In short he argues for a category of developer he calls the Expert Beginner. These are people who rose to prominence in their company or community due more to a lack of local competition than raw skill. Developers who think they are great because they are good but have no real benchmarks to compare themselves to and no one calling them out for doing things poorly. These developers not only fail to do good work, but will hold back teams because they will discourage people from trying new paths they don’t understand.
I have one big problem with his argument: he treats Expert Beginning as a finished state. He doesn’t provide a path out of that condition for anyone who has realized they are an expert beginner or that they are working with someone who needs help getting back on track to being an actual expert.
That bothers me because I realized that at times I have been, or certainly been close to being, such expert beginner. When I was first at AFSC, I was the only one doing web development and I lacked feedback from my colleagues about the technical quality of my work. If I said something could be good, it was good because no one could measure it. If I said the code was secure, it was secure because no one knew how to attack it. Fortunately for me, even before we’d developed our slogan about making new mistakes, I had come to realize that I had no idea what I was doing. Attackers certainly could find the security weaknesses even if I couldn’t.
I was talking to a friend about this recently, and I realized part of how we avoided mediocrity at the time was that we were externally focused for our benchmarks. The Iraq Peace Pledge gathered tens of thousands of names and email addresses: a huge number for us. But as we were sweating to build that list, MoveOn ran out and got a million. We weren’t playing on the right order of magnitude to keep pace, and it helped humble us.
I think expert beginnerism is a curable condition. It requires three things: mentoring, training, and pressure to get better.
Being an expert beginner is a mindset, and mindsets can be changed. Deitrich is right that it is a toxic mindset that can cause problems for the whole company but change is possible. If it’s you, a colleague, or a supervisee you have noticed being afflicted with expert beginnerism the good news is it is fixable.
Step 1: Make sure the expert beginner has a mentor.
Everyone needs a mentor, expert beginners need one more than everyone else. A good mentor challenges us to step out of the comfort zone of what we know and see how much there is we don’t understand. Good mentors have our backs when we make mistakes and help us learn to advocate for ourselves.
When I was first in the working world I had an excellent boss who provided me mentoring and guidance because he considered it a fundamental part of his job. AFSC’s former IT Director, Bob Goodman, was an excellent mentor who taught me a great deal (even if I didn’t always admit it at the time). Currently no one person in my life can serve as the central point of reference that he did, but I still need mentoring. So I maintain relationships with several people who have experiences they are willing to share with me. Some of these people own companies, some are developers at other shops, some are at Cyberwoven, and probably none think I look at them as mentors.
I also try to make myself available to my junior colleagues as a mentor whenever I can. I offer advice about programming, careers, and on any other topic they raise. Mentoring is a skill I’m still learning – likely always will be. At times I find it easy, at times it’s hard. But I consider it critical that my workplace has mentors for our junior developers so they continue progress toward excellence.
Step 2: Making sure everyone gets training.
Programming is too big a field for any of us to master all of it. There are things for all of us to learn that someone else already knows. I try to set a standard of constant learning and training. If you are working with, or are, an expert beginner push hard to make sure everyone is getting ongoing training even if they don’t want it. It is critical to the success of any company that everyone be learning all the time. When we stop learning we start moving backward.
Also make sure there are structures for sharing that knowledge. That can take many different shapes: lunch and learns, internal trainings, informal interactions, and many others. Everyone should learn, and everyone should teach.
Step 3: Apply Pressure.
You get to be an expert beginner because you, and the people around you, allow it. To break that mindset you have to push them forward again. Dietrich is right that the toxicity of an expert beginner is the fact that they discourage other people from learning. The other side of that coin is that you can push people into learning by being the model student.
Sometimes this makes other people uncomfortable. It can look arrogant and pushy, but done well (something I’m still mastering) it shows people the advantages of breaking habits and moving forward. This goes best for me is when I find places that other colleagues, particularly expert beginners, can teach me. Expert beginners know things so show them you are willing to learn from what they have to offer as well. They may go out and learn something new just to be able to show off to you again.
Finally, remember this takes time. You have to be patient with people and give them a chance to change mental gears. Expert beginners are used to moving slow but it feels fast to them. By forcing them into the a higher gear you are making them uncomfortable and it will take them time to adjust. Do not let them hold you back while they get up to speed, but don’t give up on them either.
I deal with passwords a lot. In any given day I log into five or six servers, another dozen web sites, plus my personal systems and tools. Some are stored in password managers, some are memorized, some are mine, and some are shared. To avoid losing things I need I have several patterns, schemes, and password generation tools I use to try to keep up with it all and make sure I’m using good passwords most of the time.
I don’t have a deep background in cryptography and a definitely don’t consider myself an expert in the related math. But I have spent a lot time time with users and observing behaviors, so I do consider myself reasonably knowledgeable about how people actually behave with their passwords.
A couple years ago this XKCD comic came out, and an admin’s view of it has become one of my measures of their understanding of users and passwords:
If you’re interested in understanding what technically right and wrong with the math and assumptions in that comic you might find some of the references on explain XKCD’s discussion interesting.
I don’t actually think that’s the interesting part. To me the interesting part is the number of system administrators I have talked to who are convinced the comic is wrong, because they are also convinced that anyone who doesn’t use truly random password and avoid password reuse is stupid and therefore their behavior can be ignored. These people have access to your system, and it’s your job to keep those system secure. The problem isn’t your users, the problem is we have made passwords unreasonably hard to do right.
At this point people generally know they are supposed to use good passwords, like we all know we’re supposed to brush after every meal. Sure there are people that do that, but not most of us. We all know that passwords should be long (even if we fight over the exact length to require) and use lots of different kinds of characters (although many password systems require them to be from the latin character set). But it’s too hard to follow all the rules, and security experts are so concerned about being right they don’t provide useful guidance about when the cheat.
Let me grant those who like long-random passwords the following point: if you use different passwords on every system that needs one, and they are all truly random strings, and you memorize them all so you don’t have them recorded someplace they could be stolen, you have the hardest for an attacker to crack. Great for you. But I work with people who are not perfect, have limited memories, and need to be able to have shared access on a regular basis.
Knowing the perfect random password generation pattern is useful in some cases (or so I’ve heard), but rarely are you in a case where you can use the perfect setup. I don’t care about perfect: I’m not perfect, I don’t work in a perfect office, have perfect colleagues, or perfect clients. So here are my good-enough rules for admins and developers.
0) Make it easier to do the right thing than the hard thing. This gets to be rule zero, because everything else is meant to support this idea. You want the path of least resistance to be the one that gets you the results that are secure enough to protect your systems from the attackers they face. Make sure your users have good tools for storing passwords, settings on password fields to encourage good (not perfect) behavior, and a minimum of stupid rules you don’t really understand but someone told you are “best practice”.
1) If you make it hard for people, they will find a way around you and likely weaken security. It might be post-it notes on monitors, cycles of passwords that are 5 long (because you force them to not use the last 4). If you make it hard to pick a password (because you required punctuation but not ‘, \, “, &, or !), you will end up with lots of passwords that are curse words – and your attacker will thank you for shrinking the search space. If they are using touch devices to type them, they will do things like repeat as many characters as you allow to make it easier to type (if you ban any repeats: again the attackers thank you for shrinking the search space). All things you want them to stop doing.
2) Do not have a maximum password length. Any time I hit a system with an upper bound of 12 I want to scream (although jokes about chimps might be a better tactic). Even if you are using a secure hashing system that ignores all characters after some point: who cares? Why limit the attacker’s search space to only strings between 8-12 characters?!? Sure that’s a massive search space, but not nearly as big as it could be.
3) Do have minimum lengths. Minimum lengths forces your users to do two things. First, not use passwords that could be broken in less time than it took you to read this article. Second, it gives you leverage to push them to either good phrases or generators. If you’re smart and don’t have legacy systems to support go with something like 15 or 20 characters.
5) Provide secure means to share passwords. I have more than once been sent a password in chat (running through Google, Slack, or once upon a time AOL’s servers), email, word documents, text file, and a variety of other terrible solutions. This happens not because my colleagues didn’t know it was a bad thing to do but because they didn’t have a good option. We spend so much time locking down passwords, that we don’t create secure channels to hand them around responsibly which defeats the purpose of secured storage.
6) Pay attention to how users will be using individual passwords. Not all passwords are created equal, which is why I encourage you to support throw away passwords: something short, easy to remember, and only used places it doesn’t matter if it were stolen. But even when a password is important there are issues like the ease of entering them: if I have to enter a password 4 times a day it better be easy to type or I better be able to copy and paste it. If I need it once a month it should be impossible to remember and its okay if it takes me 5 minutes to get it right. Most of us can’t type complicated passwords quickly, and if we have to enter it a bunch we want to be fast. This is even more true for people using touch interfaces where shift is an extra keystroke as is changing to a different part of the standard keyboard.
7) Stop telling people they have to use a different password every time. This is an extension of number six. People have too many passwords, and that’s not changing soon. Sure we can encourage them to use LastPass, or a tool like it, but most people aren’t going to (and if they did that could be its own problem since it creates single points of failure). Tell them to use a different password when it’s important, and to use a throw away password or scheme when it’s not.
Not everything needs to be Fort Knox so stop pretending it is. Important things like your bank account(s), your email, Facebook need their own passwords because they can be used to do real damage in the real world. Online communities, games, and other trivial places asking you to sign in do not.
8) Don’t lecture people about bad personal password habits. Honestly, this is probably the hardest one (here I am lecturing you about not lecturing them). Usually the first people to admit they are sloppy about passwords are developers and sysadmins. Sure, they will tell you about the awesome password wallet they use first, and the two factor authentication they created for their blog, but then toss off that all their production servers have the same root password and it’s 8-10 nonrandom characters. Even if you are perfect (if you are still reading this by definition you probably have room for improvement) don’t lecture people who aren’t. It just makes them feel they can’t admit when something has gone wrong, or if they don’t understand something. When you find people doing it wrong, show them how easy it is to do it right, and if it isn’t apologize and fix it.
Every week I try to ask myself: What did I do this week to make myself more valuable? Am I moving forward toward a goal, or further from it?
In technology, communications, or any other job that involves one of those two things you are either moving forward or moving backward: standing still is not an option. You are either learning new skills, trends, tools, and concepts, or you’re falling behind as other people build new tools that drive new ideas and trends.
I read lots of advice that says to plan your career two or three moves in advance. That is good advice, but I don’t think it’s wise to trust your gut that far out. The technology landscape changes too fast and too dynamically to believe you know where everything will be in three or five years. On the one hand I think it’s important to deepen my skills for the path I want to be on, but at the same time I try to broaden my skills into areas other areas that have things to teach me. In the back of my head there is always plan B and C, just in case plan A doesn’t come together they way I hope. In part, because I’ve never been able to stay on plan A very long: life always intervenes.
When I was 23 (and sure I wanted to be teacher) I was advised to read an hour a day in my field, and that it should not just be reading about the kind work I was already doing. At the time I was the new kid in IT of a mid-sized international nonprofit organization doing whatever no one else wanted to do: which is a great way to learn a variety of things. I didn’t really know anything yet about how to have a career – I had a job, and I liked my job, but couldn’t envision a career path.
But I took that advice to heart and tried to find ways to constantly learn about things I don’t know. I read books, listened to podcasts, and taught myself new skills. I learned about communications planning, economics, corporate strategy, algorithms, and a variety of other topics. The ideas I pick up from those sources help me think about technology more creatively, and helped me understand the importance of making sure I build tools that are useful not just cool to me.
The biggest thing I’ve learned from all these different inputs is that I need to live in constant fear of getting behind, outmoded, and sidelined. That fear keeps me motivated to learn more and push myself harder. By the time I retire I cannot imagine I will still be paid to be a full-time Drupal developer, I doubt that’s what I’ll be doing five years from now. Certainly by in 30 years Drupal, and the web as we know it, won’t look anything like they do today and I will be doing my job very differently. This is the blerch that keeps me motivated.
So every week ask yourself: what did I learn this week? Did I move forward or fall behind?
This week I was working with a new colleague on our account team. As with all people knew to working with technical teams and bug tracking, she’s having to learn how to create good trouble tickets for when clients report issues. This is a challenge I’ve seen played out in every place I’ve ever worked: developers want detailed tickets so we can dive in without asking 16 follow up questions, and people creating tickets don’t actually know what we want and assume we know how to find and fix the problem. And so I’d like to try to offer this explanation of what we’re looking for and why.
At the most basic level I need to know at least three things to find and fix a problem on a project (either a web site or some other tool I’m supporting):
Where in the program is the problem? This is usually a link to a sample page that has the problem.
What happened? I need a clear explanation of what went wrong. Is there a picture missing? Is the text format wrong? Is there a big red error message at the top?
What you expected to happen? What is the picture of and where exactly was the picture supposed to appear? What formatting was supposed to appear on the text? Did you do something right before the error message appeared that helps me see that message again?
These things are part of allowing me to reproduce the problem. If I can’t reproduce the problem, I can’t promise you I fixed it. If you can’t reproduce the problem, you can’t check that I’m right.
Developers will often say that if you can’t give me the step to reproduce a problem I can’t fix it. But in my experience sometimes a problem is actually really hard to reproduce, and you need a developer or a professional tester to actually figure out those steps. So it’s okay if you can’t give me perfect directions, but give me what you have.
If you find yourself writing a ticket that doesn’t say more than “Search is broken” or “Blog post didn’t look right” the problem better be massive (think big red error message level). As an account/support team member that may be all you got from the client but someone has to fill the gaps – and developers are terrible people to have fill those gaps.
As a developer there are several reasons that’s true.
First, rarely do developers get the luxury of working on one project for an extended period, and when they do those tools are large and complex. So we probably don’t have every detail in our heads at any moment. If we could store all that information we wouldn’t need task tracking systems, you could just call us and tell us about a problem as we’d call you back a few hours/days/weeks later and say “fixed”.
Second, we’re terrible at finding mistakes in our own work. Like everyone else, we need editors. If I could see the problem you are reporting, I would probably have fixed it, or at least reported it to you so we could open a task to get it fixed later.
Third, we probably don’t spend as much time in the project documentation as you do. So if someone needs to track down the original design to check for a discrepancy between the design and what’s happening a developer is probably going to be much slower at this task than you are (or you will become soon).
Also remember your developer probably will not look at the problem today unless it’s mission critical to the client. So they need to be able to figure out three weeks from now what you were talking about. If it just says: “search is broken” and I run a search in two weeks and everything looks fine, you are going to need to tell me what’s broken about it (maybe a result is missing, maybe it’s formatted wrong, maybe it’s working perfectly but the client doesn’t like the results).
Even with all that context, I know it is intimidating for many new support or account team members to crack the code developers use when talking. We over explain this, using technical terms, and get annoyed too quickly when people don’t understand us. And we often forget that teaching by analogy is helpful.
My new colleague is a baker, and so as I was trying to help her understand what I needed to be helpful on tasks I switched to bread:
If I came to you and said “my bread didn’t work out, please tell me how to fix it” how would you start?
That helped her make the connection. Just saying my bread didn’t work out, doesn’t tell her enough to help me do it right next time. She’s going to have to ask several follow up questions before she can be helpful.
Did it taste wrong or look wrong?
What kind of bread was it?
Did you follow the instructions or do something different?
Are your ingredients fresh?
Did it rise enough?
Did you knead it enough?
Did you set the oven to the right temperature?
On the other hand if I come to her and say:
I tried to make sourdough oatmeal bread over the weekend. I followed the recipe closely, but my bread turned out really dense instead of having the bready texture I expected.
Now she knows there was a problem getting the bread to rise. So we can focus questions on the yeast and other details of getting air into bread. Yes, there are still several things that could have gone wrong, but now we know where to start.
Frequently new support staff are intimidated by all the technical things they don’t know. And too often developers brush aside new staff who don’t give them the information they need and just say things like “Oh I’ll figure it myself” instead of helping their colleagues learn. Part of the solution is to help people understand that the first set of questions aren’t actually technical. Baking isn’t the right analogy for everyone, but it helped in this case. And hopefully next time I’ll do better at getting to a better explanation quickly.
In December I gave a talk at the SCDUG meeting on Drupal 8 plugins. This is based on that talk with a few improvements and additions from things I’ve learned since.
Drupal 8 provides many great improvements, but one I’ve been finding to be the most exciting is the easy ability to create my own plugins. They are actually a bit addictive, and it’s easy to start seeing projects as an excuse to create a new plugin manager. At first it can take a little bit of effort to get your head around the process, but I got great details from Unraveling the Drupal 8 Plugin System from Drupalize.me. But the problem with those, and all the examples I could find, was they were either too deep (the drupalize.me article should be considered required reading, but the example is too abstract for most people) or too shallow (the drupal.org Plugin API examples currently lack context about why I would want to do this in practice).
So my goal is to straddle that gap.
What’s a plugin?
Plugins are a design pattern that let you extend a module or service. Drupal 8 has at least four plugin discovery systems:
Annotated ← I’m mostly talking about this one, its used for blocks, views, rules, Queue API, and more.
YAML ← These are used for menus, routes, and services.
Hook ← D7 carry over hook system, still used in many places.
Discovery Decorators ← Wrap around other plugin systems to replace/improve on the classic info alter hooks from previous Drupal versions.
Static ← These are used for test code, and I’m ignoring here.
Basic Commonly Used Plugins
As a Drupal 8 developer you probably use several plugins when building a site, particularly if there is any custom functionality required. When creating controllers you’ll need a route, and likely a menu item, which means you’ll use YAML files to define those aspects. For blocks, the queue api, or Rules you’ll create an annotated plugin. Theme functions are still commonly defined with hook_theme().
Why do I create new plugin?
Between core, and major contrib modules (like Rules and Search API), plugins options abound, but it’s not always obvious why you might want to create one for a more focused project. For large projects that may have a large number of related functions, poorly defined use cases, or use cases that are expected to expand over time Plugins provide three major advantages over other approaches:
Plugins make it easy to easily keep classes small and focused.
Plugins make it easy to extend a service.
Plugins allow one module to easily extend the function of another.
Cyberwoven has a client who needed an internal training portal. They want to provide some basic gamification, particularly badges, and they wanted a more detailed tracking of user behavior than the core statistics module provides. The badges present a function that has many related, but different, use cases and is likely to grow over time. So I created a pluggable service that allows us to easily define a set of conditions for creating a new badge. Each plugin’s annotation lists the events it cares about, and when one of those events is fired the service runs the plugin’s test function. If the badge has been earned the plugin returns the information needed to generate that entity. All the event listening, entity generation, and error checking happens in the service, so each plugin is only focused on the details of the one badge. The event tracker is similar, with each plugin defining a filter for the major system events to determining what details should be recorded, and the service handling the logging of those details and reporting. As the client needs have evolved we’ve been able to add and modify the plugins to meet their needs.
The second place I’ve created a custom plugin system for was an internal project at Cyberwoven (it was actually the project I used to learn how to create custom plugins). Our testing server has always had a service to perform some basic operations on the server, like pulling repos via webhooks and similar tools to help developers. Setting up our development environments for D8 meant it was time for a new test server, and therefore new tools to manage it. From the start I wanted to create a tool that helped both developers and account managers. So I created a D8 site to manage the sites that provides various tools to perform task operations we all need (not only code handling, but also checking if security releases apply to any sites). All the various operations we want exposed through that site are created as plugins to our custom manager. Each task plugin is focused on just that one task. All the other various needed to track the sites, route requests to run a task, and render their responses are all handled in other places. It’s also now making it easy to add features that wrap around the plugins since the plugins all have a consistent interface (like adding new displays and running some tasks through cron).
Introducing Site Manager
The main module defines an entity for each site, a couple displays (like the listing page displayed above), and the annotated plugin base so we can add new features easily. Plugins can be enabled and disabled on a site-by-site basis, and can be run from the plugins dropbutton.
Once we had the basics in place (like getting the git and drush status’s via simple plugins) we started to add more powerful features to turn the tool into a dashboard. We added the module search block you see on the left as a separate module that searches all sites for a specific Drupal module (nice when security updates are released). The search module also provides a plugin for each site that lists the modules for just that site.
The last site in the screen shot has a reference to scanning the HTTPS certificate for a specific site. This is another feature we added recently as this tool increasingly starts to serve as a dashboard. The HTTPS Scanner module again provides both a batch job to scan all sites, and a task plugin that scans just one.
As of this writing, all told there are nine working task plugins across four modules (there are a couple more underway as well). Tasks can also have their results displayed in a block when you look at the details for a site.
Building your own manager
Hopefully by now your convinced that it’s worth having your own plugin manager at least sometimes. So the next question is how to build them.
Elements of a custom plugin
Plugins have several parts, most of which are small and simple:
Annotation Plugin Definition
Plugin Manager Service
The first step is to define the annotation for your plugin. This is a simple class that extends Drupal\Component\Annotation\Plugin to define the variables you want listed in the annotation comments of the plugin. This file goes in custom_module/src/Annotation, and as always the file name and class name match:
Plugin Manager Service
The plugin manager itself is a service, but the vast majority of its function comes from the parent classes. The convention is to place this file in custom_module/src/Plugin (as will the rest of the base definitions we’re about to provide). In this case we call it: TaskPluginManager.php.
The full version of this provides a few more improvements to this class (I overrode the getDefinitions() method to provide some filtering options to support disabling plugins), but those aren’t actually required for it to work well.
In short, this class just provides some general definitions to separate it from all the other annotated plugins.
And since the plugin manager is a service, we add it to our module’s service.yml file:
For this plugin I defined 4 critical public functions, three just provide basic information about a plugin, but fourth (run) will be the interesting one in a minute.
Finally we’re getting to something interesting (or at least something that requires code). The plugin base is an abstract class which all the actual plugins will extend. This goes in custom_module/src/Plugin/TaskPluginBase.php.
The things worth taking notice of here are the fact that I inject several services into the base class, so I have them for each plugin: translation, main sitemanager module configuration, and a wrapper on Symfony’s process service to make running drush and git commands easier. The three informational functions are fully defined here, but the run function stays abstract since it’s the function that actually justifies creating a plugin at all.
In theory I could stop right there. I’ve created a plugin manager and defined all the things someone else needs to be able to use it. But there are two technically optional parts that are useful if you like to work and play well with others: an example implementation and a controller for use or testing purposes.
We went through all the trouble to create the plugin manager, before you stop you should create at least one plugin to prove all the work we just did actually works.
This is a simple task to use drush to run cron for a site. The site entities know the location on the file system for Drupal root, and the simple task service handles the extra settings to control where the commands are run (and time outs and error trapping), so we can safely feed that to command and the location to get the response.
The method returns a render array, that can be used as part of a response.
The controller is truly 100% optional. There are two reasons you might want to create one: to actually run the plugin if that makes sense (it does for the site manager), and to provide for easy testing of a plugin.
For site manager I created the controller because I actually needed it to run the tasks from the dropbutton (its the link used by all the links on those buttons). But when I created the others for clients, I realized they can be very hard to test. They are buried inside several layers of complexity, making a test suite is a challenge to create correctly. And since are of the reason to use a plugin for a project with growing definitions, the test suite is often of limited use for rush additions. But by having a carefully built (and secured) controller and route, you can create an endpoint to use to test the plugin.
Because it’s optional, and well covered elsewhere (or heck just use Drupal console to generate it), I’m going to skip over most of the details of actually building the route, and linking that to the controller, and focus on the single function of running the task itself. To keep this short I’ve removed lots of extraneous details like security and error trapping – creating a tool like this isn’t for beginners so I’m assuming you know how to do those things.
The plugin manager service was injected into the controller (code not shown) so I just have it create an instance of the task requested. The controller calls the run function. If the task is successful the output (a render array) is used for the response, otherwise a simple message is send. This sends an AjaxResponse to work with Drupal’s standard JS handlers, but it could generate a full page just as easily.
Being able to create your own plugins easily is really nice. In Drupal 7 custom plugins where generally considered an advanced developer task. The improvements in the abstraction (and the fact that true plugin support is now in core not pulled from ctools or other critical modules), means that while its still not a beginner task it is something all Drupal developers should be learning.
But remember you don’t always need or want the complexity of your own plugins. I’ve found the idea a bit addictive, and I get unreasonably excited when I find a place plugins are used or that it makes sense for me to create a new plugin type. For custom work you can almost always do the same thing using extra functions on a service, controller, or event listener, so this is a judgement call in the end.
Early in my career I spent a lot of time as the only technical person on project, and therefore believed that I didn’t need to document my work carefully since I was the only person who had to understand it later. It turned out that if a project was back burnered for a few months the details were pushed out of my mind by the details of eight other projects.
Any project that takes more than a couple hours to complete involves too many details for most people to remember for more than a few days. We often think about project documentation as something for other people – and it is – but that other person may be you in six months.
I learned to start keeping notes that I could go back to, those notes would turn into documentation that I could share with other people as the need developed. My solutions were typically ad-hoc: freeform word documents or wiki pages. For a while I had a boss who wanted every piece of documentation created by IT to fit a very predictable format and to be in a very specific system. It took two years for him to settle on the system, process, and format to use. By then I had a mountain of information in wiki pages that documented the organization’s online tools in detail, and no one else is IT had anything substantial. It was two more years before the documentation of other team members got to be as good as my ad-hoc wiki.
That’s not to say a rogue solution is best, but the solution that I used was better than his proposed setup for at least three years. That experience got me to think about what makes documentation useful.
Rules of thumb for good project documentation:
Write up the notes you’d want from others when coming into a project: think of this as the Golden Rule of documentation. Think about what you’d want to have if you were coming into the project six months from now. You’d want an outline of the purpose of the project and the solution used, and places they deviated from any standards your team normally uses. You’ve probably read documents that are explaining something technical to an expert that are hard for anyone else to understand – if I’m reading the documentation I want to become an expert, but I’m probably not one already.
Keep it easy to create and edit while working: if you have to stop what you’re doing and write your notes in a totally different environment that your day-to-day work you will not do it. Wikis, markdown files, and other similar informal solutions are more likely to actually get written and updated than any formal setup that you can’t update while doing your main work.
Document as you go: we all plan to go back and write documentation later and almost none of us do. When we do get back to it, we’ve forgotten half the details we need to make the notes useful to others. So admit you’re not going to get back to it and don’t plan to: write as you go and edit as you need.
Make sure you can come in in the middle: People skim project documentation, technical specifications, and any other large block of text. Make sure if someone has skipped the previous three sections they can either pick up where they left off, or give them directions to the parts the need to understand before continuing.
Track all contributions: Use a system that automatically tracks changes so you you can see contributions from others and fix mistakes. Tools like MediaWiki, WordPress, and Drupal do this internally. Markdown or text files in a code repository also have this trait. Avoid solutions like MS Word’s track changes that are meant for editing a final document not tracking revisions over time.
Don’t fear editing: follow the Wikipedia community’s encouragement to Be Bold. You should not fear making changes to the team’s documentation. You will be wrong in some of what you write, and you should fix any mistake you find – yours or someone else’s. Don’t get mad if someone makes a change that’s not quite right, revert the change or make a new edit and more forward.
There is always an audience: even if you are the only person on the project you have an audience of at least your future-self. Even if it feels like a waste in the moment having documentation will help down the road.
Remember even if you are working alone you’re on a team that includes at least yourself today and yourself in the future. That future version of you probably won’t remember everything you know right now, and will get very annoyed at you if you don’t record what they need to know. And if the rest of your team members aren’t just versions of yourself they may expressed their frustration more directly.
I have a degree in Computer Science, with a minor in Economics. But I earned that degree at Hamilton College, a traditional liberal arts college. That meant I was forced (now students there are just encouraged) to take classes in a variety of disciplines. I went to Hamilton in part because they offered degrees in Computer Science and History, and I was interested in both fields. In addition to the courses required by my major I took classes in history, religious studies, philosophy, art, and more. And I learned critical skills for work and life because I took those classes.
Ceramics and pottery taught me to be a craftsman, accept critical feedback, and admit failure. For all the courses I took in college, the time I spent in Ceramics and Pottery studio taught me more about how to be good at what I do than any other course work (including CS). Basic pottery is a craft that doesn’t allow you to save a piece once you’ve made mistakes. If you are making cylinders the walls aren’t straight throw it away and start again. My professor in those courses set very high standards and pushed us to meet them through brutally honest, but helpful, critiques. He taught me to appreciate honest feedback, and to be skeptical of my work before showing it others. In those courses we had to do good work, recognize our weaknesses, and hit deadlines.
History taught me to care about communicating. Between the courses I took, and dating (and eventually marrying) a history major (now professor) I came to understand how important it is to communicate well in speech and in writing. I started this blog in part because I wanted to make sure I was spending more time writing to help maintain those skills. I frequently find myself in meetings having to explain highly technical issues to non-technical clients and colleagues. Knowing how to adjust to my audience, without insulting their intelligence, is a critical part of my day to day work.
Religious Studies and Philosophy taught me to make a well reasoned argument and express the ethics of my position. Religious Studies and Philosophy both require you to make arguments based primarily on the strengths of your ideas – you can’t research your way out of a bad concept. If you cannot assemble a well reasoned argument you leave yourself open to easy counter attacks. When you work at a nonprofit you often are building your ideas and your explanations from a mix of the facts of the issue, your worldview, and the assumptions you are forced to make because no one has all the information they really need.
Economics taught me how to view the world through the lens of money and trade. While I don’t think it’s the only, or even best, way to view the world it is a very important worldview that dominates the news and political spaces. Understanding the strengths and weaknesses of economics as a field helps me think about plans put forward by various politicians. It also helps me think about how the companies I work for function, and helps make sure internal systems and plans I propose have a strong business case to support them (and because I learned to throw things away in ceramics I also know that when the case is bad to stop and try something different).
The most important skill I have is the ability to learn new skills. Since I graduated from college the technology I work with every day has changed several times. I’ve had to learn tools, techniques, and strategies that didn’t exist when I was in college. And my career has evolved through multiple employers, disciplines, and areas of expertise.
This week we get a letter from Atlantic Broadband, our ISP, addressed to “Aaron & Eliza Crosman Geor”. My wife has never gone by Eliza and her last name is not “Geor”.
It’s been this way since we signed up with them, when we ask them to fix it they acknowledge that they cannot because their database cannot correctly handle couples with different last names who both want to appear on the account. Apparently it is the position of Atlantic Broadband that in 2016 it is reasonable to tell a woman she cannot be addressed by her legal name because it would be expensive for them to fix their database, and therefore she must be misaddressed or left out entirely.
I consider this unacceptable from old companies, but Atlantic BB was founded in 2004 – there are probably articles about not making assumptions about people’s names that are older than their company.
Folks, it is 2016, when companies insult people and then blame their databases it is because they do not consider all their customers worthy of equal respect.
So let’s get a few basics out of the way:
Software reflects the biases of the people who write it and buy it.
If your database tells someone their name is invalid your database is not neutral. Just because you don’t get the push-back that Facebook sees when they mess this up does not mean what you’re doing is okay.
If your database assumes my household follows 1950s social norms, the company that uses it considers 1950s social norms acceptable in 2016 – and there are probably a few of those they don’t want to defend (I hope).
When an email, phone rep, or letter calls me by my wife’s last name or her by mine, in both cases they are assuming she has my last name not that I have hers. This is a sexist assumption that the company has chosen to allow.
Of course Atlantic isn’t the only company that does this: Verizon calls me Elizabeth in email a couple times a week because she must be primary on that account (one person must lead the family plan), and Nationwide Insurance had to hack their data fields for years so my wife could appear on our car insurance card (as required by law) every time we moved because their web interface no longer allowed the needed changes. The same bad design assumptions can be insulting for other reasons such as ethnic discrimination. My grandmother was mis-addressed by just about everyone until she died because in the 1960s the Social Security Administration could not handle having an ‘ in her name, and no one was willing to fix it in the 50 years that followed SSA’s uninvited edit to her (and many other people’s) name.
In all these cases representatives all say something to the effect of “our computers cannot handle it.” And that of course is simply not true. Your systems may not be setup to handle real people, but that’s because you don’t believe they should be.
Let’s check Atlantic Broadband’s beliefs about their customers based on how they address us (I’m sure there are some additional assumptions not reflected here but these are the ones they managed to encode in one line in this letter):
They assume they are addressing one primary account holder: I happen to know from my interactions with them that they list my first name as: “Aaron & Eliza”, and my last name as “Crosman Geor”. Plenty of households have more than one, or even two, adults who expect equal treatment in their home. Our bank and mortgage company know we are both responsible adults why is this so hard for an ISP (or insurance company, or cell provider, credit card, etc)?
They assume my first name isn’t very long: They allowed 13 characters, but 4 more is too many. I went to high school with a kid who broke their database by exceeding the 26 character limit it had (they didn’t ask the kid to change his name, the school database admin fixed the database), but Atlantic can barely handle half that.
They assume my last name isn’t very long: Only 12 characters were used and they stopped in a strange place. I know many people with last name longer than that: frequently people who have hyphenated last names blow past 12. Also the kid with a 26 character first name – his surname was longer.
They assume my middle name isn’t an important part of my name: If they had a middle name field, they could squeeze a few more letters in and make this read more sensibly. But they only consider first and last names important. Plenty of people have three names – or more – they like to have included on letters.
They assume it is okay to mis-address me and my wife: The name listed is just plain wrong, but they believe it’s okay to keep using this greeting. They assume this even after they have been told it’s not, and even after we’ve reduced service with them (if another ISP provided service to my house I’d probably cut it entirely although mostly for other reasons). They believe misaddressed advertisements will convince me I need a landline or cable package again.
Now I’ll be fair for just a minute and note something they got right: they allow & and spaces in a name so Little Bobby Tablesmight be able to be a customer without causing a crisis (partially because his name is too long for them to fit a valid SQL command into the field).
Frequently you’ll hear customers blame themselves because their names are too long or they have done something outside the “norm”. Let’s be clear: this is the fault of the people who write and buy the software. Software development is entirely too dominated by men, as is the leadership of large companies. When a company lacks diversity in key roles you see that reflected in the systems built to support the work. Atlantic’s leadership’s priorities and views are reflected in how their customers are addressed because they did not demand the developers correct their sexist assumptions.
These problems are too common for us to be able to refuse to do business when it comes up. I will say that when we switched our insurance to State Farm they did not have any trouble understanding that we had different last names and their systems accommodated that by default.
If you do business with a company that makes these (or other similar mistakes) I think it’s totally reasonable to remind them every time you reasonable can that it’s offensive. Explain that they company is denying you, your loved ones, and/or your friends a major marker of their identity. Remind them they are not neutral.
If you write data systems for a living: check the assumptions you’re building into your code. Don’t blame the technology because you used the wrong character set or trimmed the field too short: disk is cheap, UTF-8 has been standard for 15+ years, and processors are fast. If the database or report layout doesn’t work because someone’s name is too long the flaw is not the name.
We all make mistakes and bad assumptions sometimes, but that does not make it okay to deny people basic respect. When we make a bad assumption, that’s a bug, and good developers are obligated to fix it. Good companies are obligated to prevent it from happening in the first place.
As part of my effort not to repeat mistakes I have tried to build a habit in my professional – and personal – life to look for ways to be better at what I do. I recently rediscovered how much you can learn when you try doing something you know well backwards: I drove on the left side of the road.
By driving on the left I discovered how many basic driving habits I have that are built around driving on the right. The clearest being that the whole time I was in New Zealand I never knew if anyone was behind me, and the whole time I couldn’t figure out why. The mirrors on the car worked just fine, but it turned out I wasn’t looking at them. Driving home from the airport after we returned to the US I realized that every few seconds my eyes jump to the upper right of my vision to check the mirror. In New Zealand I spent the whole time glancing at the post between the windshield and the driver’s side window (which had seemed massive to me while I was there) instead of the mirror. It made me conscious of my driving habits in a way I haven’t been in years, and as a consequence, I think it’s made me a better driver. I’m thinking about little details again; I’ve been more aware of where I am on the road and what I’m doing to keep track of the other cars around me.
A few years ago I was watching videos from the MIT Algorithms course to refresh some of my basics, and because I wanted to know what had been added in the decade since I’d taken that class at Hamilton. During the review of QuickSort the professor mentioned that it wasn’t originally a divide-and-conqueror process, but a loop based approach meant to work on a fixed length array (so you could use a fixed block of memory). And as I recall he suggests that the students should work out the loop based version. So riding on the train home from work I pieced it together, and found that it’s an elegant process. It’s not something I ever expect to have cause to implement, but it did help me improve my thinking about when to use recursive functions vs when to use a loop, and helped me think about when to use recursion, loops, and other tools for processing everything in a list. There was a session by John Kary at DrupalCon this year on rethinking loops that pushed me again to revise some of how I made those decisions. Again his talk took the reverse view of much of my previous thinking and was therefore very much worth my time.
If you’re feeling like you are in a good groove on something, try doing it backwards and see what you discover.