On Being Self-Taught

Eventually we are all mostly self-taught.

From time to time conversations come up among developers, and other fellow travelers, about being self-taught vs getting formal training. Over time I’ve come to realize that the further and further you get into your career, the less the distinction means anything; eventually we are all mostly self-taught.

I’ve written before about the value of my liberal arts education and I stand by my assertion that what I learned in that setting was, and is, valuable to my life and work. But just because something was useful to life does not mean it was the only way to acquire the skills. It’s a good way for many people, but far from the only way.

For anyone in a technical field, and most professional fields really, to succeed over time you need to learn new tools, skills, and techniques. The tools I knew when I graduated college are all largely outmoded or significantly upgraded, and I’ve had to learn a variety of technologies that didn’t exist in 2001.

Within the Drupal community lots of people talk about being self-taught, sometimes with pride sometimes with embarrassment, but in truth very few people were formally trained on the platform. A CS degree is not required. Lots of very successful developers in the Drupal community (and beyond) have degrees in fields like religion and art history. In fact, I’ll argue that just about every Drupaler taught themselves most of what they know about Drupal. How they did that can vary widely, but we are a community with few formal training programs and lots of people who stumbled into Drupal trying to solve a non-technical problem (communications or marketing for an organization). Even advanced workshops at conferences dig deep into one small area and expect you to generalize that knowledge to your projects, which I count as self-taught. For example, I had a friend ask the other day about how to control the PDO connection settings in Drupal 7 — which I didn’t know how to do, but knew they were similar to Drupal 8 — so I sent him my Drupal 8 instructions and he figured it out how from there. He’s now taught himself how to do what he needed for that project and in the process generalized the approach for whatever he may need next time.

So then it is important for all of us to find, and hopefully share, techniques for self-teaching — even for those who have some kind of formal training. Here are my suggestions for people who are starting out and haven’t yet found the pattern that works for them:

  1. Assume you first solution is wrong. Most of us have, or will, stumble our way through a project where we don’t really know what we’re doing without a lot of support. We usually learn a great deal in the process, and launching those projects can feel pretty good cause you’ve succeeded at something hard. It is easy to get into the habit of assuming the solutions from that project were correct because they worked. In truth those projects are really rough around the edges, and just because we got it to work does not mean the solution was good. Assuming the first solution is good enough forever is how you come an expert beginner which then takes a lot of effort to undo. Once you have a working solution, step back and see if you can think of a better one, or if you now can guess better search terms to see if someone else wrote up a different solution to the same problem. Admit your work could be better and try to improve it.
  2. Learn a few more programming languages. Most people who are self-taught from the start, and even some who have a BA/BS in Computer Science, only know 2 or 3 programming languages (PHP, JS, and CSS+HTML are often the only languages new people learn at first). One of the courses I took by chance in college forced me to learn 8 in 16 weeks. It was grueling, miserable, and darned useful. I can still learn a new language in just a couple weeks and rarely do I hit a language construct I don’t recognize. You don’t need to go that far. When I first started out a mentor told me you should learn a new language every year, and for several I did. Some of those, not the languages I learned in college, are the ones I use most day-to-day. All told I’ve spent time writing code in more than twenty different languages. That many isn’t terribly useful but the more languages you learn, the more you learn to understand the elements of your primary language.
  3. Learn basic algorithms and to measure complexity. The kind of thinking that goes into formal algorithms will help you be a better developer overall; badly thought through processes is the place I tend to see the largest gaps between developers with and without formal training. Any college-level CS program will put you through an algorithms course that teaches a variety of specific algorithms and force you to understand their structures. If you didn’t go through one of those programs, this is probably the course that will help you the most. On the one hand most of us rarely rewrite these algorithms as on modern platforms some library or another will provide a better version than we are likely to craft for our project. But learning what they are, when they are used, and how to understand their performance is useful for any project that involves lots of data or processing. MIT has a version of their algorithms course from 2011 online, or find one through another provider. Even if you just watch the lectures (really watching, not just vaguely have them on while cooking and cleaning), you can learn a great deal of useful information. I learned a lot watching those lectures as it refreshed and updated my understanding of the topics.
  4. Find and learn from mentors. Notice I used a plural there; you should try to find a few people willing to help you learn your profession, and more generally help you learn to advance in your field. Most of us benefit from learning from the experiences of multiple people, and who we need to learn from changes over time. I had the great experience of having a few wonderful mentors when I was first starting out, and much of the advice they gave me still serves me well. Some of it contradicted, and resolving those contradictions forced me to learn to do things my own way and find my own solutions.
  5. Learn other platforms. This is both a protection against future shifts in the market, and also a way to see how things work from outside your current professional bubble. Drupal developers can learn a lot from writing a WordPress plugin, or better yet an add-on for a platform in another language (think about Plone, Gatsby, or Hugo). Or try to learn to work with a platform like Salesforce or AWS. Other platforms have different communities, different learning styles, and different patterns. Like understanding additional languages, different platforms help you broaden your understanding and provide insights you can bring back to your main work.
  6. Learn to give and take criticism. Part of learning is getting feedback on your work, and part of being on a team is sharing feedback with others. If you took art or music classes in high school or college you probably learned some of the basic lessons you need here, but if you didn’t, consider taking one now at your local community college or art center. The arts are wonderful for getting experience with criticism. For all art is often open to interpretation, it also requires specific skills. If you play off-key, it sounds wrong. If your sculpture collapses under its own weight, the project failed. If your picture’s subject is out of focus, you need to re-shoot it. Sure there are brilliant artists who can violate all the rules, but if you have never experienced an art critique you are not one of those artists. The experience of getting direct, blunt, and honest feedback will help you understand its value and have to give that feedback yourself.
  7. Share what you think you know. We learn a great deal with we teach others. Both because it forces us to refine our thinking and understanding so we can explain it, and because learners ask questions we cannot answer off the top of our heads. This can be user group or conference presentations, internal trainings for your team, mentoring junior developers, writing a blog, or anything else that gets your from learning to teaching. It’s okay if you’re not 100% right, that’s part of how we learn. A few years ago I was doing a joint project with a junior developer who asked me a lot of questions, and pushed hard when she thought I was making mistakes. When she asked why I was selecting a solution or setting a pattern, she was never satisfied with “because that’s the best way to do it.” She wanted me to explain why that was the best way. If I couldn’t walk her through it right away, I went back and hunted for reference material to explain it or if that failed I tested her counter ideas against my plans to see if I was missing something. While I was usually right, not always and we did make changes based on her feedback. More importantly it forced me to show my work in fine detail which was a good exercise for me and gave her insights to help her do better work.
  8. Find your own patterns. At the start I said this list was for people who didn’t have their own patterns yet. In the long-run of your career you need to figure out what you need to know to get to where you want to go next. Eventually you will need to find a pattern that works for you and the life you are living. No one can tell you what that is, nor how to learn it all yourself. Experiment with learning styles, areas of work, roles, and types of projects as much as you are able until you feel your way to the right solutions for you.

Some Things Every Developer Should Read

Every developer needs to be learning new things all the time. We need to have a good grounding in the ideas that have come before us and those that are emerging around us within our field, and we need to understand the larger social impact of our work. To be fair this isn’t just true of developers, but we are a field that has a bad habit of arguing our work is totally new and original – which is rarely true in any meaningful way.

Part of that push for constant learning is to read books about software development, about life, about business, about writing, and really anything that pushes how and what you think. There isn’t really a settled cannon of things every developer must have read but there are definitely some works you need to be familiar with the ideas in (even if you haven’t read them) to keep in conversations over lunch at a conference like the Mythical Man Month and the Agile Manifesto (both of which came up over a lunch at my last conference). There is also a much larger set of works that many developers benefit from reading even if they aren’t really about software.

So here is my list of works that I think every developer should read at least once in their life. I’m breaking the list into two pieces: specific books I think every developer should actually read, and a category of types of things that every developer should be on the lookout to read on a regular basis. I’d love to hear suggestions about should be added and what I should have read myself.

Specific Books

These are some specific books that I’ve found helpful for making me a better developer. None of the ones here are directly about programming – that’s not an accident. Many of the books I’ve read about development and the creation of software were helpful and I am glad to have spent time with them but these are works that helped me think more broadly about the craft and the when, why, and how I write code.

  • Design of everyday things: While many of the specific recommendations for how to fix some devices are out-of-date and flawed (I will never forget reading that all office phones needed to be easy to use was a two-line digital display while working at a desk with a proof the display had not solved the problem), the general philosophy holds up. Also, it will make you justifiably angry at every poorly installed door you struggle to open.
  • Zen and the Art of Motorcycle Maintenance: This is a long, slow, dense read. And if I hadn’t need assigned it in college I probably never would have gotten through it. But I find myself coming back to its discussion of quality all the time.
  • Technically Wrong: This is the newest book on this list, and it makes the cut cause the ideas are so important. I had the opportunity to see Sara Wachter-Boettcher speak at DrupalCon before she published the book and it was one of those talks that changed how lots of people in the room think about software and the role of our work in people’s lives.
  • Elements of Style (aka “Strunk and White”): I was assigned to read this in a history course in college and then again when I was promoted to Web Director at AFSC – cause the Director of Communications wanted to be sure I remembered the importance of writing well in that job (I did, but I re-skimmed the book anyway). If everyone wrote English like Strunk and White suggest writing would be much better on the whole – although fairly boring. Read it not because their rules are perfect, read it because you should think about the rules of writing when you break them.
  • Blah Blah Blah: This is something you should read after you’ve read a lot about writing clearly since it rather pitches the reverse concept: sometimes a rough picture is clearer and more useful than a carefully written description.

Things that push and inspire

These are examples of works that are helpful to push boundaries. The specific examples I cite may or may not be valuable to you, but they are examples of things within the category I’m describing.

Books that show you code you’d never write:

These are books that talk about code and concepts that either have no application to your work, or are ways of showing off what a language can do – even when you shouldn’t follow the example. I have two examples here from opposite ends of the spectrum:

  • If Hemingway wrote javascript: This book is brilliant. It’s a series of examples of how various writers might have completed common CS class programming assignments using JavaScript: DO NOT WRITE CODE LIKE THIS. It shows off how wonderfully flexible JavaScript is as a language, and presents some of the worst possible solutions to those problems along the way. It is a great read that will teach almost anyone a few things about JavaScript, and push you to think about different ways to approach a problem. It is a light, fast, and entertaining read.
  • Beautiful code: This is a fairly dry read. It is made up of series of examples of truly well written code: WRITE CODE LIKE THIS. It looks at code in a variety of languages, not all of which are in common use, solving problems that are sometimes familiar and sometimes rare (in part because of the code you’re reading). It is a slow dense, and at times boring read, but it’ll give you insight into what other people will find graceful in your work.

A college textbook

I mean actually read and actual textbook. Sure if you went to college someone assigned bits of pieces of these things to read, maybe they even assigned all the chapters in order. But how many people actually did all the reading in order? That doesn’t mean the professors weren’t right that those things are useful – they do not get assigned for the professors health (and if you think professors get rich off assigning their own books you need to spend time learning about the economics of academic publishing). 

I have a couple around that I kept as reference after college, and one or two I’ve now actually read end to end. I will grant you these are not exciting works. I don’t do this as a regular reading habit, but it’s worth having done. If you don’t have one around, most libraries still have books and often a few reasonable high level textbooks.

Articles by someone whose ideas you don’t like

If you aren’t used to doing this I should be clear that this is about getting outside your comfort zone on topics you think you know a lot about, and reading things that push you to justify – or better yet modify – your understanding of the field. This doesn’t have to mean going and reading material you find deeply troubling or revisit ideas you know are morally reprehensible. Bothering with ideas from ideologies of hate is unlikely beneficial for most of us, think closer to home here. In this category I’m talking about project management practices you think are overblown: love Agile, read people who point out where it routinely fails; hate Agile, read some material about why it always works when done right. For me it is sometimes reading people who still argue technology is morally neutral. But best is often to find a topic you are still trying to resolve your own thinking on and absorb some uncomfortable ideas and wrestle with them. 

In Praise of B Students

There is nothing wrong with just being good at something.

When I was in school I was a solid B student. I thought of myself as a B student and I worked hard to make sure I was always working at least a B average. I worked hard, but not too hard. I could have probably been an A student if I’d worked harder, but I would not have been as happy. I didn’t feel the drive to be the best all the time, just good enough to keep up with what I was learning. Most people brag about being A students, but that wasn’t me and it has worked out pretty well in the long run.

For me a B student is someone who not only gets actually Bs most of the time in school but also knows how to work hard to get the grade they want, and how to relax so they recover and enjoy their lives along the way. If they fall a little short at some point they are perfectly capable of working a little harder to make up the ground. Life is not a sprint, it’s a marathon, we all have to save a bit for later. If you are used to running full out all the time, you do not always have the ability to step it up a little extra when needed nor how to relax and recover without stopping. In a world where we seem to be moving from helicopter parents to snowplow parents, there are still people that have fallen short from time to time, so they know how to pick themselves up and push forward.

There has been plenty written and said about the flaws of perfectionism so you don’t need to trust me, and I’m not going to waste time making the case that there are problems with pushing kids and college students to be a perfect A student all the time. And there are problems when companies only want to hire people who have those backgrounds.

In our current test-driven education culture there is a lot of push for kids to be the best. And the highly competitive nature of admissions for some top colleges can make it seem like a kid has to be perfect to succeed in life. But life isn’t Top Gun there actually are points for second place, and really more or less all the others (well paychecks and some level of happiness anyway). More importantly people who are used to a bit of failure can excel when faced with tough jobs that require a lot of on the job learning and trial and error work. That describes pretty much every modern office job, and lots of others – watch a plumber work on an old house and you’ll see plenty of trial and error being required to solve even basic problems.

Failure is a part of life. We all need to know how to acknowledge we made mistakes, how to recover, and how to ensure we don’t make the same mistake again in the future. People who earned a lot of B’s in school are familiar with all those things. We sometimes got lazy and ended up with a C instead of B – to stay a B student we then worked harder and got ourselves enough A’s to offset our missteps and bring our averages back up. Sometimes we really didn’t understand something and had to struggle through to make sure we stayed on pace.

B students also often know the difference between learning and getting the right answer on an assessment (a skill I would have told you I had but did not need anymore until I starting having to pass professional certification exams on a regular basis). We learned how to pass the test without knowing all the information, but knowing enough to get the grade we wanted. Getting a perfect score on an assessment is only useful if the assessment is perfect – most college professors and anyone who has taken a certification exam can tell you there are no perfect assessments. I had to pass tests in college, and I need to complete and maintain certifications now, but I don’t need to have every piece of information crammed into my brain all the time – that’s what Google is for. Once I pass, I’ve passed – it feels better to get more right answers, but as long as I get enough to pass I’m certified.

When I was in the 6th grade we had a whole day dedicated to studies skills taught by a guy named Mr. Gallagher. He wasn’t a regular teacher at my school, and I have no idea where they found him or what happened to him later, but some of what he taught us that day has stuck with me both in terms of skills and philosophy. He emphasized the need to understand how a test was written, both in terms of following directions so you didn’t give up silly points and so you know where to focus your time (hint: time goes first into the places you are likely to earn the most points), and when to guess and when to work through an answer carefully (again be greedy about points not the number of right answers).

He was the first person to put the idea into my head that the point of going to school is to get good grades instead of learning. I can still remember our teachers pushing back on that argument the next day, and I eventually arrived at the conclusion that they were both right: the purpose of school is to learn, but the purpose of tests is to get good enough grades so you can move forward in school. The thought exercise itself helped me see that learning material and passing a test weren’t actually always related. I had a teacher later who helped me further abstract this concept when he pointed out that he could easily write tests we all passed or all failed, so the purpose of the test writing was to find a measure of what he thought we should be able to do (that teacher and his wife were recently on Marketplace – thanks Bill that simple lesson was super useful in college and beyond). If pass and fail are at the whim of the test author, and they aren’t actually playing against me personally but really tracking a group of people, then I can play it as a game where I need to get myself into the top 80%. If I learn to play at that level reliably I’m going to do pretty well and I don’t have to stress being perfect all the time.

Yes, there are things in life that you cannot do unless you are one of the best. Not to mention the fact that the playing field isn’t level, which means some people have to work a whole lot harder than others to earn the same B and a report card full of Bs doesn’t get read the same for everyone. But we cannot all be the best, and pretending that’s not true doesn’t level the social playing fields.

And of course there are plenty of moments we need to strive to be as close to perfect as we are capable. But we also have to hit deadlines and get things done even if we know the work will be less-than perfect. Anyone who works in software development has to deal with this on a regular basis. As much as we may want to be perfect, shifting project requirements, project timelines, platform weakness, and our own mistakes all get in the way. We do the best we can with the skills, time, teams, and the tools we have. We need to know how to accept that we cannot fix every flaw and still get things done well enough to ensure the project is successful.

So when you are hiring people consider looking for people who already know how to be second best. People who know how to do really good work, but also know how to take risks, try something new, and in the process fail. People who know how to get things done well enough to be successful, even if it’s not perfect. We aren’t perfect, our work isn’t perfect, our world isn’t perfect, so don’t pretend your team has to be perfect.

Authors Note: Last month I failed to post to this blog for the first time. If you look back over the record you will see at least one post each month from the time the blog starts through August 2019 – then I missed September. Oops. I could make all kinds of excuses (I really did get sick right at the end of the month so my planned post was delayed a week), but the truth is I had time and material, I just didn’t get anything done. So now to make up for it I am trying to post every weekend this month.  I missed a goal by a little, so now I compensate to make up for it – then I’ll hopefully go back to my nice reliable routine. What can I say? I’m okay with a B.

Docksal Pantheon Setup from Scratch

I recently had reason to switch over to using Docksal for a project, and on the whole I really like it as a good easy solution for getting a project specific Drupal dev environment up and running quickly. But like many dev tools the docs I found didn’t quite cover what I wanted because they made a bunch of assumptions.

Most assumed either I was starting a generic project or that I was starting a Pantheon specific project – and that I already had Docksal experience. In my case I was looking for a quick emergency replacement environment for a long-running Pantheon project.

Fairly recently Docksal added support for a project init command that helps setup for Acquia, Pantheon, and Pantheon.sh, but pull init isn’t really well documented and requires a few preconditions.

Since I had to run a dozen Google searches, and ask several friends for help, to make it work I figured I’d write it up.

Install Docksal

First follow the basic Docksal installation instructions for your host operating system. Once that completes, if you are using Linux as the host OS log out and log back in (it just added your user to a group and you need that access to start up docker).

Add Pantheon Machine Token

Next you need to have a Pantheon machine token so that terminus can run within the new container you’re about to create. If you don’t have one already follow Pantheon’s instructions to create one and save if someplace safe (like your password manager).

Once you have a machine token you need to tell Docksal about it.  There are instructions for that (but they aren’t in the instructions for setting up Docksal with pull init) basically you add the key to your docksal.env file:

SECRET_TERMINUS_TOKEN="HASH_VALUE_PROVIDED_BY_PANTHEON_HERE"

 Also if you are using Linux you should note that those instructions linked above say the file goes in $HOME/docksal/docksal.env, but you really want $HOME/.docksal/docksal.env (note the dot in front of docksal to hide the directory).

Setup SSH Key

With the machine token in place you are almost ready to run the setup command, just one more precondition.  If you haven’t been using Docker or Docksal they don’t know about your SSH key yet, and pull init assumes it’s around.  So you need to tell Docksal to load it but running:
fin ssh-key add  

If the whole setup is new, you may also need to create your key and add it to Pantheon.  Once you have done that, if you are using a default SSH key name and location it should pick it up automatically (I have not tried this yet on Windows so mileage there may vary – if you know the answer please leave me a comment). It also is a good idea to make sure the key itself is working right but getting the git clone command from your Pantheon dashboard and trying a manual clone on the command line (delete once it’s done, this is just to prove you can get through).

Run Pull Init

Now finally you are ready to run fin pull init: 

fin pull init --hostingplatform=pantheon --hostingsite=[site-machine-name] --hosting-env=[environment-name]

Docksal will now setup the site, maybe ask you a couple questions, and clone the repo. It will leave a couple things out you may need: database setup, and .htaccess.

Add .htaccess as needed

Pantheon uses nginx.  Docksal’s formula uses Apache. If you don’t keep a .htaccess file in your project (and while there is not reason not to, some Pantheon setups don’t keep anything extra stuff around) you need to put it back. If you don’t have a copy handy, copy and paste the content from the Drupal project repo:  https://git.drupalcode.org/project/drupal/blob/8.8.x/.htaccess

Finally, you need to tell Drupal where to find the Docksal copy of the database. For that you need a settings.local.php file. Your project likely has a default version of this, which may contain things you may or may not want so adjust as needed. Docksal creates a default database (named default) and provides a user named…“user”, which has a password of “user”.  The host’s name is ‘db’. So into your settings.local.php file you need to include database settings at the very least:

<?php
$databases = array(
  'default' =>
    array(
      'default' =>
      array(
        'database' => 'default',
        'username' => 'user',
        'password' => 'user',
        'host' => 'db',
        'port' => '',
        'driver' => 'mysql',
        'prefix' => '',
      ),
    ),
);

With the database now fully linked up to Drupal, you can now ask Docksal to pull down a copy of the database and a copy of the site files:

fin pull db

fin pull files

In the future you can also pull down code changes:

fin pull code

Bonus points: do this on a server.

On occasion it’s useful to have all this setup on a remote server not just a local machine. There are a few more steps to go to do that safely.

First you may want to enable Basic HTTP Auth just to keep away from the prying eyes of Googlebot and friends.  There are directions for that step (you’ll want the Apache instructions). Next you need to make sure that Docksal is actually listing to the host’s requests and that they are forwarded into the containers.  Lots of blog posts say DOCKSAL_VHOST_PROXY_IP=0.0.0.0 fin reset proxy. But it turns out that fin reset proxy has been removed, instead you want: 

DOCKSAL_VHOST_PROXY_IP=0.0.0.0 fin system reset.  

Next you need to add the vhost to the docksal.env file we were working with earlier:

 VIRTUAL_HOST="test.example.org"

Run fin up to get Docksal to pick up the changes (this section is based on these old instructions).

Now you need to add either a DNS entry someplace, or update your machine’s /etc/hosts file to look in the right place (the public IP address of the host machine).

Anything I missed?

If you think I missed anything feel free to let know. Particularly Windows users feel free to let me know changes related to doing things there. I’ll try to work those in if I don’t get to figuring that out on my own in the near future.

SC DUG November 2018

Kaylan Wagner – Real world lessons from online games.

This fall the South Carolina Drupal User’s Group started using Zoom are part of all our meetings. Sometimes the technology has worked better than others, but when it works in our favor we are recording the presentations and sharing them when we can.

In November Kaylan Wagner gave a draft talk on using experiences in the world of online gaming to be a better remote team member.

We frequently use these presentations to practice new presentations and test out new ideas. If you want to see a polished version hunt group members out at camps and cons. So if some of the content of these videos seems a bit rough please understand we are all learning all the time and we are open to constructive feedback.

If you would like to join us please check out our up coming events on Meetup for meeting times, locations, and connection information.

Waterfall-like Agile-ish Projects

In software just about all project management methodologies get labeled one of two things: Agile or Waterfall. There are formal definitions of both labels, but in practice few companies stick to those definitions particularly in the world of consulting. For people who really care about such things, there are actually many more methodologies out there but largely for marketing reasons we call any process that’s linear in nature Waterfall, and any that is iterative we call Agile.

Classic cartoon of a tree swing being poorly because every team saw it differently.
Failure within project teams leading to disasters is so common and basic that not only is there a cartoon about it but there is a web site dedicated to generating your own versions of that cartoon (http://projectcartoon.com/).

Among consultants I have rarely seen a company that is truly 100% agile or 100% waterfall. In fact I’ve rarely seen a shop that’s close enough to the formal structures of those methodologies to really accurately claim to be one or the other. Nearly all consultancies are some kind of blent of a linear process with stages (sometimes called “a waterfall phase” or “a planning phase”) followed by an iterative process with lots of non-developer input into partially completed features (often called an “agile phase” or “build phase”). Depending on the agency they might cut up the planning into the start of each sprint or they might move it all to the beginning as a separate project phase. Done well it can allow you to merge the highly complex needs of an organization with the predefined structures of an existing platform. Done poorly it can it look like you tried to force a square peg into a round hole. You can see evidence of this around the internet in the articles trying to help you pick a methodology and in the variations on Agile that have been attempted to try to adapt the process to the reality many consultants face.

In 2001 the Agile Manifesto changed how we talk about project management. It challenged standing doctrine about how software development should be done and moved away from trying to mirror manufacturing processes. As the methodology around agile evolved, and proved itself impressively effective for certain projects, it drew adherents and advocates who preach Agile and Scrum structures as rigid rules to be followed. Meanwhile older project methodologies were largely relabeled “Waterfall” and dragged through the mud as out of date and likely to lead to project failure.

But after all this time Agile hasn’t actually won as the only truly useful process because it doesn’t actually work for all projects and all project teams. Particularly among consulting agencies that work on complex platforms like Drupal and Salesforce, you find that regardless of the label the company uses they probably have a mix linear planning with iterative development – or they fail a lot.

Agile works best when you start from scratch and you have a talented team trying to solve a unique problem. Anytime you are building on a mature software platform you are at least a few hundred thousand hours into development before you have your first meeting. These platforms have large feature sets that deliver lots of the functionality needed for most projects just through careful planning and basic configuration – that’s the whole point of using them. So on any enterprise scale data system you have to do a great deal of planning before you start creating the finished product.

If you don’t plan ahead enough to have a generalized, but complete, picture of what you’re building you will discover very large gaps after far too many pieces have been built to elegantly close them, or your solution will have been built far more generically than needed – introducing significant complexity for very little gain. I’ve seen people re-implement features of Drupal within other features of Drupal just to deal with changing requirements or because a major feature was skipped in planning. So those early planning stages are important, but they also need to leave space for new insights into how best to meet the client’s need and discovery of true errors after the planning stage is complete.

Once you have a good plan the team can start to build. But you cannot simply hand a developer the design and say “do this” because your “this” is only as perfect as you are and your plan does not cover all the details. The developer will see things missed during planning, or have questions that everyone else knows but you didn’t think to write down (and if you wrote down every answer to every possible question, you wrote a document no one bothered to actually read). The team needs to implement part of the solution, check with the client to make sure it’s right, adjust to mistakes, and repeat – a very agile-like process that makes waterfall purists uncomfortable because it means the plan they are working from will change.

In all this you also have a client to keep happy and help make successful – that’s why they hired someone in the first place. Giving them a plan that shows you know what they want they are reassured early in the project that you share their vision for a final solution. Being able to see that plan come together while giving chances to refine the details allows you to deliver the best product you are able.

Agile was supposed to fix all our problems, but didn’t. The methodologies used before were supposed to prevent all the problems that agile was trying to fix, but didn’t. But using waterfall-like planning at the start of your project with agile-ish implementation you can combine the best of both approaches giving you the best chances for success.  We all do it, it is about time we all admit it is what we do.

Cartoon of a developer reviewing all the things he's done: check technical specs, unit tests, configuration, permissions, API updates and then says "Just one small detail I need to code it."
Cartoon from CommitStrip

Thoughts on Hacktoberfest 2018

This year I took part in Hacktoberfest. Partially to see what all the fuss is about, partially to get myself involved in projects I didn’t know about, and partially for the free t-shirt (which do come in men’s and women’s cuts).  If you haven’t run into this project before it’s an effort by Digital Ocean to get people to participate in open source projects. Once you sign up they count all public pull requests you make on Github toward a goal of 5. I participated both as a developer, and by tagging a few issues on my own projects so people would find them.

As a developer:

It was a great excuse to go find new projects and look at ways I can contribute.  While I’d have plenty of experience on open source projects, often they have been outside Github or are repos I have commit access to – so I don’t open a lot of pull requests on Github. That meant that Hacktoberfest was a chance to find new projects and practice a basic process for contributing code to teams.

In that regard it was a pretty good success. I opened six PRs on four different projects. Mostly they were small stuff like linting code, updating packages, or tweaking a README file.  

In terms of drawing me into projects we’ll see. I did keep up with one after I finished the 5 required (hence having six PRs), but I didn’t dive into anything truly hard on that project.  

In terms of getting me to provide truly useful code think that was limited. The largest piece of code I wrote was initially rejected so I re-wrote in a different style, and then re-written by the project maintainer the day after he accepted the PR. He was really nice about it, and it helped him get something done that had been on the to-do list for a long time, but even that was example code to be used in classrooms (which was why he was so concerned about style – he didn’t want it to be idiomatically correct for Python he wanted to clear to beginners).

It did give me a chance to play around in other people’s code bases and I did resolve some issues for people that would have otherwise lingered longer than they already had.  It also forced me to meet other people’s standards, lint to their specifications, and pass their automated tests – all good things for everyone to do now and again to see if there are solutions you like better than the ones you use every day.

As a project owner:

Once I got through the contributions I needed to get a shirt, I figured I’d look over my own projects to see if there were issues I could label for beginners to help them find ways to get started. I listed several issues are both Hacktoberfest and good first issues. Almost all the ones I flagged as good first issues got PRs opened – sometimes more than one.

I got two problems solved that I wouldn’t have known how to solve without a bit of research, and those were great. But most of the PRs were simple things that took me longer to solve collaboratively than it would have taken me to solve myself. That’s okay, in part because some of my PRs caused the same problem for their project maintainers, and because it forced me to final learn how to setup CircleCI so the code gets checked and tested automatically when PRs are opened in the future.

What I don’t expect it caused was anyone to be truly interested in the project and helping it move forward over time. So while I solved a couple small problems, I did not get new help that going to keep engaging. That made it useful as a sprint, but not useful to helping build great projects.

But even if there is room for improvement my shirt is ordered and on the way.

Throw away information

When I was first starting out in IT I had a senior colleague who commented to me about all the throw away knowledge she’d collected and tossed over the years. She talked about learning early versions of VB, an obscure OS for an old HP minicomputer, the aging phone system we had, and all kinds of other things she’d learned in her career that was, or was about to become, useless knowledge.

Liz taught me a lot of things, but that basic observation about spending lots of time acquiring information that promptly becomes useless stands out in my mind as a useful caution. It has also become something I’ve try to plan around and compensate for as I advance.  For example I almost never bother to memorize an API: I can generally keep the documentation handy which is often updated when new features are added where my memory would lag behind.

It’s also part of why I love puzzles: they keep my brain in practice at gaining and tossing aside detailed information.  There is just about nothing as useless as the things you teach yourself while doing a jigsaw puzzle. The small details that allow you to associate pieces with parts of the puzzle. The ways you sort pieces for a particular puzzle. All kinds of things that really only apply to the puzzle you’re doing and not any other.

Puzzle underway.
The start of the cat puzzle at the top of this post as it got started. Stacks of red, white, brown, and green pieces are gathered on lower right while I finished the boarder. The wood grain and basket weave would be sorted during later passes.

There are plenty of studies that show we need to keep our brains active doing different things to help keep them healthy – some even look at the value of puzzles. Programming, and other tech work, certainly does that by default, but as any of us works with a specific platform we get used to the patterns it uses and therefore slow our rate of learning.  Puzzles also are good for finding practical applications for basic computer science concepts.

Sorting in particular is something that puzzles encourage different approaches not all of which are routinely needed in development work but are good to remember are out there. The concepts of the core of radix sorts  – or bucket sort – are extremely useful, and can even be used as a form of compression for a puzzle. My puzzle desk is a bit too small to allow me to spread out all the pieces of most incomplete puzzles, but digging through all the pieces over and over is just a waste of time.  So I often find a basic pattern I can do quick sorts by, and stack up pieces by color, or those with book covers, or other other easy to quantify detail. Then as I move through assembling the puzzle I can grab a stack of those pieces and do another round of sorting and assembly. Like a radix sort, you don’t need to first pass to be useful to make progress on the overall solution.

Puzzle pieces spread on a desk showing sample sorting collections.
The next puzzle getting started with edges sorted on the left and rough collections forming on the right. That big collection on the right is text and poster edges (puzzle is a collection of classic Star Wars posters).

Liz was right. I too learned and dropped VB; that stupidly designed phone system was mine until we replaced it with one slightly less stupidly designed; Drupal 4, 5, and 6 all had particularities I recall only to be nostalgic about the past; and the things I’m teaching myself these days about Salesforce will one day be equally useless. That’s okay, it’s what I signed up for, and if you are finding your own way in this field it’s a reality you should plan on being a part of the rest of your career.

Developers need to write more than code

Developers need to be able to in write their primary spoken languages as well as they can writing in their primary programming languages. It takes effort and practice, just like programming does. And you get rusty if you stop for awhile, just like in programming. I started blogging in part to make sure I was writing on a regular basis. When I worked at AFSC I was writing and editing everyday for my work, but the next two jobs I had didn’t expect much writing from me. I’ve written before about the value of my liberal arts education, and the last few weeks have been a testament to effort Hamilton put into making sure I learned the fundamentals of writing.

In my new job we work hard to provide a great deal of information to our clients and to make sure they can understand that information. While each team member brings a different set of strengths and weaknesses to the projects, we are all expected to communicate clearly with each other and our clients. All of us are expected to explain our parts of the project so that our clients can understand our advice and make can informed choices about how the projects should proceed.

In the last month I’ve been responsible for presenting basic overviews of platforms and highly technical reviews of parts of projects. I have contributed to large architectural design documents and focused detailed designs for small subsystems. All these materials were done collaboratively by teams of highly skilled technical people, but our client’s technical skills are much wider ranging. Often they are experts in other areas who need to use the tools we’re building. Others are experts in the technology we work with, and we are serving as added capacity to their in-house teams. We are expected to explain ourselves equally well to all these people, which means we need to both be clear and thorough.

One of the things I try to bring to the project teams is a willingness and ability to edit and be edited.  Because I spent time responsible for all the digital communications of a large organization, I have experience quickly and aggressively editing documents of all sizes. I also know how important it is to have an editor who catches your mistakes, and while I carefully review someone else’s work I am always supportive of anyone who is editing a section I wrote. Not everyone is equally comfortable with these roles, but they are things I believe all developers should try to master.

We have socially come to expect that technical people do not explain their work clearly. We are too often allowed to use a great deal of jargon, skip lightly over hard to understand details, and belittle those who cannot keep up. Complex systems are indeed complex so it does take effort to clearly explain and fully understand them. But the vast majority of developers are perfectly capable of explaining what we do when we take the time. And the vast majority of people are capable of understanding clear descriptions of our work.

Developers who are unwilling to take the time to learn to explain their work well do a disservice to their colleagues, clients, and themselves. Their work will suffer from a lack of good feedback, and so will not be as good as it could have been with more support. Those weaknesses will become bad user experiences, bugs, and other flaws in their final products.

A well done code review (one that’s meant to be supportive and doesn’t include yelling) and a good edit aren’t that different. In both one person is looking over the work of another to try to understand it and check for errors. Done well both give a chance for the reviewer to learn from the material and to help the person whose work is being reviewed do their best work.

When we explain our work well we open ourselves of up for feedback. That feedback gives us chances to validate our designs, improve the experiences of our users, and generally create more awesome products.

What else should I have asked but haven’t yet?

Every time I go through a job search either as a candidate or reviewing applications I try to learn a few things to make sure I am better prepared in the future and to help friends looking for work and talent. I recently completed a job search, so I want to share what I learned this time around.

I found a great question for a candidate to ask:  “What else should I have asked you?”

This is actually a question my wife and I started asking several years ago when we were buying large ticket items we didn’t know much about – like cars, houses, and HVAC systems. One of us, I don’t remember who, asked one of the salesman if there was anything he thought we should be asking him and his competitors. He gave us a couple small tips – likely things he thought he could answer better than his competition – and it gave us an insight into details he thought were important. Once we asked that question of all the sales people we had a list of questions to ask that covered more perspectives than we would have been able to figure out on our own.

It’s now a staple question we ask when starting anything new. Instead of trying to pretend to be experts we ask people for guidance. Often the answer is “No, I think you covered it.” but sometimes we learn things or are told about discounts, features, or services we would have otherwise missed. Outside of purchasing we’ve found the question can help spur conversations and get people to tell us things we need to know – it’s a question we use a lot as Guardian ad Litems.

Most good interviews include a time for the candidate to ask questions. This should not be a pro forma detail crammed in at the end. If the interviewer is taking your needs seriously they will give you several minutes for your questions that give you a chance to round out who you are as a candidate, this is particularly true when talking with the hiring manager (if they don’t take this seriously you should think about whether or not you want to work for that person). This portion of the interview is a critical chance to gather information about the organization, your potential role, their existing team, and vision of the future. It is also a chance to ask questions that highlight your experience and knowledge. Most advice you will find online will tell you to make sure you have a few questions you want to ask to try to draw out the information you need while showing off that you’re smart and talented. Doing this well can be hard. I discovered that having a simple, and reliably unusual, question that I can ask at the end gives a good last impression and this one has gained me unexpected insight more than once.

The exact wording isn’t important here. I’ve asked several versions:

Are there other things I should have asked but haven’t?

Are there questions you aren’t hearing from candidates that you expected?

What else do you think someone should be asking about before joining your team?

Are thing questions you would ask if you were in my shoes?

The idea is to ask an open ended question that shows you know there is always more information to be gained and gets them to think about things they haven’t shared with you, or with other candidates. The question alone often stands out, and if you get them to discuss something with you they didn’t discuss with others that helps you stand out in their minds even more. It also gives them a chance to talk about things they know and you don’t, which can help give a positive impression of you (this is same idea as dating advice that encourages getting your date to talk about themselves in part because it will make them think you’re smarter).

We all tend to want to know the same things when considering a job. This portion of the interview allows you to fill in gaps in their job ads and the conversation you’ve had so far. But since all job seekers want similar information they are asking similar questions. As for showing off, the hiring manager has likely already pre-selected a group that has shared backgrounds they are looking for so you aren’t going to easily stand out from that crowd of people with similar professional backgrounds. But by asking an unexpected question that puts the creativity on the interviewer you might be able to trigger a conversation that gives you that extra attention.

For me the question worked best in group interviews, because finding good questions is hard for me in that setting and it sometimes triggered discussion and debate within the team about things they wanted to hear candidates asking. It gave me a chance to hear a set of perspectives I wouldn’t have heard otherwise, and to see the team disagree about their vision for what’s needed. The most successful was when they fell into a mode of answer each other’s questions. For 15 minutes I moderated a discussion of what the team needed from their newest members and watched the internal team dynamics and politics played out in front of me. Usually the responses more mundane, but still helpful. Never did I feel like it was a foolish thing to have asked since the worst answer I got was a long pause and “Well that’s interesting, but I think we’ve covered everything I think you need to know.” followed by a quick check list of details the person thought it was important for candidates to know (the details of that list helped confirm why I didn’t want to work for that manager).

The question is also practical in a pinch. One of my interviews was rushed, I had just two hours to prepare after a first round interview so I didn’t have time to think of new things to ask. To add to the challenge the interviewer answered most of what I’d come up with before we got to my turn. I think I managed one or two detail questions that mostly clarified something she told me before I switched gears and asked something to the effect of “What else should I ask about before taking the job?” The question allowed me to stand out to her as asking questions that suggested I wanted to make my career move carefully (which was true and a good thing from their perspective too), and got the conversation into a productive place about the team’s role within a large organization. I start the job from that interview this week.