The Queries Part 1 of 3

This is the first in a series of posts to break down the questions from my Queries on Queries talk. The full talk is available here.

Are your tools good enough?

Our migrations live and die by our tools.

Are your tools built for the scale of your project? Do they empower you to do your best work or impede rapid progress? Would a new tool serve you better now or in the future?

Having the right tools is critical to any job. In data migration we primarily talk about ETLs (Extract, Transform, and Load): tools like Jitterbit, Informatica, Mulesoft, Talend, etc. We also use additional tools to help support the process: a task tracker like Jira, a Data Modeler like Lucidchart, staging database prep like Salesforce2Sql, and more.

It’s easy to say that it’s a poor carpenter who blames his tools, but anyone who has spent time with actual carpenters knows they care a great deal about what tools they use. They might be able to make due with poor tools, but they will do their best work with the right tools for the job.

Each tool you use needs to meet your team’s needs. It should play to your strengths, supports the kinds of projects do you do, and has an eye to the future. A tool that works great for a team of declarative Salesforce consultants might drive developers crazy. A tool that works great for 10’s of thousands of records might struggle with millions; a tool scaled for 10s of millions of records may be overly complex for a project of 30,000.

Make sure you’re using the tools that let you do your best work, now and in the future.

Do you make the data atomic for processing?

Smaller pieces of data are easier to track, manipulate, and test.

Do you divide the source data into its constituent parts? Can you process individual pieces of data easily and cleanly? Can you stop your process after each stage to validate the results?

It can be tempting to process data as it comes: handling whole rows of data in the form they were provided and treating fields as a single data point. In practice exports may have extra rows or columns to deal with related records. Organizations may have encoded multiple points of data into fields like ticket names including a show name, date, and time into the name field. Fields can also contained semi-structured data, like Joomla’s use of arbitrary JSON blobs.

To process this data it is often easier and clearer to extract it from these structures prior to direct processing. It’s not always needed, and rarely required, but doing this clean up of structure – like creating interstitial database tables or predictable data objects – can greatly ease the rest of your job.

Like many problems in software engineering, it’s easier to do good work when you are operating on atomic pieces. Think about the right ways to pull your data into constituent parts when they aren’t there already.

Can you process samples of your data set?

When you have lots of data you need to test small parts to be sure your process works.

Do you know how to create and run small segments of your total input? Are your segments made up of complete and valid samples? Does your sample include all the errors and edge cases your data set will throw at your process?

If you are working with small data sets your sample can be all the data. But when you have a large data set you need to test your process with samples. When you have a multi-step migration you likely need to test the second phase while the first phase is still under construction – again a sample data set is critical.

Having valid test data, that covers all your edge cases, is critical to making sure you have a working solution.

A few years ago I worked on a project that involved a two stage migration for a membership organization with some 600,000 active contacts. Every one of them needed to be migrated into Salesforce and then into Drupal. To test the Drupal migration we needed samples of all the types of membership statuses we would see, which involved hand creating several hundred records. At the next Salesforce Commons Sprint I raised the idea of needing a better tool for this kind of work, that question eventually helped lead to Paul Prescod‘s creation of Snowfakery. Snowfakery will build you testing data sets of any size and complexity to make sure your processes will succeed.

Queries on Queries: Improve your data migration

Last week I gave my Queries on Queries talk, intended to help you improve your data migration process, as a webinar for Attain Partners.  It’s a revised and improved version from the last time I gave it.

These questions aren’t like the old Joel Test (which is still useful) where the right answer is “yes”. These questions are designed to point you in a direction but allow you to change your answer over time. I generally answer this questions with a paragraph not a word. Use these questions as a challenge to make you and your team better.

Over the next few weeks I’m planning to publish a series that will include each query and why I think it’s useful in helping you think about how to improve your process.

Take Good Notes

A good set of notes is how we build a memory of what happened.

Good note taking is important in nearly any white collar job, particularly consulting. If we have a long conversation with a client and have to re-ask them about all the details, they will rightly be annoyed. They may demand to know what they paid for the first time we talked.

Why we take notes

In school we are taught to take notes. Teachers expect students to remember information to pass tests, write papers, and other evaluations of learning. Too often teachers will try to convince students to take notes in specific way. They may make note taking into an assignment and assessment of its own. My wife sees college students who decide that they don’t need to take notes because she does not grade them. These students do not do well. These students missed the point of taking notes. The form of the notes is not important, but the existence of them is.

When we leave school notes serve two main purposes:

  1. Record events of a meeting so there is a shared record later.
  2. Help us remember what happened so we can do our work.

Every important meeting should have someone charged with creating the first of those. How you do that task assignment is work place and team specific, but it needs to happen. Doing this well is an important skill, and every team, board, religious community, and so on needs people who do this well. But the second type of notes are often more important for day to day work; they are an important how the participants remember what needs to happen.

This second category of notes is why there is nearly always a notepad near me when I’m working. I scribble down thoughts, tasks, key points, and anything else I need to remember later. The notes I take are messy, disorganized, and useless to anyone but myself. None of that matters as long as I remember what I need to know.

Why take notes yourself

Lots of people hate to take their own notes. I have colleagues who treat note taking as a task to be avoided. Heck, I dislike being the official note taker when it’s my turn. People often fall back on the “official” notes of meetings instead of keeping their own. I have heard people go so far as to declare additional notes are just a waste of effort. I have had colleagues claim this “wasted” effort is somehow costing the client money (not true, they were in the meeting anyway).

Writing notes encourages us to engage with the content. There are no shortage of studies on the impact of note taking and memory formation. The research clear indicates that if you engage actively with information you will retain it better. Any form of note taking that encourages you to engage is a good start. That engagement can be exhausting, but that does not justify avoiding the work.

Even when in a meeting with an official note taker, our clients are best served by everyone on our team taking notes. That helps us all learn about the client needs, to contribute to the project work, as well as offer edits to official notes after a meeting wraps up.

Can an AI note taker do just as well?

The recent public emergence of generative AI has captured a great deal of attention. We are thinking of all the places that a machine can take over tasks we thought required a human – particularly those we dislike. There are already services like Otter.AI which will attend virtual meetings and generate notes for you.

My experience suggests that, right now, they are pretty terrible at their main job. The automated transcripts they require are adequate at best; their note taking ability is worse. The samples I’ve seen from meetings I was in were basically useless. Worse yet, AI tools will lie (or more accurately they generate believable, but false, information), which is getting people into trouble. After all those issues, you will need to deal with the privacy and security implications of allowing a system listen into your meetings.

One day these systems will likely be pretty good for official meeting notes, but that’s not today. Even at that point, those AI’s will not help you engage with, or retain, the information.

Do yourself a favor, no matter who or what else is taking notes, take your own.

What makes good personal notes

Fundamentally what makes good personal notes is whatever you can use to accurately recall what happened. If you are able to recall the details when you need them, your notes were good enough. If you cannot not, your notes aren’t good enough. That’s true during your education (unless a teacher is grading your notes, then play along with their instructions), that’s true in the work place.

There are several formal patterns for note taking to help you structure the information. If you are struggling to take useful notes, I recommend trying one or two to see if they work for you. Those patterns do not work well for me, and I have bad memories of being made to outline lectures, and other patterns as the “one true” solution for taking notes.

Later in my education I picked up the metric I use now: do they work. I found I retain information best when I am summarizing bits and pieces to trigger my memory. My own notes are often just a few words to draw my brain back to key points. I will write out a major decision, pronouncement, or useful quote from time to time – that extra detail emphasizes to me later that I thought was a major point at the time.

Find your own style of note taking, but do not pretend you do not need them.

Real Life Sorting

Sorting is a basic part of any computer science program. My sister and I are currently engaged in helping my parents move. As part of that effort I am spending sorting things, which has reminded me of another place where it can be useful to apply things you learn in one field to another.

Bucket and Radix sorts are basic sorting approaches taught in any good algorithms class. The process involves sorting like items into buckets, and then resorting items, either within those buckets (bucket sort), or by their next property (radix sort).

Piles of coin mid-sort
Coins mid-sort, with like piles gathered, and quarters being sorted for uniqueness and rolling.

In practice, as a developer I nearly never implement my own sorting algorithms; a language library is almost certainly going to be better optimized than what I would write, and it already exists. What’s more these two sorting processes are rarely the best in practice for most datasets. But in real life, we need to sort stuff all the time, and where these don’t make sense in many computing settings they are often the best way to sort physical stuff.

I spent my evening applying this theory to sorting the coins my father has collected over time (he’s actually a few feet away sorting more coins right now). I took the embedded picture mid-process this evening between having gathered coins by type, and was pulling out Quarters for collecting vs rolling — other denominations will come later. My sister and I have been helping prepare for the move with a similar process on a larger scale of gathering like-items together to help review and pack them as we progress. There is even science behind the concept that this is the most efficient way to sort things.

Unlike that article, my point is not just to help you sort things faster, but to support for my general argument that having well rounded education is the best form of education for life-long engagement. When I first read the article saying that radix sort was the fastest way to sort socks it made total sense to me – I already sorted socks that way because I’d realized it helped break the problem down. I took the idea from the same place the author did, my computer science training.

Generally I make this argument in the reverse; non-CS courses made me better as a developer. But it’s just as true that my CS courses taught me things that make my day-to-day life better.

No one field has all the answers to our problems as individuals or as a society.

Salesforce Data Migration Lessons

Last week I was part of a webinar for Attain Partners talking about Salesforce data migrations. One of the questions the moderator, Eric Magnuson, asked was the three lessons I’d learned doing data migration work.

The answers I chose weren’t so much from my first data migration projects, as from more recent projects. Those early migrations were tiny by my current standards. Back then, I was mainly the consumer of migrated data. When I did migrations they were small and manual. The intellectual process was similar but the scale meant I had lots left to learn about large data projects. The lessons I chose are focused on what I wish I knew when doing the migration for clients with vastly larger data sets. When I became the producer of migrated data.

My three lessons:

  1. Manage expectations from the start.
  2. Don’t be a hero.
  3. Understand your tools, and use them well.

Manage Expectations from the Start

This still surprises me, but it’s true: one of the biggest reasons data migrations get into trouble is bad expectations.

People tend to think data migrations are easy. You create a map from the old system to the new, run some processes to convert the data, load into Salesforce. But in practice, that map might involved thousands of details. Those conversions are hard to get right for every variation of the old data. And loading large amounts of data is never as easy as it looks.

When people think something is easy, and then the outcome is less then perfect, they get mad.

The problem is, data is always messy. We do migrations when we replace systems. We replace systems when the old one has problems. Problems in a system, lead to data errors.

That reality is made worse by the reality of big system switches. The freshly migrated data lands in a system the primary users are still learning. Those factors lead to confusion, mistakes, errors, and misunderstandings.

From the very first conversation I tell our clients to expect migration errors. Our first mock run of the migration is messy – sometimes very messy. The whole point is to find errors. The second mock run will be much better, but still imperfect; we’ll find more errors. No matter how many times we do test runs, there is only some much we can do with imperfect inputs.

We can make your data better then it is, but we can’t make it perfect. If you expect perfect data, you will be disappointed. I want you to be thrilled by your new system, and that means you need to understand your migrated data will have flaws.

Don’t Be A Data Hero

When I first did data migrations I was often a one-person show. I was responsible for figuring out all the details, implementing the process, reporting to the client, and fixing all the flaws. It’s a common story for people doing migrations.

We find ourselves up late at night, working through piles of detail, trying to make sure the client is satisfied. It encourages a hero mentality: I’ll make the project successful through shear will.

And most of us can do it. Being a data hero isn’t that hard if you put the hours in. It is, however, miserable.

People doing migrations need, and deserve, support. Now that I’m leading a team of people doing migrations I have added the support I should have had. I created a space for us to come together and talk about our projects.

We ask each other for help and suggestions. We offer ideas for how to improve our processes. We talk about ways to address client concerns. And yes, we complain to one another. But mostly what we do is make sure that no one is alone. Everyone, myself included, has support and back up.

A team is stronger than a person. We don’t need to be heroes. We do better work, and are happier people, when we support one another. Good work, by happy people, makes for successful projects.

Understand Your Migration Tools

Data processing tools are code generators – understanding that allows you to use them well.

Both parts of that make sense once you say it:

  • Tools that allow you to design an arbitrary process that takes inputs and generates outputs are obviously writing code at some level.
  • If you understand any tool better, you will use it better. That’s true of a hammer, a screw driver, or a piece of software.

I learned how to migrate data from people who weren’t formally trained developers. They were using the tools the best way they knew how, but didn’t have the background to apply software development best practices.

When I combined the tool usage they taught me, with the software engineering practices I already knew, I created vastly superior solutions. Our team now creates processes that are easier to setup, run faster, and allow us to fix all errors (even if we missed them until after launch).

Apply Lessons our Salesforce Data Migrations

I apply these lessons to the Salesforce data migrations I lead in my work at Attain Partners. I combine that with my queries on queries review process, and am constantly building better solutions for our clients, our team, and myself.

Becoming a Salesforce MVP

This week I learned that I am now a Salesforce MVP!

I know that I was nominated by at least Chris Pifer, Samantha Shain, Paul Ginsberg, and Cassie Supilowski. I appreciate their support, and also everything they have taught me over time. They are an august group. Two are Rock Star Admins and two are existing MVPs (three now, as Sam is also in the 2023 MVP class). It is my a privileged to know, work with, and learn from all of them.

I got here with community support

The Salesforce MVPs are a group of people who are recognized within the Salesforce community as leaders and contributors. It’s a group of people who, in addition to our day-to-day work, also support other people’s work and learning.

For me that looks like posts about Salesforce here, participating in Salesforce.org’s Commons program, and generally supporting people I meet along the way. While I contribute as I’m able, my own contributions are fueled by support from my nominators and others.

Sam, Cassie, and I are all leaders the Commons’ Data Generation Toolkit project. Chris has been a friend and colleague nearly my entire adult life – we often remark that we’ve had about every possible professional relationship. I met Paul planning Nonprofit Dreamin’ a couple years ago and we continue to talk regularly about life, Salesforce, and nonprofit (in no particular order). In other posts I have talked about the influence mentors had on my work.

Looking forward to more

Being a Salesforce MVP gives me access to tools, events, information, and opportunities that are new to me. It also comes with the expectation of continuing contributions to the community.

The good news for me is that I generally like contributing and supporting others. I am looking forward to continuing to advancing my personal projects, contribute to Data Gen Toolkit, presenting at events, and generally helping other succeed.

If your group needs a speaker reach out and let me know. I love to talk about Salesforce, data, the web, career paths, and just about anything you find me writing about here.

We Wish We Could Consult Like Sidney Freedman

Major Sidney Freedman was often the perfect psychiatrist at the perfect moment – all consultants wish we were as effective as Sidney. Major Freedman, most often referred to as Sidney, was a recurring character on M*A*S*H who comes to the 4077th a few times a season. Sidney always seems to know what his patients need; he has the perfect thing to say, while being compassionate to whoever he’s there to help (including himself).

Near Perfect Treatments

The breadth of Sidney’s skills are staggering. Sometimes simple talk therapy is all that’s needed. So when Hawkeye’s nightmares become too much Sidney pops in visit for a few sessions. And when Potter fears his surgical technique is fading a conversation with Sidney is all he needs. When everyone at M*A*S*H is stressed by their work load, a few minutes with Sidney and a prescribed bonfire steadies everyone’s nerves.

But when a really tough case comes up, Sidney is still your man. During one episode a wounded bombardier becomes convinced he’s Jesus. Sidney not only provides an emergency diagnosis but puts his career on the line to defend the man. Faced with a highly decorated soldier who is suicidal, Sidney’s quick hypnotherapy session stabilizes him. The show credits Sidney with advancing treatment of “combat fatigue” (we’ll ignore his treatment was standard 30 years before the Korean Conflict and the work of combat medics); we see him around several related patients (eventually including Hawkeye). He even remembers enough of his medical school training to pinch hit as a surgeon and offer up advice on his way out the door.

“Ladies and gentlemen, take my advice: pull down your pants and slide on the ice.”

Sindey Freedman’s best known piece of advice. Oddly fitting in the moments offered.

He even cares for himself with compassion. When patients are mad at him for his care, he knows it is part of them getting better. When he losses a patient, he knows to take a break at the 4077th and write a long letter to Sigmund Freud.

We all want to be Sidney

Everyone who works in consulting wants to be our own version if Sidney. We all want to be the person who knows exactly what’s needed at exactly the right moment. We want say all the right things. Ask the questions to get to the important truths about a project. Laugh when people are funny, then serious when we need to be. We hope to create or find the latest cutting edge solution that will provide exactly support for our client’s problem. And we want to leave them smiling and wanting more of our help.

Of course, unlike Sidney, we do not have a team of writers providing the lines. Our work doesn’t fit into tidy 22 minute episodes. New cutting edge solutions we find often don’t always work as advertised. When we are tired or frustrated, we make mistakes. When we are fresh and well rested, we still make mistakes.

The best we can do is try to follow Sidney’s example. He stays calm under stress. He stays current with his field’s research. Most importantly, he cares about his patients and provides them the best care he knows how. He even remembers to care for himself when suffering from burnout.

M*A*S*H Consulting

If you didn’t know, or can’t guess from context, my wife and I are M*A*S*H fans. We grew up with it. We bought the DVDs as they Fox released them. It’s our go-to for something comforting to watch. It’s entertaining, and full of great lines and moments to borrow for life examples. As part of marking the show’s 50th anniversary this is part of a short series of posts using consulting lessons from characters in M*A*S*H.

Do Not Consult Like Col. Flagg

Sometimes we learn best from counter examples. Col. Flagg was an recurring character on M*A*S*H. He was a CID man with Army Intelligence, and spent much of his time bloviating about his suspicions about everyone while threatening to kill people for fun.

Flagg seemed to have taken every piece of tactical advice he was ever given to absolute extreme. It leads him apply his most aggressive techniques – like threats of torture, or attempts at bribery – to every situation he encounters.

Consultants get a lot of good tactical advice about how to address our clients and our work. But if we take things too far we are only slightly less clueless than Flagg.

Col. Flagg’s Over-Generalization

Probably the best example of Flagg’s constant misapplication of techniques by generalizing their use is in The Abduction of Margret Houlihan from season 5. Chief Nurse Margret Houlihan hurries off late at night to deliver a baby in a nearby village, telling only the guard, Kilnger, who promptly goes to bed. When no one can find her in the morning, Potter calls for help searching for her. The army’s answer is to send Col. Flagg. He arrives, dressed as Mussolini, claiming to look like a Chinese double agent (if that agent chose the same get up), and proceeds to request the provisioning of several items clearly planning to torture someone.

Flagg spends the day wondering around, threatening everyone he talks with, trying to get information about Houlihan. He never finds anything of use, but does get in a number of good one-liners.

Finally Houlihan, who was never in any danger, returns to camp unaided. Flagg declares victory, throws himself out a window – so know one will see him leave – and breaks his leg.

Anyone who had been a consultant long enough can find parallels between Flagg’s useless bloviating, and consultants who try to address every client project as interchangeable.

Col. Flagg Consulting

Good advice often is context specific – you need to know when and how to use it.

Anytime I am in a meeting with a client, and I recognize that a piece of stock advice may apply, I try to reflect on the context of the moment. Does the advice actually apply, or would a different approach be better.

For example, consultants are often told to never say “I don’t know” in front of a client when asked a question we are unsure how to answer. Instead we are taught to say “I need to get back to you about that” or “let me verify with another team member before I respond.”

Up to a point this makes sense. We are paid experts; clients don’t want hire someone who does not know how to complete the job. Often making a space to make sure you are correct is the right response to an unknown. It’s far better than making up wrong answers. And, more often, better than making a client nervous because you haven’t had 30 seconds to Google the answer.

But clients can also smell something is fishy if you always claim to know everything. When I was a client, I never trusted consultants who pretended that they had seen and done everything. I knew they were lying to me because no one knows everything – leaving me to sort truth from fiction. That never ended well for the consultant.

“Nobody gets the truth out of me. I keep myself in a constant state of confusion.”

Col. Flagg

Consultants who cannot track context and adjust their approach accordingly, risk consulting like Col. Flagg.

M*A*S*H Consulting

If you didn’t know, or can’t guess from context, my wife and I are M*A*S*H fans. We grew up with it. We bought the DVDs as they Fox released them. It’s our go-to for something comforting to watch. It’s entertaining, and full of great lines and moments to borrow for life examples. As part of marking the show’s 50th anniversary this is the second in a short series of posts using consulting lessons from characters in M*A*S*H.

Consult like Father Mulcahy

Father John Patrick Francis Mulcahy (aka Father Francis John Patrick Mulcahy depending on the season), spent 11 seasons of M*A*S*H trying to balance his two realities. He was a priest, opposed to violence (aside from boxing), thrust into a war zone. As the 4077th chaplain he was responsible for the spiritual and emotional care of more or less everyone else. Father Mulcahy spends many scenes helping in OR. He supports the doctors however he can; he brings supplies and drinks to the staff; offers prayers for the dying; even assisting in surgeries at times. 

Mulcahy’s Balance

Throughout the show his story is one of a man struggling to find balance – sometimes he does it well, sometimes he misses. The writers make his goal most clear in the episode Heroes. A former boxing champion has come to came on a USO tour stop and has a stroke that is eventually fatal. As the boxer lays in a coma, he explains the influence the champ had had on his world view. 

Mulcahy talks about his struggle with the idealism in Plato and his desire not to be beaten up at school. He tells the story of the first professional fight he’d seen. The now dying champ, had asked the ref to stop the fight so he didn’t hurt his opponent too badly.

“And I realized for the first time, that it was possible to defend myself and still maintain my principles. … That was when I made up my mind to keep one foot in the ideal plane and the other foot in the real world.”

Father Mulcahy

Mulcahy Consulting

Consulting is, obviously, less challenging than Father Mulcahy’s world. But M*A*S*H has a lot of useful ideas to use to inspire us in our lives. If you aren’t familiar with the show, I highly recommend it (although new viewers might start in Season 2 or 3 since it took them a season to find their feet).

Balancing the ideal and the practical is one of the challenges in consulting. We can often see an ideal solution for a client, but the client doesn’t have the time or budget to reach that ideal. Our job is then to find a balance between that ideal solution and meeting deadlines and controlling budget. We need to keep one foot on the ideal technical plane, and the other in our practical world.

Like Mulcahy, sometimes we get frustrated that we can’t do our best work for doing for reasons that feel short-sighted. Finding ways to work on personal projects or other things that allow you to be technically purest can help keep skills sharp and scratch itches can help. Unlike Father Mulcahy, leave the choice to charge into other people’s battles to rescue a wounded soldier to the properly trained.

We may sometimes error in the other direction. Like Mulcahy trying to write the perfect sermon during a visit from a cardinal, and miss the place we should really be (hopefully we can pull it together as well as he does). It’s easy to get focused on our own goals, and miss a client’s needs go in a totally different direction.  But it’s our job as consultants to course correct and deliver on our obligations.

In the end try to consult like Father Mulcahy would.

M*A*S*H Consulting

If you didn’t know, or can’t guess from context, my wife and I are M*A*S*H fans. We grew up with it. We bought the DVDs as they Fox released them. It’s our go-to for something comforting to watch. It’s entertaining, and full of great lines and moments to borrow for life examples. As part of marking the show’s 50th anniversary this is the first in a short series of posts (not sure how many yet – at least two maybe more) using consulting lessons from characters in M*A*S*H (likely avoiding some of the obvious choices like the doctors for things we can take from others).

Thinking About Languages

I’ve been thinking about languages a lot lately.

Mostly the kind I encounter in my work, programming languages, but also human languages and the intersection between the two.

My thinking runs parallel to what I talked about bringing from my Drupal experience to my Salesforce work. I have been considering what I’ve taken from work on one programming language that made me better in others. 

A few weeks ago I asked some friends in various professional circles about their thoughts on programming languages. “What’s your personal theory about learning new languages? Have you ever learned enough? How often should you learn a new one? How do you count languages you last used 5+ years ago? What’s reasonable to expect of other developers?”

The responses ranged widely, but generally all agreed we benefit from learning more languages throughout our careers, but particularly when we are starting out. Not surprisingly the clarity of thinking ranged as well with more experienced programmers being clear about the value they had taken from different languages they had used over time.

My Experience with Languages

During my first job after college Frank Weber, an AFSC volunteer who served as a mentor of mine, suggested that I should learn a new language every year. As I work with developers who are earlier in their career than I am, I have been trying to decide how important Frank’s advice was: I’ve settled on extremely important.

Portait of Kahlil Gibran
This portrait of Kahlil Gibran I saw on display at the Telfair Academy. His writings influenced my thinking about language and poetry, but always in my native language not his.

As a kid I was taught and learned a small number of languages like BASIC, Logo, and HyperTalk. In college, I learned several more including 8 in the course of a 16 week semester – a rough but useful experience. After college, when Frank told me to learn a language a year, I did that for several years. My career has largely centered around languages I taught myself basically on my own post-college.

As a result, at one point or another, I have written programs in more than 20 different programming languages. I can learn new languages quickly, in part because it is rare for me to encounter a language that has constructs I haven’t used previously. Along the way I discovered I can even help people use a language I don’t know just based on the patterns I know from other languages.

On the other hand, I have tried to learn three human languages in addition to English: French, Russian, and Spanish – all went terribly. And my inconsistent attempts to learn some ASL have fallen in the middle: I had some success, but never put in enough effort to learn it well.

That dichotomy makes it tempting to emphasize the differences between these two types of language. In ways they are very different. Human languages evolve naturally over time; machine languages have rigidly defined rules. Human languages try to help us express the full range of our experience; machine languages are designed to get a machine to do complete tasks. 

But many concepts connect the two sets. Machine languages, like spoken languages, are tools created by people to express themselves. We label their structures as grammar in both, because we stole the formalization concepts from traditional languages when we invented programming languages. They both evolve over time. At this point they both borrow concepts from one another.

Growing Language Understanding

The thing that shifted for me recently was listening to a Hidden Brain episode Watch Your Mouth about human languages. The first section featured Lera Boroditsky talking about the impact spoken languages have on how people think. She cites examples of cultural understanding of ideas around direction and gender. There are similar ideas about our emotions that are also linked with the language we use to describe our emotions.

That had very direct correlations to programming. Hearing those discussions catalyzed the thinking I was already doing.

How I see and solve a problem in code can vary greatly with the language and platform I’m using. A Python application running on a server can use a radically different approach from a solution built in Apex on Salesforce. JavaScript running in a browser requires different solutions and details from solving that same problem in JavaScript running in a Node environment. Java largely frees you of worrying about memory handling, while many languages in the C-family force you to consider the storage implications of nearly every byte. At times when trying to work through a problem in one framework it can help think of how I’d solve it in another better suited to the task.

The second half of the Watch Your Mouth episode included a discussion of how languages change over time. It’s an interview with Linguist John McWhorter, and his work at looking at how language changes over time. He pushes back on the notion that we should all use the version of English someone tried to teach us in school: 

“It’s a matter of fashion, pure and simple. People do need to be taught what the socially acceptable forms are. But what we should teach is not that the good way is logical and the way that you’re comfortable doing it is illogical. It should just be, here is the natural way, then there’s some things that you’re supposed to do in public because that’s the way it is, whether it’s fair or not.“

John McWhorter

Programming languages have fashions and patterns – and they are rarely fair. A new language like Go will become popular in part because it’s new and used by a popular company. An older language will go out of fashion like C++ or Java, because newer languages feel more graceful even if there are technical trade-offs for that grace. They evolve, more formally than English but less formally than people pretend. Just because there is a standards committee does not mean that committee is purely selecting the “best” solution.

Technical Intersections Between Languages

XKCD with an Engineer thinking he can control the flow of languages

There is also a point where all these languages intersect: Unicode. Unicode tries to give us ways to express human languages in machine form – all written language. Every developer should understand how language is stored in files because our code is stored in that format, our servers talk to each other in that format, and humans will express themselves to our applications in that format. 

My Theory on Programming Language Acquisition

Like plumbers, electricians, doctors, nurses, and really anyone with a career: Developers need to master the tools and techniques of their trade. Programming languages are tools, and the techniques we learn from one make us better in another. We do not have to like the language designs we encounter, but we need to understand them. 

Every developer should teach themself a new language on a regular basis, either for work or for their own education. We should pick languages that are uncomfortable to us. Some should be new and popular, some should be old and foundational.

So if you’re a developer, go learn a new language. If you’re not a developer, go learn something different from what you use every day. It’ll force you out of your comfort zone and get you to learn more than you expect.

Languages I’ve written one or more programs in:

Just for fun, here’s the list of languages I can remember writing something substantive in:

  1. C
  2. C++
  3. C#
  4. Objective-C
  5. Python
  6. Ruby (with and without Rails)
  7. Java
  8. JavaScript
  9. PHP (versions 3-8, which only have passing resemblance to each other)
  10. ASP
  11. Logo
  12. BASIC (for Apple ][, and DEC Rainbow)
  13. VisualBasic
  14. VB.Net
  15. VBScript
  16. VBA
  17. HyperTalk
  18. AppleScipt
  19. Apex
  20. R
  21. Scheme
  22. Haskel
  23. Perl
  24. Prolog
  25. SQL, T-SQL, SOQL, and SOSL (sure arguably not languages which is why different flavors are listed as one)
  26. HTML/CSS (together they are mostly Turing complete, so arguably counts)
  27. XML/XSLT