Writing for Developers and Consultants: Editing

The most important thing you need to have to be a successful consultant is excellent communications skills . If your client doesn’t understand and trust what you tell them, it doesn’t matter what you suggest, create, build, or deliver.

I have been helping a few colleagues lately work on their client communication skills and figured I should write down my suggestions for how developers can communicate well with clients.

Everyone on a project team needs to be able to communicate what they need, what they are doing, and how their piece of the project works. They also need to know how to address their own errors and mistakes so the project moves forward and the client trusts that errors will be resolved.

As I sat down to write this piece I realized the topic is huge; far too large for one post. I’ve thought about the importance of good writing, editing, grammar, the kinds of email messages we send, documents we write, and presentations we give.

All of those probably justify at least one post, so I’m starting here: editing.

Why Editing is Important

Editing probably feels like an odd place to start a series on writing, but it is a thing you can start to do now that will help you write more effectively. If you have a college degree you probably got into the habit of writing papers fast, and turning them in at the deadline. But you almost certainly didn’t do enough editing of those papers (I’m married to a college professor, and I talk to college professors a lot: No. You. Didn’t.). If you don’t have a college degree, it’s also unlikely many people have sat down and talked to you about the importance of editing. If you’ve come to consulting from a writing-heavy background you’ve figured out by now the value of editing – this post isn’t really for you, so please come back for later installments.

Editing is how you take poor, or merely acceptable writing, and make it clear and compelling.

Good editing is about more than basic mechanics, it’s about testing that your idea or explanation makes sense. Editing lets you clean up awkward, but technically correct, expressions. And it gives you a chance to reflect on how your audience will read the material. As you edit, see if each sentence makes sense to them the same way it does to you.

Editors Are A Luxury

A good editor is worth their weight in gold – sometimes more. Well known authors have been known to change publishing houses to follow a good editor. They are also not part of your typical consulting team. For consultants, having an editor is a rare luxury.

For large documents, good teams will make sure one or two people give completed drafts an editing pass. Often getting support from a marketing department to edit a conference talk is possible. Another person reading an email and offering edits happens exceptionally rarely. It’s the same with internal presentations. No one will edit your chat messages except you.

So, while an outside editor is an invaluable tool, a consultant should not expect to have access to one.

Be Your Own Editor

That means you have to be your own editor. Editing your own writing is hard. If you could see the mistake the first time, you would have fixed it before you write the next sentence (those kinds of fixes are not the same as editing, that’s “fixing typos” or “not being lazy”).

There are three main tricks I use when being my own editor:

Put the document aside between writing and editing – the longer the better
Reading it aloud to yourself (yes really, out loud)
Edit on paper (use one of those printer things)

I wrote the first draft of this while sitting in the Atlanta airport. I revisited and edited a printed draft a few days later (my wife also took a pass – see above). And again after I put it into my blog. It’ll probably get a few tweaks after it’s posted.

Don’t Let Edits Be Personal

Writing, even technical writing, is very personal; editing should not be.

A good edit may be brutal to the original material. If you get too attached to your first draft, you will be tempted to skip asking for an edit or doing an editing pass yourself. That’s part of why I put a document aside for a bit between writing and editing, it gives me space to see the flaws in my words and ideas.

Several years ago I was editing a long report for AFSC, and the first draft was far too long, too detailed, had too many tangents, and didn’t focus on the intended audience. I attacked it. She and I spoke several times to make sure she understood my goal was to help make her was succeed. Through aggressive editing we made sure her great work achieved her intended goal: stopping the unchecked growth of private prisons.

I deleted entire sections, rewrote the summary (which someone else then edited for me). I pulled in favors, got my wife to edit drafts. We debated chapter order, technical details, and sentence after sentence. What started at 100 pages, ended up being 20 page report with an 8 page summary that journalists and politicians cited as they debated proposed private prison expansion. Because of that work, and follow on efforts by people using it as a model, private prison growth has slowed substantially (if that’s not a topic you know about, trust me that’s a good thing). The author and I became friends and worked together well on follow up projects and efforts. She appreciated the value of the edits even if they were painful at first.

Edit Aggressively

Whether your editing for yourself, or for someone else, dig deep and edit aggressively. If you haven’t done much editing you want to think about every word, every piece of punctuation, every detail. As you get more practice you’ll find short cuts and learn your own weaknesses (or your team’s). I use too many words, and write long sentences – I attack those. Shorter is better.

Using traditional grammar tools can be helpful, particularly if you are unsure about the basic grammar rules. But if you use tools like Grammarly or ChatGPT to rewrite things make sure you think carefully about if, and why, you like that version better. Used well Generative AIs are a useful tool, but if you lean on those tools too much you don’t actually get better yourself.

More to Come

As I said at the top, communications skills for developers and consultants is an enormous topic. I have not yet planned the series, but you should expect more to come (and I’ll edit this section to add references).

Salesforce Nonprofit and Education Scratch Orgs

During the recent Open Source Commons sprint in Chicago, I tried to create scratch orgs for nonprofit and education clouds. Despite having some of the best people in the market in the room, including two Salesforce Solution Engineers, two levels of their bosses, and of course Google, I couldn’t figure it out.

As a follow up to a conversation there, Larry Fontillas sent me links to the help docs that contain what I consider partial answers. While I’ve sent feedback to help improve those articles, I am posting my current solution to this challenge.

My Example Scratch Config

On Github I created a repo that contains two scratch org configuration files:

Each is my attempt to create a definition that works for the named cloud. If are new to scratch orgs, I suggest you start with the Trailhead Salesforce DX Quick Start. With a Devhub setup, and connected to your sf cli, you can easily create these scratch orgs from my settings with one of the two following commands:

sf org create scratch -d -f config/nonprofit-cloud.json -a npc-org

sf org create scratch -d -f config/education-cloud.json -a edu-org

Industry Org Scratch Config Breakdown

The two new clouds leve re-usable components Salesforce built, licenses, and deploys across different markets. Salesforce does not currently provide one master switch you must use. Instead you need to know what features to include and tell the Devhub which collection to enable.

That is done in two major parts of the configuration file. In my Nonprofit cloud config file there are several sections, but two are critical: features and settings → IndustriesSettings. The features list includes Salesforce components to enable. In this case I included the nonprofit specific Fundraising, Program Management, and Grantmaking modules, but also OminiStudio, Accounting Subledger, and more because they included with NP Cloud by default. Under Industries Settings you’ll also see I enable Grantmaking and support for Program Management.

The Education Cloud config file has even more detail. That’s because Salesforce as made more available for Education Cloud. The Industries settings section includes more flags as well for the same reason. As I Followed the setup guide for Education Cloud, I further adjusted some of the base-line object permissions.

Here are the features I know you need to include for each cloud:

Feature	NP Cloud	Edu Cloud	Notes
AccountingSubledgerGrowthEdition	Optional	Optional	Starter Edition is also available
AccountSubledgerUser	Optional	Optional
AnalyticsQueryService	Yes	Optional	This is listed in the docs under Fundraising, but I have not yet found a direct description.
Assessments	Yes	Yes
EducationCloud	No	Yes (requires a quantity parameter)	Main Education Cloud objects, but only a sliver of the features.
EnableSetPasswordInApi	Yes	Yes	Allows the cli to set the password, you always want this.
Fundraising	Yes	Optional
Grantmaking	Yes	Optional (Rare)
IndustriesActionPlan	No	Yes
IndustriesSalesExcellenceAddOn	Yes	Yes	This is listed in the docs under Fundraising, but I have not yet found a direct description.
IndustriesServiceExcellenceAddOn	Yes	Yes	This is listed in the docs under Fundraising, but I have not yet found a direct description.
LightningScheduler	No	Yes	Lightning Scheduler gives you tools to simplify appointment scheduling in Salesforce.
LightningServiceConsole	Optional	Yes	Allows the Lightning Service Console and access features that help manage cases faster.
MarketingUser	Yes	Optional	Provides access to the Campaigns object.
OmniStudioDesigner	Yes	Yes	Listed in lots of samples, but I have not yet found a direct description. But clearly needed for OmniStudio.
OmniStudioRuntime	Yes	Yes	More for OmniStudio.
OutcomeManagement	Yes	No
PersonAccounts	Yes	Yes	We all love Person Accounts now! Technically this is optional, although the assumption is it your default.
ProgramManagement	Yes	No	Enables the NPC Program and Case management features.
PublicSectorAccess	No	Yes	Not entirely sure about this one, but some of the features for Education Cloud seem to leverage these objects and settings.

Feedback Please!

I have tested these configurations to the degree of seeing that they work as a basic level. But I have not, yet, used them for a serious project. I am confident other people will find details that are missing, or just wrong. Please file an issue on Github or leave me a comment here with suggested changes.

Mid-Career Resumes

As we exit the Great Resignation, and move back to more traditional hiring patterns, application materials are increasingly important again. Over the course of my career I’ve been involved in a lot of hires, and read a large number of resumes. I know what I like to see, what I don’t like, and I have a bunch of friends in a similar position (although their likes and dislikes are sometimes different).

Recently, I realized that much of the advice online about resume writing is for people early in their career. That’s fair; they are the people with the least experience and need the most help. But as someone who is now mid-career, and reading resumes for other people who are also mid-career, I am noticing resumes from people who seem to still follow the early career advice.

So a few weeks ago I reached out to my friends who, like me, sometimes review mid-career resumes. While none of us are a full-time recruiter, we are the people who you need to impress if you want a job on our team. This post is a combination of my take, and the input I got from those people.

There are NO Hard Rules About Resumes

Resumes are not a regulated industry. There are no hard and fast rules. Any advice you see is just a set of suggestions. In the end, you have to decide what makes you look good and guess at what is effective.

Studies are rare, and even the best are poorly done. That is not the researchers’ fault. You cannot double blind a job hire. You cannot have 1,000 managers at different companies all hire for the same job from the same pool of applicants. Any one who knows a researcher is watching them work, likely changes their behaviors. Any study that finds bias creates legal risk for companies that participate which in turn limits participation and openness to data publication. List of problems with studying the process goes on and on.

Anyone who tells you there is one best way to create your resume, is wrong.
Anyone who is entirely focused on the hiring manager, risks failing to give advice to beat automated filters.
Anyone who is entirely focused on beating the automated filter, ignores that nearly ½ of the jobs in America are at small companies and unlikely to use such filters.

Write the best resume you can. Ask friends, particularly those who do hiring, for feedback. Consider paying a resume writer for help. But don’t expect even paid experts to be correct all the time.

Mid-Career Resumes Should Highlight Your Experience

The biggest mistake I see in resumes of people in mid-career, or even late career, is failing to highlight their experience. People who were at one employer for a long time struggle with this the most, but I’ve seen resumes for people with 15 years of experience that read like a recent graduate.

Your experience should be front and center. Everything about your resume should say “this is an experienced person.”

I like some form of summary at the top. Tell me what kind of employee and colleague you are. Not an objective section, but a summary of who you are. It can come in many forms:

a short paragraph:

Salesforce MVP, developer, administrator, and consultant with 20 years of experience in the nonprofit and higher education sectors. Seven Salesforce certifications, experience in more then 20 programming languages. Proven experience leading teams and working closely with non-technical clients.

list of titles, or key phrases

Salesforce MVP, Technical Architect, Nonprofit Fundraising Expert

After that, your job experience and skills are next. How exactly you do this can vary. Some people like skills in a sidebar. Some people put a list at the top. Some people put that list after their job experience. Frankly, as a reader, I don’t care. But I want to be able to find your list of skills and your relevant job history fast.

Your currently valid certifications should be included near your skills. But only those the reviewer will find relevant.

Think About Your Audience

Likely the person reading the resume of an experienced person is an experienced person. We have habits, routines, and work styles that are built on experience. We also have things like aging eyes, old printers, out of date external monitors, and other things that it are tempting to ignore.

Text should be high contrast, print well in black and white (there is a huge exception here for graphic designers, who benefit from showing off graphic design skills), and be generally easy to read. I don’t want your pretty three color graph, head shot, or blue text that prints light gray.

If I am reading a handful or resumes, I’ll do that on a screen and I can zoom in if I need. But if I’m digging through a big pile, I’ll print them. I will print them on my 20+ year old laser jet, blank and white, printer. When I last worked in an office and reviewed resumes, I used the office’s even older laser jet black and white printer. Your shaded background might make the whole thing unreadable on those devices. Besides, you should have too much experience to waste space on a picture (and that’s before we talk about companies trying to avoid identity based biasing who might not want reviewers to know what you look like too early in the process).

I strongly recommend going for simple, clean, classic, design approaches.

Mid-Career Resumes Should be More Than One Page.

I haven’t used a one page resume in more than 20 years. I don’t know who is still saying one page is the magic number. A new graduate might benefit from the one-pager, but if you have 10-30 years work experience, and you only need one page to tell me, it better be the most amazing page of text you’ve ever created. When I see a one-page resume, before I see the words I see a person with limited experience.

Personally, I like the two pager. Two very full pages. I want to see that you were forced to edit and format aggressively to make it fit on two pages. You want me to think you have 5 pages of content, but you compressed it effectively.

Two pages gives you plenty of room to show off, without wasting my time. It shows me you can edit and filter content. Ideally, it’ll leave me wanting more information, that gives me questions I can ask in your interview.

Some people like longer. When I spoke with friends who hire, most people liked two pages. But some were open to 3-4. Beyond four you are into academic CV land, which is a different thing entirely.

Connect the Dots

You have experience, you are showing it off well, good. But are you showing off the right experience? One of the most consistent pieces of feedback I got from friends who do hiring is that we want to know you know who we are as an employer.

No every detail, but tell us what your public persona is. Is there a values statement in the job ad? Reflect some of that language back in a cover letter. Do we work in a specific market? Make sure to include some experience that connects you to that market.

When I worked at a nonprofit, we wanted people excited by the work we did. Which means they needed to find ways to tell us in their resume, cover letter, application, and interview they knew something about that work. Since becoming a consultant, I’ve been consistently amazed that people will send resumes and come to interviews that don’t know what kind of customers we have.

Write a Cover Letter whenever Invited

This applies not just to mid-career applicants, but everyone else too. Not all jobs accept a cover letter, but when given the chance to say more: say more. The numbers I can find on resume review suggest an average of 6-7 seconds. I think that’s low in practice (see comments on studies), I know when I dig through a large stack I find ways to filter out some very fast, and others get more careful review. So an average will likely be far from my median or modal times. Even so, a resume that isn’t tossed out because it’s an applicant who is wildly unqualified, will get 15-30 seconds in my first pass. You add a cover letter, now I’m spending more time reading. You could double, or even triple, the time you get in the first review 45-90 seconds – that’s huge.

It also means you can connect some additional dots for me. If your resume includes experience that you consider related, but that might not be obvious, you have a couple sentences now to tell me that story. Are you career pivoting? Tell me what about your old career makes you better than your experience suggests. Do you volunteer in your community? Tell me what about that helps you understand our work, or support our company values.

In Mid-Career Resumes the Basics Still Matter

Details matter: fix your typos, use consistent formatting, etc. I saw a resume recently with a red-line through their summary line. That’s a bad first impression.

Write resumes you want to read: If you have read resumes as part of your job, think about the ones that impressed you and mimic those.

Get feedback from a friend: You probably have friends and professional contacts who will give you blunt feedback. Ask for it. I did as part of writing this post.

Consider hiring an expert: There are people who do this for a living. Some of them are really good. When you ask your friends for feedback, ask them for references to services they used.

Not everything is needed: Edit down your experience. Keep the stuff that says you’re awesome, cut stuff that’s not relevant to the hiring manager.

References for More Thoughts on Mid-Career Resumes:

The internet is full of advice on resume writing. Most for beginners, but some for people with more experience. Here are a few things I found useful:

The Queries Part 3 of 3

This is the third and final post in a series of posts to break down the questions from my Queries on Queries talk. The full talk is available here.

Is your solution reusable?

Migrations feel like one off processes, but teams that migrate once usually migrate again.

Have you ensured that as much of your solution as possible can be reused? Do you have a shared library of migration tools that your whole team can access? When you create new functionality are you thinking about ways to make it usable in your next project?

On any technology project you will generally benefit from designing for re-usability. I mentioned in my comments on the question about repeatability that people get tempted to see migration work as fundamentally one-off, but you need to plan for many runs. That question is focused on repeating the same project, this is about recycling parts of this project in another.

To a consultant, the value of reuse should be obvious: we like to sell projects to new clients based on successful projects for another client. For that I want libraries of tools the developer designed for rapidly assembled to meet a new client’s needs.

But even when I was the client, I was moving similar data into the same systems over and over. I created API libraries, and rough interfaces, to handle some of that work so I didn’t have to do the same tedious work again and again.

In both cases those libraries are only useful if whoever needs them knows they exist, has access to them, and can figure out how to leverage them.

Is your migration testable?

All good processes are rigorously tested.

Do you have an automated testing solution that validates your process? Can you tell if the data migrated accurately after each test run? Do your tests cover the positive and negative cases?

Testing migrations is hard. Testing software is hard. The testing tools that developers are most familiar with are unit testing tools, test one very small thing at a time. Multi-system data comparison is not their forté. The tools that do exist for such work tend to be quite expensive and/or so complex the task of creating tests is nearly as hard as the task of creating the migration jobs themselves.

But just because testing is hard does not mean you shouldn’t do what you can do within the budget and time you have. When you cannot use something like MuleSoft’s MUnit you can still create queries that sanity check the migrated and generated data. You select records for spot checking that cover edge cases you are aware of, and some that represent primary use cases. You can look for records that create invalid data states that would violate your new validation rules.

Is your work fixable?

Migrated data often needs to be updated after the jobs have all run.

Do you have a plan to fix your data if errors are found post migration? Does your plan include ensuring you have external Ids, or other connections, to be able to update all records of every type? Have you validated this plan will work in practice?

When you do a data migration, because everything is determinant, you feel like perfection is possible. But when you’re moving millions of records that were entered by humans, extracted by humans, mapped by humans, validated by humans, and represent human behaviors, there is a lot of room for human error.

You can either pretend your process is good enough to squeeze out the error, or build a process that allows you to fix the errors that slip through. Obviously I don’t believe the first is possible, so I encourage the second.

Make sure you can go back and update anything. If you’re migrating into a database that allows for a lot of easy changes – great. If you’re migrating into a financial system – make sure you understand the rules for editing.

Planning for mistakes you don’t want to have makes it far easier to recover from those mistakes when they appear.

The Queries Part 2 of 3

This is the second in a series of posts to break down the questions from my Queries on Queries talk. The full talk is available here.

Is your work repeatable?

You will need to do this more than once.

Is your process designed so you can run it over and over without error? Can you easily erase test attempts and start over from a clean slate? Do you have the capacity to do all the practice runs you need to complete your project successfully and on schedule?

Because a migration is fundamentally a one-way operation, designed to move data once, it’s tempting to build the whole process as a one-off affair. I’ve seen (even used) migration processes that required hours or days of hand polishing data to get it to load – this is a terribly way to do the job.

A good migration process should be automated. To automate anything you need to test it. If you test something you should expect it to fail many times before it works. And when it fails you need to run it again and again until it works.

By their very nature data migrations create data – in a target system no less – and so you need a way to roll back your changes to migration to a pre-run state for each subsequent test. I like to use a staging database for the main complex parts of my migrations. I created Salesforce2Sql just to make that so easy no one would be tempted to skip that step. When I create processes in an ETL, I like to have jobs start by deleting data from the staging database related to the job, so I can make Idempotent jobs as much as possible. Run, test, adjust, repeat. If you know how many times you ran your migration process, you didn’t run the jobs enough.

Is your work measurable?

To know you moved all the data, you must know how much data is going in and how much should come out.

Can you accurately predict your output data volume based on the input size? Do you have valid estimates of the running time required for each stage based on the data volumes? Are the estimates of expected data set size from a reliable source?

It seems like knowing how to measure your work should be obvious, but in truth most interesting migrations are not a simple record-in, record-out – they involve splitting records, combining tables, filtering data, converting tables to fields, fields to tables, and other similar adjustments. But the only way to know if you got it all to work out right is to work out the math wherever you can.

It’s also important to know how long a process will take. Sometimes a few thousand records here or there doesn’t matter much, but sometimes that is a matter of hours. Particularly when running samples it’s important to know the average running time. I’m working on a project right now where we know that the first 3 million records will load in about 6 hours, the last 45,000 records will take 12 hours.

In that project we’ve worked out those running times, and we have a good understanding of total records counts. In other projects we thought we knew, only to discover the person giving us the source record counts was talking about the per-year instead of total expected migration size. But with per-record estimates we can adjust expectations quickly when information changes.

Do you scope your data migrations carefully?

Limiting bad data in your system allows for better decisions in the future.

Do you only load data into the new system that you truly need? Can you easily spot the difference between new and old records? Are there data points getting loaded that have no use case or maintenance plan in the target system?

Everyone wants to keep all their data. My entire career I have understood that storage is cheap, and big data is king. AI driven data analytics have been around for a few years, and now we have all the attention on generative AIs, both benefit from large data sets.

These all tools are great, but they aren’t magic.

Big data processing, whether it be AI driven or not, is all about correlations. If you give a correlation engine bad data, it will give you bad results. Garbage in is still garbage out.

You only want to migrate data that’s good.
You only want to migrate data that’s useful.
You only want to migrate data that you will maintain.

So before you start a migration make sure you know your data will fall into those categories. Organizations can always archive data they don’t migrate.

There are other reasons more data isn’t always better.

If your system, or data archive, is ever breached that presents a risk to an organization. Privacy laws are steadily tightening, increasing the chances you will have to admit to your audience you were the cause of their information falling into the hands of bad actors.

Also, old data is often bad data. Colleges often have the email address used by their applicants squirreled away in their alumni systems. How useful do you think the AOL address I used in 1997 is to Hamilton College today? If they use it, they will fail to reach me. It provides them no value, but does provide them the chance to make mistakes. Same is true of old phone numbers, addresses, and more.

Keep the good stuff, let go of the stuff you don’t need.

The Queries Part 1 of 3

This is the first in a series of posts to break down the questions from my Queries on Queries talk. The full talk is available here.

Are your tools good enough?

Our migrations live and die by our tools.

Are your tools built for the scale of your project? Do they empower you to do your best work or impede rapid progress? Would a new tool serve you better now or in the future?

Having the right tools is critical to any job. In data migration we primarily talk about ETLs (Extract, Transform, and Load): tools like Jitterbit, Informatica, Mulesoft, Talend, etc. We also use additional tools to help support the process: a task tracker like Jira, a Data Modeler like Lucidchart, staging database prep like Salesforce2Sql, and more.

It’s easy to say that it’s a poor carpenter who blames his tools, but anyone who has spent time with actual carpenters knows they care a great deal about what tools they use. They might be able to make due with poor tools, but they will do their best work with the right tools for the job.

Each tool you use needs to meet your team’s needs. It should play to your strengths, supports the kinds of projects do you do, and has an eye to the future. A tool that works great for a team of declarative Salesforce consultants might drive developers crazy. A tool that works great for 10’s of thousands of records might struggle with millions; a tool scaled for 10s of millions of records may be overly complex for a project of 30,000.

Make sure you’re using the tools that let you do your best work, now and in the future.

Do you make the data atomic for processing?

Smaller pieces of data are easier to track, manipulate, and test.

Do you divide the source data into its constituent parts? Can you process individual pieces of data easily and cleanly? Can you stop your process after each stage to validate the results?

It can be tempting to process data as it comes: handling whole rows of data in the form they were provided and treating fields as a single data point. In practice exports may have extra rows or columns to deal with related records. Organizations may have encoded multiple points of data into fields like ticket names including a show name, date, and time into the name field. Fields can also contained semi-structured data, like Joomla’s use of arbitrary JSON blobs.

To process this data it is often easier and clearer to extract it from these structures prior to direct processing. It’s not always needed, and rarely required, but doing this clean up of structure – like creating interstitial database tables or predictable data objects – can greatly ease the rest of your job.

Like many problems in software engineering, it’s easier to do good work when you are operating on atomic pieces. Think about the right ways to pull your data into constituent parts when they aren’t there already.

Can you process samples of your data set?

When you have lots of data you need to test small parts to be sure your process works.

Do you know how to create and run small segments of your total input? Are your segments made up of complete and valid samples? Does your sample include all the errors and edge cases your data set will throw at your process?

If you are working with small data sets your sample can be all the data. But when you have a large data set you need to test your process with samples. When you have a multi-step migration you likely need to test the second phase while the first phase is still under construction – again a sample data set is critical.

Having valid test data, that covers all your edge cases, is critical to making sure you have a working solution.

A few years ago I worked on a project that involved a two stage migration for a membership organization with some 600,000 active contacts. Every one of them needed to be migrated into Salesforce and then into Drupal. To test the Drupal migration we needed samples of all the types of membership statuses we would see, which involved hand creating several hundred records. At the next Salesforce Commons Sprint I raised the idea of needing a better tool for this kind of work, that question eventually helped lead to Paul Prescod‘s creation of Snowfakery. Snowfakery will build you testing data sets of any size and complexity to make sure your processes will succeed.

Queries on Queries: Improve your data migration

Last week I gave my Queries on Queries talk, intended to help you improve your data migration process, as a webinar for Attain Partners. It’s a revised and improved version from the last time I gave it.

These questions aren’t like the old Joel Test (which is still useful) where the right answer is “yes”. These questions are designed to point you in a direction but allow you to change your answer over time. I generally answer this questions with a paragraph not a word. Use these questions as a challenge to make you and your team better.

Over the next few weeks I’m planning to publish a series that will include each query and why I think it’s useful in helping you think about how to improve your process.

Take Good Notes

A good set of notes is how we build a memory of what happened.

Good note taking is important in nearly any white collar job, particularly consulting. If we have a long conversation with a client and have to re-ask them about all the details, they will rightly be annoyed. They may demand to know what they paid for the first time we talked.

Why we take notes

In school we are taught to take notes. Teachers expect students to remember information to pass tests, write papers, and other evaluations of learning. Too often teachers will try to convince students to take notes in specific way. They may make note taking into an assignment and assessment of its own. My wife sees college students who decide that they don’t need to take notes because she does not grade them. These students do not do well. These students missed the point of taking notes. The form of the notes is not important, but the existence of them is.

When we leave school notes serve two main purposes:

Record events of a meeting so there is a shared record later.
Help us remember what happened so we can do our work.

Every important meeting should have someone charged with creating the first of those. How you do that task assignment is work place and team specific, but it needs to happen. Doing this well is an important skill, and every team, board, religious community, and so on needs people who do this well. But the second type of notes are often more important for day to day work; they are an important how the participants remember what needs to happen.

This second category of notes is why there is nearly always a notepad near me when I’m working. I scribble down thoughts, tasks, key points, and anything else I need to remember later. The notes I take are messy, disorganized, and useless to anyone but myself. None of that matters as long as I remember what I need to know.

Why take notes yourself

Lots of people hate to take their own notes. I have colleagues who treat note taking as a task to be avoided. Heck, I dislike being the official note taker when it’s my turn. People often fall back on the “official” notes of meetings instead of keeping their own. I have heard people go so far as to declare additional notes are just a waste of effort. I have had colleagues claim this “wasted” effort is somehow costing the client money (not true, they were in the meeting anyway).

Writing notes encourages us to engage with the content. There are no shortage of studies on the impact of note taking and memory formation. The research clear indicates that if you engage actively with information you will retain it better. Any form of note taking that encourages you to engage is a good start. That engagement can be exhausting, but that does not justify avoiding the work.

Even when in a meeting with an official note taker, our clients are best served by everyone on our team taking notes. That helps us all learn about the client needs, to contribute to the project work, as well as offer edits to official notes after a meeting wraps up.

Can an AI note taker do just as well?

The recent public emergence of generative AI has captured a great deal of attention. We are thinking of all the places that a machine can take over tasks we thought required a human – particularly those we dislike. There are already services like Otter.AI which will attend virtual meetings and generate notes for you.

My experience suggests that, right now, they are pretty terrible at their main job. The automated transcripts they require are adequate at best; their note taking ability is worse. The samples I’ve seen from meetings I was in were basically useless. Worse yet, AI tools will lie (or more accurately they generate believable, but false, information), which is getting people into trouble. After all those issues, you will need to deal with the privacy and security implications of allowing a system listen into your meetings.

One day these systems will likely be pretty good for official meeting notes, but that’s not today. Even at that point, those AI’s will not help you engage with, or retain, the information.

Do yourself a favor, no matter who or what else is taking notes, take your own.

What makes good personal notes

Fundamentally what makes good personal notes is whatever you can use to accurately recall what happened. If you are able to recall the details when you need them, your notes were good enough. If you cannot not, your notes aren’t good enough. That’s true during your education (unless a teacher is grading your notes, then play along with their instructions), that’s true in the work place.

There are several formal patterns for note taking to help you structure the information. If you are struggling to take useful notes, I recommend trying one or two to see if they work for you. Those patterns do not work well for me, and I have bad memories of being made to outline lectures, and other patterns as the “one true” solution for taking notes.

Later in my education I picked up the metric I use now: do they work. I found I retain information best when I am summarizing bits and pieces to trigger my memory. My own notes are often just a few words to draw my brain back to key points. I will write out a major decision, pronouncement, or useful quote from time to time – that extra detail emphasizes to me later that I thought was a major point at the time.

Find your own style of note taking, but do not pretend you do not need them.

Salesforce Data Migration Lessons

Last week I was part of a webinar for Attain Partners talking about Salesforce data migrations. One of the questions the moderator, Eric Magnuson, asked was the three lessons I’d learned doing data migration work.

The answers I chose weren’t so much from my first data migration projects, as from more recent projects. Those early migrations were tiny by my current standards. Back then, I was mainly the consumer of migrated data. When I did migrations they were small and manual. The intellectual process was similar but the scale meant I had lots left to learn about large data projects. The lessons I chose are focused on what I wish I knew when doing the migration for clients with vastly larger data sets. When I became the producer of migrated data.

My three lessons:

Manage expectations from the start.
Don’t be a hero.
Understand your tools, and use them well.

Manage Expectations from the Start

This still surprises me, but it’s true: one of the biggest reasons data migrations get into trouble is bad expectations.

People tend to think data migrations are easy. You create a map from the old system to the new, run some processes to convert the data, load into Salesforce. But in practice, that map might involved thousands of details. Those conversions are hard to get right for every variation of the old data. And loading large amounts of data is never as easy as it looks.

When people think something is easy, and then the outcome is less then perfect, they get mad.

The problem is, data is always messy. We do migrations when we replace systems. We replace systems when the old one has problems. Problems in a system, lead to data errors.

That reality is made worse by the reality of big system switches. The freshly migrated data lands in a system the primary users are still learning. Those factors lead to confusion, mistakes, errors, and misunderstandings.

From the very first conversation I tell our clients to expect migration errors. Our first mock run of the migration is messy – sometimes very messy. The whole point is to find errors. The second mock run will be much better, but still imperfect; we’ll find more errors. No matter how many times we do test runs, there is only some much we can do with imperfect inputs.

We can make your data better then it is, but we can’t make it perfect. If you expect perfect data, you will be disappointed. I want you to be thrilled by your new system, and that means you need to understand your migrated data will have flaws.

Don’t Be A Data Hero

When I first did data migrations I was often a one-person show. I was responsible for figuring out all the details, implementing the process, reporting to the client, and fixing all the flaws. It’s a common story for people doing migrations.

We find ourselves up late at night, working through piles of detail, trying to make sure the client is satisfied. It encourages a hero mentality: I’ll make the project successful through shear will.

And most of us can do it. Being a data hero isn’t that hard if you put the hours in. It is, however, miserable.

People doing migrations need, and deserve, support. Now that I’m leading a team of people doing migrations I have added the support I should have had. I created a space for us to come together and talk about our projects.

We ask each other for help and suggestions. We offer ideas for how to improve our processes. We talk about ways to address client concerns. And yes, we complain to one another. But mostly what we do is make sure that no one is alone. Everyone, myself included, has support and back up.

A team is stronger than a person. We don’t need to be heroes. We do better work, and are happier people, when we support one another. Good work, by happy people, makes for successful projects.

Understand Your Migration Tools

Data processing tools are code generators – understanding that allows you to use them well.

Both parts of that make sense once you say it:

Tools that allow you to design an arbitrary process that takes inputs and generates outputs are obviously writing code at some level.
If you understand any tool better, you will use it better. That’s true of a hammer, a screw driver, or a piece of software.

I learned how to migrate data from people who weren’t formally trained developers. They were using the tools the best way they knew how, but didn’t have the background to apply software development best practices.

When I combined the tool usage they taught me, with the software engineering practices I already knew, I created vastly superior solutions. Our team now creates processes that are easier to setup, run faster, and allow us to fix all errors (even if we missed them until after launch).

Apply Lessons our Salesforce Data Migrations

I apply these lessons to the Salesforce data migrations I lead in my work at Attain Partners. I combine that with my queries on queries review process, and am constantly building better solutions for our clients, our team, and myself.

Do Not Consult Like Col. Flagg

Sometimes we learn best from counter examples. Col. Flagg was an recurring character on M*A*S*H. He was a CID man with Army Intelligence, and spent much of his time bloviating about his suspicions about everyone while threatening to kill people for fun.

Flagg seemed to have taken every piece of tactical advice he was ever given to absolute extreme. It leads him apply his most aggressive techniques – like threats of torture, or attempts at bribery – to every situation he encounters.

Consultants get a lot of good tactical advice about how to address our clients and our work. But if we take things too far we are only slightly less clueless than Flagg.

Col. Flagg’s Over-Generalization

Probably the best example of Flagg’s constant misapplication of techniques by generalizing their use is in The Abduction of Margret Houlihan from season 5. Chief Nurse Margret Houlihan hurries off late at night to deliver a baby in a nearby village, telling only the guard, Kilnger, who promptly goes to bed. When no one can find her in the morning, Potter calls for help searching for her. The army’s answer is to send Col. Flagg. He arrives, dressed as Mussolini, claiming to look like a Chinese double agent (if that agent chose the same get up), and proceeds to request the provisioning of several items clearly planning to torture someone.

Flagg spends the day wondering around, threatening everyone he talks with, trying to get information about Houlihan. He never finds anything of use, but does get in a number of good one-liners.

Finally Houlihan, who was never in any danger, returns to camp unaided. Flagg declares victory, throws himself out a window – so know one will see him leave – and breaks his leg.

Anyone who had been a consultant long enough can find parallels between Flagg’s useless bloviating, and consultants who try to address every client project as interchangeable.

Col. Flagg Consulting

Good advice often is context specific – you need to know when and how to use it.

Anytime I am in a meeting with a client, and I recognize that a piece of stock advice may apply, I try to reflect on the context of the moment. Does the advice actually apply, or would a different approach be better.

For example, consultants are often told to never say “I don’t know” in front of a client when asked a question we are unsure how to answer. Instead we are taught to say “I need to get back to you about that” or “let me verify with another team member before I respond.”

Up to a point this makes sense. We are paid experts; clients don’t want hire someone who does not know how to complete the job. Often making a space to make sure you are correct is the right response to an unknown. It’s far better than making up wrong answers. And, more often, better than making a client nervous because you haven’t had 30 seconds to Google the answer.

But clients can also smell something is fishy if you always claim to know everything. When I was a client, I never trusted consultants who pretended that they had seen and done everything. I knew they were lying to me because no one knows everything – leaving me to sort truth from fiction. That never ended well for the consultant.

“Nobody gets the truth out of me. I keep myself in a constant state of confusion.”
Col. Flagg

Consultants who cannot track context and adjust their approach accordingly, risk consulting like Col. Flagg.

MAS*H Consulting

If you didn’t know, or can’t guess from context, my wife and I are M*A*S*H fans. We grew up with it. We bought the DVDs as they Fox released them. It’s our go-to for something comforting to watch. It’s entertaining, and full of great lines and moments to borrow for life examples. As part of marking the show’s 50th anniversary this is the second in a short series of posts using consulting lessons from characters in M*A*S*H.