Salesforce Queries and Proxies in Drupal 8

The Drupal 8 version of the Salesforce Suite provides a powerful combination of features that are ready to use and mechanisms for adding custom add-ons you may need.  What it does not yet have is lots of good public documentation to explain all those features.

A recent support issue in the Salesforce issue queue asked for example code for writing queries. While I’ll address some of that here, there is ongoing work to replace the query interface to be more like Drupal core’s.  Hopefully once that’s complete I’ll get a chance to revise this article, but be warned some of those details may be a little out of date depending on when you read this post.

To run a simple query for all closed Opportunities related to an Account that closed after a specific date you can do something like the following:

      $query = new SelectQuery('Opportunity');
      $query->fields = [
        'Id',
        'Name',
        'Description',
        'CloseDate',
        'Amount',
        'StageName',
      ];
      $query->addCondition('AccountId', $desiredAccountId, '=');
      $query->conditions[] = [
        "(StageName", '=', "'Closed Won'",
        'OR', 'StageName', '=', "'Closed Lost')",
      ];
      $query->conditions[] = ['CloseDate', '>=', $someSelectedDate];
      $sfResponse = \Drupal::service('salesforce.client')->query($query);

The class would need to include a use statement for to get Drupal\salesforce\SelectQuery; And ideally you would embed this in a service that would allow you to inject the Salesforce Client service more correctly, but hopefully you get the idea.

The main oddity in the code above is the handling of query conditions (which is part of what lead to the forthcoming interface changes). You can use the addCondition() method and provide a field name, value, and comparison as lie 10 does. Or you can add an array of terms directly to the conditions array that will be imploded together. Each element of the conditions array will be ANDed together, so OR conditions need to be inserted the way lines 11-14 handle it.

Running a query in the abstract is pretty straight forward, the main question really is what are you going to do with the data that comes from the query. The suite’s main mapping features provide most of what you need for just pulling down data to store in entities, and you should use the entity mapping features until you have a really good reason not to, so the need for direct querying is somewhat limited.

But there are use cases that make sense to run queries directly. Largely these are around pulling down data that needs to be updated in near-real time (so perhaps that list of opportunities would be ones related to my user that were closed in the last week instead of some random account).

I’ve written about using Drupal 8 to proxy remote APIs before. If you look at the sample code you’ll see the comment that says: // Do some useful stuff to build an array of data.  Now is your chance to do something useful:

<?php
 
namespace Drupal\example\Controller;
 
use Symfony\Component\HttpFoundation\Request;
use Drupal\Core\Controller\ControllerBase;
use Drupal\Core\Cache\CacheableJsonResponse;
use Drupal\Core\Cache\CacheableMetadata;
 
class ExampleController extends ControllerBase {
    public function getJson(Request $request) {
        // Securely load the AccountId you want, and set date range.
 
        $data = [];
        $query = new SelectQuery('Opportunity');
        $query->fields = [
            'Id',
            'Name',
            'Description',
            'CloseDate',
            'Amount',
            'StageName',
        ];
        $query->addCondition('AccountId', $desiredAccountId, '=');
        $query->conditions[] = [
            "(StageName", '=', "'Closed Won'",
            'OR', 'StageName', '=', "'Closed Lost')",
        ];
        $query->conditions[] = ['CloseDate', '>=', $someSelectedDate];
        $sfResponse = \Drupal::service('salesforce.client')->query($query);
 
    if (!empty($sfResponse)) {
        $data['opp_count'] = $sfResponse->size();
        $data['opps'] = [];
 
        if ($data['opp_count']) {
            foreach ($sfResponse->records() as $opp) {
                $data['opps'][] = $opp->fields();
            }
        }
    }
    else {
      $data['opp_count'] = 0;
    }
    // Add Cache settings for Max-age and URL context.
    // You can use any of Drupal's contexts, tags, and time.
    $data['#cache'] = [
        'max-age' => 600, 
        'contexts' => [
            'url',
            'user',     
        ],
    ];
    $response = new CacheableJsonResponse($data);
    $response->addCacheableDependency(CacheableMetadata::createFromRenderArray($data));
    return $response;
  }
}

Cautions and Considerations

I left out a couple details above on purpose. Most notable I am not showing ways to get the needed SFID for filtering because you need to apply a little security checking on your route/controller/service. And those checks are probably specific to your project. If you are not careful you could let anonymous users just explore your whole database. It is an easy mistake to make if you do something like use a Salesforce ID as a URL parameter of some kind. You will want to make sure you know who is running queries and that they are allowed to see the data you are about to present. This is on you as the developer, not on Drupal or Salesforce, and I’m not risking giving you a bad example to follow.

Another detail to note is that I used the cache response for a reason.  Without caching every request would go through to Salesforce. This is both slower than getting cached results (their REST API is not super fast and you are proxying through Drupal along the way), and leaves you open to a simple DOS where someone makes a bunch of calls and sucks up all your API requests for the day. Think carefully before limiting or removing those cache options (and make sure your cache actually works in production).  Setting a context of both URL and User can help ensure the right people see the right data at the right time.

Drupal Salesforce Suite Custom Field Mapping Types

The Drupal 8 Salesforce Suite allows you to map Drupal entities to Salesforce objects using a 1-to-1 mapping. To do this it provides a series of field mapping types that allow you to select how you want to relate the data between the two systems. Each field type provides handling to help ensure the data is handled correctly on each side of the system.

As of this writing the suite provides six usable field mapping types:

  • Properties — The most common type to handle mapping data fields.
  • Record Type — A special handler to support Salesforce record type settings when needed.
  • Related IDs — Handles translating SFIDs to Drupal Entity IDs when two objects are related in both systems.
  • Related Properties — For handling properties across a relationship (when possible).
  • Constant — A constant value on the Drupal side that can be pushed to Salesforce.
  • Token — A value set via Drupal Token.

There is a seventh called Broken to handle mappings that have changed and need a fallback until its fixed. The salesforce_examples module also includes a very simple example called Hardcoded the shows how to create a mapping with a fixed value (similar to, but less powerful than, Constant field).

These six handle the vast majority of use cases but not all.  Fortunately the suite was designed using Drupal 8 annotated plugins , so you can add your own as needed. There is an example in the suite’s example module, and you can review the code of the ones that are included, but I think some people would find an overview helpful.

As an example I’m using the plugin I created to add support for related entities to the webform submodule of the suite (I’m referencing the patch in #10 cause that’s current as of this writing, but you should actually use whatever version is most recent or been accepted).

Like all good annotated plugins to tell Drupal about it all we have to do is create the file in the right place. In this case that is: [my_module_root]/src/Plugins/SalesforceMappingField/[ClassName] or more specifically: salesforce_webform/src/Plugin/SalesforceMappingField/WebformEntityElements.php

At the top of the file we need to define the namespace, add some use statements.

<?php
 
namespace Drupal\salesforce_webform\Plugin\SalesforceMappingField;
 
use Drupal\Core\Entity\EntityInterface;
use Drupal\Core\Form\FormStateInterface;
use Drupal\salesforce_mapping\Entity\SalesforceMappingInterface;
use Drupal\salesforce_mapping\SalesforceMappingFieldPluginBase;
use Drupal\salesforce_mapping\MappingConstants;

Next we need to provide the required annotation for the plugin manager to use. In this case it just provides the plugin’s ID, which needs to be unique across all plugins of this type, and a translated label.

/**
 * Adapter for Webform elements.
 *
 * @Plugin(
 *   id = "WebformEntityElements",
 *   label = @Translation("Webform entity elements")
 * )
 */

Now we define the class itself which must extend SalesforceMappingFieldPluginBase.

class WebformEntityElements extends SalesforceMappingFieldPluginBase {

With those things in place we can start the real work.  The mapping field plugins are made up of a few parts: 

  • The configuration form elements which display on the mapping settings edit form.
  • A value function to provide the actual outbound value from the field.
  • Nice details to limit when the mapping should be used, and support dependency management.

The buildConfigurationForm function returns an array of form elements. The base class provides some basic pieces of that array that you should plan to use and modify. So first we call the function on that parent class, and then make our changes:

 /**
   * {@inheritdoc}
   */
  public function buildConfigurationForm(array $form, FormStateInterface $form_state) {
    $pluginForm = parent::buildConfigurationForm($form, $form_state);
 
    $options = $this->getConfigurationOptions($form['#entity']);
 
    if (empty($options)) {
      $pluginForm['drupal_field_value'] += [
        '#markup' => t('No available webform entity reference elements.'),
      ];
    }
    else {
      $pluginForm['drupal_field_value'] += [
        '#type' => 'select',
        '#options' => $options,
        '#empty_option' => $this->t('- Select -'),
        '#default_value' => $this->config('drupal_field_value'),
        '#description' => $this->t('Select a webform entity reference element.'),
      ];
    }
    // Just allowed to push.
    $pluginForm['direction']['#options'] = [
      MappingConstants::SALESFORCE_MAPPING_DIRECTION_DRUPAL_SF => $pluginForm['direction']['#options'][MappingConstants::SALESFORCE_MAPPING_DIRECTION_DRUPAL_SF],
    ];
    $pluginForm['direction']['#default_value'] =
      MappingConstants::SALESFORCE_MAPPING_DIRECTION_DRUPAL_SF;
    return $pluginForm;
 
  }

In this case we are using a helper function to get us a list of entity reference fields on this plugin (details are in the patch and unimportant to this discussion). We then make those fields the list of Drupal fields for the settings form. The array we got from the parent class already provides a list of Salesforce fields in $pluginForm[‘salesforce_field’] so we don’t have to worry about that part.  Since the salesforce_webform module is push-only on its mappings, this plugin was designed to be push only as well, and so limits to direction options to be push only. The default set of options is:    

'#options' => [
    MappingConstants::SALESFORCE_MAPPING_DIRECTION_DRUPAL_SF => t('Drupal to SF'),
    MappingConstants::SALESFORCE_MAPPING_DIRECTION_SF_DRUPAL => t('SF to Drupal'),
    MappingConstants::SALESFORCE_MAPPING_DIRECTION_SYNC => t('Sync'),
 ],

And you can limit those anyway that makes sense for your plugin.

With the form array completed, we now move on to the value function. This is generally the most interesting part of the plugin since it does the work of actually setting the value returned by the mapping.

  /**
   * {@inheritdoc}
   */
  public function value(EntityInterface $entity, SalesforceMappingInterface $mapping) {
    $element_parts = explode('__', $this->config('drupal_field_value'));
    $main_element_name = reset($element_parts);
    $webform = $this->entityTypeManager->getStorage('webform')->load($mapping->get('drupal_bundle'));
    $webform_element = $webform->getElement($main_element_name);
    if (!$webform_element) {
      // This reference field does not exist.
      return;
    }
 
    try {
 
      $value = $entity->getElementData($main_element_name);
 
      $referenced_mappings = $this->mappedObjectStorage->loadByDrupal($webform_element['#target_type'], $value);
      if (!empty($referenced_mappings)) {
        $mapping = reset($referenced_mappings);
        return $mapping->sfid();
      }
    }
    catch (\Exception $e) {
      return NULL;
    }
  }

In this case we are finding the entity referred to in the webform submission, loading any mapping objects that may exist for that entity, and returning the Salesforce ID of the mapped object if it exists.  Yours will likely need to do something very different.

There are actually two related functions defined by the plugin interface, defined in the base class, and available for override as needed for setting pull and push values independently:

  /**
   * An extension of ::value, ::pushValue does some basic type-checking and
   * validation against Salesforce field types to protect against basic data
   * errors.
   *
   * @param \Drupal\Core\Entity\EntityInterface $entity
   * @param \Drupal\salesforce_mapping\Entity\SalesforceMappingInterface $mapping
   *
   * @return mixed
   */
  public function pushValue(EntityInterface $entity, SalesforceMappingInterface $mapping);
 
  /**
   * An extension of ::value, ::pullValue does some basic type-checking and
   * validation against Drupal field types to protect against basic data
   * errors.
   *
   * @param \Drupal\salesforce\SObject $sf_object
   * @param \Drupal\Core\Entity\EntityInterface $entity
   * @param \Drupal\salesforce_mapping\Entity\SalesforceMappingInterface $mapping
   *
   * @return mixed
   */
  public function pullValue(SObject $sf_object, EntityInterface $entity, SalesforceMappingInterface $mapping);
 

But be careful overriding them directly. The base class provides some useful handling of various data types that need massaging between Drupal and Salesforce, you may lose that if you aren’t careful. I encourage you to look at the details of both pushValue and pullValue before working on those.

Okay, with the configuration and values handled, we just need to deal with programmatically telling Drupal when it can pull and push these fields. Most of the time you don’t need to do this, but you can simplify some of the processing by overriding pull() and push() to make sure the have the right response hard coded instead of derived from other sources. In this case pulling the field would be bad, so we block that:

  /**
   * {@inheritdoc}
   */
  public function pull() {
    return FALSE;
  }

Also, we only want this mapping to appear as an option if the site has the webform module enabled. Without it there is no point in offering it at all. The plugin interface provides a function called isAllowed() for this purpose:

  /**
   * {@inheritdoc}
   */
  public static function isAllowed(SalesforceMappingInterface $mapping) {
    return \Drupal::service('module_handler')->moduleExists('webform');
  }

You can also use that function to limit a field even more tightly based on the mapping itself.

To further ensure the configuration of this mapping entity defines its dependencies correctly we can define additional dependencies in getDependencies(). Again here we are tied to the Webform module and we should enforce that during and config exports:

  /**
   * {@inheritdoc}
   */
  public function getDependencies(SalesforceMappingInterface $mapping) {
    return ['module' => ['webform']];
  }

And that is about it.  Once the class exists and is properly setup, all you need to do is rebuild the caches and you should see your new mapping field as an option on your Salesforce mapping objects (at least when isAllowed() is returning true).

In Praise of B Students

There is nothing wrong with just being good at something.

When I was in school I was a solid B student. I thought of myself as a B student and I worked hard to make sure I was always working at least a B average. I worked hard, but not too hard. Likely I could have been an A student if I’d worked harder, but I would not have been as happy. I didn’t feel the drive to be the best all the time, just good enough to keep up with what I was learning. Most people brag about being A students, but that wasn’t me and it has worked out pretty well in the long run.

For me a B student is someone who not only gets actually Bs most of the time in school but also knows how to work hard to get the grade they want, and how to relax so they recover and enjoy their lives along the way. If they fall a little short at some point they are perfectly capable of working a little harder to make up the ground. Life is not a sprint, it’s a marathon, we all have to save a bit for later. If you are used to running full out all the time, you do not always have the ability to step it up a little extra when needed nor how to relax and recover without stopping. In a world where we seem to be moving from helicopter parents to snowplow parents, there are still people that have fallen short from time to time, so they know how to pick themselves up and push forward.

There has been plenty written and said about the flaws of perfectionism so you don’t need to trust me, and I’m not going to waste time making the case that there are problems with pushing kids and college students to be a perfect A student all the time. And there are problems when companies only want to hire people who have those backgrounds.

In our current test-driven education culture there is a lot of push for kids to be the best. And the highly competitive nature of admissions for some top colleges can make it seem like a kid has to be perfect to succeed in life. But life isn’t Top Gun there actually are points for second place, and really more or less all the others (well paychecks and some level of happiness anyway). More importantly people who are used to a bit of failure can excel when faced with tough jobs that require a lot of on the job learning and trial and error work. That describes pretty much every modern office job, and lots of others – watch a plumber work on an old house and you’ll see plenty of trial and error being required to solve even basic problems.

Failure is a part of life. We all need to know how to acknowledge we made mistakes, how to recover, and how to ensure we don’t make the same mistake again in the future. People who earned a lot of B’s in school are familiar with all those things. We sometimes got lazy and ended up with a C instead of B – to stay a B student we then worked harder and got ourselves enough A’s to offset our missteps and bring our averages back up. Sometimes we really didn’t understand something and had to struggle through to make sure we stayed on pace.

B students also often know the difference between learning and getting the right answer on an assessment (a skill I would have told you I had but did not need anymore until I starting having to pass professional certification exams on a regular basis). We learned how to pass the test without knowing all the information, but knowing enough to get the grade we wanted. Getting a perfect score on an assessment is only useful if the assessment is perfect – most college professors and anyone who has taken a certification exam can tell you there are no perfect assessments. I had to pass tests in college, and I need to complete and maintain certifications now, but I don’t need to have every piece of information crammed into my brain all the time – that’s what Google is for. Once I pass, I’ve passed – it feels better to get more right answers, but as long as I get enough to pass I’m certified.

When I was in the 6th grade we had a whole day dedicated to studies skills taught by a guy named Mr. Gallagher. He wasn’t a regular teacher at my school, and I have no idea where they found him or what happened to him later, but some of what he taught us that day has stuck with me both in terms of skills and philosophy. He emphasized the need to understand how a test was written, both in terms of following directions so you didn’t give up silly points and so you know where to focus your time (hint: time goes first into the places you are likely to earn the most points), and when to guess and when to work through an answer carefully (again be greedy about points not the number of right answers).

Mr. Gallagher was the first person to put the idea into my head that the point of going to school is to get good grades instead of learning. I can still remember our teachers pushing back on that argument the next day, and I eventually arrived at the conclusion that they were both right: the purpose of school is to learn, but the purpose of tests is to get good enough grades so you can move forward in school. The thought exercise itself helped me see that learning material and passing a test weren’t actually always related. I had a teacher later who helped me further abstract this concept when he pointed out that he could easily write tests we all passed or all failed, so the purpose of the test writing was to find a measure of what he thought we should be able to do (that teacher and his wife were recently on Marketplace – thanks Bill that simple lesson was super useful in college and beyond). If pass and fail are at the whim of the test author, and they aren’t actually playing against me personally but really tracking a group of people, then I can play it as a game where I need to get myself into the top 80%. If I learn to play at that level reliably I’m going to do pretty well and I don’t have to stress being perfect all the time.

Yes, there are things in life that you cannot do unless you are one of the best. Not to mention the fact that the playing field isn’t level, which means some people have to work a whole lot harder than others to earn the same B and a report card full of Bs doesn’t get read the same for everyone. But we cannot all be the best, and pretending that’s not true doesn’t level the social playing fields.

And of course there are plenty of moments we need to strive to be as close to perfect as we are capable. But we also have to hit deadlines and get things done even if we know the work will be less-than perfect. Anyone who works in software development has to deal with this on a regular basis. As much as we may want to be perfect, shifting project requirements, project timelines, platform weakness, and our own mistakes all get in the way. We do the best we can with the skills, time, teams, and the tools we have. We need to know how to accept that we cannot fix every flaw and still get things done well enough to ensure the project is successful.

So when you are hiring people consider looking for people who already know how to be second best. People who know how to do really good work, but also know how to take risks, try something new, and in the process fail. People who know how to get things done well enough to be successful, even if it’s not perfect. We aren’t perfect, our work isn’t perfect, our world isn’t perfect, so don’t pretend your team has to be perfect.

Authors Note: Last month I failed to post to this blog for the first time. If you look back over the record you will see at least one post each month from the time the blog starts through August 2019 – then I missed September. Oops. I could make all kinds of excuses (I really did get sick right at the end of the month so my planned post was delayed a week), but the truth is I had time and material, I just didn’t get anything done. So now to make up for it I am trying to post every weekend this month.  I missed a goal by a little, so now I compensate to make up for it – then I’ll hopefully go back to my nice reliable routine. What can I say? I’m okay with a B.

Arts in the Heart 2019

This year’s Arts in the Heart of Augusta was not hurricane plagued the way last year’s was, and was a great weekend of good food, performances, and generally great chances to take pictures.

Each year opens with a parade to celebrate the diverse set of communities that have settled in and around Augusta.

The whole event is built around artists selling their work. They manage to pull in people from a fairly large area, but also make space for young local artists to try to get started selling their work.

For us, and for many others, the variety of ethic foods available for purchase is a large part of the draw.

It’s also a great event for just plain people watching. Between watching the various performances, enjoying the vendors, and the many activities, there are lots of chances to watch people having a good time with their friends and family.

Mixed in with the vendors, food, family activities, stages, and other goes on are always a mix of street performers.

There are several stages setup throughout the event, spanning several city blocks. The performances range from local signers to dance troops, street performers

Until next year…

Two people walking away from the camera as the sun is setting. A few other people milling about.  The woman is wearing a traditional Mexican dress and hat.