Docksal Pantheon Setup from Scratch

I recently had reason to switch over to using Docksal for a project, and on the whole I really like it as a good easy solution for getting a project specific Drupal dev environment up and running quickly. But like many dev tools the docs I found didn’t quite cover what I wanted because they made a bunch of assumptions.

Most assumed either I was starting a generic project or that I was starting a Pantheon specific project – that I already had Docksal experience. In my case I was looking for a quick emergency replacement environment for a long-running Pantheon project. So I wanted to install Docksal from scratch for a Pantheon project. 

Fairly recently Docksal added support for a project init command that helps setup for Acquia, Pantheon, and Pantheon.sh, but pull init isn’t really well documented and requires a few preconditions.

Since I had to run a dozen Google searches, and ask several friends for help, to make it work I figured I’d write it up.

Install Docksal

First follow the basic Docksal installation instructions for your host operating system. Once that completes, if you are using Linux as the host OS log out and log back in (it just added your user to a group and you need that access to start up docker).

Add Pantheon Machine Token

Next you need to have a Pantheon machine token so that terminus can run within the new container you’re about to create. If you don’t have one already follow Pantheon’s instructions to create one and save if someplace safe (like your password manager).

Once you have a machine token you need to tell Docksal about it.  There are instructions for that (but they aren’t in the instructions for setting up docksal with pull init) basically you add the key to your docksal.env file:

SECRET_TERMINUS_TOKEN="HASH_VALUE_PROVIDED_BY_PANTHEON_HERE"

 Also if you are using Linux you should note that those instructions linked above say the file goes in $HOME/docksal/docksal.env, but you really want $HOME/.docksal/docksal.env (note the dot in front of docksal to hide the directory).

Setup SSH Key

With the machine token in place you are almost ready to run the setup command, just one more precondition.  If you haven’t been using docker or docksal they don’t know about your SSH key yet, and pull init assumes it’s around.  So you need to tell Docksal to load it but running fin ssh-key add.  

If the whole setup is new, you may also need to create your key and add it to Pantheon.  Once you have done that, if you are using a default SSH key name and location it should pick it up automatically (I have not tried this yet on Windows so mileage there may vary – if you know the answer please leave me a comment). It also is a good idea to make sure the key itself is working right but getting the git clone command from your Pantheon dashboard and trying a manual clone on the command line (delete once it’s done, this is just to prove you can).

Run Pull Init

Now finally you are ready to run fin pull init: 

fin pull init --hostingplatform=pantheon --hostingsite=[site-machine-name] --hosting-env=[environment-name]

Docksal will now setup the site, maybe ask you a couple questions, and clone the repo. It will leave a couple things out you may need: database setup, and .htaccess.

Add .htaccess as needed

Pantheon uses nginx.  Docksal’s formula uses Apache. If you don’t keep a .htaccess file in your project (and while there is not reason not to, some Pantheon setups don’t keep anything extra around) you need to put it back. If you don’t have a copy handy, copy and paste the content from the project repo:  https://git.drupalcode.org/project/drupal/blob/8.8.x/.htaccess

Finally, you need to tell Drupal where to find the Docksal copy of the database.  For that you need a settings.local.php file. Your project likely has a default version of this, which may contain things you may or may not want. Docksal creates a default database (named default) and provides a user named…“user”, which has a password of “user”.  The host’s name is ‘db’. So into your settings.local.php file you need to include at the very least:

<?php
$databases = array(
  'default' =>
    array(
      'default' =>
      array(
        'database' => 'default',
        'username' => 'user',
        'password' => 'user',
        'host' => 'db',
        'port' => '',
        'driver' => 'mysql',
        'prefix' => '',
      ),
    ),
);

With the database now fully linked up to Drupal, you can now ask Docksal to pull down a copy of the database and a copy of the site files:

fin pull db

fin pull files

In the future you can also pull down code changes:

fin pull code

Bonus point: do this on a server.

On occasion it’s useful to have all this setup on a remote server not just a local machine. There are a few more steps to go to do that safely.

First you may want to enable Basic HTTP Auth just to keep away from the prying eyes of Googlebot and friends.  There are directions for that step (you’ll want the Apache instructions). Next you need to make sure that Docksal is actually listing to the host’s requests and that they are forwarded into the containers.  Lots of blog posts say DOCKSAL_VHOST_PROXY_IP=0.0.0.0 fin reset proxy. But it turns out that fin reset proxy is out of date, instead you want: 

DOCKSAL_VHOST_PROXY_IP=0.0.0.0 fin system reset.  

Next you need to add the vhost to the docksal.env file we were working with earlier:

 VIRTUAL_HOST="test.example.org"

Run fin up to get Docksal to pick up the changes (this section is based on these old instructions).

Now you need to add either a DNS entry someplace, or update your machine’s /etc/hosts file to look in the right place (the public IP address of the host machine).

SC DUG May 2019

For this month’s SC DUG, Mauricio Orozco from the South Carolina Commission for Minority Affairs shared his notes and lessons learned during his first DrupalCon North America.

We frequently use these presentations to practice new presentations, try out heavily revised versions, and test out new ideas with a friendly audience. If you want to see a polished version checkout our group members’ talks at camps and cons. So if some of the content of these videos seems a bit rough please understand we are all learning all the time and we are open to constructive feedback.

If you would like to join us please check out our up coming events on Meetup for meeting times, locations, and connection information.

Hague Appeal for Peace Pictures

This week marks the 20th Anniversary of the Hague Appeal for Peace.

This week marks the 20th anniversary of the Hague Appeal for Peace and everything that happened (and didn’t) as part of that event and since, I decided to post some of my pictures from that adventure.

In my post on being an activist back in March I mentioned attending the Hague Appeal and the peace walk that followed. I was part of a delegation from Philadelphia Yearly Meeting; a group mostly made up of college students and a few older high school students, along with a few adults who handled the logistics and kept us on track more or less.

I have ten boxes of slides, and a few years ago I scanned them as best I could but frankly the scans aren’t great. The slides, which were more than ten years old at the time, had already started to fade and color shift as a result of their age. I did some color correction as I prepped them for this, but I also like the feel of some being somewhat faded and shifted with time. There are shared here full frame, and some are roughly cropped, but none carefully realigned. Since they are now pushing twenty I decided that I wanted to leave them all at or near full size and try to capture a bit of the way I saw the world then, and less of how I would edit it now. I like the rough visual feel they have as part of reflected on partially faded memories.

That trip was an important few weeks in my life, and I’ve been having a great time going back through the pictures. If you were with me on that trip and wonder if I have other pictures of you kicking around I might so send me a note and I’ll try to see what’s around and sent some your way.

FormAssembly Dynamic Parameter Signing

For a project I’ve been working on recently we had need to create a module that provides secure redirects from a Drupal site to FormAssembly. Overall the module does a number of things, but handling dynamic parameter signing was the thing that took the most time.

FormAssembly provides a variety of great features for creating flexible forms that integrate with Salesforce. One of the more popular features is its ability to pull data from Salesforce to prefill fields on a form. But the downside is that it is easy to create forms that leak information from Salesforce into those forms, and create privacy risks.

To address this, FormAssembly allows 3rd party tools to securely sign URLs that contain parameters (often Salesforce IDs) that could be used to extract information through an iteration attack and other basic approaches. This secure signing process can be done statically but for most interesting projects you want to sign the URLs dynamically. The dynamic signing process allows you alter the parameters on the fly and set an expiration date to limit the value of a stolen link. Our project required this approach.

But the dynamic signing process has a couple sharp corners. First, it’s rarely done outside of Salesforce so there aren’t a lot of code samples around, and none that I could find in PHP.  Second, FormAssembly is very open and honest about the fact that they do not provide support on this feature. So I had to create my own process from the documentation they provide.  The docs are good, but very Salesforce centric, with all code samples in APEX.

The process involves preparing the data for signature, generating a HMAC-SHA256 with a form specific pre-shared key (in binary mode), converted to a string using base64, and finally URL encode the result.

Their convention for preparing the data is straightforward. You format all parameters as just their key and value strung together: key1Value1key2Value2

The interesting part is the actual HMAC-SHA256, which needs to be generated in binary mode, something that is often the default mode but not in PHP (in fact most PHP devs I’ve talked don’t realize the last parameter to hash_hmac() is useful, if you are doing this in another language check out this collection of examples).

From there you encode the output in base-64 (which results in a 44 character hash), and URL encode the hash to make sure it’s URL safe, and you’ll end up a few characters longer.

Finally you add you hash to the query string, and you’re ready to go.

To help anyone else who needs to do this, I generalized this part of the solution and I created and tossed it into Gist.

On Being An Activist

When someone tries to insult you with what you often see as a compliment it is worth stopping to reflect. Am I an activist? If I’m not, should I be?

On Valentines Day this year my wife and I spent a few hours at DSS for a meeting related to some of the children we work with in the Guardian ad Litem program. In the course of a rather tense conversation a caseworker tossed out “Well, I am not an activist.” with the clear intention of implying that I am, and that activists are a problem.

It is the first time I can recall being called an Activist as an insult, and I’ve been a bit hung up on the topic ever since.

Between my personal and professional life I have a very high standard for what it means to be an activist. I have friends, including a former boss, who were arrested the recently protesting the conditions asylum seekers face coming to the US. Among them my friend Lucy who is willing to do this for more than just one cause. I was around when AFSC started to help restore the legacy of Bayard Rustin and his work planning the March on Washington and making the phrase “Speak truth to power” commonly known. My friend Tom Fox traveled around the middle east participating in peace movements until he was kidnapped and murdered in Iraq.

Those are activists.

At AFSC I had colleagues who would argue if you haven’t been arrested for a cause you aren’t really an activist. We had critics who argued that because AFSC staff were paid they couldn’t be true activists. I didn’t then, nor now, fully agree with those arguments, but my point is that when someone calls me an “activist” those are the comparisons they are drawing in my mind.

My credentials as an activist on that scale are weak at best. The first time I spent a lot of time with activists was in 1999 during the Hague Appeal for Peace and a peace walk that followed. The group walked from the Peace Palace – home of the international criminal court – in The Hague, Netherlands to Nato Headquarters in Brussels, Belgium. That picture of the water cannon firing on a crowd at the top of the page is mine, although I wasn’t willing to risk arrest that day (my sister was getting married the next week and my mother would have killed me if I’d missed it because I had been arrested in Europe – would a true activist be deterred by such things?). It was a great experience, but didn’t do a thing toward our goal of nuclear disarmament – I now live in a town supported by nuclear weapons maintenance (and soon pit production too).

After college I took a job at AFSC which consisted of largely back office functions of one type or another – while defining for my career and personally gratifying work there is an important difference between building the tools activists need to communicate and being the activist. In 2008 I was part of planning a peace conference in Philadelphia as part of the Peace and Concerns standing committee, but it is important to note that I objected to the civil disobedience that was part of that event (being a consensus driven process people feared I would block it entirely – but I stood aside so they could move forward).

I’ve been to other events and protests, although sometimes as much accidentally as purposefully. So while the account is not empty, it’s not exactly the kinda stuff that gets you into history books, or even an FBI file worth reading.

Having spent much of my professional life supporting back office functions on nonprofits, and now interacting with DSS as a volunteer who has to be careful about what I share since I have to maintain the privacy of the kids we work with, I struggle to envision myself as an activist.  I support activists sure, but I don’t see myself as one.

But when someone tries to insult you with what you often see as a compliment it is worth stopping to reflect.  Am I an activist? If I’m not, should I be?

It occurred to me this case worker has a much lower standard of what it means to be an activist than I do – anyone who simply speaks against the status quo in favor of well established laws and precedents are activists in his book. To be fair he’s not far off the suggestion Bayard Rustin, and the committee who helped him write Speak Truth to Power, were making. And as much as I am sure they would deny it, the caseworkers are the most powerful people in the lives of children in foster care: they dictate where the children live, who they can talk to, if/when they see siblings, when they buy clothes, where they go to school, what doctors they see, and without an active advocate they shape how the courts see the children. And right now in South Carolina their power is being tested and reigned in because a group of Guardians ad Litem stood up a few years ago to the rampant systemic abuses. The ramifications of that class action are still being determined, and no one really knows what the lasting effect will be. But this case worker has inspired me to make sure we honor the sacrifices they made (all were forced to stop fighting for the named children because they were “distractions”).

I’m not sure I am an activist, but I promised those kids I would stay with them until the judge ordered me to stop. No matter what taunting I get from the case workers, their bosses, and others within the power structure I can speak truth to power as long as I must.

SC DUG February 2019

Will Jackson – Local Development in Docksal

For the SC DUG meeting this month Will Jackson from Kanopi Studios gave a talk about using Docksal for local Drupal development. Will has the joy of working with some of the Docksal developers and has become an advocate for the simplicity and power Docksal provides.

We frequently use these presentations to practice new presentations, try out heavily revised versions, and test out new ideas with a friendly audience. If you want to see a polished version checkout our group members’ talks at camps and cons. So if some of the content of these videos seems a bit rough please understand we are all learning all the time and we are open to constructive feedback.

If you would like to join us please check out our up coming events on Meetup for meeting times, locations, and connection information.

Aiken State Park Photography

A few weeks ago I took a few hours to scratch an itch and went out did a little local photography. I spent much of my time wondering the trails through Aiken State Park. Although there are a few included here from another stop I made, and from the Lunar eclipse on January 21st.

SC DUG November 2018

Kaylan Wagner – Real world lessons from online games.

This fall the South Carolina Drupal User’s Group started using Zoom are part of all our meetings. Sometimes the technology has worked better than others, but when it works in our favor we are recording the presentations and sharing them when we can.

In November Kaylan Wagner gave a draft talk on using experiences in the world of online gaming to be a better remote team member.

We frequently use these presentations to practice new presentations and test out new ideas. If you want to see a polished version hunt group members out at camps and cons. So if some of the content of these videos seems a bit rough please understand we are all learning all the time and we are open to constructive feedback.

If you would like to join us please check out our up coming events on Meetup for meeting times, locations, and connection information.

SC DUG September 2018

Chis Zietlow – Using Machine Learning to Improve UX

This fall the South Carolina Drupal User’s Group started using Zoom are part of all our meetings. Sometimes the technology has worked better than others, but when it works in our favor we are recording the presentations and sharing them when we can.

Chris Zietlow presented back in September about using Machine Learning to Improve UX.

We frequently use these presentations to practice new presentations and test out new ideas. If you want to see a polished version hunt group members out at camps and cons. So if some of the content of these videos seems a bit rough please understand we are all learning all the time and we are open to constructive feedback.

If you would like to join us please check out our up coming events on Meetup for meeting times, locations, and connection information.

Drupal 8 Batch Services

For this month’s South Carolina Drupal User Group I gave a talk about creating Batch Services in Drupal 8. As a quick side note we are trying to include video conference access to all our meetings so please feel free to join us even if you cannot come in person.

Since Drupal 8 was first released I have been frustrated by the fact that Drupal 8 batch jobs were basically untouched from previous versions. There is nothing strictly wrong with that approach, but it has never felt right to me particularly when doing things in a batch job that I might also want to do in another context – that really should be a service and I should write those core jobs first. After several frustrating experiences trying to find a solution I like, I finally created a module that provides an abstract class that can be used to create a service that handles this problem just more elegantly. The project also includes an example module to provide a sample service.

Some of the text in the slides got cut off by the Zoom video window, so I uploaded them to SlideShare as well:


Quick Batch Overview

If you are new to Drupal batches there are lots of articles around that go into details of traditional implementations, so this will be a super quick overview.

To define a batch you generate an array in a particular format – typically as part of a form submit process – and pass that array to batch_set(). The array defines some basic messages, a list of operations, a function to call when the batch is finished, and optionally a few other details. The minimal array would be something like:

  <?php  // Setup final batch array.
    $batch = [
      'title'    => 'Page title',
      'init_message' => 'Openning message',
      'operations'  => [],
      'finished' => '\some\class\namespace\and\name::finishedBatch',
    ];

The interesting part should be in that operations array, which is a list of tasks to be run, but getting all your functions setup and the batch array generated can often be its own project.

Each operation is a function that implements callback_batch_operation(), and the data to feed that function. The callbacks are just functions that have a final parameter that is an array reference typically called $context. The function can either perform all the needed work on the provided parameters, or perform part of that work and update the $context['sandbox']['finished'] value to be a number between 0 and 1. Once finished reaches 1 (or isn’t set at the end of the function) batch declares that task complete and moves on to the next one in the queue. Once all tasks are complete it calls the function provided as the finished value of the array that defined the batch.

The finish function implements callback_batch_finish() which means it accepts three parameters: $success, $results, and $operations: $success is true when all tasks completed without error; $results is an array of data you can feed into the $context array during processing; $operations is your operations list again.

Those functions are all expected to be static methods on classes or, more commonly, a function defined in a procedural code block imported from a separate file (which can be provided in the batch array).

My replacement batch service

It’s those blocks of procedural code and classes of nothing but static methods that bug me so much. Admittedly the batch system is convenient and works well enough to handle major tasks for lots of modules. But in Drupal 8 we have a whole suite of services and plugins that are designed to be run in specific contexts that batch does not provide by default. While we can access the Drupal service container and get the objects we need the batch code always feels clunky and out of place within a well structured module or project. What’s more I have often created batches that benefit from having the key tasks be functions of a service not just specific to the batch process.

So after several attempts to force batches and services to play nice together I finally created this module to force a marriage. Even though there are places which required a bit of compromise, but I think I have most of that contained in the abstract class so I don’t have to worry about it on a regular basis. That makes my final code with complex logic and processing far cleaner and easier to maintain.

The Batch Service Interface module provides an interface an an abstract class that implements parts of it: abstract class AbstractBatchService implements BatchServiceInterface. The developer extending that class only needs to define a service that handles generating a list of operations that call local methods of the service and the finish batch function (also as a local method). Nearly everything else is handled by the parent class.

The implementation I provided in the example submodule ends up four simple methods. Even in more complex jobs all the real work could be contained in a method that is isolated from the oddities of batch processing.

<?php

namespace Drupal\batch_example;
use Drupal\node\Entity\Node;
use Drupal\batch_service_interface\AbstractBatchService;

/**
 * Class ExampleBatchService logs the name of nodes with id provided on form.
 */
class ExampleBatchService extends AbstractBatchService {

  /**
   * Must be set in child classes to be the service name so the service can
   * bootstrap itself.
   *
   * @var string
   */
  protected static $serviceName = 'batch_example.example_batch';

  /**
   * Data from the form as needed.
   */
  public function generateBatchJob($data) {
    $ops = [];
    for ($i = 0; $i < $data['message_count']; $i++ ) {
      $ops[] = [
        'logMessage' => ['MessageIndex' => $i + 1],
      ];
    }

    return $this->prepBatchArray($this->t('Logging Messages'), $this->t('Starting Batch Processing'), $ops);
  }

  public function logMessage($data, &amp;$context) {

    $this->logger->info($this->getRandomMessage());

    if (!isset($context['results']['message_count'])) {
      $context['results']['message_count'] = 0;
    }
    $context['results']['message_count']++;

  }

  public function doFinishBatch($success, $results, $operations) {
    drupal_set_message($this->t('Logged %count quotes', ['%count' => $results['message_count']]));
  }

  public function getRandomMessage() {
    $messages = [
      // list of messages to select from
    ];

    return $messages[array_rand($messages)];

  }

}

There is the oddity that you have to tell the service its own name so it can bootstrap itself. If there is a way around that I’d love to know it. But really one have one line of code that’s a bit strange, everything else is now fairly clear call and response.

One of the nice upsides to this solution is you could write tests for the service that look and feel just like any other services tests. The methods could all be called once, and you are not trying to run tests against a procedural code block or a class that is nothing but static methods.

I would love to hear ideas about ways I could make this solution stronger. So please drop me a comment or send me a patch.

Related core efforts

There is an effort to try to do similar things in core, but they look like they have some distance left to travel. Obviously once that work is complete it is likely to be better than what I have created, but in the meantime my service allows for a new level of abstraction without waiting for core’s updates to be complete.