2018 Jun 11

Webforms as of Drupal 8 now has a cool new feature called "remote post" under the "handlers" section. What this does is allow form submission data to also be sent to a URL. Typically when we talk about handlers in this context, we mean processes that take the form data and do something with it. Usually the common one is to send an email. Now we're going to create a remote post handler to send the data to a URL we'll setup later. 

There are primarily 2 methods to create a lead in Salesforce. The first is using an older way called "web-to-lead". This essentially allows you to build your own form and set the action to POST right to Salesforce. As long as you name your fields to align with the field names in your force account, that's all that is needed. The only thing associating your custom form to your account though is a hidden field with your org_id. This isn't ideal and frankly a little brittle, but it has been the method used for some time. 

In this post, we'll discuss instead using the Force API to create our leads. Some benefits of this approach include heightened security, flexibility (for more than just creating leads) and better handling for larger amounts of data. One key limitation of the web-to-lead approach is its limit of 500 leads per day. The API on the other hand has a much higher allowance. The API limit depends on the organization type and edition though.     

Creating a module to handle the form data

As mentioned above, we'll need a URL to send the webform data to as defined in the remote post handler. For this we'll create a new route that will be the gateway in to the processing bit of our new module. 

salesforce.info.yml

name: Salesforce Integration
description: Integration with Salesforce to take webform submissions and post to salesforce as leads
core: 8.x
package: Custom
type: module

salesforce.routing.yml

salesforce.settings:
  path: '/admin/config/salesforce/settings'
  defaults:
    _form: '\Drupal\salesforce\Form\salesforceSettingsForm'
    _title: 'Salesforce Admin'
  requirements:
    _permission: 'administer site configuration'

salesforce.generate_lead:
    path: /salesforce-generate-lead
    defaults:
        _controller: Drupal\salesforce\Controller\SalesforceController::generate_lead
    requirements:
        _permission: 'access content'

Once that is configured, you can configure the remote post handler within the webform. This config can be found under Settings -> Emails/Handlers -> Add Handler -> Remote post. Give the handler any name and put this in the "Completed URL" field: http://localhost/salesforce-generate-lead. Once the form is submitted now, this handler will send a POST request to that route, where we can then process the data in to Salesforce as a lead.

Creating a Form to store our API credentials

We also are going to create an admin form to easily manage our credentials for use with interacting with the force API. Creating an app in Salesforce is outside the scope of this post, but there are some comments inline below that give hints about where each value comes from.

salesforce.links.menu.yml

# provides links on the admin -> config page
salesforce.admin_config_salesforce:
  title: Salesforce
  route_name: salesforce.settings
  parent: system.admin_config
  description: 'Manage salesforce, son.'
  weight: 6

salesforce.settings:
  title: 'Salesforce Configuration'
  parent: salesforce.admin_config_salesforce
  description: 'Configure Salesforce API settings, etc.'
  route_name: salesforce.settings

src/Form/salesforceSettingsForm.php

<?php
// https://www.drupal.org/docs/8/api/configuration-api/working-with-configuration-forms
namespace Drupal\salesforce\Form;

use Drupal\Core\Form\ConfigFormBase;
use Drupal\Core\Form\FormStateInterface;

/**
 * Configure salesforce settings for this site.
 */
class salesforceSettingsForm extends ConfigFormBase {

  const FORM_ID = 'salesforce_admin_settings';

  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return self::FORM_ID;
  }

  /**
   * {@inheritdoc}
   */
  protected function getEditableConfigNames() {
    return [
      'salesforce.settings',
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    $config = $this->config('salesforce.settings');

    $form[$this->getFormId() . '_base_url'] = array(
      '#type' => 'textfield',
      '#title' => t('Base URL'),
      '#default_value' => $config->get($this->getFormId() . '_base_url', ''),
      '#required' => TRUE,
    );
    $form[$this->getFormId() . '_owner_id'] = array(
      '#type' => 'textfield',
      '#title' => t('Owner ID'),
      '#default_value' => $config->get($this->getFormId() . '_owner_id', ''),
      '#description' => t('The default owner of the leads created via the API.<br>
      The ownership seems to go to the Leads Owner user.<br>
      The ID was found by simply opening the user\'s record in SF and getting ID from URL.'),
      '#required' => TRUE,
    );
    $form[$this->getFormId() . '_client_id'] = array(
      '#type' => 'textfield',
      '#title' => t('Client ID'),
      '#default_value' => $config->get($this->getFormId() . '_client_id', ''),
      '#required' => TRUE,
    );
    $form[$this->getFormId() . '_client_secret'] = array(
      '#type' => 'textfield',
      '#title' => t('Client Secret'),
      '#default_value' => $config->get($this->getFormId() . '_client_secret', ''),
      '#description' => t('These two fields can be found in the connected app overview in your SalesForce account<br>
      You can find them by clicking on your account (top-right) and selecting Setup<br>
      Once that open, search for \'Apps\' in the left menu and select Build -> Create -> Apps<br>
      Your connected apps are on the bottom of the new window.<br>

      To create a new connected app, follow this: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_defining_remote_access_applications.htm<br>
      We ignore the callback url (but have to enter one) because we use the username/password oauth flow.<br>
      See: (https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_understanding_username_password_oauth_flow.htm)<br>'),
      '#required' => TRUE,
    );
    $form[$this->getFormId() . '_username'] = array(
      '#type' => 'textfield',
      '#title' => t('Username'),
      '#default_value' => $config->get($this->getFormId() . '_username', ''),
      '#description' => t('Username of the user that will be used to authenticate'),
      '#required' => TRUE,
    );
    $form[$this->getFormId() . '_password'] = array(
      '#type' => 'textfield',
      '#title' => t('Password'),
      '#default_value' => $config->get($this->getFormId() . '_password', ''),
      '#description' => t('Password of the user above with the security token appended at the end<br>
      See: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_understanding_username_password_oauth_flow.htm<br>
      To reset security token see: https://help.salesforce.com/articleView?id=user_security_token.htm&type=5<br>'),
      '#required' => TRUE,
    );

    return parent::buildForm($form, $form_state);
  }

  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    // Retrieve the configuration
    $this->config('salesforce.settings')
      // Set the submitted configuration setting
      ->set($this->getFormId() . '_base_url', $form_state->getValue($this->getFormId() . '_base_url'))
      ->set($this->getFormId() . '_owner_id', $form_state->getValue($this->getFormId() . '_owner_id'))
      ->set($this->getFormId() . '_client_id', $form_state->getValue($this->getFormId() . '_client_id'))
      ->set($this->getFormId() . '_client_secret', $form_state->getValue($this->getFormId() . '_client_secret'))
      ->set($this->getFormId() . '_username', $form_state->getValue($this->getFormId() . '_username'))
      ->set($this->getFormId() . '_password', $form_state->getValue($this->getFormId() . '_password'))
      ->save();

    parent::submitForm($form, $form_state);
  }
}

Handling the incoming POST data

Here we'll look at the part of the module that accepts the data coming in from the webform submission. We have a controller that just sucks in all POST data using Drupal's request object. Then, it passes it off to a separate class (bottom of this post) to handle the processing of this data in to Salesforce as a lead.

src/Controller/SalesforceController.php

<?php

namespace Drupal\salesforce\Controller;

use Drupal\salesforce\Form\salesforceSettingsForm;
use Symfony\Component\HttpFoundation\JsonResponse;

class SalesforceController {
  public function generate_lead()
  {
    $lead = false;
    $payload = $this->build_payload();
    if ($payload) {
      try {
        $salesforce = new \Salesforce();
        $lead = $salesforce->post_a_lead($payload);
      } catch (\Exception $e) {
        \Drupal::logger('salesforce')->error("ERROR from".__FILE__.":".__LINE__." ".$e->getMessage());
      }
      if ($lead) {
        // if it was good, say so
        \Drupal::logger('salesforce')->info("SUCCESS! Created new lead in salesforce with the following info: <pre>".print_r($payload, true)."</pre>");
        $response = new JsonResponse();
        return $response->setStatusCode(JsonResponse::HTTP_CREATED);
      }else {
        $response = new JsonResponse();
        return $response->setStatusCode(JsonResponse::HTTP_NOT_FOUND);
      }
    }
  }

  public function build_payload()
  {
    $salesforce_config = \Drupal::config('salesforce.settings');
    $payload = \Drupal::request()->request->all();
    // remove unwanted fields
    if (isset($payload['metatag'])) {
      unset($payload['metatag']);
    }
    if ($payload) {
      // add on owner_id
      $payload['OwnerId'] = $salesforce_config->get(salesforceSettingsForm::FORM_ID . '_owner_id');
      return $payload;
    }else {
      return false;
    }
  }

}

 

Quick walk-through of creating the lead

Preparing the data

The post_a_lead method is called from the controller with the data passed in. But first, the construct method first pulls in our API credentials that were stored as part of the admin page for this module. 

Then we call a helper function to prepare the data. This is essentially a whitelist of fields that exist in salesforce. We want to do this because if we try and pass a value that doesn't exist, the creation won't work. This is a key difference between using the API and the aforementioned web-to-lead method, which would continue on and just ignore fields that don't match. We also remove the campaignid field value for use later in the call to create the lead. This was setup as a hidden field in the form to store the value, but we don't want to include it as a field here, rather a parameter when sending the data.

Lastly, we break up the address webform element if used. Salesforce wants this address data passed in separately. If your clients are using the address element, we need to do that processing here. Otherwise they can be instructed to create separate fields named appropriately for the discrete data.

Sending the data to create a lead

Back in the post_a_lead method, we take the prepared form data and send it off. We've opted to use GuzzleHttp for this task as it comes with Drupal and is otherwise appropriate for a job like this. 

If we get back a created status, we then proceed to link this lead to an existing campaign. This is that hidden value we unset earlier. In Salesforce, leads are typically associated to Campaigns. We are making that link here with another helper method called link_lead_to_campaign. Once we get back a 201 for that, then it completes the creation of the lead.

salesforce.module

<?php

use Drupal\salesforce\Form\salesforceSettingsForm;
use GuzzleHttp\Client;

/**
 * Provides methods for interacting with Salesforce API
 */
class Salesforce
{
  public $client = '';
  public $salesforce_access_token = '';
  public $campaign_id = '';

  function __construct()
  {
    $salesforce_config = \Drupal::config('salesforce.settings');
    $this->client = new \GuzzleHttp\Client();

    // These two fields can be found in the connected app overview in your SalesForce account
    // You can find them by clicking on your account (top-right) and selecting Setup
    // Once that open, search for 'Apps' in the left menu and select Build -> Create -> Apps
    // Your connected apps are on the bottom of the new window.

    // To create a new connected app, follow this: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_defining_remote_access_applications.htm
    // We ignore the callback url (but have to enter one) because we use the username/password oauth flow.
    // See: (https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_understanding_username_password_oauth_flow.htm)
    $this->client_id = $salesforce_config->get(salesforceSettingsForm::FORM_ID . '_client_id');
    $this->client_secret = $salesforce_config->get(salesforceSettingsForm::FORM_ID . '_client_secret');

    // Username of the user that will be used to authenticate
    $this->username = $salesforce_config->get(salesforceSettingsForm::FORM_ID . '_username');

    // Password of the user above with the security token appended at the end
    // See: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_understanding_username_password_oauth_flow.htm
    // To reset security token see: https://help.salesforce.com/articleView?id=user_security_token.htm&type=5
    $this->password = $salesforce_config->get(salesforceSettingsForm::FORM_ID . '_password');

    // Base URL of the instance you are going to hit
    $this->base_url = $salesforce_config->get(salesforceSettingsForm::FORM_ID . '_base_url');

    // The default owner of the leads created via the API.
    // From what we could tell of the video walkthrough, the ownership seems to go to the Leads Owner user.
    // The ID was found by simply opening the user's record in SF and getting ID from URL.
    $this->owner_id = $salesforce_config->get(salesforceSettingsForm::FORM_ID . '_owner_id');
  }

  /**
   * Make sure we only pass field values to Force
   * that it wants. Also translate values to conform
   * to Force, i.e. Address field
   */
  protected function prepare_fields($form_content = []){
    $whitelisted_fields = ['firstname','lastname','email','phone','mobilephone',
    'fax','company','companydunsnumber','industry','description','annualrevenue','createdby',
    'website','title','rating','numberofemployees','name','owner','donotcall',
    'hasoptedoutofemail','status','leadsource','currencyisocode','lasttransferdate','lastmodifiedby',
    'address','city','state','country','street','postalcode','salutation','annualrevenue',
    'numberofemployees','fax','vertical_market__c'];

    // We remove the CampaignID from form_content because it doesn't exist on the Lead object in SF.
    // We will use it later to call link_lead_to_campaign
    $this->campaign_id = $form_content["campaignid"];
    unset($form_content["campaignid"]);
    // for the rest of the fields, filter out ones that Force won't accept
    foreach ($form_content as $key => $value) {
      if (!in_array($key,$whitelisted_fields)) {
        unset($form_content[$key]);
      }
    }

    // if address field is present, break up its parts and rename them to work with Force
    if (array_key_exists('address', $form_content)) {
      foreach ($form_content['address'] as $key => $value) {
        switch ($key) {
          case 'address':
            $form_content['street'] = $value;
            break;
          case 'address_2':
            $form_content['street'] .= ' '.$value;
            break;
          case 'city':
            $form_content['city'] = $value;
            break;
          case 'state_province':
            $form_content['state'] = $value;
            break;
          case 'postal_code':
            $form_content['postalcode'] = $value;
            break;
          case 'country':
            $form_content['country'] = $value;
            break;
        }
      }
      // unset the webform address field so it doesnt get included and trip things up
      unset($form_content['address']);
    }
    return $form_content;
  }

  /**
   * create a lead in SF
   *
   * All fields sent in the form_content should exist in Salesforce, otherwise you will get an error from the API and this function will crap out.
   * See this for a list of default fields: https://developer.salesforce.com/docs/api-explorer/sobject/Lead
   * They can also define custom fields, found by going into Setup and searching for Leads in the left menu
   * and selecting Build -> Customize -> Leads -> Fields. Custom fields are at the bottom.
   *
   * Dates should be in the formats indicated here: https://success.salesforce.com/answers?id=90630000000gwpvAAA
   *
   * We still need to refine how to error handle this.
   */
  public function post_a_lead($form_content = [])
  {
    // filter/prepare form values for submission to Force
    $form_content = $this->prepare_fields($form_content);

    if ($this->authenticate_to_salesforce()) {
      try {
        $res = $this->client->request(
          'POST',
          $this->base_url . 'services/data/v42.0/sobjects/Lead/',
          [
            'headers' =>
              [
                'Content-Type' => 'application/json',
                'Authorization' => 'Bearer ' . $this->salesforce_access_token,
              ],
            'body' =>
              json_encode($form_content)
          ]
        );
      } catch (\Exception $e) {
        \Drupal::logger('salesforce')->error("ERROR from".__FILE__.":".__LINE__." ".$e->getMessage());
      }

      if (isset($res) && $res->getStatusCode() == 201) {
        $contents = json_decode($res->getBody()->getContents());
        $this->link_lead_to_campaign($contents->id, $this->campaign_id);
        return true;
      }else {
        echo "Request didn't return a 201";
      }
    } else {
      echo 'Authentication failed';
    }
    return false;
  }

  /**
   * authenticate to SF
   *
   * This will return us an access token that we need for all subsequent calls.
   */
  protected function authenticate_to_salesforce()
  {
    try {
      $res = $this->client->request(
        'POST',
        $this->base_url . 'services/oauth2/token',
        [
          'form_params' =>
            [
              // grant_type will define the oauth flow used.
              // See: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_understanding_authentication.htm
              'grant_type' => 'password',
              'client_id' => $this->client_id,
              'client_secret' => $this->client_secret,
              'username' => $this->username,
              'password' => $this->password
            ]
        ]
      );
    } catch (\Exception $e) {
      \Drupal::logger('salesforce')->error("ERROR from".__FILE__.":".__LINE__." ".$e->getMessage());
    }

    if (isset($res) && $res->getStatusCode() == 200) {
      $contents = json_decode($res->getBody()->getContents());
      $this->salesforce_access_token = $contents->access_token;
      return true;
    }else {
      return false;
    }
  }

  /**
   * create a lead in SF
   *
   * Leads do not have a direct relationship to Campaigns.
   * The relationship is defined by another object called CampaignMember.
   *
   * When creating a lead in the UI, SF will automatically create a CampaignMember object for you.
   * But not via the API, so we need another call.
   *
   * Passed in the lead_id by post_a_lead for the newly created SF lead.
   * Passed in the campaign_id by post_a_lead from the form.
   *
   * Also need to fine tune the error handling.
   */
  protected function link_lead_to_campaign($lead_id, $campaign_id)
  {
    try {
      $res = $this->client->request(
        'POST',
        $this->base_url . 'services/data/v42.0/sobjects/CampaignMember/',
        [
          'headers' =>
            [
              'Content-Type' => 'application/json',
              'Authorization' => 'Bearer ' . $this->salesforce_access_token,
            ],
          'body' =>
            json_encode(
              [
                "LeadId" => $lead_id,
                "CampaignId" => $campaign_id
              ]
            )
        ]
      );
    } catch (\Exception $e) {
      \Drupal::logger('salesforce')->error("ERROR from".__FILE__.":".__LINE__." ".$e->getMessage());
    }

    if (isset($res) && $res->getStatusCode() == 201) {
      $contents = json_decode($res->getBody()->getContents());
    } else {
      echo "Couldn't link to a campaign";
    }
  }
}

Conclusion

As of this writing, there doesn't seem to be many options available in terms of contributed modules to accomplish this task. The above is admittedly quite custom, but we feel that it provides the right amount of flexibility needed when dealing with Salesforce. In also leverages the new remote post handler in Webforms, something that puts a lot of the power back in to the hands of the site admins. For a more out-of-the-box solution, one option that may be worth watching is the Salesforce Suite module.