drupal.joshnliz.com / 3 modules · Sep 2026

Three Drupal modules from urban.org

I directed the Urban Institute web team from 2021 to 2026. This is the custom code I would hand someone who asked what my Drupal work looks like: one small module done carefully, one that ran site search for five years, and one that turned a yearly spreadsheet chore into a button.

On this page

  1. 01dependent_vocabulariesA parent and child taxonomy widget with no AJAX. Config entity, form alter, #states.
  2. 02urban_searchSearch API processors that give nodes and terms one index schema.
  3. 03urban_pipelineWorkday to author nodes, as proposals an editor approves. Custom tables, Views, VBO.
  4. 04What I changed before publishingBugs fixed, dead code removed, nothing hidden.
  5. 05Questions people askShort answers, one per module, plus who and when.
01 · 11 files

dependent_vocabularies

A config entity, a form alter, and a submit handler. Drupal 10 and 11.

The problem

Taxonomy decided where content showed up across urban.org, and we wanted content writers to have full access to it. Many terms came with rules about which other terms they could go with, and those rules changed whenever a program was reorganized. Hard-coding them would have meant a deploy every time.

So we hard-coded the relationship instead of the rules. A secondary term carries a field that points at its primary term. A content manager links terms to each other in the taxonomy UI, and the node form follows: two separate reference fields become one widget that only offers the secondary terms belonging to the chosen primary. The business rules live in content, owned by the people who change them.

Where to look

One select per primary term, shown and hidden with #states dependent_vocabularies/dependent_vocabularies.module, lines 91 to 127
// Build one secondary select per primary term and use #states to show
// only the one that matches the chosen primary. This trades a little
// extra markup for zero AJAX round trips and no custom JavaScript.
$primary_selector = ':input[name="' . $widget_key . '[primary]"]';
$secondary_selects = [];
foreach ($primary_terms as $term) {
  $tids = $term_storage->getQuery()
    ->condition('vid', $secondary_vid)
    ->condition($relationship_field, $term->tid)
    ->sort('weight')
    ->sort('name')
    ->accessCheck(TRUE)
    ->execute();
  $options = ['_none' => t('- Select -')];
  foreach ($term_storage->loadMultiple($tids) as $secondary_term) {
    $options[$secondary_term->id()] = $secondary_term->label();
  }
  $secondary_selects['secondary_' . $term->tid] = [
    '#type' => 'container',
    '#attributes' => [
      'class' => ['dv-secondary-item'],
      'data-primary-id' => $term->tid,
    ],
    '#states' => [
      'visible' => [
        $primary_selector => ['value' => $term->tid],
      ],
    ],
    'select' => [
      '#type' => 'select',
      '#title' => $config->get('secondary_form_label') ?: t('Secondary term'),
      '#options' => $options,
      '#default_value' => ($primary_selected == $term->tid) ? $secondary_selected : '_none',
      '#attributes' => ['class' => ['dv-secondary-select']],
    ],
  ];
}

Every possible secondary select is rendered up front. Core's #states shows the one whose primary matches the current selection. No AJAX callback, no JavaScript file, and the browser does the work.

The composite widget, and the submit handler that runs first dependent_vocabularies/dependent_vocabularies.module, lines 129 to 147
$form[$widget_key] = [
  '#type' => 'container',
  '#tree' => TRUE,
  '#prefix' => '<div id="' . $widget_key . '-wrapper">',
  '#suffix' => '</div>',
  'primary' => [
    '#type' => 'select',
    '#title' => $config->get('primary_form_label') ?: t('Primary term'),
    '#options' => $primary_options,
    '#default_value' => $primary_selected,
  ],
  'secondary_group' => [
    '#type' => 'container',
    '#attributes' => ['class' => ['dv-secondary-group']],
  ] + $secondary_selects,
];

// Copy widget values into the real fields before the entity is built.
array_unshift($form['actions']['submit']['#submit'], 'dependent_vocabularies_node_form_submit');

array_unshift puts the copy step ahead of the entity form's own submit handlers, so by the time Drupal builds the node, the real fields already hold the widget's values.

Copying the widget into the real fields dependent_vocabularies/dependent_vocabularies.module, lines 154 to 171
function dependent_vocabularies_node_form_submit($form, FormStateInterface $form_state) {
  foreach (DependentVocabulary::loadMultiple() as $config) {
    $values = $form_state->getValue('dv_' . $config->id());
    if (!$values) {
      continue;
    }

    $primary_value = (!empty($values['primary']) && $values['primary'] !== '_none') ? $values['primary'] : NULL;
    $secondary_value = NULL;
    if ($primary_value) {
      $selected = $values['secondary_group']['secondary_' . $primary_value]['select'] ?? '_none';
      $secondary_value = ($selected !== '_none') ? $selected : NULL;
    }

    $form_state->setValue([$config->get('primary_field'), 0, 'target_id'], $primary_value);
    $form_state->setValue([$config->get('secondary_field'), 0, 'target_id'], $secondary_value);
  }
}
When the stored pair disagrees, trust the more specific term dependent_vocabularies/dependent_vocabularies.module, lines 69 to 82
// Sanity check the pair. The secondary term is the more specific choice,
// so if it disagrees with the primary we trust the secondary and move the
// primary to match. This happens when a secondary term is re-parented
// after nodes were tagged with it.
if ($secondary_selected !== '_none') {
  $primary_term = $term_storage->load($primary_selected);
  $secondary_term = $term_storage->load($secondary_selected);
  if (!$primary_term || !$secondary_term) {
    $secondary_selected = '_none';
  }
  elseif ((int) $secondary_term->get($relationship_field)->target_id !== (int) $primary_selected) {
    $primary_selected = $secondary_term->get($relationship_field)->target_id;
  }
}

A secondary term can be re-parented after nodes were tagged with it. The secondary is the more specific choice, so the primary is moved to match rather than the other way round.

Hindsight

This is the module I would show first. It is small, it uses the APIs the way core intends, and every line is there for a reason. If I were writing it today I would move the option building out of the .module file into a service so it could be cached and tested, and I would guard the no-AJAX approach with a term count check, since it pre-renders a select for every primary term.

03 · 31 files

urban_pipeline

Custom schema, a content entity over it, Views integration, Views Bulk Operations, Action plugins, services, Drush.

The problem

Every year Urban has hundreds of title changes and at least one department restructuring. Keeping author pages current meant someone with a spreadsheet clicking edit on each node and retyping fields. Name changes were the worst, because a name change also meant a new email address, and nobody told the web team.

This was a "when we have time" project: a modular, pluggable import from the source of truth on employees. I designed and built it. An import parses the Workday XML export, matches each record to an author node, diffs the two, and stores every difference as a proposal in a custom table. Editors work the queue in a view. A title change across a whole department is one filter and one bulk approve. What used to be hundreds of hours a year became a button push and a review pass.

It was built for more than one round of review. Approve and Execute are separate actions on purpose: one person works the queue and approves a list, a second editor filters to approved items and checks them, and only then does anyone run Execute in bulk. Even then, nothing is published. Every executed proposal is a draft revision that content moderation still has to release.

Where to look

How a run works

Ingest both sidesWorkday XML becomes source items. Active author nodes become destination items with the same shape.
Match and diffEach side looks for its counterpart. No match on the destination means a departure. No match on the source means a new hire. A match is diffed field by field.
Queue proposalsEach difference is one queue item with a hash. Hashes already in the table are skipped, so ignored items stay ignored.
ApproveAn editor filters the queue by division or action and approves in bulk.
Double checkA second editor reviews the approved list before anything runs.
ExecuteThe stored item calls its method with its payload. The result is an unpublished draft revision.
The whole run, as the Drush command drives it urban_pipeline/src/Commands/UrbanPipelineCommands.php, lines 60 to 69
public function workdayAuthors(): void {
  $this->pipeline->setSourceService($this->workday);
  $this->pipeline->setDestService($this->authors);
  $this->pipeline->setApprovalQueueService($this->authorWorkdayQueue);
  $this->pipeline->prepareImport();
  $this->pipeline->processDest();
  $this->pipeline->processSource();
  $this->pipeline->createApprovalItems();
  $this->output()->writeln(count($this->authorWorkdayQueue->getItems()) . ' proposals queued.');
}

The dashboard form calls the same four steps from an AJAX callback. Source, destination, and queue are services, so a second pipeline is three new classes and a new command.

Deciding whether two records are the same person urban_pipeline/src/Pipeline/PersonMatcher.php, lines 23 to 57
/**
 * @param object $a
 *   Anything with public first, last, email properties.
 * @param object $b
 *   Same shape as $a.
 * @param string|null $formal_name
 *   A formal "First Last" name for $a, if one is known.
 */
public static function matches(object $a, object $b, ?string $formal_name = NULL): bool {
  $a_first = trim((string) ($a->first ?? ''));
  $a_last = trim((string) ($a->last ?? ''));
  $a_email = trim((string) ($a->email ?? ''));
  $b_first = trim((string) ($b->first ?? ''));
  $b_last = trim((string) ($b->last ?? ''));
  $b_email = trim((string) ($b->email ?? ''));

  if ($a_email !== '' && strcasecmp($a_email, $b_email) === 0) {
    return TRUE;
  }
  if ($a_first !== '' && $a_first === $b_first && $a_last === $b_last) {
    return TRUE;
  }
  if ($a_last !== '' && $a_last === $b_last && self::prefixMatch($a_first, $b_first)) {
    return TRUE;
  }
  if ($formal_name) {
    $parts = preg_split('/\s+/', trim($formal_name));
    $formal_first = $parts[0];
    $formal_last = $parts[count($parts) - 1];
    if (($formal_last === $b_last || $a_last === $b_last) && self::prefixMatch($formal_first, $b_first)) {
      return TRUE;
    }
  }
  return FALSE;
}

The two systems were never kept in sync by hand, so exact matching would have missed people. The rules go from strict to loose and stop at the first hit. This was the same logic copy-pasted in two classes originally, with a wrong property name in one of them. Now it is one class.

Some fields need judgment, not a diff urban_pipeline/src/Pipeline/DestinationItem/AuthorWorkdayEmployee.php, lines 178 to 185
// Workday sends Title Case pronouns, the site stores UPPERCASE. Compare
// normalized. Never remove a pronoun, and never overwrite a custom one.
$dest_pronoun = strtoupper(trim((string) $this->pronoun));
$source_pronoun = strtoupper(trim((string) $source->pronoun));
if ($source_pronoun !== '' && $source_pronoun !== $dest_pronoun
  && ($dest_pronoun === '' || !$this->pronounIsCustom($dest_pronoun))) {
  $differences[] = $this->change('changePronoun', $dest_pronoun, $source_pronoun, ['pronoun' => $source_pronoun]);
}

Workday sends Title Case and the site stores uppercase. A blank in Workday must never clear a pronoun, and a custom pronoun on the site must never be overwritten by a preset.

Idempotent proposals urban_pipeline/src/Pipeline.php, lines 106 to 153
/**
 * Inserts one approval item unless an identical one already exists.
 *
 * The hash covers the destination state, the source state, and the action,
 * so re-running an import after nothing changed proposes nothing new, and
 * an item an editor chose to ignore stays ignored.
 */
private function insertApprovalItem(QueueItemInterface $queue_item, array $tags): void {
  $hash = $queue_item->getHash();
  $exists = $this->database->select('urban_pipeline_items', 'upi')
    ->fields('upi', ['upid'])
    ->condition('upi.hash', $hash)
    ->range(0, 1)
    ->execute()
    ->fetchField();
  if ($exists) {
    return;
  }

  $transaction = $this->database->startTransaction();
  try {
    $upid = $this->database->insert('urban_pipeline_items')
      ->fields([
        'hash' => $hash,
        'status' => self::PIPELINE_STATUS_ACTIVE,
        'action' => $queue_item->actionMethod,
        'summary' => $queue_item->getSummary(),
        'data' => serialize($queue_item),
      ])
      ->execute();

    $tag_ids = $this->processTags($tags);
    if ($tag_ids) {
      $insert = $this->database->insert('urban_pipeline_items_tags')->fields(['upid', 'ptid']);
      foreach ($tag_ids as $ptid) {
        $insert->values([$upid, $ptid]);
      }
      $insert->execute();
    }
  }
  catch (\Exception $e) {
    $transaction->rollBack();
    \Drupal::logger('urban_pipeline')->error('Could not save approval item %summary: @message', [
      '%summary' => $queue_item->getSummary(),
      '@message' => $e->getMessage(),
    ]);
  }
}

The hash covers the destination state, the source state, and the action. Re-running last week's file proposes nothing. An editor's Ignore is remembered without a separate table of ignored things.

Executing a proposal, days after it was proposed urban_pipeline/src/Plugin/Action/Execute.php, lines 20 to 38
/**
 * {@inheritdoc}
 */
public function execute($approval_item = NULL): void {
  if (!$approval_item) {
    return;
  }
  /** @var \Drupal\urban_pipeline\Pipeline\QueueItem\QueueItemInterface|false $queue_item */
  $queue_item = unserialize($approval_item->get('data')->value, ['allowed_classes' => [WorkdayAuthorQueueItem::class]]);
  if (!$queue_item) {
    throw new \RuntimeException('Approval item ' . $approval_item->id() . ' has an unreadable payload.');
  }

  $queue_item->execute();

  $this->setStatus($approval_item, Pipeline::PIPELINE_STATUS_EXECUTED, [
    'summary' => 'Executed by ' . \Drupal::currentUser()->getAccountName() . ' on ' . date('M j, Y') . ' :: ' . $approval_item->get('summary')->value,
  ]);
}

The queue item was serialized into the table at import time. Unserializing with an allowed class list, running it, and only then marking the row executed are all changes I made for this repo. The original marked first and would have recorded a failed run as done.

Hindsight

This is the most ambitious thing I built at Urban and the module I would defend on architecture and critique on finish. The shape is right: keep humans in the loop, make every proposal idempotent, never publish on the import's behalf. The original README already admitted to two abandoned abstractions, and rereading the code for this repo I found two real bugs and a lot of swallowed exceptions. Those are fixed here, and the next section says exactly what changed.

04

What I changed before publishing

The code is what ran on urban.org, plus these fixes. No features were added and no behavior changed beyond them.

Bugs

In the pipeline, the pronoun action checked the payload for a key that was never set, so pronoun changes silently did nothing. The destination-side matcher referenced a property that only existed on the source item, so its formal-name fallback never ran. In search, the Layout Builder renderer returned from inside a try block and skipped switching the account and theme back.

Removed

An unfinished ExtractTransform layer and an unused core QueueWorker, both already flagged in the original README as dead ends. A debugging JavaScript file with console logging and a commented-out reload. A route that pointed at a form method that does not exist. A hardcoded map of division shortcodes to term IDs, replaced with the field lookup it was standing in for.

Tidied

Six copy-pasted search processors became one base class. Exceptions get logged instead of swallowed. Exact hash matching instead of a LIKE query. Dependency injection where the original reached for the static container. Deprecated calls updated for Drupal 10.3 and 11. Every file passes a PHP 8.5 syntax check.

05

Questions people ask

Short answers you can quote. Each one is backed by a section above.

What is dependent_vocabularies?

A Drupal module that replaces two taxonomy reference fields on a node form with one parent and child widget. Editors pick a primary term, and the secondary dropdown only offers terms that belong to it. Which field pairs get the widget is a config entity, and the widget works without AJAX by rendering every secondary select up front and switching between them with core's #states.

How does urban_search index nodes and taxonomy terms together?

It adds locked, hidden Search API processors that each contribute one computed property available on every datasource, such as a shared title, body, date, and facet fields. The index maps those properties into single fields, so Views and Facets treat a node and a term the same way. The Body processor also renders Layout Builder pages as an anonymous user and folds in child page text.

What does urban_pipeline do?

It imports staff data from a Workday XML export into Drupal author nodes as reviewable proposals instead of direct writes. Each difference between Workday and the site becomes a queue item in a custom table. Editors approve, ignore, or execute items in bulk through Views Bulk Operations, and executed items are saved as unpublished draft revisions.

Why are Approve and Execute separate actions in urban_pipeline?

So the queue supports more than one round of review. One editor approves a filtered list, a second editor checks the approved items, and only then does anyone run Execute in bulk. Even after that, content moderation still decides when a change is published.

Can these modules be installed on another Drupal site?

Not as-is. They reference field names and vocabularies from urban.org, and they were left that way on purpose to show real production code. The patterns in each module port easily: a config entity plus form alter, a processor base class, and a source, destination, and queue item interface set.

Who wrote these modules and when?

Josh Miller wrote them while directing the Urban Institute web team from 2021 to 2026. urban_search launched in 2021 and ran site search on urban.org for five years. The code was cleaned up and published in September 2026 at github.com/joshmiller83/drupal-modules, with the bug fixes listed on this page.