Can Serial Groups Run Dynamic Config Pipelines in Ascending Pipeline Number Order?

Overview

Serial groups do not guarantee that every dynamic configuration pipeline runs in ascending pipeline-number order without skips.

With dynamic configuration (setup: true plus continuation), setup duration determines when continued jobs join a serial-group queue. If a newer pipeline joins first, an older pipeline from the same project is skipped (not_run) when it later requests that serial group. This is expected pipeline number priority, and the priority cannot be overridden.

To reduce setup-timing inversions, serialize setup with a separate key from the continued deploy job. This is a best-effort workaround, not a strict ordering guarantee. If every pipeline must run in order without skips, queue pipeline triggers outside CircleCI.

Symptoms

Two pipelines start close together.
Both run a setup workflow, then continue into a deploy or similar job with the same serial-group key.
Setup duration differs because of executor startup, checkout, or config-generation work.
The newer pipeline completes setup first and runs the serialized job.
The older pipeline later reaches the serialized job and is skipped (not_run), even though it started earlier.

Example timeline:

Pipeline A (older) starts setup.
Pipeline B (newer) starts setup shortly afterward.
Pipeline B finishes setup first and acquires the deploy serial-group lock.
Pipeline A finishes setup later, requests the same lock, and is skipped.

Why this happens

Serial groups apply at the job level. A job joins its serial-group queue when it becomes ready, not when its pipeline is created.

With dynamic configuration:

Setup jobs usually have no serial-group.
Setup can take different amounts of time per pipeline.
Continued jobs join the serial queue only after continuation.
Whichever pipeline continues first can request the deploy lock first.
Within the same project, an older pipeline is skipped if a newer pipeline already holds or waits on the same serial group.

Putting setup and deploy on the same key does not guarantee order across continuation. The setup lock releases when the setup job finishes; the continued deploy joins the queue afterward.

When the workaround fits

Use the separate setup-key workaround when you want:

Pipeline B's setup to wait when pipeline A's setup reached the setup queue first.
Pipeline A to continue before pipeline B, reducing the chance that setup-duration differences invert deploy queue order.
Deploy jobs to remain serialized while allowing pipeline B's setup to overlap pipeline A's deploy.

This workaround does not make pipeline B wait for pipeline A's entire continued workflow.

Serialize setup with a separate serial-group key

Use one key for setup and a different key for the continued deploy job.

Setup key: << pipeline.project.slug >>/setup-mutex
Deploy key: << pipeline.project.slug >>/deploy-mutex

If the older pipeline joins the setup queue first, the newer setup waits. The older pipeline continues first and can acquire the deploy lock before the newer pipeline reaches deploy.

A setup ──────────────────► A continue/deploy ──────────►
         B setup waits …
                              B setup (may overlap A deploy)
                                 └─► B continue/deploy waits for A continue/deploy

Setup configuration

version: 2.1
setup: true

orbs:
  continuation: circleci/continuation@1

jobs:
  setup:
    executor: continuation/default
    steps:
      - checkout
      - continuation/continue:
          configuration_path: .circleci/continue_config.yml

workflows:
  setup:
    jobs:
      - setup:
          serial-group: << pipeline.project.slug >>/setup-mutex

Continuation configuration

version: 2.1

jobs:
  deploy:
    docker:
      - image: cimg/base:current
    steps:
      - run: echo "deploy"

workflows:
  deploy:
    jobs:
      - deploy:
          serial-group: << pipeline.project.slug >>/deploy-mutex

Limitations

Trigger order does not guarantee queue-entry order.
If the newer pipeline joins the setup queue first, the older setup can itself be skipped.
The workaround does not create an atomic lock from setup through deploy.
Pipeline B's setup can overlap pipeline A's deploy.
Pipeline number priority cannot be overridden. See limitations of serial groups.

Strict ordering without skips

If every pipeline must run in ascending order without skips, serial groups alone are insufficient. Queue pipeline creation in your API caller or webhook handler, and trigger the next pipeline only after the previous pipeline reaches a terminal state.

When skipping an older pipeline is acceptable

If deployments use a linear branch history and each newer commit contains earlier merged changes, an older skipped deployment might not omit any changes. Confirm this against your deployment model, and monitor unexpected not_run serial jobs.

Additional resources