Skip to main content

Retrieve Build Data via the API

Retrieve Build Data using the v2 and v3 API endpoints

Overview:

Typically, build data will disappear after a set amount of time, based on your organization's plan. But what if I need data that is outside of my organization's plan? The CircleCI v2 API contains endpoints that will help retrieve build data on a regular basis so you can keep records going forward.

Steps to Retrieve Data:

  1. Retrieve a list of pipelines for a given project: Get all pipelines

  2. Retrieve a list of workflows for a given pipeline by the pipeline ID: Get a pipeline’s workflows

  3. Retrieve the workflow data by the workflow ID: Get a workflow by ID

  4. Retrieve the jobs of a workflow by the workflow ID: Get a workflow’s jobs

  5. Store the retrieved pipeline, workflow, and job data in your chosen data stores

  6. Repeat steps as required with the next_page_token response from step 1

To expedite to process, we would recommend performing these steps in a script that can be reused and modified to suit your needs.

Using the v3 API

If you are on CircleCI cloud, you can follow the same steps with the v3 API. A v2 pipeline is a v3 run, and v3 identifies projects, runs, workflows, and jobs by UUID instead of by slug.

  1. Get the project ID: GET /api/v3/projects?filter[org_id]=<org-id>

  2. List the runs for the project: GET /api/v3/runs?filter[project_id]=<project-id>

  3. List the workflows for a run: GET /api/v3/workflows?filter[run_id]=<run-id>

  4. List the jobs for a workflow: GET /api/v3/jobs?filter[workflow_id]=<workflow-id>

  5. Repeat with page.next from each response, sent back as page[cursor], until page.next is null

Send your personal API token as Authorization: Bearer <token>, and use curl -g when a URL contains [ ]. See the API v3 developer's guide for details. API v3 is not available on CircleCI server, so use the v2 steps there.

Limitations:

While this method may allow you to get build data that is outside of your retention period, it will not be able to get all build data for a project that is outside of your plan's retention period. This is why we recommend that users regularly retrieve their build data and store it securely.

Additional Resources:

Related Articles
Did this answer your question?