CircleCI integration summary

Stitch’s CircleCI integration replicates data using the CircleCI API v2. Refer to the Schema section for a list of objects available for replication.

CircleCI feature snapshot

A high-level look at Stitch's CircleCI (v1) integration, including release status, useful links, and the features supported in Stitch.

STITCH
Release status

Beta

Supported by

Stitch

Stitch plan

Standard

API availability

Available

Singer GitHub repository

singer-io/tap-circle-ci

REPLICATION SETTINGS
Anchor Scheduling

Supported

Advanced Scheduling

Supported

Table-level reset

Unsupported

Configurable Replication Methods

Unsupported

DATA SELECTION
Table selection

Supported

Column selection

Supported

Select all

Supported

TRANSPARENCY
Extraction Logs

Supported

Loading Reports

Supported

Connecting CircleCI

Step 1: Create CircleCI API token

  1. Log into your CircleCI application.
  2. Go to your User settings.
  3. Click Personal API Tokens.
  4. Click the Create New Token button.
  5. The the Token name field, type in a name you will remember - like Stitch Integration.
  6. Click the Add API Token button.
  7. Copy your new API token and paste it in a safe location, as you will not be able to view it in the application again.

Step 2: Create your project slugs

A project slug is a triplet of the componenets:

  • Project type - which can either be github or bitbucket
  • Organization - the name of your organization on GitHub or Bitbucket
  • Repository - the name of the repository

The project slug takes the form of <project_type>/<org_name>/<repo_name>. For example, the project slug for this CircleCI Singer tap is github/singer-io/circle-ci.

Determine which repositories you would like to include in your Stitch CircleCI integration and list them, separated by spaces, in the project slug format. Keep this list in a safe location.

Step 3: Add CircleCI as a Stitch data source

  1. Sign into your Stitch account.
  2. On the Stitch Dashboard page, click the Add Integration button.

  3. Click the CircleCI icon.

  4. Enter a name for the integration. This is the name that will display on the Stitch Dashboard for the integration; it’ll also be used to create the schema in your destination.

    For example, the name “Stitch CircleCI” would create a schema called stitch_circleci in the destination. Note: Schema names cannot be changed after you save the integration.

  5. In the Project Slugs field, paste the list of project slugs you created in step 2.
  6. In the Token field, paste the API token you created in step 1.

Step 4: Define the historical replication start date

The Sync Historical Data setting defines the starting date for your CircleCI integration. This means that:

  • For tables using Key-based Incremental Replication, data equal to or newer than this date will be replicated to your destination.
  • For tables using Full Table Replication, all data - including records that are older, equal to, or newer than this date - will be replicated to your destination.

Change this setting if you want to replicate data beyond CircleCI’s default setting of 1 year. For a detailed look at historical replication jobs, check out the Syncing Historical SaaS Data guide.

Step 5: Create a replication schedule

In the Replication Frequency section, you’ll create the integration’s replication schedule. An integration’s replication schedule determines how often Stitch runs a replication job, and the time that job begins.

CircleCI integrations support the following replication scheduling methods:

To keep your row usage low, consider setting the integration to replicate less frequently. See the Understanding and Reducing Your Row Usage guide for tips on reducing your usage.

Step 6: Set objects to replicate

The last step is to select the tables and columns you want to replicate. Learn about the available tables for this integration.

Note: If a replication job is currently in progress, new selections won’t be used until the next job starts.

For CircleCI integrations, you can select:

  1. Individual tables and columns

  2. All tables and columns

Click the tabs to view instructions for each selection method.

  1. In the integration’s Tables to Replicate tab, locate a table you want to replicate.
  2. To track a table, click the checkbox next to the table’s name. A blue checkmark means the table is set to replicate.

  3. To track a column, click the checkbox next to the column’s name. A blue checkmark means the column is set to replicate.

  4. Repeat this process for all the tables and columns you want to replicate.
  5. When finished, click the Finalize Your Selections button at the bottom of the screen to save your selections.
  1. Click into the integration from the Stitch Dashboard page.
  2. Click the Tables to Replicate tab.

  3. In the list of tables, click the box next to the Table Names column.
  4. In the menu that displays, click Track all Tables and Fields:

    The Track all Tables and Fields menu in the Tables to Replicate tab

  5. Click the Finalize Your Selections button at the bottom of the page to save your data selections.

Initial and historical replication jobs

After you finish setting up CircleCI, its Sync Status may show as Pending on either the Stitch Dashboard or in the Integration Details page.

For a new integration, a Pending status indicates that Stitch is in the process of scheduling the initial replication job for the integration. This may take some time to complete.

Free historical data loads

The first seven days of replication, beginning when data is first replicated, are free. Rows replicated from the new integration during this time won’t count towards your quota. Stitch offers this as a way of testing new integrations, measuring usage, and ensuring historical data volumes don’t quickly consume your quota.


CircleCI table reference

Replication Method :

Full Table

Primary Key :

id : _workflow_id

API endpoint :

Get a workflow’s jobs

The jobs table contains information about jobs from your CircleCI workflows.

id
STRING

The job ID.

_workflow_id
STRING

The workflow ID.

Reference:

_pipeline_id
STRING

The pipeline ID.

Reference:

approval_request_id
STRING

approved_by
STRING

canceled_by
STRING

dependencies
ARRAY

items
STRING

jobs (table), dependencies (attribute)

job_number
INTEGER

name
STRING

project_slug
STRING

started_at
DATE-TIME

status
STRING

stopped_at
DATE-TIME

type
STRING


Replication Method :

Key-based Incremental

Replication Key :

updated_at

Primary Key :

id

API endpoint :

Get all pipelines

The pipelines table contains information about piplines from your CircleCI projects.

id
STRING

The pipeline ID.

Reference:

updated_at
DATE-TIME

The time the pipeline was updated.

created_at
DATE-TIME

errors
ARRAY

type
STRING

message
STRING

pipelines (table), errors (attribute)

number
INTEGER

project_slug
STRING

state
STRING

trigger
OBJECT

received_at
DATE-TIME

type
STRING

actor
OBJECT

login
STRING

avatar_url
STRING

pipelines (table), actor (attribute)
pipelines (table), trigger (attribute)

trigger_parameters
OBJECT

vcs
OBJECT

origin_repository_url
STRING

target_repository_url
STRING

revision
STRING

provider_name
STRING

commit
OBJECT

body
STRING

subject
STRING

pipelines (table), commit (attribute)

branch
STRING

tag
STRING

pipelines (table), vcs (attribute)

Replication Method :

Key-based Incremental

Replication Key :

created_at

Primary Key :

id

API endpoint :

Get a pipeline’s workflows.

The workflows table contains a list of workflows from your CircleCI projects, sorted by pipeline ID.

id
STRING

The workflow ID.

Reference:

created_at
DATE-TIME

The time the workflow was created.

canceled_by
STRING

errored_by
STRING

name
STRING

pipeline_id
STRING

The pipeline ID.

Reference:

pipeline_number
INTEGER

project_slug
STRING

started_by
STRING

status
STRING

stopped_at
STRING

tag
STRING



Questions? Feedback?

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.