LookML integration summary

Stitch’s LookML integration connects to GitHub repositories where your Looker LookML is hosted for your Looker project. Stitch then replicates it using GitHub v3 API to extract LookML components using lkml parser.

Refer to the Schema section for a list of objects available for replication.

LookML feature snapshot

A high-level look at Stitch's LookML (v1) integration, including release status, useful links, and the features supported in Stitch.

STITCH
Release status

Beta

Supported by

Singer Community

Stitch plan

Standard

API availability

Available

Singer GitHub repository

singer-io/tap-lookml

REPLICATION SETTINGS
Anchor Scheduling

Supported

Advanced Scheduling

Supported

Table-level reset

Unsupported

Configurable Replication Methods

Unsupported

DATA SELECTION
Table selection

Supported

Column selection

Supported

Select all

Supported

TRANSPARENCY
Extraction Logs

Supported

Loading Reports

Supported

Connecting LookML

LookML setup requirements

To set up LookML in Stitch, you need:

  • Access to the GitHub repositories you want to replicate data from. Stitch will only be able to access the same repositories as the user who creates the access token.


Step 1: Create access token

  1. Sign into your GitHub account.
  2. Click the User menu (your icon) > Settings.
  3. Click Developer settings in the navigation on the left side of the page.
  4. Click Personal access tokens.
  5. On the Personal access tokens page, click the Generate a personal access token button. If prompted, enter your password.
  6. In the Description field, enter stitch. This will allow you to easily idenfiy what application is using the token.
  7. In the Select Scopes section, check the repo option:

    Highlighted repo scopes on the GitHub Personal Access Tokens page

    Note: While these are full permissions, Stitch will only ever read your data. The repo scope is required due to how GitHub structures permissions.

  8. Click the Generate token button.
  9. The new access token will display on the next page. Copy the token before navigating away from the page - GitHub won’t display it again.

Step 2: Add LookML as a Stitch data source

  1. Sign into your Stitch account.
  2. On the Stitch Dashboard page, click the Add Integration button.

  3. Click the LookML icon.

  4. Enter a name for the integration. This is the name that will display on the Stitch Dashboard for the integration; it’ll also be used to create the schema in your destination.

    For example, the name “Stitch LookML” would create a schema called stitch_lookml in the destination. Note: Schema names cannot be changed after you save the integration.

  5. In the API Token field, paste the access token you copied from step 1.
  6. In the Git Owner field, enter the owner of the repositories you want to replicate. You can find this info in the repository’s URL. For example: The Git owner of https://github.com/stitch/stitch-repo would be stitch.

    Note: While you can replicate data from multiple repositories, this integration only supports replicating from one owner at a time.

  7. In the Git Repositories field, enter the repository or repositories you want to replicate. You can find this info in the repository’s URL. For example: The repository of https://github.com/stitch/stitch-repo would be stitch-repo.

    To track multiple repositories, enter them as a comma-delimited list. For example: repo-1, repo-2.

Step 3: Define the historical replication start date

The Sync Historical Data setting defines the starting date for your LookML integration. This means that:

  • For tables using Key-based Incremental Replication, data equal to or newer than this date will be replicated to your destination.
  • For tables using Full Table Replication, all data - including records that are older, equal to, or newer than this date - will be replicated to your destination.

Change this setting if you want to replicate data beyond LookML’s default setting of 1 year. For a detailed look at historical replication jobs, check out the Syncing Historical SaaS Data guide.

Step 4: Create a replication schedule

In the Replication Frequency section, you’ll create the integration’s replication schedule. An integration’s replication schedule determines how often Stitch runs a replication job, and the time that job begins.

LookML integrations support the following replication scheduling methods:

To keep your row usage low, consider setting the integration to replicate less frequently. See the Understanding and Reducing Your Row Usage guide for tips on reducing your usage.

Step 5: Set objects to replicate

The last step is to select the tables and columns you want to replicate. Learn about the available tables for this integration.

Note: If a replication job is currently in progress, new selections won’t be used until the next job starts.

For LookML integrations, you can select:

  1. Individual tables and columns

  2. All tables and columns

Click the tabs to view instructions for each selection method.

  1. In the integration’s Tables to Replicate tab, locate a table you want to replicate.
  2. To track a table, click the checkbox next to the table’s name. A blue checkmark means the table is set to replicate.

  3. To track a column, click the checkbox next to the column’s name. A blue checkmark means the column is set to replicate.

  4. Repeat this process for all the tables and columns you want to replicate.
  5. When finished, click the Finalize Your Selections button at the bottom of the screen to save your selections.
  1. Click into the integration from the Stitch Dashboard page.
  2. Click the Tables to Replicate tab.

  3. In the list of tables, click the box next to the Table Names column.
  4. In the menu that displays, click Track all Tables and Fields:

    The Track all Tables and Fields menu in the Tables to Replicate tab

  5. Click the Finalize Your Selections button at the bottom of the page to save your data selections.

Initial and historical replication jobs

After you finish setting up LookML, its Sync Status may show as Pending on either the Stitch Dashboard or in the Integration Details page.

For a new integration, a Pending status indicates that Stitch is in the process of scheduling the initial replication job for the integration. This may take some time to complete.

Free historical data loads

The first seven days of replication, beginning when data is first replicated, are free. Rows replicated from the new integration during this time won’t count towards your quota. Stitch offers this as a way of testing new integrations, measuring usage, and ensuring historical data volumes don’t quickly consume your quota.


LookML table reference

Replication Method :

Key-based Incremental

Replication Key :

last_modified

Primary Key :

git_owner : git_repository : path

API endpoint :

Git API Search

The model_files table contains information about model files in your GitHub about using the Git API Search filename and extension filters for model and lkml.

git_owner
STRING

The GitHub repository owner.

git_repository
STRING

The GitHub repository.

path
STRING

The URL for the repository.

last_modified
DATE-TIME

The time the file was last modiefied.

content
STRING

download_url
STRING

encoding
STRING

git_url
STRING

html_url
STRING

name
STRING

sha
STRING

size
INTEGER

type
STRING

url
STRING


Replication Method :

Full Table

Primary Key :

git_owner : git_repository : path

API endpoint :

Git API Search

The models table contains information about the model file parse items in your GitHub account using the lkml filter.

git_owner
STRING

The owner of the GitHub repository.

git_repository
STRING

The GitHub repository.

path
STRING

The URL for the repository.

access_grants
ARRAY

allowed_values
ARRAY

value
STRING

models (table), allowed_values (attribute)

name
STRING

user_attribute
STRING

models (table), access_grants (attribute)

case_sensitive
STRING

connection
STRING

datagroups
ARRAY

max_cache_age
STRING

name
STRING

sql_trigger
STRING

models (table), datagroups (attribute)

explores
ARRAY

access_filters
ARRAY

field
STRING

user_attribute
STRING

models (table), access_filters (attribute)

always_filter
OBJECT

filters
ARRAY

field
STRING

value
STRING

models (table), filters (attribute)
models (table), always_filter (attribute)

always_join
ARRAY

value
STRING

models (table), always_join (attribute)

cancel_grouping_fields
ARRAY

value
STRING

models (table), cancel_grouping_fields (attribute)

case_sensitive
STRING

conditionally_filter
OBJECT

filters
ARRAY

field
STRING

value
STRING

models (table), filters (attribute)

unless
ARRAY

value
STRING

models (table), unless (attribute)
models (table), conditionally_filter (attribute)

description
STRING

extends
ARRAY

value
STRING

models (table), extends (attribute)

extension
STRING

fields
ARRAY

value
STRING

models (table), fields (attribute)

from
STRING

group_label
STRING

hidden
STRING

joins
ARRAY

fields
ARRAY

value
STRING

models (table), fields (attribute)

foreign_key
STRING

from
STRING

name
STRING

outer_only
STRING

relationship
STRING

required_access_grants
ARRAY

value
STRING

models (table), required_access_grants (attribute)

required_joins
ARRAY

value
STRING

models (table), required_joins (attribute)

sql_foreign_key
STRING

sql_on
STRING

sql_table_name
STRING

sql_where
STRING

type
STRING

view_label
STRING

models (table), joins (attribute)

label
STRING

name
STRING

persist_for
STRING

persist_with
STRING

required_access_grants
ARRAY

value
STRING

models (table), required_access_grants (attribute)

sql_always_having
STRING

sql_always_where
STRING

sql_table_name
STRING

symmetric_aggregates
STRING

view_label
STRING

view_name
STRING

models (table), explores (attribute)

fiscal_month_offset
STRING

includes
ARRAY

value
STRING

models (table), includes (attribute)

label
STRING

last_modified
DATE-TIME

map_layers
ARRAY

extents_json_url
STRING

feature_key
STRING

file
STRING

format
STRING

label
STRING

max_zoom_level
INTEGER

min_zoom_level
INTEGER

name
STRING

projection
STRING

property_key
STRING

property_label_key
STRING

url
STRING

models (table), map_layers (attribute)

named_value_formats
ARRAY

name
STRING

strict_value_format
STRING

value_format
STRING

models (table), named_value_formats (attribute)

persist_for
STRING

persist_with
STRING

sha
STRING

week_start_day
STRING


Replication Method :

Key-based Incremental

Replication Key :

last_modified

Primary Key :

git_owner : git_repository : path

API endpoint :

Git API Search

The view_files table contains information about view files, using the Git API Search filename and extension filters for views and lkml.

git_owner
STRING

The GitHub repository owner.

git_repository
STRING

The GitHub repository.

path
STRING

The URL for the repository.

last_modified
DATE-TIME

The time the file was last modified.

content
STRING

download_url
STRING

encoding
STRING

git_url
STRING

html_url
STRING

name
STRING

sha
STRING

size
INTEGER

type
STRING

url
STRING


Replication Method :

Full Table

Primary Key :

git_owner : git_repository : path

API endpoint :

Git API Search

The views table contains information about the view file parse items in your GitHub account using the lkml filter.

git_owner
STRING

The GitHub repository owner.

git_repository
STRING

The GitHub repository.

path
STRING

The URL for the repository.

last_modified
DATE-TIME

name
STRING

The name of the repository.

derived_table
OBJECT

cluster_keys
ARRAY

value
STRING

views (table), cluster_keys (attribute)

create_process
OBJECT

sql_steps
ARRAY

value
STRING

views (table), sql_steps (attribute)
views (table), create_process (attribute)

datagroup_trigger
STRING

distribution
STRING

distribution_style
STRING

explore_source
OBJECT

bind_filters
OBJECT

from_field
STRING

to_field
STRING

views (table), bind_filters (attribute)

columns
ARRAY

field
STRING

name
STRING

views (table), columns (attribute)

derived_columns
ARRAY

name
STRING

sql
STRING

views (table), derived_columns (attribute)

expression_custom_filter
STRING

filters
ARRAY

field
STRING

value
STRING

views (table), filters (attribute)

limit
STRING

name
STRING

sort
OBJECT

desc
STRING

field
STRING

views (table), sort (attribute)

timezone
STRING

views (table), explore_source (attribute)

indexes
ARRAY

value
STRING

views (table), indexes (attribute)

partition_keys
ARRAY

value
STRING

views (table), partition_keys (attribute)

persist_for
STRING

sortkeys
ARRAY

value
STRING

views (table), sortkeys (attribute)

sql
STRING

sql_create
STRING

sql_trigger_value
STRING

views (table), derived_table (attribute)

dimension_groups
ARRAY

alias
ARRAY

value
STRING

views (table), alias (attribute)

allow_fill
STRING

bypass_suggest_restrictions
STRING

can_filter
STRING

convert_tz
STRING

datatype
STRING

description
STRING

drill_fields
ARRAY

value
STRING

views (table), drill_fields (attribute)

fanout_on
STRING

full_suggestions
STRING

group_item_label
STRING

group_label
STRING

hidden
STRING

html
STRING

intervals
ARRAY

value
STRING

views (table), intervals (attribute)

label
STRING

name
STRING

order_by_field
STRING

required_access_grants
ARRAY

value
STRING

views (table), required_access_grants (attribute)

skip_drill_filter
STRING

sql
STRING

sql_end
STRING

sql_start
STRING

suggest_dimension
STRING

suggest_explore
STRING

suggestable
STRING

tags
ARRAY

value
STRING

views (table), tags (attribute)

timeframes
ARRAY

value
STRING

views (table), timeframes (attribute)

type
STRING

view_label
STRING

views (table), dimension_groups (attribute)

dimensions
ARRAY

actions
ARRAY

form_params
ARRAY

default
STRING

label
STRING

name
STRING

options
ARRAY

label
STRING

name
STRING

views (table), options (attribute)

required
STRING

type
STRING

views (table), form_params (attribute)

form_url
STRING

icon_url
STRING

label
STRING

params
ARRAY

name
STRING

value
STRING

views (table), params (attribute)

url
STRING

user_attribute_params
ARRAY

name
STRING

user_attribute
STRING

views (table), user_attribute_params (attribute)
views (table), actions (attribute)

alias
ARRAY

value
STRING

views (table), alias (attribute)

alpha_sort
STRING

bypass_suggest_restrictions
STRING

can_filter
STRING

case
OBJECT

whens
ARRAY

label
STRING

sql
STRING

views (table), whens (attribute)
views (table), case (attribute)

case_sensitive
STRING

convert_tz
STRING

datatype
STRING

description
STRING

drill_fields
ARRAY

value
STRING

views (table), drill_fields (attribute)

fanout_on
STRING

full_suggestions
STRING

group_item_label
STRING

group_label
STRING

hidden
STRING

html
STRING

label
STRING

label_from_parameter
STRING

links
ARRAY

map_layer_name
STRING

name
STRING

order_by_field
STRING

primary_key
STRING

required_access_grants
ARRAY

value
STRING

views (table), required_access_grants (attribute)

required_fields
ARRAY

value
STRING

views (table), required_fields (attribute)

skip_drill_filter
STRING

sql
STRING

sql_end
STRING

sql_latitude
STRING

sql_longitude
STRING

sql_start
STRING

style
STRING

suggest_dimension
STRING

suggest_explore
STRING

suggest_persist_for
STRING

suggestable
STRING

suggestions
ARRAY

value
STRING

views (table), suggestions (attribute)

tags
ARRAY

value
STRING

views (table), tags (attribute)

tiers
ARRAY

value
STRING

views (table), tiers (attribute)

type
STRING

value_format
STRING

value_format_name
STRING

view_label
STRING

views (table), dimensions (attribute)

extends
ARRAY

value
STRING

views (table), extends (attribute)

extension
STRING

filters
ARRAY

alias
ARRAY

value
STRING

views (table), alias (attribute)

bypass_suggest_restrictions
STRING

case_sensitive
STRING

convert_tz
STRING

datatype
STRING

default_value
STRING

description
STRING

full_suggestions
STRING

group_item_label
STRING

group_label
STRING

hidden
STRING

label
STRING

name
STRING

required_access_grants
ARRAY

value
STRING

views (table), required_access_grants (attribute)

required_fields
ARRAY

value
STRING

views (table), required_fields (attribute)

sql
STRING

suggest_dimension
STRING

suggest_explore
STRING

suggest_persist_for
STRING

suggestable
STRING

suggestions
ARRAY

value
STRING

views (table), suggestions (attribute)

tags
ARRAY

value
STRING

views (table), tags (attribute)

type
STRING

view_label
STRING

views (table), filters (attribute)

measures
ARRAY

actions
ARRAY

form_params
ARRAY

default
STRING

label
STRING

name
STRING

options
ARRAY

label
STRING

name
STRING

views (table), options (attribute)

required
STRING

type
STRING

views (table), form_params (attribute)

form_url
STRING

icon_url
STRING

label
STRING

params
ARRAY

name
STRING

value
STRING

views (table), params (attribute)

url
STRING

user_attribute_params
ARRAY

name
STRING

user_attribute
STRING

views (table), user_attribute_params (attribute)
views (table), actions (attribute)

alias
ARRAY

value
STRING

views (table), alias (attribute)

approximate
STRING

approximate_threshold
NUMBER

can_filter
STRING

convert_tz
STRING

datatype
STRING

description
STRING

direction
STRING

drill_fields
ARRAY

value
STRING

views (table), drill_fields (attribute)

fanout_on
STRING

filters
ARRAY

field
STRING

value
STRING

views (table), filters (attribute)

group_item_label
STRING

group_label
STRING

hidden
STRING

html
STRING

label
STRING

label_from_parameter
STRING

links
ARRAY

list_field
STRING

name
STRING

percentile
NUMBER

precision
INTEGER

required_access_grants
ARRAY

value
STRING

views (table), required_access_grants (attribute)

required_fields
ARRAY

value
STRING

views (table), required_fields (attribute)

sql
STRING

sql_distinct_key
STRING

suggest_dimension
STRING

suggest_explore
STRING

suggestable
STRING

tags
ARRAY

value
STRING

views (table), tags (attribute)

type
STRING

value_format
STRING

value_format_name
STRING

view_label
STRING

views (table), measures (attribute)

parameters
ARRAY

alias
ARRAY

value
STRING

views (table), alias (attribute)

allowed_values
ARRAY

label
STRING

value
STRING

views (table), allowed_values (attribute)

bypass_suggest_restrictions
STRING

convert_tz
STRING

default_value
STRING

description
STRING

full_suggestions
STRING

hidden
STRING

label
STRING

name
STRING

required_access_grants
ARRAY

value
STRING

views (table), required_access_grants (attribute)

required_fields
ARRAY

value
STRING

views (table), required_fields (attribute)

suggest_dimension
STRING

suggest_explore
STRING

suggest_persist_for
STRING

suggestable
STRING

suggestions
ARRAY

value
STRING

views (table), suggestions (attribute)

tags
ARRAY

value
STRING

views (table), tags (attribute)

type
STRING

view_label
STRING

views (table), parameters (attribute)

required_access_grants
ARRAY

value
STRING

views (table), required_access_grants (attribute)

sets
ARRAY

fields
ARRAY

value
STRING

views (table), fields (attribute)

name
STRING

views (table), sets (attribute)

sha
STRING

sql_table_name
STRING

suggestions
STRING

view_label
STRING



Questions? Feedback?

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.