LookML (v1) | Stitch Documentation

LookML is in beta
LookML is currently in beta. The info in this guide is subject to change.

LookML is supported by the Singer community
This integration is powered by Singer's LookML tap. For support, visit the GitHub repo or join the Singer Slack.

LookML integration summary

Stitch’s LookML integration connects to GitHub repositories where your Looker LookML is hosted for your Looker project. Stitch then replicates it using GitHub v3 API to extract LookML components using lkml parser.

Refer to the Schema section for a list of objects available for replication.

LookML feature snapshot

A high-level look at Stitch's LookML (v1) integration, including release status, useful links, and the features supported in Stitch.

STITCH
Release status	Beta	Supported by	Singer Community
Stitch plan	Standard	API availability	Available
Singer GitHub repository	singer-io/tap-lookml
REPLICATION SETTINGS
Anchor Scheduling	Supported	Advanced Scheduling	Supported
Table-level reset	Unsupported	Configurable Replication Methods	Unsupported
DATA SELECTION
Table selection	Supported	Column selection	Supported
Select all	Supported
TRANSPARENCY
Extraction Logs	Supported	Loading Reports	Supported

Connecting LookML

LookML setup requirements

To set up LookML in Stitch, you need:

Access to the GitHub repositories you want to replicate data from. Stitch will only be able to access the same repositories as the user who creates the access token.

Step 1: Create access token

Sign into your GitHub account.
Click the User menu (your icon) > Settings.
Click Developer settings in the navigation on the left side of the page.
Click Personal access tokens.
On the Personal access tokens page, click the Generate a personal access token button. If prompted, enter your password.
In the Description field, enter stitch. This will allow you to easily idenfiy what application is using the token.
In the Select Scopes section, check the repo option:

Note: While these are full permissions, Stitch will only ever read your data. The repo scope is required due to how GitHub structures permissions.
Click the Generate token button.
The new access token will display on the next page. Copy the token before navigating away from the page - GitHub won’t display it again.

Step 2: Add LookML as a Stitch data source

Sign into your Stitch account.
On the Stitch Dashboard page, click the Add Integration button.
Click the LookML icon.
Enter a name for the integration. This is the name that will display on the Stitch Dashboard for the integration; it’ll also be used to create the schema in your destination.

For example, the name “Stitch LookML” would create a schema called stitch_lookml in the destination. Note: Schema names cannot be changed after you save the integration.
In the API Token field, paste the access token you copied from step 1.
In the Git Owner field, enter the owner of the repositories you want to replicate. You can find this info in the repository’s URL. For example: The Git owner of https://github.com/stitch/stitch-repo would be stitch.

Note: While you can replicate data from multiple repositories, this integration only supports replicating from one owner at a time.
In the Git Repositories field, enter the repository or repositories you want to replicate. You can find this info in the repository’s URL. For example: The repository of https://github.com/stitch/stitch-repo would be stitch-repo.

To track multiple repositories, enter them as a comma-delimited list. For example: repo-1, repo-2.

Step 3: Define the historical replication start date

The Sync Historical Data setting defines the starting date for your LookML integration. This means that:

For tables using Key-based Incremental Replication, data equal to or newer than this date will be replicated to your destination.
For tables using Full Table Replication, all data - including records that are older, equal to, or newer than this date - will be replicated to your destination.

Change this setting if you want to replicate data beyond LookML’s default setting of 1 year. For a detailed look at historical replication jobs, check out the Syncing Historical SaaS Data guide.

Step 4: Create a replication schedule

Replication schedules affect the time Extraction begins, not the time to data loaded. Refer to the Replication Scheduling documentation for more information.

In the Replication Frequency section, you’ll create the integration’s replication schedule. An integration’s replication schedule determines how often Stitch runs a replication job, and the time that job begins.

LookML integrations support the following replication scheduling methods:

Replication Frequency
Anchor Scheduling
Advanced Scheduling using Cron (Advanced or Premium plans only)

To keep your row usage low, consider setting the integration to replicate less frequently. See the Understanding and Reducing Your Row Usage guide for tips on reducing your usage.

Step 5: Set objects to replicate

Is an object missing or not replicating? Verify that the object meets the requirements for selection and replication.

The last step is to select the tables and columns you want to replicate. Learn about the available tables for this integration.

Note: If a replication job is currently in progress, new selections won’t be used until the next job starts.

For LookML integrations, you can select:

Individual tables and columns
All tables and columns

Click the tabs to view instructions for each selection method.

In the integration’s Tables to Replicate tab, locate a table you want to replicate.
To track a table, click the checkbox next to the table’s name. A blue checkmark means the table is set to replicate.
To track a column, click the checkbox next to the column’s name. A blue checkmark means the column is set to replicate.
Repeat this process for all the tables and columns you want to replicate.
When finished, click the Finalize Your Selections button at the bottom of the screen to save your selections.

Important: Using the Select All feature will overwrite any previous selections. However, selections aren’t final until Finalize Your Selections is clicked. Clicking Cancel will restore your previous selections. Refer to the Select All guide for more info about this feature.

Click into the integration from the Stitch Dashboard page.
Click the Tables to Replicate tab.
In the list of tables, click the box next to the Table Names column.
In the menu that displays, click Track all Tables and Fields:
Click the Finalize Your Selections button at the bottom of the page to save your data selections.

Initial and historical replication jobs

After you finish setting up LookML, its Sync Status may show as Pending on either the Stitch Dashboard or in the Integration Details page.

For a new integration, a Pending status indicates that Stitch is in the process of scheduling the initial replication job for the integration. This may take some time to complete.

Initial replication jobs with Anchor Scheduling

If using Anchor Scheduling, an initial replication job may not kick off immediately. This depends on the selected Replication Frequency and Anchor Time. Refer to the Anchor Scheduling documentation for more information.

Free historical data loads

The first seven days of replication, beginning when data is first replicated, are free. Rows replicated from the new integration during this time won’t count towards your quota. Stitch offers this as a way of testing new integrations, measuring usage, and ensuring historical data volumes don’t quickly consume your quota.

Replication will continue after the seven days are over. If you’re no longer interested in this source, be sure to pause or delete the integration to prevent unwanted usage.

LookML table reference

Schemas and versioning

Schemas and naming conventions can change from version to version, so we recommend verifying your integration’s version before continuing.

The schema and info displayed below is for version 1 of this integration.

This is the latest version of the LookML integration.

Table and column names in your destination

Depending on your destination, table and column names may not appear as they are outlined below.

For example: Object names are lowercased in Redshift (CusTomERs > customers), while case is maintained in PostgreSQL destinations (CusTomERs > CusTomERs). Refer to the Loading Guide for your destination for more info.

models

View table schema on GitHub

Replication Method :	Full Table	Primary Key :	git_owner : git_repository : path
API endpoint :	Git API Search

The models table contains information about the model file parse items in your GitHub account using the lkml filter.

models table schema

git_owner
STRING

The owner of the GitHub repository.

git_repository
STRING

The GitHub repository.

path
STRING

The URL for the repository.

access_grants
ARRAY

Click to expand access_grants

allowed_values
ARRAY

Click to expand allowed_values

value
STRING

models (table), allowed_values (attribute)

name
STRING

user_attribute
STRING

models (table), access_grants (attribute)

case_sensitive
STRING

connection
STRING

datagroups
ARRAY

Click to expand datagroups

max_cache_age
STRING

name
STRING

sql_trigger
STRING

models (table), datagroups (attribute)

explores
ARRAY

Click to expand explores

access_filters
ARRAY

Click to expand access_filters

field
STRING

user_attribute
STRING

models (table), access_filters (attribute)

always_filter
OBJECT

Click to expand always_filter

filters
ARRAY

Click to expand filters

field
STRING

value
STRING

models (table), filters (attribute)

models (table), always_filter (attribute)

always_join
ARRAY

Click to expand always_join

value
STRING

models (table), always_join (attribute)

cancel_grouping_fields
ARRAY

Click to expand cancel_grouping_fields

value
STRING

models (table), cancel_grouping_fields (attribute)

case_sensitive
STRING

conditionally_filter
OBJECT

Click to expand conditionally_filter

filters
ARRAY

Click to expand filters

field
STRING

value
STRING

models (table), filters (attribute)

unless
ARRAY

Click to expand unless

value
STRING

models (table), unless (attribute)

models (table), conditionally_filter (attribute)

description
STRING

extends
ARRAY

Click to expand extends

value
STRING

models (table), extends (attribute)

extension
STRING

fields
ARRAY

Click to expand fields

value
STRING

models (table), fields (attribute)

from
STRING

group_label
STRING

hidden
STRING

joins
ARRAY

Click to expand joins

fields
ARRAY

Click to expand fields

value
STRING

models (table), fields (attribute)

foreign_key
STRING

from
STRING

name
STRING

outer_only
STRING

relationship
STRING

required_access_grants
ARRAY

Click to expand required_access_grants

value
STRING

models (table), required_access_grants (attribute)

required_joins
ARRAY

Click to expand required_joins

value
STRING

models (table), required_joins (attribute)

sql_foreign_key
STRING

sql_on
STRING

sql_table_name
STRING

sql_where
STRING

type
STRING

view_label
STRING

models (table), joins (attribute)

label
STRING

name
STRING

persist_for
STRING

persist_with
STRING

required_access_grants
ARRAY

Click to expand required_access_grants

value
STRING

models (table), required_access_grants (attribute)

sql_always_having
STRING

sql_always_where
STRING

sql_table_name
STRING

symmetric_aggregates
STRING

view_label
STRING

view_name
STRING

models (table), explores (attribute)

fiscal_month_offset
STRING

includes
ARRAY

Click to expand includes

value
STRING

models (table), includes (attribute)

label
STRING

last_modified
DATE-TIME

map_layers
ARRAY

Click to expand map_layers

extents_json_url
STRING

feature_key
STRING

file
STRING

format
STRING

label
STRING

max_zoom_level
INTEGER

min_zoom_level
INTEGER

name
STRING

projection
STRING

property_key
STRING

property_label_key
STRING

url
STRING

models (table), map_layers (attribute)

named_value_formats
ARRAY

Click to expand named_value_formats

name
STRING

strict_value_format
STRING

value_format
STRING

models (table), named_value_formats (attribute)

persist_for
STRING

persist_with
STRING

sha
STRING

week_start_day
STRING

model_files

View table schema on GitHub

Replication Method :	Key-based Incremental	Replication Key :	last_modified
Primary Key :	git_owner : git_repository : path	API endpoint :	Git API Search

The model_files table contains information about model files in your GitHub about using the Git API Search filename and extension filters for model and lkml.

model_files table schema

git_owner
STRING

The GitHub repository owner.

git_repository
STRING

The GitHub repository.

path
STRING

The URL for the repository.

last_modified
DATE-TIME

The time the file was last modiefied.

content
STRING

download_url
STRING

encoding
STRING

git_url
STRING

html_url
STRING

name
STRING

sha
STRING

size
INTEGER

type
STRING

url
STRING

views

View table schema on GitHub

Replication Method :	Full Table	Primary Key :	git_owner : git_repository : path
API endpoint :	Git API Search

The views table contains information about the view file parse items in your GitHub account using the lkml filter.

views table schema

git_owner
STRING

The GitHub repository owner.

git_repository
STRING

The GitHub repository.

path
STRING

The URL for the repository.

last_modified
DATE-TIME

name
STRING

The name of the repository.

derived_table
OBJECT

Click to expand derived_table

cluster_keys
ARRAY

Click to expand cluster_keys

value
STRING

views (table), cluster_keys (attribute)

create_process
OBJECT

Click to expand create_process

sql_steps
ARRAY

Click to expand sql_steps

value
STRING

views (table), sql_steps (attribute)

views (table), create_process (attribute)

datagroup_trigger
STRING

distribution
STRING

distribution_style
STRING

explore_source
OBJECT

Click to expand explore_source

bind_filters
OBJECT

Click to expand bind_filters

from_field
STRING

to_field
STRING

views (table), bind_filters (attribute)

columns
ARRAY

Click to expand columns

field
STRING

name
STRING

views (table), columns (attribute)

derived_columns
ARRAY

Click to expand derived_columns

name
STRING

sql
STRING

views (table), derived_columns (attribute)

expression_custom_filter
STRING

filters
ARRAY

Click to expand filters

field
STRING

value
STRING

views (table), filters (attribute)

limit
STRING

name
STRING

sort
OBJECT

Click to expand sort

desc
STRING

field
STRING

views (table), sort (attribute)

timezone
STRING

views (table), explore_source (attribute)

indexes
ARRAY

Click to expand indexes

value
STRING

views (table), indexes (attribute)

partition_keys
ARRAY

Click to expand partition_keys

value
STRING

views (table), partition_keys (attribute)

persist_for
STRING

sortkeys
ARRAY

Click to expand sortkeys

value
STRING

views (table), sortkeys (attribute)

sql
STRING

sql_create
STRING

sql_trigger_value
STRING

views (table), derived_table (attribute)

dimension_groups
ARRAY

Click to expand dimension_groups

alias
ARRAY

Click to expand alias

value
STRING

views (table), alias (attribute)

allow_fill
STRING

bypass_suggest_restrictions
STRING

can_filter
STRING

convert_tz
STRING

datatype
STRING

description
STRING

drill_fields
ARRAY

Click to expand drill_fields

value
STRING

views (table), drill_fields (attribute)

fanout_on
STRING

full_suggestions
STRING

group_item_label
STRING

group_label
STRING

hidden
STRING

html
STRING

intervals
ARRAY

Click to expand intervals

value
STRING

views (table), intervals (attribute)

label
STRING

name
STRING

order_by_field
STRING

required_access_grants
ARRAY

Click to expand required_access_grants

value
STRING

views (table), required_access_grants (attribute)

skip_drill_filter
STRING

sql
STRING

sql_end
STRING

sql_start
STRING

suggest_dimension
STRING

suggest_explore
STRING

suggestable
STRING

tags
ARRAY

Click to expand tags

value
STRING

views (table), tags (attribute)

timeframes
ARRAY

Click to expand timeframes

value
STRING

views (table), timeframes (attribute)

type
STRING

view_label
STRING

views (table), dimension_groups (attribute)

dimensions
ARRAY

Click to expand dimensions

actions
ARRAY

Click to expand actions

form_params
ARRAY

Click to expand form_params

default
STRING

label
STRING

name
STRING

options
ARRAY

Click to expand options

label
STRING

name
STRING

views (table), options (attribute)

required
STRING

type
STRING

views (table), form_params (attribute)

form_url
STRING

icon_url
STRING

label
STRING

params
ARRAY

Click to expand params

name
STRING

value
STRING

views (table), params (attribute)

url
STRING

user_attribute_params
ARRAY

Click to expand user_attribute_params

name
STRING

user_attribute
STRING

views (table), user_attribute_params (attribute)

views (table), actions (attribute)

alias
ARRAY

Click to expand alias

value
STRING

views (table), alias (attribute)

alpha_sort
STRING

bypass_suggest_restrictions
STRING

can_filter
STRING

case
OBJECT

Click to expand case

whens
ARRAY

Click to expand whens

label
STRING

sql
STRING

views (table), whens (attribute)

views (table), case (attribute)

case_sensitive
STRING

convert_tz
STRING

datatype
STRING

description
STRING

drill_fields
ARRAY

Click to expand drill_fields

value
STRING

views (table), drill_fields (attribute)

fanout_on
STRING

full_suggestions
STRING

group_item_label
STRING

group_label
STRING

hidden
STRING

html
STRING

label
STRING

label_from_parameter
STRING

links
ARRAY

Click to expand links

icon_url
STRING

label
STRING

url
STRING

views (table), links (attribute)

map_layer_name
STRING

name
STRING

order_by_field
STRING

primary_key
STRING

required_access_grants
ARRAY

Click to expand required_access_grants

value
STRING

views (table), required_access_grants (attribute)

required_fields
ARRAY

Click to expand required_fields

value
STRING

views (table), required_fields (attribute)

skip_drill_filter
STRING

sql
STRING

sql_end
STRING

sql_latitude
STRING

sql_longitude
STRING

sql_start
STRING

style
STRING

suggest_dimension
STRING

suggest_explore
STRING

suggest_persist_for
STRING

suggestable
STRING

suggestions
ARRAY

Click to expand suggestions

value
STRING

views (table), suggestions (attribute)

tags
ARRAY

Click to expand tags

value
STRING

views (table), tags (attribute)

tiers
ARRAY

Click to expand tiers

value
STRING

views (table), tiers (attribute)

type
STRING

value_format
STRING

value_format_name
STRING

view_label
STRING

views (table), dimensions (attribute)

extends
ARRAY

Click to expand extends

value
STRING

views (table), extends (attribute)

extension
STRING

filters
ARRAY

Click to expand filters

alias
ARRAY

Click to expand alias

value
STRING

views (table), alias (attribute)

bypass_suggest_restrictions
STRING

case_sensitive
STRING

convert_tz
STRING

datatype
STRING

default_value
STRING

description
STRING

full_suggestions
STRING

group_item_label
STRING

group_label
STRING

hidden
STRING

label
STRING

name
STRING

required_access_grants
ARRAY

Click to expand required_access_grants

value
STRING

views (table), required_access_grants (attribute)

required_fields
ARRAY

Click to expand required_fields

value
STRING

views (table), required_fields (attribute)

sql
STRING

suggest_dimension
STRING

suggest_explore
STRING

suggest_persist_for
STRING

suggestable
STRING

suggestions
ARRAY

Click to expand suggestions

value
STRING

views (table), suggestions (attribute)

tags
ARRAY

Click to expand tags

value
STRING

views (table), tags (attribute)

type
STRING

view_label
STRING

views (table), filters (attribute)

measures
ARRAY

Click to expand measures

actions
ARRAY

Click to expand actions

form_params
ARRAY

Click to expand form_params

default
STRING

label
STRING

name
STRING

options
ARRAY

Click to expand options

label
STRING

name
STRING

views (table), options (attribute)

required
STRING

type
STRING

views (table), form_params (attribute)

form_url
STRING

icon_url
STRING

label
STRING

params
ARRAY

Click to expand params

name
STRING

value
STRING

views (table), params (attribute)

url
STRING

user_attribute_params
ARRAY

Click to expand user_attribute_params

name
STRING

user_attribute
STRING

views (table), user_attribute_params (attribute)

views (table), actions (attribute)

alias
ARRAY

Click to expand alias

value
STRING

views (table), alias (attribute)

approximate
STRING

approximate_threshold
NUMBER

can_filter
STRING

convert_tz
STRING

datatype
STRING

description
STRING

direction
STRING

drill_fields
ARRAY

Click to expand drill_fields

value
STRING

views (table), drill_fields (attribute)

fanout_on
STRING

filters
ARRAY

Click to expand filters

field
STRING

value
STRING

views (table), filters (attribute)

group_item_label
STRING

group_label
STRING

hidden
STRING

html
STRING

label
STRING

label_from_parameter
STRING

links
ARRAY

Click to expand links

icon_url
STRING

label
STRING

url
STRING

views (table), links (attribute)

list_field
STRING

name
STRING

percentile
NUMBER

precision
INTEGER

required_access_grants
ARRAY

Click to expand required_access_grants

value
STRING

views (table), required_access_grants (attribute)

required_fields
ARRAY

Click to expand required_fields

value
STRING

views (table), required_fields (attribute)

sql
STRING

sql_distinct_key
STRING

suggest_dimension
STRING

suggest_explore
STRING

suggestable
STRING

tags
ARRAY

Click to expand tags

value
STRING

views (table), tags (attribute)

type
STRING

value_format
STRING

value_format_name
STRING

view_label
STRING

views (table), measures (attribute)

parameters
ARRAY

Click to expand parameters

alias
ARRAY

Click to expand alias

value
STRING

views (table), alias (attribute)

allowed_values
ARRAY

Click to expand allowed_values

label
STRING

value
STRING

views (table), allowed_values (attribute)

bypass_suggest_restrictions
STRING

convert_tz
STRING

default_value
STRING

description
STRING

full_suggestions
STRING

hidden
STRING

label
STRING

name
STRING

required_access_grants
ARRAY

Click to expand required_access_grants

value
STRING

views (table), required_access_grants (attribute)

required_fields
ARRAY

Click to expand required_fields

value
STRING

views (table), required_fields (attribute)

suggest_dimension
STRING

suggest_explore
STRING

suggest_persist_for
STRING

suggestable
STRING

suggestions
ARRAY

Click to expand suggestions

value
STRING

views (table), suggestions (attribute)

tags
ARRAY

Click to expand tags

value
STRING

views (table), tags (attribute)

type
STRING

view_label
STRING

views (table), parameters (attribute)

required_access_grants
ARRAY

Click to expand required_access_grants

value
STRING

views (table), required_access_grants (attribute)

sets
ARRAY

Click to expand sets

fields
ARRAY

Click to expand fields

value
STRING

views (table), fields (attribute)

name
STRING

views (table), sets (attribute)

sha
STRING

sql_table_name
STRING

suggestions
STRING

view_label
STRING

view_files

View table schema on GitHub

Replication Method :	Key-based Incremental	Replication Key :	last_modified
Primary Key :	git_owner : git_repository : path	API endpoint :	Git API Search

The view_files table contains information about view files, using the Git API Search filename and extension filters for views and lkml.

view_files table schema

git_owner
STRING

The GitHub repository owner.

git_repository
STRING

The GitHub repository.

path
STRING

The URL for the repository.

last_modified
DATE-TIME

The time the file was last modified.

content
STRING

download_url
STRING

encoding
STRING

git_url
STRING

html_url
STRING

name
STRING

sha
STRING

size
INTEGER

type
STRING

url
STRING

Questions? Feedback?

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.

Related	Troubleshooting
Destination & Integration Compatibility Replication Scheduling Syncing Historical SaaS Data Resetting Replication Keys Nested Data Structures & Row Count Impact	Third-Party Downtime Understanding & Reducing Your Usage Re-Authorizing Integrations Replication Issues