LookML integration summary

Stitch’s LookML integration connects to GitHub repositories where your Looker LookML is hosted for your Looker project. Stitch then replicates it using GitHub v3 API to extract LookML components using lkml parser.

Refer to the Schema section for a list of objects available for replication.

LookML feature snapshot

A high-level look at Stitch's LookML (v1) integration, including release status, useful links, and the features supported in Stitch.

STITCH
Release status

Beta

Supported by

Singer Community

Stitch plan

Standard

API availability

Available

Singer GitHub repository

singer-io/tap-lookml

REPLICATION SETTINGS
Anchor Scheduling

Supported

Advanced Scheduling

Supported

Table-level reset

Unsupported

Configurable Replication Methods

Unsupported

DATA SELECTION
Table selection

Supported

Column selection

Supported

Select all

Supported

TRANSPARENCY
Extraction Logs

Supported

Loading Reports

Supported

Connecting LookML

LookML setup requirements

To set up LookML in Stitch, you need:

  • Access to the GitHub repositories you want to replicate data from. Stitch will only be able to access the same repositories as the user who creates the access token.


Step 1: Create access token

  1. Sign into your GitHub account.
  2. Click the User menu (your icon) > Settings.
  3. Click Developer settings in the navigation on the left side of the page.
  4. Click Personal access tokens.
  5. On the Personal access tokens page, click the Generate a personal access token button. If prompted, enter your password.
  6. In the Description field, enter stitch. This will allow you to easily idenfiy what application is using the token.
  7. In the Select Scopes section, check the repo option:

    Highlighted repo scopes on the GitHub Personal Access Tokens page

    Note: While these are full permissions, Stitch will only ever read your data. The repo scope is required due to how GitHub structures permissions.

  8. Click the Generate token button.
  9. The new access token will display on the next page. Copy the token before navigating away from the page - GitHub won’t display it again.

Step 2: Add LookML as a Stitch data source

  1. Sign into your Stitch account.
  2. On the Stitch Dashboard page, click the Add Integration button.

  3. Click the LookML icon.

  4. Enter a name for the integration. This is the name that will display on the Stitch Dashboard for the integration; it’ll also be used to create the schema in your destination.

    For example, the name “Stitch LookML” would create a schema called stitch_lookml in the destination. Note: Schema names cannot be changed after you save the integration.

  5. In the API Token field, paste the access token you copied from step 1.
  6. In the Git Owner field, enter the owner of the repositories you want to replicate. You can find this info in the repository’s URL. For example: The Git owner of https://github.com/stitch/stitch-repo would be stitch.

    Note: While you can replicate data from multiple repositories, this integration only supports replicating from one owner at a time.

  7. In the Git Repositories field, enter the repository or repositories you want to replicate. You can find this info in the repository’s URL. For example: The repository of https://github.com/stitch/stitch-repo would be stitch-repo.

    To track multiple repositories, enter them as a comma-delimited list. For example: repo-1, repo-2.

Step 3: Define the historical replication start date

The Sync Historical Data setting defines the starting date for your LookML integration. This means that data equal to or newer than this date will be replicated to your data warehouse.

Change this setting if you want to replicate data beyond LookML’s default setting of 1 year. For a detailed look at historical replication jobs, check out the Syncing Historical SaaS Data guide.

Step 4: Create a replication schedule

In the Replication Frequency section, you’ll create the integration’s replication schedule. An integration’s replication schedule determines how often Stitch runs a replication job, and the time that job begins.

LookML integrations support the following replication scheduling methods:

To keep your row usage low, consider setting the integration to replicate less frequently. See the Understanding and Reducing Your Row Usage guide for tips on reducing your usage.

Step 5: Set objects to replicate

The last step is to select the tables and columns you want to replicate. Learn about the available tables for this integration.

Note: If a replication job is currently in progress, new selections won’t be used until the next job starts.

For LookML integrations, you can select:

  1. Individual tables and columns

  2. All tables and columns

Click the tabs to view instructions for each selection method.

  1. In the integration’s Tables to Replicate tab, locate a table you want to replicate.
  2. To track a table, click the checkbox next to the table’s name. A blue checkmark means the table is set to replicate.

  3. To track a column, click the checkbox next to the column’s name. A blue checkmark means the column is set to replicate.

  4. Repeat this process for all the tables and columns you want to replicate.
  5. When finished, click the Finalize Your Selections button at the bottom of the screen to save your selections.
  1. Click into the integration from the Stitch Dashboard page.
  2. Click the Tables to Replicate tab.

  3. In the list of tables, click the box next to the Table Names column.
  4. In the menu that displays, click Track all Tables and Fields:

    The Track all Tables and Fields menu in the Tables to Replicate tab

  5. Click the Finalize Your Selections button at the bottom of the page to save your data selections.

Initial and historical replication jobs

After you finish setting up LookML, its Sync Status may show as Pending on either the Stitch Dashboard or in the Integration Details page.

For a new integration, a Pending status indicates that Stitch is in the process of scheduling the initial replication job for the integration. This may take some time to complete.

Free historical data loads

The first seven days of replication, beginning when data is first replicated, are free. Rows replicated from the new integration during this time won’t count towards your quota. Stitch offers this as a way of testing new integrations, measuring usage, and ensuring historical data volumes don’t quickly consume your quota.


LookML table reference

model_files

The model_files table contains information about model files in your GitHub about using the Git API Search filename and extension filters for model and lkml.

Replication Method

Key-based Incremental

Primary Keys

git_owner

git_repository

path

Replication Key

last_modified

Useful links

model_files schema on GitHub

LookML API method

content

STRING

download_url

STRING

encoding

STRING

git_owner

STRING

git_repository

STRING

git_url

STRING

html_url

STRING

last_modified

DATE-TIME

name

STRING

path

STRING

sha

STRING

size

INTEGER

type

STRING

url

STRING

models

The models table contains information about the model file parse items in your GitHub account using the lkml filter.

Replication Method

Full Table

Primary Keys

git_owner

git_repository

path

Useful links

models schema on GitHub

LookML API method

access_grants

ARRAY

allowed_values

ARRAY

name

STRING

user_attribute

STRING

case_sensitive

STRING

connection

STRING

datagroups

ARRAY

max_cache_age

STRING

name

STRING

sql_trigger

STRING

explores

ARRAY

access_filters

ARRAY

field

STRING

user_attribute

STRING

always_filter

OBJECT

filters

ARRAY

field

STRING

value

STRING

always_join

ARRAY

cancel_grouping_fields

ARRAY

case_sensitive

STRING

conditionally_filter

OBJECT

filters

ARRAY

field

STRING

value

STRING

unless

ARRAY

description

STRING

extends

ARRAY

extension

STRING

fields

ARRAY

from

STRING

group_label

STRING

hidden

STRING

joins

ARRAY

fields

ARRAY

foreign_key

STRING

from

STRING

name

STRING

outer_only

STRING

relationship

STRING

required_access_grants

ARRAY

required_joins

ARRAY

sql_foreign_key

STRING

sql_on

STRING

sql_table_name

STRING

sql_where

STRING

type

STRING

view_label

STRING

label

STRING

name

STRING

persist_for

STRING

persist_with

STRING

required_access_grants

ARRAY

sql_always_having

STRING

sql_always_where

STRING

sql_table_name

STRING

symmetric_aggregates

STRING

view_label

STRING

view_name

STRING

fiscal_month_offset

STRING

git_owner

STRING

git_repository

STRING

includes

ARRAY

label

STRING

last_modified

DATE-TIME

map_layers

ARRAY

extents_json_url

STRING

feature_key

STRING

file

STRING

format

STRING

label

STRING

max_zoom_level

INTEGER

min_zoom_level

INTEGER

name

STRING

projection

STRING

property_key

STRING

property_label_key

STRING

url

STRING

named_value_formats

ARRAY

name

STRING

strict_value_format

STRING

value_format

STRING

path

STRING

persist_for

STRING

persist_with

STRING

sha

STRING

week_start_day

STRING

view_files

The view_files table contains information about view files, using the Git API Search filename and extension filters for views and lkml.

Replication Method

Key-based Incremental

Primary Keys

git_owner

git_repository

path

Replication Key

last_modified

Useful links

view_files schema on GitHub

LookML API method

content

STRING

download_url

STRING

encoding

STRING

git_owner

STRING

git_repository

STRING

git_url

STRING

html_url

STRING

last_modified

DATE-TIME

name

STRING

path

STRING

sha

STRING

size

INTEGER

type

STRING

url

STRING

views

The views table contains information about the view file parse items in your GitHub account using the lkml filter.

Replication Method

Full Table

Primary Keys

git_owner

git_repository

path

Replication Key

last_modified

Useful links

views schema on GitHub

LookML API method

derived_table

OBJECT

cluster_keys

ARRAY

create_process

OBJECT

datagroup_trigger

STRING

distribution

STRING

distribution_style

STRING

explore_source

OBJECT

bind_filters

OBJECT

from_field

STRING

to_field

STRING

columns

ARRAY

field

STRING

name

STRING

derived_columns

ARRAY

name

STRING

sql

STRING

expression_custom_filter

STRING

filters

ARRAY

field

STRING

value

STRING

limit

STRING

name

STRING

sort

OBJECT

desc

STRING

field

STRING

timezone

STRING

indexes

ARRAY

partition_keys

ARRAY

persist_for

STRING

sortkeys

ARRAY

sql

STRING

sql_create

STRING

sql_trigger_value

STRING

dimension_groups

ARRAY

alias

ARRAY

allow_fill

STRING

bypass_suggest_restrictions

STRING

can_filter

STRING

convert_tz

STRING

datatype

STRING

description

STRING

drill_fields

ARRAY

fanout_on

STRING

full_suggestions

STRING

group_item_label

STRING

group_label

STRING

hidden

STRING

html

STRING

intervals

ARRAY

label

STRING

name

STRING

order_by_field

STRING

required_access_grants

ARRAY

skip_drill_filter

STRING

sql

STRING

sql_end

STRING

sql_start

STRING

suggest_dimension

STRING

suggest_explore

STRING

suggestable

STRING

tags

ARRAY

timeframes

ARRAY

type

STRING

view_label

STRING

dimensions

ARRAY

actions

ARRAY

form_params

ARRAY

default

STRING

label

STRING

name

STRING

options

ARRAY

label

STRING

name

STRING

required

STRING

type

STRING

form_url

STRING

icon_url

STRING

label

STRING

params

ARRAY

name

STRING

value

STRING

url

STRING

user_attribute_params

ARRAY

name

STRING

user_attribute

STRING

alias

ARRAY

alpha_sort

STRING

bypass_suggest_restrictions

STRING

can_filter

STRING

case

OBJECT

whens

ARRAY

label

STRING

sql

STRING

case_sensitive

STRING

convert_tz

STRING

datatype

STRING

description

STRING

drill_fields

ARRAY

fanout_on

STRING

full_suggestions

STRING

group_item_label

STRING

group_label

STRING

hidden

STRING

html

STRING

label

STRING

label_from_parameter

STRING

links

ARRAY

map_layer_name

STRING

name

STRING

order_by_field

STRING

primary_key

STRING

required_access_grants

ARRAY

required_fields

ARRAY

skip_drill_filter

STRING

sql

STRING

sql_end

STRING

sql_latitude

STRING

sql_longitude

STRING

sql_start

STRING

style

STRING

suggest_dimension

STRING

suggest_explore

STRING

suggest_persist_for

STRING

suggestable

STRING

suggestions

ARRAY

tags

ARRAY

tiers

ARRAY

type

STRING

value_format

STRING

value_format_name

STRING

view_label

STRING

extends

ARRAY

extension

STRING

filters

ARRAY

alias

ARRAY

bypass_suggest_restrictions

STRING

case_sensitive

STRING

convert_tz

STRING

datatype

STRING

default_value

STRING

description

STRING

full_suggestions

STRING

group_item_label

STRING

group_label

STRING

hidden

STRING

label

STRING

name

STRING

required_access_grants

ARRAY

required_fields

ARRAY

sql

STRING

suggest_dimension

STRING

suggest_explore

STRING

suggest_persist_for

STRING

suggestable

STRING

suggestions

ARRAY

tags

ARRAY

type

STRING

view_label

STRING

git_owner

STRING

git_repository

STRING

last_modified

DATE-TIME

measures

ARRAY

actions

ARRAY

form_params

ARRAY

default

STRING

label

STRING

name

STRING

options

ARRAY

label

STRING

name

STRING

required

STRING

type

STRING

form_url

STRING

icon_url

STRING

label

STRING

params

ARRAY

name

STRING

value

STRING

url

STRING

user_attribute_params

ARRAY

name

STRING

user_attribute

STRING

alias

ARRAY

approximate

STRING

approximate_threshold

NUMBER

can_filter

STRING

convert_tz

STRING

datatype

STRING

description

STRING

direction

STRING

drill_fields

ARRAY

fanout_on

STRING

filters

ARRAY

field

STRING

value

STRING

group_item_label

STRING

group_label

STRING

hidden

STRING

html

STRING

label

STRING

label_from_parameter

STRING

links

ARRAY

list_field

STRING

name

STRING

percentile

NUMBER

precision

INTEGER

required_access_grants

ARRAY

required_fields

ARRAY

sql

STRING

sql_distinct_key

STRING

suggest_dimension

STRING

suggest_explore

STRING

suggestable

STRING

tags

ARRAY

type

STRING

value_format

STRING

value_format_name

STRING

view_label

STRING

name

STRING

parameters

ARRAY

alias

ARRAY

allowed_values

ARRAY

label

STRING

value

STRING

bypass_suggest_restrictions

STRING

convert_tz

STRING

default_value

STRING

description

STRING

full_suggestions

STRING

hidden

STRING

label

STRING

name

STRING

required_access_grants

ARRAY

required_fields

ARRAY

suggest_dimension

STRING

suggest_explore

STRING

suggest_persist_for

STRING

suggestable

STRING

suggestions

ARRAY

tags

ARRAY

type

STRING

view_label

STRING

path

STRING

required_access_grants

ARRAY

sets

ARRAY

fields

ARRAY

name

STRING

sha

STRING

sql_table_name

STRING

suggestions

STRING

view_label

STRING


Questions? Feedback?

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.