The Replication Frequency setting, found in the Integration Settings page, controls how often Stitch will attempt to replicate data from an integration.
The more frequently an integration is set to replicate, the higher the number of rows that will replicate through Stitch.
Sync Scheduling & Replication Frequency
Stitch schedules syncs based on the start time of the previous sync.
If the previous sync is still in progress when it would be time to start the next sync, Stitch will wait until the next recurrence of the Replication Frequency to begin.
For example: A Facebook Ads integration has a Replication Frequency of 30 minutes. The Start and End Times for this integration’s syncs might look like this:
|Sync #||Start Time||End Time||Duration (minutes)|
Notice Sync 2’s Start Time. This is because Sync 1 took 42 minutes to complete, or longer than the defined 30 minute Replication Frequency. In this case, Sync 2 began at the next recurrence of the Replication Frequency, or
Row Count Impact & Replication Frequency
Replication Frequency affects not only how often your integrations are set to replicate, but how much data is replicated from the integration over time.
Even though we’ll send out notifications when you’re nearing your row limit, we recommend taking the following into consideration before setting the Replication Frequency for an integration:
- Replication Frequency applies to the entire integration, not individual tables. This means all tables set to sync will replicate according to the set frequency.
The number of syncing tables in the integration. While you can sync individual tables in database integrations and the SaaS integrations that support whitelisting, most SaaS integrations have all tables set to sync by default.
SaaS integrations with high table counts like QuickBooks can result in a high number of replicated rows if set to replicate frequently.
The Replication Methods syncing tables use. Because it can reduce latency and your row count, we recommend using Incremental Replication for database integration tables whenever possible. A high Replication Frequency (such as 30 minutes) and a large number of tables that use Full Table Replication can quickly use up your row count and possibly lead to overages.
If some tables have to use Full Table Replication, check out the Different Frequencies for Full & Incrementally Replicating Tables section below for a solution that can significantly reduce row usage.
For info on the Replication Methods used by SaaS integration tables, refer to the Schema section on any integration’s page.
How the integration replicates as a whole. This typically applies to SaaS integrations and the querying strategies Stitch employs to extract data.
For example: Every time a sync runs for Facebook Ads, the past 28 days’ worth of data will be replicated. This is to account for changes that can be made during the 28 day attribution window on campaigns. Even though all tables in this integration use Incremental Replication, the number of replicated rows can still be huge because of the strategy Stitch uses to query.
For more info on how SaaS integration data is queried/replicated, refer to the Replication section in the SaaS integration docs.
- The importance of completely fresh data. If you don’t need data to be constantly replicated or if tasks can still be completed with slightly older data, consider setting integrations to replicate less frequently.
Define an Integration’s Replication Frequency
When you initially create and set up an integration, you’ll be asked to define its Replication Frequency.
You can either use the default (which is 30 minutes for most integrations) or define a custom setting. In some cases, you’ll have to uncheck the Use Integration Default box:
Then, Stitch will replicate data from all syncing tables in the integration based on their individual Replication Methods and the Frequency you select.
Update an Integration’s Replication Frequency
If at any time you want to change the Replication Frequency for an integration, here’s what you need to do:
- Click into the integration from the Stitch Dashboard page.
- Click the Settings tab, next to Tables to Replicate.
- Scroll down to the Replication Frequency section.
- Change the Replication Frequency to your desired setting.
- Click Save Integration.
Define Different Frequencies for Full & Incremental Tables
Additionally, while this will work for some SaaS integrations, some SaaS integration providers may not allow more than one API session to be open at a time. NetSuite is just such an example. We are unsure of the exact impact this may have on any API quotas imposed by the integration provider.
To help keep row counts low, you can create two integrations: one to sync tables using Full Table Replication, the other to sync Incremental tables. You can then set each integration’s Replication Frequency appropriately.
Let’s say you have two tables in a PostgreSQL database that you want to sync:
files table contains 50,000 rows and uses Full Table Replication, while
orders contains 100,000 rows and updates incrementally.
On an average day, 500 rows are added or updated to the
orders table. For
files, perhaps 10 rows are added or updated each day. It isn’t necessary to have
files replicate as often as
To help keep your row count down, you could:
- Create a PostgreSQL integration and name it
Postgres - Full Tableor something similar.
- Only select tables using Full Table Replication to sync. In this case, that would be the
- Set the integration’s Replication Frequency to every 24 hours.
Then, to ensure the
orders table is updated regularly:
- Create an additional PostgreSQL integration and name it
Postgres - Incrementalor something similar.
- Only select tables using Incremental Replication to sync. In this case, that would be the
- Set the integration’s Replication Frequency to every 30 minutes, 1 hour, etc.
Did this article help? If you have questions or feedback, please reach out to us.