Amazon S3 Setup Requirements
To set up Amazon S3 in Stitch, you need:
An Amazon Web Services (AWS) account. Signing up is free - click here or go to
https://aws.amazon.comto create an account if you don’t have one already.
Permissions to create and manage S3 buckets in AWS. Your AWS user must be able to create a bucket (if one doesn’t already exist), add/modify bucket policies, and upload files to the bucket.
An up and running Amazon S3 bucket. Instructions for creating a bucket using the AWS console are in Step 1 of this guide.
Step 1: Create an Amazon S3 Bucket
- Sign into AWS.
- Click Services near the top-left corner of the page.
- Under the Storage option, click S3. A page listing all buckets currently in use will display.
- Click the + Create Bucket button.
- On the first screen, Name and region, complete the following:
- Bucket name: Enter a DNS-compliant name for the bucket.
- Region: Select the region you want the bucket to be located in.
- When finished, click Next.
- As Stitch doesn’t require any particular configuration, how you define the settings in the Set properties and Set permissions screens are up to you. Follow the on-screen prompts to complete these steps.
- When you reach the Review screen, verify that all the bucket’s information and settings are correct.
- When ready, click Create bucket.
Step 2: Configure Stitch Settings
In this step, you’ll define the file and object key format Stitch will use to load data into Amazon S3.
First, enter the name of the bucket in the Bucket Name field. Enter only the bucket name: No URLs,
https, or S3 parts.
Step 2.1: Select Data Storage Format
The data storage format defines the type of file Stitch will write to your Amazon S3 bucket. Supported options are:
- JSON: Data will be stored as JSON files (
- CSV: Data will be stored as CSV files (
For examples of how data in each format will be stored, click here.
If CSV is selected, there are a few additional configuration options for the files Stitch will write to your bucket:
- Delimiter: Select the delimiter you want to use. Stitch will use the comma (
,) option by default, but you may also use pipes (
|) and tabs (
- Quote all elements in key-value pairs: If selected, Stitch will place all elements of key-value pairs in quotes. For example: Numerical fields will appear as
Step 2.2: Define Webhook Loading Notifications
As part of an Enterprise plan, you can set up configurable webhooks to notify you when fresh data has finished loading into your destination. Contact Stitch Sales for more info.
Webhooks allow external services to be notified when an event happens. If you choose, you can configure a webhook for Stitch to notify you when data is successfully loaded into your bucket.
Webhook notifications are sent on a per-integration basis. This means that every time Stitch successfully loads data for an integration, a summary webhook will be sent to the URL you define.
To enable this feature, check the Post to a webhook URL each time loading to S3 completes box and paste a webhook URL in the Webhook URL field.
More info about webhook loading notifications, including a list of attributes and sample use cases, can be found here.
Step 2.3: Define S3 Object Key
In Amazon S3, Object Keys are used to uniquely identify objects in a given bucket.
The S3 Key setting determines the convention Stitch uses to create Object Keys when it writes to your bucket. For example: If the default Key is used:
This could create an object with an Object Key of:
You can opt to use the default Key, which is pre-populated, or define your own using the elements in the next section.
S3 Key Elements
The following elements are available to construct an S3 Key:
|Required Elements||Optional Elements|
All of the following elements must be included in the S3 Key, in any order:
The following elements are optional:
Additionally, keep in mind that Keys cannot exceed 500 characters or include spaces or special characters (
As you update the values in the S3 Key field, Stitch will validate the entry. If the Key doesn’t include all required elements or contains spaces or special characters, you will be prompted to make corrections.
After you’ve finished defining the Key, click Continue.
Step 3: Grant and Verify Bucket Access
Next, Stitch will display a Grant & Verify Access page. This page contains the info you need to configure bucket access for Stitch, which is accomplished via a bucket policy. A bucket policy is JSON-based access policy language to manage permissions to bucket resources.
Note: The policy Stitch provides is an auto-generated policy unique to the specific bucket you entered in the setup page.
For more info about the top-level permissions the Stitch bucket policy grants, click the link below.
|Permission Name||Operation||Operation Description|
Allows the addition of objects to a bucket.
An alternate form of
|Initiate Multipart Upload||
Allows a multipart upload and return of an upload ID.
Allows for the upload of a part in a multipart upload.
|Complete Multipart Upload||
Allows for the completion of a multipart upload by assembling previously uploaded parts.
|PUT Object - Copy||
Allows for the upload of a part by copying data from an existing object as the data source.
Allows for the retrieval of objects from Amazon S3.
Allows for the retrieval of metadata from an object without returning the object itself.
|s3:ListBucket||GET Bucket (List Objects)||
Allows for the return of some or all (up to 1,000) of the objects in a bucket.
Used to determine if a bucket exists and access is allowed.
Step 3.1: Add the Stitch Bucket Policy
To allow Stitch to access the bucket, you’ll need to add a bucket policy using the AWS console.
- Sign into AWS in another tab, if you aren’t currently logged in.
- Click Services near the top-left corner of the page.
- Under the Storage option, click S3.
- A page listing all buckets currently in use will display. Click the name of the bucket you want to connect to Stitch.
- Click the Permissions tab.
- In the Permissions tab, click the Bucket Policy button.
- In the Bucket policy editor, paste the bucket policy code from Stitch.
- When finished, click Save.
Leave this page open for now - you’ll come back to it in the next step.
Step 3.2: Verify Bucket Access
Next, to ensure that Stitch can access the bucket, you’ll create a blank file that Stitch will use to test the permissions settings.
In Stitch, just below the bucket policy code, is the Verify your bucket section. In this section is a field containing the unique name of the test file you need to create:
Note: This file name will only display once. If you navigate out of this screen without saving the file name, you’ll need to start over.
Create a blank file using the name displayed in this field. Do not save the file with an extension (file type) like
.txt. In the image below, notice that there isn’t any kind of file extension after the challenge file name:
- Switch back to the AWS console and click the Overview tab.
- Click the Upload button and follow the prompts to upload the file.
- After the file has been uploaded to the bucket, switch back to where you have Stitch open.
- Click Check and Save to save and test the connection to Amazon S3.
Troubleshooting connection errors
If you receive an error message, we recommend trying these troubleshooting steps for common destination connection problems before reaching out to support.
We’ve found that the majority of the time, these resources can resolve most issues.
Learning about Stitch and Amazon S3
Now that your Amazon S3 data warehouse is up and running, we recommend learning about how Stitch will load your data into it.
- Amazon S3 Destination Overview: This overview will give you the high-level basics for working with your Amazon S3 data warehouse.