Great news for Stitch / AWS Redshift users
AWS just announced the launch of second-generation Dense Compute nodes for use with AWS Redshift warehouses. These DC2 nodes boast an impressive doubling of warehouse performance at the same price. For Stitch users, the upgrade process is simple and low-risk.
If your Redshift cluster is hosted in one of the regions that support the new DC2 nodes, this post will guide you through the upgrade process our team used.
While AWS notes in their release article that you can perform this upgrade by restoring your cluster from a snapshot, we found that modifying the node type of a running cluster worked just fine.
The process outlined here requires resizing your Redshift cluster, so the cluster will be restarted in read-only mode for the duration of the operation. This means that any active connections or queries being executed will be terminated once you start, then again when the resize completes.
We suggest waiting until non-business hours to perform this upgrade, as there will be some downtime during which Stitch will be unable to load data to your Redshift cluster. Specifically, you’ll see occurrences of the following two errors during the resize process:
ERROR: Cannot execute query because system is in resize mode Detail: System is in resize mode, and ONLY read-only queries are allowed to execute.
Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
These indicate that Stitch was blocked from loading data to your warehouse due to either the temporary read-only functionality or cluster restart. Stitch will automatically reattempt loading once the resize has completed, so no data will be lost and resolution won’t require any action on your part.
The resize duration will vary depending on the amount of data your cluster contains, as well as the number of nodes. For us, the entire process took exactly 2 hours to upgrade our cluster from five dc1.large nodes to five dc2.large nodes with 50% of disk capacity utilized at the time.
The upgrade process
To start the upgrade process, log in to the AWS Management Console and click on the Services tab.
Select Amazon Redshift from the Database listing.
Now that the Redshift console is open, click on the Cluster pane.
In the Cluster column, click the name of the cluster you’d like to upgrade. This will open the Cluster Details page.
- From here, locate and click the Cluster dropdown, where you’ll see an option to “Resize”:
This will open a Resize Cluster window with three different fields: node type, cluster type, and number of nodes.
- Click on the Node type field and select either dc2.large or dc2.8xlarge (depending on your team’s current node type). Leave the Cluster type and Number of nodes unchanged.
- To store the changes and begin the resize process, press the Resize button at the bottom of this window. And that’s it. Really.
You can check on the progress of the resize by clicking on the Status tab, which displays the completion percentage and average rate of transfer, as well as the elapsed time and expected remaining time.
Once the resize has completed, Stitch will resume loading to your Redshift instance from the point where it left off.
We’re just beginning to try out these new node types. Some of our team members have already noted reduced execution time on their reporting. AWS mentions that the new DC2 nodes will support I/O speeds of up to .60GB/s, compared to only .20GB/s with the first-generation DC1 nodes, so we’re excited to see how warehouse performance improves.
Not already a Stitch user? Sign up for a free trial today and check out the 60+ supported data sources that can be integrated with Redshift using Stitch.