One ETL platform to handle SaaS data and internal services
OrderMyGear provides an online platform for buyers and sellers of group apparel and gear. The company's director of engineering, Daniel Hodges, says, "We have a lot of critical account and contact information that was only stored in Salesforce. We needed to combine that information with the data from our application so that we could better understand the performance of specific segments of our customer base." The solution would involve a Google BigQuery data warehouse and a number of data sources – not only SaaS platforms like Salesforce and Zendesk, but also internal services exposed via a REST API. OrderMyGear needed an ETL tool that could handle all of them.
Hodges says, "We looked at four ETL tools. For one of the tools, one of our team members would have had to become an expert in order to use it effectively. Another seemed to be targeted toward large organizations that have difficulty getting the appropriate resources to execute new integrations. We were strictly focused on building a data pipeline for our analytics tools. A third seemed targeted at business analytics teams that need to push data into their warehouse without any engineering effort. We actually discussed Stitch with them, and they said, 'Stitch is more targeted towards engineering teams,' and we said, 'perfect.'"
"We initially used Stitch only to push Salesforce and Zendesk data into our data warehouse. I was able to set up those flows myself in a matter of minutes. We set up new views in Looker the same day and instantly gained new insights on our data.
"Once we figured out that we could use the open source Singer tool along with Stitch, our data engineer implemented a tap that could fetch data from our API and then push it to the Stitch target. The initial proof-of-concept implementation of this tap took just a few days to implement. After we determined that we wanted to continue in this direction, the majority of our data engineer's effort was spent setting up Apache Airflow in our production environment and configuring it to run jobs that pushed API data to Stitch. We also had to update several of our internal service APIs to support fetching the data we needed to warehouse.
"Stitch became a critical part of our new data pipeline, which was focused around accuracy and ease of use. Adding a new data source to our old pipeline used to take weeks of effort. Stitch enables us to add a new data source to our pipeline in a matter of days.
"And Stitch has solved problems we didn't realize we had. Shortly after we committed to using Looker as our analytics tool, our COO insisted on having Salesforce data available in it. I expected that project to take a month; we completed it in a day with Stitch."
Stitch became a critical part of our new data pipeline, which was focused around accuracy and ease of use.
Director of Engineering
Not everything went smoothly 100% of the time, however. "Our data engineer worked with the Stitch Support team when we ran into some trouble pushing some poorly formatted data into our warehouse. I was impressed with the fact that when we were having issues, we didn't have to re-push our data to Stitch. Once the issues were resolved, the data flowed directly into our data warehouse without any manual work on our side."
Moving forward, Hodges says, "We're going to continue to expand the number of internal services we extract data from. Our company also loves to invest in new software services all of the time, and we need data from those services to be pushed into our data warehouse so that we can connect the dots in our analytics toolset.
"Stitch has become one of our most valuable team members. It sits in the corner, quietly chugs along, and makes our entire team look like rock stars," Hodges says. "Thank you for being awesome."