Cloud data warehousing has facilitated innovation and modernization in data pipeline builds. ELT is the new standard of processes for data pipelines and allows large data loads to be handled fast and efficiently.
This page is Daasity’s ever-growing resource launchpad for ELT information and content:
- What is ELT?
- What is Daasity? Is Daasity an ELT solution?
- EL (Extract, Load) Resources, including comparison between Daasity + other EL solutions
- Information on Daasity’s data models
- How to approach ELT/data stack builds
What Is ELT?
ELT stands for “Extract, Load, Transform:” these are the three major steps required to sync data from a data source (e.g., Shopify Plus or Klaviyo) into a repository, which is commonly a cloud data warehouse (e.g., Snowflake or BigQuery).
- Extract: Copying data from the data source, often via API
- Load: Replicating the data, storing in a database
- Transform: Reformatting and/or normalizing data for analysis and (commonly) visualization.
For information about how ELT compares to ETL, head here: ETL vs ELT.
What Is Daasity? Is Daasity an ELT solution?
Daasity is the first and only Modular Data Platform (MDP) built for omnichannel consumer brands, and yes: Daasity is used for ELT. In fact...
Daasity can be used as an E, EL, or ELT solution. Daasity can also be used as a complete data solution: ELT, prebuilt and customizable dashboards in a visualization tool (e.g., Looker or Tableau), reverse ETL, and data orchestration. We're modular, so brands can choose the functionality they need.
Brands use Daasity as an extractor-only solution, such as to pull data from key sources, such as Shopify Plus, Amazon, and retail POS. Brands also use Daasity as their complete data and analytics stack. Brands can integrate Daasity into their Modern Data Stack, or they can fully replace the Modern Data stack with Daasity.
EL (Extract, Load) Resources
Daasity connects to data sources that EL tools cannot, such as Amazon and retail POS, and to get more data from certain data sources (e.g., Shopify) compared to leading extractors.
Daasity vs. Other Extraction Solutions for Consumer Brands
Daasity vs. Fivetran
There are some key differences between Daasity and Fivetran.
- Fivetran is an EL tool that must be connected to a transformation tool, such as dbt. Through dbt, it offers some transformation capability via dbt Core-compatible models.
- Fivetran does not connect to Amazon or retail POS. Daasity has a robust Amazon integration and connects to several retail POS.
- Fivetran requires a separate data orchestration tool. Daasity has built-in data orchestration.
- Full implementation of a Modern Data Stack that includes Fivetran can take 6-18 months, depending on size of brand and complexity of data. Full implementation with Daasity takes less than a month.
For a much deeper dive, we've built a full comparison here: Daasity vs. Fivetran.
Daasity vs. Stitch Data
As with Fivetran, there are some key differences between Daasity and Stitch Data.
- Stitch has both "Stitch-certified" and "community-supported" extractors via the Singer framework. All Daasity extractors are built, maintained, and optimized by Daasity.
- Stitch does not have any out-of-the-box transformation functionality. It must be connected to a transformation tool, such as dbt or its parent company, Talend.
- Like Fivetran, Stitch does not connect to Amazon or retail POS.
- For data orchestration, Stitch has error handling with notifications, it logs extraction/load, and brands can choose their replication frequency. For its higher pricing tiers, Stitch offers greater data orchestration functionality. Daasity offers robust and customizable data orchestration functionality out-of-the-box, based on a merchant’s integrations, ensuring that data transformation runs once extraction is complete. Merchants can also leverage Daasity's Custom Workflows functionality. Merchants can choose the integrations, transformation scripts, and schedule(s) they want to run.
For a thorough comparison, head over here: Daasity vs. Stitch.
Data Transformation Resources
Daasity’s Data Models
Unified Order Schema (UOS)
Daasity's Unified Order Schema is a core data model within the Daasity transformation module that helps accelerate development of analytical capability by normalizing all commerce data: eCommerce, Marketplace, Retail and Wholesale. We break down the "why" of each component of the data model, and we link to our documentation and link to the corresponding KB doc that covers the "what" by providing a column-by-column breakdown of the data model.
Unified Notifications Schema (UNS)
Daasity's Unified Notification Schema is another core data model that make it easy to centralize and analyze data from multiple communication platforms, such as:
- Email via an Email Service Provider (ESP)
- SMS via SMS platforms
- Push notifications or in-app notifications via relevant providers
Building an ELT Pipeline and Choosing a Data Warehouse
If you’re looking for resources on building an ELT process for your brand, we’ve got you covered.
Building ELT + Analytics Using Data Tools (e.g., Fivetran, dbt)
If you’re looking to Stitch together (bad pun) a data pipeline using data tools out there, you’ll need to connect the following:
- An Extract/Load provider (such as Fivetran, Stitch)
- A Data Warehouse provider (such as Snowflake or BigQuery)
- A Transformation tool (such as dbt)
- A visualization tool (such as Looker or Tableau)
- A data orchestration tool (such as Airflow)
- A reverse ETL tool (such as Hightouch)
We’ve built a piece on building your own data stack stack that covers this process, as well as expected timelines and costs (both initial and ongoing costs) associated with the project.
Choosing a Cloud Data Warehouse
As an expansion on one element of the DIY analytics stack, we cover choosing a cloud data warehouse in greater detail. In this article, we compare Snowflake, BigQuery, and Redshift.
Building ELT from Scratch
If you’re interested in or looking to learn more about what it takes to build an ELT pipeline from the ground up, we wrote an article about what would go into this build, as well as the complexities and challenges of the process.
We use Shopify’s API as an example of a data extraction source, and talk about the Transformation side of the process.
