​
What is reverse-ETL?

Reverse-ETL is the process of loading selective data from a data warehouse into operational systems and SaaS tools. Business teams interacting with customers can have visibility into this data in their tools and personalize interactions with customers, creating a much richer experience for customers. In this way, business teams can stay away from having to access data warehouses and from their complexity. Thus reverse-ETL benefits business teams directly by simplifying access to data. More importantly, the data pipelines managed by reverse-ETL systems ease the workflows of data engineers who will have to move data for business teams in the absence of reverse-ETL systems. In addition, a purpose-built reverse-ETL system like valmi.io is more robust to deploy and manage than a system built in-house. Thus, both data teams and business teams can leverage valmi.io in their workflows.

​
The Modern Data Stack and valmi.io reverse-ETL in it.

Modern Data Stack

The data stack is a collection of technologies that are used to collect, store, process, and analyze data. The modern data stack revolves around the central storage component called a Data Warehouse. The data warehouse is a central repository for data that is used to store large amounts of data and to perform complex queries. The data warehouse is the heart of the modern data stack. We have talked about the importance and evolution of a data warehouse here.

​
Components of the modern data stack

There are several components in the modern data stack. These components include:

  • Data ingestion: Data sources are the systems that collect data. Data sources can include operational systems, SaaS tools, and sensors. These are upstream data sources that feed data into the data stack. The process of collecting data from data sources is called data ingestion, also known as Extract, Load of ELT. Airbyte is an open-source data integration platform that helps you replicate your data in your warehouses, lakes and databases. You can read more about Airbyte here. Fivetran is a commercial vendor in this category.
  • Data warehouses: Data warehouses are central repositories for data. Data warehouses are used to store large amounts of data and to perform complex queries. Cloud warehouses such as Snowflake, BigQuery, Databricks and Redshift are prime examples of data warehouses.
  • Data processors: Data processing tools are used to clean, transform, and analyze data. Data processing tools can include data wrangling tools, machine learning tools. The principal technology in this segment is dbt. This makes up the Transform part of the ELT process.
  • Data activation: The word ‘Activation’ refers to the process of putting data into action and making data available to business users. Data activation tools can include reverse-ETL tools, customer data platforms, and marketing automation tools. This is a new category and soon becoming a key component of the modern data stack. In this segment, Valmi.io open source reverse-ETL is a key player.

The data stack is a complex system, and it can be difficult to manage. However, the data stack is essential for organizations that want to collect, store, process, and analyze data.

In the next section, we describe how to quickly get up and running with valmi.io reverse-ETL.

Quick Start