About Client
The client is a billion-dollar home leasing company with an estimated 80,000 rental homes in 16 markets across the United States.
Background
The client initiated a modernization effort for their data architecture. As part of this process, the Enterprise Data Warehouse was moved from SQL server to the cloud-based Snowflake platform.
At the same time, data from nine different sources got added with plans to incorporate more in the future.
However, ensuring data quality and consistency throughout the ingestion process became crucial for minimizing errors and achieving robust data governance. The client desired for a modern, scalable, and cost-effective solution that could handle complex analytical tasks, reduce operational complexity, and facilitate efficient data ingestion.
Challenge
The lack of a systematic approach posed our client with the following challenges:
- Errors and difficulties in data integration and transformation during ingestion.
- Monitoring errors during the pipeline execution and capturing source-related data inconsistencies became tedious.
The client aimed to minimize errors during ingestion through a quality data framework.
