Our Solution: Microsoft Fabric
The transformation followed a structured, phased approach that adopted Microsoft Fabric as the cloud data platform, implementing a Lambda Architecture that supports both batch and real-time processing workloads. The architecture comprises three layers:
Lambda Architecture Layers
| Batch Layer | ELT pipelines via Azure Data Factory and Fabric Notebooks, processing data into a medallion architecture (Bronze → Silver → Gold) |
| Speed Layer | Microsoft Fabric Event Stream for near-real-time ingestion via Azure Event Hub into KQL Database and Lakehouse Delta Tables |
| Serving Layer | T-SQL and Spark endpoints exposing data to Power BI, Excel, and other consumers across all data stores |
All data is standardized in Delta Lake format within OneLake, eliminating redundant copies across compute engines. Microsoft Purview integration was incorporated for governance and data lineage. The platform was sized at F128 Microsoft Fabric capacity — exceeding the baseline F64 estimate — to accommodate additional workloads and future growth. DevOps/CI-CD pipelines and a data quality framework were established from Phase 1 as foundational standards applied across all subsequent phases.
Oracle Data Migration (Phase I)
Migrate the Oracle EDW on-premises ETL and all associated Power BI reporting to cloud. Oracle was selected as the Phase 1 workload because it is the most loosely coupled dataset in the EDW – minimizing dependencies – and requires a source-layer change from Oracle ATP to Oracle Object Storage, which is best handled as a standalone effort. Two distinct Oracle data streams are in scope: GL (General Ledger) and Discover. Both run as separate batches in the on-premises EDW and will be independently managed in the cloud pipeline.
Data Pipeline Architecture
| Oracle Object Storage | → | Azure Blob Storage (ADF) | → | Fabric Bronze Layer | → | Fabric Silver Layer | → | Fabric Gold Layer |
Each medallion layer maps to a specific transformation stage: Bronze ingests raw files as-is; Silver applies transformations to match the existing ATP table structure; Gold processes data into the final structure mirroring the on-premises EDW.
Migrated Oracle ERP data (GL and Discover) along with common reference datasets (LocationList, WLCEmployee, RLS) to the cloud. This phase established the foundational architecture using a medallion model (Bronze, Silver, Gold), implemented end-to-end pipelines, migrated historical data, and transitioned Power BI dashboards. Success was defined by full data and dashboard parity with improved ETL performance, enabling decommissioning of on-prem Oracle ETL.
Success Criteria
| ✓ | On-premises EDW data matches the Cloud platform data – row counts, aggregates and key business metrics validated |
| ✓ | Power BI dashboards fed from on-premises data match those fed from the cloud platform |
| ✓ | Cloud ETL runtime is equal to or better than the on-premises runtime (currently 8 hours avg.) |
Phase 1 Exit Milestone: Upon successful completion and validation, on-premises ETL for Oracle sources will be permanently decommissioned.
Expansion & Platform Scaling (Phase II)
Extend cloud migration to operational and workforce data sources, including Compass (CRM), NWF (workforce data), and Medallia (survey/API data) while building the batch processing layer of the data platform. This phase involved building 47 tables each in bronze and silver layers, and 30 tables in the gold layer, over a 19-week timeline. The objective was to migrate all ETL and reporting pipelines for these sources from on-premises systems to the cloud, covering the full data lifecycle from ingestion to summary layers. The scope included setting up source connectivity, migrating historical data, developing pipelines across bronze, silver, and gold layers, transitioning Power BI dashboards, extending CI/CD frameworks from Phase 1, and executing proofs-of-concept for SharePoint Shortcuts and custom Airflow-style orchestration.
Success Criteria
| ✓ | Data and dashboard parity, along with improved ETL performance |
| ✓ | Key workstreams progressed from initial POCs to source integration, pipeline development, and reporting migration |
| ✓ | Seamless transition of reporting layers to cloud, UAT, production deployment, and documentation. |
Future Phases (Planned)
Phase 3: Data quality, reliability, and model refinement
Phase 4: Migration of remaining data sources and reporting layers
Overall Migration Roadmap
Phase 1 is the first of four phases in the full EDW cloud migration. Each phase builds on the foundation established by the previous one.
| Phase | Focus | Objective | Exit Condition |
| Phase 1 | Oracle Migration | Migrate Oracle ETL (GL & Discover) and Oracle Power BI dashboards to the cloud | Oracle on-prem ETL decommissioned |
| Phase 2 | Bronze & Silver (All Sources) | Ingest and transform all remaining source data into bronze and silver layers | Cloud Bronze/Silver validated; on-prem still running |
| Phase 3 | Gold Layer (All Sources) | Build Dimensions, Facts, and Summaries for Phase 2 sources in Gold layer | Full cloud EDW parity achieved |
| Phase 4 | Full Reporting Cutover | Migrate all remaining Power BI dashboards/datasets to cloud and decommission on-prem | On-prem EDW fully retired |
