24/7 data observability
Implementation of a monitoring framework with Great Expectations and Airflow to validate critical pipelines and publish alerts in Slack.
Key components
- More than 60 automated test suites for critical tables.
- Metadata and results stored in an Azure data lake.
- Monitoring dashboards in Grafana with coverage metrics.
Repository and documentation
Code and CI/CD configuration documented in an internal repository, with public examples available for reference.