Job Description
Responsibilities
- Automate data quality and reconciliation checks across varied storage layers, including Snowflake, SQL, and RDF/SPARQL databases.
- Test and verify data lineage, governance, and visualization components using Snowflake, data catalogs (e.g., DataHub), Thoughtspot, and other visualization tools.
- Integrate test suites into the core infrastructure orchestrated by Apache Airflow and utilizing Iceberg table formats, while monitoring data pipeline health, alerting, and observability metrics with Prometheus and Grafana Cloud.
- Establish AI evaluation loops and guardrails: build rigorous verification protocols—including structural tests, checks, and watchdog agents—to validate AI‑generated artifacts, catch false positives, and ensure outputs are secure, reliable, and free from hallucinations.
- Integrate automated testing workflows into CI/CD pipelines using GitHub Actions, ensuring continuous stability and quality gates ...