Job Description
We're looking for a hands‑on Lead Data Engineer to support the delivery of a new real-time Operational Data Store (ODS) platform during a critical phase of the programme. You'll take ownership of designing, building, and optimising high‑throughput data pipelines while providing technical leadership across the engineering team.
Experience you'll bring
- Experience with PySpark , Spark , and Structured Streaming
- Proven background building real‑time / near‑real‑time data platforms
- Experience with operational data stores (ODS) and ingestion‑heavy architectures
- Strong understanding of data quality, reconciliation, duplicates, ordering, and late-arriving data
- Performance tuning Spark workloads for scalability, stability, and low latency
- Ability to troubleshoot complex issues and make pragmatic technical decisions under delivery pressure
- Comfortable providing technical...