Location: Chester (Hybrid)
A leading global financial institution is seeking a Senior Data Platform Engineer to help build and enhance a strategic enterprise platform responsible for ingesting, processing and assuring large-scale communication, collaboration and transactional data.
We're looking for an engineer who can own the entire data lifecycle, from source-system integration and streaming ingestion through to transformation, storage and the delivery of production-grade compliance and assurance controls. You'll work at the intersection of software engineering, data engineering and regulatory technology, building scalable solutions used across a highly regulated environment.
The role offers exposure to modern data-platform technologies, complex distributed systems, enterprise-scale datasets and emerging AI-enabled engineering practices.
What You'll Be Doing End-to-End Platform Engineering
- Design and develop resilient enterprise-scale data pipelines
- Build integrations across vendor APIs, streaming technologies and distributed data platforms
- Engineer high-volume batch and near real-time processing solutions
- Develop reusable ingestion and processing frameworks rather than one-off solutions
- Implement robust data handling patterns including replay, backfill, recoverability and schema evolution
- Define and enhance operational telemetry, monitoring and platform observability
- Build and optimise solutions using:
- Apache Kafka
- Apache NiFi
- Apache Spark / PySpark
- Hadoop ecosystem technologies
- Hive and large-scale data storage platforms
- Develop scalable matching, enrichment and reconciliation services
- Create canonical data models spanning multiple source systems
- Ensure data lineage, quality and governance requirements are met
- Translate regulatory and operational requirements into deterministic control logic
- Develop reconciliation, completeness, configuration and assurance controls
- Produce auditable outputs, exception reporting and evidence frameworks
- Build monitoring and alerting capabilities for compliance-critical processes
- Work closely with engineering, architecture, risk and operational stakeholders
- Write clean, maintainable and testable Python and PySpark code
- Contribute to CI/CD pipelines and automated deployment processes
- Participate in code reviews and engineering standards initiatives
- Deliver solutions through Development, UAT and Production environments
- Support troubleshooting and root-cause analysis across distributed systems
Core Technical Skills
- Python development in enterprise environments
- Apache Spark / PySpark
- Apache Kafka
- Apache NiFi
- Hadoop and Hive
- Advanced SQL
- REST API integration and development
- Data modelling and data lineage
- Distributed systems engineering
- CI/CD and modern software delivery practices
- Git and collaborative development workflows
- Microsoft Azure
- Microsoft Graph
- Entra ID
- Service principals and secure identity patterns
- Grafana, Splunk, Tableau or Power BI
- Java or another strongly typed language
- AI-assisted development tools including GitHub Copilot
- Exposure to agentic AI, orchestration frameworks or automation tooling
- Financial services
- Banking and capital markets
- RegTech
- Large-scale enterprise data engineering environments
- Telecommunications or communications platforms
- Highly regulated industries
What Success Looks Like
- New data sources are onboarded through reusable engineering patterns
- Pipelines are resilient, observable and fully recoverable
- Control logic is transparent, auditable and trusted
- Data quality and lineage are clearly understood
- Solutions are delivered through a disciplined SDLC into production environments
- The platform scales to support new technologies, controls and business requirements
