Senior Lead Software Engineer, Big Data & Cloud Engineering — Risk Central (London).

• Lead end-to-end delivery of complex initiatives across ingestion, transformation, storage, and consumption layers. 
• Drive engineering best practices (design reviews, code reviews, testing standards, CI/CD, documentation). 
• Mentor engineers and raise the bar on technical quality, ownership, and operational excellence. 
• Design and build robust batch and streaming pipelines (high-volume, high-throughput) using distributed compute. 
• Implement efficient data modeling, partitioning, and performance tuning strategies for large datasets. 
• Build reusable frameworks/components to accelerate onboarding of new datasets and analytics use cases.
• Engineer data products and workflows that span Databricks and Redshift, including ingestion patterns, transformations, and serving layers. 
• Define approaches to reconciliation/consistency, lineage, and controls across both warehouses. 
• Optimize query performance and cost across platforms; establish best practices for workload placement. 
• Build cloud-native solutions on AWS (e.g., S3, EMR, Lambda, Kinesis/MSK, Glue, EventBridge, DynamoDB, Redshift, EKS—depending on team standards).
• Leverage Spark-based processing (including PySpark/Scala Spark) and modern lake/lakehouse patterns where applicable. 
• Own production stability: monitoring/alerting, incident management, root-cause analysis, and preventative engineering. 
• Define and improve SLAs/SLOs for critical data deliveries and platform uptime. 
• Stakeholder partnership - work closely with product managers, quants, risk managers, traders, and controllers to translate needs into scalable technical solutions. 
• Communicate clearly with senior stakeholders on progress, risks, dependencies, and trade-offs. 
• Contribute to planning, refinement, execution, and retrospectives; help teams deliver predictably with high quality.

   

•  Extensive professional software engineering experience, including delivery of production systems at scale.
• Strong programming skills in Python (and/or Java/Scala), with strong CS fundamentals (data structures, algorithms, OO design). 
• Hands-on experience with distributed data processing (e.g., Spark) and building data pipelines (batch and/or streaming). 
• Experience with at least one of: Databricks, Amazon Redshift, or equivalent enterprise data warehouse/lakehouse platforms; ability to design and tune performant workloads. 
• Strong SQL skills and understanding of data modeling and analytics patterns. 
• Practical experience with cloud engineering concepts (security, networking basics, IAM/access controls, encryption, observability). 
• Proven ability to troubleshoot production issues and drive operational improvements. 
• Strong communication skills and experience working with globally distributed teams.

Preferred Qualifications

• Experience in Markets technology and familiarity with the trade lifecycle, risk concepts (market risk measures, sensitivities, P&L explain), and common products (FX, rates, credit, equities; derivatives basics). 
• Streaming technologies (Kafka/MSK/Kinesis) and near-real-time analytics patterns. 
• Experience with governance, controls, and data quality frameworks (reconciliations, lineage, auditability, entitlements). 
• Familiarity with lake/lakehouse table formats and tooling (e.g., Iceberg/Delta concepts) and columnar storage formats (e.g., Parquet). 
• CI/CD, infrastructure-as-code, containerized workloads, and DevOps practices. 
• Prior experience leading initiatives across multiple teams (tech lead responsibilities, mentoring, cross-team coordination).