SWE, Data Ingestion

Before the detail, here's the challenge you'd help us solve.

We build the embodied intelligence that moves real vehicles safely, and the ecosystem a billion machines will run on in the future. Very few people in AI can say this. Every role here, whatever the team, plugs into that.

Here’s what this particular role covers.

The Role

We are looking for a Data Ingestion Engineer to help build and strengthen the data foundations behind Wayve’s self-driving technology.

At Wayve, we do not hand-code cars to drive. We train them to drive from data. That makes data ingestion one of the most important parts of our learning system. The faster, more reliably and more intelligently we can process real-world driving data, the faster we can improve our models and bring embodied AI into the world.

This is a hands-on permanent role for an engineer who enjoy solving practical, high-impact problems at scale. You will help keep our ingestion pipelines running smoothly, unblock critical data flows, and contribute to the long-term evolution of the systems that support annotation, data science, model training and evaluation.

Our data platform operates at significant scale, with over 500,000 hours of driving data, equating to 100’s of PBs. As our ADAS and autonomy work grows, we need ingestion systems that are robust, efficient and cost-effective. A single bad data segment can block a pipeline, build up queues and slow down downstream teams, so this role has a direct impact on how quickly Wayve’s AI can learn.

Key Responsibilities

You will work within the Data Ingestion team to improve the reliability, efficiency and throughput of the pipelines that move real-world driving data through Wayve.

- Debug and resolve failing or blocked ingestion pipelines.

- Investigate issues caused by corrupt, malformed or unexpected data.

- Design and implement more resilient pipelines so individual bad data segments do not block wider workflows.

- Improve how we handle varied data formats from partners, suppliers and third-party sources.

- Support orchestration across multi-step ingestion workflows, including dependencies, retries and queue management.

- Optimise Spark jobs and data-processing pipelines for throughput, compute efficiency and reliability.

- Reduce operational toil around failed jobs, stalled pipelines and manual interventions.

- Work on high-volume batch-processing systems where throughput, reliability and cost all matter.

- Help prioritise and unblock important datasets for downstream annotation, data science and model training teams.

- Partner with engineers across Data Platform and downstream teams to deliver both immediate improvements and scalable long-term solutions.

- Contribute to the technical direction, maintainability and operational excellence of Wayve’s ingestion platform.

About You

You are an experienced Data Engineer, Platform Engineer or Distributed Systems Engineer who enjoys working on large-scale production data systems.

You have seen how data pipelines behave in the real world: messy inputs, strange edge cases, corrupt files, stalled queues, unexpected formats and failures that only appear at scale. You are comfortable digging into those problems, finding the root cause and making systems better as a result.

You combine strong technical depth with a practical, collaborative approach. You can take ownership of complex systems, work effectively across teams and balance urgent operational needs with thoughtful, durable engineering improvements.

Essential

- Strong production experience with Apache Spark.

- Strong Python engineering experience.

- Experience building, debugging or operating large-scale data-ingestion, ETL or data-processing pipelines.

- Experience with distributed data-processing systems.

- Ability to optimise jobs for throughput, compute efficiency and reliability.

- Experience debugging production pipeline failures.

- Comfort working with messy, corrupt, incomplete or inc …