Find x. Make it work.
I'm a software engineer and data scientist. Before that, I spent ten years in mathematical physics, which gave me plenty of practice getting unstuck on difficult problems.
Current work
At CARIAD, the Volkswagen Group's software company, I was brought in to an existing vehicle-data analysis effort to develop more advanced analyses and explore where machine learning could help.
That work quickly exposed limits in data processing and architecture. Making those analyses practical at petabyte scale became a larger software and data effort, and within six months a dedicated team was formed around the work I was leading. I mentored the team as they learned to work with Spark and vehicle data at that scale.
I set the initial technical direction, designed the architecture, and built the first data pipelines and interactive analysis tools. As teams needed more from the platform, I kept taking the lead on problems that called for a new approach. That included reworking earlier solutions when they were no longer enough. The platform is now developed by four teams, supporting recurring analyses across multiple Volkswagen Group teams.
Although my main focus is now newer projects built on and around the platform, with and without AI/ML, I remain involved in wider design decisions across the teams developing it.
From bespoke analysis to a reusable dataset
I designed a system that turns many terabytes of raw recordings each day into a reusable analytical dataset. Previously, new questions meant inspecting recordings in specialist software or writing bespoke analyses that decoded the raw data again. The ingestion path now decodes binary measurement formats, network protocols, and software logs once, retaining as much data as practical. This costs more processing and storage up front, but lets investigations begin with SQL over the decoded history.
At this volume, unnecessary reads and copies add I/O and memory costs. I rebuilt the critical path in Rust as a single-pass extractor, tuning the balance between serial and parallel decoding for end-to-end throughput.
On top of the resulting tables, we built an interactive application for ad hoc questions, while recurring analyses run as optimized SQL queries on a schedule. Partitioning and clustering reduce reads by letting the query engine skip data that cannot match a filter.
Replacing the existing tools was a substantial part of the work. They were trusted and well suited to the bespoke analyses they had been built for. The new path introduced Rust, unfamiliar protocols and binary formats, and additional storage costs. We had to build confidence in the extractor's correctness and help engineers adopt the new tooling.
The extractor now converts data faster and with less compute than the previous path. As other engineers learned to extend it, we became less dependent on the old tool. Maintaining that tool alongside the new extractor eventually became a liability, and it was deprecated.
Learning together
At CARIAD, I started a Pythonista club within my team that grew into a biweekly seminar attended by colleagues from across the Volkswagen Group. I also bring colleagues together for sessions about Spark, Git, and other tools and design decisions we encounter in our work. I have also co-organized our participation in Advent of Code.
Apparently, leaving academia did not mean giving up teaching.
A return to software
I wrote software long before I became a mathematical physicist, freelancing alongside my studies and contributing to open source. I then spent ten years in research and teaching, working on mathematical physics and quantum field theory, and later on deep learning.
I used to spend too long looking for the best approach. Research helped me trust my judgment about when I knew enough to begin, even if the work would eventually lead me to rethink the approach. Teaching and supervising students were part of that work too. In 2022, I joined CARIAD as a data scientist, bringing software development back to the centre of my daily work.
- to now
Senior Data Scientist
Developing analytical methods and building the processing systems needed to apply them to vehicle-development data at scale.
- to
- to
Freelance developer
Alongside my studies
Building web applications and contributing to open source when Ruby on Rails was still new.
Outside work
I live with my wife and our two children in Berlin. I have tried an unreasonable number of sports. I miss cross-country skiing, rowed for years, and enjoy bouldering despite disliking heights. I am also a good cook.
I sometimes get carried away with Advent of Code. In 2020, I used it to learn Julia and ended up reading LLVM IR, writing explicit SIMD, and injecting a little IR directly to see how fast an idiomatic solution could get.