From bottleneck to backbone: scaling experimentation as developers speeds up

“Engineers are shipping faster than ever. AI agents like Claude and Gemini have changed what a single developer can deliver in a day. When the delivery of engineers speeds up, the entire product lifecycle needs to keep pace. Delivering features without experiments to measure their impact is not just risky, it also prevents us from learning what customers truly want. Experimentation allows us to learn incrementally and with purpose. And since engineers can deliver twice as fast, shouldn’t we be able to learn twice as fast too?

The catch is: experiment analyses take much too long, and analytics became the new bottleneck in the product lifecycle. But value is created under friction. People from different teams, different domains and different crafts, most of whom had never worked together, came together around that shared problem. We made use of our complementary skills to create a faster way to evaluate experiments. And more specifically, a way where not just analysts, but anyone can read the results and act on what they reveal.

What we built is an experimentation analytics tool that uses AI to generate readable reports from raw results, turning an analysis that used to take days into something anyone in the product team can pull up and act on. This talk is about that problem, the unlikely team that formed around it and how we turned each other’s specialisms into something valuable for the entire product and tech domain. And, in the end, for our customers.”