Metrics Store Summit 2022

Thank you so much for attending the Metrics Store Summit 2022! Whether or not you were able to join us live, we hope you enjoyed the discussion surrounding the metrics layer. Recordings of each session are available below.

Vision Keynote: Metrics as the language of data

Nick Handel, Co-founder and CEO of Transform will share his vision for handling metrics as code and dive deep into elements of a strong metrics framework. He’ll also look at the modern analyst workflow and how metrics layers are changing the way that data teams work and interact with business teams.

Nick Handel

Co-Founder and CEO of Transform

Building the metrics store before it was cool

Long before the concept of the “metrics layers” hit Twitter feeds and Substack newsletters, technology companies built metrics stores internally to understand their business, conduct experiments, and share metric logic. Hear from data experts at Airbnb, Spotify, and Slack on what they’ve learned along the way. We will end the mini sessions with a Q&A with all of the experts.

The magic behind Airbnb’s Minerva

Airbnb’s Minerva helps the organization manage the entire lifecycle for metrics. Minerva provides a single source of truth for metrics, and powers many data applications in Airbnb. Xiaohui Sun, Software Engineering Manager at Airbnb will cover: Why the team built Minerva, how Minerva’s architecture covers the full lifecycle of metrics, and an overview of applications powered by Minerva including reporting, analytics, experimentation, metadata management, etc.

Experimentation and metrics at Spotify

Five years ago Spotify set out to build a new experimentation platform. Their goals? To increase the quality of metrics and make analysis simpler. Johan Rydberg, Principal Product Manager at Spotify, will share learnings, including how his team is now implementing the second iteration of their internal metrics store, with the goal of standardizing metrics for all of Spotify, not just experimenters.

Metrics-at-scale at Airbnb and Slack

Aaron Keys, Senior Principal Data Scientist at Slack, will compare and contrast two large-scale metrics repository projects that he worked on at Airbnb and Slack. He will discuss the design tradeoffs that were considered by his team, successes, and mistakes made along the way. Aaron will close with views about how these types of internal projects relate to open source projects and the new companies in this space.

Engagement Metrics Framework at Facebook

Siva will talk about how the data engineering team at Facebook built a framework to compute growth and engagement metrics at scale. The talk will focus on why Facebook needed this framework, the secret sauce and the cost of not building it.

Aaron Keys

Principal Data Scientist at Slack

Johan Rydberg

Principal Product Manager, Experimentation at Spotify

Xiaohui Sun

Software Engineering Manager at Airbnb

Siva Kolappa

General Partner at Data Tech Fund

Philip Weiss

Software Engineer, Airbnb

Standing on the shoulders of metrics stores

For as long as analysts have relied on SQL, those in the data community have been trying to figure out ways to make analysis reusable. On the surface, it seems imminently possible and immensely valuable: Software is eating the world; we’re all becoming tech companies; we all use the same tools and try to solve more or less the same problems. Yet, despite that, data practitioners still have to rebuild many of the same models and metrics—from pipelines that clean up Salesforce data to dashboards of daily active users—from scratch.

Our struggles to solve this problem are explained by two phenomena: It’s neither as easy nor as useful to build analytical “libraries” as it appears. There are no frameworks for sharing analytical scaffolding, and the small distinctions in our businesses get amplified into very meaningful differences in our reporting needs. As a result, little of the analysis we create is repurposed across the community, and collective best practices only advance through the slow propagation of tribal knowledge.

The metrics layer could change all this. It, along with other recent developments in the data stack, could provide the technological foundation that finally makes community reusability possible. In this talk, I’ll share how metrics layers have the potential not just to unify a company’s metrics governance, but how they could also shape the giant’s shoulders we’ve long been trying to stand on.

Benn Stancil

Chief Analytics Officer at Mode

Measuring as making: Workshop with Stefanie Posavec

Within her creative practice, Stefanie uses data-gathering and data visualization as a design process to communicate warm, subjective and often emotional messages. She will show how in an era of ever-increasing data, we all can – through channeling our inner 'anoraks’ – start to see observing and measuring as valid forms of making and creating as well as an integral part of being human.

After her talk, Stefanie will take you through a fast-paced workshop where you will learn data visualization basics through visualizing your personal data points and learn more about yourself in the process!

Materials needed: Plain letter paper and a thick dark pen.

Stefanie Posavec

Information designer / artist / author

"Jobs-to-be-done" in Modern BI: Self-serve in a metric-first world

In a previous time, BI was the only interface to data and the data warehouse. Today, that’s no longer true. It’s allowing data teams to be more impactful to organizations than ever before. In this talk, we’ll talk about the five workflows data teams are tackling and the new technology that is enabling them, as well as what this means for “self-serve analytics” in a new metric-first world.

Emilie Schario

Data Strategist in Residence at Amplify Partners

Metrics and the Modern Data Stack

You can’t talk about metrics stores without talking about how they fit within the modern data landscape. We’ve invited experts from across the modern data stack—from experimentation and observability to business intelligence—to discuss how the metrics layer will impact analyst workflows, both in the short term and the future.

Allegra Holland

GTM Strategist at Transform

Prukalpa Sankar

Co-Founder of Atlan

Barry McCardel

Co-Founder and CEO of Hex

Chetan Sharma

Founder and CEO of Eppo

Kashish Gupta

Founder of Hightouch

Oliver Laslett

Co-Founder & CTO of Lightdash

Building an analytics product team

There has been a lot of buzz about running data teams like product teams. But once you get to this point, how do you manage the team? Borrowing management strategies from engineering organizations enables analytics teams to scale and better measure their impact on decision making. Belinda Bennett of Transform and Cynthia Beal of Lithic explore how to leverage product and engineering practices to run, measure, and grow your analytics organization.

Cynthia Beal

Analytics at Lithic

Belinda Bennett

Data Science Manager at Transform

The future of metrics tooling: Open source & enterprise

The metrics layer is a relatively new concept in the data space and naturally, different vendors are approaching this concept in new and exciting ways. Hear from vendors about their plans for the metrics layer and their vision for the future. You’ll also have the chance to ask questions about their products in our Q&A.

Dave Mariani

Chief Technology Officer, Founder at AtScale

Nick Handel

Co-Founder and CEO at Transform

Artyom Keydunov

Founder of Cube