Metrics Store Summit 2022
Thank you so much for attending the Metrics Store Summit 2022! Whether or not you were able to join us live, we hope you enjoyed the discussion surrounding the metrics layer. Recordings of each session are available below.
Vision Keynote: Metrics as the language of data
Nick Handel, Co-founder and CEO of Transform will share his vision for handling metrics as code and dive deep into elements of a strong metrics framework. He’ll also look at the modern analyst workflow and how metrics layers are changing the way that data teams work and interact with business teams.
Building the metrics store before it was cool
Long before the concept of the “metrics layers” hit Twitter feeds and Substack newsletters, technology companies built metrics stores internally to understand their business, conduct experiments, and share metric logic. Hear from data experts at Airbnb, Spotify, and Slack on what they’ve learned along the way. We will end the mini sessions with a Q&A with all of the experts.
The magic behind Airbnb’s Minerva
Airbnb’s Minerva helps the organization manage the entire lifecycle for metrics. Minerva provides a single source of truth for metrics, and powers many data applications in Airbnb. Xiaohui Sun, Software Engineering Manager at Airbnb will cover: Why the team built Minerva, how Minerva’s architecture covers the full lifecycle of metrics, and an overview of applications powered by Minerva including reporting, analytics, experimentation, metadata management, etc.
Experimentation and metrics at Spotify
Five years ago Spotify set out to build a new experimentation platform. Their goals? To increase the quality of metrics and make analysis simpler. Johan Rydberg, Principal Product Manager at Spotify, will share learnings, including how his team is now implementing the second iteration of their internal metrics store, with the goal of standardizing metrics for all of Spotify, not just experimenters.
Metrics-at-scale at Airbnb and Slack
Aaron Keys, Senior Principal Data Scientist at Slack, will compare and contrast two large-scale metrics repository projects that he worked on at Airbnb and Slack. He will discuss the design tradeoffs that were considered by his team, successes, and mistakes made along the way. Aaron will close with views about how these types of internal projects relate to open source projects and the new companies in this space.
Engagement Metrics Framework at Facebook
Siva will talk about how the data engineering team at Facebook built a framework to compute growth and engagement metrics at scale. The talk will focus on why Facebook needed this framework, the secret sauce and the cost of not building it.
Standing on the shoulders of metrics stores
For as long as analysts have relied on SQL, those in the data community have been trying to figure out ways to make analysis reusable. On the surface, it seems imminently possible and immensely valuable: Software is eating the world; we’re all becoming tech companies; we all use the same tools and try to solve more or less the same problems. Yet, despite that, data practitioners still have to rebuild many of the same models and metrics—from pipelines that clean up Salesforce data to dashboards of daily active users—from scratch.
Our struggles to solve this problem are explained by two phenomena: It’s neither as easy nor as useful to build analytical “libraries” as it appears. There are no frameworks for sharing analytical scaffolding, and the small distinctions in our businesses get amplified into very meaningful differences in our reporting needs. As a result, little of the analysis we create is repurposed across the community, and collective best practices only advance through the slow propagation of tribal knowledge.
The metrics layer could change all this. It, along with other recent developments in the data stack, could provide the technological foundation that finally makes community reusability possible. In this talk, I’ll share how metrics layers have the potential not just to unify a company’s metrics governance, but how they could also shape the giant’s shoulders we’ve long been trying to stand on.
Measuring as making: Workshop with Stefanie Posavec
Within her creative practice, Stefanie uses data-gathering and data visualization as a design process to communicate warm, subjective and often emotional messages. She will show how in an era of ever-increasing data, we all can – through channeling our inner 'anoraks’ – start to see observing and measuring as valid forms of making and creating as well as an integral part of being human.
After her talk, Stefanie will take you through a fast-paced workshop where you will learn data visualization basics through visualizing your personal data points and learn more about yourself in the process!
Materials needed: Plain letter paper and a thick dark pen.
"Jobs-to-be-done" in Modern BI: Self-serve in a metric-first world
In a previous time, BI was the only interface to data and the data warehouse. Today, that’s no longer true. It’s allowing data teams to be more impactful to organizations than ever before. In this talk, we’ll talk about the five workflows data teams are tackling and the new technology that is enabling them, as well as what this means for “self-serve analytics” in a new metric-first world.
Metrics and the Modern Data Stack
You can’t talk about metrics stores without talking about how they fit within the modern data landscape. We’ve invited experts from across the modern data stack—from experimentation and observability to business intelligence—to discuss how the metrics layer will impact analyst workflows, both in the short term and the future.
Building an analytics product team
There has been a lot of buzz about running data teams like product teams. But once you get to this point, how do you manage the team? Borrowing management strategies from engineering organizations enables analytics teams to scale and better measure their impact on decision making. Belinda Bennett of Transform and Cynthia Beal of Lithic explore how to leverage product and engineering practices to run, measure, and grow your analytics organization.
Cynthia Beal
Analytics at Lithic
Belinda Bennett
Data Science Manager at Transform
The future of metrics tooling: Open source & enterprise
The metrics layer is a relatively new concept in the data space and naturally, different vendors are approaching this concept in new and exciting ways. Hear from vendors about their plans for the metrics layer and their vision for the future. You’ll also have the chance to ask questions about their products in our Q&A.