North Star Metrics

"Like any good PM, you should track AI adoption as inputs (who’s using AI) and outputs (what business value it’s creating). You should also reward employees who are leading the charge to keep the momentum going."

"Ideally this is a specific metric, with a defined goal, that you can easily measure. Ideally it directly connects to your team’s KPI’s. Ideally it is based on hard data about the opportunity size, investment size, and a heuristic from past experiments."

"The CFO was experiencing a similar version of this problem, so we came up with the idea of establishing a north star metric: nights booked. We would use this as a common denominator for projects so we could align on where to prioritize investment. An added (and unexpected) benefit of doing this was it propelled the company's use of experiments."

"As we became convinced that the product is stable, we focused our attention more on outcome metrics. For example, since the product helps sellers book meetings, we looked at the number of meetings booked by a typical seller to ensure that we deliver the intended value. As we became convinced that the product delivers value, we focused our attention on more operational metrics, for example, how long it takes a new customer to launch."

"First, you need to figure out which features to focus on. That means you need to analyze your data to identify which of your features are most correlated with growth, and then analyze the engagement levels of those features to highlight which features aren’t getting the user love they deserve."

"One final important metric to measure is success rate. This doesn’t apply to every feature, but many features have a concept of a successful outcome for the user. For instance, what percentage of resumes submitted for a job opening on Indeed receive a call back from the employer?"

"Correlation does not imply causation, but it’s the simplest and fastest way to locate which features or metrics correlate the most with high or low user engagement. In my work, I like to run a correlation analysis first thing when I am coming fresh to the data and looking for connections between metrics."

"Linear regression takes correlation analysis further and shows how much one variable affects another and, more importantly, whether you can use the pattern of one variable to predict and estimate the behavior of another. Importantly, just like correlation analysis, a linear regression doesn’t prove causation, but it does give you more confidence that there’s a strong connection between variables."

"If you can swap X and Y and get the same result, use correlation. If changing them affects your outcome, use regression. If your analysis aims to answer if there is a relationship between X and Y, use correlation. If you aim to answer how X affects Y or have X predict Y, use regression."

"The aha moment, or inflection point, was a 3 minutes ETA. Under 3 minutes felt instantaneous. When it was over 3 minutes, you were incentivized to shop around, take the bus, walk, or take another ride sharing company. This goal gave our local teams a north star."

"Most companies use logins or app opens as main events for the “active users” definition.But given that I always aim for the cleanest and most precise data reporting, I’d recommend using the main user action as the activity event"

"As a result, we reoriented the entire company, board, and investor base to focus on the metric of weekly active companies, not ARR. Our North Star was now to get as many companies using Equals as possible, with the belief that we’d later monetize them."

"Whether you are a marketplace, SaaS, direct-to-consumer, social network, or any other type of business, you can think of your data as a company organizational chart: Revenue is the 'CEO' (it’s an outcome metric). Acquisition, retention, and monetization are part of the 'leadership team,' strategizing how to make the company successful."

"Pair the audit with data on how many people are on your website, how many go through the signup process, how many reach the 'aha' moment, and how many are successful in self-checkout. This combination of data and user experience will help identify the most significant areas of opportunity, often activation and conversion."

"- **Paid company conversion:** % of free companies that convert to paid within X days - **Payback period**: Average time to pay back CAC - **Gross margins**: Net sales revenue minus the cost of goods sold"

"Surprisingly (though not that surprisingly) Google Sheets was by far the most used tool, followed by Profitwell and Google Data Studio (sitting on top of Google Sheets)."

"As you learn more about what levers most impact the health and growth of your business (e.g. retention, margins, acquisition, etc.), focus on those. Try to narrow in on the two or three metrics that most directly drive your business’s success. I’ve learned this lesson many times over: greater focus leads to greater impact."

"If fill rate is the ultimate measure of marketplace health, bookings is the best way to track marketplace growth. Unlike GMV (which is also incredibly important, as you’ll see below), bookings removes confounding variables like average order values, pricing changes, and outlier purchases. It directly tells you whether your marketplace usage is growing or shrinking."

"If you’re launching a brand new independent product, or pivoting the product, you likely have nothing to compare this new product against (other than it *not* existing). In this case, you’re better off just setting independent success criteria (e.g. a specific retention rate, or a new user signup threshold), vs. coming up with an awkward experiment."