North Star Metrics (NSMs) are key performance indicators that reflect the core value your LLM product delivers to users. They focus on outcomes rather than outputs, aligning teams around a single, measurable goal that drives long-term success and product growth.
Identify the user behavior or interaction that best represents your product’s value, such as meaningful conversations or task completions. Track this metric continuously while filtering noise from short-term signals. Use it to prioritize features, improve model responses, and balance latency, accuracy, and cost.
For AI product managers, a well-defined NSM ensures focus on user engagement and satisfaction, optimizing resource allocation. It drives scalable improvements in user experience, reduces unnecessary compute costs, and provides a unified framework to measure business impact and technical feasibility.