Do Metrics Under Deliver?
Contributed by Tom HuntingtonOn the second Friday of each month Neil Baron assembles a handful of product executives to share best practices. Last month the ranks swelled to three-dozen executives as we discussed product metrics.How does the seemingly simple task of measuring progress get so distorted? The following themes emerged.Create responsible ground rulesWe're biased toward self-congratulation. Neil kicked off the discussion with the Dilbert cartoon above. Should metrics reporting be moved out of the Product organization to an impartial group like Finance? Such impartiality comes at a cost. The goal is to improve behavior, not just generate data. If the Product organization does not feel ownership in the metrics, it will not use the analysis. Responsible ground rules can ensure effectiveness, regardless of who tallies the metrics.
- Buy-in: The Product organization should have an active voice in designing metrics so that it takes ownership of results.
- Context: With proper context, metrics become less subjective. Two useful contexts are historical trends and prior expectations of current results.
- Transparency: Share results and allow for comment. Sunlight is the best disinfectant.
Share commitmentShould we assign a single owner for each metric? Proponents argued that a single neck to choke is necessary to produce results. Opponents pointed out that each metric is part of a larger chain that impacts others. Narrow ownership invites misalignment. In the pursuit of overall sales, salespeople will overlook new product introduction. In defense of the status quo, a product owner will fight creative destruction. Producing results is table stakes, but teams that share commitments can be more strategic.Identify the resulting actionSeveral executives distinguished between tactical and strategic metrics. For example, defect rates may be tactical while new customer penetration is strategic. However, this classification is not stable. If product defects are causing customer attrition then defects are highly strategic. The importance of each metric is relative to the resulting action. If defects rise above 2%, then a resulting action could be to shift a resource from Development to Quality Assurance. This resulting action brings relevance into focus. With this context, the team can then figure out the right metrics, meaningful thresholds and who to involve.The right metric gives you leverage
The group ended with a brief discussion of specific metrics themselves. What are the best metrics to track? My personal favorite is referral rate. When referrals are high, you have a virtuous cycle. However this answer is no better than off-the-shelf metrics like revenue and cash flow. The right metric is specific to each company. It should describe the unique economic engine of your firm. In his book “Good to Great,” Jim Collins defines the magic number as a denominator because it drives desired results like revenue and cash. The right metric gives you leverage.Referral rates are high at the Product Executives Forum. Enough people were seated along the walls that it’s not exactly a roundtable any more. Nonetheless in Neil’s capable hands, the sessions remain rich and intimate in their pursuit of better product management.
Tom Huntington is co-founder and CEO of Windshield, collaboration for purpose-driven teams. [Ed. note: Additional thanks to Tom for all images in this post.]