Best Practices Summary:

• Show the value of analytics to leaders to create important allies and support investments

• Focus on the metrics that matter the most by utilizing the HEART framework to analyze product user experiences 

• Even if measures appear to be “vanity metrics”, produce scorecards and dashboards to build perceived value and trust with stakeholders 

Case Study:

An online travel shopping company marketed the lodging inventory of partners, but each promotion was done by market managers based on their opinions rather than data. Lodging partners such as hotels often had excess room inventory that went unsold. The partners, company, and market managers could all benefit when sales conversion increased. 

The Director of Data Science addressed the need for more insight by aligning closely with the business and then providing analytics to help meet growth goals. He found a sympathetic Vice President with whom he socialized the value proposition for analytics on their products. His support was crucial in aligning on the metrics and leading indicators of business outcomes like partner bookings and sales. An analytics program was developed with a regularly updated scorecard that provided baseline measures such as monthly active users. A recurring cadence of engagement with the business was started to help uncover additional analytic needs and opportunities. The engagement included weekly standups, analytic insight newsletters, and monthly forums that included a readout on key investment areas. These vehicles helped increase demand for insights further. To help home in on the most important metrics, the HEART method, originally developed at Google, was applied. The HEART method helps evaluate product user experiences around five areas including Happiness, Engagement, Adoption, Retention, and Task success. The company chose a maximum of three goals, signals, and metrics for each of the five areas. The method helped with focus on which metrics were truly most important and informative for decision-making on the product design and features. The dashboard for HEART metrics was launched and its use was tracked. At first there was little uptake, however with inclusion in regular newsletters, supportive messages from leaders, scheduled discussions, promotion over Slack, and publication of a wiki that described the metrics use of the HEART dashboard dramatically increased. 

The combination of business and product team engagement along with production of analytics focused on the most important metrics for product improvement helped to increase analytics adoption at the company. The partner portal benefited as market managers were able to focus their efforts on lodging inventory that could most easily be sold through promotions. The data-driven decision-making helped to increase bookings and revenue for the partners and company and, helpfully, commissions for market managers.