Best Practices Summary:

• Don’t underestimate the percentage of portfolio effort and time that will be dedicated to continuously building and maintaining "basic" situational awareness of internal customer expectations, and the value it brings with broader community data literacy and adoption of more complex solutions down the road.

• Pace capability and capacity to stakeholder expectations. Invest in sufficient automation so that the technical team can keep up when stakeholder expectation ignites.

• Don't fall into the trap of assuming that impact value is implied and understood.  Data analytics teams must invest in value translation.

Case Study:

A video game publisher’s growth analytics team was focusing their efforts on projects they identified as high priority and high impact, and as a result began bypassing some projects they considered “vanity metrics”.  “Vanity metrics” meant data and reporting that helped teams feel good about their success but didn’t lead to any changes in their strategy or decision-making. A game studio working with the publisher was preparing for their big product launch moment after two years of effort fine-tuning everything from game play optimization to art and storyline.  With a previous launch, the growth analytics team had provided a real time monitoring dashboard for day one installs, however this time no one at the game studio explicitly requested it and the analytics team did not prepare one given their effort to avoid “vanity metrics” and focus on high impact metrics with clear growth and sales attribution.  The game studio was disappointed, and the publisher risked losing their trust.

The growth analytics team quickly realized that they had underestimated the value of dedicating sufficient time and resources into building and maintaining basic situational awareness and trust through high level dashboards and KPIs, even when they aren’t tightly aligned with actionable insights.  The director needed the game studio team’s trust only a few weeks later when he wanted to go forward with building algorithms that fed personalization of content in the gaming app. The algorithms aligned the right message with the right user to drive up engagement and follow-on sales in an automated way.  While the personalization was anticipated to produce value, the game studio didn’t trust the analytics team because they “couldn’t even do a basic launch dashboard”.

Recognizing the importance of regaining trust, the team stopped everything on the more advanced, complex analytics and algorithms just four days before launch, affirming the game studio’s need for a launch dashboard.  While the metrics monitored may not have been directly actionable, providing the dashboard built trust and adoption interest, and provided a medium for the team to celebrate and share their enthusiasm about reaching launch and watching their new game reach the hands of gamers.

When the analytics director came back to the game studio a few weeks later to present the personalization algorithms they quickly aligned and got onboard with proceeding, riding the wave of trust and excitement from the recent launch and the support the analytics team had provided at crunch time.  The analytics team now thinks more widely than just business impact, also weighing trust building, adoption, engagement, peace of mind, and increases in data literacy when identifying priority and impact of a project.  It may be less measurable, but the feeling matters and can unlock the potential for more measurable KPIs as a next step.  They started by giving people wins with basic dashboards and technical deliverables, built the relationship, and with time when the more controversial, less intuitive proposals of complex analytics come along stakeholders were ready to move forward.  With this approach the game publisher growth analytics team continued to improve adoption and partner with stakeholders on advanced solutions that look across their portfolio for opportunities to maintain and increase customer engagement throughout the year.