Best Practices Summary:

• Treat data analytics solutions like products by applying best practices from product management philosophies and principles

• Understand user personas, including their use cases and motivations

• Leverage a project brief at the start of any project to articulate the problem statement, hypotheses, input data signals, expected outcomes, and expected business impact. Include stakeholders in the development of the brief document.

• Use telemetry (e.g., engagement metrics, attribution) on the use of dashboards and reports to approximate the impact of the analytics on the business. 

Case Study:

A large technology company with a digital advertising platform sought to increase their advertising sales revenue while reducing third party advertiser churn. A central advertising data analytics team was tasked with providing insights to sales, marketing, operations, strategy, and partnerships teams to help drive growth through insights. However, there was often limited adoption of insights by these teams in the past. In addition, when analytics were used for decision-making, it was often difficult to attribute the impact of analytics on business results.

The Director of Data Analytics for Advertising sought to increase adoption of analytics and insights by treating analytics as a product and applying product management principles. His data analytics team performed three functions: a) data pipelines and storage, b) business intelligence applications, and c) analytics and insights. These teams shifted their focus from analytics to better understanding business needs and their internal customers. The product mindset helped guide them to start their analytics work by understanding the user personas. A helpful approach for getting a better understanding of these personas was using the Five Whys which is root-cause analysis technique/framework that helps participants explore and uncover underlying problems and needs. The insights from user persona cohorts helped them complete a project brief that articulated the problems, hypotheses, data signals, solutions, and expected outcomes. By writing the project brief and then getting signoff from the internal customers both alignment on what was needed and commitment to use the analytics increased. Once a project reached the briefing stage it was assigned to a product backlog. The product owner pitched the value and feasibility, and it was added to the team’s roadmap.

While it remained challenging to understand the impact of analytic insights on the business directly, a good proxy was measurement of the use of dashboards and reports. For example, data scientists developed advanced machine learning models that were surfaced via dashboards. They could measure their adoption by tracking dashboard engagement metrics via telemetry. The number of potential users was identified to serve as an ongoing comparison with active users. When monthly active users or the percentage of active vs. potential users declined it meant either exploring improvement opportunities or deprecating the analytics/reporting entirely to free up capacity for higher impact work.

As a result of the product management-based approach, there were substantial increases in the use of analytics by the sales, marketing, operations, strategy, and partnerships functions. The insights helped contribute to increases in sales revenue as selected functions could better perform activities such as optimizing ad placement, targeting advertisers with pricing and offers to reduce churn, and running the advertising platform more efficiently.