Best Practices Summary:

• Use an organizational model for analytics where individuals are embedded in the business and have accountability back to a central analytics group

• Develop a cross-business team within the central group with a structure that facilitates analytics manager and peer consultation on the work to improve it

• Apply the Five Whys technique to enhance understanding of the needs of the business and how analytics can best provide value-added and actionable insight 

• Utilize the right level of analytics for the business problem - analyses don’t have to be extraordinarily sophisticated to provide value

Case Study:

The payments division of a large marketplace company sought to ensure that analytics better influenced product design, features, and performance. Many product companies struggle to harness analytics effectively as part of product design and feature development by engineering teams. On the one hand, analytics team members are often too far removed from the business to understand what insights would be most helpful. On the other hand, when analysts are embedded in the business unit, they may have a better understanding of business needs but lack best practices in analytics. 

The Head of Global Data Science addressed the challenge of influencing product design with analytics through both organization design and work management techniques. From an organizational design perspective, he deployed an organizational model with analytic resources that are embedded in the business yet have performance accountability back to his central analytics group. With this model, analytics team members worked closely with product teams enabling them to ensure relevance of analyses. However, since product teams often lack experience leveraging analytics they could often ask for data and insights that had suboptimal value (e.g., ‘vanity metrics’). The central analytics group provided a structure for improving the value of analytics through work management techniques. These included the use of a weekly standup, quarterly retrospectives, and feedback sessions all facilitated by an analytics professional. In the feedback sessions, a particularly useful approach was to ask the Five Whys, which is an interrogative technique that helps participants explore and uncover underlying problems and needs.  This helped drive insight and accountability on the analytics teams to solve real challenges that drove value for the product. For example, one of the data scientists on the team worked with the payments group that sought to improve purchase checkout efficiency. The data scientist asked why payments failed initially while a portion of them later went through. By asking and analyzing why customers were able to resolve failed payments themselves he noticed that the successful customers usually simply tried again. To address this opportunity, a machine learning model was implemented that automatically tried payments again when they failed dramatically increasing the overall payment success rate without burdening the customer. 

The combination of an effective organization design and analytics team management techniques led to increased adoption of insights as their relevance, quality, and value increased. The business was able to grow the impact of their product and associated revenue through application of these winning insights.