Driving a Culture of Data Science

Data Science is having its ‘day’ in the sun. This strange stew of computer science, algorithmic skills, hacking abilities, statistical inferencing, probabilistic modeling and scientific enquiry which started off as a rebranding of statistical work became a bona-fide job title only in 2008. Yet, it is now the ‘sexiest job of the 21st century’. Amidst all the hoopla there is a lot of confusion on what constitutes a data scientist. However tempting it may be to try and define a high-priesthood of data science, the fuzziness comes from the fact that different businesses require and therefore rightly organize themselves around different flavors of data science. However we choose to define it, data science requires a fundamentally interdisciplinary mindset to problem solving. Data science at MSD is centered around building data products using machine learning, statistical inferencing and modeling of domain knowledge. Some musings on our data science culture –

A reliable data platform is the cornerstone of an effective data science team. Our data platform instruments every product with usage data, pipelines and consolidates them strategically at different fidelities for use in data science workflows. Careful deliberation about scale is a key player in our data architecture. Data comes in different shapes and sizes and the platform must open doors for them all.

Building a data-driven culture early on and embedding this deep into the DNA of the company is important for effective machine learning and data science. Data Science projects are characterized by weak contracts and experimental iterations defined by measures of accuracy and variability and various tradeoffs. A process of scientific inquiry and decision-making must be followed to allow the algorithms to evolve.

Decades later this is still relevant to data science today. We begin by building an empirical model which makes generalizations about the environment it operates upon based on evidence and prior knowledge, then revises these beliefs under the right conditions.

In order to extract true signals from a firehose of various modalities of data including visual, textual, behavioral etc. coupled with human expertise we must understand the underlying semantics of entities and their interconnections. Building abstractions of knowledge and connecting the causal dots is key to mining the patterns that will inform our data products.

Unless you’ve been living under a rock lately you would have heard of Deep Learning — the Jack Bauer of AI, playing a key role in many of the recent technical milestones such as the AlphaGo win. Often missing from popular commentary, the role of reinforcement learning, for instance Monte Carlo Tree Search in AlphaGo. Building intelligent agents that rationalize and course-correct on the fly is a key piece of the AI puzzle.

A personalized ‘nudge’ is the de-facto icing on all our data products. Personalization, when done right, serves as a veritable force multiplier, especially in e-commerce where multiple websites are vying for customers’ limited attention. However it can quickly devolve into a caricature without a deeper conceptual understanding of customer needs. Enabling delightfully surprising yet relevant discovery is a key objective of our algorithms.

A predictive modeling mindset is necessary to detect the seasonality and trends intrinsic to the data which in turn paves the way for proactive rather than reactive data products. Running blind without sensible forecasting for foreseeable spikes and expected seasonal behavior leads to enormous missed opportunities for conversion. For instance — running controlled experiments during festive seasons resulting in significant loss of revenue.

In order to transform data into product behavior we must craft a compelling story to explain the data in a meaningful way. Data storytelling is so crucial that it is often said to be the Data Scientists’ ‘real job’ — a gross oversimplification but nonetheless important. Transcending counting and finding the compelling narrative that lies beneath is a major part of our data products’ life-cycle.

These are some of our thoughts in the ‘path to less wrongness’. A year and a half in the trenches — this is still early days for our data science team. As our data evolves so will our beliefs and sweeping generalizations. Watch this space for more.

Related posts:

Yelp Review is a component of Yelp.com, the largest review stage on the web. The Yelp review is about how buyers rate the organization and offer their evaluation of items, administration or even…

Every day you open up your browser and navigate to your favorite website — Google. You go to this page by typing in a url or clicking on a link to the page. Have you ever thought about what happens…