5 rules on how to excel at Data-Driven decision-making

‍

What do you usually do when you’re charged up with emotions, under stress and overwhelmed? Having spent 6 years on a trading floor, I’d say I have this down pat so here is what I do: 1. I step away from keyboards (we’ve all sent that angry email and regretted it right after), 2. try to remain calm and rational, and 3. I prioritise facts and data above all else. Now imagine you’re on top of a big marketing department, or a Product Manager at a fancy tech company or the CFO of a promising scale up and you are faced with the challenge of having to make important business decisions in times of uncertainty and crisis. What do you do? 

Don’t stress, here are 5 rules on how to excel at Data Driven Decision Making.

1. Data collection is a means not an end, so be strategic around data collection. Define the business need, the objective and the strategy THEN collect the information that will assist you to reach said objective. Data should drive the business not overwhelm your teams. 
   

2. Invest wisely in your data stack. Collecting data is great, but you want to ensure you have the tools in place to treat, warehouse and transform that data. Imagine you want to make a nutritious smoothie, so you go and get all the best and fresh organic ingredients. You even go one step further and start thinking of presentation ideas (thank you Pinterest) and all the magical ways this healthy smoothie is going to nurture your body. Not having a robust data stack is like not having a good blender to make your smoothie. So what does a data stack typically include?
   

• Ingestion: Load and move data from one place to another. Fivetran
• Warehousing: Store your data in one place, cloud based warehousing is the basis of the modern data stack. Snowflake, Bigquery, Redshift.
• Transforming: Make your data usable and ready for analytics. dbt, Airflow, ETLeap.
• Analysis & Business Intelligence: This is the step where data starts funneling into ML models so that business reports and dashboards start to get created. Looker, Mode, Periscope, Chartio, Metabase, Redash.
  

3. Implement Data Quality Monitoring, at scale. As you introduce new data ingestion sources (often third party), you expand to more business uses and data users and increase the complexity of your data pipelines which in turn increases the instances of missing, outdated, stale and inaccurate data. Ultimately this leads to a “garbage in garbage out” situation which could negatively impact your decision making. And let’s face it, you don’t need that and especially during tense times. 

   A Data Quality Monitoring framework should at a minimum cover the following aspects:

• Although sometimes SQL is the answer, I strongly believe any modern data quality monitoring framework should rely on some form of AI to tackle the issues at scale. 
• The framework should have a predefined set of rules specific to your business needs.
• The framework should be scalable, i.e., it should support the addition of new rules and should be able to support data quality monitoring across multiple data sources. 
• The Data Quality framework should be “actionable”, i.e. it should not only alert of anomalies in your data sets but also assist in resolution by providing more context around the data.  
  

4. Be strategic when hiring data talent, and make data observability a part of your company culture. This goes from hiring Data Scientists and Analysts that adhere to the “self service analytics” concept, but more importantly making sure you build and invest in a data platform that will allow your Data Engineers to reach their full potential. 
   

5. Make sure the data is presented in a format that speaks to the decision maker. Not only should the data be relevant and accurate, the way it is presented to the decision maker is almost as important as the message you are trying to convey. Whether you prefer the old school ppt presentation way or opt for a more interactive Tableau or Qlikview dashboard, the decision maker should be able to uncover the business intelligence in order to make a sound data driven decision.