In this third blog we focus on the importance of - and the capabilities required - to stitch a narrative together that delivers actionable insight.
Data narratives require three things, one of which, as technology stands today, can only come from human beings:
- Quality data from both qualitative as well as quantitative sources
- A good understanding of mathematics, in particular, concepts such as probability, correlation vs. causation, significance and margins of error
- A good understanding of the context
Every organisation has data, both quantitative and qualitative, but few organisations are able to use this data to create insight. Whilst there are a range of organisational challenges to producing insight, it is also common for an organisation to struggle with a more fundamental issue in that the quality of the data is simply not of a standard to enable effective insight generation.
Data “quality”, in simple terms, is the level of trust that an organisation should have in its data and in most cases the data that is available is of very low quality, failing to meet one of more of the key quality criteria below:
- Accurate – the data has to be correct; it is not helpful to report traffic volume as x if it is actually y
- Unbiased – are there any biases to the data being produced, are teams within the organisation marking their own homework and if so, do they have an agenda that they are looking to support with their data?
- Reproducible – can you reproduce the data that has been produced, if we cannot replicate a data point, we don’t use it
- Directional – is there evidence of an increase or decrease over time. For example, reporting conversion rate for a single period of time is less useful than reporting over time, is it going up or down?
- Explained - what exactly is being reported, for example how do you define conversion rate; are you measuring transactions, orders or customers against users or sessions?
Many organisations produce data but without a consideration of these five criteria, the result being is that the data simply is not good enough to produce actionable insights.
If data quality is the level of trust an organisation should have in its data, mathematical competence is a necessary skill for assessing data quality. Organisations that produce a lot of data may not necessarily have an understanding of the nuances of that data, and so the “quality” of that data is reduced.
Two common examples of mathematical competence, or lack thereof are detailed below:
- Average order value – every E-Commerce organisation reports average order value, but “average” can mean one of three things, the mean average (total revenue / number of orders), the modal average (the most common order value) or the median average (the middle order value). By focusing solely on mean average, organisations miss the real picture; for example if five orders were made at £20 and a sixth was made at £150, the average order value (as typically calculated) would be £41.67 but clearly this does not represent real customer behaviour
- Statistical significance – much like average order value, statistical significance is a common mathematical concept with the most common application being AB testing whereby 95% “statistical significance” is the target level of significance to declare a winning variation. However, statistical significance is only part of the story. Due to the means of calculation, statistical significance only provides an overall view of performance throughout the test period, it cannot factor for variation in performance over time. It is entirely possible, and often the case, that the “statistically significant” result was generated due to an anomalously high performing day for a given variation. This anomaly biases statistical analysis towards the variation and so a “statistically significant” test result may in fact simply be a fluke.
The above are two common examples of where mathematical competence is required to validate data despite the data being factually sound. Without this level of familiarity with the nuances of data and, more importantly, what exactly is being reported, organisations simply cannot produce actionable insights no matter the breadth of data available to them.
Whilst machine learning can deliver both the above, it cannot bring insight on areas for which it wasn’t programmed, it cannot apply experience of which those who programmed it did not have and apply an understanding of motivations for behavioural change drawn from qualitative responses that reads ‘between the lines’. This last part is what differentiates human beings, indeed any sentient being, from something that is programmed to process data points.
In scientific endeavour this last part doesn’t matter. Nature has patterns and mass data processing can help identify these far more rapidly and with far greater accuracy than any human-led activity. In the analysis and understanding of human behaviour, however, the basic irrationality of human beings, their whimsicality, their sheer genius for change is, for now at least, beyond any artificial processing that is available to commercial leaders – whatever the marketing claims made.
Given the importance of the narrative in e-commerce, leaders increasingly are looking to find platforms, solutions or approaches that can deliver actionable insight. A level or degree level mathematics is a rare skill-set in marketing and commercial disciplines. Related disciplines that can be useful such as econometrics and statistics equally so. This means that building in-house data narrative solutions is challenging.
Buying expertise has until now, been equally difficult and when found, often expensive as it is often embedded in consulting or technical services.
Creating 4Front has allowed us to offer data narratives as a stand-alone product and to do so with a human-led proposition that ensures ‘context’ and the ability to ‘read between the lines’ and connect the dots to create a full picture.
For anyone looking for advanced mathematics, assured data quality and thoughtful, careful interpretation then 4Front offers it all.