Innovation in the age of big data

Dave Patterson looks at how big data can drive disruptive innovation in financial services, and how it is the established companies who have the biggest potential advantage.

At the height of the Crimean war in 1854, a ‘frail young woman’ called Florence Nightingale arrived at the Selimye Barracks in Turkey. Accompanied by her team of volunteer nurses, she encountered the full horror of 19th century military medical care, where insanitary conditions and diseases such as cholera caused ten times more deaths than battlefield wounds. Her team’s work saw the death rate reduce from 42% to just 2%, and she quickly became a hero of her day, acquiring the famous epitaph, ‘the lady of the lamp’, and her legacy as the founder of modern nursing was assured.

However, her most innovative and transformative work happened after the Crimean war, when she spent many years gathering data from the British army in India. By combining large amounts of data gathered over many years and creating new ways of visualising the data (Chart 1), she demonstrated that by setting out clear objectives, designing your approach fastidiously and collecting sufficiently accurate data, the results can be nothing less than revolutionary. Thus she was in many respects a founder and pioneer of a very modern concept, driving innovation through the use of big data. 

Chart 1: Diagram of the causes of mortality in the army in the east
Chart 1: Diagram of the causes of mortality in the army in the east
Source: https://upload.wikimedia.org/wikipedia/commons/1/17/Nightingale-mortality.jpg

In the 19th century gathering data was the primary challenge, fast forward 150 years and the data available has multiplied to staggering and sometimes incomprehensible levels. The ubiquitous nature of technology means our lives are now recorded on a dramatic scale. Data such as location, velocity, even vital signs and sleep patterns are being recorded constantly through our ever-present smartphones. In addition, in the future self-driving cars could soon be generating as much as 4,000 gigabytes of data per hour.1 IBM estimated in 2012 that the planet is producing 2.5 exabytes (2.5 billion gigabytes)2 of data every single day.

Whilst for a long time storage and transmission of this data was the biggest headache, various innovations including cloud computing have eased this concern, and now the challenge is analysing and interpreting this data in order to drive potentially highly disruptive innovation. Although the quantities of data may have dramatically increased, the principles and approach Nightingale perfected nearly two centuries ago remain key to harnessing this new commodity and driving successful change.

As is often the case, it has been the rise of another technology, artificial intelligence (AI) and machine learning, which has produced the paradigm shift in the big data revolution, and it has been the changing nature of the data captured which was a primary catalyst for this change. Where once data was well structured (e.g. Mr, Mrs, name, age) and predictably captured, thanks to social media and the Internet of things (think: search queries, photos on social media or sensors in a nuclear reactor), it is now becoming largely unstructured and continuous.

Four companies – Amazon, Google, Facebook and Microsoft – have most effectively shown the value of big data, often dubbed ‘the new oil’, and have shown how effectively it can be monetised. Jeff Bezos of Amazon points to machine learning as the real stand-out performer for Amazon. Whilst Echo, Amazon’s voice assistant, and Prime Air, their drone delivery service, are stealing the headlines, behind their success is a tremendous amount of machine learning: 

‘Machine learning and AI is the horizontal enabling layer. It will empower and improve every business, every government organisation, every philanthropy – basically there’s no institution in the world that cannot be improved with machine learning.’3

At Amazon, the quiet revolution is in the improved search results, improved product recommendations as well as inventory planning and forecasting which have been only made possible by the vast amounts of purchasing data that the company now has access to.

Whilst it is tech companies which are leading the way in the field of big data, other industries can also provide inspiration and ideas to the financial services industry. The pharma industry has long had the challenge of dealing with massive amounts of unstructured data. From diseases identification and personalised treatment through to drug discovery and clinical trial analysis, the possibilities for innovation are legion. Even beyond the data already held by the drug companies themselves, there is freely available data which can also be tapped into. A company called ProMed is tapping into information openly available on social media and the Internet to anticipate and predict likely outbreaks of epidemics across the world.4

In financial services, big data has probably seen most traction around fraud detection and prevention. American International Group (AIG) uses big data and data visualisation to help fight fraud.5 By taking both structured and unstructured data from claims databases and handwritten adjuster notes it can provide operators with a priority list of claims to investigate.6

Some of the most innovative uses are in areas of customer services and retention. American Express has been using sophisticated predictive models to analyse historical transactions, monitoring up to 115 different variables to forecast future churn. They now believe that they can identify with a good degree of certainty 24% of the accounts that will close within the next four months.7

However, the real opportunity could be found in the intrinsic value of the data itself, as a commodity which can be sold to other companies. Google and Facebook started out using data to match advertising to page content or explicit user preferences. Companies are starting to realise that they identify implicit user presences and work out an individual’s preference for certain products and services based on many more and varied data points.

For this purpose, the data that Google and even Facebook can access is limited compared to the daily financial data a bank gathers on its customers. Neither the tech giants or the fintech start-up companies can match this rich seam of information that – as yet – remains largely untapped, although some such as Apple and PayPal are starting to change this and some of the fintech companies are not far behind. 

In his book ‘Social Physics: How Good Ideas Spread’, Alex Pentland analysed 10 million transactions of eToro users (a social trading platform) and found that those traders who were isolated and those who over-connected did worse than those who struck a balance. By encouraging the loners and slightly discouraging the most active users, he was able to double the profitability of the group. 

The value that retailers would place on predicting likely future purchasing patterns, preferences and life events is enormous. A customer buying gardening products for the first time combined with a change in rent or mortgage payments would signal the perfect time for a gardening company or lawn mower company to engage with that consumer.

Big finance then is well placed to capitalise on the burgeoning opportunities represented by big data, if it can overcome the inevitable technical and legal hurdles. It cannot afford to wait around, however, with tech giants like Google, PayPal and Apple eating into their revenues. Accenture estimates that competition from non-banks could erode one third of traditional banks’ revenues by 2020. There has never been a better time to tap into the potential of big data nor a worse time to pass up the opportunity.

Commerzbank Disclaimer

The views expressed in this article are those of the author and may differ from the published views of Commerzbank Corporate Clients Research Department, the communication has been prepared separately of such department. No representations, guarantees or warranties are made by Commerzbank with regard to the accuracy, completeness or suitability of the data.