Skip to content

The size of Big Data

Capital at Risk: All financial investments involve an element of risk. Therefore, the value of your investment and the income from it will vary and your initial investment amount cannot be guaranteed.

Part 1:
Lots of data sources, as well as the “right” data

Are you capturing enough data?

Investment teams looking to harness the power of Big Data must cast a wide net when seeking to find the right data to enhance investment outcomes. Why is that? The global economy and financial markets are highly complex, so the data they generate are often unstructured and noisy. It is important to leave no stone unturned and to evaluate as much data as possible to determine which may be most useful in an investment process . That is why SAE researchers trialed over 70 new datasets in 2017 alone.

Total buy-side spend on alternative datasets ($mm)

Source:, as of March 31, 2018. The above estimates are those of There is no guarantee that forecasts made will come to pass.

How to achieve quality over quantity?

The data itself needs to be tested for its quality and additivity to a forecasting model. When SAE on-boards a new dataset, we clean it and then run it through our own battery of statistical tests to determine the value of the information it provides. In technologically advanced regions such as the US where we already have a lot of information about the firms we invest in, the bar to finding new data that is useful is high. After years of experience, we have learned that new data is not always additive to existing models. However, you never know that unless you try! And even in projects that are not successful immediately, these are often ideas that can be used for other projects in the future.

Example case study:
Big Data forecasting in retail

We explain how analysing data on consumer behaviour seek to produce forecasts to enhance investment outcomes.


GPS data

Geo-location data sourced from mobile phone beacons can be useful in recording consumer foot traffic in physical stores. But, foot traffic in and of itself does not mean any sales occurred. Fortunately, there are many data sources that can shed light on consumer intentions and could help inform our assessment of a retailer’s sales growth.

For illustrative purposes only.

Internet search history

At one end of the spectrum, analysing aggregated internet search activity can capture changing consumer sentiment towards a company’s brand or products. Still, data on internet search activity has a long forecasting horizon and less forecasting accuracy given there are several steps consumers must take after an internet search before they purchase.

For illustrative purposes only.

Electronic receipts

At the other end of the spectrum, aggregated consumer transaction data as recorded by banks and credit card providers can enable our forecasting models to track actual consumer spending (and ultimately, reported revenues).

For illustrative purposes only.

Booked sales

By using multiple data sources to corroborate one another and answer the same investment question, we can significantly improve the quality of our forecasts.

For illustrative purposes only.

The impact of consumer behaviour on sales growth

Click on each icon to find out more.

GPS traffic

Source: BlackRock, as of May 31, 2018. Provided for illustrative purposes only, not meant to depict actual data.

Download a Big Data due diligence checklist

Many asset managers hope to discover unique investment insights by applying machine learning and Big Data analysis. Download a checklist of questions to ask when conducting due diligence on this topic.

Any data collected will be processed according to BlackRock's privacy policy, which can be found here.

Take advantage of a changing world
Our new Advantage range brings together technology, Big Data analysis and responsible investing to reflect the world we live in today.