Big Data Is Coming to Pension Funds

Authored By: Irene Aldridge | Publish Date: April 13, 2018

While Big Data is on the radar of many pension fund managers, most often, it is still only in their peripheral vision. Still, in the next five years, Big Data analysis will likely become the must-have skill in the pension industry. Big Data in portfolio analysis, in particular, is well worth the pension fund industry investment.

Outside of portfolio management, Big Data is already everywhere: in social media analytics, digital video recognition, 5G cellular technology, and much much more. The capabilities of Big Data are nothing short of explosive and well-established. Indeed, the core capabilities of Big Data first emerged on a large scale as a spying technology in the World War II and has since been well tested.

Set Of Techniques

What is Big Data? Loosely, it is a set of techniques embedded in the latest, most sophisticated technologies. At the heart of Big Data is the concept of data as a structure that obeys well-defined principles. The principles hold for any data set, regardless of how random, disjointed or even empty your data table might be. Big Data techniques help distill the important drivers in the data and create meaningful inferences.

Consider the following example from our latest research paper, ‘Big Data in Portfolio Allocation.’ An image, such as the one shown in Figure 1, is a dataset, a matrix of pixels, whereby every pixel represents information about the color of that dot on the computer screen. With Big Data techniques, one can digitally extract the most important information within the image in literally microseconds without telling the computer anything about what it should look for in the picture. Indeed, as shown in Figure 2, the computer effortlessly identifies areas in the image with the most detail and variation as the key information-carrying regions.

Figure 1. Original Image

Figure 2. The image with key information extracted and rest discarded, without any human input

Why is this important for portfolio managers? Many portfolio managers literally swim in data. And the more data, the merrier, think the modern financial industry stalwarts such as Goldman Sachs. Indeed, Goldman Sachs has recently created a central data ‘lake’ — a repository of all sorts of data accessible to all channels of the organization. Synthesizing the data and making inferences quickly and efficiently is what the Big Data techniques do best. In other words, separating the wheat from the chaff in the data with Big Data analytics takes only seconds, not weeks, as is traditionally the case.

Superior Out-of-sample Returns

Big Data in Portfolio Allocation’ is the first to study the Big Data properties of the inverse of the correlation matrix, commonly used in portfolio optimization. As it shows, the inverse is much more informative than the correlation matrix itself, from the Big Data perspective. Subsequently, the paper proposes Big Data approaches to harness the correlation inverse and to deliver superior-out-of-sample returns.

Armed with Big Data techniques, researchers can identify key trends, drivers, and even missing or latent variables quickly, seamlessly, and with great precision. Huge amounts of data become trivial and information of all sorts is more valuable than ever. Most importantly, researchers, analysts, and portfolio managers can save countless hours and ramp up productivity by letting technology do the gruntwork. Then, researchers and portfolio managers can focus their efforts on creating inferences from smaller, much more powerful datasets condensed by Big Data.

Irene Aldridge is President and Head of Research of, a Big Data information source for financial markets, and co-author of ‘Real-Time Risk: What Investors Should Know About Fintech, High-Frequency Trading and Flash Crashes’ (Wiley, 2017). Follow her on Twitter:

Like what you just read?

Consider signing up for our Daily News Alert Email to receive relevant industry news, headlines and articles

Subscribe Now