Big data to shrink as Covid fuels end of data munchers

The age of big data is finally drawing to close, with Gartner predicting that, by 2025, almost three-quarters (70%) of companies will have shifted their focus to small and wide data, and the obsession with gathering every last scrap of information will make way for a “less is more” approach to provide more context for analytics.

In a new report which gives the game away in the title of “Top Trends in Data and Analytics for 2021: From Big to Small and Wide Data”, Gartner argues that big data is simply too cumbersome.

The report claims that AI-powered systems are too data hungry, because they gather virtually everything and often this information becomes obsolete faster than it can be used. So instead of big data, data and analytics teams will shift their attention towards techniques known as “small data” and “wide data”.

Small data is a technique where businesses try to extract more insights from less data, while wide data refers to the analysis of various small and large, unstructured, and structured data sources. It aims to find links between data sources and different data formats.

Gartner research vice president Rita Sallam explained: “Disruptions such as the Covid-19 pandemic are causing historical data that reflects past conditions to quickly become obsolete, which is breaking many AI and machine learning models.

“In addition, decision making by humans and AI has become more complex and demanding, and overly reliant on data hungry deep learning approaches.”

Gartner believes that the retail industry will be a major beneficiary of the new approach, as it can forecast demand or improve customer experiences. Physical security and fraud detection are also areas that can be improved.

Sallam concluded: “Small and wide data approaches provide robust analytics and AI, while reducing organisations’ large data set dependency. Using wide data, organisations attain a richer, more complete situational awareness or 360-degree view, enabling them to apply analytics for better decision making.”

The term big data has been in use since the 1990s, with many crediting American computer scientist, director and entrepreneur John Mashey for coining the term. Not that everyone is convinced of its provenance, however. REaD Group chief executive Jon Cano-Lopez, a man who has now worked in the data industry for nearly 35 years, once said: “It makes me laugh when I read about big data. It hasn’t just appeared – it’s been around for years.”