Analysts slam big data 'bullshit'

Big data is facing a growing backlash from those at the sharp end – the data analyst community – who are becoming increasing critical over the amount of time it is taking to sort the wheat from the chaff in large data-sets.
Many on the direct marketing industry have criticised the phenomenon virtually since the phrase was first coined by Garter in 2001, although with a recent IDC forecast claiming the market will be worth $41.5bn (£27bn) by 2018, most have been forced to embrace it.
The latest issue of Science News details the limits of big data in a series of articles, while some have pointed out that if it is as effective as promised how come IBM – one of the biggest vendors in the market – has not been able to halt a three-year sales slide. The company recently signed an £800m deal with marketing services giant WPP.
In a blog post, MailChimp data scientist John Foreman, who has built systems for Coke and Dell, said: “There is this idea endemic to the marketing of data science that big data analysis can happen quickly, supporting an innovative and rapidly changing company. But in my experience and in the experience of many of the analysts I know, this marketing idea bears little resemblance to reality.
“A lot of vendors want to cast the problem as a technological one. That if only you had the right tools then your analytics could stay ahead of the changing business in time for your data to inform the change rather than lag behind it. This is bullshit.
“If you want to move at the speed of light, pick your big data analytics battles. If your business is currently too chaotic to support a complex model, don’t build one. Focus on providing solid, simple analysis until an opportunity arises that is revenue-important enough and stable enough to merit the type of investment a full-fledged data science modeling effort requires.”