Executive director of the Royal Statistical Society, Hetan Shah, stated “It’s happened. We are now living in a data economy. The UK needs to skill up fast when it comes to data, statistics and analysis, or we will miss the boat.” And yet businesses are struggling to employ those able to effectively analyse this data.

It seems that businesses of all sizes can and will continue to benefit from data crunching at some level, though understandably there is a big push for financial, budgeting and marketing insights in particular. ‘Data crunching’ is a term that refers to the analysis of vast quantities of data resulting in finding previously unforeseen patterns in order to give a business a competitive edge. While so much business data comes from recent years, this means that it remains unstructured. Therefore, it is not neatly in a database for a CEO to glance over, instead, it needs a highly qualified and experienced professional to ‘crunch’ it, so to speak. Yet those qualified enough to do so, a term known as ‘data scientists’, are often unsupported.

These ‘data scientists’, are usually of Ph.D or post-doctorate education level and therefore have a very strong academic background, thus making them well practised in highly developed, mathematical and scientific analysis and theses. This then makes them well suited to a profession exploring large volumes of often un-structured data which can come in many different forms from numerical data to images or even social media content.

The problem with this, however, is that “true data scientists are rare”, as claimed by the head of business intelligence and big data at Telefonica, Ricard Benjamins. While it is easy to find an employee that excels in either coding, statistics or business theory, it is almost impossible to find one talented in all three. And it is even harder to find someone talented in all three that can then drive forward the ideas and opportunities found.

While a data scientist’s starting salary can reach up to $200,000 in America and at least £40,000 in the UK, this makes them fairly unobtainable for smaller businesses. Therefore, many businesses have adopted technology such as budgeting and forecasting software alternatives. These successfully rid of the need for an employee who needs to excel in all data crunching skills processing by doing what computer processes are good at, processing large volumes of data quickly. By having the relevant information shown immediately, recognising trends and patterns can be easy for owners and managers not necessarily trained in data science.

Such forecasting is effective and saves time and money, much more so than having a data scientist fully employed and supported. Moreover, by doing the crunching electronically, the risk of inaccuracies and human error is removed, as well as the fact that computational analytics methods do not require human communication that can be easily misinterpreted and / or misconstrued.

Computers and programmes are so advanced now that many are capable of making accurate predictions based on the data seen without human intervention. By processing the vast amounts of data that would ordinarily take a data scientist months, these softwares are able to leverage the data, identify patterns and anomalies and associating similar and differentiating projects almost immediately. So with this technology readily available and still rapidly improving, why is there such a demand for data scientists of the human kind?

Sources:

Nesta News

Bloomberg News