A couple of years ago a shortage of data scientists would not have been an issue for most organisations; but as more businesses aim to squeeze extra value from their data the imbalance is being felt more widely. People who can prove the grasp of data structures, algorithms and the business value are hard to find, and some businesses are struggling to fill short term gaps and asking how to meet the long term need.

It is possible to the find the skills on a contractual basis, but it is an immature and patchy market.

Chris Yapp, a consultant in technology and futures thinking, says some consultancies are offering a service, but that he has doubts about their data science capabilities. There are individuals with the skills, some of them in particular areas such as sentiment analysis, who are picking up limited term contracts or can be tempted into staff jobs. But Yapp says the people he knows who fit the bill are job hopping, benefitting from word of mouth reputations to get more from employers or clients.
Tony Venus, head of standards and qualifications at e-skills UK, suggests the market is tilted further against employers by a lack of understanding of the role.

“The problem a lot of employers will have is to articulate their skills and needs, then to align and match the providers in the market against it,” he says. “There is a lot of confusion around the business value of doing it and a lot of missed opportunities.

Identifying providers of services is problematic.”

Venus says there are instances in which buying in the skills short term can provide good results: “If you have a well organised, well structured database, you may go out and subcontract the work on it. It depends on the nature of the investigation. If it’s more procedural it may well be covered by outsourcing.”

Looking to the long term, it is more likely an organisation will only get the best from a data scientist who really understands the business and its market. This is raising questions about how to develop the capability in-house.

Yapp suggests that a focus on projects to fix problems in a business could lay the ground for developing the skills. For example, obtaining sales and performance reports daily rather than retrospectively could be a step towards building the skills base in-house.

“Start from that and build around it, rather than building a data science department,” he says, adding: “You carry out projects and learn what you can from them.”

There can also scope to build on the role of structured query language (SQL) programmers. Mike Hoskins, chief technology officer of big data analytics specialist Actian, says that while their skills have been developed on relational databases, new platforms provide the visual tools and scripting interfaces to help them run analytics of unstructured data in Hadoop frameworks.

“The first step is to look at SQL programmers,” he says. “Its how we’ve grown to ask questions of our data, and the smartest short term answer is to leverage their talents.”

He says the creation of relevant disciplines in universities will take time to bring results, but the focus of e-skills UK suggests this is where the long term answer can be found.

“We have to ask which degrees provide an underpinning,” says Tony Venus. “If I want to recruit a computer scientist, does their degree help with the technical and business insights into data? Or do I want to see new degrees more focused on linking the technological and business aspects of analytics?

“Or if I want to recruit a ready-skilled person, how do I benchmark their skills? How do I know which questions to ask and develop a checklist of the skills needed? This comes back to national occupational standards.”

e-skills UK has begun to work with employers on developing new occupational standards that, despite some questions over the terminology, will take in the analytics performed by a data scientist. Along with the creation of relevant university degrees it will take years before it impacts on the market, but organisations that see the long potential in data scientists will keep a close eye on their development.