Among European countries, the UK is ahead on Big Data. As the government has recognised, the UK already leads in research and can also lead in commercialisation. Much of this is because Big Data exploits two national advantages. The first is that because the UK economy is relatively liberal, it’s more acceptable to go out and make money by studying consumer behaviour. The second national advantage is that much of the world’s electronic data is collected and stored in English.

But even in the world’s most developed Big Data market, the US, most of the available information is not currently being exploited. Data science is a new field. The expertise needed to draw meaningful inferences from large data sets is hard to come by, and few people know what to do with unstructured data.

Knowing what to do with unstructured data is, in fact, one of the hottest topics in Big Data. Orange Business Services estimates that two-thirds of the data collected by companies is unstructured data from audio, videos, images, and social media. NoSQL platforms promise new ways of storing and searching unstructured information, but it’s still early days.

Even Google admits they don’t yet know what they’re doing. On the Charlie Rose show, Google CEO Eric Schmidt observed that when you use Google and get more than one answer to your question, you’re experiencing a bug in Google. “We have more bugs per second than anything else in the world,” Schmidt says, “We should be able to give you the right answer just once. We should know what you meant, we should give it to you in your language, and we should never be wrong.”

The field is new, but UK organisations should get ahead of the game. They should get good at managing and analysing large data sets to support business strategies. Research firm Gartner advises IT leaders to “ensure Big Data initiatives are tied to organisational goals and processes, and demonstrate the insights and value that these initiatives can bring to the business. Gartner sees many organisations getting stuck in the experimentation phase - finding interesting insights from new data sources, but often not connecting them to business processes.”

And by the way, whether you are ready to exploit Big Data to your own advantage, remember there is a flip side. While most companies are potential beneficiaries of Big Data, every company is a potential victim.

Your organisation may fall prey to these new capabilities by unknowingly giving away company secrets. If your employees are using social media, or doing just about anything on the public internet, chances are, some unscrupulous party is following what they are doing, and learning something about your company.

For the time being, what’s clear is that every IT director needs to do at least one thing: minimise the risks of employees unwittingly giving away secrets on the internet.

[Part 1 - What every organisation needs to do about Big Data]

[Part 2 - Ethical questions around Big Data]