The Big Data hype is growing exponentially. If you believe the hype in the time it took write this three gazillion petabytes of big data marketing collateral has been created. Big data sounds cool and is trendy, but what is big data? Should CIOs be implementing big data strateies and if so how?

Paraphrasing Wikipedia: “Big data is a collection of data sets so large and complex that it becomes impossible to work with using traditional database management tools and approaches”.

So how big is big?

There doesn’t seem to be a standard definition out there. But examples of how data is growing and is so easily created come from social media. Twitter creates over 500 million tweets per day, Netflix houses five billion film ratings.

The potential to derive new customer insight from huge datasets such web click stream traffic, social media, population data without needing to structure it could provide CIOs and their organisations with a valuable resource.

Now there are good tools to filter huge datasets (like DataSift). And there a number of frameworks that could allow organisations to spot patterns and trends within complex interactions and datasets (such as Acunu). But spotting a pattern is pointless without turning it into insight and action that will allow organisations to personalise, segment, target, predict and recommend in ways not possible before.

Is there a quicker and cheaper way for organisations to gain insight is a question overlooked amongst all the PR and analyst hype? Are organisations really using all the data they already have? If an organisation already has a CRM system and it’s not driving business decisions, then the distraction of kicking off a big data project will cost time and money without adding anything to a situation weak in people and/or process.

Techniques like multi-variate testing can be quicker and cheaper to implement than big data and deliver significant business impact.

If I haven’t put you off and you are desperate to write that cheque consider these exercises:

  • Set up and create a create a closed loop process to collect data, process it, derive insight and implement changes.
  • Don’t underestimate the costs and difficulties in getting the data where you want’s the hardest part.
  • Don’t forget the “Three Quotes Rule” and to talk to more than one provider to look at cost/benefits of the proposal.
  • It will be much easier to demonstrate ROI if you set and hit a mixture of easy, hard, and game changing goals.

Despite all the hype the Big Data approaches are here to stay and some companies genuinely will derive game changing insights from them.

From the CIO archive: 

Betfair CTO Rory Devine placed his bet on people