Where to put all that Big Data

The rate of change of data that is being generated has been noteworthy for sometime and from where i’m standing in a professional services environment, the proliferation of material is very clear to see.

The general gist is that growth is exponential and this trend is not the preserve of big business.

Collaboration inter business, and more so beyond the boundaries of the business, has blossomed beyond the traditional email but now encopasses voice and video traffic at the very least.

What is clear, the consumerisation agenda has stimulated an increase on data volumes particularly as mobile devices has driven multimedia traffic flowing round the enterprise, legitimate or otherwise.

Initiatives like docx from Microsoft will assist with slowing the storage requirements but this will not be an overnight sensation.

Data storage is moving forward as you would expect but more interestingly the choices on offer for in-house teams is expanding. Buying vast arrays of disk is not a default position.

StaaS is now a very credible alternative in passing on the considerable management burden and skill required to manage data stores.


The smart money here is spend on techniques to actively move data between high speed storage and other less expensive alternatives.

Removing duplicates is necessary. Those of us running an in-house IT organisation have to tolerate the annoyance that is enthusiastic users emailing very graphical powerpoint presentations to a large audience.

Balancing what gets stored where and deploying some of these other techniques requires a skill set likely to be beyond even the most modest of SME’s.

Some of the high end storage vendors have strong offerings but this level of investment should follow through to the support of it which can mean people and more than one for failover.

StaaS is a serious contender in addressing the accessibility to high end tech and the important management wrap around it. With StaaS comes all the capex avoidance benefits the cloud market bang on about.

Big data has another dark cloud for IT leaders, which is the recovery position. A cornerstone of our internal standard operating condition is the provision of a credible data recovery plan that fits into a more rounded business continuity plan.

The more data there is swilling around, the larger the recovery time and potential complexity. More organisations are adopting a disk-to-disk backup strategy with mag media used only as a last resort.