CIOs have a big challenge in managing data. Companies are sitting on huge stacks of historical data, with many now increasingly looking to leverage these data sets for business gain, such as improved customer loyalty and enhanced operational efficiency.

Not only that but digital businesses are now looking to utilise new, real-time data coming from Internet of Things (IoT) devices, smart cities and autonomous vehicles to derive similar insights.

This data offers CIOs an opportunity to analyse how their business is performing almost instantaneously, pushing their data-driven organisations to become more efficient and profitable as a result.

It’s to little surprise then that Big Data – the term that refers to technologies and initiatives that involve data that is too diverse, fast-changing or huge for conventional technologies, skills and infrastructure to address efficiently -- is a prominent issue near the top of the CIO’s agenda, as well as the CEO’s. It has been for a long time.

In , 89 percent of C-level executives said that Big Data is very important to their transformation into digital, with executives adding that their companies used Big Data moderately or extensively to identify new sources of revenue (94 percent), retain and acquire customers (90 percent), and develop new products and services (89 percent).

A more recent report from Economist Intelligence Unit found that almost a quarter of business leaders (23 percent) believe Big Data will revolutionise the way businesses are managed.

This further crystallises the need for CIOs and CEOs to work together to transform the business, and this partnership is key given there are still many barriers to Big Data adoption.

Privacy, security and the risk of legal sanctions are potential barriers, while the terabytes of data now flowing in and out of business promises to stretch existing IT architectures and infrastructures.

Yet the biggest danger is arguably the failure to seize on the opportunities of Big Data. Laggard businesses face ceding competitive advantage to rivals, while CIOs could miss their opportunity to take a central role in driving business transformation.

Why firms can’t avoid Big Data

There are already several stellar examples of Big Data technologies changing how businesses operate; Netflix is using open-source tools such as Hadoop and Cassandra to analyse viewing behaviour to offer related content, while Airbus is using Internet of Things (IoT) sensors and data platforms to improving flight testing. John Deere, meanwhile, helps farmers to boost crop efficiency through the analysis of radar data.

Retailers like Macy’s, Tesco and Walmart have adopted such technologies to roll-out real-time pricing, search and predictive maintenance, while oil and gas companies are taking the output of IoT sensors in drilling equipment to make more efficient and safer drilling decisions.

These are industry-leading examples and many more are following suit; a 2015 study by BARC Research revealed over 40 percent of companies worldwide analyse Big Data and many are now enjoying a variety of benefits, such as better strategic decisions (69 percent), improved control of operational processes (54 percent), better understanding of customers (52 percent) and cost reductions (47 percent).

The key to this, however, is ensuring that organisations are analysing the right data at the right time. The ascent of IoT, in particular, means organisations are increasingly collecting terabytes of data – much which could be worthless. At a recent conference, it was revealed that modern GE aircraft engines can collect data every 62 milliseconds across 5,000 sensors – which could result in data overload.

CIOs need to focus on collecting and analysing the right data at the right time to make smarter strategic decisions. By doing so, they should be able to identify hidden trends in data and do everything from reduce fraud and improve security to predicting and optimising business performance.

Big Data benefits go beyond IT

Businesses must see Big Data as a business opportunity, rather than a siloed IT project.

A study by technology organization Wikibon revealed that the revenue from big data is expected to grow from $5 billion to more than $50 billion by 2017, that for a typical Fortune 500 company just a 10 percent increase in data accessibility will result in more than $65 million additional net income.

The BARC study further indicated that those organizations able to quantify their gains from analysing big data reported an average eight percent increase in revenues and a 10 percent reduction in costs.

The strategic CIO, who is increasingly being asked to “step up” by CEOs to drive business gains, must drive these Big Data initiatives.

So, how do you go about adopting Big Data technologies? Experts say that the CIO must start by defining the business problem, before establishing the partners and technologies you need to use, some of which may be open-source, like Hadoop, Kafka and Spark.

Enterprises will need multi-model data platforms that can handle this copious amounts of data, coming from disparate sources, especially as firms migrate to IoT. Such platforms enable firms to move to real-time decision making, and support multiple transactions at the speed and scale organizations need.

CIOs, and their IT teams, will require rapid access to data to drive decisions, as is the ability to respond to that data.  CIOs know now they must step up to drive the Big Data transformation.