"We see a reduction year-on-year in the amount of energy we supply, but we have other businesses that grow," reveals David Cooper, CIO of British Gas and Centrica, as we discuss Hive, the British Gas entry into the Internet of Things market that is about to transform and disrupt the CIO world. "Just listening to customers does change the way you do things, and Hive is much closer to consumers and what they want."

British Gas is the retail and services arm of Centrica, once a nationally-owned organisation that in 2011 was worth £15 billion as FTSE-listed organisation.

"Centrica is about oil and gas exploration, and spends a lot of capital on that. British Gas buy gas and electricity in bulk and sells it in retail," Cooper told us at Centrica's headquarters in Windsor. British Gas remains the dominant energy provider in the UK market, despite new entrants, supplying 16 million UK homes.

"It is a really odd brand. We have a lot of political criticism," he says of how in the past 18 months politicians from both Labour and the Conservatives have used energy firms as a brickbat to appeal to the public. "But we also have the trust of the people to fix things and come right into their homes and people's lives." British Gas is also one of the UK's largest suppliers of home servicing and repairs to central heating, electrics and plumbing.

"Outside of the living room, technology hasn't really changed how we manage our homes," wrote CIO's sister title TechAdvisor recently. "The way we heat, power and light our homes has not changed for decades. The last mainstream innovation in our home could be described as the mass adoption of central heating in the 1970s." But smart home devices like Nest, acquired by Google in 2014 for £1.9 billion, and Hive are bringing the Internet of Things into daily life.

Studies of early Hive and Nest adopters shows that 40% of users interact with their application at least once a day via a smartphone app. The combination of connected devices, environmental and financial concern, and the ease of use and prevalence of smartphone usage means the Internet of Things is finding a home, at home. British Gas believes consumers could save £150 per year as hive users.

A further change to the energy sector is a government mandate that the UK must have fully converted to smart meters by 2020, ending years of consumer's meter reading and instead allowing your energy supplier to monitor online how, when and the volumes of energy you use. Together these technologies will radically alter the way British Gas operates, but Hive demonstrates an organisation and CIO that is embracing the disruption and moving its business model in response. Cooper was a new entrant to the 2014 CIO 100, and went straight into the top 15 because the judging panel were impressed with Hive and the attitude towards change at British Gas.

"Who'd have thought that British Gas would be in the Apple Store," Cooper chuckles at the rate of change at the energy firm. He admits that creating the Hive brand as an almost separate entity to British Gas has done a lot to attract customers to Hive, which doesn't require that you buy your energy from British Gas. It originally launched as British Gas Remote Heating Control.

"The board realised it had some potential and changed the brand. It is now seen as one of the engines to drive the business forward. Connected homes are a massive focus," the CIO explains.

Hive also brings out the technologist in Cooper. "There are some new functions coming that enable predictive intelligence and geolocation, so it can tell if you are near home. There is more around the Internet of Things for intelligent boilers that like cars will tell us when a part is about to fail and on our infrastructure side as well," he says of the opportunities for real transformation.

"With increased energy usage, the energy grids need upgrading which can be difficult, so we may need to control device usage. Grid investment is always a very large undertaking. The grid is carrying more capacity than ever as the number of households is going up. Plus, we all have more devices, despite efficiency savings. We can help.

British Gas CIO David Cooper

"It is the peaks that are hurting the grids. We can see economies as the Internet of Things grows, from different tariffs to promote a change in usage pattern, such as cheaper rates at the weekends. These have already been trialled in the US. It is the same as when the telecoms companies changed their tariffs to encourage people to make calls in the evenings to ease the strain on the network," he argues. And the mobile telecoms world is also driving great change at British Gas.

"We are seeing traffic shift away from PCs to smartphones, and expect it to be at least 50% this year, as it has been a 14-15% swing every year.

"Everything was originally done for the PC-based web, now there's a huge trend with two thirds of our online communication coming from a smartphone, and these users can solve their own problems. So for us, it's about doing things in a more customer focused frame of mind. My team is an integral part of what it is that we can do to make our customers lives better.

"There are huge implications for the older systems at the back end, bill changes must be correct, otherwise it drives a huge amount of calls. So we are entering a slightly different world of readings every 15 minutes rather than every 18 months. Combining Hive with a Smart meter, as we can tell a lot about your house," Cooper reveals.

Smart energy

Like his peer at First Utility Bill Wilkins, Cooper and British Gas are developing smart energy reports that allow consumers to compare themselves to a typical user and for the energy provider to guide customers to save money. Utility companies, akin to the software industry, are having to rethink business strategies and revenue sources, as consumers change their attitudes and behaviour.

"It doesn't have to be a negative thing. We will have other business that will grow. We are a service business, we install new boilers or solar. We are making a virtue out of something that could be 'woe is me'," he says in direct opposite to what so many IT vendors say.

Before joining British Gas in 2011, Cooper had spent the bulk of his CIO career in telecoms at TalkTalk, Hutchinson 3G and BT. It was his experience of a customer-focused sector that like utilities has seen customers move away from one revenue source – voice call consumption – to another. "They are very similar businesses. It is about customer service at the end of the day. The winning telcos are the ones that are all about the customer experience and it is the same for us," the CIO told us.

When Cooper arrived at British Gas, he admits to discovering a Whitehall mentality that 'it was different'. "I would say the back end is the same as a telco. If you accept that, you can use open source and you can learn from other sectors and companies," he explains of both his technology strategy and culture change.

The previous attitude of being different was not winning fans at any level of the business either. "The management team at the time felt that IS was not as responsive as it could be. All the noise has gone now and there are no complaints," he says.

British Gas has been rationalising its application estate and as a result making significant cost savings, when Cooper started at British Gas, the IT budget was £450 million, today its £290 million.

"When I joined, British Gas was still suffering from the consequences of a massive programme failure in 2002-2008 in the billing and CRM space. The compromise architecture, system stability and resilience issues, and tactical system deployed during this era and the outstanding problems were causing significant customer and business operations team issues.

British Gas CIO David Cooper

"I drove a programme to review the known issues and defined the strategic changes required to remove all of these issues, and we have seen a dramatic improvement in the experience from both a business operations team and customer point of view. Many of the workarounds and failures have been removed with consequent IT cost reductions and business efficiency improvements.

"The business wanted to keep their old processes and specify new systems to correct the customer problems, which resulted from such a fragmented landscape. My solution was to use the software licences the company had previously purchased, and using the latest version with very little customisation change the business processes. This had a significantly accelerated timescales, reduced whole life costs and eventually dramatic reductions in the size of the workforce.

"Things had been left because they worked, but people didn't realise the cost or that the applications were holding back the business. Now we can see the 360-degree perspective. You have to have good systems to sell and be a service business. I had to convince the CEO why he should trust me and go with my solution for the business. He did and as we near the end of the programme, it was the correct approach. It is quite difficult to face the whole business with a different vision but immensely rewarding, as it is now seen as a major success," he says of the transformation.

A challenge for British Gas and Cooper is the diversity of the business, from straightforward selling of gas and electricity to installation teams, maintenance teams, plumbers, sparkies, and plumbers at Dyno-Rod. "To control all of that you need a huge amount of compute power. There are 26 different field force all doing different jobs within British Gas. We are investigating system changes to get better at leveraging of the field force solutions to improve efficiency," he explains.

With Hive, smart meters and an army of field workers collecting data, British Gas has huge amounts of data. But as CIOs have said before in these pages, it is not the amount of data that matters but the actionable insights that can be drawn from it. Cooper and his team have been working on ensuring that British Gas can illuminate useful information from this data through the adoption of open source Apache Hadoop database architecture.

"I want it on Hadoop to get a more rounded perspective. We are using PowerBI and Qlikview as these tools mean we can allow the users to do so much more themselves. With these tools, users can fish in the lake and pull out what they want. The traditional technologies are limiting and struggle to scale. Relational databases were based on computer power," he says of the move away from traditional analytics tools.

"Hadoop scales massively and its heritage is with Google, LinkedIn and Facebook and there is a cost saving, which is a nice by-product. But more importantly it is much easier for us to answer those 'what if' questions."

In the clouds

Cooper has moved around 50% of his architecture to the cloud, partly because it hadn't been refreshed and to drive the transformations.

"The first phase has been completed and now we are building an SAP environment on HP. Dynamic allocation for development and test will go live in September, as we do see huge peaks and troughs in our demands. We can see great opportunities in the future for production infrastructure as boilers only break when under full load, so the customer traffic into British Gas can be huge if there is a cold snap. Or media coverage on price rises or regulation changes means we have to notify everyone quickly. Being able to dynamically provision based on weather forecasts or other drivers means we can turn services up and down to serve customers better," Cooper reveals.

"We actively identify the transactions that occupy significant call centre time and that are growing, and then we automate them; for example, you couldn't change your direct debit on the App, but you can now. So we are offloading traffic from the call centre, so they are free to deal with more complex problems, and they are more successful at that and we can see that in their transactions."

As part of the data centre transformation and cloud adoption, Cooper has opted for a RedHat open source architecture and away from Unix.

"I wanted to remove the impediments to moving to the cloud and I can move to anyone's cloud. These are the technologies that the social media space use. If it works for them, I think I can make it work for me, they too are dealing with massive data volumes. All our Oracle platforms are on the cloud," he says.

You only need glance at Cooper to see he's a cyclist, he's lean and lithe, and at least once a year takes part in a major cycling challenge, either over the Alps or the UK's steeper areas.

"You have to be fit before you go and it makes you feel better, there are no frustrations at the end of the ride."

David Cooper CV

2011 – present: CIO, British Gas
2009 – 2011: CIO, TalkTalk Residential
2006 – 2009: Chief operations and technology officer, Hutchison 3G
2003 – 2006: Chief technical officer, Hutchison 3G
2002 – 2003: IT director, Hutchison 3G
1999 – 2001: Head of network and systems design, BT

British Gas CIO David Cooper