Benchmarking in the IT industry has a long and lively history. At times, suppliers have behaved as though their survival depended on a single good performance rating. However, in recent years, marginal advantages in performance have rarely seemed to be critical.

But the emergence of green issues and the eco-imperative means that there is now a new and perhaps vitally important axis for measuring IT performance. In our special report, Eco-efficient IT: The eco-imperative and its impact on suppliers and users, we identified 12 existing or proposed benchmarks that can be used to measure or compare energy efficiency. Even since writing the report, a few more have come to light.

Some of these are well established and simple, such as the Energy Star rating system for assessing how much power a machine uses relative to its processing power or storage capacity, based either on system labels or supplier data. Some are innovative and, by happy coincidence, highly favourable to the suppliers that are backing them. Sun Microsystems’ SWaP (space, watts and performance) benchmark for systems and Pillar Data Systems ‘storage efficiency’ equivalent – which both produce a quotient based on performance, power use and physical space – make technical sense but have little backing. Others, such as the Green Grid’s Datacentre Performance Efficiency (DCPE) benchmark, are more ambitious. DCPE is an attempt to measure energy use relative to an elusive measurable called ‘useful work.’ Perhaps unsurprisingly, Green Grid representatives working on the standard admit that defining the term ‘useful work,’ is proving difficult.

Taken on their own, the role and importance of most of these eco-efficiency benchmarks will probably follow a well-charted course. R&D teams will use them as targets to beat, or hurdles to get over in order to leap frog rivals, or simply as measurements of their progress, all of which encourages innovation.

Marketeers and even savvy customers will use them when it is helpful to do so, but will otherwise ignore or pay lip service to them, perhaps favouring their own alternatives.

Market mechanisms
But eco-efficiency also involves a new element that is going to make measurement, benchmarking and certification more important and more complicated: stakeholders, lobbyists and legislators want to encourage or even mandate the accurate measurement of energy use and of energy efficiency. And having done so, they want to put in place measures – voluntary or otherwise – that reward good practice and punish bad practice.

In the IT arena, this will bring two important changes to the usual measurement and benchmarking process. First, the focus of most of these initiatives will not be on establishing the merits of supplier claims about equipment efficiency – important though this is – but on verifying and rewarding demonstrable improvements in energy efficiency at the customer’s premises. And second, trading systems or rebate programmes may mean that good results carry a monetary value, and poor results a monetary penalty, so that the entire process may be scrutinised by auditors, regulators, utilities and government departments.

None of these energy efficiency programmes are, at present, directly aimed at IT, but because IT’s use of energy is now so high (more than 50 per cent of corporate power bills, in many cases), the CIOs, their suppliers and their outsourcing or consulting partners will all be drawn in. And it may not stop there: as IT’s proportion of corporate energy bills grows, and as it is increasingly recognised that IT is a big and inefficient user of energy, specific legislation aimed at datacentres is a distinct possibility.

Whatever shortcomings there are in the benchmarks, legislators attempting to reduce carbon emissions are unlikely to leave IT alone, even if they don’t address it directly, and even if there are no widely agreed user-side mechanisms for providing incentives or punishment for efficiency levels. There are many indicators of energy use and efficiency, even if none are absolute or entirely fair, and these will be used in conjunction with raw energy measurements to change corporate behaviour. The consensus among legislators committed to tackling climate change is that willpower and goodwill, by individuals or corporations, will not be enough.

Andy Lawrence is research director for eco-efficient IT at The 451 Group