Critics argue that service oriented architectures (SOA) have failed to deliver value. But the real problem might just be that most firms fail to properly measure the costs and benefits of their SOA projects.

While service oriented architectures (SOAs) have garnered plenty of headlines over the past few years, some question whether the technology has made the headway that was expected of it. Indeed, earlier this year, analyst firm Burton Group announced that the SOA experiment had failed, as SOA had not delivered all of the promised benefits. In fact, Burton said, SOA had delayed projects and increased costs. Since last year SOA is generally said to have fallen off the tech radar.

An SOA is a software architecture used to build and maintain applications in an enterprise. Rather than designing applications from the ground up, an SOA allows developers to reuse pieces of code packaged as a service and share them between applications and company departments. This lets developers combine resources from all over the company. It typically yields cost savings because of that, while better aligning IT services with business processes. This has caused SOA to be hailed as a major technology breakthrough, but at the same time it requires vast changes in corporate policies and investments in new software products to manage the architectures.

While most observers said that the Burton Group had overstated the problems and that SOA still generates plenty of interest, the Burton study did put its finger on an important issue: how should one measure a project's success? And what metrics should an organisation use for this process?

SOA guru David Linthicum, author of the forthcoming book Cloud Computing and SOA Convergence in Your Enterprise, thinks that the Burton report has been misinterpreted and made some important statements about SOA. What Burton highlighted was the difficulty of assessing the success of a project, he says.

For Linthicum there are three basic approaches to measure a project's success. An enterprise can use one of the methodologies from the likes of the Open Group, that follow a standards-based assessment approach. It can also use methodologies from a vendor, or a firm can use its own methodology devised in-house. Linthicum says that the most successful approach is one that marries a more rigid methodology with an enterprise's own customisation.

He also points out than an SOA is a process, not something tacked on to an existing way of doing business. "SOA is something you do, not something you buy," he says. "Anyone treating it like something you can buy is doomed to fail."
One of the reasons why projects are hard to value is that SOA is based on a variety of factors. A project has both an IT element - the overhaul of an organisation's entire computing system - and a business element - creating applications by combining trusted services. Such disruption has to be driven by a business case, and that is what needs to be defined from the outset.

IBM is a major player in the SOA market. The company's SOA business leader, Gary Gomersall, argues that a high level of co-operation is required to ensure the success of an SOA installation, while the business case has the highest priority. "I can't emphasise strongly enough how important it is to develop SOA close to the business process." But, he says, the fact that business people and IT staff have different priorities and talk a different language should not necessarily be a barrier to success. "Once the IT guys are talking to the business guys, they can often find short cuts between them that give real value and save a lot of complexity later on."

Gomersall says that after having determined objectives, the next step is to draw up a two-dimensional matrix, based on a series of what he calls Key Agility Indicators and Key Performance indicators. The first would include features such as ‘time to market for product' or ‘time to new channel', the latter indicate factors at the SOA level that drive business success.
He also adds that it is important to keep metrics down to a manageable number. "We're not talking about hundreds of web services. Enterprises can only actively manage 40 or 50 metrics effectively," he says.

David Linthicum agrees that the important thing to do is to make sure that the business side has plenty of input into the process, as the business need should be top priority. He emphasises that any SOA project should have a powerful business sponsor and requires the co-operation of stakeholders from all levels of the business.

To sum up, a successful SOA project requires close co-operation between the business side and IT side of the company, with the business team taking the lead. It needs a corporate executive sponsor as well as a clear methodology, defined from the outset, which keeps the number of metrics to a minimum.