Autonomy CEO Mike Lynch is arguably the biggest star of the UK business software scene and yet when we meet for lunch in Soho, London, he walks in free of PR minders- or posse and apologises for being a few minutes late. He is unassuming, unpretentious and quite the opposite of the ego-crazed Master of the Universe types that run most modern tech giants. Instead he recalls - a little starry-eyed - having been photographed in the nearby legendary jazz club Ronnie Scott's for a profile piece that riffed on this sax-player's love of music.

He's a popular profile subject, of course, having set up Autonomy in the mid-1990s - the company now turns over more than $500m per year. Analysts love the firm so much that it has a premium market capitalisation of £3.5bn and Lynch has been lauded by fawning journalists - perhaps wide-eyed at his personal- holding - who make wild comparisons to Bill Gates.

You get the sense that none of this means that much to Irish-born, Essex-raised Lynch himself. He has always been confident about his company but overt marketing, hype and aggrandisement are foreign to him. When I ask him about the recent wave of market sentiment in favour of Autonomy, he bats it off.

"You can be lulled into the idea that it's always going to be like that. We went public in 1998 and not much happened," he recalls. But he is proud that his still-young company has reached one peak, having overtaken Sage as the most highly valued UK business software concern.

In part this is down to a smart segue from enterprise search (or "meaning-based computing" as the firm likes to describe its mission) to the hot sector of - e-discovery. Because of new demands for corporate governance, companies are having to stump up relevant information in the face of government, legal or industry regulatory probes and are seeking software to help them do so. Since its 2007 acquisition of a specialist in the sector, Zantaz, Autonomy has an solution in hand that is a nice source of income as many firms pull back their spending on less urgent matters.

In this world "FRCP [and similar rules] is the biggest problem for the CIO", Lynch contends, referring to the Federal Rules of Civil Procedure, and adding that such governance strictures change the legal rules of engagement for businesses. The bottom line today is that organisations need to provide an audit trail of their actions - or have a very good reason for not doing so.

"The principles of discovery go back to the Romans," Lynch argues, adding that the more recently developed "computer ate my homework" defence is no longer valid. "Senior management wants all the boxes to be ticked and nobody wants to be on the board. [What is required is an] early case assessment and an ‘am I guilty' button."

More subtle "sleuthing" is becoming available, he says, adding that audio-based sourcing of evidence is becoming more important and that it is now possible to create "conceptual understanding" so that rogue traders and their miscreant activities can be spotted earlier and with greater -certainty than was previously possible.

Three certainties

In a market where spending on potential revenue-spinning projects is at a low, this need to invest on behalf of IT buyers is, I suggest, reminiscent of Computacenter boss Mike Norris's statement that there are only three certainties in life: death, taxes and IT spending.

"Well, death is at number one and then there's taxes... but someone else has got a monopoly on that one," Lynch says. "The legal system is catching up with reality - this is stuff they can't defer. But the driver [of the overall Autonomy business] is still unstructured information."

The rise of unstructured information - essentially, text strings, images, audio, web pages and anything else that isn't numerically based and therefore easy fodder for databases and spreadsheets - has been meteoric, with various analyses suggesting that 80 per cent of the world's data is unstructured and that unstructured data growth is currently outpacing that of structured data.

In this area, most experts give Autonomy a significant lead and note that its code is resold by hundreds of licensees. Lynch sees the segment breaking into the tripartite model first suggested by Forrester Research consisting of free tools at the bottom, a mid-range sector- that includes Microsoft and a high-end led by Autonomy.

Despite this, he is realistic about how far Autonomy has come and how far it has yet to go in terms of achieving the seemingly impossible task of interpreting, classifiying and indexing the incredible complexity of human language, gestures and the rest of the ways we communicate.

"The reason is why Autonomy doesn't have any direct competition - it's very different and [the area is] very difficult," he says. "The problems are not [easily] solvable. [To engineer software in this context] you look at the real world and massively simplify it, but the real world is incredibly subtle, complex and changes all the time."

Despite this, Lynch says that the sector stands fair to host one of the biggest changes in business software since the relational database or even the internet, and the payback is obvious in terms of reducing the number of manual inputs required.

"The real value is you're removing a person," he says. So is the rush of the new, new thing enough to keep Lynch at the helm for a while longer? The answer would appear to be ‘yes'.

"The first 40 years [of business IT] were a diversion. We think it's the biggest revolution since the ‘i' [in internet]. If you're a technology company it's a one-off, once-in-a-lifetime opportunity. If you're on top of the wave keep riding it."

Purchase power

Following the success of buying Verity for $500m in 2005 and the afore-mentioned $375m Zantaz capture, Autonomy followed up early this year with the $775m acquisition of Interwoven, a big name in enterprise content management. However, Lynch is not interested in following the likes of Larry Ellison's Oracle in seeing value in providing more licence-accruing money to the company. He is more interested in moving quickly as Interwoven will allow Autonomy to do in the legal sector with Autonomy's IDOL core technology acting as an underlying "jet engine".

"Software acquisitions are actually a very bad idea so you only do them when they're perfect," Lynch says. "Verity was the biggest company in the space when we were two men and a dog. We're not interested in an Ellison-esque roll-up model [but] when you have the lead, you have to want to accelerate."

Lest we think that everything is rosy in the garden, however, Lynch remains outspoken on the subject of being a public company. On one previous occasion when I interviewed him he seemed to suggest that he would not have entered the business if he had known just how difficult running a public company could be. This time around he is particularly critical of certain practices used to pump up or deflate stocks in an irresponsible, artificial manner.

"It's a bit like going for a swim in the sewer," he says. "There were periods when it was really dodgy. The problem is that if you want to be a world player it's hard to do it as a private company."

Such is his success with Autonomy, Lynch is now regarded by many as an éminence grise of technology. He is a non-executive director of the BBC's executive board, insisting that whatever criticisms it attracts, the national broadcaster is an asset to be treasured. He also advises venture capital firms and startups, including Isabel Healthcare, a company that has developed a decision-support system to be used in clinical diagnoses.

He could, if he so desired, probably reinvent himself as a superior version of Sir Alan Sugar, prognosticating on the future of UK (and Irish) business, entrepreneurialism and suchlike. But his view of these matters is less populist and probably too subtle for TV and today's five-second attention span.

"I only get involved with companies if I think they can do something good," says Lynch. "We have amazing talent here [in the UK and Ireland] but we've been very bad at building the industry."

Banking on regulation

Similarly, he declines to give the bankers the kicking that many pundits have proffered in the past few months, and can see both sides of the argument on controls.

"In the UK the regulator has been a little bit too much in bed with the poachers. The FSA has a lot of ex-bankers running it but then does anybody but an ex-banker understand how it was done?"

Lynch is dismissive of Nicholas G. Carr's Does IT Matter? critique of binary sources of differentiation ("It should be retitled Does The Database Matter?" he says) and says that audio e-discovery will bring major changes. He is too much of a technophile to buy into the notion of digital commoditisation. When I say that I must leave to attend a press conference on Microsoft's Surface technology - a sort of table-top, gesture-sensitive computer and visualisation system - he says he wished he could go too.

The search for search supremacy

Although Autonomy is the current leader in enterprise search - at least as it stands when applied to contextual understanding - there are plenty of other firms attempting to knock it off its perch. A list of some of the better known contenders would include Microsoft (especially since the acquisition of FAST), IBM, Google and Endeca.
However, for many organisations, the business of understanding the masses of data held in their datacentre is only just beginning. According to recent research conducted by Vanson Bourne on behalf of another search firm, Recommind, just 29 per cent of staff members at financial services firms are able to tap into the knowledge or expertise of colleagues.

Knowledge management is clearly lacking at many organisations but so is the ability to search for even basic data and the same research found that many employees spend half a day a week looking for the information to let them do their jobs.

Recommind European director Simon Price said that "invisible silos" build up due to the accretion of data, departments and tools available as companies evolve, merge, demerge and make strategic changes.

However, software alone won't fix the problem. Instead, firms must commit to process-driven, standardised tagging of data in order to make it discoverable to colleagues and future employees. Without such processes, firms run the risk of "drowning in a sea of data" and leaving themselves open to security vulnerabilities and governance compromises, as well as charges of plain inefficiency.