Ever since the Web became part of the collective consciousness, search tools have become critical applications for computer users, from Alta Vista via Excite and Inktomi to the long-running and ongoing leader today, Google. However, more quietly and with some lag in time, there has also been a very significant rise in the use of enterprise search tools that identify and index business content so it can be searched for by permitted users inside or outside the organisation.
Traditionally, enterprise search has been a means of harnessing knowledge management for internal users and selected partners on an intranet-style model although increasingly it also underpins consumer-facing websites, providing a means for owners to attract and keep customers and prospects through more intelligent use of taxonomy and other tools.
Gartner estimates that the total enterprise search market will be worth over $1.2bn by 2010 but definitions of what constitutes the enterprise search market today should be treated with caution as the sector morphs into other areas.
Indeed, it's becoming difficult to say where search ends and other related fields begin as the segment has sucked in classification, analytics and presentation layers by most people's definitions.
Demand for search and related tools has driven the rapid rise of companies like Autonomy (which helped ignite the sector- by buying Verity in 2005), FAST (now owned by Microsoft) and Endeca Technologies, as well as Google, which is hoping- to translate its consumer success to the business-to-business world.
For many companies, these vendors are providing the new critical applications that can make or break the fortunes of organisations - and for the CIOs that -select them as partners.
At Google, the company is trying to repeat tricks from the consumer side to change the face of enterprise search.
"Enterprise search or universal search is an exciting part of our enterprise business," says Robert Whiteside, the firm's regional head of enterprise business. "People are looking at how they get the best out of their business and it's a way to get information to people who make decisions. In challenging times people want to get the most out of what they've got and what's needed is simple to use yet powerful search, supporting the day-to-day job of finding information. There's a huge opportunity and people need to leverage their investments."
As an example, Telegraph Media Group - publisher of the Daily Telegraph, Sunday Telegraph and telegraph.co.uk increased its traffic and public-facing conversion rate as well as providing internal staff with better insight, by using the Google Search Appliance, a server-based solution.
Google claims to have about 30,000 business search customers globally,- selling direct and through partners but without offering professional services. Customers include BP, BA and Apple, while partners include Capgemini and SCC.
What next in search? Some vendors may head off towards niches, following the likes of Recommind in the legal sector. Following Google's lead, hardware-housed search might catch on more broadly and companies like Thunderstone or Scan Computers are already pursuing the line.
Other pundits believe that enterprise search could follow sectors such as email, CRM and sales force automation in being delivered as a web-based service carrying a subscription tariff. Also, for those uncertain as to their next steps, there -exists a plethora of free offerings from IBM, Microsoft and others to give firms a toe in the water.
Another broader trend is seeing search become tied inextricably with content management, archiving, portal and collaboration tools, and several firms from IBM to Oracle and Autonomy to Open Text, and even Microsoft, will offer themselves as one-stop-shops.
As Gartner analyst Tom Eid has said, "No one vendor can solve all information access needs. Search and information access is not a one-size-fits-all marketplace, and the market will continue to develop, whether within individual devices, through organisational intranets, across the Web, or specialised in a digital asset type."
But the sector that is booming today is search as an adjunct to e-commerce and one company that is behind many of the new search-enabled sites is Endeca.
Jody Goodall is head of research and development for Trader Media, owner of Autotrader, the UK's leading car marketplace magazine and Autotrader.co.uk website that receives about one billion impressions per month. Like many others in the commercial media sector, Autotrader has seen its revenues and profits swing from physical to digital and it is using Endeca's software to maximise the stickiness and profitability of its site.
"We can throw everything into Endeca to see relationships between what we are doing and sales, and ensure we don't go down negative paths or zero-view indexes," Goodall says.
"It's scary because you get this lens sometimes on things you don't want to know. We know that the Aston Martin DB9 has been more viewed more times than the number of models that have ever been made and that there is a nine million audience drop during Top Gear."
On the plus side, Goodall can gain instant insights into what search criteria users are employing, create breadcrumb trails of their visits and see where and why customers' searches go cold, or where to place related contents and other objects.
For Omar Haque, a senior IT consultant at RS Components, a UK reseller of electronics products and parts, Endeca offers the chance to let customers find obscure SKUs.
"ERP and mainframe aren't where the value is," he says. "It's at the edge, not at the heartbeat. It's ‘How do you find the resistor in the haystack?'"
Similarly, a third Endeca customer, Shop Direct, is using the firm's tools to help it transform from a predominantly high-street and catalogue retailer to a multi-channel force.
Endeca chief strategist Paul Sonderegger says the firm offers a way to "change the way companies compete in a way regular people can understand".
Spying an opportunity, Google late last year launched a hosted enterprise service called Commerce Search, arguing that it could help retailers provide up to 10 times better conversion rates than the current three per cent.
Footwear seller Birkenstock USA said the service had improved satisfaction ratings. "Bounce rates have decreased and we're seeing more return customers,' said Jeff Kilmer, Birkenstock's COO.
"The search results are ultra-fast, so customers more easily find the specific products they're looking for. We deployed quickly and easily, and we've seen dramatic conversion improvement since implementation."
However, newer players are also making their presences felt in areas that impinge on the customer-facing enterprise search sweet spot. Fredhopper of the Netherlands has built a $5bn business in just 10 years with customers including Clarks, B&Q, Jessops, Toys R Us and House of Fraser.
But the search sector continues to transform and no company better exemplifies its mercurial nature than Autonomy.
"Regulation and litigation remain search drivers and we're seeing a lot more on e-commerce so companies understand what people are interested in and what they're trying to buy rather than just relying on the old keyword box," says Nicole Eagan, chief marketing officer, pointing to the John Lewis site as an example.
Rich media search is also becoming more important. "It's associated more with consumers but we're hearing CIOs asking ‘what are the effects of You Tube and Tweets on my brand?'"
Audio and video evidence is also increasingly being called up as evidence ion e-discovery processes, for example from voice calls and WebEx meetings and search could even play a part in anticipating broader trends.
"We're helping CIOs identify patterns in information graphically for example through call centre enquiries and trading data. You could argue that search could have prevented the banking crisis..."