Rising data levels and real-time web applications are driving a shift towards NoSQL databases a move that could be as significant as the switch from mainframe to relational databases 30 years ago.
Today's relational database management systems (RDMS) are good at dealing with generic data, but their performance is limited when it comes to high volumes of unstructured information.
This less structured data is much better served by NoSQL, which uses a more open approach to how information is stored and manipulated. The open nature of NoSQL has already won it a prominent place with large-scale internet firms, including Amazon, Google and Facebook.
As data requirements change, an increasing number of organisations are being enticed by the flexibility and low cost that open source database tools such as Apache Cassandra can provide. NoSQL appeals to CIOs for its ability to scale and is ideal for start ups as well as the growing breed of organisation using data heavy applications.
However, NoSQL databases won't suit all businesses. Transactional data is best served within relational databases and NoSQL cannot be used without add-ons such as analytical tools. NoSQL also work in an entirely different way to relational databases, forcing CIOs to recruit new skills and practices.
Unlike relational databases, NoSQL is a schema-less platform, so users do not need to design the tables, columns and rows using the same methods.
This means changes to the way the database works can be made "on the fly", says Clive Longbottom, analyst at Quocirca. "You don’t need to do table joins or other actions that are needed within the relational world."
Firms opt for NoSQL because it allows multi-data centre performance workloads, which can not simply be added to a relational database, according to Matt Aslett, research director at 451 Research.
As organisations start to migrate data heavy applications across to NoSQL, it calls into question the role of the traditional IT vendors such as Oracle.
"It's a new market and an interesting dynamic," Aslett says. "There are workloads deployed on NoSQL that were previously on relational solutions, so in terms of database we have more choice - and people are more prepared to look around for options."
The NoSQL market is gradually growing. According to 451's data, NoSQL revenue was $184 million in 2012 and will be $1.1 billion by 2016. Geographically, adoption is highest in the US, on the west coast and Silicon Valley which is where many projects emerge. "It's here that they have the people with skills sets," says Aslett. "But there is growing interest across the board."
Adoption of NoSQL is increasing in specific industries, and experts say it's slowly hitting the mainstream, making particularly big waves in telecoms and finance.
NoSQL is also increasingly used for web applications, as well as social networking and on customer facing websites such as retail. "If you look at a company's web presence, you can figure out the different databases selling different parts of user experience - getting back user credentials what you might be interested in," Aslett says.
Unlike relational databases, which are adopted by database admin teams and central IT departments, NoSQL take up is developer led. "So there could be an interesting tension where enterprises get to the point that adoption happens outside of IT department," Aslett says.
As a developer who built his career on Oracle tools, NoSQL was not the obvious next step for Billy Bosworth, CEO of DataStax, the vendor that distributes and supports a version of NoSQL database Apache Cassandra.
But when Hadoop and NoSQL started to emerge in around 2007, Bosworth saw a shift starting to take place. "I started to think; is this going to be a paradigm shift? Or is this going to absorb into main databases? This stuff was all brand new and required fundamental re-architecting of these databases."
Cassandra reduces the big teams needed to work on traditional databases. The cost is also lower; about a tenth of the price of Oracle licences, Bosworth says.
Most of DataStax's customers sit in specific sectors, with retail, media and finance making up the top three. The firm is also seeing demand from the energy and utility sectors as well as healthcare companies.
"In those industries they feel the need for endless customer touch and customer service," says Bosworth. "And you can't keep up with ever increasing data; they have to change their systems."
Among the reasons for migration to NoSQL, Bosworth says customers are seeing the technical limitations of traditional databases. "We hear, 'we've hit a wall' over and over again. They want the advantages of active everywhere, multiple data centres, to run in the cloud. They want an on premise data centre and to expand into cloud; they want both."