Mike McCarthy SkillPages

NoSQL's flexibility makes database technology ideal for start ups. Taxi app Hailo chose the NoSQL database Cassandra because it "makes it easy to do global distribution and it has high availably characteristics", says Dave Gardner, architect at the firm. Cassandra was also the best way for the fast-growing company to scale.

"We wanted to expand globally, and we wanted people to be able to use one app," Gardner says. "We wanted people to fly from the UK to New York and get off the plane and hail a cab. People trust us to be there and to work; we didn't want maintenance windows or downtime due to data centre outages."

Hailo still uses MySQL, but is planning to move its entire app based storage requirement to Cassandra in long term. Hailo will retain relational databases for any information it does not require when a user needs a taxi.

"If we could use Cassandra for everything we would," Gardner adds. "We've kind of installed it and left it. It just works."

Starting on a larger scale, fast-growing music streaming service Spotify began as "pretty heavy PostgreSQL users", according to Nicholas Harteau, director of engineering at Spotify.

But then things got complicated; the company needed to replicate data and had high level problems that drove it to look at other storage solutions.

Driven by its high availability and replication capabilities, Spotify adopted Cassandra in 2010.

"It really is our go-to storage solution," Harteau says. "We still do a lot of PostgreSQL; but less and less as we need it for transactional and relational uses."

Spotify has about 35 clusters for different parts of the service, alongside 350 nodes and around 35TB of data.

Nowadays, the firm values NoSQL so much that it is looking at expanding its use of Cassandra. "In general we are looking at specific use cases," says Harteau. "We need to export large amounts of active data from Hadoop and current strategies don't work."

NoSQL is also a valuable tool for firms who need to process large amounts of analytic information. This is true of online video technology company Ooyala, which currently processes over two billion analytic events a day.

Ooyala's first analytics system was based on Oracle's MySQL, but problems started to appear when the firm needed to scale. "We started running into a lot of problems; having a global footprint became difficult," says Peter Bakas, the firm's director of platform engineering and operations.

And although Ooyala still uses Oracle's MySQL to store certain types of data, Cassandra's ability to scale out geographically was attractive. "Your availability is only as high as your lowest available component, so having a system that is available is key," says Bakas. "Since having Cassandra, if a node goes down, your whole system doesn't go down."

Meanwhile, for website and mobile social app SkillPages, the move to NoSQL and Cassandra was simple and fitted the business. The company is growing fast, with 2.7 million users at the end of 2011, 10.5 million in 2012, reaching around 18.5 million this year.

"Traditionally we started off as a Microsoft house," says CTO Mike McCarthy (pictured). "But as business grew, we used open source to help with scale; we look for tech that works."

As a growing business, scale is important to SkillPages. The firm, which syncs social media accounts, estimates it is processing three to four billion objects on a daily basis.

"Infrastructure out the box is essential to us," says McCarthy. "We could have built it ourselves but it would have been a huge overhead. It's scaling out the cluster now, which isn't a huge deal."

Yet despite its many benefits, Cassandra takes skill and time to use, and can require extensive training, especially for CIOs used to relational databases.

Using Cassandra certainly isn't easy; even experts can shoot themselves in the foot, says Jimmy Mardell, storage software engineer at Spotify. He says that the firm encountered performance problems "before we realised what you should and shouldn't do".

Hailo's Gardner, who runs a Cassandra meet up group in London, says he also ran into challenges when first using the technology. "Initially you have to dedicate a lot of energy to it," he says.

"One of the things about NoSQL is you're making trade offs like experience; we haven't had a single person who's joined the company having worked on Cassandra before, apart from me," says Gardner. "It's a challenge because it's entirely possible for a developer to shoot themselves in the foot. If you get people in with 10 years of experience, it is probably relational."

This experience is integral to smooth running of the system because the way that relational and NoSQL databases work is entirely different. For example, some firms might not realise they will lose analytical queries as part of the move to Cassandra. To mitigate this, Hailo is using Acunu, an analytics platform that sits on top of Cassandra. The firm has taken this approach with all its use of NoSQL. "We've taken the sorts of things that MySQL used to do and we've split it so that a different tool does all the jobs," says Gardner.

Another issue is security: a lack of protocols is presenting a downside to the growing use of NoSQL platforms. Security issues could arise in standard practice such as authorisation, and access to data, including which apps have access to it, says Aslett. "NoSQL has to fit within existing policies but it is not designed to fit into these."

Despite a lack of industry protocols, NoSQL users are taking their own steps to secure data. Security is not necessarily a problem if a firm knows what they are doing; the open source model actually increases the security of software, Spotify's Harteau says. "The more code is transparent and not locked away the more that vulnerabilities are found and discussed. The other aspect from an architectural perspective is we don't rely on Cassandra for security - we use other tools."

With this in mind, much of Cassandra is about protecting data, or what Bosworth calls "disaster avoidance". One of Cassandra's biggest users, streaming website Netflix, takes such an extreme view of this that it has created 'Chaos Monkey', a resiliency tool that helps applications tolerate random instance failures.

Meanwhile, Hailo secures its database by running Cassandra inside a virtual private cloud so it can not be accessed from the outside, and each app runs on its own key space. "Security is more about policy," says Gardner, adding that Hailo has internal meetings, tech talks and industry meet ups to mitigate risks.

Migration to NoSQL is beginning and as adoption increases, it adds much-needed choice to the market. The space is getting bigger; the next tranche of users will be "the mainstream ones", says analyst Longbottom.  "Some of these will go down the mixed SQL and NoSQL route by conscious choice. The rest will find themselves going down this route as their existing databases are upgraded as they review application usage."

But will the traditional IT vendors such as Oracle eventually be pushed out as NoSQL takes hold? Not at this point. "People have written off mainframe - but it continues," says Aslett.

"Relational databases have a very specific role and NoSQL makes up just a tiny proportion. We predict significant growth, but it's still going to be a small percentage of the overall software market."