In March 2013, I read with great interest the results of the University of Cambridge analysis of some 58,000 Facebook profiles. The results predicted unpublished information like gender, sexual orientation, religious and political leanings of the profile owners.
In one of the biggest studies of its kind, scientists from the university’s psychometrics team developed algorithms that were 88 per cent accurate in predicting male sexual orientation, 95 per cent for race and 80 per cent for religion and political leanings. Personality types and emotional stability were also predicted with accuracy ranging from 62 to 75 per cent. The experiment was conducted over the course of several years through their MyPersonality website and Facebook Application. You can sample a limited version of the method for yourself at http://www.YouAreWhatYouLike.com.
Not surprisingly, Facebook declined to comment on the analysis, but I doubt this information is news to anyone at Facebook. In fact it’s just the tip of the iceberg. Facebook as a global technology company (arguably a global data company) has without a doubt, far more complex algorithms trawling, interrogating and manipulating its vast and disparate data warehouses, striving to give its demanding user base ever richer, more unique and distinctly customised experiences.
As an IT leader, I’d have had to be living under a rock to have missed the Big Data buzz. Vendors, analysts, well-intentioned executives and even my own staff – everyone seems to have a Big Data opinion lately, and most of those opinions imply that I should spend my budget on Big Data.
It’s been clear to me for some time that we are no longer in the age of ‘what’s possible’ when it comes to Big Data. Big Data is big business and the companies that can unlock, manipulate and utilise data and information to create compelling products and services for their consumers are going to win big in their respective industries.
Data flow around the world and through organisations is increasing exponentially and becoming highly complex; we’re dealing with greater and greater demands for storing, transmitting and processing it. But in my opinion, all that is secondary. What’s exciting is the developments to enable better customer service and bespoke consumer interactions that significantly increase value along all our service lines in a way that was simply not possible just a few years ago. This is what’s truly compelling. Big Data is just a means to an end, and I question whether we’re losing sight of that in the midst of all the hype.
Why do we want bigger or better data? What is our goal? What does success look like? How will we know if we have attained it? These are the important questions and I sometimes get concerned that – like so often before in IT – we’re rushing (or being pushed by both consultants and solution providers alike) to find solutions, tools and products before we really understand the broader value proposition.
Let’s not be a solution in search of a problem. We’ve been down that supply-centric road too many times before.
For me it’s simple – innovation starts with demand, and demand is the force that drives innovation. It all starts with a problem that needs solving, a value experience for our customers. Only through a deep understanding of what value means to the customer can we truly be effective in searching out solutions. This understanding requires an open mind and the innovative resolve to challenge the conventions of ‘How we’ve always done it’.
Candidly I hate the term Big Data. It is marketing verbiage coined by Gartner which covers a broad ecosystem of problems, tools, techniques, products, and solutions. If someone suggests you have a Big Data problem, that doesn’t say much as arguably any company operating at scale, in any industry, will have some sort of challenge with data. But beyond tagging all these challenges with the term ‘Big Data’, you’ll find little in common across diverse industries, products or services.
Given this diversity across industry and within organisations, how do we construct anything resembling a Big Data strategy?
We have to stop thinking about the supply of Big Data tools, techniques and products peddled by armies of over-eager consultants and solution providers. For me technology simply enables a business proposition. We need to look upstream, to the demand. This demand presents itself in business terms. For example in financial services you might look at:
• Who are our most profitable customers and, most importantly, why?
• How do we increase customer satisfaction and drive brand loyalty?
• How do we take excess and overbearing processes out of our supply chain and speed up time to market?
• How do we reduce losses to fraud without increasing compliance and control costs?
Importantly, asking these questions may or may not lead us down a Big Data road. But we have to start by asking them.
Similarly, the next set of questions are also not about specific solutions but about framing the demand and the potential solutions:
• How do we understand the problem today? How is it measured? What would improvement look like?
• What works in our current approach, in terms of the business results? What doesn’t? Why? What needs to improve?
• What are the technical limitations in our current platforms? Have new techniques and tools emerged that directly address our current shortcomings?
• Can we develop an experimental approach to test these new techniques, so that they truly can deliver an improvement?
• Having conducted the experiment, what did we learn? What should we abandon, and what should we move forward with?