We are facing uncertainty in the technology world. The future is less predictable (if it ever was before) and the pace of technological change is rapidly advancing. Old methods of delivery, slow, deliberate and costly are being shunned in favour of Agile working methods in all senses in which the term is used.
But whilst the masters of Scrum can deliver greater effectiveness in the manufacture of software, achieving actual business change and benefit through technology is as elusive as ever. In many ways, the advent of Agile has made many of us turn our backs again on the thorny issue of managing the changes in behaviours and attitudes that are a necessity of successful delivery. If Facebook can garner a seventh of the world's population within it's membership, surely the days of having to worry about adoption are over?
Recently, though, I wonder if there are valuable lessons that can be learned from an approach to technology delivery that had its conception in, of all places, the coal mines of post-War Britain...
By the end of the Second World War 70 years ago, the mining industry in the UK was on its knees. Subjected to decades of under investment, and newly nationalised, the decision was taken to adopt the best coal extraction technology available on the market. Significant investment was made, with the adoption of the Long Wall method for mining which had been proven in its efficacy in the United States. The new technique relied on electrical conveyer belts, drills and picks, adding a whole wealth of technology to the dynamite, pickaxes and brute force that had come before.
The result of this massive investment into the best technology? Mediocrity. Having seen steady declines over many decades, the Long Wall approach made marginal difference. The improvements expected didn't materialise. Coal production barely made it back to pre-war levels.
A group of researchers from human resources experts Tavistock Institute were tasked to investigate. In their meticulous examination of what had happened, it became clear that whilst the new technology had been implemented, nothing else had been. From worker shift patterns to inter-team communications, the adoption of Long Wall had been on a "suck it and see" principle. It was thought that the workforce would adapt around the new technology. Mine productivity proved otherwise.
Heaven forbid that today, 70 years later, we should run projects where we expect that implementing best of breed technology will miraculously deliver significant business benefit. Yet time and again I see organisations looking to the US, selecting best of breed services, and then expecting something organisationally complex like, say, increased collaboration, emerge phoenix-like from the ashes of failed IT implementations past.
The traditional and modern methods of software development are good when you know what it is, in absolute terms, that you are trying to deliver and have case history to show what you should do to achieve that goal. But for something nebulous like collaboration they fall short: waterfall methods struggle to engage when an organisation doesn't know specifically what it wants. Agile methods can iterate, but again if end goals are fuzzy then iteration can perpetuate forever. Internal projects, with political will behind them, don't have the hard stop that does for 90% of tech startups.
In 1966 the philosopher Karl Popper gave a lecture in which he described a world that was made up of logical, complicated but ultimately knowable things that were like clocks, and chaotic, amorphous, things that were like clouds. The metaphor is a good one if you can cope with the double meaning we now have with Cloud from the "as-a-service" industry: clock-like problems can be addressed by analysis to eventually find the right answer that can then be objectively implemented. Gathering more and more data about cloud-like problems just leaves one more and more confused, and the only way to deal with a cloud is to just get in and start working with it. It will change its shape around you the minute that you do.
The Tavistock Institute is well known for Socio-technical thinking, which defines where we have the most to learn to address Cloud-like problems. I'd argue most of the thorny business challenges that we face today are distinctly cloudy. In 1976, Tavistock's Albert Cherns published a paper in which he described six key facets of a socio-technical approach:
1. Compatibility of process
The way in which you run any change initiative needs to be able to resonate with the desired outcomes of the project. So, for example, if you are running a project to increase the collaborative capabilities of your organisation, you need to ensure that it itself is run in a collaborative manner, with proper engagement from people across your organisation. You can't command and control greater collaboration.
2. Organisms or mechanisms
The tradition of the industrial era has been to distill the workings of an organisation into machine-like (indeed, clock-like) structures. Roles for people are turned into cogs in a bigger device. Division of labour, as first evangelised by Adam Smith, turns people into replaceable, divisible units of work.
That's all fine if you are addressing a clock-like problem where the outcomes are known and little-changing. If you are producing pins all day, every day, then you can align labour and machinery in ways that optimises delivery. But if tomorrow you need your pin factory to start producing staples. or paper clips, or even cheese - well, that inflexibility is the pay-off for productive efficiency. Cloud-like problems need multi-skilled, T-shaped workers who can adapt and change as circumstances fit.
3. Minimum critical specification
If you don't know what your outputs need to be until you start delivering them, don't bother trying to document every option. The world of Agile has the concept of Minimum Viable Product which, on first look, appears very similar to the much old concept of Minimum Critical Specification. But too often with Agile-in-the-Enterprise, the list of wants and needs is still exhaustive, and when early iterations appear there is still the disappointment of "where's my feature?".
For Cloud-like problems, keep the list of wants and needs to a bare minimum. It's not until you are interacting with the world that you'll really understand what the specification should be.
4. Controlling Variance at Source
In a world of unknowns, problems and challenges will arise. In an organic organisation addressing Cloud-like problems, the power to address those challenges needs to be given as near to the source as is possible. This is a lesson that has been well learned in the world of manufacturing, with pioneering organisations like Toyota. White-collar work is perversely often still much more hierarchically appraised.
5. Appropriate information flow
If you are going to give everyone the ability to fix problems, you also need to give them the information to spot them in the first place. We are familiar with management reporting; we are also increasingly familiar with the usage reporting that is delivered on Web-based applications. But how often is meta-information about the ways in which systems are used being delivered directly to the users to help them shape the ways in which they can evolve the use of the technology?
6. Boundary locations
The final area that Cherns identified is another where the world of office work is bizarrely behind the world of manufacturing. As a rough rule of thumb, if you want to find where there are problems in an organisation, look to where there are hand-offs. Workflow between departments is almost certainly going to be the cause of strife.
A machine shop used to be organised on the basis of the skills deployed: the milling shop, the grinding team, the forge, the polishing shop. But again the modern manufacturing models used in industries like automotive place teams around the product.
The white-collar functions of the modern organisation, from sales to marketing to finance to HR, are structured functionally like a machine shop of the early 1900s. Matrix management is a tacit admission that the organisational structures don't work - and cloud-like problems might push these structural issues to the forefront.
There are a number of adoption success stories within the world of IT that are often overlooked because, to a great extend, their adoption had nothing to do with IT: email (loved or loathed) is the base communication channel for almost all businesses; Excel is the place in which most business logic happens; mobile was the story of BlackBerry wheedling its way into businesses through end user demand. What all of these services have in common is the way in which they are platforms that empower users to build upon them.
Taking a socio-technical look at how to deliver change in cloud-like scenarios could well provide a template within which we can help to shepherd change in ways that our traditional, output focused project methods have often failed in the past.