EuroSTAR is the annual gathering in Europe for the software testing industry. I gave a keynote address at the November 2011 event in Manchester, and attended it again in November 2012 at the Amsterdam RAI Centre.
I have written about software testing before, and declared an interest – I chair the not-for-profit TMMI Foundation, promoting a process maturity model designed to help evaluate the effectiveness and the efficiency of testing processes in our increasingly complex age.
In this fast evolving era of the virtual, the cloud and mobile, what testing can and should be about is changing. There is a real risk that testing’s vital contribution to business performance and business operational risk management is not well understood at board level. It does not help that the testing profession can come across as highly ‘techie’ in its work.
Yet, as I described in my June 2011 column Testing 101, when Geoff Thompson of the testing consultancy Experimentus and I asked “What is the essential purpose of (software and systems) testing?”, we concluded that it was confidence in outcomes. The board needs to have confidence that, when it invests significant resource in new software and systems capabilities, intended outcomes will be delivered, on time and to budget. And that is what testing is really about – assurance.
I spent a day at EuroSTAR 2012 visiting exhibitors with one objective: to see how the ecosystem of technical offers is developing in the context of the impact of the virtual, the cloud and the mobile.
My first observation was of a battleground of theology and terminology. Two broad approaches to software development and related testing methods (so called ‘dev and test’) are now in play: Waterfall and Agile. Each is a discrete and powerful methodology in its own right and each has its own broad arena where its best practice is of genuine business relevance and application, even if such broad labels as ‘Waterfall’ and ‘Agile’ often confuse rather than clarify.
But clarity and insight were there at EuroSTAR 2012 with the work of the US research house, Voke Research. In a study published in June 2012, Voke opens with its version of the importance of confidence in the delivery of outcomes, that, with software at the core of most business operations and frequently a key competitive differentiator, ‘a software failure is a brand failure’. This is testing as risk management.
Voke focuses on the reality that testing has to address all aspects of the full lifecycle of a new software or systems capability: from developing requirements specifications (which will need to flex over time), through ‘dev and test’, into operational life on increasingly virtual ‘compute and network’ infrastructures. The company maps and evaluates the wide range of testing technologies, approaches and vendors that this fast developing marketplace has on offer.
While confirming a leading position for HP in this field, Voke details the importance of a great variety of smaller, innovative ventures responding to and exploiting the triple opportunity of the virtual, the cloud and the mobile.
At EuroSTAR 2012, we met a host of such firms: Codenomicon, a Finnish company focusing on assurance of the security dimensions of new software – a horrible corporate moniker (‘defensics’) but a powerful test capability; Electric Cloud (US) with its focus on the automation and acceleration of ‘dev and test’ through the exploitation of the virtual and the cloud; Neotys (US) with its focus on load testing web applications at the ‘dev and test’ stage and during their deployment process to assure operational robustness; the Dutch Jamo Solutions, focused on mobile apps; and many more.
Perhaps the new identifier for software and systems testing should be software and systems assurance. A CIO, seeking support for investment in new software-enabled capabilities, will now need to deliver to the board assurance across a broad front: delivery of intended business performance objectives, to budget and on time; operationally robust and secure (especially across the internet and in the cloud); full operational compatibility across a diversity of mobile platforms; and full protection of corporate data assets exploited by the new systems.
Is it time we had a CSSAO (Chief Software and Systems Assurance Officer)? A career motivator for the ambitious software testing professional, surely, and certainly for those who know their EuroSTAR from their Eurostar.