This is a guerilla blog post from PNSQC.
My great friend and fellow testing blogger Trish Khoo recently wrote a post inspired by Goranka Bjedov’s keynote at STANZ 2011. In her post, Trish reacts to some news delivered by Goranka at STANZ: “Quality is Dead.”
This morning I’ve had the pleasure of listening to Julian Harty play understudy for an ailing Goranka with his opening keynote for PNSQC. I am guessing that although Julian gave a talk that seemed very much his own, the heart of what he was saying is similar to what Goranka was exposing at STANZ. (You can bet that as you are reading this, I have absconded to watch the video of her keynote.)
What I noticed not just in Julian’s keynote, but the subsequent presentations I attended today, was not a death so much as a rebirth, a liberation and even a tearing down of walls. At the end of the keynote, I tweeted that I can feel another paradigm shift happening not only testing but also in software.
In the keynote, Julian talked about how Dr. Barry Boehm has moved on from saying that bugs are found more cheaply earlier in the software process. Instead, Boehm has been writing that it is now just as cheap to fix bugs in production. This was the perfect setup for Keith Stobie’s presentation, “Testing Services in Production”. Keith drove home the point that he is currently much more comfortable testing in production than testing in any type of “staging” environment.
Aside from moving testing into a production environment, there is some testing that Keith doesn’t even call testing anymore. He calls it “monitoring.” This is where the wall between testing and production really crumbles away. He talked about moving asserts from tests into the production code. I’m sure that this is nothing especially new. What’s new is that now there are people relying on this code as an indication of system health.
In his presentation, “An Introduction to Customer Focused Test Design“, Alan Page (who got us to lunch on time) talked about how his team has been getting away from even doing functional tests by focusing on “ility” testing instead of functional test design. This in essence, means that his team is much more focused on performance testing.
What does all of this mean for those of us who consider ourselves “QA” or “Testers” or some similar combination. It means that our jobs are changing and will probably look very different in a few years. I lost count of the number of times Julian mentioned “testing not being around,” “automation experts being out of a job,” “testers being out of a job” or even questioning the validity of a testing conference.
As someone who is currently employed to primarily work with test automation and test infrastructure, dId this make me uncomfortable? Not at all. In fact, it’s very exciting. You see, I’m here because I love change and innovation. If there is something better, I’m all for tearing away the cruft and exposing the new. It might mean that I start adding code to the app I work with for web timings or that I end up with a job title that has no trace of “test” or “qa.” The focus is undoubtedly shifting away from looking for bugs and presuming to know what customer’s want to making sure real data is available about system health and that we know how to interpret it.
How does all of this apply to my talk on “Hard Lessons About Soft Skills?” All of this means that we’ll be more embedded with the devs than ever before. In fact, it might even mean that we will be the devs. If they don’t want us on their team anymore because we’ve burned our bridges with them and we are sheltering behind the wall of QA, what’s gonna happen when the wall comes down?
No matter how things shift, I can guarantee that I’ll still be around and giving a shit about how well the software is working and whether or not customers are happy with it. Are you ready to be liberated?
I’ll leave this guerilla testing post with a classic from LEGENDARY rockers, “The Scorpions.”
As a grizzled veteran of the IT world then colour me cynical when paradigm shifts come along ( 4GL and the death of programmers anyone ? ) so I’ll see what comes out of this
“just as cheap to fix bugs in production” – Hmmm,OK, I dont think I want a Boeing 797 to be uploading a patch mid-flight nor a BMW 456 to be finding it has a defect in the braking system as the traffic lights approach. A modern car has around 100 million lines of code – is this going to be crowdsourced ? Are banks going to monitor Twitter to see if their customers are complaining about their accounts being wrong ?
What about the massive amount of legacy code out there ? Would be great if all that could be wiped out and we could start afresh with this new approach but thats not gonna happen. How are fixes and enhancements going to be tested – or is this the area going to become the elephants graveyard for those testers that cannot change ?
And in the week when Steve Jobs was lost, would he tell people to fix the bugs in production ? It might be ‘as cheap’ but is it the best thing to do ?
I’n all for a paradigm shift and new approaches – finding the same kind of defects I was writing and finding 10-15 years ago is very dull ( which is why I’m making a radical job and location move myself ) – so will keep an eye on what comes out of the conference. keep those guerilla posts coming
And if there aren’t going to be any testers, who will be buying the forthcoming “how we test software at Google” book ?
Thanks, Phil, for such a terrific comment.
Let me clarify what I mean when I say that we are in a paradigm shift. When I say “paradigm shift” I’m talking about large scale changes that happen over a period of years. An example of what I consider a paradigm shift is the move from software in a box on a shelf, to an online service such as Flickr that is consistently updated.
Here’s another question: do we want testing to be the same thing forever or is there room for exploring new ways of critically eyeing an application. The way applications are delivered and what they even encompass is constantly changing. Doesn’t that mean there are places in testing that change as well?
Regarding defects in airplanes, Julian talked about the Qantas Airbus 380 which was forced to land last year because of engine trouble. It was known that the engines in these planes might have this problem, but the maker of the engine, Rolls Royce, decided to wait and see if it was really a problem. Like it or not, we are already flying in planes which are testing in production.
Regarding the new book about testing at Google: someone should ask them if the strategies in the book are what they are using today. I know that their testing organization has been disbanded.
So, we can call this a fad, but I don’t think it is. We are all dealing with this paradigm shift now as testers and as consumers. Grumbling and saying it’s a fad are complaints I expect. Personally, I might keep an ear open for grumbling that I think is substantial, but I’m also going to be pretty busy embracing the changes to find what about them works and what doesn’t.
Thanks for the reply and clarification
I really really dont want to be testing the same way and finding the same problems so I hope something comes out of the conference apart from the headline grabbing quotes
One concern/question I have is that whilst this approach might be working for Google/Facebook they also happen to have some of the top talent – what happens when firms with less talent try to copy their approach ?
And I’ll definitely find out if the Google book is about current practices before I think about buying it :)