Last year, I visited Microsoft to give a presentation. Alan Page, one of the authors of How We Test Software at Microsoft was my host. When he introduced me to the audience, he gave me an autographed copy of his book and a pad of quadrille paper (paper with squares instead of lines). He told me it was for drawing models. Apparently this is quite the popular way for testers to understand software at Microsoft (and I hear, at Google as well). I’ve read a lot of HWTSM but I must admit, I had not looked very closely at the model based testing chapter.
The paper Alan gave me has made it to Australia, and I’ve been using it for keeping up with life and stuff. Every time I look at it, however, I keep thinking, “What the hell is this model-based testing about?” So I decided we would check it out for this week’s weekend testing. I told Alan of my plans on twitter and he replied:
Hmm…keeping this in mind, off I went to read his chapter on model-based testing. The models are really just finite state machines. If you’ve taken a discrete math class, you’ve seen these. If you haven’t…it’s pretty simple. Have a look at wikipedia’s example and you’ll see what I’m talking about. In reading through HWTSM I noticed that emphasis was placed on using models to understand specifications. Dr. Cem Kaner’s sessions with Weekend Testing from a while ago verified this to me. I read through the transcript from Weekend Testing 21 in which Dr. Kaner is describing his suggested use of models.
Dr. Kaner suggests 2 ways in which models can be helpful. The primary reason he suggests for using a model is when there is so much information that a tester is having difficulty wrapping their head around all of the possible states they can create and need to test within a system. In this case, model-based testing is used to deal with information overload. It looked to me as though he was less concerned about necessarily having a finite state machine and more concerned with the tester having some way of visually mapping the system in a way that made sense.
The secondary reason for using a model, was as a way to approach sensitive devs about holes in their logic. Saying to a dev, “here’s this diagram I made of the system, but I seem to have a gap, can you help me fill this in?” is much less confrontational than approaching them with their spec and telling them they forgot stuff.
Dr. Kaner’s primary reason for using a model intrigued me because it is contrary to Alan Page’s suggestion in HWTSM that models can get too big. Dr. Kaner is using models as a remedy for information overload and he uses a decision tree he made showing reasons to buy or sell stock as an example. It’s not a small picture, but maybe that’s because I’m not used to testing with models or even looking at them on a daily basis. Here’s what Alan has written about the size of models:
Models can grow quite quickly. As mine grow, I always recall a bit of advice I heard years ago: ” ‘Too Small’ is just about the right size for a good model.” By starting with small models, you can fully comprehend a sub-area of a system before figuring out how different systems interact.
-Alan Page, How We Test Software at Microsoft. p162 (inspired by Harry Robinson)
This week’s mission, is to make some models and compare notes about how successful this strategy is for varying levels of model size/complexity. Since all iGoogle gadgets have some type of specification, I picked a few google gadgets:
Small: The Corporate Gibberish Generator
Medium: XKCD
Large: Thinkmap Visual Thesaurus
Since models can also be useful for api’s, if anyone is feeling super-geeky, you can try modelling some api calls from twitter. I just blogged using twitter with curl, so that might help those choosing to do api modeling. Alan writes about how modeling can be useful for testing api’s and it made me very curious. (Off topic, I have to wonder: what happens when you throw some models at a genetic algorithm? Learning? Useful tests? Who knows. I’m saving that one for later.)
I also have what I’ll call a modeling “experiment.” This may or may not work. It may or may not teach you something, but I think it will make your afternoon/evening/morning interesting to say the least. This link is to a painting in the Museum of Modern Art. The painting is The City Rises by Umberto Boccioni. As I read about Dr. Kaner’s approach of using modeling to combat information fatigue, I was immediately reminded of this painting. There is so much going on in this painting and, if you explore, you will find relationships that make it a masterpiece. Can a model pull out and define the power of this painting? Let’s find out.
For graphics software, I suggest Gliffy. It is browser based so no worries about operating system, and for our purposes, sign-up is not required.
You should check out Harry Robinson’s stuff too, he is known for his work in MBT.