Software Testing – Page 5

Is complexity really all about the source code?

At this point, the work I’m doing on my master’s thesis is starting to congeal. I’ve been reading about data visualization and how it can assist quality. There seem to be several levels at work. There is the source code level where developers and, to some extent, qa examine individual pieces of code. This level is addressed by unit tests generally written by a developer or specialized white box tester. Higher up is the system test level which may or may not be automated. Recently, frameworks and tools such as STAX/STAF and Selenium have helped to automate and provide more consistency for some level of system test. At the highest level, quality is less about tests and more about metrics, in particular, lines of code.

In my research, I have found many papers about providing analysis for source code. There are also plenty of papers which address the production of metrics. Software and User Interfaces have developed to the point where the idea of zooming is going to bring these layers together. If you think about Google Maps and the zooming capability that it has, think about how that type of zooming could be applied to the Software Development process. James Whittaker, the architect of Microsoft Team Test certainly has. His vision is that this type of interaction will result in a Heads Up Display for quality.

Assuming that this is where we are going. I can see some challenges in how projects are organized. The main challenge will be mapping from one level to the next. Moving from a system test level to a unit test level will highlight how system tests are mapped to unit tests. This goes back to requirements and design which I know is still an issue at not all but plenty of software shops. Apart from unit to system test mapping, there is also the issue of how LOC is examined and how that maps to source code and maybe even system tests. What if two developers have worked on the same code? How would management assess the LOC count?

I know that, technically, system tests, unit tests and source code are all supposed to be organized based on requirements, but some of us live in the real world!!!! I know that there are places where requirements are maybe not great, but usable. That said, there are plenty of testers who don’t work in shops where we have the benefit of usable requirements or they may be usable in a very loose sense of the word. If a tester is working in a group with bad requirements, they are still responsible for the bugs they didn’t catch.

In this case, complexity is not just about source code anymore, but about how well tests cover the system.

Software Testing Conferences

In my work as a 1 woman test team, figuring out best practices in testing has taken some work. Books help a lot. Advice from friends at my company who also test has helped out too, but I have a need to connect with other testers most of whom probably work on teams with much more structured testing guidelines. As if that weren’t enough, I have this master’s thesis I’m putting together that will deal with testing, and I’m looking for places to submit as a presentation or a poster.

Attending the Google Test Automation Conference last year opened my eyes to the fact that it is important to keep up with the best practices of other companies. Talking with other professionals in testing helped me gauge what I’ve been doing right at my job and where I can stand to improve. I’ve been looking around for other conferences to attend, and have found a few of them. Reading the blog posts of people who have attended or presented at these conferences is also interesting. In addition to including the links for some testing conferences, I’m also including links to some blog posts about those conferences (when I could find them).

Obviously my list is not exhausitive and will probably get dated over time, but I don’t mind adding to it. Likewise, if you have blogged about a testing conference you enjoyed or have comments about one of the conferences I listed, feel free to leave a comment.

Star East
Star West: JW on Test
PNSQC: Testy Redhead
CAST: Adam Goucher
GTAC: You can read my other posts or, for something completely different, check out The Automated Tester
Swiss Testing Day

5 Lessons from the 2008 Google Test Automation Conference

I’m back from Seattle and ready to write some tests. Here is a short list of what I learned (or what was reinforced) at the 2008 Google Test Automation Conference.

1. If your group isn’t building continuously, they are behind the times.
In hearing people talk about how tests are run, it was generally taken for granted that builds are now being continously integrated using a tool such as Hudson or Bamboo. I found this incredibly large spreadsheet comparing CI Tools.

2. Everyone loves a visual representation
Throughout the conference, besides hearing people talk about Selenium, people were absolutely crazy about a visual representation of the Windows Vista source code presented by James Whitaker in his keynote. This link is for a treemap created by The New York Times which shows the change in the financial sector’s market capitalization over the past year. I’ve circulated it at work as an example of how powerful good visualization can be.

3. Unit testing is ubiquitous.
I talked with people about the types of testing that happens at their work, and the consensus is that much more emphasis is being placed on creating automated tests and unit tests. What surprised me is how many shops actually do use unit testing. This is not some crazy new-fangled thing anymore, this is what teams are doing in conjunction with continuous builds.

4. Live blogging isn’t easy at all.
Before I sat down for the first presentation of this conference I thought that live blogging would be a piece of cake. It turned out to be a challenge because I had to balance providing information about what the speakers were talking about with writing about my opinion of their presentation. Take into account that I’m talking with the people next to me about what the speaker is saying and at times, trying to understand the presentation and suddenly, writing about a presentation becomes quite challenging. Blogging all of them got to be too much, but luckily, I wasn’t the only one blogging the conference. David Burns of The Automated Tester was also blogging. Between the two of us, I think we got most, if not all, of the presentations blogged. I’m happy to link to any other GTAC blogs.

5. There is always someone smarter.
It’s always tough to admit that you are not the smartest person in the room (or maybe even close to being the smartest) but in situations like this one, I really don’t mind. This conference put me in such close proximity to so many brilliant testers that I couldn’t help but be inspired. Every conversation that I had was really amazing and helped me grow as a tester. I will always take any opportunity I’m offered to hang out in such great company.

It was great to meet so many smart and amazing people at GTAC. For anyone who has the opportunity to attend this conference, I highly recommend it.