A few more Vampire Testing Lessons

Yellow Porsche Carrera GT with Hardtop (US Ver...
Image via Wikipedia

I just love it when bloggers mix pop culture with testing. Recently, the Testy Redhead posted a few lessons about testing she adapted from her reading of the Twilight books. I love these books in all their somewhat-poorly-written-but-ultimately-addicting glory so I couldn’t help but put together a few lessons of my own. The last lesson is a spoiler, but I’m guessing that there are not hordes of Twi-hards reading this blog.

You can build a working car from a bunch of disparate parts.
Jacob totally rebuilds a Volkswagon Rabbit using parts that he collects over the length of a couple of the books. This reminds me of some of the great open source tools that are now available for testing such as Bugzilla and Selenium. It also reminds of the Automated System Test Framework I’m building from scratch at my job. I started out with a bunch of short scripts that I wrote, but, with some perserverance I’m close to having a system in place that will greatly assist me in testing.

Sometimes the yellow Porsche really is what you need.
I’ve noticed a real disdain for expensive tools among testers, but sometimes they are the right answer the same way the yellow Porsche was the right car for Bella and Alice in New Moon. When I started my testing job, I was not a tester and I did not know what I was doing. My tester friends in another group had shown me HP Quality Center, and I realized that I desperately needed this assistance with test case management. It helped me transition off of spreadsheets and gave me a structure for repeatable testing.

Don’t read the last one if you don’t want to read the spoiler.

Testers ARE the Shield
In the last book, Bella protects everyone using her special super power. Testers also have a super power, and that power is the right to say, “This product really stinks and is not ready to be released with our team’s name on it.” This is not the most obvious power, but it can protect a team or even a company from releasing a product and regretting it. I’ve had to say this before to a most “busy and important” developer who let me know how busy and important he was, but I knew that I was protecting consumers by saying it.

Reblog this post [with Zemanta]

Is complexity really all about the source code?

At this point, the work I’m doing on my master’s thesis is starting to congeal. I’ve been reading about data visualization and how it can assist quality. There seem to be several levels at work. There is the source code level where developers and, to some extent, qa examine individual pieces of code. This level is addressed by unit tests generally written by a developer or specialized white box tester. Higher up is the system test level which may or may not be automated. Recently, frameworks and tools such as STAX/STAF and Selenium have helped to automate and provide more consistency for some level of system test. At the highest level, quality is less about tests and more about metrics, in particular, lines of code.

In my research, I have found many papers about providing analysis for source code. There are also plenty of papers which address the production of metrics. Software and User Interfaces have developed to the point where the idea of zooming is going to bring these layers together. If you think about Google Maps and the zooming capability that it has, think about how that type of zooming could be applied to the Software Development process. James Whittaker, the architect of Microsoft Team Test certainly has. His vision is that this type of interaction will result in a Heads Up Display for quality.

Assuming that this is where we are going. I can see some challenges in how projects are organized. The main challenge will be mapping from one level to the next. Moving from a system test level to a unit test level will highlight how system tests are mapped to unit tests. This goes back to requirements and design which I know is still an issue at not all but plenty of software shops. Apart from unit to system test mapping, there is also the issue of how LOC is examined and how that maps to source code and maybe even system tests. What if two developers have worked on the same code? How would management assess the LOC count?

I know that, technically, system tests, unit tests and source code are all supposed to be organized based on requirements, but some of us live in the real world!!!! I know that there are places where requirements are maybe not great, but usable. That said, there are plenty of testers who don’t work in shops where we have the benefit of usable requirements or they may be usable in a very loose sense of the word. If a tester is working in a group with bad requirements, they are still responsible for the bugs they didn’t catch.

In this case, complexity is not just about source code anymore, but about how well tests cover the system.

Software Testing Conferences

In my work as a 1 woman test team, figuring out best practices in testing has taken some work. Books help a lot. Advice from friends at my company who also test has helped out too, but I have a need to connect with other testers most of whom probably work on teams with much more structured testing guidelines. As if that weren’t enough, I have this master’s thesis I’m putting together that will deal with testing, and I’m looking for places to submit as a presentation or a poster.

Attending the Google Test Automation Conference last year opened my eyes to the fact that it is important to keep up with the best practices of other companies. Talking with other professionals in testing helped me gauge what I’ve been doing right at my job and where I can stand to improve. I’ve been looking around for other conferences to attend, and have found a few of them. Reading the blog posts of people who have attended or presented at these conferences is also interesting. In addition to including the links for some testing conferences, I’m also including links to some blog posts about those conferences (when I could find them).

Obviously my list is not exhausitive and will probably get dated over time, but I don’t mind adding to it. Likewise, if you have blogged about a testing conference you enjoyed or have comments about one of the conferences I listed, feel free to leave a comment.

Star East
Star West: JW on Test
PNSQC: Testy Redhead
CAST: Adam Goucher
GTAC: You can read my other posts or, for something completely different, check out The Automated Tester
Swiss Testing Day

Project Management Class and Scrum

This past Fall, the only class I took was Project Management. The reason I signed up for this class was because I thought it would be a nice break from the classes I had been taking like Distributed Systems and Artificial Intelligence. Ha Ha Ha. Now I know why the PM’s at work have such long hours and pained looks on their faces (or maybe it’s because we use waterfall).

In order to give students in the class a full-on project management experience, our instructor broke us into groups, gave us some requirements from which we had to produce a working application. I ended up being elected the PM for my group. Since there has been a serious lack of coverage for Agile Software Development in my classes, I organized us to use Scrum because of an interaction I’d had at the Google Test Automation Conference (GTAC).

When I attended GTAC last Fall, I had a great conversation with a man named Pete from F5 Networks. We were sitting at a lunch table where we were supposed to be discussing Agile development. He told me that he had used Scrum and been pretty happy with it on his team. I didn’t get the chance to ask him how he had implemented Scrum, but he did tell me an interesting fact he had read. He said that he had read somewhere that of all teams implementing XP or Agile software development, the teams that used Scrum were more like to stick with and have success with that process. This stuck with me when I returned home and had to quickly organize our group.

Since I didn’t know anything about Scrum, I looked for a reference on O’Reilly Safari and found The Enterprise and Scrum
by Ken Schwaber. Appendix A, “Scrum 1-2-3” was most helpful. Not only did it have an overview of the whole process, but also included examples of the documentation typically used for a Scrum project.

 

Scrum documentation for a project manager consists of a Product Backlog and a Burndown Chart. The product backlog is a high-level listing of activities that must be completed, the number of hours the activity is expected to take, and a listing of actual hours spent as the weeks progress. Activities are grouped on the chart according to Sprints. (The sprint is the basic unit of time around which activity is organized, 2-4 weeks). The burndown chart is a more detailed listing of who has taken responsibility for which activity and how they are progressing. This chart, produces a weekly graph of how the team is progressing.

Based on our academic calendar, I broke up our project into 5 1-week sprints. It took, no kidding, 8 hours just to break down our tiny set of requirements into the 2 charts. Following the scrum methodology of teams being self-organizing, once I had the activities listed, I went back to my team and asked team members to volunteer for assignments.

As the weeks progressed, we worked hard on our project. We filled out the chart every week, activities were finished and our application was completed. No one stayed up all night and we delivered exactly what we said we would. Oh yeah, and I made an A! So, what worked and what didn’t?

Stuff that didn’t work so well:

Team members who don’t pull their weight mess up all of the assignments. We had one team member who volunteered in front of the whole team to take half of the programming assignments. Afterward, he decided not to do any work. Since the burndown chart was already filled in with his name on several assignments, it was difficult to figure out how to deal with the hours he was supposed to work but didn’t. We had to reshuffle several activities among team members which threw off our initial chart. We handled this by re-organizing the chart.

Activities that slipped became difficult to track. Since each activity is supposed to be listed in one sprint category, I wasn’t sure how to track that activity if it wasn’t finished by the end of the assigned sprint. We ended up adding a column for the actual total hours required since our chart initially only had estimated total hours required.

One of the basics of Scrum is having a 15 minute daily “stand up” meeting where everyone answers the questions, “what did you do yesterday,” “what are you doing today,” “what is blocking you.” Those daily 15 minute meetings did not happen for my group because our schedules did not match up at all. That said, I think these meetings are crucial to the success of a scrum project. People need to know what’s going on and who is blocking who. If we had been having the daily meetings, we would have known that our slack programmer was slacking much earlier. To me, this also means that Scrum works better if teams are co-located. My team was not co-located, which put a serious damper on having any meetings. This was also partly due to the fact that not all of us had equal access to computers at home.

Stuff that really worked:

We were able to re-organize every week depending on how well we’d met the previous week’s goals. This is really the greatest part of Scrum organization for me. When the one guy decided he didn’t want to program (to be fair, it was his last semester), we were able to completely re-organize for the next week’s work. We were also able to adjust the workload depending on who had more or less time to do work.

The charts made bottlenecks very clear. For about a week, I was the scrum master and the lead programmer. Everyone on the team, including me, knew that we couldn’t finish the semester this way, and our instructor noticed it from our chart and the number of assignments with my name attached. Because there is so much detail to be tracked for the Product Backlog and the Burndown chart, there is no way that one person can shoulder updating the charts and producing significant amounts of code. As I said, the whole team knew that we’d have to make changes, and we did. I pretty much handed scrum-master activities to someone else when we were half-way through the project.

Testing happened throughout the project which meant that no one had to stay up all night at the end. For this reason, I think that Scrum or any iterative process is highly effective. It was still challenging to keep the test team busy during the early and middle stages of the project, but we all felt better knowing that what we had produced was working correctly.

My conclusion about Scrum:

Project tracking, in general, is much more complicated than I had anticipated. Producing the artifacts for Scrum took some time and they were not always easy to maintain. Even so, I can’t speak for the whole team, but I know that I had a much better grasp of what was happening because of our documentation. I’m sure documentation is a challenge on any project, and the fact that we were able to keep up with ours is extra points for scrum. I think that iterative software development is much more effective and gives teams a much better chance to course-correct if something goes awry. The self-organizing aspect of choosing assignments also appealed to me. I am a tester, but I could see helping out with documentation or maybe switching and doing some coding on certain assignments. This aspect of Scrum seems likely to keep team members from burning out project-to-project while also providing opportunities to learn something new. I liked using scrum. Hopefully, I’ll have opportunities in the future to work on a team using Scrum.

Reblog this post [with Zemanta]

5 Lessons from the 2008 Google Test Automation Conference

I’m back from Seattle and ready to write some tests. Here is a short list of what I learned (or what was reinforced) at the 2008 Google Test Automation Conference.

1. If your group isn’t building continuously, they are behind the times.
In hearing people talk about how tests are run, it was generally taken for granted that builds are now being continously integrated using a tool such as Hudson or Bamboo. I found this incredibly large spreadsheet comparing CI Tools.

2. Everyone loves a visual representation
Throughout the conference, besides hearing people talk about Selenium, people were absolutely crazy about a visual representation of the Windows Vista source code presented by James Whitaker in his keynote. This link is for a treemap created by The New York Times which shows the change in the financial sector’s market capitalization over the past year. I’ve circulated it at work as an example of how powerful good visualization can be.

3. Unit testing is ubiquitous.
I talked with people about the types of testing that happens at their work, and the consensus is that much more emphasis is being placed on creating automated tests and unit tests. What surprised me is how many shops actually do use unit testing. This is not some crazy new-fangled thing anymore, this is what teams are doing in conjunction with continuous builds.

4. Live blogging isn’t easy at all.
Before I sat down for the first presentation of this conference I thought that live blogging would be a piece of cake. It turned out to be a challenge because I had to balance providing information about what the speakers were talking about with writing about my opinion of their presentation. Take into account that I’m talking with the people next to me about what the speaker is saying and at times, trying to understand the presentation and suddenly, writing about a presentation becomes quite challenging. Blogging all of them got to be too much, but luckily, I wasn’t the only one blogging the conference. David Burns of The Automated Tester was also blogging. Between the two of us, I think we got most, if not all, of the presentations blogged. I’m happy to link to any other GTAC blogs.

5. There is always someone smarter.
It’s always tough to admit that you are not the smartest person in the room (or maybe even close to being the smartest) but in situations like this one, I really don’t mind. This conference put me in such close proximity to so many brilliant testers that I couldn’t help but be inspired. Every conversation that I had was really amazing and helped me grow as a tester. I will always take any opportunity I’m offered to hang out in such great company.

It was great to meet so many smart and amazing people at GTAC. For anyone who has the opportunity to attend this conference, I highly recommend it.

Reblog this post [with Zemanta]