Selenium – Marlena's Blog

Webinar with me: Using Firefox Add-ons for testing

As a tester of web applications, I’ve become familiar with how web pages are constructed and rendered through the browser. Firefox Add-ons provide a great set of tools for doing this and have been a mainstay of my testers toolbox for quite a while. One of my jobs as a Mozilla employee was to show others how to use some of these addons.

Below is a chat I had a while back with a contributor (name anonymized for privacy) about Firebug and Firepath, 2 addons that I use all the time for getting information about the elements, called locators, on a webpage. We use these all the time in Selenium testing and it is a taste of what I’ll be talking about in a free webinar I’ll be giving for the Software Association of Oregon on Tuesday, May 15 from 12:00pm to 1:00pm. Registration is free :o)

In addition to Firebug and Firepath, I’ll be talking about an add-on developed at Mozilla, Mem Chaser and some browser functionality that started as an add-on but has made its way into the Firefox browser, Tilt.

marlenac Have you used Firebug before?
olivier No i have heard of it
olivier Its a firefox extension
marlenac You’ll want to install 2 addons: Firebug and Firepath into firefox
marlenac I use both of these all the time to help with the locators. I’m also looking up a good link on CSS for you.
marlenac This blog post explains some of it and has a bunch of other links for working with Selenium and CSS: http://blog.reallysimplethoughts.com/2010/10/12/a-quick-introduction-to-css-locators-in-selenium/
olivier How does firebug help with locators?
olivier Does it generate the expression?
marlenac It allows you to inspect web pages so you can see what the locators are.
marlenac If you’ve got it installed, choose an element on a web-page, right click and choose “inspect element”
olivier Kk
marlenac As an example, I’m on the homepage for addons: https://addons-dev.allizom.org/en-US/firefox/
marlenac If I hover the mouse over the big “ADD-ONS”, right click, and choose inspect element
marlenac It will split the window of the browser and show me the html of the page at the bottom.
olivier Kk
marlenac the line starting with “<a title=”Return to the Firefox…” is highlighted.
marlenac Just above that is a css class “site-title”
olivier Right
marlenac If I want to select the link at “ADD-ONS”, the selector will be “.site-title > a”
marlenac We can check that this is correct with Firepath.
marlenac In the Firebug pane, there are several tabs at the top. “HTML”, “CSS” and further over is “Firepath”
olivier Kk
marlenac If you paste in “.site-title > a” without the quotes, it will highlight the element for you.
marlenac It’s pretty great!
olivier Thx so much u just made my life easier
marlenac I know!!!!!! We’d all be suffering without Firebug and Firepath!!
marlenac A couple of tips.
marlenac Use .blah to specify a class name
marlenac Use #blah to specify an id.
marlenac Use the “>” to get to a child element.
olivier What if there are many child element?
marlenac You can keep going with it.
marlenac or if there is a list, you can use “nth” to specify the 1st, 2nd, 3rd.
marlenac This post has a great explanation of that: http://saucelabs.com/blog/index.php/2010/01/selenium-totw-css-selectors-in-selenium-demystified/
marlenac I’m just finding an example in our code as well.
olivier I think i get it
marlenac Cool!
olivier I will ask i get stuck
olivier Thx again
marlenac You’re welcome :)

Isolating a performance bug

Image via Wikipedia

Last week, I isolated a performance problem that started with the failure of an automated test. I’m blogging it because the bug has an interesting story which highlights some of the weirdness I’ve typically found when isolating performance bugs.

A teammate of mine, Alin Trif, noticed one of our automated checks failing. It writes a review for an add-on. He replicated the failing test manually and wrote up the bug. Since a few others in our group couldn’t replicate the problem, it was subsequently closed. In true tester fashion, Alin mentioned it to me anyway. I thought “what could it hurt to try reproducing it.” So, after opening our staging site, addons-dev.allizom.org, I went through the steps…and reproduced the problem.

Note that I did not just re-open the bug. Personally, I think re-opening bugs is a great way to alienate people who are ready to move on from a particular issue and make testing more irrelevant on your team. This doesn’t mean dropping the ball on a problem. It means that it’s time to uncover more or better information which would likely result in a different bug being opened anyway. If you read to the bottom, you’ll find that this was, indeed, something else entirely.

Two of my favorite web testing tools are Firebug and Skitch. Although I generally use Firebug for inspecting web pages and finding the locators I need for writing Selenium-Webdriver checks, it will also show metrics for requests and responses. To access this, open Firebug and click on “net”

In this case, while the request was off in never-never land, I opened Firebug, switched to the net tab and took a snapshot. When the request eventually finished, I took another snapshot. This time, the culprit is pretty clear.

Although I was testing the addons staging site, the bug is actually for Browser-ID, Mozilla’s new solution for single sign-on.

With the screenshot in hand I logged this bug against browser-id.

Interesting take aways from this bug:

Unless you are actively looking for performance bugs, they are extremely easy to dismiss. In this case, it looked like it wasn’t reproducible. These are also easy to miss unless you are hyper-sensitive to slowness. (Every tester ought to know their physical triggers of impatience for slowness. Do you tap your fingers and/or roll your eyes and/or cuss at the screen?)
If you’re frustrated about one of your bugs getting closed or marked invalid, it’s time to talk to someone about it vs. only leaving a comment.
Using Firebug and your screenshot tool of choice makes it fairly painless to document these bugs. I’ve caught bugs like this before and ended up installing a third party tool for looking at the time taken for requests and responses. This is now much easier if you download the Firebug add-on because the tool is right in your browser.

The next time, something “feels slow” or “slower” to you, give this a try. You might find something you weren’t expecting.

Hat tip to Alin Trif for finding the bug and asking about it even after it was initially closed. That’s good testing.

Anatomy of Switching Selenium from RC to Webdriver

If you’ve ever wanted to see what it looks like to switch tests from the Selenium-RC api to the Selenium-Webdriver api, this github pull request will give you a very good idea of the changes that have to be made.

There are a few reasons why these tests are a particularly good example of what happens.

We use page objects. If there’s ever been proof that page objects help ease the pain of refactoring, this would be it. We made significant changes to our page objects, but the changes made to tests are relatively minor.
We have over 100 tests covering many of the areas in addons.mozilla.org. It’s all open source, so you can see the tests, the website they are testing AND the code of the website they are testing.
Our tests are written in python which is a pretty easy language to read and get to know.

A few highlights and bogans:

When you’ve got css locators for list items and you are using “nth” you have to bump all of your numbers up by one because rc starts at 0 but webdriver starts at 1. (Insert expletives here.)
When assigning locators, the format changes and makes the flavor of your locator much more obvious than it was in Selenium-RC. It’s not that the information is COMPLETELY MISSING in Selenium-RC, it just isn’t in my favorite font-type OF ALL TIME. (<3 <3 <3)
Instead of using get_css_count, you have to get the list of items instead and test the length.
Native events don’t work as well on mac, so we had to set up windows vms in our selenium grid. This was not a trivial task and deserves it’s own post which I will write if I get over the vm trauma of the past couple of weeks.
You can see Webdriver navigating across the different elements of a page which is really freaking cool!
You can write tests for hover elements. We’ve got lots of this in addons at present and Selenium-RC was preventing us from tests that incorporated these. (Open the diff view and search for “hover!”
Instead of working mainly with locators, Selenium-Webdriver uses Web elements. If you don’t know, these are dom elements and have a bit more meat to them than locators. Whereas a locator is a string describing a location in a web page, a web element encompasses tags and everything within them.
Currently, we have to run the tests sequentially in Firefox even though they can be run concurrently in other browsers. This is, apparently, not a new problem. Fortunately, after talking with the Firefox devs, they are working hard on a fix for us which should benefit anyone wanting to run webdriver tests in Firefox.
There are lots of lovely methods which can be chained together to mimic user actions. I know it’s a machine and not a person with a brain running these tests, but personally, I can’t wait to write a mobile test that uses the “double tap” method. Cue Zombies!!!!

There’s much more to this change, but I’m dog tired and it’s Friday. Knock yourselves out with that pull request!

Special thanks to the team that wrote all of that code, Teodosia Pop, Florin Strugariu, Zac Campbell with special guest-star appearances made by Dave Hunt.

Next Week: moz-grid-config workshop for setting up a local selenium grid

Next Friday will be my 2nd AMO Automation testday at Mozilla and the first one I’ve helped to organize. AMO stands for addons.mozilla.org and it’s the web-site for which I’ve been writing and reviewing automated checks (sometimes I call them tests, but I like referring to them as checks).

As part of the last test day, I ran a github workshop to help people figure out how to set up and work with github. Here’s a link to that blogpost.

For the upcoming testday, I’ll be running a workshop to help people who would like to run our addons selenium checks with our grid configuration. Although it’s easier, when starting out to write and execute tests using a standalone selenium jar, it’s good to understand how to run tests with selenium grid as well. All of the tests for AMO are run using grid, so this is the setup I use when I’m doing code review or writing checks.
This is the link to the github repo containing our grid configuration. I’ll be posting again next week with a few more instructions on how to modify the repo for running checks locally.

The testday starts at UTC/GMT 15:00:00 and continues throughout most of the day until 5:00pm Pacific time. (We’re global, baby!)

A few local times for the moz-grid-config workshop:
UTC/GMT: 17:00:00
Pacific: 10:00am
London: 6:00pm
Bucharest: 8:00pm
Bangalore: 10:30pm

Here is the link to info about our testday on the Mozilla QA Blog.

By the way…if you’re in the UK and close to London, you should checkout the London Selenium User’s Meetup which is happening next week. (I hear these are frakking awesome) :)