GTAC Liveblog: Using Cloud Computing to Automate Full-Scale System Tests by Mark Elian-Begin and Charles Loomis

Mark is presenting a project called ETICS which is a European project funded by CERN. CERN has most recently been in the news for their Large Hedron Collider.

Automating System Tests
They use machines to investigate and have problems getting the machines allocated which means that by the time he can run the test, the system has been deployed and the problem exists in production. This was partly becase the release dynamics are spread across stakeholders which creates complexity for automation. In order to get around this he started using Amazon’s Web Services to run some of the tests.

ETICS’ goal was to show vision of automated testing. Mark wanted to “capture people’s worries in a common automated platform.” To do this his team built a testing framework on top of a backend grid system called Condor. This is where problems started to appear. Challenges included limited cm, limited sys admin, limited resource to implment automated testing. People were using this to do tests but their tests kept crashing because of inconsistencies in the complex grid environment. In order to simplify management of the system the decision was made to move the backend from Condor to Amazon’s Web Services.

Lessons Learned
Tools MUST be simple or people won’t use them. ROI must be obvious.

The use case Mark presented was for the Diligent project which used ETICS to automate testing. Diligent goes through large set of documents for queries. Used ETICS for deployment tests.
Full process automation is not required for ROI
If you don’t have access to machine, troubleshooting becomes difficult. I can see why they chose to use Web Services in this instance. Amazon allows an amazing amount of control for the boxes being used.

Cloud Computing
build -> package -> install-> deploy-> test
cloud helps with building lots of test beds.

He prefers using RESTful Web Services because of their ease of use

Mark touched on the hardware virtualization runtime environment called Xen for this and says Microsoft is about to announce their cloud solution. I remember reading about the different ways to use AWS and they really seem to push using it through REST

The next step, he says is kvm which goes from para virtualization (xen) to full virtualization(kvm).

Public cloud vs. Private cloud – they are seeing a need for clouds that work with senstive services.

Q&A
Q: are there projects being worked on with this tech?
A: “I wish I could be more open about this” that means YES.
Q: I looked into cost of 24×7 AWS and iwas cost prohibitive. If you use AWS for 10 minutes they charge for an hour how do you keep costs down
A: Its like mobile phone contracts It’s a problem. personally he’d like to see this go away.
A..from audience
had dinner with a friend doing performance testing using aws to attack his system. He uses it for an hour and then turns it off
Q: performance testing?
A: wrote study between grid and cloud it’s on the CERN web-site. Amazon uses compute unit to guide user in their choices. We have little information on how Amazon puts their back end together.

OMG the crowd RAN OUT of questions.