Copyright 2009, Build Lackey Labs (see last slide for licensing terms) http:// buildlackey.com - Selenium - Browser-Based Automated Testing of Web Apps Under Continuous Integration Presented By Chris Bedford Founder & Lackey at Large Build Lackey Labs Company Overview Founded 2008 Services • Release engineering & Build/Test Automation for Java Environments • Spring/Hibernate & Groovy/Grails App Development • Training: Spring/Groovy/Grails & Build/Test Automation Technologies Employees • Chris Bedford • Mike Jackson
36
Embed
Copyright 2009, Build Lackey Labs (see last slide for licensing terms) - Selenium - Browser-Based Automated Testing of Web Apps.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
- Selenium -Browser-Based Automated Testing of Web Apps Under
Continuous Integration Presented By
Chris Bedford Founder & Lackey at Large Build Lackey Labs
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Agenda
Different Ways To Test a Grails App Unit, Integration, Functional Selenium Demo of IDE Architecture Alternatives (Canoo WebTest)
Automating Testing Using Selenium Maven Demo Cargo Viewing Test Results Lab 1 – maven project
Ant
Continuous Integration With Hudson
How to write functional tests with
Selenium
KEY
TAKE-AWAYS
How to set upSelenium Testing On Grails Apps In Continuous
Integration
Building With AntBuilding
with maven
Continuous Integration
With Hudson
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Unit
Unit Tests are created and run developer and run using framework like Junit or TestNG• test class or package functionality in isolation• heavy use of mocks and stubs • tests should execute quickly and be run often to catch problems early on
Integration
• If you are dependent on someone else’s stuff, the test includes your stuff and theirs• Verifies that two or more components work together,
• usually directly, and not with mocks• To simplify test set-up components typically execute same JVM process
Functional Testing
• Run application in container • Exercise functionality via a client side agent that executes Javascript.
•Agent could be either:• your actual target browser (Firefox, IE, etc.) [Selenium's approach]• a Javascript engine embedded into test framework [Canoo Webtest's approach]
Different Ways To Test a Java Web App
Real http requests
Separate client
Mock http requests
Client requests from same process
Package or class level scope
increasingly coarse grained components under test
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Manual Steps Involved In Running Selenium
Before running Selenium Functional Tests we need to
• Compile classes
• Run unit and integration tests • bail on functional tests if we catch problems with lighter weight tests
• Package .war file
• Install and start the container in which we want to run the .war
• Deploy the .war to the container • unless your running Jetty, which runs embedded in your app
• Launch target browser
• Launch Selenium on desired test suite
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Launching Selenium
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Recording New Selenium Tests
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Exporting test commands to 3GL (Java, etc.)
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Individual tests referenced by the suite are recorded using their paths relative to the suite. For simplicity put your suite and all tests in the same directory (to start)
Saving New or Modified Selenium Tests
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Selenium Components
• Selenium IDE
• Selenese Commands• Enable test author to
• simulate navigation, clicks• Make assertions about expected responses
• Selenium RC
• Client side library that enables you to program more sophistication into your tests than the IDE allows (conditions, looping, error handling)
• The Selenium Server which launches and kills browsers, interprets and runs the Selenese commands passed from the test program, and acts as an HTTP proxy,
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
(client side java script / Ajax portion of application underTest – originates from here )
Application under test(server side)
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
A Look Same Origin Policy (Which Selenium’s Architecture Circumvents)
Same origin policy:
Forbids JavaScript code running on a web page from interacting with resources which originate from any web site other than the one which served up that web page
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Selenium RC Server Acting As Proxy To Avoid Same Origin Policy Restrictions
2) Selenium-RC server launches a browser (or reuses an old one) with URL that injects Selenium-Core’s javascript into browser-loaded web page.
3) client-driver passes a Selenese command to the server e.g.: open command
4) Server interprets the command and then triggers the corresponding javascript execution to execute that command within the browser (say open page in app under test)s
5) Request to open the page is routed through proxy server
6)
7)
Proxy forwards request to app server
App server returnsresponse
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Functional Test Alternatives: Canoo vs Selenium
Canoo Web Test
built on HtmlUnit
pros:•excellent test reporting allows you to pin point errors in test very easily.•faster to run (no browser spin up overhead)•better support for non HTML content (like spread sheets)
cons:•Weaker IDE (for test recording and playback)•develop tests in Ant or Gant only
Selenium
pros:•develop tests in HTML markup or 3 GL's like Java, Ruby, etc.•run test in actual browser •vs.embedded Javascript engine used by NO popular browser platform
cons:•slower to start.•see 'pros' listed Canoo
RECOMMENDATION: I’d go with Canoo for testing a REST-ful API
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Canoo Web Test Reports
Canoo's reports show overall test results and let you drill down into any test
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Canoo Web Test Reports (cont.)
Click to review a copy of the response HTML page corresponding to the first test step that failed
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Gluing together the steps in your build process
Before running Functional Tests need to • Compile• Run unit and integration tests
• bail on functional tests if we catch problems with lighter weight tests• Package .war file• Install the container in which we want to run the .war (optional)• Deploy the .war to the container (optional – can just do ‘grails run-app’)• Launch your target browser• Launch Selenium on your desired test suite
To automate this process you can use• ant• maven• Groovy/Gant scripts• Gradle
Our example uses Maven
(demo) (tour of pom.xml files that wire together our build steps)
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
deploy
Compile
Unit Test
Integration Test
Package .war file
DownloadAnd InstallTomcat
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Structure of our demo project
mvn install
Maven Repository
Lives in $HOME/.m2/repostitory or /Docuemts and Settings/<user>/.m2/repository
declares the artifact it produces to be org.example:demo:1.1
declares aa dependency onorg.example:demo:1.1
org.example:demo:1.1
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Demo
All Tests Pass
Some Tests Fail
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Maven Basics
•Convention over configuration• Standardizes where things live and what they are named • Lets you know where to look for the things you need…
• and what to do when you find them
•Project Object Model (pom.xml) specifies the complete ‘recipe for your build’• what artifact type are your producing ? (the name, the version…)• what are the dependencies (things you need) to produce that artifact ?• What plug-ins activate at what phases in the build life cycle ?
•Shared Repository for storing artifacts that result from a build• Convention for naming artifacts
•Build Life Cycle• each project is typically responsible for producing a distinct artifact (a.k.a. packaging) type
.jar, .war, .ear, etc. • each packaging type has an associated life cycle (ordered set of build phases)
•Nested build project (module) structure• overall driver for your build project lives at top level of a directory hierarchy• sub-modules underneath the parent can be either
• individual components of your product …or…. • key phases in your build
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Maven Nested Module Structure, Dependencies and Shared Repo
mvn install
Maven Repository
Lives in $HOME/.m2/repostitory or /Docuemts and Settings/<user>/.m2/repository
declares the artifact it produces to be org.example:demo:1.1
declares aa dependency onorg.example:demo:1.1
org.example:demo:1.1
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Maven pom.xml – Nested Module Structure
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Maven pom.xml – Dependency Relationships
Maven Repository
org.example:demo:1.1
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Hooking Maven Plug-ins Maven Into the Build Life Cycle
Installer installer = new URL("http://somewhere/tomcat-6.0.zip"));installer.iZipURLInstaller(new nstall();
LocalConfiguration configuration = new DefaultConfigurationFactory().createConfiguration("tomcat6x")...)
container = new DefaultContainerFactory() .createContainer("tomcat6x"....);container.setHome(installer.getHome());
WAR deployable = new WAR("foo.war); deployable.setContext("ROOT");configuration.addDeployable(deployable);
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Launching Selenium Via Ant
Walk through of ant script that launches Selenium server in separate JVM,
Waits for server ready, then launches Selenium tests, then waits for shut down
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Continuous Integration
Dedicated box runs regular full builds (including tests) of your software
Build triggers whenever any developer checks into SCM
Team is notified of any failures
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Continuous Integration Benefits
Replaces big (and long) integration cycles with small frequent ones.
Same benefits as continous compilation in Eclipse
Immediate feedback when an error is introduced. Fewer deltas between when it worked and when it broke => easier resolution of failures
Lower ripple through impact when colleagues checks in broken code CI server build fails & team is notified Other developers know it is not safe to pull from source Mail sent out when build goes back to normal
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Workflow After Adopting Continous Integration
CI Server:
0- Receive notification of change in SCM repo1- Check out latest sources 2- Build from scratch3 Build passes? yes: publish, e.g., deploy .war to QA server no: send out email, flash red lights ...
Developer:
0- Check CI server for current build status broken ? don't update ! ... otherwise..1- check out from SCM2- code new feature 3- run automated build on your box 4- Tests pass ? no? go back to 2 !5- Commit changes
SCM Repo
Build ok?
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Continuous Integration With Hudson
Hudson orchestrates the execution of repeatable jobs periodic builds (e.g., nightly) builds that are triggered by some event (like checkin to source control)
Keeps History of Past Builds (Trend Reports)
Notifies When Builds Fail
Written in Java and runs from a .jar (no install required)
Integrates with wide range of SCM systems (SVN, P4, etc.)
Supports ant, maven, shell script based builds
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Thank You !
Copyright 2009, Build Lackey Labs (see last slide for licensing terms)
http://buildlackey.com
Content Licensing Terms for This Work
You may present, distribute, copy, or incorporate into your own work, any and all portions of this presentation as long as such copies, or derivative works are made available without charge.
If you would like to redistribute this work on any type of fee-for-use or subscription basis, or if you wish incorporate any or all portions of this work into content which you charge for, please contact info <at> buildlackey.com to discuss licensing terms.