Automated Web UI Testing on Steroids Angel Todorov www.infragistics.com www.istabg.org
Automated Web UI Testing on Steroids
Angel Todorov
www.infragistics.com www.istabg.org
www.infragistics.com www.istabg.org
About Infragistics
Worldwide leader in user experience controls
and components targeting jQuery & HTML5,
Windows Forms, Windows Phone, ASP.NET,
Silverlight, WPF, SharePoint, iOS, Android,
Windows UI
Focus on high performance
scenarios for line of business
and data visualizations
Contents
• Background & Motivation
• Good Tests
• Improving JavaScript unit
testing
• Improving scenario testing
• Other Behavior-Driven
Development applications
www.infragistics.com www.istabg.org
Background & Motivation
• Execute all unit tests as part of every CI build
• Execute all integration scenarios as part of every
nightly build
• Be able to quickly automate submitted bugs
• Limit misunderstanding between groups and promote
collaboration – everyone contributes to testing
www.infragistics.com www.istabg.org
Run tests before check-in – only the
feature
Run tests on CI build trigger – single browser
Run all tests on a nightly build trigger
– all browsers
No single solution !
What makes a good test?
• Easy to write
• Very fast to execute – avg. couple of seconds
• Reliable
• Customer-focused
• Valuable (effective)
• Maintainable
• Part of CI
www.infragistics.com www.istabg.org
JavaScript unit testing
• QUnit
• PhantomJS/Awesomium & custom runner script
• Integrates nicely with CI builds
• No interference on the developer machine
• Headless = layout engine + JavaScript engine
www.infragistics.com www.istabg.org
Scenario testing: Common challenges
• Test Beds take a long time to create
• Achieving good reuse of test code is not
straightforward (copy / paste)
• Hard to investigate / debug failing tests
Wait a sec…
Writing tests
should be
fun !
www.infragistics.com www.istabg.org
Scenario testing: Common challenges – Cont’d
• Different documents not always in sync & up to date
(test code, test plans, etc.)
• Gap between stakeholders & engineers
• Tests are hard to understand and validate by non-
engineers (designers, user experience guys)
www.infragistics.com www.istabg.org
Cucumber & BDD
• Behavior driven-development: specialized version of
Test-Driven-Development
• Cucumber: A Ruby tool for running automated
acceptance tests written in a BDD style
• Various implementations for other platforms –
example: SpecFlow (.NET)
• Scenarios grouped in feature files in the Gherkin
format:
Given / When / Then Steps describe a scenario
www.infragistics.com www.istabg.org
Cucumber & BDD – example
www.infragistics.com www.istabg.org
…Which maps to:
www.infragistics.com www.istabg.org
Cucumber & BDD – analyze results (demo)
www.infragistics.com www.istabg.org
IG BDD Test Framework
• Ability to generate test beds (HTML & MVC) based
on Given’s
• Out of the box reuse – every step is independent
• Built-in Visual Studio Intellisense:
www.infragistics.com www.istabg.org
IG BDD Test Framework – Cont’d
• Feature files => Actual test plan
• Easy for PMs and UX to follow the interactions,
expressed in Gherkin
• Tags and templates bring a lot of semantics (Demo)
www.infragistics.com www.istabg.org
IG BDD Test Framework – Cont’d
Test project (component under test)
e.g. Grid Column Moving
IG BDD Test Framework
.feature files
written in Gherkin
(Step Definition
implementations,
templates, tags &
scenario lifecycle
management)
SpecFlow.NET / MSTest / Selenium
WebDriver API
Low-level APIs for
verifications, Assertions,
test-running & reporting
www.infragistics.com www.istabg.org
Other Applications of BDD
• Puppet setup:
1 Master, 1 slave running CentOS on Oracle VirtualBox
@generate-puppet-manifest
Scenario: Ensure Apache is provisioned on nodes
Given I enable Apache in the main manifest
When I update my puppet slave nodes
When Changes take effect
Then Apache should be running on my slave nodes
on port 80
www.infragistics.com www.istabg.org
Resources
• Selenium Web Driver:
http://seleniumhq.org/docs/03_webdriver.html
• Cucumber: http://cukes.info/
• SpecFlow: http://www.specflow.org/specflownew/
• GhostDriver: https://github.com/detro/ghostdriver
• PhantomJS: http://phantomjs.org
• Qunit: http://qunitjs.com/
• Awesomium: http://awesomium.net/
• Puppet: http://puppetlabs.com/
www.infragistics.com www.istabg.org
Thank you!
www.infragistics.com www.istabg.org
Q & A
www.infragistics.com www.istabg.org