Web Test Automation Framework with Open Source Tools powered by Google WebDriver Kapil Bhalla & Nikhil Bhandari Intuit Inc. (Bangalore)
May 17, 2015
Web Test Automation Framework with Open Source Tools powered by Google WebDriver
Kapil Bhalla & Nikhil Bhandari
Intuit Inc. (Bangalore)
Understand Present Scenario
Look at Pain Points
Framework
Q&A Discuss
What we will be doing?
Agenda Understand Present Scenario
ü Present Scenario
ü Product
ü Development
ü Customer
ü Agreement
ü What can we Do ?
Present Scenario - Product
Decade Old Product
Interleaved with multiple technologies
Focus on new development technologies and framework
Increasing team size with newer people
Flex
Java Script
HTML
DOJO
AJAX
SOAP
Web Services
REST
Present Scenario – Development
Agile Development
Shorter Release Cycles
Long Testing Cycles…growing continuously with every release.
No Automation Yet !!
Need for Speed
Present Scenario – Customer Expectation
Should work on my current browser !
Should work on Operating System I use!
No Compromise on Quality !
Reliability is a must for us!
Even higher expectation from Customer Support
Google Chrome
Internet Explorer 7
Internet Explorer 8
Internet Explorer 6
FireFox 3
FireFox 3.7
Windows XP SP2Windows XP SP3
Windows Vista
Windows 7
10 / 10
We are in agreement !
What Can we Do ?
Automate
We know That!
Then What ??
Agenda Pain Points
ü Pain Points in Automation
ü Agreement
Pain PointsAbsence of Single “Silver Bullet” tool.
Poor Ease of automating test cases. (The coding way)
Learning Curve for new automation libraries, frameworks.
Diversified domain / functional knowledge among team members.
Enabling multiple contributors to automation projects.
Expected Low maintenance in long run .
Always expensive to switch to a different automation tool/framework.
We are in agreement !
Agenda Framework
ü Overviewü Onboardingü Architectureü What did we achieve
Framework an OverviewObject Oriented way to test a Web Application
It is a mechanism to enable
• Structured testing effort
• Multiple team members to contribute in automation
• Leverage work done by others
Separation of concerns
• Test code
• Web Page Objects (Page modeling code)
• Suite Formations
Loose Test-Tool Coupling
On demand Test Launch
Framework On-boarding : Page ObjectA Page Object is Object representation of a Web Page in Java Programming World.
All elements seen on the web page can be interacted with using the respective methods in the page
object. Page object uses information in Web Element Repository to interact with an element.
Framework On-boarding : Web Element Repository
Refers to XML based collection of meta information about elements seen on
the Web Page. Framework uses this information to identify the Web Element
and then interact with it.
<element key="SAVE_BUTTON" findBy="ID" id="save" tag="input" name="“ xPath=""/>
Here this button is identified by using the ID and the id is "save".
Framework On-boarding - Tests
Test Cases : Test cases are assertions/verifications made on compilation of
workflows using one or more real web pages. Interaction with the web pages is
done using Page Objects. Example- Hit the login page, assert that the page
loaded completely and it is login page indeed.
Test Suites : These are collection of 1 or more test cases.
Framework On-boarding - PackagingIt is an attempt to enable multiple contributors to contribute to the automation
project aligned with the team structure. An product area based segregation
helps both in division of work/impact areas and gives liberty to the area owner
to experiment with different ways of automating his/her area.
Packages aligned with common functionality provided by framework, like
utilities, logging features, framework code, etc.. should be clearly demarcated.
Framework On-boarding …
Reporting : The users can write messages to logs using the APIs exposed by the
framework. These messages are displayed in the final reports generated by the TESTNG
report engine.
Browser : Browser is a place where most of the noticeable action happens.
Test Engine :Test Engine takes in test cases and test suites and execute them.
Framework On-boarding - WebDriverWeb Driver : Page Objects indirectly interact with the set of Web Driver libraries
to hook on to the Web Elements and to interact with them.
Some typical scenarios are finding an element, sending text to element, clicking
a button, reading value of an element, and so on..
User can use the libraries directly or through the APIs provided by the
framework (Recommended way is Framework APIs )
Framework Architecture
CONFIDENTIAL
Framework What did we achieve
• Ease of use - Its only Simple Java APIs
• Easy to extend
• Reuse code and domain knowledge
• Track execution flow.
• Flexible and Powerful
• Straightforward integration with other tools/libraries (Html Unit, Selenium)
Framework is …
aggregation of various Open Source Powers enabling us to do:
§ UI testing on Multiple Browsers and Operating Systems
§ Business Logic layer testing using APIs
§ Profiling data (like page load time, first/last request, http timing, status, etc)
§ Integration with continuous integration build system with different Test Suits classes
§ Integration Virtual Test Execution Lab and Test Case Management System.
Discussion
References
Google WebDriver Project homepage
http://code.google.com/p/webdriver/
Google Test Automation Conference (GTAC) 2007 Presentation on WebDriver
http://www.youtube.com/watch?v=tGu1ud7hk5I
Latest Revision of this presentation slides
http://docs.google.com/leaf?id=0B_NzEJt8br0aNzgzYjE5OWItMjJiZC00YjhmLWEyY2YtOWE0YmY4ZjEzNjk5&hl=en
Contact Info
Kapil Bhalla
http://in.linkedin.com/pub/kapil-bhalla/7/a47/a21
Nikhil Bhandari
http://in.linkedin.com/in/nikhilbhandari
Why WebDriver?• Multi-platform support (Mac & Win)
• Multi-browser support (IE, FF, Safari, Chrome)
• Faster adoption of new technologies (Win7, new browsers, etc.)
• Free. The test developers may have fewer hassles obtaining licenses
• Tests are written in Java programming language
• Connections to browsers are optimized for each browser (IE uses COM, FF uses
a plug-in, Safari uses AppleScript). Selenium executes javascript inside the
browser, which causes some testing complications and a risk of a "single
source error“
• API is object oriented and "cleaner" than Selenium
• Manipulates browsers directly (instead of hacking JavaScript code into the site)
• Selenium and WebDriver merging! - Best of both worlds
• Eclipse IDE
Browser Drivers
Decision Criteria – Web DriverDeveloper Productivity
Object/API discoverability Good: Using tools like FireBug, IE Developer, Or source code directly (hit objects using Text, partial text, id - best way usually, xpath, etc...)
Training/Consulting available Active Web Driver Community, Internal Users
Ease of use Simple Java Programming concepts, Normal Web Elements awareness - Buttons, Anchor, Tables …
Language knowledge in team Java. Beginners 1 to 2 days are good to get going
Vendor responsiveness Its GOOGLE Open Source Community - Google Web Driver. Issues get fixed and tracked. Have seen post replys from simon stewart WD creator.
Cross Browser/Platform testing IE 7/8 YES Both FF 3.0/3.5 YESSafari 4.0 Mac Should workTool/Language Features Program Control (loops, decisions, etc.) ALL Available, you can do almost anything
Ability to handle popups Can handle pop ups that are certain, remaining need to be figured out.
Incremental Cost One Time effort to build up Specific Infrastructure. (Tool Independent) Then equivalent to : writing simple Java program.
License availability FREE (Open Source)Other Tools Integration Selenium, TestNG, Java Robots, Fitnesse YES, Any tool with which Java can be easily integrated
Decision Criteria …
Criteria Selenium WebDriver Watir Sahi
Browser/Platform Support
All All Some (immature Safari support)
All
Development/Community
Great Good Decent Small, not well known
Infrequent releases
Ease of use
Requires server Concept of Page Object, simple API
Ruby learning curve
manual proxy, no frames + multiple domains
Hacky multiple-domain support
Language SupportMany! Java Ruby, Java
comingJavaScript
Modal Dialogs Coming? Coming… Limited Limited