Test Automation and Innovation with Open Source Tools

Testautomation & Innovation with Open Source Tools

Michael Palotas Head of Quality Engineering Europe eBay International

Zürich, May 2012

Founded in 1995 Based in San Jose, California 27770 Employees worldwide

eBay Inc. Facts

104 million active buyers and sellers worldwide

68 Billion USD transaction volume 2011

2000 USD transaction volume every second

Platform Facts

9 petabytes of data In our Hadoop and Teradata clusters

2 billion page views each day

75 billion database calls each day

Data Facts

70,000,000 Downloads of the eBay apps

5 billion USD transacted 2011 on mobile devices

1 million items are listed with using mobile apps every week

Mobile Facts

Who am I?

Michael Palotas Head of Quality Engineering Europe email: mpalotas@ebay.com

•  Introduction •  Testautomation why? •  Automation history at eBay •  Why open source? •  Testautomation who? •  Testautomation how? •  Outlook

Topics today

To get early and instant Feedback

regression tests only

safety net to save time & which is reinvested into manual testing

Test Automation – why?

NOT to replace manual testing

End-to-End Automation History at eBay

http://www.argyllhoteliona.co.uk/home_baked.php http://www.seleniumhq.org

Home baked solution Selenium IDE

Selenium RC Selenium 2 Selenium GRID

(Forget that) It’s free

Faster innovation cycles

Independence

Engagement / opportunities to grow

Hiring easier

It’s free

Why open source?

Support for tool drops

Your competitor uses the same tool

Need a different mindset

You need to solve your problems yourself

Potential risks

Testautomation – WHO?

http://gigidavidge.com/

Testautomation – HOW?

High Level Test Architecture

Biz Domain Layer

Browser

Flow Objects

Tests

WebDriver

Page Objects

Biz Domain Layer

•  Modularization and reuse of components: maintainable & extendable e2e tests

•  Crucial to have in depth knowledge of the domain

•  Enable data driven testing by generic test cases

eCommerce shopping site like eBay

Checkout Page Success Page Shopping Cart

Page View Item

Page

Pages and Flows

View Orderlist

View Category

Update

Delete View Product

Search Item

View Category Review

Select Payment Provider

Order Summary

Instant Feedback - Selenium Grid

Sequential Execution Execution time: 7000 minutes (~116 hours = ~5 days)

Test 1

Test 2

Test …

Test 4500

Execution Time

Test 3

Footnote about the scale of the tests @eBay Europe: -  4500 end-to-end regression test cases -  2-3 minutes execution time for each test

Parallel Execution Execution time: 60 minutes by using 150 Virtual Machines / Browsers using the WebDriver GRID

Test Test Test

Execution Time

Test

Test Test Test Test

Test Test Test Test

Par

alle

l Exe

cutio

n

Par

alle

l Exe

cutio

n

Summary & Outlook

•  Automation is changing

•  Open source •  Skillset

•  Mobile … stay tuned

• Try it yourself

Thank You! Questions or Comments?