Testdroid: How to Leverage Appium in Your Mobile App Testing

webinar

How to Leverage Appium in Your Mobile App Testing

7 January 2014

Ville-Veikko HelppiTechnical Product [email protected]

Saad ChaudrySr. Software Engineer

[email protected]

mailto:[email protected]



Agenda• Different Ways to Automate Your App Testing• Appium and How It Compares Against Other

Test Automation Frameworks• Appium with Real Devices (@Testdroid Cloud)• 10 Tips for Professional Mobile App Testing• Demonstration• Q&A

webinar

© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.

2



webinar


3

Different Approaches To Testing webinar


4

Different Ways of Doing Test Automation

webinar

Automatic test exercisers Record and PlaybackHand written test scripts

Benefits:

Accurate, specific to your testing needs, plenty of options with frameworks, tools

Fast to create, accurate, not as sensitive to human-errors as hand-written tests, tools avail’ty

Fastest & extremely automated, excellent for smoke testing/quick testing, availability

Tradeoffs:

Takes a lot of time, ties resources to write test cases/scripts, error-prone (humans)

Compelling Recorder+Playback tools available for only few test automation frameworks

Not accurate as real test cases


5

Option #1: Cloud-Based Testingwebinar

Developers

Source code

Repository

Build failed

Polling

Notifying

Successful build

QA & Testers

Building

Error reporting

Test Cases

Application

Users / Consumers

Rele

ase

Com

mit


6

Option #2: Continuous Integrationwebinar

Developers

Source code

Repository

Build failed

Polling

Notifying

Successful build

QA & Testers

Building

Error reporting

Test Cases

Application

Users / Consumers

Rele

ase

Com

mit


7

Manualwebinar

Automation

Smaller coverage, More money burnt & time wasted, Error-prone

Large coverage, quickly

completed, Less money & time wasted, Exact

results.


8

Why Apps Fail?webinar


9

Why Real Devices are Must-to-Have?• Emulators cannot help you testing…– User Experience and Usability– Hardware– Software– Infrastructure

webinar

0 % = the percentage of your app users that use emulator to run your app!


10

How to Improve the App Quality• Automate generic things as much as you can• During the development your app changes –

the same must apply for testing!• Carefully select (testing) technology & partner• Use all 24 hours per day with test automation• Cloud-based platform is the only solution to

get you quickly covered in the global markets

webinar


11



webinar


12

Android Example: The Family Tree of Test Automation Frameworks

webinar

JUnit

Android Instrumentation Framework

Robotium Espresso

UI Automator

Calabash

Appium


13

Appium Introductionwebinar

• Uses Selenium Webdriver (W3C working draft) as a scripting framework– Standard for browser automation – libs for various languages

• Supports native Android, native iOS and mobile web:– Android via uiautomator (API level >=17) and Selendroid (API

level <17)– iOS via UI Automation– Mobile web as Selenium driver for Android and iOS

• You can write your Appium scripts on almost any programming language (Haskell/Go/Clojure/Java/Ruby)


14

Appium: How it works?webinar

• Appium is an HTTP server that creates and handles WebDriver sessions

• It starts an Appium server on the device that is listening commands from the main Appium process

• It receives json requests from client libraries over HTTP

• On Android Appium executes these commands as either uiautomator or Selendroid commands depending on the API level of the device

• Tests are driven from a Selenium script on Appium server

Device

Your app

Bootstrap.jar

Test server

UI Automator controller or

Selendroid driver

Selenium script


15

Appium: Test Lifecyclewebinar

1. Appium installs Bootstrap.jar as an instrumentation package which contains an uiautomator server and a tcp server

2. The commands from a Selenium script are executed on the main Appium server that relays the commands over TCP/IP to the tcp server running on the device

3. The UI Automation server converts the Selenium commands to uiautomator commands on the fly

4. If the device has API level <17 Appium installs a Selendroid server that converts the Selenium commands to Android Instrumentation (JUnit) commands

5. All test verification and result processing is happening on server side in the main Appium Server


16

Appium: Code Examplewebinar

# wait for hellosleep(3)textFields = driver.find_elements_by_tag_name('textField')assertEqual(textFields[0].get_attribute("value"), "Hello")

# click sign-in buttondriver.find_elements_by_name('Sign in')[0].click()

# find the text fields again, and enter username and passwordtextFields = driver.find_elements_by_tag_name('textField')textFields[0].send_keys("twitter_username")textFields[1].send_keys("passw0rd")

# click the Login button (the first button in the view) + sleepdriver.find_elements_by_tag_name('button')[0].click()sleep(3)

# click the first button with name "Compose” + type in the tweet message + press “Send”driver.find_elements_by_name('Compose')[0].click()driver.find_elements_by_tag_name('textField')[0].send_keys(”#Testdroid is awesome!")driver.find_elements_by_name('Send')[0].click()

# exitdriver.quit()


17

How Appium Compares to Others?webinar

Appium Robotium uiautomator Espresso Calabash

Android Yes Yes Yes Yes Yes

iOS Yes No No No Yes

Mobile web Yes (Android & iOS)

Yes (Android)

Limited to x.y clicks

No Yes (Android)

Scripting Language

Almost any Java Java Java Ruby

Test creation tools

Appium.app Testdroid Recorder

UI Automator viewer

Hierarchy Viewer

CLI

Supported API levels

All All 16 => 8, 10, 15-19 All

Community Active Contributors Google Google Pretty quiet


18



webinar


19

Appium @ Localhostwebinar


20

Test Script

Test Case

Desired Capabilities

{ “device”: “Android”, “app”: “/Users/user/ApiDemos.apk” “app-package”: “com.example.android.apis” “app-activity”: “.ApiDemos”}

Appium @ Localhost



21

Test Script

Test Case


{ “app”: “com.bitbar.testdroid.BitbarIOSSample”}

Appium @ Localhost



22

Test Script

Test Case


Appium @ Localhost

WebDriverhttp://localhost_4723/wd/hub Appium Server

4723

Device



Test Script

Test Case


From Localhost to Testdroid Cloud

WebDriverhttp://localhost_4723/wd/hub

*Testdroid Caps

http://appium.testdroid.com/wd/hub



Test Script

Test Case


From Localhost to Testdroid Cloud

WebDriver { “testdroid_username”: “[email protected]”, “testdroid_password”: “p4s$w0rd”, “testdroid_project”: “My First Project”, “testdroid_testrun”: “Test 1”, “testdroid_device”: “iPad Mini 7.0.4 A1432”, “testdroid_app”: “http://domain.com/app_v1.ipa” . . “app”: “com.bitbar.testdroid.BitbarIOSSample”}

Projects and Test Runs


25

webinar

Appium @ Testdroid Cloud


26

webinar

Appium @ Testdroid


27

webinar

Appium @ Testdroid


28

webinar

Appium @ Testdroid


29

webinar

Appium @ Testdroid


30

webinar

Appium @ Testdroid


31

webinar

Running tests on multiple devices


32

webinar



33

webinar



34

webinar

Appium Broker

5 Appium Ready

sessionid



35

Test Script

Behind the Scene

Device Cluster

Device 1 Device 2 Device 3

1 WebDriver Session Request@ http://appium.testdroid.com/wd/hub/

Desired Caps, .apk / .ipa

2 Configure project

3Wait for device to become available

4 Start Appium

Session Map

WebDriver Session response6

sessionid

7 Test Run

sessionid Proxy

Appium

Device 1



webinar


36

Tip #1: Test early, Test oftenwebinar


37

Tip #2: Plan What to Automatewebinar

People

Infrastructure

Tools

Training

Time

MAN

UAL

AUTO

MATED


38

Tip #3: Use only Real Deviceswebinar


39

Tip #4: Use Atomic Test Unitswebinar

Test iteration #1 Test iteration #2


40

Tip #5: Create Hermetic Testswebinar


41

Tip #6: Use All Possible Deviceswebinar

Testdroid Cloud’s 250+ Android devices = 93-95% global Android volumes!


42

Tip #7: Separate Apps and Testswebinar

Your App


43

Tip #8: Output Everything to Logswebinar


44

Tip #9: End-to-End Testingwebinar


45

Tip #10: Integrate Dev & Testingwebinar

Com

mit

Developers

Source code

Repository

Build failed

Polling

Notifying

Successful build

QA & Testers

Building

Error reporting

Test Cases

Application

Rele

ase

Users / Consumers


46



webinar


47



webinar


48

webinar


49

Testdroid: How to Leverage Appium in Your Mobile App Testing

Technology

testing copyrights

app testing appium

bitbar technologies

testing needs

real test cases copyrights

app users

app quality

test casesscripts