Yvonne and others - Mobile test automation the ING mobile app

@eddybruin

@smaljaars

#yvonne

22 mei 2013 2

22 mei 2013 3

22 mei 2013 6

22 mei 2013 7

iOS Mobile App @ ING Netherlands

• Developed in-house

• Working on 11th major release

since v1.0 November 2011

• Customized User Interface

• Native application

• Customer feedback is pivotal in

release planning

8

22 mei 2013 9

Challenges in mobile testing

10

11

22 mei 2013 12

13

13

Top 6 Android users by February 2013

User analytics based testing

14

Device Marketshare

Source: http://developer.android.com/about/dashboards/index.html

Usage of Android versions – March 2013

User analytics based testing

14

Device Marketshare

Source: http://chitika.com/ios-version-distribution /

Usage of iOS versions – March 2013

User analytics based testing

15

1 Full force manual testing

22 mei 2013 17

2

There is

no silver

bullet (yet)

• 1 Tool

• Test all device, OS combinations

• 100% tested

• Low cost

• Quick results

• Easy maintainable

• Easy to learn

What does management want?

Mobile Test Automation Eco-system

22 mei 2013 18

Technique

Framework

Infrastructure

Mobile Test Automation Eco-system

22 mei 2013 52

• Technique to interact with the (web) app

• OCR vs. Code elements

Technique

Mobile Test Automation Eco-system

22 mei 2013 53

• Maintainability

• ATDD

Framework

Mobile Test Automation Eco-system

22 mei 2013 54

• Deal with fragmentation

• Test management

• Cloud vs. private

• Automation vs. manual

• Extra test coverage

Infrastructure

Mobile Test Automation Eco-system

22 mei 2013 18

Technique

Framework

Infrastructure

3

Mobile test automation eco-system =

Technique, Framework and

Infrastructure tools

22 mei 2013 19

Technique

Infrastructure Framework

Device coverage

22 mei 2013 20

Technique

Infrastructure Framework

Platform coverage

22 mei 2013 21

22 mei 2013 22

Technique

Framework

Infra.

Robotium & UIautomator

Experitest / Silk mobile

Experitest / Silk mobile

22 mei 2013 26

22 mei 2013 26

4

Onion structure of UIA code

enables maintainability,

accessability and readability

Test case: should make payment

Test steps:

Actions: tap, fill

Elements: amount input field

Test data

1

2 3 4

Amount

Creditor name

Account number

Description

1

2

3

4

fill fields &

send payment

• Enforced consistency coding

• Test data central location

• Pixel perfect testing

• Exporting logging

22 mei 2013 29

Cooking the onion

22 mei 2013 30

Technique

Framework

Infra.

KIF

Frank

22 mei 2013 32

22 mei 2013 33

5

22 mei 2013 34

22 mei 2013 35

OPS PM

DEV QA

22 mei 2013 36

6 Yvonne guarantees test

coverage over multiple iOS

devices and OS versions

22 mei 2013 37

22 mei 2013 38

Technique

Framework

Infra.

KIF

Frank

39

Use the cloud

42

Use the crowd

22 mei 2013 43

Technique

Framework

Infra.

KIF

Frank

22 mei 2013 44

Technique

Framework

Infra.

22 mei 2013 45

Picture creativity 7

Yvonne half a year later….

22 mei 2013 46

Tweet aways Mobile testing

22 mei 2013 47

1

2

3

4

5

6

7

Exhaustive manual testing is impossible

There is no silver bullet (yet)

Mobile test automation eco-system =

Technique, Framework and Infrastructure tools Structure your UIA code

Continuous integration powerful enabler

Test automation = team effort

Yvonne guarantees test coverage over

multiple iOS devices and OS versions

22 mei 2013 48

@eddybruin

@smaljaars

#yvonne

http://goo.gl/95Tvp

Yvonne’s tale continues…