Top Banner
Why a Mobile Test Strategy is just Test Strategy 27 | 06 | 2015 HCMC Software Testing Club 2015
30

Why a Mobile Test Strategy is just Test Strategy

Mar 21, 2017

Download

Technology

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Why a Mobile Test Strategy is just Test Strategy

Why a

Mobile Test Strategy

is just Test Strategy

27 | 06 | 2015

HCMC Software Testing Club 2015

Page 2: Why a Mobile Test Strategy is just Test Strategy

2

Page 3: Why a Mobile Test Strategy is just Test Strategy

3

Page 4: Why a Mobile Test Strategy is just Test Strategy

4

Page 5: Why a Mobile Test Strategy is just Test Strategy

Are we scared of Mobile?

Challenges:

• Creating separate teams

• Managing multiple devices

• Webservers don’t run low on battery get taken home

• Browser, OS, Device, Platform combinations are more than we have ever had to manage

But it still boils down our mission – Validate products, report risk

5

Page 6: Why a Mobile Test Strategy is just Test Strategy

Our role is greater than a mobile test strategy

Building a mobile approach is a key point in your overall strategy….

…But we must do more!

Our mission, to report risk, must include a focus on where risk is introduced

We must have a seat at the table:

- What is the mobile strategy of the organization?

- What will the next technology be?

- Where will our content and products be accessed?

6

Page 7: Why a Mobile Test Strategy is just Test Strategy

What must we understand

7

Page 8: Why a Mobile Test Strategy is just Test Strategy

What must we understand

8

Page 9: Why a Mobile Test Strategy is just Test Strategy

What must we understand

• For public and consumer applications,

mobile does not just mean an “app”

• For internal, private applications, our

users or still customers and usability

can’t be taken for granted

9

Page 10: Why a Mobile Test Strategy is just Test Strategy

There are two primary drivers to Mobile Quality

10

Page 11: Why a Mobile Test Strategy is just Test Strategy

There are two primary drivers to Mobile Quality

11

Page 12: Why a Mobile Test Strategy is just Test Strategy

But our apps aren’t really meant for mobile

12

Page 13: Why a Mobile Test Strategy is just Test Strategy

What are we really testing?

• Native apps – Yes

• Hybrid apps – Yes

• Mobile web – Yes

• Responsive design – Yes

• What about:

- TV’s?

- Cars?

- Refrigerators!!!

13

Page 14: Why a Mobile Test Strategy is just Test Strategy

What are we really testing?

14

Page 15: Why a Mobile Test Strategy is just Test Strategy

What are we really testing?

15

Page 16: Why a Mobile Test Strategy is just Test Strategy

What are we really testing?

16

Page 17: Why a Mobile Test Strategy is just Test Strategy

What are we really testing?

17

Page 18: Why a Mobile Test Strategy is just Test Strategy

But, I just want to make sure it renders

We have a lot more to test than functionality and

usability

18

Page 19: Why a Mobile Test Strategy is just Test Strategy

But, we don’t have the devices

19

Page 20: Why a Mobile Test Strategy is just Test Strategy

What does this mean for the tester?

• Defect escape risk is exponentially higher with

changes in platform and technology

• Our challenges are increasing and so are our

options

• Working with product teams on strategy can manage the

risk

• Knowing that you’re mobile even when you’re not

• Communicate cost of defects and customer loss

• Building for future needs will strengthen non-mobile

aspects of your strategy

20

Page 21: Why a Mobile Test Strategy is just Test Strategy

Mobile is not only about mobile

• Building for future mobile needs will strengthen

non-mobile aspects of your strategy:

- Usability is not a uniquely mobile trait

- Customer adoption and community acceptance

isn’t either

- Size, performance, battery consumption, and

connectivity options all affect functionality

- Biometrics

- Augmented reality

21

Page 22: Why a Mobile Test Strategy is just Test Strategy

How my strategy should change for mobile

• Clearly, new techniques are needed

• The functional tester is no longer only able to

focus on functionality!

• But, we can still break this down into similar

buckets:1. Estimation

2. Requirements

3. Environment

4. Test Phases

5. Risk Analysis

22

Page 23: Why a Mobile Test Strategy is just Test Strategy

Strategy – Estimating for Mobile

The complexity of combinations is a significant drain on test estimation

We must focus on production statistics (if Available) or expected usage to drive initial, risk based estimations

Permutations of device, OS, Browser will create distraction – These are important, but must be based on target usage

AUT loading, charging, OS selection, and setup are also factors

23

Page 24: Why a Mobile Test Strategy is just Test Strategy

Strategy – Managing Mobile Requirements

This is a change in the role of the tester:

• Functional requirements will not address the

infrastructure and technical needs of a mobile

application or web experience

• We must create best practices for product owners

and architects to follow that drive quality solutions

• Testing must become the voice of the mobile strategy

24

Page 25: Why a Mobile Test Strategy is just Test Strategy

Strategy – Managing Mobile Requirements

Functional Requirements:

• Usability, accessibility, notifications and interactions with OS

Infrastructure Requirements:

• Wi-Fi vs. Network, lost transmission, security, and on device

storage

Consumption requirements:

• Battery, data, bandwidth

Input requirements:

• Camera, biometrics, keyboard types

Error Handling:

• Call interruption, multi-tasking, background usage

25

Page 26: Why a Mobile Test Strategy is just Test Strategy

Strategy – Managing Mobile Environments

Traditionally, the test environment is managed by the

infrastructure or technical team

But with mobile, who do we “request” an environment

from?

And what constitutes a mobile environment?

It is now the tester’s job to load OS, AUT, prepare the

environment, and ready for testing

26

Page 27: Why a Mobile Test Strategy is just Test Strategy

Strategy – Managing Mobile Test Phases

Traditional web\desktop testing has included a standard

approach that usually consists of: Functional, System,

Integration, NFR, Regression and UAT (along with many

other phases)

Often, with mobile, we are forced to only consider

functional (does it render) when we must also include:

• Usability

• Performance

• Security

27

Page 28: Why a Mobile Test Strategy is just Test Strategy

Strategy – Managing Mobile Quality\Risk Analysis

Between native\hybrid apps, “m.”, responsive design,

and mobile web, our risk reporting must be focused on

financial impact, customer adoption, and usability

Risk reporting now includes intended

platform\OS\Device\browser combinations

Our roles in this process will be continuing to change

but we must define risk criteria as part of our strategy

from the onset

28

Page 29: Why a Mobile Test Strategy is just Test Strategy

Summary - Quick Hits

• Mobile Strategy is as much a part of test strategy as regression, especially for customer\public portals

• Responsive design must be implemented in any external website and can be tested outside of the device

• Test strategy must evolve to include influence to design and development based on intended use

• Risk mitigation includes the identification of core aspects of product usage

• Mobile is not just for phones and tablets!

29