iPhone & Android Application Testing Author: Syeda Fatima Waseem
Jan 15, 2015
iPhone & Android Application Testing
Author: Syeda Fatima Waseem
• Android is a Linux-based operating system designed primarily for touch screen mobile devices
First Android-powered phone was sold in October 2008, and by the end of 2010 Android had become the world's leading smart phone platform
It had a worldwide smart phone market share of 59% at the beginning of 2012, and as of third quarter 2012, there were 500 million devices activated and 1.3 million activations per day
What is Andriod???
• iPhone is a line of smart phones designed and marketed
by Apple
• The iPhone runs Apple's iOS mobile operating system, originally named iPhone OS
• Applications for the iPhone are written in Objective-C
• Development of iPhone began in 2004 and first iPhone was released in 2007
What is iPhone???
• Messaging:
Apple iOS: SMS,MMS,eMailAndroid: SMS,MMS,eMail and C2DM
• Technology Access:
Apple iOS: 3G,3.5G,Wi-Fi,Bluetooth(HSDPA,HSUPA, UTMS)Android: 2G,3G,3.5G and 4G(GSM,EDGE,CDMA,EV-DO,UMTS,Bluetooth,NFC,Wi-
Fi,LTE and WiMAX)
• Compatible Devices:
Apple iOS: iPad,iPod Touch, iPhonesAndroid: Any Devices
• Web Browser:
Apple iOS:- SafariAndroid: Chrome’s V8 JavaScript engine
Difference between iOS & Android
• Connectivity:
Apple iOS: Wi-Fi, BluetoothAndroid: Wi-Fi, Bluetooth and NFC
• Some Android OS Features:
3D Google Map
Chrome to phone
3rd Party Branded OS
Adobe Flash Support
Automatic App Update
• Some Apple iOS Features:
Airprint, AirPlay
Find my iPhone
Messages with text tone
Keyboard and Directory enhancement
Difference between iOS & Android
• Mobile devices have witnessed a phenomenal growth in the past few years and there are more mobile phones in the world than televisions and PCs combined
• Market of mobile applications is growing every day and even going to become more demanding as technologies are rapidly developing
• In Mobile-Internet era people use smart phones to connect to internet for work, networking, gaming, shopping and other everyday activities, testing software applications across mobile platforms becomes indispensible
Introduction
• Testing is expensive and time consuming but it is needed
to ensure that your consumers have a positive experience when they use your mobile applications
• Fail to do a good job of testing and your customer will leave forever
Why to perform Testing???
• Primary goal of your testing efforts is not to find errors
• Goal in testing should be to understand the quality of your offerings i.e.
Goals of your Testing
Mobile application testing is a process by which application software developed for hand held mobile devices is tested
for its functionality, usability and consistency
Mobile Application Testing
Application testing Ensures
•There are number of client devices that could be used on your mobile site, and all of
them must be considered when testing your mobile application
• Number of devices can be reduced to an extent, but each time you reduce the
number of device types that you test against, you are taking a chance that your
application might not work on a device, locking out a number of potential customers
• To handle the device challenge, you have two options: You can test using real
devices, or you can test using emulated devices
Biggest Mobile testing Challange
The critical factors that determine the success of a mobile
testing program are:
•Use of emulators and actual devices
•Use of test automation
Critical Factors of Mobile Testing
Choosing Testing Strategy
Advantage:
• Real devices have the advantage of having all of the limitations and
quirks present in the actual client hardware
• Hardware Exceptions handling is possible e.g. low battery, power-offs,
insufficient memory, etc
Disadvantage:
• Testing with real devices is incredibly expensive
• Real handsets are not designed with testing in mind so limited
processing power and storage of the handsets does not allow on-board
diagnostic software to be loaded
Real Device
Advantage:
• Emulated devices are easier to manage one can switch device types by simply loading a new device profile
• Emulators run on more powerful PCs/servers and designed with testing in mind
• Emulated devices are cost effective
Disadvantage:• Emulated devices lack the quirks and faults that only the real device can
provide
• Hardware Exceptions handling is not possible e.g. low battery, power-offs, insufficient memory, etc.
Emulator
• Android is fast simulator but smaller screen of the phone makes things
a little impossible to test the entire application
• We cannot zoom in once we have the modal window, and in the original
zoomed out size we can hardly see anything for testing a specific
feature
Issues with Android Emulator
They are extremely slow and Problems occur while doing testing
on Mobile Applications
Issues with iPhone Simulator
• A simulator is a piece of software that duplicates as precisely as
possible the processor to "run" code to see if it is
correct
• An emulator generally is a piece of hardware which duplicates the
features and functions of a real system; it is self-contained, and is able
to be hooked to some kind of development environment
Difference between Simulator & Emulator
Types of Mobile Application Testing
Types of Mobile Application Testing
• Primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect
• Each unit is tested separately before integrating them into modules to test the interfaces between modules
Unit Testing
• Integration testing is the phase in software testing in which individual software modules are combined and tested as a group
• It occurs after unit testing
Integration Testing
• Functional testing is a type of Black box testing in which mobile applications are tested to ensure that it is working as per the requirements
• Functions are tested by feeding the input and examining the output
Functional Testing
• Performance testing is performed to determine how a system performs in terms of responsiveness and stability and it uncovers what needs to be improved before the product goes to market
• Check the performance and behavior of the application under certain conditions such as low battery, bad network coverage, low available memory, simultaneous access to application’s server by several users and other conditions
• As application is installed on Mobile Phones of different memory, processor and screen size so it should perform at its best despite of these variants
• Evaluates the response time (speed) of Application to execute its required functions in comparison with different Mobile devices
• Testing is performed until the Application under test achieves the expected levels of performance
Performance Testing
Performance Testing
• System tests (booting the phone, or resuming from standby)
• User interface tests (rotating the screen, displaying a menu)
• Media (starting a video)
• Phone (calling a contact)
Performance Testing: Check List
• Compatibility testing is conducted on the application to evaluate the application's compatibility with the environment Operating systems (MVS, UNIX, Windows, etc) Browser compatibility (Firefox, Netscape, Internet Explorer,
Safari, etc)
• Mobile devices have different size, resolution, screen, version
and hardware so application should be tested across all devices
to ensure that the application works as desired
Compatibility Testing
Following are some basic scenarios which should be considered while testing mobile
application:
• UI of the app is as per the screen size of the device, no text/control should be
cutting off
• Text should be readable
• Suspend/Resume (Call/SMS/Alarm) when app is running
• If app has timer/sound then Suspend/Resume timer/sound
• Behavior of app on Flip/Slider close
• Behavior of app when memory of the device is almost full
• Behavior of when network is not available
• Behavior of app if app is running for longer period of time
• Behavior of app when keys are pressed randomly
Compatibility Testing: Check List
• Load testing is done to check stability of the application when desired
concurrent number of users accesses the mobile application at a given time
• Objective is to test whether the mobile application qualifies and sustains the
required number of users with maximum load for minimum to maximum period of
time
• It helps to identify the maximum operating capacity of an application as well as
any bottlenecks and determine which element is causing degradation
Load Testing
• Stress testing is used to determine the stability of a given system when load placed on the system is raised beyond normal operational capacity often to be a breaking point, in order to observe the results
• Process of feeding largest tasks it can operate with, and constantly increasing the load on the system via automated tools to simulate real time scenario with virtual users
Stress Testing
• Regression testing seeks to uncover new software bugs in existing areas of a system after changes have been made to them
• Intent of regression testing is to ensure that a change, such as a bug fix, did not introduce new faults
• Determine whether a change in one part of the software affects other parts of the software
• Common methods of regression testing include rerunning previously run tests and checking whether program behavior has changed and whether previously fixed faults have re-emerged
Regression Testing
Other Testing
• Brute Force:
A Brute Force attack is an automated process of trial and error used to guess a
person’s username, password or credit -card number
•Insufficient Authentication:
Insufficient Authentication occurs when an application permits an attacker to access
sensitive content or functionality without having to properly authentication
• Weak Password Recovery Validation:
Weak Password Recovery Validation is when an application permits an attacker to
illegally obtain, change or recover another user’s password
• Insufficient Session Expiration:Insufficient Session Expiration is when an application permits an attacker to reuse old
session credentials or session IDs for authorization
Security Testing: Check List
• Brute Force:
A Brute Force attack is an automated process of trial and error used to guess a
person’s username, password or credit -card number
•Insufficient Authentication:
Insufficient Authentication occurs when an application permits an attacker to access
sensitive content or functionality without having to properly authentication
• Weak Password Recovery Validation:
Weak Password Recovery Validation is when an application permits an attacker to
illegally obtain, change or recover another user’s password
• Insufficient Session Expiration:Insufficient Session Expiration is when an application permits an attacker to reuse old
session credentials or session IDs for authorization
iPhone & Android ApplicationTesting: Check List
• Page scrolling
• Navigation between screens
• Data testing
• Network Testing: (if the application is Network based) Verify the behavior of application when there is Network problem and user is
performing operations for data call User should get proper error message like “Network error. Please try after
sometime”
• Application Side Effects: • Installed application should not cause other applications of device to hamper
iPhone & Android ApplicationTesting: Check List
• Check Application start/stop behavior
• Check if No disruption to voice calls: With the application installed and running use a second phone to call the test device
• Check if No disruption to text messages: With the application installed and running, send a text message to the test device
• Check for Multitasking: No disruption to key device applications
• Check for social networking options such as sharing ,posting and links etc
• Memory testing: check the memory by filling and emptying it ,then compare the application results
• Check if any payment gateway occurs like Paypal, American Express etc
iPhone & Android ApplicationTesting: Check List
• It is preferred to have a quick and easy app with less functionality than a slow and
difficult application with many features
• An app that is easy-to-use and intuitive, and similar to industry-accepted
interfaces will tend to do well
Icons & Buttons
• Buttons should have the right size and be suitable to big fingers.
• Buttons in the same place of the screen to avoid confusion.
• Natural and consistent icons.
• Buttons that have the same function should have the same color
Tips for Android Usability Testing
Touch•Tapping should zoom-in and zoom-out•There must be a way to go back or undo an action, as touching the wrong item is done quickly•Minimize keyboard input
Menu•Contextual menus should not be overloaded because it has to be used quickly
Text•Keep text simple and clear•Short sentences and paragraphs are readable•The font size should be big enough, not too big and not too small
Messages
•When the app starts downloading a lot of data, the user should be warned
•Test the closing of the app from different states and check if it re-opens in the same state
Tips for Android Usability Testing
+92-42-35782346
www.Twitter.com/SwaamTech
www.FaceBook.com/SwaamTech