About This eBook - تیستنtisten.ir/wp-content/uploads/2019/02/Daniel-Knott-Hands-On-Mobile-… · Mobile Usability Testing Accessibility Testing Battery Usage Testing Stress and

AboutThiseBook

ePUBisanopen,industry-standardformatforeBooks.However,supportofePUBanditsmanyfeaturesvariesacrossreadingdevicesandapplications.Useyourdeviceorappsettingstocustomizethepresentationtoyourliking.Settingsthatyoucancustomizeoftenincludefont,fontsize,singleordoublecolumn,landscapeorportraitmode,andfiguresthatyoucanclickortaptoenlarge.Foradditionalinformationaboutthesettingsandfeaturesonyourreadingdeviceorapp,visitthedevicemanufacturer’sWebsite.Manytitlesincludeprogrammingcodeorconfigurationexamples.To

optimizethepresentationoftheseelements,viewtheeBookinsingle-column,landscapemodeandadjustthefontsizetothesmallestsetting.Inadditiontopresentingcodeandconfigurationsinthereflowabletextformat,wehaveincludedimagesofthecodethatmimicthepresentationfoundintheprintbook;therefore,wherethereflowableformatmaycompromisethepresentationofthecodelisting,youwillseea“Clickheretoviewcodeimage”link.Clickthelinktoviewtheprint-fidelitycodeimage.Toreturntothepreviouspageviewed,clicktheBackbuttononyourdeviceorapp.

Hands-OnMobileAppTesting

AGuideforMobileTestersandAnyoneInvolved

intheMobileAppBusiness

DanielKnott

NewYork•Boston•Indianapolis•SanFranciscoToronto•Montreal•London•Munich•Paris•MadridCapetown•Sydney•Tokyo•Singapore•MexicoCity

Manyofthedesignationsusedbymanufacturersandsellerstodistinguishtheirproductsareclaimedastrademarks.Wherethosedesignationsappearinthisbook,andthepublisherwasawareofatrademarkclaim,thedesignationshavebeenprintedwithinitialcapitallettersorinallcapitals.

Theauthorandpublisherhavetakencareinthepreparationofthisbook,butmakenoexpressedorimpliedwarrantyofanykindandassumenoresponsibilityforerrorsoromissions.Noliabilityisassumedforincidentalorconsequentialdamagesinconnectionwithorarisingoutoftheuseoftheinformationorprogramscontainedherein.

Forinformationaboutbuyingthistitleinbulkquantities,orforspecialsalesopportunities(whichmayincludeelectronicversions;customcoverdesigns;andcontentparticulartoyourbusiness,traininggoals,marketingfocus,orbrandinginterests),pleasecontactourcorporatesalesdepartmentatcorpsales@pearsoned.comor(800)382-3419.

Forgovernmentsalesinquiries,[email protected].

ForquestionsaboutsalesoutsidetheU.S.,[email protected].

VisitusontheWeb:informit.com/aw

LibraryofCongressCataloging-in-PublicationDataKnott,Daniel.Hands-onmobileapptesting:aguideformobiletestersandanyoneinvolvedinthemobileappbusiness/DanielKnott.pagescmIncludesindex.ISBN978-0-13-419171-3(pbk.:alk.paper)—ISBN0-13-419171-41.Mobileapps—Testing.I.Title.QA76.76.T48K642015006.3—dc232015009688

Copyright©2015PearsonEducation,Inc.

Allrightsreserved.PrintedintheUnitedStatesofAmerica.Thispublicationisprotectedbycopyright,andpermissionmustbeobtainedfromthepublisher

priortoanyprohibitedreproduction,storageinaretrievalsystem,ortransmissioninanyformorbyanymeans,electronic,mechanical,photocopying,recording,orlikewise.Toobtainpermissiontousematerialfromthiswork,pleasesubmitawrittenrequesttoPearsonEducation,Inc.,PermissionsDepartment,200OldTappanRoad,OldTappan,NewJersey07675,oryoumayfaxyourrequestto(201)236-3290.

ISBN-13:978-0-13-419171-3ISBN-10:0-13-419171-4

TextprintedintheUnitedStatesonrecycledpaperatRRDonnelleyinCrawfordsville,Indiana.Firstprinting,May2015

Formywife,Sarah.Thankyouverymuchforyoursupportand

encouragementwhileIwrotethisbook.

Contents

Preface

Acknowledgments

AbouttheAuthor

Chapter1:What’sSpecialaboutMobileTesting?UserExpectationsMobilityandDataNetworksMobileDevicesMobileReleaseCyclesMobileTestingIsSoftwareTestingSummary

Chapter2:IntroductiontoMobileDevicesandAppsOverviewofMobileNetworksMobileDevicesandtheYear2007TheBigTwoWhatTypesofAppsExist?NativeAppsHybridAppsWebApps

BusinessModelsofMobileAppsFreemiumPaidTransactionChoosingaBusinessModel

AppStoresSummary

Chapter3:ChallengesinMobileTestingTheCustomer

CustomerSummaryMobilePlatformsandFragmentationMobileDeviceGroupsMobileDeviceLabs

SensorsandInterfacesAmbientLightSensorProximitySensorAccelerationSensorGyroscopeSensorMagneticSensorPressure,Temperature,andHumiditySensorsLocationSensorTouchlessSensorSensorsSummaryTouchscreenMicrophonesCamera

SystemAppsInternationalization(I18n)andLocalization(L10n)MobileBrowsersSummary

Chapter4:HowtoTestMobileAppsEmulator,Simulator,orRealDevice?ManualTestingversusAutomatedTesting“Traditional”TestingMobile-SpecificTestingMobileFunctionalTestingMobileUsabilityTestingAccessibilityTestingBatteryUsageTestingStressandInterruptTesting

PerformanceTestingStandbyTestingInstallationTestingUpdateTestingDatabaseTestingLocalStorageTestingSecurityTestingPlatformGuidelineTestingConformanceTestingCheckingtheLogFilesBeCreative!

Checklists,Mnemonics,andMindMapsChecklistsMnemonicsMindMaps

HowtoFileMobileBugsBugIDDescriptionStepstoReproduceExpectedResultActualResultWork-aroundReproducibleOperatingSystem,MobilePlatform,andMobileDeviceMobile-Device-SpecificInformationBrowserVersionSoftwareBuildVersionNetworkConditionandEnvironmentLanguageTestDataSeverity

BugCategoryScreenshotorVideoLogFilesTesterWhoFoundtheBugThreeMorePoints

AppQualityAllianceSummary

Chapter5:MobileTestAutomationandToolsTheFlippedTestingPyramidTheMobileTestPyramid

DifferentTypesofTestAutomationToolsImageRecognitionCoordinate-BasedRecognitionOCR/TextRecognitionNativeObjectRecognitionCaptureandReplayToolTypeRecommendation

WhatShouldBeAutomated?Emulator,Simulator,orRealDevice?Emulator/SimulatorProsEmulator/SimulatorConsRealDeviceProsRealDeviceConsWheretoAutomate?

HowtoSelecttheRightMobileTestAutomationToolSelectionCriteriaforaTestAutomationTool

CurrentStateofToolsAndroidToolsiOSToolsMobileTestAutomationToolsSummary

ContinuousIntegrationSystem

BetaDistributionToolsSummary

Chapter6:AdditionalMobileTestingMethodsCrowdTestingPrivateCrowdTesting

MobileCloudTestingPrivateCloud

Cloud-BasedTestAutomationSummary

Chapter7:MobileTestandLaunchStrategiesMobileTestStrategyDefineRequirementsTestingScopeDefineTestLevelsandTestTechniquesTestDataSelectTargetDevicesandTestEnvironmentManualandin-the-WildTestingMobileChecklistsandToursTestAutomationProductRisks

MobileLaunchStrategyPre-Launch—ChecktheReleaseMaterialPost-Release—WhatHappensafteranAppLaunch?CommunitySupportReviewsCrashReportsTrackingandStatistics

Summary

Chapter8:ImportantSkillsforMobileTestersSkillSetofaMobileTester

CommunicationCuriosityCriticalThinkingTenacityConstantLearnerCreativityCustomerFocusProgrammingandTechnicalSkills

HowtoImproveYourMobileTestingSkillsLearnfromOtherAppsCrashesonSpecificDevicesObserveTakePartinCompetitionsandTestCyclesTheMobileCommunityandtheMobileWorldValuableSources

Summary

Chapter9:What’sNext?AndFinalThoughtsInternetofThingsConnectedHomeConnectedCar

WearablesSmartWatchesandFitnessWristbandsSmartGlasses

HealthAppsFinalThoughtsFiveKeySuccessFactors

Summary

Index

Preface

Mobilephoneshavebeenaroundsincethemiddleofthe1970s.Thedeviceshaveofcoursechangedprofoundlysincethen,butthebiggestchangecamein2007whenApplepresenteditsfirstiPhone.Fromthatmomenton,themobilesmartphonemarkethasknownonlyonedirection—UP!Eightyearslater,touchdevicessuchassmartphonesandtabletshavebecomeubiquitous.Morethantwomillionappsareavailablefordownloadinthestoresofthebiggestvendors,andthisnumberisstillrising.1Thereareappsforeveryaspectofourlives,rangingfromphotosandmusic,toofficeapplicationsandgames,andontofitnessandhealth.Butwhataboutthequalityofthoseapps?Aretheyreliable,trustworthy,easytouse,welldeveloped,andtested?

1.iOSStorenumbers,www.engadget.com/2014/06/02/apples-wwdc-2014-in-numbers-40-million-on-mavericks-and-more/;AndroidPlayStorenumbers,www.appbrain.com/stats/number-of-android-apps.NumbersarefromJune2014.

Thisbookisapracticalguidetomobiletestingforanyonewhoworksinthemobilebusiness,butitisespeciallyaimedatmobiletesters.

WhyIWroteThisBookItallstartedin2010whenIhadtheopportunitytoworkonmyfirstmobileproject.ThemobileteamIworkedinwasresponsiblefordevelopingamobileWebapp,anativeAndroidapp,andanativeiOSapp.Thiswasthecompany’sfirstmobileprojectandacompletelynewtestingenvironmentforthequalityassurancedepartment.Togetherwithacolleague,Ihadthechancetobuildamobiletestingstrategyfromscratch.Weevaluatedseveraltestautomationtoolstoseewhichonefitbestinoursoftwaredevelopmentlifecycle.Atthattime,mobiletestingtoolswerefewandfarbetween,andataveryearlydevelopmentstage.Wethentriedseveraltestingapproachesandtools.Ofcoursewefailedwithsomeofthem,butintheendthewholeteam,thecompany,andourcustomerswerehappy.AnotherreasonwhyIwrotethisbookwasbecauseofmyblog,

www.adventuresinqa.com.Istartedbloggingin2011aftergivingapresentationattheAgileTestingDaysinPotsdam,Germany.Thiswasmyfirsttalkatamajortestingconference,andIwastheonlyspeakerontheagendawhospokeaboutmobiletesting.AftermypresentationIwasverybusyfortherestoftheconferenceasalotofpeopleapproachedmetoaskaboutmobiletesting,theapproachesIuse,whatkindoftoolsIuse,andsoforth.Thehugeinterestinand

thelackofknowledgeaboutmobiletestingconvincedmetostartwritingablog.Thegoalwastosharemyknowledgeofmobiletestingandtoexchangeviewsandideaswithothermobiletesters,whilealsoimprovingmywrittenEnglishskills.SofarI’vewrittenabout90postscoveringmobileappsandtesting,andIneverexpectedsomanypeoplefromaroundtheworldtotakeaninterestinmyblog.ThefeedbackI’vegottensofarhasbeengreat,anditconvincedmetotakethenextstep.Thatstepiswhatyou’rereading:abookaboutmobiletestingthatcapturesmy

practicalexperienceandknowledgeforanyoneinvolvedinthemobilebusiness.Ihopeyouenjoyreadingthisbookandcanlearnsomethingnewaboutthemobiletestingbusiness.

WhoShouldReadThisBook?Thisbookisaimedatanyonewhoisinterestedinmobileappsandmobiletesting,rangingfromjuniortoexpertmobiletesterswhoarealreadyinvolvedinmobiledevelopmentteams.Thisbookisalsoidealforsoftwaretestmanagerswhoneedtomanagemobile

testingteamsortoselectamobileteststrategy.It’salsogreatforsoftwaretesterswhoarenewtothistopicandwanttoswitchtomobiletechnologies.Softwaredeveloperswhowanttoknowmoreaboutmobiletestingandtesting

theirmobileappshavealsocometotherightplace.Thisbookisalsointendedforproductmanagerslookingtogainfurther

insightsintothechallengingjobofmobiletesting.

TopicsCoveredinThisBookThisbookcontainsthefollowingchapters:

•Chapter1:What’sSpecialaboutMobileTesting?Thefirstchapterfocusesonthespecialcharacteristicsofmobiletesting.Itprovidesanintroductiontomobileuserexpectations,mobiledatanetworks,mobiledevices,andwhymobiletestingissoftwaretesting.•Chapter2:IntroductiontoMobileDevicesandAppsChapter2introducesmobiledatanetworksandwhatisimportanttoknowaboutthem.Thechapteralsodescribesthemobiledeviceevolutionfromdumbphonestothecurrentsmartphones.Furthermore,thischapterintroducesthedifferenttypesofappsandpossibleappbusinessmodels.•Chapter3:ChallengesinMobileTestingChapter3isallaboutmobile

testingchallengesandhowtohandlethem.Therearechallengessuchasthecustomer,devicefragmentation,sensorsandinterfaces,systemapps,andmobilebrowsers.Eachsectionofthechapterprovidessolutionsforhandlingthosechallengesinyourdailybusinessasamobiletester.•Chapter4:HowtoTestMobileAppsChapter4isallabouthowtotestmobileapplications.Thischapterexplainsthedifferencesamongemulators,simulators,andrealdevices.Italsoexplainswheretotestamobileapp.Furthermore,thischapterprovidesseveralfunctionalandnonfunctionalapproachestotestingamobileapp.Inaddition,thischapterpresentsmobiletestingmindmaps,mnemonics,andcheckliststoimproveyourmobiletestingefforts.•Chapter5:MobileTestAutomationandToolsChapter5coversthetopicofmobiletestautomation,whichisaveryimportantone.Thechapterintroducesthedifferenttestautomationtooltypesandapproaches.Itprovidesideasforhowtoselecttherightmobiletestautomationtoolforyourtestenvironment.Additionally,thechapterprovidesanoverviewofthecurrentstateofmobiletestautomationtoolsforAndroidandiOS.•Chapter6:AdditionalMobileTestingMethodsChapter6providesanoverviewofadditionalmobiletestingmethodssuchascrowdandcloudtesting.Bothmethodsareexplained,includingtheprosandconsandwhereitmakessensetousetheminyourmobiletestingapproach.•Chapter7:MobileTestandLaunchStrategiesChapter7dealswiththetopicofmobiletestandlaunchstrategies.Itisveryimportantfordevelopersofmobileappstohavebothinplaceinordertodevelop,test,andlaunchamobileappwithhighquality.Thischapterprovideslotsofideasaboutandexamplesofhowtoestablishmobiletestandlaunchstrategies.•Chapter8:ImportantSkillsforMobileTestersChapter8describestherequiredskillsetofamobiletester.Furthermore,thechapterprovidesideasandsolutionsonhowtoimprovetheskillsofamobiletester.•Chapter9:What’sNext?AndFinalThoughtsChapter9isthefinalchapterofthisbookanddealswithpossibletopicsthatsoftwaretestersmayhavetohandleinthenearfuture.ThechaptercontainstopicssuchastheInternetofThings,connectedhomes,connectedcars,andwearables.Attheend,fivekeysuccessfactorsareprovided.

Eachchapterfocusesonthepracticalsideofmobiletesting.Sure,therewillbesometheoreticalparts,butmostofthecontentisbasedonreal-lifeexperience

asamobiletester.

HowtoUseThisBookThisbookisapracticalguidetomobiletesting.Youcanreaditfromfronttobacktogetanoverviewofmobiletesting,oryoucanjumpstraighttothechaptersyou’remostinterestedin.There’soneimportantpieceofadviceyoushouldbearinmindwhilereadingthisbook:makesureyouhaveatleastonemobiledevicenexttoyousoyoucantryoutthethingsyouread.Ifyouwanttogetstartedwiththetestautomationtoolsmentionedinthis

book,nowwouldbeagoodtimetogetyourcomputer.

Acknowledgments

ThankstoTobiasGeyerforbeingmydedicatedreviewer.Withoutyourcontribution,encouragement,help,feedback,andcriticalquestionsthisbookwouldnothavebecomewhatitis.ThankstoDominikDaryforyourawesomefeedbackonmobiletest

automationandhelpingmeshapethecontentofseveralchaptersinthisbook.ThankstoRudolfGrötzforyourgreatideasaboutseveralgraphicsand

images.Thankyouverymuchforyourhelpandcontributiontothecrowdandcloudtestingchapter.ThankstoDagmarMatheswhogavemetheopportunitytobepartofthe

mobiletestingbusiness.Thanksforyourtrust,support,andencouragement.ThankstoSergejMudrukandChristophWielgusforyoursupportinreading

mybookandprovidingmewithveryusefulfeedback.ThankstoAndrewRennisonforbeingmygreatcopyeditor,foryourhelpand

constantcontributiontomytext.Andlastbutnotleast,Iwanttosaythankyoutoallsoftwaretestersoutthere

forsharingyourknowledgeofvarioussoftwaretestingandmobiletestingtopics.Thanksforyourcontributiontothecommunityandallofthetimeandeffortyouputin.

AbouttheAuthor

DanielKnotthasbeenworkinginthefieldofsoftwaredevelopmentandsoftwaretestingsince2003.HestartedhiscareerasatraineeatIBMwherehewasinvolvedinenterprisesoftwaredevelopmentandtesting.AfterhistimeatIBM,DanielstudiedcomputerscienceattheUniversityof

AppliedSciencesinWiesbaden,Germany.Softwaretestingbecameapassionduringhistimeatuniversityandisthereasonhechoseacareerinthefield.DanielhasworkedatseveralcompaniesinvariousindustrieswherehewasresponsiblefortestingWeb,desktop,andmobileapplications.DuringanumberofprojectshedevelopedfullyautomatedtestingframeworksforAndroid,iOS,andWebapplications.Danielisawell-knownmobileexpert,aspeakeratvariousconferencesinEurope,andablogauthor(www.adventuresinqa.com).Furthermore,Danielisthefounderandorganizeroftwolocalsoftwaretesting

usergroupsincentralGermany.OneistheSoftwareTestUserGroupRheinMain(www.stugrm.de)andtheotheristheRheinMainMobileQualityCrew(www.meetup.com/Rhein-Main-Mobile-Quality-Crew).

Chapter1.What’sSpecialaboutMobileTesting?

BeforeIstartdescribingtheuniqueaspectsofmobiletesting,I’dliketoshareatruestorywithyou.What’sspecialaboutmobiletesting?Someoneaskedmethisexactquestion

severalyearsagowhileatatestingconference.Istartedtalkingaboutmobiletechnologies,apps,howtotestthem,andwhat’sspecialaboutmobiletesting.Theguysimplysmiledatmeandsaid,“Butit’ssoftwarejustonasmallerscreen.There’snothingspecialaboutit.”Hewasreallyarrogantanddidn’tseethechallengespresentedbymobiletesting.NomatterwhichargumentsIusedtoconvincehim,hedidn’tbelieveintheimportanceofmobiletechnologies,apps,andtesting.Imetthesameguyagainin2014whileatatestingconferencewherehe

talkedaboutmobiletesting.Hespokeabouttheimportanceofappsandhowimportantitistotestthem.Asyoucansee,it’sveryeasytounderestimatenewtechnologies.Asa

softwaretesterit’sespeciallyhelpfultobecuriousaboutlearningsomethingnewandexploringnewtechnologiestobroadenyourskills.Solet’scomebacktotheinitialquestion:What’sspecialaboutmobiletesting?

IthinkIcanassumeyouhaveatleastonemobiledevice,namely,asmartphone.Ormaybeyouhaveatablet,orevenboth.Ifyoulookatyourdevice(s),whatdoyousee?Justasmallcomputerwithlittleshinyiconsonitsscreen?Ordoyouseeaverypersonalcomputerwithlotsofsensorsandinputoptionsthatcontainsallofyourprivatedata?Pleasetakeaminutetothinkaboutthat.Mysmartphoneandtabletareverypersonalcomputersthatholdalmostallof

mydata,beite-mails,SMS,photos,music,videos,andthelike.IcanaccessmydatanomatterwhereIamandusemysmartphoneasanavigationandinformationsystemtofindoutmoreaboutmysurroundings.ForthatreasonIexpectmyappstobereliable,fast,andeasytouse.InthosethreesentencesIdescribedmypersonalexpectationsofmobile

devicesandapps.Butyoumayhaveentirelydifferentexpectations,asdoesthenextperson.Andthisbringsmetothefirstspecialcharacteristicoruniqueaspectofmobiletesting:userexpectations.

UserExpectationsInmyopinion,theuserofanappisthemainfocusandmainchallengefor

mobileteams.Thefactthateveryuserhasuniqueexpectationsmakesitdifficulttodevelopanddeliverthe“right”apptocustomers.Asseveralreportsandsurveyshaveshown,mobileusershavefarhigherexpectationsofmobileappsthanofothersoftwaresuchasbrowserapplications.1Themajorityofreportsandsurveysstatethatnearly80%ofusersdeleteanappafterusingitforthefirsttime!Thetopfourreasonsfordeletionarealwaysbaddesign,poorusability,slowloadingtime,andcrashesimmediatelyafterinstallation.Nearly60%ofuserswilldeleteanappthatrequiresregistration,andmorethanhalfofusersexpectanapptolaunchinundertwoseconds.Iftheapptakesmoretime,itgetsdeleted.Again,morethanhalfofusersexperiencecrashestheveryfirsttimetheystartanapp.Anaverageusercheckshisorhermobiledeviceeverysixminutesandhasaround40appsinstalled.Basedonthosenumbers,youcandeducethatmobileusershavereallyhighexpectationswhenitcomestousability,performance,andreliability.Thosethreecharacteristicswerementionedmostoftenbyfarwhenuserswereaskedabouttheirexperiencewithmobileapps.

1.http://offers2.compuware.com/rs/compuware/images/Mobile_App_Survey_Report.pdf

Currentlytherearemorethantwomillionappsavailableintheappstoresofthebiggestvendors.Alotofappsperformthesametask,meaningthatthere’salwaysatleastonecompetitorapp,whichmakesitveryeasyforconsumerstodownloadadifferentappasit’sjustasingletapaway.Herearesomepointsyoushouldkeepinmindwhendevelopingandtestingamobileapp:

•Gatherinformationaboutyourpossibletargetcustomergroup.•Askyourcustomersabouttheirneeds.•Yourappneedstosolveaproblemfortheuser.•Usabilityisreallyimportant.•Yourappneedstobereliableandrobust.•Appperformanceisreallyimportant.•Appsneedtobebeautiful.

Thereare,ofcourse,aplethoraofotherthingsyoushouldtakeintoaccount,butifyoupayattentiontothesepoints,yourusersarelikelytobehappy.You’veprobablyalreadyheardoftheKISSprinciple.2KISSisanacronymfor

KeepItSimple,Stupidandisalwaysausefulreminder—especiallyforsoftwareprojects—tonotinflatethesoftwarewithjustanotherfunctionoroption.Keepingitsmall,easy,andsimpleisbestinmostcasesandislikelytomakeyourcustomershappy.InspiredbyKISS,Icameupwithmyownprinciplefor

mobileapps:KIFSU(seeFigure1.1).Thisabbreviationisagoodmnemonictohelpyoucovercustomerneedsandaconstantremindernottoinflateappswithuselessfunctions.

2.http://people.apache.org/~fhanik/kiss.html

Figure1.1KIFSU

MobilityandDataNetworksAnotherchallengemobileappshavetodealwithmorethansoftwarerunningoncomputersisthefactthatusersaremovingaroundwhiletheyuseapps,whichoftenrequiresanInternetconnectiontofetchdatafromthebackendandservetheuserwithupdatesandinformation.Mobileappsneedtobetestedinreallife,inrealenvironmentswherethe

potentialuserwillusethem.Forexample,ifyou’retestinganappforsnowboardersandskiersthataccessesslopeinformation,onethatisabletorecordthespeedofthecurrentdownhillrunandmakesitpossibleforuserstosharerecordsdirectlywiththeirfriends,youneedtotestthesefunctionsonaslope.Otherwiseyoucan’tguaranteethateveryfeaturewillworkasexpected.Ofcourse,therearepartsofanappthatyoucantestinalabsituation,suchas

slopeinformationavailabilityorwhetherornottheappcanbeinstalled,butwhataboutrecordingaperson’sspeed,theweatherconditions,ortheInternetconnectionatthetopofamountain?Theweatherconditionsonamountain,inparticular,canbeverydifficultto

handleastheycan,ofcourse,rangefromsunshinetoasnowstorm.Insuchscenariosyouwillprobablyfindlotsofbugsregardingtheusabilityanddesignofanapp.Maybeyou’llalsofindsomefunctionalbugsduetothetemperature,whichmayhaveanimpactonyourhardwareand,inturn,yourapp.AsIalreadymentioned,thespeedandavailabilityofInternetconnections

couldvaryinsuchregions.Youwillprobablyhaveagoodnetworkconnectionwithhighspeedatthetopofthemountainandareallypooronedowninthe

valley.WhathappensifyouhaveabadornoInternetconnectionwhileusingtheapp?Willitcrashorwillitstillwork?Whathappensifthemobiledevicechangesnetworkproviderswhiletheappisbeingused?(Thisisacommonscenariowhenusingappsclosetoaninternationalborder,suchaswhensnowboardingintheAlps.)Allofthesequestionsareveryhardtoanswerwhentestinganappinalab.

Youasamobiletesterneedtobemobileandconnectedtodatanetworkswhiletestingapps.Asyoucansee,it’simportanttotestyourappinreal-lifeenvironmentsandto

carryouttestsindatanetworkswithdifferentbandwidthsasthebandwidthcanhaveahugeimpactonyourapp;forexample,lowbandwidthcancauseunexpectederrormessages,andtheswitchbetweenhighandlowbandwidthcancauseperformanceissuesorfreezes.Here’sanexerciseforyou.Takeanyappyouwantandfindthreeusage

scenarioswheretheenvironmentand/ornetworkconnectioncouldcauseproblems.

MobileDevicesBeforeyoucontinuereading,pickupyourmobiledeviceandlookatit.Takeyourdeviceinyourhandandlookateverysideofitwithoutturningiton.Whatdoyousee?Youwillmostlikelyseeadevicewithatouch-sensitivescreen,adevicewith

severalhardwarebuttonswithacharger,aheadphoneconnection,andacamera.That’sprobablyit—you’renotlikelytohavemorethanfivehardwarebuttons(exceptforsmartphoneswithaphysicalkeyboard).Inanerawhenthewordscellphonehavebecomesynonymouswith

smartphone,it’simportanttorememberthatthereusedtobeothertypesofcellphones,so-calleddumbphonesandfeaturephonesthathavelotsmorehardwarebuttonsformakingacallortypingamessage.Withaconventionaldumbphoneyouareonlyabletomakeacall,typeamessage,orstoreacontactlist;they’renotusuallyconnectedtotheInternet.Themoreadvancedones,thefeaturephones,havegames,acalendar,oraverybasicWebbrowserwiththeoptiontoconnecttotheInternet.Butallthesephonesarereallybasicintermsoffunctionalityandexpandabilityasusersaren’tabletoinstallappsoreasilyupdatethesoftwaretoanewerversion,ifitall.Bothtypesofphonesarestillavailable,especiallyinemergingmarkets,butsince2013moresmartphoneshavebeensoldworldwidethandumbphonesorfeaturephones,3andthistrendis

likelytocontinueastimegoeson.Infact,inthenextcoupleofyearsdumbphonesandfeaturephoneswillbeathingofthepast.

3.www.gartner.com/newsroom/id/2665715

Thephonesweusenowadaysarecompletelydifferentfromthe“old”ones.Currentsmartphonesareminisupercomputerswithlotsoffunctionalityintermsofhardwareandsoftware.They’repackedwithvarioussensorssuchasbrightness,proximity,acceleration,tilt,andmuchmore.Besidesthat,allmodernsmartphoneshavebothfront-andrear-facingcameras,variouscommunicationinterfacessuchasBluetooth,nearfieldcommunication(NFC),andGlobalPositioningSystem(GPS),aswellasWi-FiandcellularnetworkstoconnecttotheInternet.Dependingonthemobileplatformandmobilemanufacturer,youmayfindanarrayofotherhardwarefeatures.Fromasoftwarepointofview,smartphonesofferlotsofapplication

programminginterfaces(APIs)formanufacturers,developers,anduserstoextendsmartphonecapabilitieswithapps.Ifyoujustfocusonthemajormobileplatforms,iOSandAndroid,thereare

plentyofhardwareandsoftwarecombinationsthatmobiletestershavetodealwith.Thefactthattherearesomanycombinationsisknownasfragmentation.Mobiledevicefragmentationisahugetopicandyetanotherchallengewhenitcomestomobiletesting.Youcan’ttestyourappwitheverypossiblehardwareandsoftware

combination.Andthefactthatyoushouldtestyourappinarealenvironmentmakesitevenmoreimpossible.Mobiletestersneedtofindastrategytodownsizetheeffortoftestingondifferentdevicesandtofindawaytotestontherightdevices.Buthowcanthatbeaccomplished?Bytestingonjustonemobileplatform?

Bytestingonjustthelatestdevice?Bytestingwithjustthelatestsoftwareversion?Beforeyoudefineastrategy,youshouldkeepinmindthateveryappis

unique,hasuniquerequirements,hasotherproblemstosolve,andhasauniqueuserbase.Withthesepointsinmind,youcanaskyourselfthefollowingquestionstofindthe“right”mobiledevicesfortesting:

•Whoismyuserbase?•Howoldistheaverageuser?•Howmanymenorwomenareinmytargetusergroup?•Whichplatformisusedmostamongthatuserbase?

•Whichdeviceisusedmost?•Whichsoftwareversionisinstalledonmostofthephones?•Whatkindofsensorsdoesmyappuse?•Howdoestheappcommunicatewiththeoutsideworld?•Whatismyapp’smainusecase?

Ofcourse,therearelotsmorequestionstoask,butifyouanswermostoftheonesIsuggest,thelistofpossibledevicesyoushouldconsidertestingismuchshorter.InlaterchaptersIwilldescribeothertechniquesforselectingtherightdevices

formobiletesting.

MobileReleaseCyclesNowthatyouknowhowtofindtherightdevicesfortestingyourapp,itdoesn’tmeanthattheprocessisover.Tobehonest,it’snevergoingtoend!Themainmobilemanufacturersreleaseanewflagshipphonewithmore

featureseveryyear.Inandaroundthosereleasestheybringoutotherphonesfordifferentuserscenariosandusergroups.ThisisespeciallytrueintheAndroidworldwhereeverynewphonecomeswithanewversionoftheoperatingsystempackedwithnewfeatures,designs,orAPIs.Therearemultiplesoftwarereleaseswithinthecourseofayear,rangingfrombugfixestofeaturereleases.Youasamobiletesterneedtobesurethatyourappwillrunonthelatesthardwareandsoftware.Buthowshouldyouhandlethesesituations?Bybuyingeveryphonethat

appearsonthemarket?Byconstantlyupdatingtothelatestoperatingsystemversion?Again,themostimportantfactorsareyourtargetcustomergroupandtheapp

you’retesting.Whenyouknowthatyourtargetgroupalwaysusesthelatestandfastestphonesonthemarket,youneedtobuythosephonesassoonastheyappear.Regardlessofwhetherornotyourtargetgroupisup-to-date,youshouldalwaysmonitorthemobilemarket.Youneedtoknowwhenthemainvendorsareduetoreleasenewflagship

phonesthatalotofpeoplearelikelytobuy.Youalsoneedtoknowwhentheoperatingsystemsreceivepatches,newfeatures,ornewdesignpatterns.Sotheanswertothequestionofwhetheryouneedtobuyeveryphoneand

constantlyupdatetheoperatingsystemsisyesandno.Ofcourseyoudon’tneedtobuyeveryphonethat’sonthemarket,butyoushouldconsiderupdatingtothe

latestoperatingsystemversion.Whendoingso,keepinmindthatnoteveryuserwillinstalltheupdate.Manypeopledon’tknowhowtodothat,ortheydon’tcareaboutnewversions.Youneedatleastsomephonesthatarerunningolderversionsoftheoperatingsystemtoseehowtheappreactsinthatenvironment.Olderversionsoftheoperatingsystemarealsoneededtoreproducereportedproblemsandbugs.Agoodwaytomanageallthisistostickwiththesameoperatingsystem

versiononthephonesthatyouhaveandbuynewphoneswiththelatestsoftwareversion.Thisofcourseleadstoanotherproblem—it’sreallyexpensive!Noteverymanagerwantstospendsomuchmoneyonmobiledeviceswhenaphoneisgoingtobeusedforonlyacoupleofmonths.Asolutionforthatistorentdevices.ThereareseveralprovidersandOpenDeviceLabswhereyoucanrentadeviceforacertainperiodoftime(alistofproviderscanbefoundinChapter3,“ChallengesinMobileTesting”).Anotherwaytorentdevicesisthemobiledevicecloudasthereareanumberofproviderswhogivemobiletestersexclusiveaccesstothephysicaldevicestheyhavemadeavailableinthecloud.Justuseyoursearchengineandcheckthemout.InthemobileprojectsI’veworkedon,wealwayshadthetoptento15devices

usedbyourtargetusergroupindifferentvariationsfordevelopingandtesting.Thiswasagoodnumberofdevicesthatcoverednearly90%ofourtargetgroup.Withthosetento15deviceswewereabletofindmostofthecriticalbugs;theremaining10%ofdeviceswedidn’thavewereofnomajorconsequencetotheprojectoruserexpectations.Inordertohandlethefastpaceofmobilereleasecycles,youshouldkeepthe

followingthingsinmind:•Monitorthemobiledeviceandsoftwaremarket.•Knowwhennewphoneswillberolledout.•Findoutaboutthenewfeaturesoftheoperatingsystems.•Keepaneyeonyourtargetcustomergrouptoseeifnewdevicesareshowingupinyourstatistics.•Thinktwicebeforeupdatingaphonetothelatestoperatingsystemversion.•Buynewphoneswiththelatestoperatingsystemversion.•Ifbuyingisnotanoption,rentthedevices.

Updating,buying,andmaintainingallofyourdevicesisachallengingtaskandshouldnotbeunderestimated!Atsomepoint,dependingonthenumberoftestdevicesusedwithinaproject,thiscouldbeafull-timejob.

MobileTestingIsSoftwareTestingLet’scomebacktothestoryItoldatthebeginningofthischapterwhentheguyattheconferencedidn’tbelieveintheimportanceofmobiletesting.Hehadtheattitudethatmobiletestingisnotrealsoftwaretesting.Inhisopinion,mobileappswereonlysmallprogramswithlessfunctionalityandnorealchallengeswhenitcomestosoftwaretesting.Butthisisdefinitelynotthecase.IfyoulookatthetopicsIdescribedinthischapter,youshouldhaveaninitialimpressionaboutthechallengingjobofamobiletester.MobiletestingistotallydifferentfromtestingsoftwareapplicationssuchasWebordesktopapplications.Withmobileapps,physicaldeviceshavefarmoreinfluenceoverthesoftwarethatisrunningonthemwhencomparedtoothersoftwaresuchasWebapplications.Becausetherearesomanydifferentsmartphonesavailableonthemarket,mobiletestersneedtofocusalotmoreonhardwareduringthetestingprocess.Inaddition,usersmovingaroundandusingdifferentdatanetworksforcemobiletesterstobeonthemovewhiletesting.Besidesthehardware,userexpectationsplayanimportantpartinthedaily

businessofamobiletesterandneedtobetakenseriously.Therearemanymoretopicsandissuesmobiletestersneedtoknowaboutin

ordertohelpthewholeteamreleaseasuccessfulapp.Therestofthechaptersinthisbookwillcoverthefollowingtopics:

•Morechallengesformobiletestersandsolutionstothosechallenges•Howtotestmobileappssystematically•Howtoselecttherightmobiletestautomationtool•Thedifferentconceptsofmobiletestautomationtools•Howtofindtherightmobiletestingstrategy•Additionalmobiletestingmethods•Requiredskillsformobiletesters

Keepthetopicsfromthischapterinmindasastartingpoint.Keepyourappsimpleandfast(rememberKIFSU).Testwhileyou’reonthemove,andtestondifferentdevicesbasedonyourtargetcustomergroup.

SummaryThefirstchapterofthisbookmentionedsomeveryimportanttopicsfromthemobiletestingworld.Asyouhaveseen,mobiletestingiscompletelydifferentfromtestingonothertechnologiessuchaslaptopsordesktopcomputers.Thebiggestdifferencebetweenmobileandothertechnologiesisthatthemobileuser

isonthemovewhileheorsheisusingyourproduct.Therefore,itisveryimportanttoknowaboutthedifferentdatanetworksandthedifferenttypesofmobiledevices.Thischapteralsoprovidedafirstoverviewofmobileusers’highexpectations.

ItisreallyimportanttokeepKIFSUinmindwhendesigning,developing,andtestingamobileapp.Itwillhelpyoutofocusontheimportantelementsandnotwastetimeonunnecessaryfeaturesthatyouruserswon’tuse.Andlastbutnotleast,thischaptershouldremindyoutoneverunderestimatea

newtechnology.Beopen-mindedandcurioustoimproveyourdailyworklife.

Chapter2.IntroductiontoMobileDevicesandApps

BeforeIdivedeeperintothetestingchapters,I’dliketointroduceyoutothehistoryofmobiledevicesandcellularnetworks.Thismaysoundabitboring,butasyouwillseebytheendofthischapter,it’sreallyimportanttoknowthebackgroundofthemobileworldandtohavethoroughknowledgeofformermobiletechnologies.LaterinthischapterIdescribethedifferentmobileapptypesandtheappbusinessmodelsandprovideyouwithashortoverviewofthecurrentstateofmobileappstores.Let’sstartwiththewordmobile.ItcomesfromtheLatinwordmobilis,which

itselfisderivedfromtheLatinverbmovere,“tomove”—tobeabletomovearoundfreelyandeasilybywalking,driving,orflying.Thisdefinitionsoundsreallysimpleandcomprehensible,andI’msureyou

hadsomethingsimilarinmind.Ifyoulookatthewordmobilefromatechnologicalpointofview,it’snotquiteassimpleduetothevastchangesinthewaypeoplehavebeguntousemobiletechnologiesoverthepastfewdecades.Solet’sgobackafewdecadesintime.

OverviewofMobileNetworksBeforewecancommunicatewithanykindofmobiledevice,acommunicationinfrastructuremustbeavailable.Themobileinfrastructureiscurrentlyinitsfourthgeneration,knownaptlyas4GorLTE(Long-TermEvolution).1Beforethatwesawthegenerations0G,1G,2G,and3G,andeachgenerationwasamilestoneatitstimeofintroduction.

1.www.etsi.org/technologies-clusters/technologies/mobile/long-term-evolution

Thezerogeneration—theearlypredecessors—includedjustanalogradiocommunicationandwasmainlyusedinthe1960s.ItisalsoknownastheMobileRadioTelephoneSystem.Communicationatthistimewashalfduplex,meaningthatonlyonepersonwasabletotalkatatimewhiletheotherlistened.ThezerogenerationconsistedofdifferentmobiletechnologiessuchasMobileTelephoneService(MTS),MobileTelephonySystemD(MTD),AdvancedMobileTelephoneSystem(AMTS),andOffentligLandmobilTelefoni(OLT).Thecellphoneswerereallyheavyandwereinstalledmostlyintrucks,trains,andothervehicles.Thephoneconsistedoftwoparts,thetransceiver(transmitterandreceiver)andthehead.Thetransceiverwasresponsibleforestablishingtheconnectiontothelocaltransmitterstations,andtheheadwas

wiredtothetransceiverandconsistedofdialkeys,adisplay,andahandset.Thisgenerationhadalotofproblemswithconnectivityandhadlimitednumbersofsubscribers.Thefirst-generation(1G)cellularnetworkwasanimprovementoverthezero

generationandwasintroducedinthe1980s.1GstillusedanalogradiosignalstotransmitinformationusingtheAdvancedMobilePhoneService(AMPS)orNordicMobileTelephone(NMT)technology.ThefirstnetworkswerelaunchedinJapan,followedbyDenmark,Finland,Norway,Sweden,andtheUnitedStates.Acoupleofyearslaterothercountriesbuiltuptheir1Gnetworkinfrastructure.Thebiggestadvantageoverthezerogenerationwasthat1Gwasabletoaccommodateuptotentimesmoreusersbydividingthelocalareaintosmallercells.Thisgenerationhaditsdrawbackswhenitcametosecurityasuserswereabletolistentosomeoneelse’sconversationandhackthesystemtomakefreecalls.Thebiggestimprovementinmobilecommunicationnetworkswasintroduced

withthesecondgenerationofcellularnetworks.2Gwasfirstlaunchedin1991inFinlandusingtheGSM(GlobalSystemforMobileCommunications)standard.2AcoupleofyearslaterCDMA(CodeDivisionMultipleAccess)waslaunchedintheUnitedStates.3Thesenewstandardsformedthebasisoftoday’smobilecommunicationinfrastructureandofferedthreemainadvantagesovertheirpredecessors:

2.www.etsi.org/technologies-clusters/technologies/mobile/gsm3.www.etsi.org/technologies-clusters/technologies/mobile/w-cdma

•Forthefirsttimeever,communicationwasdigitalandencrypted.•2Gwaswaymoreefficientandprovidedbetterglobalcellphonecoverage.•Dataserviceswereintroduced,themostcommonlyknownonebeingtheSMS.

The2Gnetworkwasbuiltmainlyforvoiceandtextcommunicationandonlyhasslowdatatransmission.Afterthe2Gnetworkswereestablished,mobileservicesusageincreasedanddatatransmissionbecametooslow.Toachievehigherdatatransferrates,the2GnetworkwasextendedwiththeGPRS(GeneralPacketRadioService)4andEDGE(EnhancedDataratesforGlobalEvolution)5standards.GPRSisalsocalled2.5GandEDGE,2.75G.Bothtechnologieshavehigherdatatransferrates(GPRS=56Kbit/supto115Kbit/s,EDGE=upto236Kbit/s)thanthenormal2Gnetworkandarethepredecessorsofthe3Gnetwork.

4.www.etsi.org/index.php/technologies-clusters/technologies/mobile/gprs5.www.etsi.org/index.php/technologies-clusters/technologies/mobile/edge

Thethirdgenerationofmobilenetworks(3G)hasbeenaroundsince2001andisanevolutionoftheexisting2Gnetworks.ThethirdgenerationusestheUMTS(UniversalMobileTelecommunicationsSystem)6andCDMA2000standards.3Goffershigh-speeddatatransferratesupto21Mbit/sdependingontheuser’scurrentlocation.Thishighdatatransferrateallowssmartphone,tablet,orcomputeruserstomakevideocalls,watchmobileTV,andsurftheInternetwhileonthemove.The3Gnetworkswiththeirhigh-speeddatatransferrateshavehadamajorinfluenceonthesuccessofmobiledevicesandapps.

6.www.etsi.org/technologies-clusters/technologies/mobile/umts

Thefourthgenerationofmobilecommunicationnetworksaccommodatedthehugeamountsofdatanowbeingtransferredoverthenetworkbyincreasingdatatransferratesevenfurther.The4Gnetworkisbasicallydividedintotwostandards:WiMAX(WorldwideInteroperabilityforMicrowaveAccess)7andLTE.WiMAXoffersadownloadtransferrateofupto128Mbit/sandanuploadrateofupto56Mbit/s.LTEoffersdownloadratesofupto100Mbit/sandanuploadrateofupto50Mbit/s.Ifbothstandardsarefullyimplemented,thedownloadspeedcanincreaseto1Gbit/s.

7.www.wimaxforum.org/index.htm

Dependingonyournetworkproviderandcountry,yoursmartphoneisconnectedtoeitheraWiMAXorLTEnetwork.Ifyourphonesupports4Gnetworks,youwillseealittleLTEor4Giconinthestatusbarofthephone.Thefifthgenerationofmobilenetworksiscurrentlyunderdevelopment.

Severalresearchgroupshavebeenformedtodescribeanddevelopthenextgenerationofmobilecommunicationandarchitecture.However,thisstandardwillnotbeavailablebefore2020.8

8.http://europa.eu/rapid/press-release_IP-13-159_en.htm

Thiswasahigh-leveloverviewofthetechnologybehindmobilenetworks,buteventhishigh-levelviewisimportantwhenyoustarttestingyourappindifferentnetworksusingdifferentstandards.It’simportanttoknowwhatkindofnetworkstandardsareavailableandwhichnetworkspeedsareprovided.Furtherinformationonhowtotestthedifferentdatanetworkswillbeprovidedinlaterchapters.Ihighlyrecommendthatyouusethisknowledgeinyourdailyapptesting.

MobileDevicesandtheYear2007Priorto2007mostphoneswereso-calledfeaturephones(asdescribedinChapter1,“What’sSpecialaboutMobileTesting?”)whosefunctionalitycould

notbeextendedwithsoftware(apps).NotallofthemwereabletoconnecttotheInternet,evenwhenthemobilenetworkswereabletohandledatatransmission.AtthattimeNokia,Motorola,BlackBerry(ResearchinMotion),andsomeothermobilemanufacturersdominatedthemobiledevicemarket.Mostofthedeviceshadasmallscreenandaphysicalkeyboardandweregoodjustformakingacallortypingatextmessage.However,thosedeviceswerenofuntousewhentryingtosurftheWeborsearchforcontactswithinthephone.ThesmartphonerevolutionstartedinJanuary2007whenApplelaunchedits

firstiPhone.SteveJobspresentedthefirstgenerationoftheiPhonewiththefollowingsentence:“Today,Appleisgoingtoreinventthephone.”AndSteveJobswasright.Themobiledevicemarkethasofcoursechanged

dramaticallysince2007.Justoneyearlater,GooglepresentedthefirstAndroidsmartphonedevelopedbyHTC,theHTCDream(alsoknownasT-MobileG1).InthefollowingyearslotsofothermanufacturersbuilttheirownAndroidsmartphoneswithdifferentAndroidsoftwareversions.WhenAppleannouncedtheiPhone,onlyGooglewasfastenoughtoadaptand

buildanothermobileplatform.However,MicrosoftandBlackBerry(ResearchinMotion)haveupdatedtheirmobiletechnologiestoclosethegapbetweenthemselvesandAppleandGoogle,butuntilnowtheystillhaven’tmanagedtocatchup.Since2007,smartphoneshaveconstantlybeenimprovedwithnewhardware

andsoftwarefeatures,andsometimesit’shardtofollowallthenewfeaturesthatarerolledoutintothemobilemarket.Youneedtoknowwhat’sinsideasmartphonetogetanoverviewofits

physicalpartsandwhateachofthemdoes.Thisknowledgewillhelpyouunderstandthedevicesyouwanttotest,whichinturnwillhelpyoutestmoreeffectively.Ifyoulookatthemobiledevicenexttoyou,youwillseeasmall,thin,and

flatorcurvedpieceofglass,plastic,ormetal.Allthehardwarethatisneededtobringthislittlethingtolifeispackedintothesmallcaseandisnotseenbytheuser.Butwhat’sinside?Atypicalsmartphoneconsistsofthefollowinghardwarecomponents:•Mainboardorlogicboard•CPU(centralprocessingunit)•GPU(graphicsprocessingunit)•Memory

•Differentantennasandinterfaces:•Cellularnetworkchipstoconnectto2G,3G,or4G•Wi-Fi•NFC(nearfieldcommunication)•GPS(GlobalPositioningSystem)•Bluetooth

•Varioussensors(notnecessarilyalloftheonesinthislist):•Ambientlightsensor•Proximitysensor•Accelerationsensor•Gyroscope•Magneticsensor•Pressuresensor•Temperaturesensor•Humiditysensor

•Battery•Vibrationmotor•Slotforadditionalmemorycard•SlotforSIMcard

Herearethepartsyoucanseewhenyouhaveyoursmartphoneinyourhand:•Smartphonecase•Touchscreen•Hardwarebuttons(powerbutton,volumeupanddownbutton,navigationbuttons)•Headphonejack•Speakersandmicrophones•Charger/USBconnector•Frontandrearcameras•Flash

Ifyouwanttogetmoredetailedinformationaboutaspecificphone,justusetheInternetandsearchfor“teardowns”ofthedeviceorcheckoutthedevicemanufacturer’sWebsite.ThehardwareIlistedalsoappliestotabletswithsome

variations.Asamobiletester,it’sveryimportantthatyouknowallthepossible

componentswithinamobiledevice.Thisknowledgeisneededtoidentifyandnarrowdownproblemsorbugsthatcouldberelatedtothedevicehardwareoryourapp.

TheBigTwoAsmentionedinthepreviouschapter,othermobileplatformssuchasWindowsPhoneandBlackBerryhaveverylittlemarketsharecomparedtoAndroidandiOS.9AsofFebruary2015,marketshareswereasfollows:Android,55.26%;iOS,23.82%;WindowsPhone,2.32%;andBlackBerry,1.66%.Themissing17%includesdevicesrunningSymbian,Series40,andotheroutdatedmobileoperatingsystems.

9.Exactfiguresonmarketsharecanbefoundhere:http://gs.statcounter.com

ImportantThesefigureswillofcoursevarydependingongeographicallocation,butthey’reagoodindicatorofthebigpicture.

SinceBlackBerryandWindowsPhonehaveverylittlemarketshare,I’llfocusonthebigtwofortherestofthechapter:iOSandAndroid.Whatarethedifferencesbetweenthetwooperatingsystems?Whatdothey

haveincommon?Table2.1comparestheoperatingsystemsbasedoncertaincriteria.

Table2.1ComparisonofAndroidandiOS

Asyoucansee,bothplatformshavelotsoftechnologies,functions,andappsincommon,suchasappsforvoicecommands,maps,videochats,e-mail,acalendar,andmuchmore.Butifyoulookatthesourcemodelandprogramminglanguages,you’llnoticethemaindifferences.Partsoftheopen-sourceAndroidoperatingsystemarewritteninCandC++.AndroidappsarewritteninJava.iOSisalsowritteninCandC++andisclosedsource.iOSappsarewritteninObjective-CorSwift.iOSusestheopen-sourceDarwinoperatingsystemasasystembasis,butthefinishediOSversionasweknowitisclosedsource.Anotherobviousdifferencebetweenthetwoisthemanufacturers.Apple

producesiOSdevicesonitsown,whereasGooglegivesothermanufacturerstheopportunitytobuildtheirownhardwaredevicesbasedontheAndroidoperatingsystem.ThemanufacturersareabletoextendtherawAndroidoperatingsystemandbuildacustomizedAndroidversiontailoredtothehardware.Mobiletestersneedtoknowallaboutthetailoringofthedifferenthardware

manufacturersbecausetheuserinterfacesonAndroidcandifferalotandthereforeinfluencethebehaviorofanapp.ThefragmentationofAndroiddevicesandsoftwareversionsisoneofthemainchallengesthatmobiletestersneedtohandle.However,thereisalsoapureAndroiddeviceavailableonthemarketasGooglecooperateswithsomehardwaremanufacturerstobuildtheirNexusdevices.Last,Iwanttopointoutthedifferencesbetweentheuserinterfaces.Both

platformsprovidetouchinterfacesthathavelotsofgesturesincommonsuchasswiping,tapping,pinch,andzoom,buteachhasitsownUIanddesignpatterns.ForcompleteinstructionschecktheAndroiddesignguide10andtheiOSdesignguide.11Witheachnewversionoftheoperatingsystem,checkthoseguidelinesagain,becausechangesarelikely.Thesepatternsmustbefollowedinordertopublishanapp.Ifyourappdoesn’tfollowtheguidelines,itmaywellberejectedfromtheappstore,whichismorelikelytohappenintheAppleAppStorethanintheAndroidappstores.Togetanoverviewofpossiblereasonswhyanappisrejectedfromanappstore,lookatthe“CommonAppRejections”pageprovidedbyApple.1210.https://developer.android.com/design/index.html11.https://developer.apple.com/design/12.https://developer.apple.com/app-store/review/rejections/

Anothergoodreasontofollowthedesignprinciplesistomakecustomershappybecausetheyknowhowtousetheplatform-specificfeaturessuchasswipingfromlefttorighttoswitchviews,ortopulldowntorefreshthecurrentview.Whenthedevicehasbooted,bothplatformspresentahomescreentotheuser

thatissimilartoacomputerdesktop.WhiletheiOShomescreencontainsonlyrowsofappiconsspreadoverseveralhomescreens,Androidgivestheusertheabilitytocustomizethehomescreenwithappsandwidgets.13Widgetsareabletodisplaymoreinformationonthedesktop,suchascontentfromthee-mailapp,Twitter,ortheweather.Widgetscanberesizedandplacedwherevertheuserwantstohavethem.SinceiOS8,usersareabletoplacewidgetsintheiOSnotificationcenteraswell.

13.https://developer.android.com/guide/topics/appwidgets/index.html

Bothhomescreenshaveadocksectionatthebottomofthescreentopinthemostimportantappsthatareavailableoneveryhomescreen.Also,bothplatformshaveastatusbarthatisavailableatthetopofthescreenanddisplaysdevice-relevantinformationsuchasthebatterystatus,thenetworkstrength,thecurrenttime,andanynotificationsfrominstalledapps.Again,onAndroidthestatusbarcandisplaymoreinformationsuchasreceivede-mails,messages,phonecalls,orreminderslinkedtoinstalledapps.Ifyou’refamiliarwithonlyoneplatform,youshouldbuyorrentthe

unfamiliaronetolearneverythingaboutit.Youwillneedtoknowasmuchaspossibleaboutthemajorplatformstosucceedinthelongrunasamobiletester.

WhatTypesofAppsExist?Ifyouunlockyourphonetogettothehomescreen,youwillseeyourinstalledapps.Butwhichtypesofappsdoyouhaveinstalled?Aretheynative,hybrid,orWeb-basedapplications?Ifyoujustlookattheappicon,youprobablycan’ttell.Maybeyoucanfindoutbytappingonanappicontoopenit.Let’stryitout.Getyourmobiledevice,unlockit,andopenanappofyour

choice.Whatdoyousee?Isabrowserwindowshown,oristheappvisibleinfull-

screenmode?Ifyouseeabrowserwindow,yourappisaWeb-basedapplication.Butcanyouseethedifferencebetweenahybridandanativeapp?Thatwill

dependonhowwellthehybridappisdevelopedandoptimizedforyourphone.InthefollowingsectionIwilldescribethedifferentmobileapptypesandlist

theprosandconsofeachofthem.

NativeAppsNativeappsareprogrammedwithaspecificprogramminglanguageforthespecificmobileplatform.Forexample,AndroidappsaredevelopedinJava,whereasiOSappsarewritteninObjective-CorSwift.Nativeappshavefullaccesstoallplatform-specificlibrariesandAPIsinordertotakeadvantageofallthefeaturesamodernsmartphonehastooffer.Assumingtheuserhasgrantedthenecessarypermissions,theapphasdirectaccesstothecamera,GPS,andalltheothersensors.DevelopersareabletobuildappsthatmakeuseofsystemresourcessuchastheGPUandCPUtobuildpowerfulapps.Nativeappsgenerallyexhibitexcellentperformanceandareoptimizedformobileplatforms.Inmostcases,nativeappslookandfeelgreatandareabletosupportevery

possiblegestureonthetouchscreen.Appdistributionisalsoquitesimpleasyoucanuploadyournativeapptothe

appstoresofthedifferentvendorsandstartsellingit.Someappstorevendorshaveanapprovalprocess,meaningthatitcantakesometimeuntilyourappisavailable.Thesameprocessapplieswhenanalreadyreleasedappisupdated,whichcanbeaproblemespeciallywhenyouwanttofixareallyurgentbuginyourapp.

Pros:•Nativeappshavefullaccesstoplatform-specifichardwareandsoftwarefeatures.•Nativeappshavegoodperformancebecausetheyareoptimizedforthespecificmobileplatform.•Nativeappshaveagoodlook-and-feel.•NativeappsoffergoodusabilityiftheplatformUIguidelinesaremet.•Nativeappshavefullaccesstoalltouchgestures(ifimplemented).•Nativeappdistributioniseasy.Userscansearchforyourapp.•Nativeappscanstoredataoffline.

Cons:•Theamountofdevelopmentworkincreaseswitheachsupportedplatformbecauseeachplatformneedsitsowncodebase.•Theapprovalprocesscanbequitelong.•Updatingareleasedappmaytakesometime(whichisannoyingwhenitcomestourgentbugfixes).•Developmentcostscanbehigher.•Youmustshare30%ofyourapprevenueswiththeplatformprovider.

HybridAppsHybridapps,asthenamesuggests,areappsthatconsistofdifferentWebtechnologiessuchasHTMLorJavaScript.OncetheWebparthasbeenbuilt,developersareabletocompilethiscodebasetothedifferentnativeformats:Android,iOS,WindowsPhone,orBlackBerry.TocompiletheWebcodeintonativemobilecode,developersneedtouseahybriddevelopmentframeworksuchasPhoneGap.14SuchframeworksofferAPIstoaccessthedevice-specifichardwarefeatureswithintheWebpartoftheapp.14.http://phonegap.com/

Howdoessuchaframeworkwork?

ImportantThedescriptionhereisaverysimplisticviewofhybridmobileframeworks.

Theframeworkbuildsaso-calledbridgetotheWebcodeviaanHTMLrenderingengine.AsmallpartoftheapprunsonthenativeoperatingsystemandcommunicateswiththeWebcodeintherenderingengineviathebridge.Withtheaidofthisbridge,theWebcodecanaccesssomeofthenativehardwarefeatures.TheHTMLcontentorcomponentsofhybridappscanbehostedonaserver.

Thisapproachmakesitveryeasytomakesmallupdateswithoutupdatingthewholeappthroughtheappstoresubmissionprocess.Storingtheinformationandelementsontheserverhasonebigdrawback;however,thecontentandelementsdon’tworkwhenthephoneisoffline.Thesepartsareavailableonlyifthedeviceisconnectedtoadatanetwork.However,youcanputallthecontentandelementsintotheappforfullofflinesupport,butthensmallonlineupdatesarenolongerpossible.Ifyourteamisdevelopingahybridapp,keepthosepointsinmind.

Pros:•Thereisonecodebasefordifferentmobileplatforms.•Frameworksofferaccesstohardwarefeatures.•Smallupdatescanbeperformedontheserver.•Appdistributioniseasy.•Userscansearchforyourapp.

Cons:•Performanceisbadwhenthecontentandcomponentsareaccessedfromtheserver.•Meetingthedesignguidelinesofthedifferentmobileplatformsisnoteasy.•Platform-specificfeaturescan’tbedeveloped,becausetheymaynotbeavailableontheotherplatforms.•Theapprovalprocessmaybelong.

Anicecomparisonofdifferentmobiledevelopmentframeworkscanbefound

onthe“MobileFrameworksComparisonChart”Website.1515.http://mobile-frameworks-comparison-chart.com/

WebAppsAmobileWebappisaWebsitethatcanbeaccessedfromthedevice’sWebbrowser.SuchWebsitesareoptimizedformobilebrowserusageandareindependentofthemobileplatform.MobileWebappsaredevelopedwithWebtechnologiessuchasHTMLandJavaScript,particularlywithHTML5,16CSS3,andJavaScript.1716.http://dev.w3.org/html5/html-author/17.www.w3.org/Style/CSS/

HTML5offersdevelopersthecapabilitytoimplementmobileWebsiteswithanimatedandinteractiveelements.Theycanintegrateaudioorvideofilesandusepositioningfeaturesaswellassomelocalstoragefunctionality.TheuseofHTML5,CSS3,andJavaScriptmakesiteasytodevelopmobileWebapps.Furthermore,mobileWebappsrequirenoappstoreapprovalandcanbeeasilyandquicklyupdated.However,mobileWebappshavesomedrawbacks.Forexample,theyoffer

onlyverylimitedtonoaccesstothedevicehardwarefeaturessuchasproximityoraccelerationsensors.MobileWebappshavenoaccesstothecamera,compass,microphone,oranykindofnotifications.Theytendtobeslowerthannativeorhybridappsbecausetheyneedtodownloadalltheinformationthatisshownonthescreen.Dependingonthemobilebrowser,mobileWebappscanworkandbehave

differentlybecausenotallmobilebrowserssupportthefullstandardsofHTML5,CSS3,andJavaScript.ThiscanhaveamajorinfluenceonthemobileWebapp,meaningthatdifferentmobileWebbrowsersneedtobecoveredduringthetestingprocess.Tosummarize,Webappshavethefollowingadvantagesanddisadvantages:

Pros:•Populartechnologiesareusedfordevelopment.•Webappsarefasterandcheapertoimplementthannativeandhybridapps.•Theyaremobileplatformindependent.•ThereiseasyaccesstothemviaaWebbrowser(noinstallationrequired).

•Noappstoresubmissionisrequired.•Theupdatingprocessisfastandeasy.

Cons:•Thereislimitedaccesstohardwarefeatures.•Thereislimitedofflinebehavior.•Largemediafilessuchasimagesorvideoscantakealongtimetodownload.•TheyhavedifferentWebbrowsersupportfortheHTML5,CSS3,andJavaScriptstandards.•Theyarenotasconvenienttouseasnativeapps.•Thereislimitedusageoftouchgestures.•Userscan’tfindtheappintheappstores.

BusinessModelsofMobileAppsWhere’sthemoneyinmobileapps?Andhowcanyoutesttomakesuremoneyisbeingearned?Thesetwoquestionsareimportantwhendevelopingandtestinganapp.Nearlyeveryappdeveloperandcompanywantstomakemoneyfromin-apppurchases!Youthereforeneedtotestthepaymentmodelstobesurethatthemobileapp

isgeneratingrevenues,whichinturnmeansthatyouneedtobeawareofcurrentbusinessmodels:

•Freemium•Paid•Transaction

FreemiumThefreemiummodelisdesignedtoreachasmanyusersaspossible.Oncetheappisinstalled,thereareseveralwaystogeneraterevenuesoutof

thefreeapp:•Themostwidelyusedapproachisthefreeversionoftheapp.Thefreeversionislimited,suchasintermsoffunctionalityorcontent.Ifauserwantstohavefullfunctionality,heorshecandownloadtheenhanced(paid)versionoftheapp.Thisapproachisthemostwidelyusedfreemiumappmodel.

•Thesecondmostusedapproachforgeneratingrevenuesissellingadvertisementswithintheapp.Therearedifferentkindsofadvertisementformatsthatcanbeaddedtoanappinordertogeneraterevenues.Advertisementsarepartofnearlyeveryfreeappandcanoftenbereallyannoyingandfrustratingfortheuser.Thinkverycarefullybeforeaddingadstoanappasyouruntheriskoflosingcustomers.DeveloperscanimplementadframeworkssuchasAdMob18oriAd.19

18.www.google.com/ads/admob/19.http://advertising.apple.com/

•Thethirdapproachforgeneratingrevenuesisthe“in-apppurchase.”Thisisoftenusedwithingamingappswherenewlevelsormoretoolscanbeboughttohavemorefunwiththegame.Lotsofnewspaperappsalsoofferin-apppurchasestogetthelatestversionoftheirdailynews.Someappscanbecomeadfreeiftheuserpaysforit.

PaidThepaidbusinessmodelisquitesimple:beforeuserscandownloadtheapp,theyhavetopayforit.ThisisacommonusecaseforgamingappsorappsthatfulfillaspecialtasksuchasapplyingfilterstoimagestomakethemlooklikePolaroids.

TransactionInthetransactionbusinessmodeltheuserpaysonlyaftercompletingatransactionwiththeapp.AnexampleofatransactionalappisGoogleWallet,whereusersareabletosendmoneytoanotheraccountusingtheircreditordebitcard.20Oncethetransactioniscomplete,asmallfeeispaidthatdependsonthetransactionamount.20.www.google.com/wallet/

ChoosingaBusinessModelResearchcarriedoutbyGartnershowsthatfreemiumappscontainingin-apppurchasesaredownloadedthemost(approximately90%)fromtheappstores,whereaspaidappsaredownloadedfarlessfrequently.21Developersthereforeneedtoputsomethoughtintotheirapp’sbusinessmodelandprice.21.www.gartner.com/newsroom/id/2592315

Whenchoosingabusinessmodelforyourapp,makesureyoukeepthedifferentapptypesinmind.Noteverymodelcanbeappliedtoeveryapptype.

Forexample,ifyouwanttodevelopapaidmobileWebapp,yourappneedsalogintoidentifytheuser’ssubscriptionsoastogainaccesstothepaidcontent.Theloginfunctionmaynotbenecessarywithinanativeorhybridappbecausepaymentismadewithintheappstore.

AppStoresAppstoresformthecoreofthemobileworldwhereappscanbedownloadedandreviewed.Withouttheappstores,smartphoneswouldn’tbeasintelligentandfunctionalaswewantthemtobe.Atthetimeofwriting,theappstoresofthebiggestmobileplatforms—Google22andApple23—containmorethantwomillionapps.Sofarmorethan100billionappdownloadshavebeencounted.Thesearehugenumbersthatareboundtoincreaseinthefuture.22.GooglePlaystore,https://play.google.com/store23.AppleAppStore,https://itunes.apple.com/us/genre/ios/id36?mt=8

BesidesthebigtwoappstoresfromAppleandGoogle,thereareotherstoressponsoredbydevicemanufacturersandnetworkoperators.Thefollowinglistisnotcompletebutcontainssomeotherappstoresforthedifferentmobileplatforms:

•Amazon•AT&T•ChinaMobile•Mozilla•Samsung•T-Mobile•Vodafone

Butwhyaretheresomanydifferentstores?EspeciallyintheAndroidworldthereismorethanoneappstoreavailable,suchastheAmazon24andSamsung25stores.Theanswertothisquestionissimple:everyappstoreproviderwantstomakemoneyoutofthemobileappbusiness!24.www.amazon.com/mobile-apps/b?node=235014901125.http://apps.samsung.com/

Let’staketheSamsungstoreasanexample.SincethelaunchofitsGalaxydeviceseries,SamsunghasgrowntobecomeoneofthebiggestandmostsuccessfulAndroiddevicemanufacturersandhassoldandcontinuestosellmillionsofdevicesaroundtheworld.HavingitsownappstorepreinstalledoneverySamsungphoneisahugeadvantagebecauseitdirectspotentialmobile

appcustomersstraighttoSamsungandawayfromGoogle.Ifmillionsofusersusethisstore,thestoregeneratestraffic,whichmeansitcansellads.Ontopofthat,appsalesviatheappstoreallowSamsungtogenerateadditionalrevenues.Inmostoftheappstores,30%ofthesalepricegoestotheplatformprovider,andthesameappliestoin-apppurchases.Ithinkthisisindicativeofwhytherearesomanydifferentmobileappstores.

IfyousearchtheInternet,I’msureyou’llfindevenmoreofthem.Theothermobileplatformsalsohaveappstores.BlackBerryappscanbe

downloadedfromtheofficialBlackBerryWorld,26andWindowsPhoneappscanbedownloadedfromtheMicrosoftStore.2726.BlackBerryWorld,http://appworld.blackberry.com/webstore/?

d=android&o=m&countrycode=US&lang=en27.MicrosoftStore,www.windowsphone.com/en-us/store

Beforeamobileteamdistributesanappinthebiggeststores,theyshouldthinkaboutthegoalsoftheapp.Maybethereareotherstoresthatarebettersuitedtotheirappthanthebigplayers.Forexample,somestoresofferabetterrevenuesharethantheusual70%(developerrevenues)/30%(storeproviderrevenues)split,ortheyofferbetterapptargeting,forexample,indifferentmarketssuchasAfricaorAsia.However,uploadinganapptooneofthestoresrequiresonethinginparticular

—knowledgeofthestore’sreviewandpublishingprocess.Yourappneedstofulfillthereviewandpublishingguidelinesofthevariousvendorsandappstores;otherwiseyourappisverylikelytoberejected.Knowledgeofthedifferentreviewguidelineswillallowyoutobettersupport

yourteamwhiledevelopingandreleasingyourapp.ThereviewandpublishingguidelinesofthemajormobileappstorescanbefoundonthefollowingWebsites:

•AmazonAppstoreGuidelines(https://developer.amazon.com/help/faq.html)•AppleAppStoreGuidelines(https://developer.apple.com/app-store/review/)•BlackBerryWorldGuidelines(http://developer.blackberry.com/blackberryworld/vp_checklist.html)•GooglePlaystoreGuidelines(http://developer.android.com/distribute/googleplay/publish/preparing.html)•SamsungAppStoreGuidelines

(http://developer.samsung.com/distribute/app-submission-guide)•WindowsPhoneStoreGuidelines(http://msdn.microsoft.com/en-us/library/windows/apps/br230835.aspx)

SummaryChapter2coveredtheevolutionofmobiledatanetworksandmobiledevices.Itisveryimportanttoknowthedifferencesamongthedatanetworks,theirspeed,andtheirtechnologies.Thisknowledgeisrequiredwhentestingamobileappwhileyouareonthemove.Besidesthedatanetworks,knowledgeaboutmobiledevicesandtheir

evolutionisalsoveryimportant.Amobiletestermustknowallthehardwareandsoftwarecomponentsofamodernsmartphoneinordertotestmobileappsinvarioussituationsandwithdifferenthardwareandsoftwarecombinations.InalatersectionofthischapterIexplainedthedifferentapptypesthatare

currentlyavailableonthemarket.Thedifferencesamonganative,hybrid,andWebappaswellastheirprosandconsshouldbeclearbynow.Thedifferentbusinessmodelsformobileappswereexplained.Theclosing

sectionofthischapterdealtwiththedifferentmobileappstoresthatareavailableforeachplatformandwhatisimportanttoknowaboutthem.

Chapter3.ChallengesinMobileTesting

InChapter1,“What’sSpecialaboutMobileTesting?”Idescribedtheuniqueaspectsofmobiletesting,coveringuserexpectations,datanetworks,mobiledevices,andmobilereleasecycles.Butthereare,ofcourse,othertopicsthatmakemobiletestingaspecialandchallengingjob.Thischaptercontainsmoremobiletestingchallengestogetherwiththeir

possiblesolutions.

TheCustomerAsImentionedinChapter1,customersandtheirexpectationsareoneofthemainchallengesformobiledevelopersandtesters.Tosatisfycustomers,it’sreallyimportantthatyougatherinformationabout

yourpossibletargetcustomersandtheirneeds.Ifyoureleaseanappwithoutanykindofknowledgeofyourtargetgroup,theappwillmostlikelynotbeinstalledoritwillreceivereallybadreviews.Thisleadstofewerdownloads,andcustomersmayevendownloadanappfromyourcompetitor.Inordertohandlethecustomerchallenge,youneedtogatherasmuch

informationaboutyourpotentialusersaspossible.This,inturn,meansthatyouneedtoincorporatethespecificsofyourtargetgroup,suchasage,gender,andgeographicalbackground,duringthedevelopmentandtestingprocess.Youneedinformationsuchasthefollowing:

•Gender•Age•Monthlyincome(importantforpossibleappbusinessmodels)•Educationalbackground•Geographicalbackground(e.g.,dotheyliveinacity?)•Whatappstheyuseandwhatkind•Smartphonehabits(howoftentheychecktheirsmartphone)•Whethertheyuseacompetitor’sappsandifso,how,andaretheyhappywiththem•Whatdevicestheyuse

Important

Becarefulwhenaskingpeoplepersonalquestionsasyoucouldendupinfringinguponprivacylaws.

Anotherwayofgettinginformationaboutyourtargetgroupistoconductinterviews:inviteyourcustomerstoyourofficeandaskthemabouttheirmobileusageandhabits.Ifyouencounterproblemsgettingtheanswerstoallofthesequestionsorarenotallowedtoaskyourpotentialcustomersdirectly,youcanuseservicessuchasMobilePersonas1togetgeneralinformationaboutthebehaviorofmobileusers.

1.www.mobilepersonas.com/

Ifyougather,analyze,andgroupthatkindofinformation,youwillmostlikelyhavemorethanonetypeofpersontheappwillbemadefor.Tosatisfythedifferenttypesofcustomers,itishelpfultocreateso-calledpersonas2torepresenttheirvariousneeds.PersonaswereintroducedbyAlanCooper3in1998inhisbookTheInmatesAreRunningtheAsylum.

2.www.usability.gov/how-to-and-tools/methods/personas.html3.www.cooper.com/journal/2008/05/the_origin_of_personas

Personasarefictionalcharactersthatarerepresentativeofyourrealcustomers.Personasareacommonandveryusefulwaytoidentifyusermotivations,expectations,problems,habits,andgoals.Personascanhelpyoutomakedecisionsaboutyourapp,suchasfeatureset,

possiblegestures,anddesign.Personascanhelpamobileteamgetafeelfortheircustomers’needsandproblems.Youasamobiletestercanalignyourdailyworkflowtothepersonadescription.Table3.1presentsanexampleofatypicalpersona.

Table3.1PossiblePersonaDescription

Youcanalsowriteapersonadescription,suchasthisone:Martinisa28-year-oldtech-savvymalewithamaster’sdegreeincomputerscience.HelivesinNewYorkandisafrequentsmartphoneuserwhocheckshissmartphonearound150timesaday.MartinusesappslikeTwitter,Facebook,Spotify,andTumblr.Hismonthlyincomeisabout$3,000.Martinisafriendly,smart,andpolitepersonwholikestomeetfriends.

Withtheaidofpersonasandknowledgeoftheirhabits,it’sabiteasiertotestamobileappasthetestingprocessismorefocusedoncustomerneedsthanonthetester’sexpectationsorhabits.Personasareagoodwaytohandlethecustomerchallengewhenitcomesto

mobiletesting.Oncethepersonasareinplace,themobileteamshouldtrytofindrealcustomerswhomatchthepersonas.Onceyou’vefoundsomecustomers,talktothemandaskthemquestionsaboutyourappanddiscusspossibleadditionalfeatures.Inviteuserstoausabilitytestingsessioninyouroffice.Thiswaytheusersfeelconnectedwiththecompanyandyourapp.Anotherwayofgettinguserfeedbackorengagementisabetatestingphase.Thisisacommonapproach;somebigmobileappvendors4invitecustomerstotestbetaversionsofnewappstocollectfeedbackatanearlydevelopmentstagesotheycanimprove

theappbeforegoinglive.4.www.sonos.com/beta/screen/

Othervaluablesourcesofinformationaboutyourcustomersaretheappstoresofthedifferentvendorsandtheratingsforandcommentsaboutyourapp.Readthecommentscarefullyandlearnfromthem.Ofcourse,theremaybelotsofcommentsthatdon’tdeliveranyusefulinsights.However,thereareuserswhocomplainaboutusability,bugs,orproblemsyouhaveneverheardof,andthosecommentsareextremelyusefulforyouandyourteam.Moreinformationaboutappstorereviews,ratings,andcommentscanbefoundinChapter7,“MobileTestandLaunchStrategies.”

CustomerSummaryDothefollowinginordertomaximizethechancesthatyouwillsatisfyyourcustomers:

•Gatheruserinformation.•Createpersonas.•Usepersonaswhiletesting.•Invitecustomerstousabilitytests.•Interviewcustomersaboutyourproduct.•Invitecustomerstobebetatesters.•Checktheappstorereviewsandlearnfromtheusefulcomments.

MobilePlatformsandFragmentationInthepreviouschaptersIexplainedthedifferentmobilevendorsandmobileplatforms,soyounowknowwhat’sinsideamobiledevice.Youalsoknowthatforsomemobileplatformsthereismorethanonemobiledevicemanufacturer.Fragmentationisahugeprobleminthemobileworldandespeciallyinthe

Androidworld.BasedonthenumbersfromOpenSignal,5nearly19,000Androiddevicesareavailableonthemarket.It’ssimplynotpossibleand,asyouwillsee,notnecessarytotestonallofthosedevices.Thisproblemisn’tjustlimitedtoAndroid;othermobileplatformssuchasiOS,WindowsPhone,andBlackBerryarealsoaffectedbyfragmentation.Thepossiblehardwareandsoftwarecombinationsonthoseplatformscanalsobeaproblem.

5.http://opensignal.com/reports/2014/android-fragmentation/

Thenextseveralsectionsofthischapterpresentsomesolutionsforhandlingfragmentationwhiletesting.

MobileDeviceGroupsOnesolutionforhandlingdevicefragmentationinyourmobiletestingprojectistogroupyourmobiletestdevices.Ifthedevicesaregrouped,youcantestononlysomedevices(onetothree)fromeachgroup,whichhelpsyoudownsizetheamountoftestingwork.I’veadoptedthisapproachinallofmymobileproj-ectsandithasprovenveryefficient.Basedonyourtargetcustomergroup,youcancreatethreedevicegroups(thisexampleassumesthatthetargetgroupisreallytechsavvy).Thefirstgrouphasthehighestpriority:A.Devicesinthisgrouparemost

likelytobenewdeviceswithpowerfulhardwareandabigscreenwithahighresolutionandpixeldensity.Theyalsousuallyhavethelatestoperatingsystemversioninstalled.Devicesinthisgroupmustbefullysupportedbyyourappintermsoffunctionality,design,andusability.

•Group1,PriorityA:•High-enddevices•Dual/quad-coreCPU•RAM>=2,048MB•Displaysize>=5″•Retina,fullHDdisplay•Latestoperatingsystemthatisavailableforthedevice

Thesecondgrouphasmediumpriority:B.DevicesinthisgrouparemidrangedeviceswithaveragehardwaresuchasasmallerCPU,screenresolution,andsizethanthedevicesingroupA.Theoperatingsystemversionisprobablylessthanoneyearold.Thedevicesinthisgroupshouldfullysupporttheappintermsoffunctionalityandusability.Thedesigndoesn’tneedtobeperfectforthisgroupduetothesmallerscreens.

•Group2,PriorityB:•Midrangedevices•Dual-coreCPU•RAM<2,048MB•Displaysize<5″•NoRetinaorfullHDdisplay•Softwarelessthanoneyearold

Thethirdgrouphaslowpriority:C.DevicesinthisgrouphaveasmallCPU

andasmallscreenresolutionanddensity.Thesoftwareversionismorethanoneyearold.Devicesinthisgroupstillhavetofullysupporttheappintermsoffunctionality,butthedesignandusabilitymaydifferfromtheothergroupsbecausethehardwaremaybetooslowtoprovidesufficientresponsiveness.

•Group3,PriorityC:•Slowdevices•Single-coreCPU•RAM<1,024MB•Displaysize<4″•Lowscreenresolution•Operatingsystemmorethanoneyearold

Onceyouhavedefineddevicegroups,youneedtomakesureyoukeepthemcurrentbymonitoringthemobiledevicemarketfornewdevicesthatmatchyourtargetgroups.Ontheotherhand,youcanremoveolderdevicesfromyourdevicegroupsifyourcustomersdon’tusethemanymore.Andlastbutnotleast,youneedtocheckyourgroupcriteriafromtimetotimetomakesuretheystillsufficientlycoveryourcustomerspectrum.Withtheaidofsuchdevicegroups,you’llfinditmucheasiertohandledevice

fragmentationandhavetherightdevicesfortesting.AniceWebpageprovidedbyGoogleis“OurMobilePlanet,”6whereyoucan

getinformationbasedonthecountry,theage,thegender,andthebehaviorofusers.Suchinformationcanbeusedwhenthetargetcustomerisunknown.

6.http://think.withgoogle.com/mobileplanet/en/

ImportantDevicegroupsmayvarygreatlyfromprojecttoproject!

MobileDeviceLabsDependingonthemobileappproject,youmayneedlotsofdevicesfortesting,whichisofcourseveryexpensiveandtime-consuming.Agoodalternativetobuyingallofyourtestingdevicesistorentthem.Youcanusemobiledevicelabsordevicecloudstorentthetestingdevices

youneed.However,beforeyourenttestdevices,keepthedevicegroupinginmindtodownsizetheamountoftestingrequiredforallthosevirtualandphysicaldevices.

Currentlythereareplentyofmobiledevicelabprovidersthatoffermobiletestdeviceswithinthecloud.Mobiledevelopersandtestersareabletouploadtheappfiletothecloud,selectthedevices,andstartmanualorautomatedtesting.Themainadvantageofsuchservicesisthatyoudon’thavetoworryabout

buyingnewdevicesandmaintainingthem.However,beforechoosingamobilecloudprovider,youshouldcheckwhethertheproviderofferstestingonrealdevices,simulators,oremulators.Amobilecloudtestingprovidershould:•Testonrealdevices•Beabletouseemulatorsorsimulatorsaswell•Notusejailbrokenorrooteddevices•Beabletotestondifferentmobileplatformsifpossible•Beabletowritetestautomationscriptsindifferentprogramminglanguages•Offercontinuousintegration•Havesomeperformancemonitoringinplace•Generateatestreportattheendofthetestcycle

Testingyourmobileappinthecloudonemulatorsorsimulatorscanbeveryannoyingduetoperformanceissues.Usingemulatorsorsimulatorsonalocalmachineisoftenalreadyfairlyslow;usingthemontheInternetviaaWebGUIcanbeevenslower.Eveniftheproviderofferstestingonrealdevices,it’sstillpossiblethatthe

performancemaynotbethesameaswhentestingonalocalphysicaldevice.Thelatencymaybetoohigh,whichcanleadtoveryslowreactionsonthedevice.Scrolling,tapping,orjustloadingtheappcantakealongtime,whichmayhaveabigimpactonyourtestingandthesubsequentresults.Anotherthingtolookoutforwhenchoosingacloudprovideristomakesure

youhaveexclusiveaccesstothephysicaldevicesandthatyourappisdeletedoncethetestsessioniscomplete.Checkthattheprovideroffersaprivatetestcloud;otherwiseyourappmaystillbeinstalledonthetestdevicesforsubsequentcustomerstosee.Discoveringsuchanissuewhileevaluatingaproviderisaclearwarningsignthatshouldnotbeignored.Ifsecurityandprivacyareimportanttoyou,considerpickinganotherprovider.Thefollowingprovidersofferamobiledevicetestcloud:•AppThwack(https://appthwack.com/)

•CloudMonkeyLabManager(www.cloudmonkeymobile.com/labmanager)•KeynoteMobileTesting(www.keynotedeviceanywhere.com/)•MobileLabs(http://mobilelabsinc.com/)•PerfectoMobile(www.perfectomobile.com/)•SauceLabs(https://saucelabs.com/)•Testmunk(www.testmunk.com/)•TestObject(http://testobject.com/)•TouchTest(www.soasta.com/products/touchtest/)•XamarinTestCloud(http://xamarin.com/test-cloud)

ImportantThislistisbynomeanscomplete.JustbrowsetheInternettosearchformoreprovidersthatfitinyourenvironment.

Ifyoudon’twanttobuyeverytestingdeviceforyourcompanyordon’twanttouseacloudprovider(ormaybeyou’renotallowedtouseone),thereisanotherwayofgettingphysicalmobiledevices:ODLs,orOpenDeviceLabs.OpenDeviceLabswereintroducedbythemobilecommunitywiththeaimof

establishingphysicalstoreswheremobiletesters,developers,andanyoneinvolvedinthemobilebusinesscanobtaindevicesfortestingpurposes.TheprimarygoaloftheseOpenDeviceLabsisthateveryonecanborrowdevicesforfree!Inordertomakethisideasuccessful,OpenDeviceLabsneeddevicedonationsfromindividualsorcompanieswhowanttosupportthemobilecommunity.Dependingonyourgeographicalregion,youmighthaveanOpenDeviceLabnearby.TheWebsite7providesanicemapthatshowsyouwhereyournearestOpenDeviceLabislocated.Youshoulddefinitelytrythem!

7.http://opendevicelab.com/

Chapter7includesmobiletestlabsandOpenDeviceLabsaspartofthemobiletestingstrategyandalsoprovidesfurtherinformationonthistopic.Inthispartofthechapteryouhavelearnedthattherearethreepossible

solutionstotheproblemofhowtohandlemobilefragmentation:•Groupingthedevicesyouneedfortesting•Mobiledevicelabsinthecloud•UsingphysicaldevicesfromanOpenDeviceLab

SensorsandInterfacesEverysmartphonehasavarietyofsensorsandinterfacesthatcanbeaccessedbytheinstalledappstoprovideusefulfeaturestousers.Theactualsensorsandinterfacesuseddependontheapp’sspecificusecase.Theinstalledsensorsandinterfacesarereallycomplexandarepotentiallysusceptibletoelectricalinterference.Asamobiletesterit’syourresponsibilitytoensurethattheimplemented

sensorsandinterfacesareusedcorrectly.It’salsoimportanttoverifythatfailingsensorsdon’taffecttheappinanegativeway.

AmbientLightSensorTheambientlightsensorisabletodeterminehowmuchlightisavailableinthecurrentlocationandautomaticallyadjustthescreen’sbrightnessbymeansofsoftwareinordertoprolongthedevice’sbatterylife.Ifyourappmakesuseoftheambientlightsensorinanyway,youshouldtest

yourappindifferentlocationswithdifferentprevailinglightsituations.Forexample,testyourappinadarkroom,outsideinthesunshine,oratyourworkplacetoseeiftheapprespondscorrectly.Becreativewithyourtestingenvironments.Differentlocationscouldinclude:•Adarkroom•Yourworkplacewithadesklampon•Outsideinthesunshine•Roomswithdifferentlightsontheceiling

ProximitySensorAnotherlittlehelperistheproximitysensor,whichdetermineshowclosethephoneislocated,suchastoafaceorsurface.Itcanbeusedtoautomaticallyturnoffthedisplaywithoutanyphysicalcontact.Thispreventsyoufromaccidentallyclickingortouchingbuttonsonanactivescreen.It’salsousefulforsavingbatterylife.Makesureyoualsotesttheusageoftheproximitysensorindifferent

locationswithdifferentlightsituationsaswellaswithdifferentkindsofsurfacestoseewhethertheappisusingthesensorcorrectly.Differentlocationsandsurfacescouldincludethefollowing:

•Adarkroom•Yourworkplacewithadesklampon•Outsideinthesunshine•Yourhandmovingoverthedisplay•Yourhandclosetothedisplay•Thedevicemovingtowardyourface•Thedevice’sdisplaymovingtowardamirrororpaneofglass

AccelerationSensorTheaccelerationsensordetectschangesinthedevice’smovement.Themostcommonusecaseforthissensoriswhenthedeviceisrotatedbetweenportraitandlandscapemodes.Thissensorisusedinnearlyeveryappifthedevelopersimplementedaportraitandlandscapeview.Ifbothmodesaresupported,youshouldchangetheorientationofthedevicealotduringtesting.Bydoingso,youwillprobablyfindlotsofUIglitchesbecausetheUIelementscouldbemovedtoadifferentposition.Itisalsolikelythattheappwillcrash,forexample,whendataisfetchedfromthebackendwhileaUIrefreshisbeingperformed.

GyroscopeSensorThegyroscopesensorisusedtoeithermeasureormaintaintheorientationofthedevice.Unlikeanaccelerometer,whichmeasuresthelinearaccelerationofadevice,agyroscopemeasuresthedevice’sexactorientation.Thismeansthatthedeviceisabletodetect360-degreemotion.Thankstotheaccelerometerandgyroscopesensors,thedeviceisabletooperateonsixaxes—leftandright,upanddown,forwardandbackward—andtrackroll,yaw,andpitchrotations.Thecombinationofbothsensorsismostlyusedingamingappssuchasflight

simulationstocontrolaplanewithrealphysicalmovements.Keepthesixaxesinmindwhentestingyourapp,andtesteachaxisseparatelyaswellasmultipleaxesatoncetobesuretheappisrespondingcorrectly.

MagneticSensorAmagneticsensorisabletomeasurethestrengthandorientationofmagneticfieldsaroundthedevice.Thissensorismostlyusedbyappsthatrequirecompassinformation,suchasfornavigation.Withtheaidofthissensorthedeviceisabletodeterminethedirectioninwhichit’sfacing:west,east,north,orsouth.Ifyourappusesthemagneticsensor,youshouldalsotestitindifferent

locations.Forexample,ifyou’reusingtheappinabuildingwhereyou’resurroundedbylotsofmetal,themagneticsensormaydeliverfalseinformationordata,whichcanleadtoadversesideeffectsinyourapp.Differentlocationscouldincludethefollowing:•Insideabuilding•Outsideonthestreet•Whenstuckintraffic•Nearbuildingswithlotsofmetal

Pressure,Temperature,andHumiditySensorsThesethreesensorsarenotincludedineverysmartphoneyet,buttheywillbesoon.Allthreesensorscanbeusedtocollectmoredataabouttheuser’scurrentlocationtoprovideappswithusefulinformationsuchasthecurrenttemperature,currentaltitudebasedonatmosphericpressure,andhumidity.Thesesensorsareused,forexample,byoutdoororweatherapps.Again,make

sureyoutestthesesensorsindifferentlocationswithdifferenttemperatures,pressures,altitudes,andhumidities.Youcan,ofcourse,simulatepressureorhumidityinatestlabsituationtocheckthatthesensorsareworkingcorrectly,buttestingtheminrealsituationscouldproducesideeffectsthataren’tlikelytohappeninalab.

LocationSensorYoucertainlyknowthelocationsensor,betterknownasGPS.ThankstoGPS,appscandeterminethecurrentdevicelocation.GPSisusedinlotsofdifferentkindsofappssuchasmapapps,cameraapps,andsocialmediaapps.Userscansharetheircurrentlocationwithappsandsendtheircurrentposition,forexample,tofriendstoletthemknowwheretheyare.IfyourappusesGPStodeterminethedevice’scurrentposition,besurethat

theGPSsensorisswitchedoffafteruse.Otherwisethedevice’sbatterywillbeemptyverysoon.GPSfunctionalityshouldofcoursebetestedinvariouslocations,suchasin

thecountrysideorindowntownsprawlwithlotsofhugebuildings.InbothscenariosyoumayfindthatthecurrentlocationisincorrectornotfoundbytheGPSsensor.Thisisespeciallytrueincitiesduetointerferencecausedbysurroundingbuildings.IftheGPSsignalisnotstrongenoughtodeterminethecurrentlocation,checkthattheWi-Fiornetwork-basedlocationfindingfeature

ofthephoneisactivatedtolocatethephone.AlowGPSsignalcanalsocauseperformanceproblemswithinyourappduetodifficultiesincalculatingthecurrentposition.WhentestingtheGPSfunctionofyourapp,keepaneyeonthetemperatureofyourdevice.UsingGPSwillheatupyourdevice,consumelotsofbattery,andthereforepossiblyinfluencetheperformanceofyourapp.IalsorecommendthatyouturnGPSofftoseehowyourappreactstothisscenario.

TouchlessSensorAnothersensorthatisnotbuiltintoeverysmartphoneisatouchlesssensor.Inmostcasesthissensorisbuiltintothefrontofthedevicetoaccepttouchlessgesturesfromafingerorahand.Whatthismeansisthatyoucanswipebetweenphotosinyourphotogallerysimplybywavingyourhandoverthescreen.Oryoucanansweraphonecallbyholdingthedevicenexttoyourearwithouttouchingtheanswerbutton.Ifyourappsupportstouchlessgestures,besureeverygesturealsoworksat

differentangles.

SensorsSummaryMostofthesensorsjustdescribedareinfluencedbytheuser’scurrentlocationandsurroundings.Anykindofmovementisveryimportantwhiletestingyourappandthedevice’ssensors.It’salsousefultotestthesensorsincombinationwithoneanothertocheckthatyourapprespondscorrectlytochangesofsensordata.Possiblesensortestscouldincludethefollowing:•Walkwhileusingtheappasthissimulatesback-and-forthmotion.•Whilewalking,suddenlystopforafewmomentsbeforecontinuingonyourway.•Shakeandrotatethedevice.•Simulatevibrationsasifyou’reonatrainorinacar.•Performextrememovementssuchasspinningorrotatingthedevice.•Usetheappoutdoorsinsunny,cloudy,anddarklocations.•Usetheappindoorswithnormallightorunderalamp.•Waveyourhandsinfrontofthedevice.•Testtheedgecases,forexample:GPScoordinates0,0;temperatureat0degrees;waterproofdevicesinareaswithhighhumidity.

•Checktoseeiftheappcanbeusedondevicesthatdon’thavebuilt-insensorsandinterfaces.

Whileperformingthesetasks,watchoutforappcrashesorfreezes.CheckthattheUIelementsarevisibleanddisplayedcorrectlyonthescreen,especiallyindifferentlightsituations.Keepaneyeontheperformanceoftheappandthesmartphone’sbatterylifetimeaswell.

TouchscreenThebiggestandmostimportantinterfaceinsmartphonesisthetouchscreen.Thankstothetouch-sensitivescreen,userscanuseoneormorefingersonthedevicetocreategesturesthatthephoneisabletoconvertintocommands.

ImportantThedescriptionoftouchscreentechnologieshereissimplified.Ifyouwanttogetmoredetailedinformationaboutthem,pleasedosomeonlineresearch.

Therearebasicallytwokindsoftouchscreentechnologiesavailable.Thefirstistheresistivescreenthatismadeoutofvariouslayersandreactstopressure.Usuallythiskindoftouchscreenismadeforusewithastylus.Wheneveryouhavetosignforaparceldelivery,youprobablysignonaresistivescreen.Thistechnologyhasonebigdrawback:itdoesn’tsupportmultitouchgestures.Thatisthereasonwhythesecondtechnology,capacitivetouchscreens,isused

insmartphones.Capacitivescreensreacttotouchratherthanpressureandsupportmultitouchgestures.Capacitivescreensconsistofaninsulator,whichinmostcasesisglasscoated

withatransparentconductorlikeindiumtinoxide.Sincethehumanbodyisalsoanelectricalconductor,touchingthecapacitivescreenresultsinadistortionofthescreen’selectrostaticfield.Thisdistortionisthenconvertedintodatathatthedevicehardwareisabletounderstand.Thefollowinggesturesarepossibleonacapacitivetouchscreen:•Touch:Touchthescreenwithafingertip.•Longtouch:Touchthescreenforalongertime.•Swipe:Moveyourfingertipoverthescreen.•Tap:Brieflytouchthescreenwithafinger.•Doubletap:Brieflytouchthescreenwithafingertiptwice.

•Drag:Moveafingeroverthescreenwithoutlosingcontactwiththescreen.•Multitouch:Usetwoormorefingersonthescreenatthesametime.•Pinchopen:Touchthescreenwithtwofingersandmovethemapart.•Pinchclose:Touchthescreenwithtwofingersandbringthemclosertogether.•Rotate:Usetwofingersonthescreenandrotatethem.Someapps,suchasmapapps,willrotatethecontentintheapp.

Thevarietyofpossibletouchgesturesposesaspecialchallengewhiletestingamobileapp.Youshouldkeepallthepossibletouchgesturesinmindandusethemwhiletesting.Agoodwaytoseeifanappcanstanduptotouchgesturesistousemultiplefingersonthetouchscreenatthesametime.Youshouldalsoperformseveraldifferenttouchgesturesveryquicklyonthescreentoseehowtheappreactsandhandlestheinputs.AlsowatchoutforperformanceissuesorUIglitchesthatcanhappenwhileusingmultitouchgestures.Oneimportantfactortonoteabouttouchscreensisthecurrentweather

conditions.Theycanaffectyourfingersandpreventtouchesfrombeingregisteredproperly.Therefore,it’sagoodideatoalsousethedifferentgesturesindifferentweatherconditionssuchasoncoldorhotdaysoronadaywithhighorlowhumiditytoseehowtheappreactstoyourgesturesonthetouchscreen.

MicrophonesAnotherwayofcommunicatingwithyourappisbyusingyourvoiceorsound.Mostsmartphoneshavemorethanonemicrophoneinstalled.Usuallythereareuptothreemicrophones,oneatthefront,oneattheback(nearthecamera),andoneatthebottomofthedevice.Thethreemicrophonesensureverygoodvoicerecordingfromallpossibleanglesandregardlessofthephone’sposition.Whentestingsoundinputsviathemicrophone,makesureyoudothe

following:•Testvoiceinputindoorswithnormalsoundandinnoisysituations.•Testtheappindoorswithmorebackgroundnoisesuchasinanofficeorrestaurant.•Testtheappoutdoorswithbackgroundnoisescomingfromastreetorcars.•Checkthattheappcanhandletheusermutingandunmutingthephoneorthemicrophone.•Startotherappsthatgeneratesounds,suchasmusicapps,andcheckthe

behaviorofyourapp.•Usethevolumeupanddownbuttonstoincreaseordecreasethesoundlevelandcheckthebehavioroftheapp.•Checkthatvoiceinputsareprocessedcorrectly.•Ifthevoiceinputisstoredonthephone,checkthatplaybackisworkingproperly.•Testthevoiceinputinareal-lifeenvironment(thepurposeoftheapp).

Whiletestingthesoundinputsinallthedifferentsurroundings,watchoutforinputdelaysordistortionintherecordingorplaybackmode.Keepaneyeontheperformanceoftheappandthephonetoseeifthedevicefreezesorcrashesduetoalackofhardwareresources(especiallywhensavingtheinputs).Anddon’tforgettocheckthedigitalrightsmanagement(DRM)whenworkingwithvoiceorsoundinputs.

CameraAtypicalsmartphonehasatleastonecamera,usuallylocatedatthebackofthesmartphone.However,mostoftoday’ssmartphonesalreadyhavetwocameras:arear-facingandafront-facingcamera.Therearcameraisusedtotakehigh-resolutionpictures,andthefrontcameraismostlyusedforvideochattingwithalowerresolution.Also,mostoftherear-facingcamerashaveaflashincluded.Camerasareusedinavarietyofmobileappsfortakingpicturesorvideos.

SomeappsusethecameraasascannertocaptureinformationusingOCR(opticalcharacterrecognition)orotherkindsofshapes.GoodexamplesofthisincludeQR(QuickResponse)codes,scanningapps,orappsthatscanbusinesscardstotransformthemintodigitalcontactsinyourphone.SomeappsjustusetheLEDflashtoactasaflashlight.Ifyourappusesthecamera,testitwithavarietyofmobiledevicesthatmatch

yourtargetcustomergroup.Everysmartphonehasauniquecamerawithauniquelens,flash,andresolution.Thedifferentresolutionsofthecamerahaveanimpactontheimagesize.Thehighertheresolution,thebiggerthepicturesare.Besuretotestthecamerafunctionwithdifferentresolutionsandcameratypesinordertoseeiftheappisabletoprocessbothsmallandlargeimages.Besidesthat,youshouldchecktheperformanceoftheappwhilethecamera

modeisactivated.Thecameratakesupalotofthedevice’shardwareresources,whichcanhaveanadverseimpactontheappandcauseittocrashorfreeze.Heavycamerausecanheatupthecamerasensorandcancausehardwaredamage,sobesurethatthisisnotpossiblewithinyourapp.Finally,don’tforget

totestthattheappcanuseanimagestabilizertopreventthecamerafromtakingblurrypictures.Thesameappliestovideocapturingwithinyourapp.

SystemAppsMostpreinstalledappsfromdevicemanufacturersaresystemapps.Thereareoftenpreinstalledappssuchasacontactapp,aphoneapp,acalendarapp,andthelike.However,onmostofthemobileplatformsusersareabletoinstallappsthatreplacethesystemapps.Themainreasonsusersdothisisbadusabilityorthelackoffunctionalityonthepartofthesystemapps.Ifyouchecktheappstoresofthedifferentvendorsforcalendarapps,forinstance,you’llfindplentyofappswithfarmorefeaturesthanthepreinstalledones.Areallyinterestingexampleofthisiskeyboardappsonsmartphones.Such

appsarelikelytohaveanimpactonyourapp.OnAndroidphonesandtablets(and,sinceiOS8,alsooniOSdevices),usersareabletoreplacethepreinstalledkeyboardappwithspecialkeyboardsthatofferatotallydifferentwayoftypingandkeyboardlayouts.Thereareconventionaltapkeyboards(likeQWERTY),keyboardswithdifferentlayouts(multiplesymbolsperkey),keyboardswithtap-sliderfunctions,andkeyboardsthatofferaswipingmethodtoinserttext.Changingthekeyboardcanhaveaninfluenceonyourapp.Forexample,the

keyboardmaybemuchtallerthanthestandardoneandthereforehideUIelementsthatareimportanttointeractwith,orthescreenmightnotrecognizethatitneedstoscroll.Youmaycomeacrossimportantkeysthataremissingfromthekeyboard,thuspossiblyrenderingyourappunusableinsomecases.Itisalsopossibleforsettingstogetlostwithinyourphone,andyourappmightcausefreezesorcrashes.Thisisjustoneexampleofhowreplacedsystemappscanhaveanimpacton

yourapp.Ifyourappinteractswithoneofthepreinstalledsystemapps,keepinmindthatusersareabletoreplacethemwithotherapps.Youneedtohaveanoverviewofpopularappsthatreplacesystemappsinordertotesttheirintegrationandinteractionwithyourapp.

Internationalization(I18n)andLocalization(L10n)Anotherchallengethatneedstobehandledduringthemobiledevelopmentprocessistheinternationalization(I18n)andlocalization(L10n)ofmobileapps.I18nistheprocessofdesigningsoftwareapplicationsinsuchawaythattheycanbeadaptedtovariouslanguagesandgeographicalregionsthroughouttheworld,withoutchangingthecodebaseandwhilethesoftwareisrunning(some

appsandsomemobileplatformsrequirearestart).L10nistheprocessofadaptingtheinternationalizedsoftwaretoaspecific

languageorregionbyaddinglocale-specificelementsortranslatingthetext.It’sreallyimportanttotestbothI18nandL10nifyourappisdesignedtobe

usedindifferentcountries.YoumustbesurethatthedifferentlanguageswillnotbreakyourUIelementsorhaveaninfluenceontheapp’susability.

ImportantManylanguageshavetheirowncharacterset,andonewordcanhaveaverydifferentwidthandheight.

Havealookattheexamplewordlogout.IfyoucomparethewordwithtranslationsfromGermany,France,Turkey,andRussia,youwillseelotsofdifferenceincharacters,width,andheight:

•Logout(English)•Ausloggen(German)•Déconnexion(French)•Çıkışyap(Turkish)•(Russian)

BoththeGermanwordAusloggenandtheFrenchwordDéconnexionaremuchlongerthanLogout.TheTurkishversionevenconsistsoftwowords.Asasample,Asianlanguagescanbeusedas“short”languages,whileGermanandPortugueseareconsidered“long”languages.AlltranslationscanleadtoaUIglitchorevenbreakyourdesignrules.Whentestinganappindifferentlanguages,checkthateverytexttranslationwillfitintotheUIelementsandthateveryscreenhasthesamelook-and-feel.Alsocheckthatthedifferentcharactersandfonttypescanbestoredinthelocaldatabase.Thesameappliestodateformatsthatareusedwithinapps.Dependingonthe

countryorregioninquestion,differentdateformatsarealsoused.Youneedtocheckthatthecorrectformatisshowndependingontheregionsettingsofthephone.Youshouldalsotesttheparsingfromonedateformattoanother.Thefollowingdateformatsarecommonlyused:•DD/MM/YYYY(Day/Month/Year)•DD.MM.YYYY•DD-MM-YYYY

•MM/DD/YYYY(Month/Day/Year)•MM.DD.YYYY•MM-DD-YYYY•YYYY/MM/DD(Year/Month/Day)•YYYY.MM.DD•YYYY-MM-DD

Ifyourappisgoingtobeusedindifferentcountriesorregionsintheworld,makesureyouaddtheimportantlanguagesbasedonyourtargetcustomergroupinordertoprovideagooduserexperience.Whentestingtheapp,youneedtocheckitineverylanguage.Youalsoneedtocheckthatthelanguageswitchisworkingtobesurethatonlyasinglelanguageisshowndependingonthedevice’slanguagesettings.Alsocheckthefallbacklanguageofyourapp.Totestthis,changethedevicelanguagetoonethatisnotsupportedandcheckthattheappusestheimplementedfallbacklanguage.Languagesanddateformatsshouldneverbemixedupwithintheapp.Ifyouwanttotestiftheusedlanguageiscorrect,youshouldaskanativespeakerinordertoavoidusingmisleadingtranslationsortextthatissimplywrong.

ImportantThedifferentlanguagesanddateformatsmustbeconsideredduringthedesignphaseoftheappsothatdesignerscanplanthelook-and-feeloftheappinalloftheprovidedlanguages.Latechangesduetotranslationissuescandelaythereleasedateoftheappornegativelyaffecttheapp’sdesign.

MobileBrowsersIfyou’retestingamobileWebapp,youwillofcourseneedtodosoinamobileWebbrowser.MobileWebbrowsersareoptimizedtodisplayWebcontentsuchthattheycanbeaccessedonsmallerscreens.Unfortunately,thereismorethanonemobileWebbrowseravailableindifferentversionsofthedifferentmobileplatforms.Besidesthat,thebrowsersusedifferentlayoutenginessuchasthefollowing:

•Blink(www.chromium.org/blink)•Gecko(https://developer.mozilla.org/en-US/docs/Mozilla/Gecko)•Presto(www.opera.com/docs/specs/)

•Trident(http://msdn.microsoft.com/en-us/library/aa741312(v=vs.85).aspx)•WebKit(www.webkit.org/)

Dependingonthelayoutengine,browsersettings,andbrowserversion,mobileWebappsmaylookandbehaveverydifferently.Thisisespeciallythecaseforthedifferentbrowserlayoutengines.EverybrowserlayoutenginehandlesstandardslikeHTML,CSS,andJavaScriptdifferently.Noteverybrowserhasimplementedthecompletefeaturesetorthelatestversionofthevariouslanguages(HTML,CSS,JavaScript),whichcanleadtodifferencesinbehavior.TobesurethatyourmobileWebappworksondifferentbrowsers,testiton

differentmobileplatformssuchasAndroid,iOS,WindowsPhone,orBlackBerrytogetherwithdifferentbrowserversions.Thisisofcoursethesameproblemnativeappshave:fragmentation.However,testingmobileWebappscanbemoredifficultbecausenowyouhaveanothervariableaddedtoyourtestingmatrix:thedifferentWebbrowsers.Thismeansthatyouhavetofocusonthedifferentbrowserversions,mobileplatforms,andoperatingsystemversions.ThemobileWebbrowserslistedinTable3.2areavailableforthedifferent

mobileplatforms.Toseewhichbrowserlayoutengineisusedbywhichbrowserversion,checkthebrowservendorWebsites.

Table3.2OverviewofMobileBrowsers

ImportantNoteverybrowserisavailableoneveryplatform.Table3.2showsthemostusedmobileWebbrowsers.8Thetableisnotcomplete.

8.http://akamai.me/1EQZbP0

Asyoucansee,testingamobileWebappondifferentmobilebrowserscanalsobearealchallenge.TokeeptheamountoftestingworkneededformobileWebbrowserstoaminimum,youshouldcreatemobilebrowsergroupsoraddmobilebrowserrequirementstothedevicegroups.Ifyoucreatebrowsergroups,prioritizethembasedonyourtargetcustomergroupandonlytestyourWebapp

withinthesegroups.

SummaryChapter3wasoneofthefirsthands-onchaptersofthisbook.Youarenowabletocreateyourownpersonasbasedontheinformationaboutyourtargetcustomergroupandtheusageofyourapp.Thosepersonaswillhelpyoutofocusyourdevelopmentandtestingeffortsonyourtargetgroupandnotwastetimeonunnecessaryfeatures.Inadditiontopersonas,younowknowhowtohandledevicefragmentationin

themobileworld.Testingyourapponeverymobiledeviceisnotpossibleoreconomical.Basedonyourtargetgroup,youcandefineso-calledmobiledevicegroupstotestonlyonthedeviceswithineachgroup.Thiswillhelpyoutodownsizethetestingeffortdramatically.Anotherapproachtohandlingdevicefragmentationistousemobiledevicelabs.Asyouknow,mobiledevicesarepackedwithlotsofsensorsandinterfaces,

andifyourappusesthem,youneedtotestthemaswell.Testingthedifferentsensorsandinterfaceswasanotherbigtopicinthischapter.Foreachsensorandinterface,testingideaswereprovided.Whenyourappisavailableindifferentcountriesandsupportsdifferent

languages,thesection“Internationalization(I18n)andLocalization(L10n)”shouldhelpyouremembertotestyourappagainstthoselanguagesandsettingsrelatedtolocationsfromaroundtheworld.Thechapterclosedwithanoverviewofthedifferentmobilebrowsersthatare

currentlyavailableonthemarket.

Chapter4.HowtoTestMobileApps

Inthepreviousthreechaptersyoulearnedalotaboutthemobileappworld,rangingfromthedifferentnetworktypes,apptypes,businessmodels,appstores,customerexpectations,andchallengesformobiletesterstodevice-specifichardwareelements.Inthischapteryouwilllearnhowtotestmobileapps.Thisisahands-onchapter,andIsuggestthatyouhaveatleastonedevicewithanappofyourchoicenexttoyouwhilereadingit.Useyournewlyacquiredknowledgetotesttheappandseeifyoucanfind

bugsorotherdiscrepancies.

Emulator,Simulator,orRealDevice?Beforeyoustarttotestanapp,there’soneimportantquestiontoanswer:Areyougoingtotestitonarealdevice,inasimulator,orinanemulator?Mobiledeviceemulators,suchastheAndroidEmulator,1aredesktop

applicationsthattranslatetheinstructionsofthecompiledappsourcecodesothattheappcanbeexecutedonadesktopcomputer.Theemulatoractsexactlylikethemobiledevicehardwareandoperatingsystem,thusallowingthedeveloperandtestertodebugortesttheapplication.Sincetheappisexecutedonacomputer,notallofthemobile-specifichardwareelementssuchasthesensorsortouchgesturescanbeemulated.However,emulatorscanbeveryusefulatanearlystageofthedevelopmentprocessinordertoobtainquickfeedbackabouttheimplementedfeatures.

1.http://developer.android.com/tools/help/emulator.html

Simulators,suchastheiOSsimulator,2arelesscomplexsoftwareapplicationsthatsimulateasmallsubsetofthedevice’sbehaviorandhardware.Incontrasttoemulators,simulatorsareonlysimilartothetargetplatformandsimulatetherealdevice’shardware,makingthemmuchfasterthanemulators.Itisalsonotpossibletotestdevice-specifichardwareelementswithsimulators.However,simulatorsareusefulatanearlystageofthedevelopmentprocessinordertoobtainfeedbackabouttheimplementedfeatures.

2.https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/iOS_Simulator_Guide/Introduction/Introduction.html

Thebiggestdifferencebetweenasimulatorandanemulatoristhatasimulatorattemptstoduplicatethebehaviorofthemobiledevice,whileanemulatortriestoduplicatetheentireinnerarchitectureofthemobiledeviceandistherefore

closertothetargetplatform.Dependingonthemobileplatform,vendorsprovideeitherasimulatororanemulator.AppleandBlackBerry(ResearchinMotion)offerasimulator;GoogleandMicrosoftprovideanemulator.Asyouhavelearnedinthepreviouschapters,mobiletestingrequires

movementanddifferenthardware,meaningthatyouneedtotestyourapponphysicaldevicestobesurethateverythingworkstogetherinreal-lifesituations.Emulatorsandsimulatorsshouldbeusedonlyforverybasictestssuchas

simplefunctionality(isthebuttonclickable?)ortomakesurethelook-and-feeloftheappisOK.

ManualTestingversusAutomatedTestingThere’sanotherimportantdecisiontomake:Areyougoingtotesttheapppurelywithautomatedtests,purelywithmanualtests,orwithacombinationofboth?Thisdecisiondependsonyourapp.Simplyperformingtestautomationwillnotworkandisnotsufficientfor

severalreasons.Noteverymobile-device-specificfunctioncanbeautomated—forexample,locationdata—andotherenvironmentalsensordataisreallyhardtotestinalabsituation.Becauseoftheselimitations,yourappwillmostlikelycontainlotsofbugsandproblemsthatyourcustomerswillfind.Onlyperformingmanualtestingcanwork,butit’salsonotsufficient.You

shouldperformonlymanualtestsofyourappifitmeetsthefollowingcriteria:•Yourappisverysimpleandbasic.•Yourapphasonlyverylimitedfunctionality.•Yourappisavailableforonlyalimitedtimeintheappstores.

Inallothercasesyoushouldcombinemanualandautomatedtesting.Beforeperformingtestautomation,youshouldalwaysdomanualtesting.Everynewfeaturemustbemanuallytestedsystematicallyondifferentdevices.Onceyou’vecompletedmanualtesting,youcanthendefinethepartsoftheappthatrequiretestautomation.InChapter5,“MobileTestAutomationandTools,”Idescribethedifferent

mobiletestautomationconcepts,explainhowtoselectamobiletestautomationtool,andprovideyouwithanoverviewofpossibletools.

“Traditional”TestingMobileapplicationsaresoftwareapplications.Besidestestingthemobile-specificfunctionsandelements,itisstillnecessarytotestmobileappsinthe

samewayyoutestWebordesktopapplications.Youstillneedtodesignthetestcases,managethetestdata,andofcourserunthetests.IfyoulookatFigure4.1,youcanseethetypicalactions/stepsthatshouldbe

takenduringthesoftwarequalityassuranceprocess.Therearetwotypesofsoftwarequalityassurancemeasures:productfocusandprocessfocus.Theproductfocusphaseisusedtofindbugs,whereastheprocessfocusphaseaimstoprotectthesoftwarefrombugs.

Figure4.1Softwarequalityassurancemeasures

Theprocessfocusphaseofsoftwarequalityassurancecoversthemethods,languages,tools,guidelines,standards,andprocesseswithwhichthesoftwareisdeveloped.Inthisphase,softwaretestersareresponsibleformakingsure,forexample,thattheguidelines,developmentmethods,andtoolsaswellasthestandardsaremetduringthewholedevelopmentprocess.Theyshouldguideandhelpdevelopersandotherteammembersinfulfillingthedefinedprocessesinordertopreventanybugsfromarising.Theproductfocusphaseisdividedintostatictestinganddynamictesting.Instatictesting,thesoftwareisnotexecuted.Testersanddevelopersshould

performreviewsduringthisstageofthesoftwaredevelopmentprocess,eithercodereviewsbeforecommittingthecodetoarepositoryorreviewingthedocumentsandspecificationsbeforedevelopmentstarts.Duringthestaticphase,theapplicationcodewillbecheckedwithtoolstoseeifitmatchescoding

guidelinesorcontainsanymemoryleaksorraceconditions.ThefollowinglistcontainssomeJavastaticanalysistools:•Checkstyle(http://checkstyle.sourceforge.net/)•FindBugs(http://findbugs.sourceforge.net/)•Lint(http://tools.android.com/recent/lint)•PMD(http://pmd.sourceforge.net/)

ImportantTherearestaticanalysistoolsavailablefornearlyeveryprogramminglanguage,justhavealookforthemontheInternet.

Indynamictesting,theapplicationcodeisexecutedtoseehowtheapplicationbehaveswhilebeingrun.Dynamictestingisagainsplitintotwotypes:whiteboxtestingandblackboxtesting.Whiteboxtestingisawayoftestingsoftwarewithknowledgeoftheinternal

structureofmethodsandclasses.Whiteboxtestingisusuallydonebydeveloperswhotesttheircodeonaunitlevel,inmostcasesusingtoolssuchasJUnit,3TestNG,4XCTest,5orsimilarunittestingtools.

3.http://junit.org/4.http://testng.org/doc/index.html5.https://developer.apple.com/library/prerelease/ios/documentation/DeveloperTools/Conceptual/testing_with_xcode/Introduction/Introduction.html

Whiteboxtestingtechniquesincludethefollowing:•Statementcoverage•Pathcoverage•Branchcoverage•Decisioncoverage•Controlflowtesting•Dataflowtesting

Blackboxtestingisawayoftestingsoftwarewithoutanyknowledgeoftheinternalstructureofthemethodsorclasses.Thisapproachisusuallydonebysoftwaretesterswhoneedtoknowwhatthesoftwareshoulddoanddon’tneedtoknowhowthesoftwareactuallydoesit.Writingtestcasesandplanningtestingactivitiesareimportantpartsofblackboxtesting.Therearetestdesigntechniquesthatshouldbeconsideredinordertodeveloptherighttestcases,for

example,boundaryvalues,decisiontables,andequivalenceclasspartitioning.Boundaryvaluetestingwasdevelopedfromtheexperiencethatbugsoccuron

theboundariesandedgesofconditionsandlists.Thesebugsmaycausethesoftwaretocrash,freeze,orbehaveincorrectly.Decisiontableswillhelpyoutosplitcomplexapplicationsandconditionsinto

smallersections,whichpreventsyoufromforgettingimportantfeatures.Theyhelpyouobtainanoverviewofthecriticalelementsofthefeatureset.Equivalenceclasspartitioningisatechniqueusedtodefinetestcasesthat

uncovererrorclasses.Equivalenceclasspartitioninghelpsyoureducethenumberoftestcasesthatneedtobedevelopedduringyourtestingactivities.Thisapproachistypicallyusedtotestpossibleinputsoftheapplication.Moreexamplesofblackboxtestingtechniquesfollow:•Cause-effectgraph•Errorguessing•Statetransitionanalysis•All-pairstesting•Exploratorytesting

ImportantThissectiongivesjustaverybriefoverviewofpossibletestingmethodsandtechniques.Iroughlysummarizedthefoundationsofsoftwaretestingandanalysis,soifyou’renotfamiliarwiththedifferenttesttechniques,methods,andapproaches,IhighlyrecommendthatyoureadsomesoftwaretestingbooksortakeatestingclasssuchasISTQB6foundationlevelorBlackBoxSoftwareTesting.7

6.www.istqb.org/7.www.testingeducation.org/BBST/

Mobile-SpecificTestingInthispartofthechapteryouwilllearnhowtotestyourappindifferentcategoriesandensurethatitisrobust,stable,usable,andasfreeofbugsaspossible.PleaseusetheknowledgeyougainedinChapter3,“ChallengesinMobileTesting,”aswellwhentestingyourapp.Keepthesensors,networks,gestures,customers,languages,andsystemappsinmind.

Beforeyoucontinuereading,getyourmobiledeviceoutofyourpocketandstartanappsothatyoucantryoutthethingsyou’reabouttoread.

MobileFunctionalTestingTestingthefunctionalityofyourappshouldbethefirstthingyoudo.Executethedesignedtestcaseswithyourmobileapptomakesurethatthefeaturesandrequirementshavebeencorrectlyimplemented.Besidestestingyourtestcases,itisimportantthatyoualsotestthedefinedacceptancecriteria.Testtomakesurethatallthefunctionsperformastheyshould,forexample,

inputs,outputs,tappablebuttons,navigation,anddataprocessing.Testthefunctionalityindifferentmobileuserscenariosandenvironments.Keepthesensorsandinterfacesofamobiledeviceinmindwhiletestingthefunctionality(seeChapter3again).Furthermore,usethequalityassurancemeasurestotestthefunctionalityof

yourappinastaticanddynamicway.

ImportantTestingthefunctionalityisoneofthemostimportantaspectsofeverysoftwareproject.Dependingonyourapp,functionaltestingwillbedifferentineveryproject.

MobileUsabilityTestingUsability,especiallymobileusability,isahuge,complex,andimportanttopic.Withinyourmobiledevelopmentteamyoushouldhaveausabilityexpertwhocoversthispartofthedevelopmentprocess.Goodusabilityrequireslotsofrefining,intenseuserresearch,andevenmoretestingwithrealusers.Itisimportantthatyourappbeeasytouse;otherwiseitmightendupwithlowratings,whichcausedamagetotheapp’sandpossiblyyourcompany’sreputation.Toavoidsuchreputationdamage,mobiletesterscanperformtheactionsdescribedinthispartofthechapterinordertohelptheteamdevelopausableapp(keepKIFSUinmind).Thefollowinglinksareagoodstartingpointifyouwanttolearnmoreabout

mobileusability:•Usabilityheuristics(www.nngroup.com/articles/ten-usability-heuristics/)•Mobileusability(www.nngroup.com/articles/mobile-usability-update/)•GoogleBestPractices(www.google.com/think/multiscreen/#mobile-best-

practices)•Usabilityprinciplesandtechniques(www.usability.gov/what-and-why/index.html)•Mobileuserinterfaceandusabilitydesignprinciples(www.neobytesolutions.com/mobile-user-interface-and-usability-design-principles/)

Oryoucanrefertothefollowingbook:•MobileUsabilitybyRalucaBudiuandJakobNielsen(www.nngroup.com/books/mobile-usability/)

LessIsMoreWhentestingtheusabilityofanapp,thedesignprinciple“Lessismore”isagoodstartingpoint.Checktheappforuselessnavigationelementsandbuttons.Ifyoufindany,talktoyourteamortheusabilityexpertandgetthemremoved.Thesameappliestotextandwords.Removethemiftheyarenotrequiredtointeractwiththeapp.Trytofocusontheprimarycontentoftheapp.Keepthegoalandtheproblemtheappshouldsolveinmind,andgetridofanythingthatisdistracting.

Self-ExplanatoryCheckiftheappisself-explanatory.Askyourselfthesequestions:Istheuserabletoseetheelementswithwhichheorshecaninteract?Iseveryelementonthescreenclearandunderstandable?Thisaspectofusabilitytestinggetsmoredifficultovertimebecausethemoreyouworkwiththeapp,themoreblindyouwillbecometothingsthatarenotself-explanatory.Youwillmostlikelybeusedtopotentialusabilityissues.Inthatcaseitmakessensetoaskcolleaguesnotinvolvedintheappdevelopmentprocessoryourcustomersforearlyfeedback.

PayAttentiontoPatternsEveryappshouldfollowausabilityanddesignpattern.Ifsuchapatternisinplace,verifythatyourappfollowsthoserules.Ifthereisnopatternavailable,keepaneyeonthelook-and-feeloftheapp.Sametype-elementssuchasbuttonsortextlabelsshouldhavethesamespacings,sizes,andcolors.Checkthatalloftheelementsareaccessible,forexample,thatbuttonscanalsobetappedbyapersonwiththickerfingersandondifferentscreensizesanddensities.Alsocheckthedefaultpatternsthatareprovidedbythemobileplatformsasyourappshouldatleastfollowthosepatterns.

AllowMistakesInordertoprovideatrulyusableapp,yourappmustallowuserstomakemistakes.Ifauser,forexample,missesarequiredinputfield,providefriendlyandusefulerrormessages.Checkthattheerrormessagesarenottoogenericandthattheydescribetheerrorinawaythatpeoplewithoutatechnicalbackgroundcaneasilyunderstand.Furthermore,niceerrorhighlightingshouldbeinplacetoshowtheuserthemistakeheorshemade.Keepyourtargetcustomergroupinmindwhenverifyingerrormessagesasthiswillhaveanimpactonthewording.Besidesthat,yourappshouldprovideundoandgobackactionsinordertooffertheuseraneasywaytocorrecterrors.Anotherapproachmayinvolveuserinputrecommendationsorso-calledtype-aheadsearches,whichprovidetheuserwithpossibleinputsolutions;forexample,whenheorsheistyping“NewY,”theappwillsuggestNewYorkasapossibleinput.

CheckWorkflows,Details,andNavigationIfyourappfollowsaspecialworkflow,checkthatitfollowsthesameworkflowineverysection.Keepaneyeonthedetailsofthoseworkflows.Besurethatbuttons,labels,andotherelementsarebigorsmallenoughtobeused.Screendivisionisimportant,sochecktheapponsmallerandbiggerscreens.Don’tforgettochecktheappforimportantdetails.Aretheyprominentenough?Istheuserabletofindthem?Alsokeepaneyeonvisualtransitionssuchasanimationsorelementsthatfadeinorout.Isthetransitionofthoseelementssmoothordoesitfeelchoppy?Makesureyoualsouseslowdevicestocheckthatallofyouranimationsaresmoothenough.Checkthatthemainnavigationandsub-navigationoftheappareeasytouse.

Arethereanyunnecessarytapsthattheusermustperforminordertoreachhisorhergoal?Checkthatthenavigationhasnounnecessarysteps,andchecktoseeifsomeofthestepscanbecombined.Isthenavigationdoablewithonehand?Askpeoplewhohavedifferenthandsizesandareleft-orright-handedtoperformthistask.Havetheplatform-specificnavigationpatternsbeenmet?

ChecktheWordingVerifythatthetextusedwithintheappisclearandeasytounderstand.Askdifferentpeopleiftheyunderstandthewordingandthefeaturebehindit.Ifyourappcontainstext,itmustbefreeoftyposastheycanbeveryembarrassingandhaveanimpactonyourstorereviewsandratingsaswellasyourreputation.Ifyourappsupportsmorethanonelanguage,besurethetextfitsintoeveryUIelementandthatthetranslationiscorrect.Don’tforgettotestyourappfor

placeholdertextslike“Loremipsum”oranyotherdefaulttextsfromdevelopersordesigners.

CheckConsistencyAreallyimportanttaskwithinmobileusabilitytestingistocheckforconsis-tency.Verifythatyourappisconsistentineverysectionorview.AsIalreadymentioned,theUIelementsmusthavethesamelook-and-feel,thesametext,spacings,colors,andimages.Furthermore,itisreallyimportanttocheckthatyourappisconsistentoverallofthesupportedmobileplatformssuchasiOS,Android,WindowsPhone,orBlackBerry,notjustintermsofnavigationorpatterns(eachmobileplatformhasitsownrules),butbyensuringthatthetexts,colors,andimagesarethesame.MakesureyoualsochecktheconsistencyofanyofferedWebordesktopapplicationsthatprovideadifferentviewtoyourapplication.Allofyourapplicationsmustfollowthecorporateidentityguidelinesofyour

companyinordertolendafamiliarlook-and-feeltoyourproduct;otherwisetheusermaygetconfused.Askforyourcorporateidentityguidelines,whichwillincludetopicssuchascolors,fonts,logos,images,andtext.

WriteItDownWhiletestingyourapp,itisimportantthatyouwritedowneverythingthatfeelsdifferentorsomehowjumpsoutatyou,nomatterhowtrivialitmayseem.Detailedobservationsarereallyimportantandwillhelpimprovetheapp.Youcancreatescreenshotsoftheentirescreenusingthemobile-platform-specificbuttoncombinationorscreencapturevideoswithaWebcamwhileusingtheapptoshowthese“wrong”thingstoyourproductownerordevelopersothatyoucandiscussthem.Animportantpointis:don’tgiveupwritingdownsuchthings,evenifthe

productownerorthedeveloperrejectsyourfindingstimeandtimeagain.Fightforthosethingsandkeepraisingthem.However,dobecarefulwithyourbugremindersasitislikelythatyourcomplaintswillbeignoredifyouconstantlyraisethesameissues.Isuggestthatyoucollectandcategorizerejectedbugsandpresentthemasanimprovementstorytoyourproductmanagerordeveloperforoneofthenextreleases.Anotherapproachwouldbetoincludefixesofoldbugsineveryreleasetoimprovethemobileappandcleanupthebugpool.Talktoyourteamandfindthesolutionthatworksbestwithyourapp.Testersneedtobecuriousandtenacious,andyourcustomerswillloveyoufor

that!

AccessibilityTestingAsyouhavelearnedfromthepreviouschapters,lotsofpeoplearoundtheworldareusingmobiledevicesandmobileappsintheirdailylife.Appsareusedforawholerangeofdifferentreasons,suchasnavigation,hotelbookings,ticketpurchases,listeningtomusic,orplayinggames.Whendevelopingandtestinganapp,didyouthinkaboutpeoplewith

disabilities?Therearemanypeopleouttherewhoarevisually,audibly,physically,and/or

cognitivelyimpaired.Makingmobileappsaccessibleisnoteasyandit’salotofwork,butitdoesn’tjustbenefitpeoplewithdisabilities;italsoimprovesyourcompany’sreputationandincreasesyouruserbase.Itisthereforeveryimportantthateveryoneinvolvedinyourmobileapp

developmentprocessbeawareofaccessibilityandknowabouttheaccessibilityrequirementsofmobileapps.Therearefourtypesofimpairmentthatarerelevanttomobileappdevelopment:visual,auditory,physical,andcognitive.

VisualImpairmentPeoplewithvisualimpairmentmaybeblind,havelimitedvision,becolor-blind,orhaveacolordeficiency.Duetotheirvisualimpairment,theymaynotbeabletoseeallofthebuttons,labels,text,andotherelementsinyourappandarethereforeunabletoaccessorcontroltheappcorrectly.Toadaptyourmobileappforvisuallyimpairedpeople,youcanimplementthe

followingfeatures:•Makeallelementsavailableforscreenreaders.Nearlyeverymobileplatformofferstheoptiontouseabuilt-inscreenreadertoreadoutthecontentandtheelementsofthemobileapp.Iftherearenobuilt-inscreenreadersavailable,youcaninstallthemfromthird-partyvendors.•Everymobileplatformoffersabuilt-infeaturetoenlargethefontsizeoftheentirephoneandapps.Implementyourappinsuchawaythatthefontsizecanbeadjustedwhileretainingapleasantlayout.•Everymobileplatformoffersafeaturetochangetheforegroundorbackgroundcolorofthescreen,ortochangethecontrast.Therefore,youshouldusecolors,brightness,andcontrastoftheUIelementsthatworkwiththedifferentforegroundandbackgroundsettings.•Built-inscreenmagnifierscanbeusedtoenlargetheelementsonthescreentomakethemeasiertoread.

•Anotherwayofhelpingvisuallyimpairedpeopleisthroughtheuseofvoicerecognition.Thisallowsmobileuserstocontroltheentiremobileappthroughvoicecommands.

AuditoryImpairmentSomepeoplehavetroublehearing,meaningthattheydon’thearifsomeonecallsthemoriftheyreceiveaudionotifications,instructions,oranyotherkindofaudiocontent.Tohelppeoplewithhearingproblems,youcandothefollowing:•Usethebuilt-invibrationorvisualnotificationtoinformuserswhentheyreceiveacall,message,oranyothernotification.•Ifyourappreliesonvideocontent,usesubtitlestoinformtheuseraboutthecontentofthevideo.•Implementadjustablesoundorvolumecontrols.Thisisreallyimportantforpeoplewithahearingaid.•Ifyourappgeneratesanysounds,implementafeaturetoswitchthedevicetomonoaudio.Thiswillhelppeoplewhocanhearwithonlyoneear.

PhysicalImpairmentPhysicallyimpairedpeoplehaveproblemsperforminggestureswiththeirhands.Theyarenotabletousegesturessuchaspinch,tap,orswipetocontroltheelementsonthephoneorwithinthemobileapp.Tohelpthemuseyourapp,youcandothefollowingthings:•Usevoicerecognitionasthisallowsmobileuserstocontroltheentiremobileappthroughvoicecommands.•Implementafeaturetoincreaseordecreasethespeedofpressingbuttonsasthiswillmakeiteasierforphysicallyimpairedpeopletocontrolyourapp.

CognitiveImpairmentCognitivelyimpairedpeoplemayhaveproblemsrelatedtoattention,communication,memory,computation,andreadingskills.Thosepeopleexperienceproblems,forexample,withunderstandingcomplexnavigationsorinstructionsthattheyneedtofollowwithinanapplication.Tohelpthemuseyourapp,youcandothefollowingthings:•Implementaneasyandintuitiveuserinterface.Makeitassimpleas

possibletounderstandanduse.•Usescreenreadersincombinationwithsimultaneoustexthighlighting.•Implementanauto-textfeatureforinputfieldssothatpreloadedanddefinedtextelementsarealreadyinplace.•Implementyourappinsuchawaythatcognitivelyimpairedpeoplehaveenoughtimetocompleteanoperation.

AccessibilityGuidelinesAsyouhaveseen,accessibilitytestingisareallychallengingtask,andyouandyourteamneedtofindawaytosupportsuchfeatures.Ifyourappisaccessible,itisveryimportantthatyoutestitwithpeoplewhohavesuchimpairmentsinordertobesurethattherequirementshavebeencorrectlyimplemented.Providinganaccessibleappwillincreaseyouruserbaseandimproveyourreputation.TheW3Ccreatedanaccessibilityinitiativethatcoversmostofthe

accessibilityconcerns.Youcanfindinformationatwww.w3.org/WAI/orwww.w3.org/WAI/mobile/.Togetdetailedmobileaccessibilityinformationforthedifferentmobileplatforms,checkthemanufacturers’Webpages.TheguidelinesforAndroidcanbefoundhere:•Androidaccessibilityguide(http://developer.android.com/guide/topics/ui/accessibility/index.html)•Androidaccessibilitytesting(http://developer.android.com/tools/testing/testing_accessibility.html)

TheguidelinesforiOScanbefoundhere:•iOSaccessibilityintroduction(https://developer.apple.com/library/ios/documentation/UserExperience/Conceptual/iPhoneAccessibility/Introduction/Introduction.html•AccessibilityoniPhone(https://developer.apple.com/library/ios/documentation/UserExperience/Conceptual/iPhoneAccessibility/Accessibility_on_iPhone/Accessibility_on_iPhone.html•iOSaccessibilitytesting(https://developer.apple.com/library/ios/technotes/TestingAccessibilityOfiOSApps/TestingtheAccessibilityofiOSApps/TestingtheAccessibilityofiOSApps.html

TheguidelinesforWindowsPhonecanbefoundhere:•WindowsPhoneaccessibilityguide(www.windowsphone.com/en-us/how-to/wp8/settings-and-personalization/accessibility-on-my-phone)

TheguidelinesforBlackBerryarehere:•BlackBerryaccessibilityguide

(http://ca.blackberry.com/legal/accessibility.html)

BatteryUsageTestingWhiletestingamobileapp,youneedtotestbatteryconsumptionduringuse.Ifyourappconsumestoomuchpower,userswilldeleteitandmoveontoanotherone.Totestyourapp’sbatteryusage,youcanrunthroughthetwoscenarios

describedinthefollowingsections.

FullyChargedBatteryInthefirstscenario,youhaveafullychargedbattery.Afterinstallingandstartingtheapp,leaveitopenandputthedeviceinstandbymode(yourappisnowrunningintheforeground).Nowyouhavetowaitacoupleofminutes(orhours)toseewhethertheappisconsuminglotsofbattery.Checkthebatteryusagefromtimetotimeandnotethebatterylevel.Youshoulddothiswithdifferentdevicestogetbetterresults,anddon’tforgettocloseanyotherappsthatarerunningonthedevicetoeliminateanysideeffectstheymaycause.Thesametestshouldbedonewhentheappisrunninginthebackground.Toputtheappinthebackground,juststartitandthencloseit(bypressingthehomebuttonorclosebutton).Onmostmobileplatforms,theappwillnowruninthebackgroundormultitaskmode.Again,checkthedevicefromtimetotimetomonitortheapp’sbatteryusage.Usethesmartphonehardwarefeatures(ifusedwithinyourapp)suchasGPS

orothersensorsandcheckthebatterylevelofyourdevicewhileusingthosefeatures.It’simportantthatyourappswitchoffdevicefeaturesoncetheyarenolongerneeded.Ifyourappdoesn’tdothis,itwillconsumelotsofbattery.Checkthattheappisnotsendinganyunnecessaryrequeststoabackendsystemwhilerunninginthebackgroundorforeground.Unnecessaryrequestswillhaveanimpactonbatterylife.Tocheckthis,useaproxytoolsuchasCharles8orFiddler9tosendallyourrequestsandcommunicationsfromthedevicethroughyourworkstationtothebackendsystem.

8.www.charlesproxy.com/9.www.telerik.com/fiddler

Whileusingthedeviceandtheapp,checktoseeifthedevicegetshot.Insomecasesdevice-specificfeatureswillcausethedeviceandbatterytoheatup,whichwillhaveanimpactonyourdevicehardwareandmayultimatelydamageit.

Alsocheckthatyourappreallyisclosedafterremovingitfromthemultitaskthread.Toverifythis,openthemobile-specificappsystemsettingsandcheckthatyourappisnotcurrentlyrunningthere.

LowBatteryThesecondscenarioyoushouldtestiswhenthebatteryofthephoneisnearlyempty,whenithas10%to15%ofitscapacityleft.Inthisbatterystate,mostmobiledevicesswitchofffeaturessuchasnetworkconnections,GPS,andsensorsinordertoprolongthebatterylife.Ifyourappusesanyofthosefeatures,checkhowithandlesthislowbatterystate.Watchoutforfreezes,crashes,andperformanceissues.Onsomedevicesyoucanactivateapower-savingmodethatswitchesoff

sensorsorInternetconnections.Again,don’tforgettocheckhowyourapphandlesthispower-savingmode.Areallyimportanttestistoseehowyourapphandlesacompletelyempty

battery.Useyourappuntilthephonebatterydies.Thenpluginachargerandstartthephoneagain.Oncethephonehasbootedup,checkyourappfordatalossordatacorruption.Theappshouldworkasexpectedandnodatalossorcorruptionshouldoccur.Youshouldalsotestyourappinthetransitionfromagoodtoabadbattery

state.Mostdeviceswillpopupanalertinformingtheuserthatthebatteryisnearlyempty.Thistransitioncanhaveanimpactonyourapp,socheckitforfreezes,crashes,andperformanceissues.Thesameshouldbedonetheotherwayaround.Useyourappwhilethebatteryischargingasthiscanalsoimpacttheapp.

BatteryConsumptionToolsSomemobileplatformsofferabatteryusagestatistictoseewhichappisconsumingthemostbattery.Android,forexample,providessuchanoverviewtoseethecurrentstateofthebattery(seeFigure4.2).Usethistoolforyourbatterystatetesting;it’sreallyusefulformobiletesters!

Figure4.2Androidappbatteryusageoverview

Ifyouoryourdeveloperswanttoconstantlymeasurebatteryconsumptionduringyourproject,therearetoolsavailableforthat.ForAndroidthereisatoolcalledJouleUnit.10JouleUnitisanopen-sourceenergyprofilingtoolforAndroidappsthatfindsanyunnecessarybatteryusageofappswhentheyarerunning.ItalsomeasurestheusageoftheCPU,Wi-Fi,ordisplaybrightness.ThetoolbasicallyhasthesamestructureasJUnittestsandisreallyeasytointegratewithinyourdevelopmentenvironmentsoyoucangetearlyfeedbackaboutthebatteryconsumptionofyourapp.10.https://code.google.com/p/jouleunit/

iOSallowsyoutousetheenergyusagefunctionwithinInstruments11toprofiletheenergyusageofyourapp.InstrumentsispartoftheXcodedevelopmentenvironmentandisfreefortestersanddevelopers.11.

https://developer.apple.com/library/mac/documentation/developertools/conceptual/instrumentsuserguide/Introduction/Introduction.html

ThereareacoupleofusefuldocumentsonlineaboutenergyusagewithinInstruments:

•EnergyUsageInstrument(https://developer.apple.com/library/ios/documentation/AnalysisTools/Reference/Instruments_User_Reference/EnergyUsageInstrument/EnergyUsageInstrument.html

•LoggingenergyusageiniOS(https://developer.apple.com/library/ios/recipes/Instruments_help_articles/LoggingEnergyUsageinaniOSDevice/LoggingEnergyUsageinaniOSDevice.html

Batterytestingiseasytodoandshouldbepartofyourtoolchain.Combinethemanualwiththeprofilingapproachtogetgoodresults.

StressandInterruptTestingStressandinterrupttestingisanimportantpartofthemobiletestingprocess.Withtheaidoftools,mobiletestersareabletodetermineanypotentialperformanceorstabilityissuesexhibitedbyanapp.Totestyourappagainstinterrupts,youcanmanuallytriggerlotsofnotificationstothedevicewhileusingtheapp.Notificationscanbeincomingmessages,calls,appupdates,orpushnotifications(softwareinterrupts).Furthermore,pressingthevolumeupanddownbuttonsoranyotherkindofhardwarebuttonisalsoaninterrupt(hardwareinterrupt)thatcanalsohaveanimpactonyourapp.Doingallofthesetasksmanuallyisalotofworkandverytime-consuming.In

mostcases,thesetestscenarioscan’tbedonemanuallybecauseitisveryhardtosimulatefastandmultipleuserinputswithoneortwohands.Butitcanbedonewiththeaidoftools,anditisreallyeasytointegratethemintothedevelopmentandtestingprocess.ForAndroidapps,atoolcalledMonkey12canbeusedwhichispartofthe

AndroidSDK(SoftwareDevelopmentKit).Monkeycanrunoneitheraphysicaldeviceoranemulator.Whilerunning,itgeneratespseudo-randomusereventssuchasatouch,click,rotate,swipe,mute,Internetconnectionshutdown,andmuchmoretostress-testtheappandseehowithandlesallthoseinputsandinterrupts.12.http://developer.android.com/tools/help/monkey.html

ThepackagenameoftheAndroid.apkfileisneededtobeabletorunMonkey;otherwiseitwillexecuteitsrandomcommandstotheentirephoneinsteadofjusttheappundertest.Withaccesstotheappcode,thepackagenamecanbefoundinthe

AndroidManifest.xml.Ifonlythecompiled.apkfileisavailable,mobiletesterscanusetheAndroidAssetPackagingTool13(aapt)togetthepackagenamefromtheapp.aaptislocatedinthebuild-toolsfolderoftheinstalledAndroidSDKversion.13.http://elinux.org/Android_aapt

Thepathtoaaptmaylooklikethis:Clickheretoviewcodeimage

/../daniel/android/sdk/build-tools/android-4.4/

Withthefollowingcommand,thepackagenamecanbereadoutfromthe.apkfile:Clickheretoviewcodeimage

./aaptdbadging/daniel/myApp/myApp.apk|grep'pack'

...package:name='com.myApp'versionCode=''versionName=''...

Whenthepackagename(inthiscasecom.myApp)isavailable,executeMonkeywithadb14(AndroidDebugBridge):14.http://developer.android.com/tools/help/adb.html

Clickheretoviewcodeimage

./adbshellmonkey-pcom.myApp-v2000

Thenumber2000indicatesthenumberofrandomcommandsthatMonkeywillperform.Withanadditionalparameter-sforseed,Monkeywillgeneratethesamesequenceofeventsagain.ThisisreallyimportantforreproducingabugthatmayoccurwhenrunningMonkey.ForiOSappsthereisasimilartoolcalledUIAutoMonkey.15UIAutoMonkey

isalsoabletogeneratemultiplecommandstostress-testaniOSapp.TouseUIAutoMonkey,aUIAutomationInstrumentstemplatemustbeconfiguredwithinXcode.Afterthetemplateisconfigured,aJavaScriptfileneedstobewrittentotellthetoolhowmanyandwhichcommandsshouldbeexecutedduringthestresstestingsession(seeListing4.1).15.https://github.com/jonathanpenn/ui-auto-monkey

Listing4.1UIAutoMonkeyScript

Clickheretoviewcodeimage

config:{numberOfEvents:2000,delayBetweenEvents:0.05,//Inseconds

//EventsthatwillbetriggeredonthephoneeventWeights:{tap:30,drag:1,flick:1,orientation:1,clickVolumeUp:1,

clickVolumeDown:1,lock:1,pinchClose:10,pinchOpen:10,shake:1},

//ProbabilitythattoucheventswillhavethesedifferentpropertiestouchProbability:{multipleTaps:0.05,multipleTouches:0.05,longPress:0.05}},

Ifthescriptiswritten,itcanbeexecutedwithinXcode.Attheend,bothtoolsgenerateanoverviewofpossibleerrorsandproblemswithintheapp.

ImportantPleasevisitthetoolmanufacturer’sWebsiteforinstallationinstructions.

Asyoucansee,thesetoolsmakeitsimpletostress-andinterrupt-testamobileapplication.Besidesthat,usingthemisahugebenefitformobiletestersasithelpstheteambuildareliableandrobustmobileapp.Bytheway,it’susefultocombinebatterytestingwithstressandinterrupttestingtoseehowthebatteryisusedwhenlotsofinterruptsanduserinputsaretriggeredthroughouttheapp.

PerformanceTestingPerformancetestingisoneofthekeytestingareasineverysoftwaredevelopmentprojectandespeciallyformobileapps.IfyourememberthehighuserexpectationsdescribedinChapter1,“What’sSpecialaboutMobileTesting?,”mobileperformancetestingisareallyimportantandcriticalpartofappdevelopment.Mobileusersexpectanapptostart/loadwithintwoseconds;otherwisetheyareunhappyandmaydeleteit.Testersanddeveloperscanuseperformanceteststodiscoverpotential

bottlenecksintheirsoftwareapplication.Normally,performancetestingisdoneonserversorbackendsystemstocheckhowthesystemsorsoftwarecanhandlehugenumbersofrequests,tomeetacceptableresultsfortheusers.Performancetestsmustbeexecutedinadefinedstateoftheapplicationwith

hardwareresourcesthatareequaltothelivebackendenvironment.Thecollected

datamustthenbeanalyzedtofindpossiblebottlenecksandproblemsinthesoftware.Performancetestingisacomplextopicthatshouldneverbeunderestimatedorpostponedtotheendoftheproject;itshouldbedoneasearlyaspossible.Keyperformancefiguresshouldbepartoftherequirementsphaseofappdevelopmentinordertostartplanningrightfromthebeginning.Formobileapps,performancetestsaremorecomplexandneedtocovermore

systemsinordertogetmeaningfulresultsthatimproveperformance.Thetypicalmobileappreliesonabackendsystem(server)withwhichit

communicates.Theappsendsrequeststotheserver,whichthenhandlesthoserequestsandsendsbackaresponse.Tosendasimplerequesttoabackendsystem,therearethreecriticalperformanceareasthatneedtobecovered:

•Theserver•Themobiledatanetworks•Themobiledeviceandappitself

Totesttheperformanceofmobileapps,youneedtoperformance-testatleastthebackendsystemandtheappitself.Itisnotpossibletotesttheperformanceofthemobiledatanetworks,anditmakesnosenseforyouasamobiletester.Thenetworkspeedandlatencycanbesimulatedduringthetestingphase,butthenetworkspeedwillbetotallydifferentwhileauserisonthemoveinrealdatanetworks.Inthischapter,Iwanttofocusonperformancetestingofthemobileappitself.

KeepthebackendsystemperformanceinmindandlookforfurtherinformationontheInternet.Thereareplentyofgoodtoolsavailabletohelpyousetupaperformancetestingsuite.Ifyouwanttodoperformancetestingwithinyourprojectandusetoolstodoso,keepthefollowingstepsinmindastheywillhelpyoutodefineaclearstrategy:

1.Planperformancetestsearlyinthedevelopmentphase.2.Designtheperformancetestsforyourapp.3.Preparetestdataandinfrastructure(theseshouldbethesameasfortheliveenvironment).

4.Executetheperformancetests.5.Collect,analyze,andevaluatethedata.6.Findpotentialbottlenecksandrefinetheapp.7.Monitortheapplicationchangesagaintoseeiftherefinementsweregoodenough.

MobileAppPerformanceTestingWhentestingtheperformanceofamobileapp,keepitsimpleandfocusontheUIoftheapp.Forexample,useastopwatchandmeasuretheapplicationlaunchorthedelaysbetweencertainoperations.Measuretheloadingtimeofthecontentsuchasimages,text,oranimationsthatneedtobecreatedorprocessedbytheapp.Performthosetestsseveraltimesandnotetheperformedstepsinordertoreproduceandtrackpossibleperformanceissues.Also,writedownhowoftentheproblemoccurred.Ifithappensallthetime,that’sfine;thatisaneasyfix.Butsomeproblems,especiallyperformanceproblems,mayhappenonlythreeoutoftentimes.It’sthereforeimportanttofindthebugandreproducethebehavior.Whiletestingtheapp,writedowneverythingthatseemsslowasitcouldbea

potentialperformanceissue.Manualperformancetestingshouldbedoneonseveraldeviceswithdifferenthardwarespecificationsinordertogetmeaningfulresults.Anothertestthatshouldbedoneisacomparisonbetweenthecurrentapp

versionthatisliveandavailablefordownloadintheappstoresandthenewreleasecandidateapp.Comparebothversionsintermsofapplaunchtimeandallotherareas.Thenewreleasecandidateshouldnotbeslowerthanthecurrentversion;otherwisetheappwillgetbadfeedbackfromusers.

ImportantThecomparisonshouldbedoneonthesamehardware;otherwisetheresultswilldiffer.

Ifyourappcontainsthird-partyelementssuchasadvertisingornewsfeeds,checkthatthoseelementshavenoimpactontheperformanceoftheapp.Asanexample,youcanuseaWebproxytoollikeFiddlertosendthethird-partyrequesttoatime-outtoseethatithasnoimpactontheapp’sperformance.Anotherwaytotesttheperformanceoftheappistoprofileandmeasurethe

processandoperationtimeofthecode.Thereareplentyofprofilertoolsavailabletochecktheappcodeforpotentialbottlenecksandperformanceissues.Thistaskshouldbedonebythedevelopers,soencouragethemtousesuchtools.Tosummarizethesimplemobileappperformancetests:•Measuretheapp’slaunchtime.•Checkfordelaysduringoperationsoruserinteractions.

•Measuretheloadingtimeofthecontent.•Noteeverythingthatseemsslow.•Testondifferenthardware,especiallyonslowerphones.•Comparetheliveappversionwiththenewreleasecandidate.•Checkforthird-partyelements.•Useprofilingtoolstomeasuretheprocessandoperationtimeofmethodsandclasses.

Asyoucansee,thesearesimplestepstotesttheperformanceofamobileapp.Ifyouwantmorecomplexanddetaileddataabouttheperformanceoftheapp,thebackend,andthedatanetworks,you’llneedtouseaperformancetestingtoolorsolutionthatcoversallparts.Thereareplentyofmobileappperformanceprovidersavailable,sosearchforthemontheInternetandseewhichonebestfitsyourdevelopmentandtestenvironment.

StandbyTestingStandbytestingisreallyeasyandsimpletodobutcanshowsomenicecrashes,freezes,andUIglitcheswithintheapp.Whiletheappisrunning,putthedeviceintostandbymodebypressingtheoffbuttononce.Dependingontheappundertest,wakeupthedeviceafteracoupleofseconds,minutes,orhourstoseehowtheappreactsuponwake-up.Mostappsfetchdataupdatesfromthebackendsystemafterawake-uptorefreshthecurrentUI.It’spossibleforabugtopreventtheappfromdisplayingthenewlyfetcheddatacorrectly,oritmayfreezeorcrash.IfaUIviewupdatemechanismisimplemented,youmustmakesurethismechanismworksafterthedevicewakesupandfetchesthelatestdata.AlsotestthescenarioofnothavinganInternetconnectionwhiletheappisin

standbymode.Totestthat,opentheapp,closethedevice’sInternetconnection,andputthedeviceintostandby.Afteracertainamountoftime,wakeupthedeviceandcheckthebehavioroftheapp.Itwillprobablycheckforupdates,butthereisnoInternetconnection.Inthiscase,theappmustshowapropererrormessagetotheuser,informinghimorherofthecurrentsituation.Don’tforgettocheckthatthedeviceisabletocommunicatewithdevice-

specifichardwareelementsafterwakingup.Feelfreetogetcreativewhentestingtheappforstandbyandwake-upproblems.

InstallationTestingTheinstallationprocessisthefirstimpressionamobilecustomerhasofanapp.

Iftheinstallationfailsduetoerrorsorproblems,thecustomerwillnottrytodownloadtheappagainandwillmoveontoanotherone.Toavoidsuchproblems,installationtestingmustbemadepartofthemobiletestingprocessandshouldbeperformedatleastbeforeanewversionisdueforrelease.Totestyourappforinstallationproblems,performthefollowingtasks:•Verifythattheappcanbeinstalledsuccessfullyonthelocalstorageormemorycardofthedevice.•CheckthattheinstallationworkswithdifferentInternetconnectionssuchasWi-Fiormobiledatanetworks.•ChangetheInternetconnection(Wi-Fito3G,forexample)whiletheappisinstalling.•Switchtootherappswhiletheappisinstalling.•Switchthedevice’sInternetconnectionoff,forexample,toairplanemode,whiletheappisinstalling.•Trytoinstalltheappwhenthereisnotenoughspaceleftonthelocalstorage.•Trytoinstalltheappviadatacableorsyncingfrommobile-specificsoftwareapplications.

Whenperformingthoseactions,watchoutforerrormessagesintheappaswellascrashesandfreezes.Whentheappsuccessfullyinstalls,mobiletestersshouldalsotesttheother

wayaroundbytestingtheuninstallprocess.Souninstalltheappandcheckthatitiscompletelyremovedfromthedevicewithnodataleftonthehardwareorlocalstorage.Toverifythattheappwasremovedcompletely,checkthedevice’smemoryandfoldersforleftoverdata.Anotherwaytoverifythattheappwasremovedsuccessfullyistoinstalltheappagainandtocheckthat,say,auserisnotloggedinautomaticallyorthattheappdoesn’tshowanydatafromthepreviousinstallation.Thesetestsarereallyimportantbecausesomedevicesaresharedwithinacompanyorfamilyandleftoverdatacanleadtoserioustrouble.Somemobileplatformsofferdifferentwaysofuninstallinganapp,soyou

havetotestallofthem.Whileuninstallingtheapp,watchoutforerrormessages,crashes,andfreezes.

UpdateTestingAsyouhavelearnedintheprevioussections,testinganappduringtheinstallanduninstallprocessesisreallyimportant.Besidesinstallinganduninstallingan

app,usersarealsoabletoupdatetheappfromoneversiontoanother.Duringthisupdateprocess,lotsofthingscangowrongandneedtobetestedbeforeanewappversionissubmittedtotheappstores.Testingtheupdateprocessincludesscenariossuchasthefollowing:•Logged-inusersshouldnotbeloggedoutfromtheappaftertheupdateisinstalled.•Theupdatewillnotaffectthelocaldatabase;thatis,existingdatawillnotbemodifiedordeleted.•Theappisinthesamestateasbeforetheupdatewasinstalled.•Testingtheupdateprocesswillsimulatetheupdateprocessintheappstores.

Youshouldalsotesttheupdateprocessfromafarolderappversiontothelatestonetoseewhathappenstotheapp.Whiledoingupdatetesting,keepaneyeoutforerrormessages,crashes,freezes,andperformanceissuesrightaftertheupdate.Inthefollowingtwosections,Iwilldescribehowtoperformupdatetestingon

iOSandAndroid.Nevertheless,ifyouaretestingaWindowsPhone,BlackBerry,oranyotherkindofapp,youshouldofcoursedoupdatetestingaswell.

iOSUpdateTestingTherearetwowaysofsimulatingtheupdateprocessforiOSapps.ThefirstonecanbeperformedwithiTunesbyfollowingthesesteps:

1.BuildanadhocversionoftheappthatiscurrentlyliveintheAppleAppStore.HINT:Thisversionmusthavethesamebundleidentifier(packagenameorstructureoftheappclasses)asthenewapp.

2.BesurethatnoolderversionoftheappisinstalledwithiniTunesandonthetestdevice(syncwithiTunestobesure).

3.Dragtheappfromstep1intoiTunesandsynctheversiontoyourtestdevice.

4.Launchtheappanddosomemanualtestingtomakesurethattheappisworking.

5.Buildthenewreleasecandidateversionoftheapp,dragittoiTunes,andsyncittothedevice.iTunesshouldconfirmthattheolderversionwillberemoved.

HINT:Donotdeletetheoldbuild!InthenextstepiTuneswillinstallthenewappovertheoldoneandsimulatetheupdatefromtheAppStore.

6.LaunchthenewversionoftheappandcheckthateverythingisOK.ThesecondwayoftestingtheupdateprocessforiOSistousetheApple

Configurator.16Thistoolisfareasiertouse,especiallyifyouwanttotesttheupdateprocedureonseveraliOStestdevicessuchasaniPhone4(S),iPhone5(S),oriPad.16.http://help.apple.com/configurator/mac/1.7/?lang=en

1.BuildanadhocversionoftheappthatiscurrentlyliveintheAppleAppStore.HINT:Thisversionmusthavethesamebundleidentifierasthenewapp.

2.Besurethatnoolderversionoftheappisinstalledonthetestdevice.3.Usethetooltoinstalltheappfromstep1tothedevicesforwhichyouwanttochecktheupdateprocess.

4.Launchtheappanddosomemanualtestingtomakesurethattheappisworking.

5.Buildthenewreleasecandidateversionoftheappandinstallitwiththetool.Theupdateprocesswillbesimulated.

6.LaunchthenewversionoftheappandcheckthateverythingisOK.Forfurtherinformation,pleasereadthetechnicalnoteTN228517fromApple

abouttestingupdatesoniOSdevices.17.

https://developer.apple.com/library/ios/technotes/tn2285/_index.html#//apple_ref/doc/uid/DTS40011323

AndroidUpdateTestingThesameupdatetestingcanbedonewithAndroidapps.TotesttheupdateprocessforAndroidapps,youcanusetheadbtoollocatedintheAndroidSDKfolder:

1.InstallthecurrentGooglePlaystoreversionoftheapptothetestdevice:Clickheretoviewcodeimage

./adbinstallRELEASED_APP_NAME.apk

2.Checkthatthisversionworks.3.BuildareleasecandidateoftheAndroidapp.

HINT:BesuretosignthereleasecandidatewiththePlaystorekeystore.4.Usethefollowingcommandtoinstallthenewversionoftheappandtotesttheupdateprocedure:

Clickheretoviewcodeimage

./adbinstall-rNEW_VERSION_APP.apk

Theoption-rmeanstheappwillbereinstalledandretainitsdataonthephone.

5.Thenewversionoftheappisnowinstalledandcanbetested.Asyoucansee,updatetestingisveryeasytodo.It’sreallyimportantto

performthesechecksbeforesubmittinganapptothevariousappstores.

DatabaseTestingAppsuselocaldatabases,inmostcasesaSQLite18database,topersistdataonthephone.Storingthedataorthecontentofanappinalocaldatabaseenablesmobileappstopresentthecontentwhenthedeviceisoffline.ThisisahugeadvantagecomparedtomobileWebappsthatrelyonastableInternetconnectioninordertoworkproperly.Thefactthatmobileappsusedatabasesmeansthatmobiletestersneedtotestthemandtheactionsthatwillbeexecutedonthedatabase.18.www.sqlite.org/

Localdatabasetestingcanbeperformedmanuallyorautomated.Thegoalistotestdataintegritywhileediting,deleting,ormodifyingthedata.Inordertoachievegooddatabasetesting,youneedtoknowthedatabasemodelwiththetablenames,procedures,triggers,andfunctions.Withthehelpofdatabasetools,youcanconnecttothedevicedatabasetotestandverifythedata.Thefollowingtypesoftestsshouldformpartofyourmobiledatabasetesting:•Databasevalidationtesting•Databaseintegrationtesting•Databaseperformancetesting•Procedureandfunctiontesting•Triggertesting•CRUD(Create/Read/Update/Delete)operationstestingtomakesuretheywillworkonthedatabase•TestingthatthedatabasechangesareshowncorrectlyontheUIoftheapp

•Searchandindexingfunctiontesting•Databasesecuritytesting•Testingthedatabaseformigrations

Becausedatabasesanddatabasetestingarehugetopicsinthemselves,Irecommendfurtherreadingtogathermoreinformationaboutthedifferentdatabasetechnologies.The“BooksaboutSQLite”19pagecontainslotsofusefulbooksonthistopic.19.www.sqlite.org/books.html

LocalStorageTestingLocalstoragetestinghasnothingtodowiththeappdatabase;instead,youshouldcheckhowtheappreactsindifferentstatesofthedevice’slocalstorage.Everydevicehasacertainamountofstoragecapacityformusic,images,apps,andanyotherkindofdatathatcanbestoredonthedevice.Somedeviceshaveonlyasinglecentralandpermanentlyinstalledlocalstorage,andusersarenotabletoextendthosedeviceswithadditionalstorage.However,therearemanydevicesthatofferthepossibilitytoextendthelocalstorage,forexample,withamicroSDcard.Whentestingamobileapp,youshouldtesttheapptogetherwithdifferent

statesofthelocalstoragetobesurethattheappcanhandlethemproperly.Thefollowingscenariosshouldbetested:

•Testtheappwhenthelocalstorageisfull.•Testtheappwhenthelocalstorageisfullbuttheextendedstoragestillhassomespaceleft.•Testtheappwhenthelocalstoragehassomespaceleftbuttheextendedstorageisfull.•Testtheappwhenbothstorageareasarefull.•Testtheappwhenthelocalstorageisnearlyfull.Performsomeactionswithintheappthatwillwritelotsofdatatothelocalstorageinordertofillitup.•Removethedevice’sextendedstorageandcheckthebehavioroftheapp.•Ifpossible,movetheapptotheextendedstorageandcheckthebehavior.•Testtheappwhenthelocalstorageisempty.

Whenperformingthesescenarios,watchoutforappcrashes,errormessages,appfreezes,performanceissues,UIglitches,andanyotherkindofstrange

behaviorthatcouldindicateaproblem.

SecurityTestingSecuritycanbebusinesscritical—forexample,whenattackersstealyourcustomerdata—thusmakingitaveryimportantpartofthedevelopmentandtestingprocessofyourmobileapp.Securitytestingisacomplextopicthatrequiresknowledgeinmanydifferentareas,suchasclient-servercommunication,softwarearchitecture,andsystemarchitecture.Becauseofitscomplexnatureandthespecializedskillsetrequired,securitytestingisbestdonebyexperts.Itincludesmethodssuchasmanualorautomatedpenetrationtestingwithman-in-the-middleattacks,fuzzing,scanning,andauditingthesoftware.Penetrationtestingisanapproachusedtofindsecurityweaknessesinan

applicationthatallowaccesstoitsfeaturesanddata.Thereareseveralopen-sourceandenterprisetoolsonthemarketthatcantesttheapplicationforcommonvulnerabilities.AlistofcommonsecuritytestingtoolscanbefoundontheOWASP(OpenWebApplicationSecurityProject)page.2020.www.owasp.org/index.php/Appendix_A:_Testing_Tools

Withtheaidofaproxytool,anattackercanchangeormonitorcommunicationbetweenaclientandaserver.Thisallowstheattackertoreadsensitivedatasuchasusernamesandpasswordsormanipulatethebehavioroftheclientbysendingforgeddatatoit.Thismethodisknownasaman-in-the-middleattack.Fuzzingisamethodtogenerateandsendrandomdatatoasysteminorderto

seehowthesystemreactstothedatainpreparationforapossibleattack.Thankstoscanningtools,thesystemcanbecheckedforknownvulnerabilitiesthatcanbeexploitedinordertogainaccess.Auditsaremostlikelyperformedbycertifiedsecurityproviders.Withthehelp

oftheexternalpartners,theapplicationwillbecheckedmanuallyandautomaticallyforvulnerabilitiesandpossibleattacks.Recallthatthedefaultmobileapparchitectureincludesthefollowing:•Themobileappitself•Themobiledatanetworks•Thebackendsystem

Youhavethreepossibleareasthatneedtobetestedforsecurity.Itwillbenighonimpossibletotestthedatanetworksforsecurity,butthedatanetwork

providersalreadydothistoensureasafeandsecurecommunicationenvironment.Therefore,twoareasremainforsecuritytesting:theappandthebackend.Tosimplifythings,Iwanttofocusonmobileappsecuritytesting.

CommonAppSecurityMistakesThefollowinglistcontainsthemostcommonsecurityproblemsofmobileapps:

ImportantThesepointscanbeusedduringthedevelopmentphasetocheckforcommonmistakes.However,lookingforthesecommonmistakesisnotareplacementforexpertsecuritytestinglikethatmentionedearlier.

•Cachestorage:Sensitivedatasuchaspasswordsortokensiscachedonthedevice.•Unintendedlocalstorage:Sensitivedatasuchaspasswords,tokens,orcreditcarddetailsisstoredbyaccident.•Encryption:Sensitivedatasuchaspasswordsisnotencryptedonthedevicestorage.•Client-sidevalidation:Passwordverificationisperformedonlyontheclientside.•Unencryptedcommunication:Communicationfromtheapptothebackendsystemsisnotencrypted.•Unnecessaryapppermissions:Appsusepermissionsfordevicefeaturesthattheydonotneedoruse.

Toavoidthesemistakes,youshouldkeeptheminmindandtestforthem.Toverifythecachestorageofthedevice,testyourappforinputsanddatathatarenotstoredonthedevice.Checkthedevicecachestoragefordatayoujustenteredandverifythatthedataiscachedforonlyacertainamountoftime.Toverifythecachestorage,useanappthatisabletolookatthefilesystemlevelandcheckthatthecachefolderofyourappcontainsonlythepermitteddata.Turnthedeviceoffandonagainandcheckthedeviceandappcacheasbothmustbeemptyinordertoensurethatthereisnosensitivedataleftonthedevice.Thesameappliestodatathatisstoredonthedevicebyaccident.Checkthedevicestorageandfilesfordatathatisnotallowedtobeonthe

device.Toverifythelocalstorageofthedevice,useappsordevelopertoolsthat

grantaccesstothelocalstorage.Ifyourappusesalocaldatabasetopersistthecontentordatasuchasthelogin

credentialsofyourapp,besurethatsuchdataisencryptedinthelocaldatabase.Checkthataformofencryptionisinplaceatthedatabaselevel.Ifyourappusesalogintograntaccesstothefeaturesandcontentdelivered

fromabackendsystem,besurethattheuserisnotvalidatedontheclientside.Validationshouldalwaysbeperformedonthebackendsystem.Ifthedevicegetslostandvalidationiscarriedoutonlyontheclientside,itisveryeasyforattackerstochangethevalidation,tomanipulateorstealsensitivedata.Mostappsrelyonabackendsystemfortheusertosendorreceive

informationanddata,toreadthelatestnews,tocommunicatewithfriendsinsocialnetworks,ortosende-mails.Ifthecommunicationbetweentheappandthebackendcontainssensitive,unencrypteddata,itmakessensetoencryptit,suchaswithTLS(TransportLayerSecurity).Ifanappwantstouseadevice-specificfeaturesuchasthecameraorwantsto

synchronizecontactswiththeaddressbook,specialpermissiontoaccessthoseelementsisrequired.Whendevelopingandtestinganapp,keepaneyeonthosepermissions.Useonlythepermissionsyourappreallyneeds.Otherwisetheusermaynotuseyourapp,becauseheorsheisskepticalaboutitormayfeellikeheorsheisbeingwatched.Theuseofunnecessarypermissionscanalsobeavulnerabilityfortheappandthedatastoredonthedevice.Ifyoudon’tknowthepermissionsofyourapp,talktoyourdevelopersaboutitandaskcriticalquestionsaboutthepermissionsandtheirpurpose.

SecurityRequirementsAnalysisThesecurityrequirementsanalysisshouldbepartofthefirstrequirementsanalysisphaseofeachmobileappproject.Youshouldraisethetopicasearlyaspossibletoavoidproblemsattheendoftheproject.Thefollowinglistcanhelpyouwiththesecurityrequirementsanalysis:

•Identifythepossibleuserrolesaswellastheirlimitationsandpermissionswithinthearchitecture(appandbackend).•Doestheuserrolehaveanimpactontheexistingsecurity,forexample,thebackend?•Isanexternalauditrequired?Whatshouldbepartofthisaudit?•Whatkindofsecuritytestingapproachesandtoolsarerequiredinordertoachieveagoodsecuritylevel?

•Dowehaveenoughskillsforsecuritytesting?Thislistisbynomeanscomplete.Dependingontheappanditscomplexity,thislistcanbemuchlonger.Agoodsourceofvaluableinformationaboutmobilesecurityisthe“OWASP

MobileSecurityProject”page.21Theprojectcollectsinformationaboutmobilesecuritysuchasthefollowing:21.www.owasp.org/index.php/OWASP_Mobile_Security_Project

•MobileTools(www.owasp.org/index.php/OWASP_Mobile_Security_Project#tab=Mobile_Tools•MobileSecurityTesting(www.owasp.org/index.php/OWASP_Mobile_Security_Project#tab=Mobile_Security_Testing•MobileCheatSheet(www.owasp.org/index.php/OWASP_Mobile_Security_Project#tab=Mobile_Cheat_Sheet

Thisinformationcanbeveryusefulduringboththesecurityrequirementsanalysisphaseandtheactualsecuritytestingphase.AnotherinterestingandusefulpartoftheOWASPprojectisthelistof“Top

10MobileRisks.”22Thelistincludesthefollowingtopics,withagoodexplanationofeachtopicandpossiblesolutions:22.www.owasp.org/index.php/OWASP_Mobile_Security_Project#tab=Top_10_Mobile_Risks

1.WeakServerSideControls2.InsecureDataStorage3.InsufficientTransportLayerProtection4.UnintendedDataLeakage5.PoorAuthorizationandAuthentication6.BrokenCryptography7.ClientSideInjection8.SecurityDecisionsviaUntrustedInputs9.ImproperSessionHandling10.LackofBinaryProtection

MobileSecurityTestingSummarySecuritytestingisacomplexanddifficultpartofthesoftwaredevelopmentandtestingprocess.Itrequiresspecialskillsandknowledgeoftechnologytoensurethatthesoftwareorappissecureagainstattacksandvulnerabilities.Youasa

mobiletestershouldalwayskeepsecuritytestinginmind.Talkaboutthetopicasearlyaspossibleintherequirementsanalysisphase.Ifyou’renotsurewhetheryoucantestthesecurityoftheappsufficiently,youshouldgetsomehelpfromexpertsasearlyaspossibleintheprocess.Keepthefollowingpointsinmindtoensureasecuremobileapp:•Testtheappfordifferentinputs;forexample,haveapreparedlistofpossibleattackstringsavailable.•Dopenetrationtestingontheappandthebackendsystem.•Useaproxy,afuzzer,andascannertoverifyeachpartoftheappandthebackendarchitecture.•Checktheappforcommonmistakes.•HavealookattheOWASPMobileSecurityProjectandfollowtheguidelinessetoutthere.•Checktheapp’scertifications.•Keepup-to-datewiththelatestmobiletechnologiesandsecuritynews.•Hireexperts.

ImportantThesecuritytopicscoveredinthischapterareaveryroughoverviewofpossiblethingstoconsider.Mobilesecurityisahugeandcomplextopic,sopleasescourtheInternetforfurtherinformation.

PlatformGuidelineTestingAnothertopicyouneedtobeawareofisthemobile-platform-specificguidelines.Eachapphastofollowsuchguidelinesinordertomeetthedesign,usability,andplatform-specificpatterns.Iftheappfailstofollowtheseguidelines,itmightberejectedfromtheappstore.Inaddition,followingthedesignprincipleswillmakeyourcustomershappybecausetheyknowhowtouseplatform-specificfeaturessuchasswipingfromlefttorighttoswitchviews,ortopulltorefreshandupdatethecurrentview.Itisimportantthatsuchguidelinesbeincludedinthedesignphaseofanapp.

Followingtheguidelinesfromtheverybeginningoftheprojectwillsaveyoutimeattheendbecauseyouwillencounterfewerbugsthanyouwouldbynotfollowingtheguidelines.

Whentestingamobileapp,keeptheplatform-specificguidelinesinmind.Youshouldknowthembyheartoratleastknowwheretofindthem:

•AppleiOSdesignresources(https://developer.apple.com/library/ios/design/index.html#//apple_ref/doc/uid/TP40013289•AppleiOSapprovalguidelines(https://developer.apple.com/appstore/resources/approval/guidelines.html•Androidguidelines(https://developer.android.com/design/patterns/index.html)•WindowsPhoneguidelines(http://dev.windowsphone.com/en-us/design)•BlackBerryguidelines(http://developer.blackberry.com/design/bb10/)

ImportantMakesureyou’refamiliarwiththelatestmobileplatformguidelines.Theychangewitheverynewmobileoperatingsystemversionthat’sreleased.

ConformanceTestingConformancetestingaimstoensurethatthesoftwaremeetsadefinedsetofstandards.ThesestandardsaredefinedbyindependententitiessuchastheEuropeanTelecommunicationsStandardsInstitute23(ETSI),theWorldWideWebConsortium(W3C),24andtheInstituteofElectricalandElectronicsEngineers(IEEE).2523.www.etsi.org/24.www.w3.org/25.www.ieee.org/index.html

Ifyourapphastoimplementdefinedstandards,youmustverifythatthosestandardsaremet.Itisimportantforyoutobeawareofthosestandardsandthatyouknowhowtocheckthem.However,thestandardsshouldalreadyformpartofyourrequirementsanalysisphase.Whenitcomestothedevelopmentphase,talktoyourdevelopersaboutimplementingthestandardsbyaskingquestions,performingacodereview,andwalkingthroughthedefinedstandardsdocumentation.Dependingonthemobileapptypeandpurpose,anexternalinstitutionmaybe

requiredtoverifyyourappinordertocheckthatyoucorrectlyimplementedtheirstandards.Thiswillbethecasewithmedicalappsorappsthatareusedbygovernments.Insomecases,theapphastobetestedforlegalcompliance.

ImportantKeepconformancetestinginmindandaskquestionsattheverybeginningoftheproject.

CheckingtheLogFilesWhiletestingyourmobileapp,connectyourdevicetoyourcomputerandchecktheapp’slogfileswhileusingit.Togetaccesstothelogfiles,youneedtoinstallthedevelopertoolsonyourcomputersoyoucanruntheappindebugmode.Whencheckingthelogfiles,watchoutforerrors,warnings,orexceptionsthat

occurwhileusingtheapp.Savetheinformationfromthelogfilealongwithstepsforhowtoreproducetheproblemandpasstheinformationontothedevelopers.Logfileinformationisextremelyusefultodevelopersforfindingandsolvingproblems.Besideslookingforerrorsandcrashes,youshouldlookforsensitivedatasuchastokensorpasswordsthatarevisibleinthelogs.Whentestingyourappwithoutacableconnectiontoyourcomputer,andthe

appcrashesorexhibitsstrangebehavior,leavetheappandthedeviceinthisstateandconnectitassoonaspossibletoyourcomputerandthedevelopertoolssoyoucancheckthelogfilefortheerror.Beforesubmittinganapptotheappstores,youhavetocheckthelogfilesfor

debuginformation,warnings,anderrorsagain.Debuginformationaswellaswarningsanderrorsshouldnotbepartofthereleaseversionoftheapp.Alsokeepthelogfilesofthebackendsystemsinmindastheylogdifferent

kindsofinformationsuchasrequestsandresponsesaswell.

ImportantSomebugsarevisibleonlyintheapp’slogfiles.Suchbugsmaynotbeshowntotheuserbutcouldhaveahugeimpactonthefunctionalityoftheapp.

BeCreative!Upuntilthispoint,thisbookhaspresentedasystematicapproachtomobiletestinganditsspecifics.Buteveryonewhohaseffectivelytestedsoftwareknowsthatmostofthetime,thereallynastybugsdon’toccursystematically.Bugscomeindifferentshapes,sizes,andsituations,andsometimesit’sreallyhardtofindthem.

Afteryouhavetestedyourmobileappinasystematicway(orbeforestartingwiththesystematicapproach),itisusefultothinkoutofthebox.Leanback,stepawayfromallthetestcasesandsystematicapproaches,andjusttrytobreakit!Testtheappforalimitedtimeframe—30minutes,forexample—andtrytofindsomebugsusingyourowncreativeapproaches.Becreativewithyourtesting.Trytothinkofedgecasesthataremostlikelyto

happenintherealworldwhenlotsofpeoplewillbeusingthemobileappinatotallydifferentwayfromwhatyouandyourteamexpect.Docrazythingswiththeappandthedevice.Rotateorflipthedevice,use

morethanonehandtointeractwiththeapp,andpressbuttonsasfastaspossible.Whiledoingthat,watchoutforanyunexpectedbehavior,crashes,freezes,

errormessagesthatmakenosense,andanyotherstrangethings.Isuggestthatyourecordthiskindoftestsessionbecauseitisverylikelythatanybugsthatoccurwillbehardtoreproduce.Anotherapproachcanbepairtesting,whichinvolvesonepersontestingtheappwhileanotherpersonwatchesandtakesnotesabouttheperformedtests.

Checklists,Mnemonics,andMindMapsThispartofthechapterisallaboutreminders.Yourdailytestingbusinessisstressfulandtherearesomanythingstoremember,do,andexplore.Youmaywanttousesomemodelsortoolstopreventyoufromforgettingimportantaspectsofyourwork.Threeveryniceapproachesyoucandrawuponare

•Checklists•Mnemonics•Mindmaps

ChecklistsEverynewfeaturethatwillbedevelopedneedstobetested.Softwaretestersusuallydefinetestcasesinordertosystematicallytesteverynewfeatureandavoidforgettinganything.Whenthetestcasesarefinished,asoftwaretesterusuallyprioritizesthemanualtestcasesforthetestautomationtoextendtheregressiontestsuiteandavoidmanuallytestingthenewfeatureoverandoveragain.However,noteveryfeatureortestcasecanbeautomatedbecausethetestmaybetoocomplexandwillmostlikelyendupcausinglotsofmaintenancework.Butwhatcanyoudoifsuchatestorfeatureiscriticalfortheapporthe

business?Neverexecutingitagainissimplynotanoption.Tohelpyouavoidforgettingimportantthings,youmayfinditusefultowrite

achecklistwithpartsoftheappthatneedtobemanuallytestedagainbeforegoingliveorafteraprojectmilestonehasbeenreached.Checklistscanbeverygenericandhighlevelsothattheycanbeusedin

severaldifferentprojects,ortheycanbereallylowlevelwithspecificstepstotestacertainfeature.ThemobiletestingchecklistinTable4.1isagenericonethatcoverslotsof

importantaspectsofthemobileapptestingprocess.Thislistcanbeusedforseveraldifferentmobileapps.

Table4.1GenericMobileTestingChecklist

ImportantThislistisbynomeanscomplete;itismerelyanexampleofwhatamobiletestingchecklistcouldlooklike.

Thischecklistshowssomepossiblegenericmobileteststhatcouldbeexecutedontopofyoursystematictestingapproach.Itmayalsobeusefultocreateareleasechecklistforthewholeteamsotheydon’tmissanythingimportanttotheprocess.However,checklistscanbereallyspecifictocoveraveryspecialpartoftheapp,suchasthepaymentandbillingprocesses,orthedifferentsensorsandinterfaces.

ImportantBecreativewithchecklists.Createyourowncheckliststhatarespecifictoyourappandproject.

MnemonicsSFDPOT,FCCCUTSVIDS,andISLICEDUPFUNarenottypos.They’reabbreviationsformnemonics.Mnemonicsarelearningtechniquesthataidinformationretention.Whenitcomestomobileapptesting,eachletterstandsforatestingapproachortechniqueusedforsoftwareapplications.Mnemonicsareusedtorememberimportantaspectswhiletestingsoftwareapplicationsandareveryusefulforsoftwaretestersbecausetheyareeasytorememberandincludepowerfulideasandbestpracticesonhowtotestsoftware.SFDPOT26isamnemonicfromKarenNicoleJohnsonthatsheadaptedfrom

JamesBach’smnemonicSFDPO27(betterknownas“SanFranciscoDepot”)onmobiletesting.Karenincludedthefollowingpoints:26.http://karennicolejohnson.com/2012/05/applying-the-sfdpot-heuristic-to-mobile-testing/27.www.satisfice.com/articles/sfdpo.shtml

•Structure:Testwhatit’smadeof.•CanIdownloadtheapp?•CanIdownloadanupdate?

•Function:Testwhatitdoes.•Doestheapporsiteperformthetasksorfeaturesitwasdesignedto?•Doestheapporsiteprevent,block,ornotincludefeaturesnotintendedonamobiledevice?

•Data:Testwhatitdoesitto.•Doestheappfindtime-relateddatabasedonthedevicetime?•Doestheappfindlocationsbasedonmylocation(suchasmovietheatersorhotels)?

•Platform:Testwhatitdependsupon.•Doestheappuselocationservices?•Doestheappdependonanydevicesettings?

•Operations:Testhowit’sused.•DoestheappfunctionwhenIammovingandtravelingaround?•WhathappenswhenIswitchtoWi-Fiversus3G?

•Time:Testhowit’saffectedbytime.•Whathappensifthetimezoneisswitched?•Whathappenswhenmylocationisswitched?

Foreachpart,Karenwrotesomequestionstoaskwhiletestingamobileapp.Checkoutthefulllistofquestionsinherblogpost“ApplyingtheSFDPOTHeuristictoMobileTesting.”Useandadaptthemtotestyourapp.AnothermnemonicisFCCCUTSVIDS28byMikeKelly,whonoteddown

hisideasintourstoexploreandtestapplicationsinseveralways:28.http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html

•Featuretour•Exploretheapplicationandgettoknowallthecontrolelementsandfeatures.

•Complexitytour•Findthemostcomplexpartsoftheapplication.

•Claimstour•Findalltheinformationaboutthesoftwarethattellsyouwhattheproductdoes.

•Configurationtour•Findallthewaystochangethesettingsoftheapplication.

•Usertour•Imaginehowthepossibleusersoftheapplicationwillusethesoftwareandwhattheyexpectfromit.

•Testabilitytour•Findallfeatureswithinthesoftwarethatcouldbetestedwiththehelpoftools.

•Scenariotour•Imaginepossiblescenariosastohowthesoftwarewillbeusedbyitsusers.

•Variabilitytour•Trytofindwaystochangetheapplication.

•Interoperabilitytour•Doesthesoftwareinteractwithothersoftware?

•Datatour

•Findthedataelementsoftheapplication.•Structuretour•Gatherasmuchinformationaspossibleabouttheapplication,suchastheprogramminglanguage,APIs,hardware,andsoon.

Thedescribedtoursareanexcellentwayoffindingoutpossiblesettings,features,orconfigurationsofthesoftwareapplication.Besidesthat,thetourscovertheuser’sperspectiveandareidealforexploringtheapplication.Allofthementionedtourscaneasilybeadaptedtomobileapps.AnothergreatmobiletestingmnemonicisISLICEDUPFUN29byJonathan

Kohl,whoalsoadaptedhismnemonicfromJamesBach’sSFDPOforusewithmobileapps.Hecoveredveryspecificmobileareasthathavetobetestedduringthedevelopmentphase:29.www.kohl.ca/articles/ISLICEDUPFUN.pdf

•Inputsintothedevice•Built-inkeyboard/keypad•Touchscreengesturesandtyping•Syncingwithotherdevices

•Store•Submissionspecifications•Developmentguide•Userguideforerrorhandling,locationservices,permissionsforuserprivacyitems,accessibility,andsoon

•Location•Geolocationerrors•Movement,stoppingsuddenly•Connectionissuesduetointerference

•Interactions/interruptions•Runningmultipleapplications,multitasking•Usingotherapplications,thenusingtheapplicationyouaretesting(e-mail,calendar,texting,notetaking,others)•Notificationsappearing

•Communication•Phone

•Texting•E-mails

•Ergonomics•Smallscreenscanbehardontheeyes.•Asmalldevicemeansthereisnoergonomichelpfromadeskorchair—youoftenhunchovertointeractwithit.•It’snotuncommontogetasoreback,fingers,oreyeswhenusingadeviceforanylengthoftime.

•Data•Typesofinput—seeiftheappusesspecialcharacters,differentlanguages,andsoon.•Media—seeiftheappdependsonanoutsidesourcetoplaymusic,videos,oranythingelse.•Sizeoffiles—iftheapplicationusesoutsidefiles,tryusingdifferentfiletypes.

•Usability•Noteandloganyissuesthatmakeyouuncomfortable,frustrated,angry,orupsetwhileusingtheapp.

•Platform•Android•iOS•WindowsPhone•BlackBerry

•Function•Canyouidentifyeverythingthattheapplicationdoes?•Haveyouworkedthroughalltheaspectsoftheapp?Clickedeverybutton?Filledineveryform?•Tryatouroftheproducttoidentifyeverythingitdoes.

•Userscenarios•Howisthisapplicationsupposedtobeused?•Whatproblemsdoesitsolveforusers?•Whatarethegoalsofendusersthatthisapplicationhelpsthemsolve?

•Network

•Wi-Fi•Wirelessbroadband•Deadspots

ISLICEDUPFUNisaverygoodmnemonic,andmobiletestersshouldknowitinordertoimprovetheirtestingworkandtoremembertheimportantmobileaspects.Besideshismnemonic,JonathanwroteanexcellentbookaboutmobiletestingcalledTapintoMobileApplicationTesting,30whichisavailableonLeanpubandcoverslotsofmobiletestingtopics.Thetourchapterisreallyinterestingandanexcellentsourceofknowledge.30.https://leanpub.com/testmobileapps

ThelastmnemonicaboutmobiletestingIwanttomentionisCOPFLUNGGUN,31createdbythecompanyMoolya.Thismnemoniccoversthefollowingtopicsandisworthlookingat:31.http://moolya.com/blogs/2012/04/121/Test-Mobile-applications-with-COP-who-FLUNG-GUN

•Communication•Orientation•Platform•Function•Location•Userscenarios•Notifications•Guidelines•Gesture•Updates•Network

ImportantMnemonicsaregreatapproachesyoucanusetoremindyourselfofimportantthings.Theywillhelpyouorganizeyourtestingapproach,testingstrategy,andthoughts.

MindMapsMindmapsareanothergreatwaytovisuallyorganizeinformationastheycanhelpyoutoprocessyourthoughtsandideasoncertaintopicssuchasmobile

testing.RosieSherry,thewomanbehindMinistryofTesting,32aplatformandprofessionalcommunityforsoftwaretesterswholeadthewayinsoftwaretestingcontent,training,andevents,createdandpublishedtwogreatmindmapsaboutmobiletesting.OneofhermindmapsisshowninFigures4.3through4.8.

Figure4.3Mobiletestingmindmap.CourtesyofRosieSherryandKarenNicoleJohnson.Foralargerversionofthisimage,see

informit.com/title/9780134191713

Figure4.4Mobiletestingmindmap:mobileplatforms.CourtesyofRosieSherryandKarenNicoleJohnson

Figure4.5Mobiletestingmindmap:mobileoperations.CourtesyofRosieSherryandKarenNicoleJohnson

Figure4.6Mobiletestingmindmap:mobilefunctions.CourtesyofRosieSherryandKarenNicoleJohnson

Figure4.7Mobiletestingmindmap:mobilestructure.CourtesyofRosieSherryandKarenNicoleJohnson

Figure4.8Mobiletestingmindmap:mobiledata.CourtesyofRosieSherryandKarenNicoleJohnson

32.www.ministryoftesting.com/

BothmindmapscanbedownloadedinhighresolutionfromMinistryofTesting133andMinistryofTesting2.34ThemindmapsRosiecreatedarebasedonthemnemonicfromKarenNicoleJohnson.33.www.ministryoftesting.com/2012/06/getting-started-with-mobile-testing-a-mindmap/34.www.ministryoftesting.com/2012/05/mobile-testing-course-pictures-and-a-mindmap/

AnothergreatinteractivemindmapprojectistheTestingMap,35whichcoverslotsofdifferentareasofsoftwaretestingandprovidessomereallyusefulideas.35.http://thetestingmap.org/

ImportantTrytocreateyourownmindmapbasedonyourappbyaddingpossibletestingtasks.Furthermore,printitoutandputitupintheoffice.Thatway,yourcolleagueswillstandtobenefitfromitaswell.

HowtoFileMobileBugsWe’venowcometothelasttopicinthischapter,“HowtoFileMobileBugs.”Ifyoufindabugwithinamobileapp,youneedtoreportitinordertogetitfixed.Filingmobilebugreportsrequiressomeadditionalinformationthatthe

developersneedinordertoreproduceandfixthebug.Butwhatisimportantwhenfilingamobilebug?Whatshouldabugreport

looklike?BeforeIanswerthosetwoquestions,Iwanttoraiseanotherone:Whyevensendabugreport?Bugreportsareveryimportantfortheproductowner,productmanager,and

thedevelopers.First,abugreporttellsthedevelopersandtheproductowneraboutissuestheywerenotawareof.Italsohelpstoidentifypossiblenewfeaturesnoonehasthoughtof,andlastbutnotleast,itprovidesusefulinformationabouthowacustomermayusethesoftware.Allofthisinformationcanbeusedtoimproveyoursoftware.Wheneveryoufindsomethingstrange,ifsomethingbehavesdifferentlyor

looksweird,don’thesitatetofileabugreport.Let’scometothequestionofwhatabugreportshouldlooklikeandwhat’s

importantwhenfilingit.Abugreportshouldcontainasmuchinformationaspossibleinorderto

identify,reproduce,andfixthebug.Thatsaid,yourreportshouldonlyincludeinformationthat’simportanttohandlingthebug,sotrytoavoidaddinganyuselessinformation.Anotherimportantpointisthatyoushoulddescribeonlyoneerrorperbugreport.Don’tcombine,group,orcreatecontainersforbugs.It’slikelythatnotallofthebugswillbefixedatthesametime,sorefrainfromcombiningorgroupingthem.Theinformationdescribedinthefollowingsectionsshouldbeincludedina

bugreport.

BugIDAbugmusthaveauniqueidentifiersuchasanumberoracombinationofcharactersandnumbers.Ifyou’reusingadefectmanagementtool,thetoolwillhandlethebugIDsforyou.Ifnot,thinkaboutauniqueIDsystemforyourproject.

•Bad:123isauniqueID,butyoumighthaveseveralprojectswhoseIDsarethesame.•Good:AppXYZ-123isgoodbecauseyou’recombininganIDwithaprojectabbreviationandanumber.

DescriptionCreateashortbutmeaningfuldescriptioninordertoprovidethedeveloperwith

aquickoverviewofwhatwentwrongwithoutgoingintodetail.Youshould,forexample,includeerrorcodesorthepartoftheapplicationwherethebugoccurred.

•Bad:“Theappcrashed,”“Whitepage,”“Sawanerror,”“Bug”•Good:“ErrorCode542ondetailmessageview”or“Time-outwhensendingasearchrequest”

StepstoReproduceThisisoneofthemostimportantpoints.Providetheexactstepstogetherwiththeinputdataonhowtoreproducethebug.Ifyouareabletoprovidethiskindofinformation,thebugwillbeveryeasytofixinmostcases.

•Bad:“Itriedtoexecuteasearch.”•Good:“Starttheappandenter‘MobileTesting’intothesearchinputfield.Pressthesearchbuttonandyou’llseetheerrorcode783onthesearchresultpageheader.”

ExpectedResultInthissectionyoushoulddescribewhatyouexpectedtohappenwhenthebugoccurred.

•Bad:“Itshouldwork”or“Ididn’texpectittocrash.”•Good:“Iexpectedtoseeasearchresultspagewithascrollablelistof20entries.”

ActualResultWhathappenedwhenthebugoccurred?Writedowntheactualresult,whatwentwrong,ortheerrorthatwasreturned.

•Bad:“Itjustwon’twork.”•Good:“Thesearchresultspagewasempty”or“Igottheerrorcode567onthesearchresultspage.”

Work-aroundIfyou’vefoundawaytocontinueusingtheappandavoidthebug,explainyoursteps.Thosestepsareimportanttoknowsincethework-aroundcouldcauseotherproblemsorindicateawayinwhichtheappshouldnotbeused.Ontheotherhand,awork-aroundcanbeveryusefulforthecustomersupportteaminordertohelpcustomerssolvethecurrentproblemuntilthebuggetsfixed.

•Bad:“Ifoundawork-around.”•Good:“Ifyouputthedeviceintolandscapemode,thesearchbuttonisenabledandtheusercansearchagain.”

ReproducibleIfyoufoundareproduciblebug,that’sfine,butdoesitoccureverytime?Ifithappenseverytime,that’sgreatasitshouldbeaneasyfixforthedeveloper.Butifthebugoccursonly20%ofthetime,itismuchhardertofindasolution.Makesureyouprovidethisinformation,asitisveryusefulforthedeveloperandwillpreventthebugfrombeingclosedwiththecomment“Can’tbereproduced.”

•Bad:“Sometimes”•Good:“Thebugoccurstwooutoftentimes.”

OperatingSystem,MobilePlatform,andMobileDeviceAnotherimportantcomponentofabugreportisinformationabouttheoperatingsystem,themobileplatform,andthemobiledevice.Writedowntheoperatingsystem,mobileplatform,anddeviceonwhichthebugoccurred.

•Bad:“OnAndroid”or“OniOS”•Good:“Android,Version4.1.2GoogleNexus4”or“iOS,Version6.1iPhone4S”

Mobile-Device-SpecificInformationMobiledeviceshavelotsofinterfacesandsensorsthatcouldhaveanimpactonyourapp.Thebatterycouldalsoaffecttheappyou’retesting.Writedownallofthisinformationinyourbugreport.

•Bad:Noinformation•Good:“GPSsensoractivated,changedtheorientationfromlandscapetoportraitmode”or“Usedthedeviceinasunnyplace”or“Batterystatewas15%”or“Batterystatewas100%.”

BrowserVersionIfyourappisamobileWebappandyoufoundanissue,it’sveryimportanttonotethebrowserversionwhereyoufoundthebugasitmayoccuronlyinacertainbrowserversion.

•Bad:“GoogleChrome”or“MozillaFirefox”

•Good:“GoogleChromeversion45.35626”or“MozillaFirefox27.6”

SoftwareBuildVersionAnotherreallyusefulpieceofinformationisthecurrentbuildversionoftheappwherethebugoccurred.Thiswillpreventthedeveloperfromwastingtimetryingtoreproduceabugthat’salreadybeenfixedinthecurrentcodebase.

•Bad:Noinformation•Good:“Appbuildversion1.2.3”

NetworkConditionandEnvironmentWhenfilingamobilebug,it’simportanttoprovidesomeinformationaboutthenetworkconditionandtheenvironmentinwhichthebugoccurred.Thiswillhelptoidentifytheproblemmoreeasilyandwillpossiblyshowsomesideeffectsnoonehasthoughtof.

•Bad:Noinformationor“Happenedonmywaytowork”•Good:“Iwasconnectedtoa3GnetworkwhileIwaswalkingthroughthecitycenter.”

LanguageIfyourappsupportsseverallanguages,providethisinformationinyourbugreport.

•Bad:Noinformation•Good:“IwasusingtheGerman-languageversionoftheapp.”

TestDataAcompletebugreportmustincludethetestdatathatwasusedtoreproducethebug.Simpletestdatacanbelogincredentialswithusernameandpassword.However,insomecasesitisnotenoughtoprovidejustausernameandpassword.Itisverylikelythatyouneedtoprovidecompletetestdatasets,forexample,asanSQLdumporatestdatascriptthatwillgeneratethenecessarydata.

•Bad:Noinformation•Good:“FindtheattachedSQLscripttoputthedatabaseinthedefinedstate”or“Enter‘MobileTesting’intothesearchinputfield.”

Severity

Everybugyoufindneedsaseveritylevel.Eitheryourdefectmanagementtoolwillofferyousomecategories,oryouhavetodefinethemwithyourteam.Itisimportanttogiveabugaseveritylevelasitwillallowtheteamtoprioritizetheirbug-fixingtimesothatcriticalandhigh-prioritybugswillbefixedfirst.Ifthisinformationisnotprovided,ittakesmuchmoretimetofindthebugsthatneedtobefixedbeforetherelease.ThedefaultseveritiesareCritical,High,Medium,andLow.

•Bad:Noinformation•Good:“Critical”or“Medium”

BugCategoryInadditiontotheseveritylevel,thebugcategoryisalsoaveryusefulpieceofinformation.Theproductownerorthedevelopercanfilterbycategorytogetanoverviewofthecurrentstatusofbugspercategory.Forexample,iftherearelotsofUXbugs,thismaybeanindicatorofapoorUIandUXoramissingdesignexpertintheteam,meaningthattheappneedssomemoredesignimprovements.

•Bad:“Noinformation”•Good:“Functionality”or“UX”or“Performance”

ScreenshotorVideoWheneveryoufindabug,trytocreatescreenshotsoravideotoprovidethedeveloperwithmoreinformation.Whenprovidingascreenshot,useanimage-editingtooltomarkthebuginthescreenshot.Avideoisalsoagreatwaytodescribeabugyou’vecomeacross.Itisalsoveryusefultogivethescreenshotorthevideoagoodnameordescription.

•Bad:“Noscreenshotsorvideosattached”or“Screenshot1.png”•Good:“01_InsertSearchTerm.png,02_SearchResultPageWithError.png”

LogFilesIfyourappcrashesorfreezes,connectthedevicetoyourcomputerandreadoutthelogfiles.Inmostcasesastacktracewillbeshownwithadescriptionoftheerror.Thiskindofinformationisextremelyusefulfordevelopers,astheyknowrightawayinwhichclassthebugortheerrorhasoccurred.

•Bad:“Noinformationprovidedwhentheappcrashed.”•Good:“Providedthefullstacktraceinthebugreport”or“Attachedthelogfiletothereport.”

TesterWhoFoundtheBugWritedownyournameorthenameofthetesterwhofoundthebug.Developersorproductownersmayhavesomequestionsaboutthereportedbug,andtheywillofcoursewanttogetintouchdirectlywiththetesterwhofoundit.Inmostcases,thisisautomaticallydonebythedefectmanagementsystemwhereeachuserhashisorherownaccount.Ifnot,makesureyouaddyoure-mailaddressand/orphonenumber.

•Bad:Noinformation•Good:“DanielKnott,[email protected]”

ThreeMorePointsAsyouhaveseen,thereisalotofinformationthatshouldbeincludedinabugreport.Therearethreeotherpointsyoushouldkeepinmindwhenwritingabugreport.ThefirstoneisDon’tgetpersonal.Whenfilingabugreport,describethe

softwaremisbehaviorratherthanthedeveloper’smindsetorthequalityofhisorherwork.Don’tuseoffensiveoremotionallychargedwordsasthosekindsofreportswillbeignoredbythedeveloperandwillendupcausingbadbloodwithintheteam.ThesecondoneisIt’snotyou.It’snotyourfaultthatthebugoccurred.Itis

thesoftwarethat’sbroken,andyouandyourcolleaguesneedtofixit.AndthethirdpointisKeepitsimple.Trytowriteyourbugreportinsucha

waythatsomeonewithnoideaabouttheprojectortheappisabletounderstandtheproblem.Ifthebugreportisthateasy,everydeveloperwithintheteamwillbeabletofixitandnontechnicalcolleaguescanunderstandtheproblemandwillvalueyourwork.

AppQualityAllianceIfyouwanttogetfurtherinformationabouthowtotestyourAndroidoriOSapp,havealookatthenonprofitgroupAppQualityAlliancesite.36ThisgroupisheadedbydifferentcoremembersandknowledgecontributorssuchasAT&T,LGE,Microsoft,Motorola,Oracle,Samsung,andSonyMobile.Themainaimofthegroupistoworkverycloselywiththeindustrytoimprovethequalityofmobileapps.36.www.appqualityalliance.org/

ThegroupdevisedtestingcriteriaforAndroidandiOSapps.Youcan

downloadbothPDFdocumentsviathefollowinglinks:•TestingcriteriaforAndroidapps(www.appqualityalliance.org/AQuA-test-criteria-for-android-apps)•TestingcriteriaforiOSapps(www.appqualityalliance.org/AQuA-test-criteria-for-iOS-apps)

Bothdocumentscontainlotsoftestcasesthatyourappshouldrunthroughbeforeyousubmitittotheappstores.Thegroupalsoprovidesinformationaboutmobileappperformancetestingas

wellasbest-practiceguidelinesfordevelopingqualitymobileapps.Youcandownloadbothdocumentsviathefollowinglinks:

•Performancetestingcriteria(www.appqualityalliance.org/aqua-performance-test-criteria)•Best-practiceguidelinesfordevelopingqualitymobileapps(www.appqualityalliance.org/AQuA-best-practice-guidelines)

Thegroup’sWebsiteisdefinitelyworthcheckingout,andmaybeyou’devenliketocontribute.

SummaryChapter4isoneofthemainchaptersofthisbookandcontainslotsoftestingideasandsolutionsthatcanbeusedinyourdailyworkinglife.Thechapterstartedwithadescriptionofthedifferencesamongemulators,simulators,andrealdevicesandwhatisimportanttoknowaboutthem.Beforethehands-onpartofthechapterIexplainedthedifferencebetweenmanualandautomatedtestingaswellastheroleoftraditionaltestinginthemobiletestingbusiness.Ifyouarefamiliarwith“traditional”softwaretestingsuchasforWebordesktopapplications,thatknowledgeisalsousefulformobileapps.LotsofthetestingapproachesforWebordesktopapplicationsarealsovalidformobileappssuchaswhiteboxandblackboxtestingtechniques.However,whenitcomestomobiletesting,therearemoreareasyouneedtotestinordertobesurethatyourappisworkingondifferentlevelsandwithdifferenthardwareandsoftwareversions.InthischapterIdescribedthefollowingmobile-relatedtopics:

•Mobileusabilitytesting•Accessibilitytesting•Batteryusagetesting•Stressandinterrupttesting•Performancetesting

•Standbytesting•Installationtesting•Updatetesting•Databasetesting•Localstoragetesting•Securitytesting•Platformguidelinetesting•Conformancetesting•Checkingthelogfiles

Ialsoprovidedchecklistsofsampletestcasesthatcanbeexecuted,suchasbeforetheappissubmittedtoanappstore.Furthermore,Ishoweddifferentmnemonicsandmindmapsthatcanbeusefulforyourmobiletestingactivities.Thechapterendedwithadetailedoverviewofhowtofileamobilebugwith

lotsofdo’sanddon’ts.

Chapter5.MobileTestAutomationandTools

Thischapterisallabouttools.Idescribethedifferenttypesandconceptsofmobiletestautomationtoolsandgiveyousomeadviceonwheretoautomateandhowtoselectamobiletestautomationtoolforyourappanddevelopmentenvironment.Ialsoexplaincontinuousintegrationandbetadistributiontools.Thischapterdoesn’tcoverhowtoinstallandconfigurethedifferentmobile

testautomationtools.Thereasonforthisissimple:Thetoolsprovideinstallationandconfigurationguidesalready,soyoucanjustvisittheirWebsitestofindalltheinformationyouneed.Detailsabouteachtoolwillbeprovidedintherelevantsectionsofthischapter.

TheFlippedTestingPyramidBeforeIstartwiththemobiletestautomationtools,Iwanttobrieflyexplainthetestautomationpyramid.AnyonewhoisinvolvedinsoftwaretestingandsoftwaretestautomationshouldknowthetestautomationpyramidintroducedbyMikeCohn.1

1.www.mountaingoatsoftware.com/

AsyoucanseeinFigure5.1,thetypicalpyramidconsistsofthreelayers.Atthebottomthereistheautomatedunittestinglayer;inthemiddle,theautomatedintegrationtestinglayer;andatthetop,thereistheautomatedend-to-end(E2E)testinglayer(includingtheuserinterfacetests).Eachlayerhasadifferentsize,indicatingthenumberofteststhatshouldbewrittenwithineachstage.Manualtestingisnotpartofthetestpyramid;henceitisshownasacloudforadditionaltestingwork.

Figure5.1Defaulttestautomationpyramid.BasedonafigurebyMikeCohn.

Butthispyramidisnotapplicabletomobileappsandmobiletestautomation.Asyoulearnedseveralchaptersago,mobiletestingrequiresatotallydifferentsetoftests—movement,sensors,differentdevicesandnetworks—fromothersoftwaresuchasdesktoporWebapplications.Lotsofmanualtestingisrequiredtobesurethatamobileappisworkingasexpectedinthedifferentusagescenarios.Mobiletestautomationtoolsarenotcurrentlyasmatureastheircounterparts

forWebanddesktopapplications,whichleadstoaflippedtestautomationpyramid.Asthetoolsbecomeincreasinglymature,thispyramidislikelytoflipbackagainbecausethedefaulttestautomationpyramidisbasedonamorestablefoundation(seeFigure5.1).Thedefaultpyramidthereforecan’tbeusedasan

indicatoroftestautomationandmanualtestinginthemobileworld.TheflippedtestingpyramidlookslikeFigure5.2.

Figure5.2Theflippedtestingpyramid

Inthisversionofthepyramid,theautomatedunittestinglayeristhesmallestone.Thisisthecasebecausenoteveryunitormethodofmobileappscanbetestedinanisolatedmanner.Insomecases,differentAPIs,layers,orsystemsmayneedtobefakedormockedinordertogetthesmallunittowork.Thisisalsothecaseforeveryothersoftwareapplication,butinsomecasesmockingorfakingothersystemsformobileappsismuchmorecomplex.Thisisnotefficientfromatechnicaloreconomicpointofview.However,it’snoexcusefornotwritingmobileunittestsatall.Thebusinesslogicofanappmustbetestedattheunitlevel.Thenextstageistheend-to-endtestautomationlayer.Withinthislayer,the

appistestedfromauserperspectivetomakesurethewholesystemisworking,fromtheapp’suserinterfacethroughtothebackendsystemviaawirelessnetwork,includingintegrationtestingwithdifferentlibrariesorAPIs.Theintegrationtestinglayeristhereforepartoftheend-to-endlayer.

Thebiggestchangeinthispyramidisthatmanualtestingispartofit.Mobiletestingrequireslotsofmanualtesting,andthiscan’tyetbereplacedbytestautomationoranyothertools.Nevertheless,mobiletestautomationisareallyimportanttopic,andevery

mobiletestershouldbeabletowriteautomatedregressionteststhatprovidefastfeedbackaboutthecurrentqualitystateofanapp.Furthermore,testautomationhelpstheteambuildareliableandrobustmobileappthatmakesthecustomershappy.

TheMobileTestPyramidTheflippedtestingpyramidhasnostablefoundation,andmobiletestingrequireslotsofmanualtesting,whichiswhyIcreatedmyownmobiletestpyramidconsistingoffourlayersincludingmanualandautomatedsteps(seeFigure5.3).Thebiggestlayerofthepyramid,manualtesting,formsthestrongfoundationforeverymobileappproject,followedbyend-to-endtesting,betatesting,andatoplayercomprisingunittesting.Thegraypartsofthepyramidindicatetheautomatedsteps,andthewhitepartsarethemanualtestingsteps.Thebetatestinglayerisnewtothepyramidbutessentialtoeverymobileappproject.Keepingthehighexpectationsofmobileusersinmindrequiresthatthislayerbepartofeverymobileprojecttogetearlyfeedbackfromyourmobilecustomers.Youcaneitheruseacrowdtestingapproachforyourbetatesting,oryoucanaskyourcolleaguestobeta-testearlyversionsofyourapptoprovideimportantfeedback.

Figure5.3Themobiletestpyramid

Ihaveusedthismobiletestpyramidinseveralprojectsandithelpedmesetupareliable,effective,andvaluabletestingprocess.

ImportantKeeptheproblemoftheflippedpyramidinmindandusethemobiletestpyramidinyourprojecttohaveagoodmixofmanualandautomatedtesting.

Inthischapter,Ifocusmoreontheend-to-endtestautomationtoolsbecausethosearethetoolsmobiletesterswillmostlikelyworkwith.However,someunittestingtoolsarealsomentioned.

DifferentTypesofTestAutomationToolsBeforeyoustartwithmobiletestautomation,itisimportantthatyouunderstandtheunderlyingarchitectureofthedifferentmobileplatforms.Youneedtoknowhowtoaccessthedifferentobjectsofapp-likebuttons,labels,lists,views,and

anyotherkindofelementinordertointeractwiththoseelementsduringthetestrun.Youalsoneedtobeabletowritecodeandbuildupreliabletestautomation

suitesthatareintegratedwithinyourapp’sdevelopmentpipeline.Whenselectingatool,youshouldknowthedifferenttypesandconceptsof

mobiletestautomationtoolsandhowthosetoolsaccessthedifferentobjects.Thisisimportanttoknowwhenselectingatoolforyourneedsandprojectbecauseeachapproachhasitsprosandcons.

ImageRecognitionToolsthatusetheimagerecognitionapproachcompareimagesinordertodrivetheuserinterfaceofanapp.Whenwritingthetestautomationscript,youtakescreenshots,forexample,ofbuttonsorlabelsthatareembeddedintoyourscript.Whenthescriptisexecuted,theimagerecognitiontoolcomparesthecurrentscreenwiththestoredbaselineimage.Ifthestoredimageisfoundonthescreen,thescriptwillexecutetheprogrammedsteps.ThosekindsoftoolsarereallyusefuliftheUIoftheappdoesn’tchangetoo

often,isdevelopedforseveralmobileplatforms,andhasexactlythesameuserinterfaceandcontrolelements.Inthatcase,imagerecognitiontoolsareagreatandfastwayofwritingtestautomationforcross-platformmobileapps.Thebiggestdisadvantageofimagerecognitiontoolsisthehighmaintenance

involvedwiththetestscripts.Thescriptswillnotworkiftheorientationofthedeviceischanged,suchasfromportraittolandscapemode,onbiggerscreens,orifthescreenresolutionchanges.Anotherdrawbackistestinganappindifferentlanguages,whichisnotpossiblesincethecapturedimagesareavailableonlyinonelanguageandwillnotworkwithotherlanguages.Herearesomeexamplesofimagerecognitiontools:•eggPlant(www.testplant.com/eggplant/testing-tools/eggplant-mobile-eggon/)•RoutineBot(www.routinebot.com/)•Sikuli(www.sikuli.org/)•TestObject(https://testobject.com/)

Coordinate-BasedRecognitionCoordinate-basedrecognitiontoolsrelyonpredefinedxandyaxiscoordinatestoaccessandinteractwiththeUIelementsoftheapp.Ifthecoordinatesofthe

elementchange,thewholescriptneedstobeadaptedtothenewvalues.Thishasamajorimpactonthemaintenanceofthescriptsinordertobuildreliabletests.Herearesomeexamplesofcoordinate-basedrecognitiontools:•MonkeyTalk(www.cloudmonkeymobile.com/monkeytalk)•PerfectoMobile(www.perfectomobile.com/)

OCR/TextRecognitionMobiletestautomationtoolsthatusetheOCRandtextrecognitionapproachobtainthetextofthecontrolelementsthatarevisibleonthescreenofthemobiledevice.Todetermineifthetextisvisibleonthescreen,OCRtechnologyisused.OCRandtextrecognitiontoolscanhandledifferentscreenresolutions,

orientations,andsizes.However,suchtoolscanverifyonlytextelementsthatarevisibleonthescreen.Ifthetextchangesorisremovedfromtheapp,theUIelementisverydifficult(orimpossible)toidentify.Youarenotabletocheckif,forexample,aUIviewisvisible,ifalistispresent,orifelementswithouttextareshownonthescreen.AnotherdrawbackofOCRrecognitiontoolsisthattheyareveryslowbecausethewholescreenneedstobescannedforthetext.HerearesomeexamplesofOCR/textrecognitiontools:•eggPlant(www.testplant.com/eggplant/testing-tools/eggplant-mobile-eggon/)•MonkeyTalk(www.cloudmonkeymobile.com/monkeytalk)•Robotium(https://code.google.com/p/robotium/)•SeeTest(http://experitest.com/)•TestObject(https://testobject.com/)

NativeObjectRecognitionToolsthatusethenativeobjectrecognitionapproachdetecttheUIobjectswithaUIelementtree.TheUIelementscanbeaccessedusingXPath(XMLPathLanguage),CSS(CascadingStyleSheet)locators,orthenativeobjectIDoftheelement.Thenativeobjectrecognitionapproachisverycommoninthedifferenttestautomationtoolscoveringnative,hybrid,andmobileWebapps.Thisapproachenablesyoutogetaccesstothenativeelementssuchasbuttons,labels,views,lists,andotherkindsofUIelements.IftheIDsorthelocatorsarewelldefinedandwritten,thetestscriptsarereliableintheeventofchangesandthereforereusableonotherdevicesaswell.ThisisahugeadvantagecomparedtoalloftheothertoolsbecausethescriptsdonotdependonchangestotheUI,

resolution,orientation,orthedeviceitself.Themajorityofthetestautomationtoolssupportthisapproachfordetectingtheelements.Herearesomeexamplesofnativeobjectrecognitiontools:•Appium(http://appium.io/)•Calabash(http://calaba.sh/)•Espresso(https://code.google.com/p/android-test-kit/)•Robotium(https://code.google.com/p/robotium/)•Selendroid(http://selendroid.io/)•Selenium(http://docs.seleniumhq.org/)•TenKodEZTestApp(www.tenkod.com/ez-testapp/)

CaptureandReplayLotsoftoolmanufacturersadvertisecaptureandreplayfunctionality.Toolsthatsupportthistechnologyareabletorecordactionssuchasclicking,scrolling,swiping,ortypingintoascript.Withthehelpofthereplayfunction,thesoftwarewillexecutetheexactsameactionsoverandoveragain.Thistechnologysoundsgreatintheory,butinrealityandday-to-daytestautomationwork,captureandreplaytoolsshouldbeusedandhandledwithcare.Butwhy?Ihaveworkedwithdifferentcaptureandreplaytools,andeverytool

presentedthesameproblems.Thecapturedscriptswerenotreliableatall.TheywereaffectedbyUI,orientation,andscreenresolutionchanges(mostofthemusecoordinate-basedorimage-basedrecognition).Thescriptswerenotreusablefromdevicetodeviceorondifferentmobileoperatingsystemversionsofthesamemobileplatform.Oftenthescriptswerenotabletostartbecausethedevicewasnotinadefined

state,meaningthatthetoolwasnotabletoreplaythescript.Oftenthescriptsmustbeeditedmanuallytobemorestableandreliable.Anotherproblemthatalwaysoccurredwasatimingissueasthetoolswereeithertoofastortooslowtointeractwiththeapplicationwhilethescriptwasreplayed.Theonlywaytosolvethisissuewastoaddwaitoperations,butthisisnotagoodapproachforwritingtestautomationscripts.Usingwaitorsleepoperationsisflawedandleadstounreliabletestresults.However,therearealsosomegoodqualities;forexample,mobiletesterswho

havenoprogrammingskillscanusethetoolstogeneratesomebasictestautomationscriptswithverylittletraining.Ontheotherhand,theyarenotabletomodifythescriptafterward.

However,don’tusecaptureandreplaytoolstobuildupahugetestautomationsuiteforyourmobileapp;otherwiseyouwillendupwithanonmaintainabletestsuite.IrecommendthatyouusethesekindsoftoolsasastartingpointtorecordbasicinteractionswithinyourappandgettheUIselectorsoftheelements.TherecordedUIselectorsorIDscanthenbeusedasafoundationforfurtherprogrammingtobuildupatestautomationsuitebasedonyourownprogramming.

ToolTypeRecommendationBeforeIrecommendatooltypeandapproach,Iwanttobrieflysummarizethemobiletestautomationtoollandscape.Mostofthetoolsarespecifictoamobileplatform,irrespectiveofwhethertheyusetheimage,coordinate,OCR,ornativeobjectrecognitionapproach.ThereareveryfewtoolsthatsupportmorethanonemobileplatformsuchasAndroid,iOS,WindowsPhone,orBlackBerry.However,notoolisabletoautomatemultiplemobileplatformswithonetestcodebase.Therearesometestautomationtoolproviderswhopromisetobeabletoautomateeverythingoneverymobileplatform,butthat’snottrue.Everymobileplatformhasspecificrequirementsforusabilityornavigationusageofanapp,meaningthatyoualsoneedtohavemultipletestcodebasesinordertoautomatethem.Inmostmobileprojects,youhavetodealwithmultiplecodebasesofdifferent

programminglanguagesandtestautomationtoolsinordertobuildupatestinganddevelopmentpipeline.Inmyopinion,amobiletestautomationtoolmustbeabletoaccessthenative

objectpropertiesthatareusedtoidentifyandtointeractwiththenativeelements.Thisisthebestandmostefficientwaytowritetestautomationscriptsformobileapps.Writingsuchtestscriptsrequiresmoreeffortandprogrammingskills,butthescriptsaremuchmorereliableandrobusttochangesandcanbeexecutedondifferentdevicesandscreenresolutions.Youshouldthereforeusenativeobjectrecognitiontoolsinyourprojectbased

ontheapptypeyouaretesting(native,hybrid,orWebapps).

ImportantRememberthatsomemobiletestautomationtoolssupportmorethanonerecognitionapproach.Combinetheapproachesinyourtestscripts,butkeeptheprosandconsinmindaswellasthefactthatnotoolisperfect.

WhatShouldBeAutomated?Whenyouhavethetaskofdeterminingwhichtestsshouldbeautomatedornot,keeponethinginmind:Itisimpossibletoautomateeveryfeatureortestcaseofamobileapp.Now,youhavetoanswerthisquestion:Whatshouldbeautomated?Creatingasmartmobiletestautomationplanrequirescarefulplanningand

designworkthatshouldbeperformedbeforeautomatingasingletestcase.First,youshoulddefineagoalforyourmobiletestautomationanddeterminewhatkindsoftestsyouwanttoautomate.Keepingthemobiletestpyramidinmindcanhelpyoutodefinetheunittestsandend-to-endusertests.Butyoushouldstillrememberthatyouneedlotsofmanualmobiletestingandthatnoteverythingcanbeautomated.However,youcanturnsomemanualtestsintoautomatedteststosavetimeandtestingeffort.Possiblegoalsforyourmobiletestautomationcouldbe:•Automatethebusiness-criticalparts.•Automateuserworkflowsandscenarios.•Automateonlycomplexappscenarios.•Automatesequencesthatneedtoberepeatedseveraltimes.•Automateonlytheacceptancecriteria.•Builduparegressiontestsuite.•Automateonlyifitiseconomicallyreasonable.

Ifsometestcasesareperformedonlyafewtimes,it’sprobablymoreeconomicaltoleavethemformanualtesting.Ifyou’reworriedaboutforgettingthem,addthemtoachecklist.Teststhatneedtoberunfrequentlyandrequirelotsofinputsandactionsshouldbeautomated.Butdon’tcreatelargeandcomplexautomatedtestscenariosasthosekindsoftestsaredifficulttomaintain,edit,anddebugandarelikelynottobestable.Keepthetestssmallandindependentofoneanother.Itisalsoimportanttokeepthecriticalpartsoftheappinmindsuchasthe

loginorpaymentprocessastheyhaveahugeimpactontheapp,yourbusiness,andthepeoplewhouseit.Ifthecriticalpartsarenotwelltestedorautomated,youmaysufferalotofdamagetoyourreputation.Anotherimportantfactorwhendecidingwhichtestsaretobeautomatedisthe

timeandthemoneyinvolved.Ifyouneedlotsoftimetoautomateamobiletestscenario,askyourselfifitisworthautomatingasalotoftimespentonone

specificscenariowillcostlotsofmoney.Andistheinvestedtimesavedlateronwhenthetestisautomatedratherthanmanual?Askyourselfthisquestionfromtimetotime;youmayfindthatit’sOKtotestmanually.Whenautomatingatestforamobileapp,keepthedifferentmobiledevicesin

mind.Onlyautomatetheteststhatcanbeexecutedoneverytargetdevice.Trytoautomatethetestsinasgeneralamanneraspossiblesoyoucanexecutethemonseveraldevices.Anddon’tforgettoautomatewithdifferentlanguagesifyourappisgoingtobeavailableinvariouscountries.UsetheUIelementIDstoidentifyanddrivethoseelementsandtoimplementrobusttests.Thereisnogenericanswertothequestion“Whatshouldbeautomated?”The

answerwillbedifferentforeverymobileappanddependontheapptype,mobileplatform,andapppurpose.Everymobiletestershouldkeepthefollowingpointsinmindwhenanswering

thisquestion:•Itisnotpossibletoautomatetestingoftheentireapporallofitsfeatures.•Defineagoalforyourtestautomation.•Defineend-to-endtests.•Starttestautomationasearlyaspossibletogetherwiththeappdevelopment.•Keepthetimeandthecoststhatareneededforthetestautomationinmind.•Keepthetestssmall,fast,simple,andindependent.•Keepthedifferentmobiledevicesinmind.•Executethetestsasearlyaspossibleandasoftenaspossible.

ImportantDon’ttrytoautomateeverything.Definetestautomationgoalsdependingonyourappandstartautomatingthosegoals.

Emulator,Simulator,orRealDevice?Thenextquestionyouneedtoansweriswhetheryoushouldautomatewithemulators,simulators,orrealdevices.InChapter4,“HowtoTestMobileApps,”Idescribedthedifferencesamongemulators,simulators,andrealdeviceswhenmobiletestingisperformedmanually.Myrecommendationtherewastousetheemulators/simulatorsforverybasictestsandtodomanualtestingonreal

devices.Butisthisalsothecasewithmobiletestautomation?BeforeIanswerthisquestion,let’shavealookattheprosandconsof

emulators/simulatorsandrealdevices.

Emulator/SimulatorProsThebiggestadvantageofemulators/simulatorsistheprice.BotharefreetouseandarepartoftheSDKfromthedifferentmobileplatforms.Besidesthat,theyarereallysimpletouseandoffervariousoptionsfordevelopersandtesters.Aftertheemulator/simulatorisinstalled,youcancreatethesimulators/emulatorsusingdifferentconfigurationssuchasoperatingsystemversionsorscreenresolutions.

Emulator/SimulatorConsIfwelookattheconsoftheemulators/simulators,therearesomemorepointsthatyoushouldbeawareof.Usingemulators/simulatorsincreasestheriskofmissingimportantbugsthatoccuronlyonrealdevices.Emulators/simulatorsarenotthesameastherealenvironment,whichisahugedisadvantage.Additionally,emulators/simulatorsofferonlya“plain”andsimplemobileoperatingsystem.Theyoffernodiversityintermsofthedifferentdevices,operatingsystems,andadaptationsoftheuserinterface.ThisisespeciallythecasewithAndroidwhereseveraldevicemanufacturerschangetheUIoftheAndroidsystemtosuittheirneeds.Fromahardwareperspective,theyalsofailtoofferarealdeviceenvironmentwithalloftherelevantsensorsandinterfaces.Thenextdisadvantageofmobileemulators/simulatorsisthatthedata

networksarenotreal.Networkspeedscanbesimulated,butthiswillnotcovertherealdatanetworkswithtrafficlossorchangeinnetworkspeedsandtechnologies.And,finally,emulators/simulatorsdonotofferthesameperformanceasrealdevicesintermsofCPU,GPU,memory,orsensors.Everymobiletestershouldkeepthefollowingpointsinmindwhenusing

emulatorsorsimulatorswithinaproject:•Itisriskynottotestinarealenvironment.•Thereisnodiversityintermsofhardwareandsoftware.•Thenetworkenvironmentissimulated.•Thereisnorealdeviceperformance.•Thereisnoaccesstodevice-specifichardwareelementssuchasthecamera,GPS,orothersensors.

RealDeviceProsTestingonrealdevicesofferslotsofadvantagescomparedtoanemulator/simulator.Thetestswillbeexecutedinarealuserenvironmentandthereforetheresultswillbeclosertoarealuserexperience,includinganydiscoveredbugs.Testingonrealdevicesalsoofferstheoptionofusingthefulldevice’shardware,software,anddevice-specificfeaturessuchassensors,CPU,GPU,andmemory.Usingtherealdevicefortestingshowstherealbehaviorintermsofperformance.Thefollowingpointssummarizetheprosofusingrealdevicesfortesting:•Realdevicetestingoffersreliabletestresults.•Realhardwareandsoftwarefeaturesareused.•Therealuserexperienceandappperformancecanbetested.•Testsaredoneinrealdatanetworks.

RealDeviceConsTestingonrealdevicesalsohassomedisadvantages.Thebiggestoneisthecostofbuyingallofthedifferentmobiledevicesfordevelopingandtestingpurposes.Ifyouwanttotestonrealdevices,youhavetobuynewdevicesnearlyeverymonthinordertoverifythatyourapprunsonallthenewfeatures(hardwareandsoftware).Inaddition,youneedsomeonewhoisresponsibleformaintainingallthosedevices.It’snotenoughjusttobuythem;youneedtocomeupwithastrategyforhowtoupdatethedevicestonewerversionsandhowtousethemwithinyourcompany.Thefollowingpointssummarizetheconsofusingrealdevicesfortesting:•Thecostofhavingtofrequentlybuynewdevicesishigh.•Maintenanceofallthedevicesistime-consuming.

WheretoAutomate?Ifwelookatalltheprosandcons,Ithinktheanswertothequestion“Wheretoautomate?”isprettysimple:ontherealdevice!Verysimpleandbasicfunctionaltestsandtestautomationcanofcoursebe

performedonanemulator/simulatortogetfastfeedbackaboutthecurrentstateofamobileapp,andthisisespeciallyusefulfordevelopers.However,ifyouasamobiletesterwanttobesurethatyourappisusingall

thedevice-specificelementssuchasthehardwareandsoftwareresourcesduring

thetestautomationsession,youneedtoexecutethetestsonrealdevices.Executingthetestautomationscriptsonrealdevicesprovidesyouwithmuchbetterresultsintermsofreliability,performance,andreal-worldbehavioroftheapp.Furthermore,youareabletoexecutethetestsonseveraldevicesatthesametimetoquicklydeterminewhetherornotanapphasaproblemonacertainmobiledevice.Agoodapproachistofindahealthymixofemulators,simulators,andreal

devicestogetthebestoutofyourtestautomation.Itisenoughif,forexample,developersdotheirtestautomationonemulatorsandsimulatorsbecausescalingandexecutingtestautomationscriptsinparallelonemulators/simulatorsismucheasierandcheaperthanonrealdevices.Ifyouwanttoscaleandbuildupanemulator/simulatormatrixforyourtestautomation,havealookattheGoogleTechTalk“BreakingtheMatrix—AndroidTestingatScale.”2

2.www.youtube.com/watch?v=uHoB0KzQGRg

Youasamobiletestershouldwritetestautomationscriptsthatwillrunonrealdevicesaswellassimulators/emulators.

ImportantWhenselectingamobiletestautomationtool,verifythatthetoolisabletoexecutethetestsonbothrealdevicesandemulators/simulators.

HowtoSelecttheRightMobileTestAutomationToolSofaryouhavelearnedaboutthedifferenttypesoftestautomationtoolsaswellaswhatshouldbeautomatedandwherethetestautomationshouldbeperformed.Nowitistimetofindtherightmobiletestautomationtoolforyourappandyourtestanddevelopmentenvironment.Whenselectingamobiletestautomationtool,youshouldkeepsomepointsin

mind.Thefirstpointisthatthereisno“onesizefitsall”testautomationtoolavailableonthemarket.Everytoolhasitsprosandcons,andnoteverytoolissuitedtoeverydevelopmentenvironmentandpipeline.ToolAcouldworkwellwithprojectAbutnotwithprojectB,meaningthattheevaluationhastoberepeatedforeveryproject.Tosavesometimeinthetoolevaluationphase,itisagoodapproachto

implementasampleappthatincludesalloftheelementsyourproductionappwillhaveinordertoseeifthetestautomationtoolisabletohandleandinteract

withthem.Ifthetoolisabletofulfillallofyourrequirementsforthesampleapp,youhaveprobablyfoundtherightmobiletestautomationtoolforyourproject.Thesecondpointyoushouldkeepinmindisthatmobiletestautomation

requiresprogrammingskillsinordertobuilduparobust,maintainable,andstabletestautomationsuite.Donotuseacaptureandreplaytooltocompensateforalackofprogrammingskills;youwillendupwitharealnightmarescenariobecauseyoucan’tfixthebrokenscripts.Ifyouhavenoprogrammingskills,trytolearnthemasyouwillneedtheminthefuture.Besidesdevelopingasampleapptoevaluatethemobiletestautomationtool,

youcanusethechecklistinthenextsectionwithsomeselectioncriteria.Thislistwillhelpyoutofindthetoolbestsuitedtoyourdevelopmentandtestingprocess.

SelectionCriteriaforaTestAutomationToolThefollowingpointsshouldbeconsideredwhenselectingamobiletestautomationtool:

1.Doesthetoolsupportdifferentmobileapptypes(native,hybrid,Webapps)?

2.Whichmobileplatformsaresupported(Android,iOS,WindowsPhone,BlackBerry)?

3.Whichrecognitiontechnologydoesthetooluse(native,image,text,coordinate)?

4.Doesthetoolchangetheappyouwanttotest(forexample,byaddingaserver,instrumentation)?

5.Isthetoolabletoexecutethetestsonrealdevicesaswellasonemulatorsandsimulators?

6.Isthereareportavailableattheendofthetestrun?7.Isthetoolabletotakescreenshotswhilethetestsareexecuted,andarethosescreenshotspartofthetestreport?

8.Canthetestsuitebeexecutedonseveraldevicesatthesametime?9.Howlongisthetestexecutiontime?Isitsufficientforyourneeds?10.DoesthetoolsupportalloftheUIandcontrolelementsofthemobile

platform?11.Istheresupportforachangeoforientationfromportraittolandscapeand

viceversa?12.Isthetoolabletoawakethedevicefromthesleeporstandbymode?13.Areallgesturessupported,suchasswipe,scroll,click,tap,orpinchto

zoom?14.Isthetoolabletosimulatenativebuttonssuchasthebackorhome

button?15.Doesthetoolusethedevice’ssoftkeyboardtoenterdata?16.Cantheappbetestedinseverallanguages?17.Doesthetoolrequiremodificationtomatchtherealdevice(jailbreak,

rooting)?18.Doesthetoolsupportaprogramminglanguagewithwhichyouareableto

writetestscripts?19.Isthetoolabletoexecutethetestsfromthecommandline?20.Canthetoolbeintegratedintoyourdevelopmentenvironment(IDE)?21.Canthetoolbeintegratedintoacontinuousintegrationsystem?22.Canthetoolbecombinedwithothertoolssuchasadefectmanagementor

testmanagementtool?23.Isthetoolabletoconnecttoatestcloudproviderinordertoexecutethe

testswithinacloud?24.Isthetoolwelldocumented?25.Isthetoolopensourceorclosedsource?26.Istherealargecommunity/supportbehindthetool?27.Sincewhenhasthetoolbeenavailableonthemarketandisitusedby

othercompaniesformobiletestautomation?28.Doesthetoolsupportcross-platformtests?Asyoucanseefromthecriterialist,therearelotsofthingstoconsiderwhen

selectingamobiletestautomationtool.Theevaluationphaseisveryimportantandshouldnotbeunderestimated.Ifyouchoosethewrongtool(duetotimeconstraints,forexample)beforetheprojectstarts,itisverylikelythatyouwillstrugglewiththetoolduringtheproject.Point18inthelist,“Doesthetoolsupportaprogramminglanguagewith

whichyouareabletowritetestscripts?”isveryimportant.Youshouldtrytofindatoolthatsupportstheprogramminglanguagewithwhichyouareabletowritecode;thiswillreduceyourlearningcurvebecauseyouneedtobecome

familiarwithjustthetoolandnottheprogramminglanguage.You’llalsosavelotsoftimeandmoneyonadditionaltraining.

ImportantDevelopasampleapporuseachecklistwithyourcriteriatofindthetoolthatbestfitsyourtestanddevelopmentprocess.

CurrentStateofToolsThispartofthechaptershouldgiveyouanoverviewofpossiblemobiletestautomationtools.Iselectedmainlyopen-sourceend-to-endtestingtoolsfortheiOSandAndroidmobileplatformsandtoolsthatIhaveusedinmobileproj-ects.Furthermore,Ichosetoolsthatarewellknownformobiletestautomationandusedbymostcompanies.Iwillgiveyousomerecommendationsandusefulinformationaboutthetoolsandhowtousethem.Iwillnotexplainordescribetheinstallationorconfigurationprocessofthetoolsbecausethisinformationcan(orwill)beoutdatedsoon.Allofthementionedtoolsrequireprogrammingskillsinorderforyoutobeabletoworkwiththemefficiently.And,asalways,thelistoftoolsmentionedhereisbynomeanscomplete.

AndroidToolsMostoftheAndroidtestautomationtoolsarebasedontheAndroidInstrumentationFramework3fromGoogle.TogetstartedwithAndroidtestautomation,youneedtounderstandtheviewhierarchyofanAndroidapp.Youalsoneedtoknowwhatkindsofcomponentsandelementstheappusesaswellashowalltheseelementsarearrangedonthescreenandwhattheyrepresentinthelogicalstructureoftheapp.

3.http://developer.android.com/tools/testing/testing_android.html

GoogleprovidesaveryusefultoolcalledUIAutomatorViewer4toinspecttheapp’sviewandlayouthierarchy.Thistoolletsyouviewtheproperties—thenameorIDofeachUIcomponentthatisdisplayedonthescreen.Youneedthiskindofinformation(nameorIDofanelement)towriteyourmobiletestscripts.

4.http://developer.android.com/tools/testing/testing_ui.html

YoucanfindtheUIAutomatorViewerintheAndroidSDKlocationonyourcomputer,forexample:Clickheretoviewcodeimage

/android/sdk/tools/uiautomatorviewer.sh

ThistoolisparticularlyimportantwhentheAndroidappcodeisnotavailableandyoujusthavethecompiled.apkfiletowriteyourmobiletestautomation.However,someoftheAndroidtoolsmentionedoffertheirownUIautomatorviewerthatcanbeusedtoinspecttheelements.

RobotiumRobotium5isthedefactostandardopen-sourcetoolforAndroidtestautomationandwasoneofthefirstAndroidtestautomationtoolsonthemarket.RobotiumisablackboxtoolthatprovidesfullsupportfornativeandhybridAndroidapps.RobotiumisanextensionoftheaforementionedAndroidInstrumentationFrameworkandprovidesaverysimpleandbasicAPIforwritingUItests.Itsuppliestheso-calledsoloobjecttocallmethodssuchasclickOnTextorenterText.CheckoutthecodeinListing5.1forsomepossibletestactions.

5.https://code.google.com/p/robotium/

Listing5.1SampleCodefromRobotium

Clickheretoviewcodeimage

/*Robotiumwillclickonthetext"Welcome"*/solo.clickOnText("Welcome");/*RobotiumwillenterthestringMySecretPasswordintotheinputfieldwiththeID2*/solo.enterText(2,"MySecretPassword");/*Robotiumwillclickonthebuttonwiththelabel"Login"*/solo.clickOnButton("Login");/*Robotiumwillsimulateaclickonthenativebackbutton*/solo.goBack();

Robotiumrequiresonlyminimalknowledgeoftheappyouwanttotestandprovidesexcellentreadability.Byjustreadingthetestmethodsyoualreadyknowwhatishappeningandwhatwillbetestedontheviewofthedevice.TheUIAutomatorViewersuppliesalloftheviewinformationyouneedtowriteyourtests.RobotiumtestsarewrittenintheJavaprogramminglanguageandcanbeexecutedonarealdeviceoronemulators.Testscanbeexecutedononlyasingledeviceatatime,andRobotiumisabletotestonlytheappthatisbeingtested,meaningthatthereisnowayoftestingoutsidethisapplication.AttheendofthetestrunaJUnitreportisgenerated.Thewrittentestscanbeexecutedeitherfromthecommandline,fromanIDE,

orfromacontinuousintegrationserverusingMaven,Gradle,orAnt.

IfyouwanttoknowmoreaboutRobotiumandhowtogetstarted,checkouttheRobotiumwikipage:

•RobotiumGettingStarted(https://code.google.com/p/robotium/wiki/Getting_Started)

AlsocheckouttheRobotiumprojectpages:•LatestRobotiumversionandsamples(https://code.google.com/p/robotium/wiki/Downloads)

RobotiumExtensionsIn2014,thedeveloperofRobotium,RenasReda,foundedthecompanyRobot-ium6toprovidetheso-calledRobotiumRecorder,whichallowsdevelopersandtesterstorecordRobotiumtestsinsteadofwritingthecodemanually.

6.http://robotium.com/

AnotherniceextensionforRobotiumistheExtSolo7projectfromthecompanyBitbar.ExtSoloaddssomeveryusefultestmethodstoRobotiumsuchasthefollowing:

7.http://docs.testdroid.com/_pages/extsolo.html

•changeDeviceLanguage(java.util.Localelocale):switchesthecurrentlanguageofthedeviceduringthetestexecution•setGPSMockLocation(doublelatitude,doublelongitude,doublealtitude):setsthedevice’scurrentGPSlocation•turnWifi(booleanenabled):turnsWi-Fioffandon,toseehowtheapphandlestheconnectionloss

FullAPIdocumentationisavailablehere:•ExtSoloAPIdocumentation(http://docs.testdroid.com/_static/extSolodocs/com/bitbar/recorder/extensions/ExtSolo.html

SpoonSpoon8isanotherpowerfulAndroidtestautomationframework,developedbythecompanySquare.9SpoonisalsoanextensionoftheexistingAndroidInstrumentationFramework.ThemaindifferencefromRobotiumisSpoon’sabilitytosimultaneouslyexecutethesametestsonmultipledevicesoremulators.Spooncanexecutethetestsoneverytarget(deviceoremulator)thatisvisibletoadb(AndroidDebugBridge)andisconnectedtothetestserver.

8.http://square.github.io/spoon/

9.https://squareup.com

ThetestsarealsowritteninJavaandcanbeexecutedfromthecommandline,fromanIDE,orfromacontinuousintegrationserver.Spoon’sstructureandtestcodearealsoverysimple,asyoucanseeinthecodeinListing5.2.

Listing5.2SampleCodefromSpoon

Clickheretoviewcodeimage

Spoon.screenshot(activity,"Login_Screen");assertThat(password).hasNoError();instrumentation.runOnMainSync(newRunnable(){@Overridepublicvoidrun(){password.setText("MySecretPassword");}});

Afterallofthesetestshavebeenexecutedoneverydevice,SpoongeneratesastaticHTMLreportwithdetailedinformationabouteachdeviceandtest(seeFigure5.4).Thereportcomparesthetestresultsonthedifferentdevicesinaniceoverview.Ifscreenshotsaretakenduringthetestrun,SpoongeneratesananimatedGIFimagefromthemsoyoucanseetheexecutedteststepsagaininthetestreport.

Figure5.4Spoonsamplereport

Spoonalsorequiresverylittleknowledgeofthecodefortheappyouwanttotest.SpooncanbecombinedwithotherAndroidtestautomationtools,soyoucan,forexample,usethescreenshotfunctionfromSpoonandtestmethodsfromRobotiumorEspresso.ThelatestversionofSpoonandanexampletestappcanbefoundonGitHub:•Spoonsamplesandlatestversion(https://github.com/square/spoon)

SelendroidSelendroid10isatestautomationtoolfornativeAndroid,hybrid,ormobileWebapps.ThenameSelendroidcomesfromthewordsSeleniumandAndroid.SelendroidisfullycompatiblewiththeJSONWireProtocol,11andthetestsarewrittenusingtheSelenium2ClientAPI.1210.http://selendroid.io/11.https://code.google.com/p/selenium/wiki/JsonWireProtocol

12.http://docs.seleniumhq.org/docs/03_webdriver.jsp

IfyouarefamiliarwithwritingautomatedtestswithSelenium2forbrowser-basedapplications,itisveryeasytowritetestautomationcodewithSelendroidforAndroidapps.SelendroidisabletosimultaneouslyexecuteandinteractwithmultipleAndroiddevices(realdevicesoremulators).Listing5.3showsacodeexample.

Listing5.3SampleCodefromSelendroid

Clickheretoviewcodeimage

WebElementloginButton=driver().findElement(By.id("startLogin"));WebElementpasswordInput=driver().findElement(By.id("password"));passwordInput.sendKeys("MySecretPassword");loginButton.click();

TheUIelementsofthemobileappcanbefoundbydifferentlocatortypes,forexample,byID,name,linktext,class,tagname,orXPath.ToinspecttheUIcomponentsoftheappundertest,SelendroidprovidesaveryusefultoolcalledSelendroidInspector,13whichprovidesahierarchyviewerthatletsyouseetheUIcomponentproperties.Itisabletocreateascreenshotwiththeviewlocator,recordtheclickactionsonthemobileapp,displaytheHTMLsourceofaWebview,andprovideaveryusefulXPathhelpertoidentifytheWebelements.13.http://selendroid.io/inspector.html

Inordertosupportdifferentgestures,SelendroidusestheAdvancedUserInteractionsAPI.14Thewrittentestscanbeexecutedfromthecommandline,fromanIDE,orfromacontinuousintegrationserver.14.https://code.google.com/p/selenium/wiki/AdvancedUserInteractions

SelendroidcanbefullyintegratedasanodeintoaSeleniumGrid15forscalingandparalleltesting.And,finally,theappundertestwillnotbemodifiedforautomationpurposes.15.https://code.google.com/p/selenium/wiki/Grid2

AdditionalusefulinformationonSelendroidcanbefoundontheWebsite:•Selendroidgettingstarted(http://selendroid.io/setup.html)•ScalingSelendroid(http://selendroid.io/scale.html)•LatestSelendroidversionandsamples(https://github.com/selendroid/selendroid)

CalabashforAndroidCalabash16isacross-platformmobiletestautomationframeworkfornativeandhybridAndroidandiOSapps.ThetoolmakesitpossibletowriteautomatedUIacceptancetestsinCucumber.17WiththehelpofCucumber,youcanexpressthebehavioroftheappyou’retestingusinganaturallanguage.Thisapproachiscalledbehavior-drivendevelopment(BDD),anditcanbeveryhelpfulwhenbusinessexpertsornontechnicalcolleaguesareinvolvedintheacceptancecriteriaprocess.16.http://calaba.sh/17.http://cukes.info/

CucumberusesGherkin18asthedomain-specificlanguage(DSL)toannotatethebehavioroftheapplication.18.https://github.com/cucumber/cucumber/wiki/Gherkin

PleaserefertothecodeinListings5.4and5.5forexamples.Listing5.4showsaCucumber(Gherkin)scenariousingrealtexttodescribethebehavioroftheapplication.

Listing5.4SampleGherkinCode

Clickheretoviewcodeimage

Feature:AsauserIwanttologinScenario:LoginusingvalidcredentialsGivenIamontheloginscreenWhenIenter"Username"intotheuserfieldAndIenter"PWD"intothepasswordfieldAndIclicktheloginbuttonThenImustseemyuseraccount

Listing5.5showstheRubycodeneededtomaptherealtextintocommandsthatthecomputerneedstounderstandinordertocommunicateandinteractwiththeapp.

Listing5.5SampleStepDefinitionCodeforOneGherkinStep

Clickheretoviewcodeimage

When(/^Ienter"(.*?)"intotheuserfield$/)do|username|fill_in("IDUserName",:with=>"username")end

Asyoucansee,actualtestautomationisperformedwithRubyandwithintheso-calledstepdefinitions.Gherkinisresponsiblefordescribingthebehavioroftheapplication,Rubyisneededfortheactualcoding,andCucumberistheframeworkthatexecuteseverythingtogetherontherealdevicesoremulators.TheCalabashtestscanbeexecutedfromthecommandline,fromanIDE,orfromacontinuousintegrationserver.Calabashsupportsascreenshotfunction,isabletouselocalizationwithinthe

app,andsupportsdifferentgestures.AdditionalusefulinformationaboutCalabash,Gherkin,andCucumbercanbe

foundontheGitHubprojectpage:•CalabashforAndroid(https://github.com/calabash/calabash-android)•PredefinedstepdefinitionsforAndroid(https://github.com/calabash/calabash-android/blob/master/ruby-gem/lib/calabash-android/canned_steps.md)

AppiumAppium19isanopen-source,cross-platformtestautomationtoolfornative,hybrid,andmobileWebapps.AppiumsupportsthemobileplatformsAndroid,iOS,andFirefoxOS.LikeSelendroid,AppiumusestheWebDriverJSONWireProtocoltodriveandtotesttheUIofthemobileapps.Youcanuseseveralprogramminglanguagestowriteyourtestautomation.Currently,Appiumsupportsthefollowinglanguages:19.http://appium.io/

•C#•Clojure•Java•JavaScript•Objective-C•Perl•PHP•Python•Ruby

Thewrittentestscanbeexecutedbyemulators/simulatorsorrealdevices.ThecodeinListing5.6showssomeoftheteststeps.

Listing5.6SampleAppiumCode

Clickheretoviewcodeimage

WebElementloginText=driver.findElement(By.name("TextLogin"));assertEquals("TextLogin",loginText.getText());WebElementloginTextView=driver.findElementByClassName("android.widget.TextView");assertEquals("TextLogin",loginTextView.getText());WebElementbutton=driver.findElement(By.name("Login"));button.click();

OneofthemainadvantagesofAppiumisthatthetoolnotonlyisabletocommunicatewiththeappbeingtestedbutisalsoabletostartanotherappfromtheappundertest,suchasthecameraapporthecontactsapp.Besidesthat,theappundertestwillnotbemodifiedwhenusingAppiuminordertoautomateit.MoreinformationaboutAppiumcanbefoundontheGitHubpageaswellas

onthemanufacturer’spage:•AppiumonGitHub(https://github.com/appium/appium)•Appiumintroduction(http://appium.io/introduction.html)•AppiumAPIreferencedocumentation(http://appium.io/slate/en/master)

EspressoEspresso20istheAndroidtestkitprovidedbyGoogle.EspressoisbasedonanimprovedInstrumentationTestRunnercalledGoogleInstrumentationTestRunner21tomakeAndroidtestautomationmorereliableandfaster.20.https://code.google.com/p/android-test-kit/wiki/Espresso21.https://code.google.com/p/android-test-kit/wiki/GoogleInstrumentationTestRunner

Espressoprovidesasmallandeasy-to-learnAPItointeractwiththeUIelementsofanativeAndroidapp.Espressoismainlyaimedatdeveloperswhohaveaccesstothecodebaseinordertowritefastandreliabletests.However,ifyouareabletowriteJavacodeandhaveaccesstothecodebaseoftheappyouwanttotest,Espressoisanicelittletoolforwritingthetestautomation.Asanexample,pleaserefertothecodeinListing5.7.

Listing5.7SampleCodefromEspresso

Clickheretoviewcodeimage

onView(withId(R.id.login)).perform(click());onView(withId(R.id.logout)).check(doesNotExist());onView(withId(R.id.input)).perform(typeText("Hello"));

Espressoisabletoexecutetestseitherfromthecommandline,fromanIDE,orfromacontinuousintegrationserveronrealdevicesoremulatorsbutnotinparallel.However,testexecutionismuchfastercomparedtothatofanyotherAndroidtestautomationtools.AdditionalusefulinformationaboutEspressocanbefoundontheGoogle

projectpage:•Espressostartguide(https://code.google.com/p/android-test-kit/wiki/EspressoStartGuide)•Espressosamples(https://code.google.com/p/android-test-kit/wiki/EspressoSamples)

MoreAndroidTestingToolsAsIsaidatthebeginningofthischapter,thelistofAndroidtestautomationtoolsmentionedinthisbookisbynomeanscomplete.Therearesomanyopen-andclosed-sourcetoolsavailableonthemarket,andmoretoolsareboundtofollow.ThefollowinglistcontainsthenamesofsomeotherAndroidtestautomationtoolsthatyoushouldtry.Thislistcontainssomeclosed-sourceenterprisetoolsaswellassomeunittestingtools.

•eggPlant(www.testplant.com/eggplant/)•Experitest(http://experitest.com/)•JamoSolutions(www.jamosolutions.com/)•Keynote(www.keynote.com/solutions/testing/mobile-testing)•MonkeyTalk(www.cloudmonkeymobile.com/monkeytalk)•PerfectoMobile(www.perfectomobile.com/)•Ranorex(www.ranorex.com/)•Robolectric(http://robolectric.org/)•Siesta(https://market.sencha.com/extensions/siesta)•SilkMobile(www.borland.com/products/silkmobile/)•SOASTA(www.soasta.com/products/soasta-platform/)•TenKodEZTestApp(www.tenkod.com/ez-testapp/)•TestObject(https://testobject.com/)

•UIAutomator(http://developer.android.com/tools/help/uiautomator/index.html)

AndroidToolRecommendationRecommendingamobiletestautomationtoolisnoteasy.Therearesomanyfactorsthatneedtobeconsideredwhenchoosingamobiletestautomationtool,andthosefactorsaredifferentforeachappandproject.FrommypointofviewandjudgingfromtheappsIhaveworkedwith(socialmediaandbookingapps),IrecommendtakingacloserlookatRobotium,Spoon,Appium,andSelendroid.Allofthetoolsaregreattoworkwith.Theyofferfullsupportfornative,

hybrid,andWeb-basedapps.Besidesthat,allofthetoolscomewithgooddocumentation,codesamples,andagreatcommunityifyouwanttoaskquestions.Andlastbutnotleast,writingtestcodewiththetoolsisveryeasyandlotsoffun.WhenchoosingRobotiumasatestautomationtool,Ihighlyrecommend

combiningitwithSpoon.Spoon’stestreportingfeatureisexcellent,andtheoptiontorunyourtestsonseveraldevicesatthesametimeisunbeatable.Robotiumisawell-developedAndroidtestautomationtoolwithahugecommunityandlotsofsupportbehindit.AppiumandSelendroidarealsotoolsyoushouldkeepinmind.Bothoffera

greatwaytodevelopyourautomatedtestsinseveralprogramminglanguages.BothtoolshavegreatoptionsforscalingyourtestingprocessinacloudoraSeleniumGrid.

ImportantKeeponethinginmind:Nomatterwhichtoolyouusefortestautomation,usetheresourceIDsoftheUIcomponentsifpossibleasdoingsospeedsuptestautomationandmakesitmorereliable.

iOSToolsLet’sgofurtherwithsomeiOStestingtools.WhatIalreadymentionedfortheAndroidtoolsalsoappliestoiOStools:

•Theselectedtoollistisnotcomplete.•Ihaveincludedend-to-endtestautomationtools.•Allmentionedtoolsrequirecodingskills.•BeforeyoustartwithiOStestautomation,makesureyou’refamiliarwith

theiOSUIstructureofiOSapps.

UIAutomationUIAutomation22istheiOStestingtoolthatispartofInstruments23providedbyApple.WiththehelpofUIAutomation,youareabletoeitherrecordthetestsorwritethemmanuallyusingJavaScript.IfyouarefamiliarwithiOSapps,youknowthatiOSappsuseso-calledaccessibilitylabelstodescribetheUIelementsandmakethemaccessible,suchasforscreenreaders.MostiOStestingtoolsandUIAutomationusetheseaccessibilitylabelsinordertocommunicateandinteractwiththeappbeingtested.Ifyourapphasnodefinedaccessibilitylabels,youarenotabletowritetestautomationforit.22.

https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html23.

https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/Introduction/Introduction.html

UIAutomationisabletosimulaterealuserinteractionssuchastap,swipe,scroll,pinch,ortype,eitheronarealdeviceorontheiOSsimulator.ThecodeinListing5.8showssometestactionsthatcanbeperformed.

Listing5.8SampleCodefromUIAutomation

Clickheretoviewcodeimage

app.keyboard().typeString("Sometext");rootTable.cells()["ListEntry7"].tap();alert.buttons()["Continue"].tap();

UIAutomationisabletochangethedeviceorientationfromportraittolandscapemodeandbackagain.Itisalsoabletohandledifferentalertsthatmayoccurduringthetestrunonthemobiledevice.Theautomatedtestscanbeexecutedfromthecommandline,withintheIDE,andfromacontinuousintegrationserver.MoreinformationaboutUIAutomationcanbefoundonApple’sdeveloper

pages:•UIAutomationJavaScriptreference(https://developer.apple.com/library/ios/documentation/DeveloperTools/Reference/UIAutomationRef/_index.html

CalabashforiOSCalabash24isacross-platformmobiletestautomationframeworkfornativeand

hybridAndroidandiOSapps(ifyouhavealreadyreadaboutitintheAndroidsection,youcanskipahead).ThetoolenablesautomatedUIacceptancetestswritteninCucumber.25WiththehelpofCucumber,youcanexpressthebehavioroftheappyou’retestingusinganaturallanguage.Thisapproachiscalledbehavior-drivendevelopment(BDD),anditcanbeveryhelpfulwhenbusinessexpertsornontechnicalcolleaguesareinvolvedintheacceptancecriteriaprocess.24.http://calaba.sh/25.http://cukes.info/

ImportantIalreadydescribedthefeaturesofCalabashintheAndroidtoolssection,sopleasereferbackthere.AlmostexactlythesameprocessisinvolvedwhenwritingthefeatureandstepdefinitionfilesforAndroidandiOS.

TogetmoreinformationaboutCalabashforiOS,checkouttheCalabashiOSprojectsite:

•CalabashforiOS(https://github.com/calabash/calabash-ios)•GettingstartedwithCalabashforiOS(https://github.com/calabash/calabash-ios/wiki/01-Getting-started-guide)•PredefinedstepsforCalabashiOS(https://github.com/calabash/calabash-ios/wiki/02-Predefined-steps)

ios-driverios-driver26isabletoautomatenativeiOS,hybrid,andmobileWebappsusingtheSeleniumWebDriverAPI.ItusesthesameapproachasSelendroidbutforiOSapps.ItimplementstheJSONWireProtocolinordertocommunicateandtestiOSappsusingInstruments.Thetoolisabletoexecutethetestseitheronarealdeviceoronasimulator.LikeAppiumandSelendroid,ios-driveroffersyouadifferentsetofprogramminglanguageswithwhichyoucanwriteyourtestscripts.Youcanchoosefromthefollowing:26.http://ios-driver.github.io/ios-driver/

•C#•Clojure•Java

•JavaScript•Objective-C•Perl•PHP•Python•Ruby

ThecodeinListing5.9showssomeofthepossibletestcommandsforanativeiOSappwritteninJava.

Listing5.9SampleCodefromios-driver

Clickheretoviewcodeimage

Bybutton=By.id("Login");WebElementloginButton=driver.findElement(button);Assert.assertEquals(loginButton.getAttribute("name"),"Login");loginButton.click();

InordertoidentifytheUIelementsoftheapp,ios-driverprovidesaUIinspector27similartoSelendroidthatidentifiesandviewsthepropertiesoftheUIelements.ios-driverisabletohandlelocalizedappsanddoesn’trequireanychangestotheappundertest.Thetestscanbeexecutedfromthecommandlineorfromacontinuousintegrationserver.Furthermore,thetoolcanbeusedasanodewithinaSeleniumGridtoscaleandparallelizethetesting.27.http://ios-driver.github.io/ios-driver/?page=inspector

Moreinformationaboutios-drivercanbefoundonthemanufacturer’sWebsite,aswellasontheGitHubprojectpage:

•ios-drivergettingstarted(http://ios-driver.github.io/ios-driver/?page=setup)•Sourcecodeandsamples(https://github.com/ios-driver/ios-driver)

KeepItFunctionalKeepItFunctional28(KIF)isanopen-sourceiOStestingtooldevelopedbythecompanySquare.29KIFisabletoautomatenativeiOSappsusingtheaccessibilitylabelsprovidedbytheapp.Thistoolusestheso-calledtesterobjecttobeabletosimulateuserinputssuchastouch,swipe,scroll,andtyping.Objective-CisusedtowriteautomatedtestscriptsforKIF,andKIFisableto

executethetestsonarealdeviceoriOSsimulator.28.https://github.com/kif-framework/KIF29.http://corner.squareup.com/2011/07/ios-integration-testing.html

HavealookatthesamplecodeofKeepItFunctionalshowninListing5.10.

Listing5.10SampleCodefromKeepItFunctional

Clickheretoviewcodeimage

[testerenterText:@"userone"intoViewWithAccessibilityLabel:@"UserName"];[testerenterText:@"Mypassword"intoViewWithAccessibilityLabel:@"LoginPassword"];[testertapViewWithAccessibilityLabel:@"Login"];

KIFcanbefullyintegratedwithinXcodetostartanddebugthetestautomationscripts.Furthermore,theautomatedtestscanbeexecutedfromthecommandlineorfromacontinuousintegrationserversuchasBots.3030.https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/xcode_guide-

continuous_integration/ConfigureBots/ConfigureBots.html

There’sonethingyouneedtokeepinmindwhenautomatingtestswithKIF:itusesundocumentedAppleAPIs.Thisisnotaproblemwhentestinganapp,butit’scrucialthatyourtestscriptsnotbepartoftheproductioncode.Iftheyare,ApplewillrejectyourappduetotheuseofundocumentedAPIs.IfyoufollowKIF’sinstallationinstructions,thisshouldnotbeanissue.

AppiumAppium31isanopen-source,cross-platformtestautomationtoolfornative,hybrid,andmobileWebapps(ifyouhavealreadyreadaboutitintheAndroidsection,youcanskipahead).AppiumsupportsthemobileplatformsAndroid,iOS,andFirefoxOS.LikeSelendroid,AppiumusestheWebDriverJSONWireProtocoltodriveandtesttheUIofthemobileapps.31.http://appium.io/

ImportantIalreadydescribedthefeaturesofAppiumintheAndroidtoolssection,sopleasereferbackthere.

MoreiOSTestingTools

AsIdidforAndroid,I’dliketoprovideyouwithalistofadditionaliOStestingtools.Thefollowinglistcontainsunittestingandend-to-endopen-andclosed-sourcetestingtools;itisbynomeanscomplete:

•Experitest(http://experitest.com/)•Frank(www.testingwithfrank.com/)•GHUnit(https://github.com/gh-unit/gh-unit)•JamoSolutions(www.jamosolutions.com/)•Keynote(www.keynote.com/solutions/testing/mobile-testing)•Kiwi(https://github.com/kiwi-bdd/Kiwi)•MonkeyTalk(www.cloudmonkeymobile.com/monkeytalk)•OCMock(http://ocmock.org/)•PerfectoMobile(www.perfectomobile.com/)•Ranorex(www.ranorex.com/)•SilkMobile(www.borland.com/products/silkmobile/)•SOASTA(www.soasta.com/products/soasta-platform/)•Specta(https://github.com/specta/specta)•Subliminal(https://github.com/inkling/Subliminal)•XCTest(https://developer.apple.com/library/prerelease/ios/documentation/DeveloperTools/Conceptual/testing_with_xcode/testing_2_testing_basics/testing_2_testing_basics.html#//apple_ref/doc/uid/TP40014132-CH3-SW3)•Zucchini(www.zucchiniframework.org/)

iOSToolRecommendationRecommendinganiOStestautomationtoolisalsonotaneasytask.JustasforAndroid,therearesomanyfactorstoconsiderwhenchoosinganiOStestautomationtool.Irecommendthatyoutakeacloserlookatios-driver,Appium,andKeepItFunctional.Allofthetoolsprovidereallygoodandpowerfulfeaturesinordertobuild

reliableandrobusttestautomationscriptsforiOSapps.IfyoujustwanttoautomateanativeiOSapp,KIFwouldbeagoodchoiceasyoucansetupandwritereliableandrobustautomatedtestsveryquickly.AnotheradvantageofKIFisthatthetestscriptsarewrittenwithObjective-C,thesamelanguagewithwhichtheappwillbewritten.IfyoustrugglewithObjective-C,youcansimplyaskyourdevelopersforsupportorhavethemwritethetestautomation.

IfyouwanttoautomateahybridiOSorWebapp,youshoulduseeitherios-driverorAppiumasbothoffergreatsupportforvariousprogramminglanguagesaswellastheoptiontousetheminacloudorSeleniumGridenvironment.Thisprovidespowerfulscalingandparalleltestexecutiononseveraldifferentdevicesandoperatingsystems.Allthreetoolscomewithgooddocumentationandverygoodcodesamples,

areeasytouse,andhaveahugecommunitybehindthemthatisonhandtohelpifyourunintoanyproblems.

MobileTestAutomationToolsSummaryAsyouhaveseen,therearemanydifferentmobiletestautomationframeworksavailableonthemarket.Eachtoolhasitsownstyleofwritingtestscriptsandsupportsdifferentfeaturesets,differentmobileplatforms,anddifferentmobileapptypes.Everytoolcurrentlyavailableonthemarkethasitsprosandcons.Youshouldreallykeepinmindthatnotoolisperfect,beitanopen-orclosed-sourcetool.Beforeselectingamobiletestautomationtool,scanthemarketforpossibletoolsandsolutionstohelpyoumaketherightdecision.Useasampleapporachecklisttoevaluatethevarioustools.Andlastbutnotleast,it’simportanttostartsimplewithamobiletest

automationtool.Don’ttryandfindTHEoneandonlytestautomationsolutionforyourmobileapp.Maybeyouneedtousemorethanonetoolortocombinetoolsinordertobuildupatestautomationsuitethatcoversyourneedsandrequirements.Itisbettertohaveonlyacertainamountoftestautomationinplacethatcovers,forexample,thecriticalpartsofyourappinsteadofeverypart.Whenchoosingatool,askyourselfthequestion“Whatshouldbeautomated?”

ContinuousIntegrationSystemContinuousintegration(CI)isnothingnew,andthisdevelopmentpracticeofintegratingandtestingthecodefromacentrallysharedcoderepositoryseveraltimesadayhasbeeninuseforseveralyearsnow.Everycheck-inisthenverifiedbyadifferentsetofautomatedbuildstepstoensurethatthelatestcodechangeswillnotbreakthesoftwareandintegrationwithothermodules.ACIservershouldbeavailableineveryproject,nomatterifthesoftwareisa

desktop,Web,ormobileapplication,asitwillhelptheteamreducetheriskofbrokensoftware,givefastfeedbacktoeveryoneinvolvedintheproject,andintegratesmallersoftwarepartsintoothersasearlyaspossiblewithinthe

process.Nowadays,thereareplentyofopen-andclosed-sourceCIsystemsavailable

onthemarket.IfyouhaveaCIsysteminyourteam,integratetheautomatedmobiletestsintoit.NearlyeverymobiletestautomationtoolcanbeintegratedintoaCIsystem.Ifthisisnotpossiblewiththetoolyou’reusing,you’llhavetofindawaytointegrateitsuchaswithexternalbuildscriptsthatwillrunoutsidetheCIenvironmenttofulfillthetask.Thisisveryimportantfortheprojectasawholesothatacompletebuildpipelineincludingallbuildandtestscriptscanbeestablished.Whenthetestautomationtoolhasbeenintegrated,defineabuildandtest

strategywithyourteam.Talktoyourdevelopersanddefinewhichtestsshouldbeexecutedaftereverycommitorduringthenight.IfyourautomatedtestsstarttoturnfastfeedbackfromyourCIsysteminto

slowfeedback,splitthemintoseparatetestsuites.Forexample,youcandefineasmoketestsuitecontainingteststhatcheckwhetherthemainpartsoftheapplicationarestillworking.Thistestsuiterunsforonlyacoupleofsecondsorminutesandshouldbeexecutedaftereverycommit.Anothertestsuitecanbearegressiontestsuitethatruns,say,fourtimesadaytochecktheappinmoredetail.Andanothersuitecanbeafulltestsuitethatrunseverytesteverynighttomakesurethecodechangesfromthepreviousdayhavenotaffectedtheexistingpartsoftheapp.AnotherimportantpointwhenaddingamobiletestautomationtooltoaCI

systemistestreporting.TheCIsystemmustbeabletodisplaydifferentkindsoftestreportformatsinordertoprovidethewholeteamwithvisualfeedback.Thereportingcomponentofthesystemshouldbeeasytoreadandunderstand.OncetheCIsystemandallofthemobiletestinganddevelopmenttoolshave

beenintegrated,defineacompletebuildandtestpipelineforyourmobileapplication.Thebuildpipelineshouldbeabletostartautomaticallywithoutanyuserinputs,forexample,byeitherlisteningtoacentralcoderepositoryortriggeringthebuildsatacertaintimeduringthenight.Furthermore,thebuildstepsshouldtriggerotherbuildstepsinordertounit-

test,end-to-end-test,buildanapplicationondifferentstagingsystems,buildalphaorbetaversionsoftheapp,oruploadtheapplicationtoabetadistributionserver.Hereisanexampleofapossible,verysimplemobilebuildpipeline:1.Performstaticcodeanalysis,suchaswithPMD,FindBugs,Lint,orCheckstyle.

2.Performunittests.3.Performend-to-endUItests.4.Buildamobileappversionondifferentstagingsystems.5.Buildabetaversionofthemobileapp.6.Uploadthebetaversiontoabetadistributionsystem.7.Signandbuildareleasecandidateoftheapp(theonlybuildstepthatshouldbetriggeredmanually).

Buildsteps1and2shouldbeexecutedonthedeveloper’scomputerbeforeheorshecommitsthecodetothecentralrepository.IfyouhaveaCIsystemforyourmobileappinplace,don’tforgettoplugreal

devicesintothatserverinordertoexecuteallofthetestautomationontherealdevice.ThefollowingaresomeoftheavailableCIsystems:•Bamboo(www.atlassian.com/de/software/bamboo)•Bots(https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/xcode_guide-continuous_integration/ConfigureBots/ConfigureBots.html)•Buildbot(http://buildbot.net/)•CruiseControl(http://cruisecontrol.sourceforge.net/)•Janky(https://github.com/github/janky)•Jenkins(http://jenkins-ci.org/)•TeamCity(www.jetbrains.com/teamcity/)•TravisCI(https://travis-ci.org/)

ImportantHaveaCIsysteminplaceandintegrateyourmobiletestautomationtooltogetfastfeedbackaboutthequalityoftheappaftereverycodechange.

BetaDistributionToolsAsyouhavelearnedfromthepreviouschapters,mobileusershaveexpectationsintermsoftheusability,performance,andfeaturesofmobileapps.Youandyourteamthereforehavetobesurethatyourmobileappprovidesagreatuserexperienceandisfast,reliable,andfuntouse.Tomeetalloftheseexpectations,

youandyourteamhaveachallengingjobandneedtotesttheappwithotherpeopletogetfeedbackasearlyaspossibleinthedevelopmentprocess.Togetthisfeedbackfromotheruserssuchascolleaguesorusersfromyour

targetcustomergroup,youneedatooltodistributebetaversionsofyourapp.Withthehelpofthistool,youcangivepotentialusersaccesstoabetaversionofthenextreleasecandidate.Betadistributiontoolsincludeseveralusefulfeaturessuchasover-the-airapp

distribution,crashreporting,bugreporting,anddirectin-appfeedback.Sometoolsprovideso-calledcheckpointswithintheappwhereyoucanplacequestionsfortheuseraboutthefeatureheorshejustused.Anothernicefeatureisso-calledsessions,whichcanbeincludedtotrackhowthebetatesterusestheapporafeature,thushelpingyoutoidentifyunexpectedappusage.Thetoolsalsoprovidedataandstatisticsaboutthemobileoperatingsystemversions,devicehardware,andinterfacelanguage.Alloftheinformationprovidedbyabetadistributiontoolisreallyimportant

toknowbeforeyourappisusedbythemajorityofyourtargetcustomergroup.Youcandrawonthisinformationtorefineanddevelopyourappintherightdirection,thusmakingitfarmorereliable,stable,andofcoursefuntouse.Whenusingabetadistributiontool,itisveryimportanttoinformthepotential

betatestersaboutallofthesefeaturesandthatinformationisgatheredaboutthedeviceandtheuser.Asastartingpoint,useabetadistributiontoolwithinyourcompanybyasking

yourcolleaguestotesttheappandprovidefeedback.Noteverymobileappcanbedistributedasabetaversiontotheoutsideworldduetonetworkrestrictions,companyguidelines,orpertinentlaw.Hereisalistofbetadistributiontools:•Appaloosa(www.appaloosa-store.com/)•AppBlade(https://appblade.com/)•ApplauseSDK(www.applause.com/mobile-sdk)•BetabyCrashlytics(http://try.crashlytics.com/beta/)•BirdFlight(www.birdflightapp.com/)•GooglePlaynativeAppBetaTesting(https://play.google.com/apps/publish)•HockeyApp(http://hockeyapp.net/)•HockeyKit(http://hockeykit.net/)

•TestFlight(https://developer.apple.com/testflight/index.html)GoogleandApplealsoprovideawaytodistributeabetaappversiontoa

wideruserbase.WithinGooglePlay32youareabletoaddbetatesterswiththeirGmailaddresseswhocanthendownloadthebetaversionfromtheGooglePlaystore.Alternatively,youcandefineastagedrolloutwhereanewversionofyourappisavailableonlytoacertainnumberofusers,forexample,10%ofthecurrentuserbase.Iftheappworksasexpected,youcanincreasetheapprollouteithertoyourentireuserbaseorbyanotherincrement.32.https://support.google.com/googleplay/android-developer/answer/3131213?hl=en

OntheApple33side,youarealsoabletobuildabetaversionofyourappanddistributeittoregisteredbetatesters.ThebetatestershavetoberegisteredwiththeiruniquedeviceID(UDID)usinganadhocprovisioningprofile.However,youareabletoregisteronly100testdeviceswithinonemembershipyear.YoucangetaroundthistestdevicerestrictionifyourcompanyispartofApple’senterpriseprogram.33.

https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/AppDistributionGuide/TestingYouriOSApp/TestingYouriOSApp.html

ImportantWhereverpossible,useabetadistributiontooltogatherearlyfeedbackfrombetatestersinordertobuildabettermobileapp.

SummaryThefifthchapterofthisbookconcentratedonmobiletestautomation.AtthebeginningofthechapterIexplainedtheproblemwiththetraditionaltestautomationpyramidandmobileapps.Iintroducedandexplainedtheflippedtestingpyramidandshowedanewpyramid—themobiletestpyramid.Thispyramidcontainsautomated,aswellasmanual,testingtofitalltherequirementsofthecurrentstateformobileapps.InthenextsectionofthischapterIdescribedthedifferentapproachesand

typesofmobiletestautomationtools.Thesearetoolsthatuseimagerecognition,coordinate-basedrecognition,textrecognition,ornativeobjectrecognition.Iexplainedeveryapproach,withprosandcons.Furthermore,Iassignedtheavailabletoolsonthemarkettothedifferentapproachestoprovideyouwithanoverview.InanothersectionIexplainedwhyitisabadideatojustusecaptureand

replaytoolsforyourtestautomation.Thosetoolsdoprovideagoodstarting

pointtogetsometestautomationupandrunninginthefirstplace,butinthelongruntheywillcauselotsofmaintenancetroubleandthetestsaren’treliableatall.Inthesection“WhatShouldBeAutomated?”Iexplainedwhichpartsofyour

appneedtestautomationandwhichdonot.Forexample,itisagoodapproachtoautomatethebusiness-criticalpartsoftheapp.Ontheotherhand,thosepartsthatarelikelytochangeofteninthenearfuturearefineformanualtestingbecausethetestautomationwillnotruninastablefashioniftheyareincluded.Tohelpyouselecttherighttestautomationtoolforyourmobileapp,I

includedalistwithselectioncriteriatofindthetoolthatfitsbestintoyourdevelopmentandtestenvironment.Thebiggestpartofthischaptercoveredthecurrentstateofmobiletest

automationtoolsfortheiOSandAndroidplatforms.Iexplainedthedifferenttoolswithcodesamplesaswellasprosandcons.Idescribedthefollowingtools:

•Robotium•Spoon•Selendroid•CalabashforAndroidandiOS•Appium•Espresso•UIAutomation•ios-driver•KeepItFunctional

Theclosingsectionsofthischaptercoveredthetopicsofcontinuousintegrationandbetadistributionofmobileapps.IoutlinedasamplemobileCIbuildpipelinethatcaneasilybeadaptedtoyourenvironment.IaddedalistofCItoolsthatcanbeusedformobileapps.Inthebetadistributionsection,Iexplainedthepurposeofdistributingabetaversionofyourapptocolleaguesorbetatesterstogetearlyfeedbackandbugreports.

Chapter6.AdditionalMobileTestingMethods

Sofaryou’velearnedaboutmobiletechnologiesandhowtomanuallytestmobileappsindifferentscenariosaswellaswhileoutandabout.You’velearnedaboutmobiletestautomationandtheconceptsbehindthedifferentmobiletestautomationtools.You’renowawareofseveraltoolsforthedifferentmobileplatformsandknowhowtoselecttherighttoolforyourtestingprocess.Toextendyourknowledgeandyourtoolbox,inthischapterIintroducesome

otherpossiblemobiletestingapproaches:crowdandcloudtesting.Bothoftheseapproachescanbebeneficialinyourdailyworkwithinamobileteam.

CrowdTestingAcompanyhasthreepossiblewaystoestablishsoftwaretestingwithinanorganization.Testingcanbedonewiththeaidofanin-housequalityassurancedepartment,viaoutsourcing(nearshoring/offshoring),orusingacrowdtestingapproach.In-housetestingandoutsourcingarenothingnew,andbothareconsideredestablishedapproachesamongvariousorganizationsandindustries.Butthisisnotthecasewithcrowdtesting.Thetermcrowdsourcingwas

introducedbyJeeHowe1in2006andisacombinationofthewordscrowdandoutsourcing.Inthesoftwaretestingbusinessthewordcrowdsourcinghasbecomecrowdtesting.

1.www.crowdsourcing.com/

Withthehelpofacommunityofexternalsoftwaretesters,severalcrowdtestingprovidersofferanewwaytoperformsoftwaretesting.Theexternalsoftwaretesterscomefromdiversebackgrounds,bothgeographicallyandintermsoftheirleveloftechnicalknowledge.Dependingonthecrowdtestingprovider,thecrowdcanrangefromafewpeopletoseveralthousandtestersworldwide,anditmayincludesoftwaretestingexpertsandpeopleofanyage,gender,profession,andeducationalbackground.Furthermore,thecrowdtestershavelotsofdifferentdeviceswithdiversehardwareandsoftwarecombinationsandaccesstoanumberofdifferentdatanetworks.Crowdtestingistantamounttotestinginthewild.Withtheaidofcrowdtesters,anappwillbetestedunderasetofrealistic

scenariosthatcan’tbecreatedbyanin-housetestingteam.Themobileappwillbetestedunderreal-worldconditions—withdifferentdatanetworks,hardware,andsoftwareaswellasdifferentusers.Externaltestersprovideafreshsetof

eyesforyourmobileappandwilldoubtlessreportlotsofbugs,deviations,aswellasperformance,usability,andfunctionalissues.Crowdtestingprovidersofferaplatformwherecrowdtesterscanregisterand

createaprofilestatingtheirdevices,skills,anddemographicbackground.Theclientcanaddtheappundertest,thepreconditions,samplescenarios,instructions,knownbugs,anddetailedtestplans.Theclientisalsoabletodefinethedemographicbackground,targetcustomergroup,skillset,anddevicesonwhichthecrowdshouldtestthemobileapp.Somecrowdprovidershaveaprojectmanagementframeworkinplace,

includinggovernanceandlegalstructuresforthetestingphase.Inaddition,testprovidersassignaprojectmanagerwhoisresponsibleforthetestcycle.Theprojectmanagerisalsothepersonwhofilters,rates,andcategorizesthebugsreportedbythecrowdandsummarizesthetestingcyclefortheclient.Somecrowdtestingprovidershaveanassessmentcenterortrialprojectin

placewherepossiblecrowdtestersmustparticipateinordertoverifytheirtestingabilitiesbeforebeingacceptedintothetestingcommunity.Mostcrowdtestingproviderschargeafeefortheirservices,butcrowdtesting

isrelativelycheapbecauseyoupayonlyforthepackagethatyouhaveagreeduponwiththecrowdtestingprovider.Therearedifferentkindsofpackagesavailable,suchassimplebugreporting,exploratorytesting,and/orexecutingdefinedtestcases.Figure6.1showsthetypicalcrowdtestingprocessfromthefirstbriefingwith

thecrowdprovidertothefinalpresentation.Duringthetestcycle,theclientisalwaysabletoseetheliveprogressofthecrowdtesters.

Figure6.1Typicalcrowdtestingprocess

1.Thefirststepistheinitialbriefingbetweenthecrowdproviderandtheclient.

2.Thecrowdproviderselectsthecrowdbasedontheclient’srequirements.Duringthetestcycletheprovidersupportsthecrowd.

3.Thecrowdteststhesoftware.4.Thecrowdtestersfilereportsdependingontheaimofthetestcycle,forexample,bugs,feedback,oranyotherkindofproblems.

5.Thecrowdproviderensuresthatthequalityofthereportsisgoodenough.Heorshewillfollowuponbugsifmoreinformationisneeded.

6.Attheendofthetestcyclethecrowdproviderwritesthefinaltestreport.7.Thereportisthenpresentedtotheclient.However,therearesomechallengesyouneedtobeawareofifyouwantto

useacrowdtestingapproachinyourproject.Itcantakealongtimetoprepareandorganizeacrowdtestingcycle.You

needtodefinetheexactgoalforthetestcycleandcollectandpreparealloftheinformationneededforthecrowdinordertogetvaluableresults.Youneedto

briefthecrowdtestingprovider,andattheendofthetestcycleyou’llneedsomemoretimetoreviewallofthebugandtestreportsyoureceive.However,thesestepsarealsonecessaryifyouwanttobriefanewin-housetesteronanewproject.Thereportedbugsmaybeofverylowqualitybecauseofthecrowd’slackof

knowledge.Thecrowdtestingproviderwillofcoursefilterandcategorizethebugsandpaytestersonlyforthoseofrealvalue,butit’salsolikelythatthebugreportsaren’tdetailedorpreciseenoughforyourneeds.Itmaybeverydifficultforcrowdtesterstoaccessthedevelopmentandtest

systems.Dataprivacyandsecurityissuesmaypreventyoufromgrantingexternalaccesstointernalstagingsystems,forcingthecrowdtesterstousetheproductionenvironmentwhich,inturn,hastoworkandinteractwithabetaversionofyourapp.Maybeyouneedtocreateanisolatedsectionwithintheproductionenvironmentthatcanhandlethebetarequestsandbeusedforthecrowdtestingcycle.Attheendofthetestcycleyouneedtobesurethatthecrowdtestersarenolongerabletoaccesstheproductionenvironmentandthattheappwillbeofnousetothem.AnotherchallengecomesintheformoflegalhurdlesandNDAs

(nondisclosureagreements).Ifyourappisconfidential,forexample,crowdtestingmaysimplynotbeanoption.Theprosofcrowdtestingincludethefollowing:•Thecrowdincludesdifferenttestersfromaroundtheworld,withdifferentdemographicbackgroundsandskillsets.•Lotsofdifferentmobiledeviceswithdifferenthardwareandsoftwarecombinationscanbeusedfortesting.•Themobileappistestedinreal-worldconditionswithrealusers.•Thecrowdprovidesafreshsetofeyesforyourapplication.•Lotsofissueswillbereported.•Crowdtestingprovidersfilterandcategorizethebugs.

Theconsofcrowdtestingincludethefollowing:•Crowdtestersarenotgenerallytestingexperts.•Youdon’treallyknowwhomyou’redealingwith.•Bugreportsmaybeoflowquality.•Accesstostagingsystemscanbeverydifficultduetolegalhurdles,dataprivacy,andsecurityconcerns.

•Crowdtestingcantakealongtimetoprepare.•Communicationwiththecrowdcanbedifficult.•Reproducingbugscanbedifficult.•Thereisariskthatthemobileappwillcontinuetobeusedbythecrowdafterthetestcyclehasended.

ImportantSomecrowdtestingprovidershaveamechanismtoautomaticallyuninstalltheappundertestfromtesters’mobiledevices.

Thefollowinglistincludessomecrowdtestingproviders(thislistisbynomeanscomplete):

•99tests(http://99tests.com/)•Applause(www.applause.com/)•crowdsourcedtesting(https://crowdsourcedtesting.com)•GlobalAppTesting(http://globalapptesting.com/)•Mob4Hire(www.mob4hire.com/)•passbrains(www.passbrains.com/)•Testbirds(www.testbirds.de/)•testCloud(www.testcloud.io/)•TestPlus(www.testplus.at/)

ImportantThecrowdtestingapproachisagoodextensionofyourin-housetestingteam.However,crowdtestingwon’tandshouldn’treplaceyourin-housetestingactivities.

PrivateCrowdTestingIfyou’renotabletouseapubliccrowdtestingproviderbecauseoflegalrestrictions,NDAs,ordataprivacyanddatasecurityconcerns,orbecauseyou’renotabletograntaccesstoyourdevelopmentenvironment,youcanuseaprivatecrowdtestingapproach.Youcanstartandbuildupaninternalcrowdtestingsessionwithyour

colleagues.Dependingonthesizeofyourcompany,you’llbesurroundedbylots

ofpeoplefromdifferentbackgroundsworkinginvariousdepartments,soyoucancollectfeedbackfromdevelopers,designers,productmanagers,projectmanagers,management,sales,andmarketingcolleagues.Withtheinputfromyourcolleagues,you’llbeabletosimulaterealusersindifferentusagescenariostogetaninitialimpressionofyourapp.Oneofthebiggestadvantagesyou’llnoticewhenestablishingaprivatecrowd

testingapproachisthatyou’llneedlesstimetoprepareandorganizeitthanwithapubliccrowdtestingapproachbecauseyourcolleaguesarealreadyfamiliarwiththecorporateenvironment,theproduct,anditsfeatures.Employeescanalsogainaccesstothedevelopmentorstagingenvironment,whichallowsyoutosidestepanylegalrestrictionsandNDAs.Furthermore,itisveryeasytocommunicatewithyourcolleaguesduringthe

testcycleandtoobservethemwhiletheyareusingtheapp.Thiswillprovideyouandyourteamwithextremelyvaluableinsightsintohowusersactuallyinteractwithyourappintermsofusabilityandfunctionality.Attheendofatestcycleyoucaninterviewyourcolleaguesabouttheappanditsnewfeaturestogathermoreinformationabouttheiropinionsoftheappandanyproblemstheyencounteredduringtesting.Theinternalcrowdtestingsessioncanbekeptshortandcanberepeated

severaltimesduringyourapp’sdevelopmentphase.Thisgivesyoumoreflexibilitywhenitcomestoreactingtopossibleusability,functional,orperformanceproblems.Onewaytokeepyourcolleaguesdedicatedandmotivatedistointroduceyour

internaltestingsessionasacompetition.Trycreatingcategoriesforthecompetitionandawardprizesforeachcategory.Possiblecategoriescouldincludethefollowing:

•Bestusabilitybug•Bestfunctionalbug•Bestperformancebug•Bestsecuritybug•Bestfeedbackprovided•Bestbugreport•Bestoveralltestengagement

Prizescouldincludethefollowing:•Companymugs

•Funnystickers•T-shirtswithfunnyslogans•Vouchers

Prizesdon’thavetobeexpensive;theyjustneedtomotivatepeopletofindandreportasmanybugsaspossible.Furthermore,competitionwillgiveyourcolleaguesanincentivetotakepartinupcomingtestcycles.

ImportantTrytocreateaprivatecrowdtestingsessionwithinyourcompanyandseehowyourcolleaguesperformastesters.You’llbesurprisedatthebugsandthereportedresults.

MobileCloudTestingManufacturersofmobilecloudtestingsolutionsprovideawiderangeofcurrentmobiledeviceswithdifferenthardwareandsoftwarecombinationsinthecloud.Cloudtestingprovidersusethecharacteristicsofcloudcomputingtoprovidethisservicetomobilecompanies,teams,andmobiletesters.Suchcharacteristicsincludethefollowing:

•Thecloudusesadynamic,shared,andvirtualITinfrastructure.•Thecloudprovideson-demandself-service.•Thecloudisscalablebasedontheload.•Thecloudispricedaccordingtoconsumption.•Thecloudisavailableacrossdifferentnetworkconnections.

MobilecloudtestingsolutionsareaccessibleviatheWebandprovidedifferenttypesoftestingthatcanbeperformedwithinthecloudonrealoremulated/simulateddevices.Thedifferenttestingtypesavailableare:

•Functionaltesting•Performancetesting•Loadtesting•Mobiledevicetesting•Cross-browsertesting

AsisthecasewithOpenDeviceLabs,themobilecloudprovidesyouandyourteamwitheasyaccesstoacomprehensiverangeofcurrentmobiledevicesusingallofthedifferentmobileplatforms,nomatterwhereyouareintheworld.

Ifyouwanttotestyourmobileappinthecloud,allyouneedtodoisallocatethephysicaldevice,uploadandinstallyourapp,andstartyourmanualorautomatedtesting.Providersofferdifferentkindsofadditionalservicessuchasreportingfeatures,screenshots,andvideosofyourtestsessionoranAPItoexecuteyourtestautomationscriptsonseveraldevicesinparallel.Thefactthatmobiletestcloudsaredistributedindifferentgeographical

regionsallovertheworldmakesiteasytosimulatedifferentnetworktechnologiesandscenariosfrompotentialmobileusers.Onemajoradvantageofamobiletestcloudisthatyoudon’tneedtobuynew

phonesforyourdevelopmentortestingeffortsasthiswillbedonebythecloudprovider.Furthermore,youdon’tneedtomaintainallthedifferentdevices,whichmayhaveapositiveimpactonyourprojectcosts.However,therearealsosomelimitationsifyouwanttotestyourmobileapp

withinamobilecloud.Forexample,ifyourappusesaBluetoothconnectiontoconnecttootherphysicalelementssuchasspeakers,itisnotpossibletotestwithinacloud.Itisalsonotpossibletouseallthesensorsandinterfacessuchastheproximitysensor,brightnesssensor,accelerationsensor,orgyroscopesensorbecausetheclouddevicesaremountedandconnectedtoaserverwithinadatacenter.Ontopofthat,youcan’ttestyourmobileappforinterruptionsornotificationsfromotherappsorthedeviceitself.Anotherdrawbackoftestingyourappmanuallywithinamobilecloudisthat

you’reinteractingwithyourappthroughthecomputermouse.Youhavenophysicalcontactwiththedeviceorappwithyourhandorfingers,makingitverydifficulttogetafeelfortheusabilityandresponseoftheapp.Furthermore,multitouchgesturescan’tbeperformedonthetouchinterface.And,lastbutnotleast,securityandprivacyissuesshouldn’tbe

underestimatedwhenusingamobiletestcloud.Youneedtobesurethattheproviderwillcompletelyremoveyourappanditsdatafromthetestdevicesafterthetestsessionhasbeencompleted;otherwisethenextcloudtestingusermightbeabletoseeanduseyourapp.Beforechoosingamobilecloudtestingprovider,checktheprovidedfeatures

andcomparethemwiththoseofothervendors.Youshouldalsoweightheprosandconsofmobilecloudtestingtoseeifthisapproachfitsyourprojectanddevelopmentenvironment.Theprosofmobilecloudtestingincludethefollowing:•Easyaccesstothephysicaldevices•Easyaccesstoemulatorsandsimulators

•Fastandeasysetupofthemobiledevices•Accessibilityfromanywhereintheworld•Lowercostsasthere’snoneedtobuynewdevices•Nodevicemaintenancecosts•Differenttesttypes•Simulatednetworkprovidersfromallovertheworld•Goodreportingfeaturessuchasreports,screenshots,andvideos

Theconsofmobilecloudtestingincludethefollowing:•Youhavelesscontroloverthemobiledevices.•Networkissuescanaffecttheavailabilityandthefunctionalityofthemobiletestcloud.•Securityandprivacyissues:Othercompaniesusethesamedevices,soyouneedtobesuretheappwillbedeletedinfullafterthesessionhasbeencompleted.•Performanceproblems:TestingtheappviatheInternetcanhaveanimpactontheappexecutiontimeandtestresults.•Firewallsetup:Youneedtochangesomeofthefirewallsettingsinordertogainaccesstothedevelopmentandtestenvironments.•Ifthecloudhasanysystemproblemsoroutages,yourapptestenvironmentmayalsoexperiencepoorperformanceoroutages.•Itisdifficulttotrackdownintermittentproblemsbecauseyouhavenoaccesstothesystem.•Noteverythingcanbetestedwithinacloud,forexample,sensors,interfaces,interrupts,andnotifications.•Youhavenophysicalcontactwiththedevice.

Thefollowingaresomemobilecloudtestingproviders(thislistisbynomeanscomplete):

•AppThwack(https://appthwack.com/)•Appurify(http://appurify.com/)•CloudMonkey(www.cloudmonkeymobile.com/)•Experitest(http://experitest.com/cloud/)•Keynote(www.keynote.com/)•Neotys(www.neotys.com/product/neotys-cloud-platform.html)

•PerfectoMobile(www.perfectomobile.com/)•Ranorex(www.ranorex.com/)•SauceLabs(https://saucelabs.com/)•TestChameleon(www.testchameleon.com/)•Testdroid(http://testdroid.com/)•Testmunk(www.testmunk.com/)•TestObject(https://testobject.com/)•XamarinTestCloud(http://xamarin.com/test-cloud)

ImportantMobilecloudtestingisagoodextensiontoyourin-housetestingworkbutcomeswithsometestinglimitationsthatneedtobetakenintoconsideration.

PrivateCloudIfthedrawbacksoutweightheadvantagesofapubliccloud,butyou’restillinterestedinusingacloudtestingapproach,considerusingaprivatemobiletestingcloud.Almosteveryproviderintheprecedinglistcanprovideaprivatemobiletestcloud.Aprivatemobilecloudcanbeofferedasahostedorlocallyinstalledsolution.

Thehostedsolutionisthemorecommononebecauseiteliminatesthelogisticsandcostsincurredbybuyingnewphonesandmaintainingdevices,suchasinstallingupdatesandperformingconfigurations.Privatecloudprovidersofferasecureareawithinthedatacenterthatgrantsexclusiveaccesstothephysicaldevices.Ontopofthat,theyoffervarioussecurityoptionstomeetacompany’ssecuritypoliciesandrequirements.Thelocallyinstalledsolutionisalsoknownasaprivatemobiletestlab.Ifyou

decidetousethissolution,themobilecloudproviderwillprovideyouwithamobiletestingrackincludingdevicemanagementsoftwaretomaintainandallocatetestdeviceswithinyourcompanytodevelopersortesters.Youcanalsoextendtherackwithnewdevicesonyourown.Thisdevicerackwillbelocatedbehindyourfirewall,anddevicesare

connectedtothedevelopmentenvironment,whichalleviatesanyproblemswiththespeedandconnectionproblemsofapubliccloud.Securityandprivacyconcernsarealsonolongeranissuewiththissolution.Bywayofexample,refer

tothemobiletestlabfromMobileLabs.22.http://mobilelabsinc.com/products/deviceconnect/

Alocallyinstalledsolutionalsocomeswiththeproblemthatyou’reresponsibleforbuyingandmaintainingnewdevicestobeincludedinthedevicerack.Furthermore,aprivatemobiletestingcloudcanbereallyexpensiveasyouhaveexclusiveaccesstothetestdevices,thecloudvendorprovidesyouwithexclusivesupport,andbothyouandyourcompanyneedtotrainyourcolleaguestobeabletoworkwiththeprivatecloudsoftwareandsystem.Theprosofaprivatemobiletestcloudincludethefollowing:•Easyaccesstothephysicaldevices•Fastandeasysetupofthemobiledevices•Accessibilityfromanywhereintheworld•Nodevicemaintenancecosts(hostedsolutiononly)•Differenttesttypes•Simulationofvariousnetworkprovidersfromallovertheworld(hostedsolutiononly)•Exclusiveaccesstothetestdevices•Nosecurityconcerns

Theconsofaprivatemobiletestcloudincludethefollowing:•Itisfarmoreexpensivethanapublicmobiletestcloud.•Networkissuescanstillaffecttheavailabilityandthefunctionalityofthemobiletestcloud.•Firewallsetup:Youneedtochangesomefirewallsettingstogainaccesstothedevelopmentandtestenvironments(hostedsolutiononly).•Ifthecloudhasanysystemproblemsoroutages,yourappwillperformpoorlyandmayalsoexperienceoutages(hostedsolutiononly).•Itisdifficulttotrackdownintermittentproblemsbecauseyouhavenoaccesstothesystem(hostedsolutiononly).•Noteverythingcanbetestedwithinacloud,forexample,sensorsandinterfaces.•Additionaltrainingisrequiredtoworkwiththecloudprovidersoftwareandsystem.

Important

Becauseofthemanualtestinglimitations,youshouldconsiderusingmobiletestcloudsformobiletestautomationpurposesonlywiththeaimofhandlingautomatedtestingandfragmentationacrossseveraldevices.Manualtestingshouldstillbeperformedonrealdevicesinreal-worldenvironmentsandwhileonthemove.

Cloud-BasedTestAutomationInChapter5,“MobileTestAutomationandTools,”Iexplainedthedifferentconceptsofmobiletestautomationtools.IalsodescribedsomemobiletestautomationtoolsfortheiOSandAndroidplatforms.Whenchoosingamobiletestautomationtool,checktoseeifthetoolisabletoexecutethetestscriptswithinamobiletestcloud.SomeofthemobiletestcloudprovidersofferanAPIforvariousmobiletestingtoolssoyoucanexecuteyourscriptswiththeirservices.ThisAPIcanhelpyouscaleyourtestingeffortsandtestdifferentdevicesinparallel.SomeprovidersofferthepossibilityofwritingtestautomationscriptsontheWebwithinthecloudtestingsoftware.Theadvantagesofamobiletestcloudalsoapplytotestautomationwithinthe

cloud.Amobiletestcloudcanhelpyoucreateanautomatedon-demandtestenvironmentforyourmobileappandmobileplatform.However,usingacloud-basedtestautomationapproachalsohassome

additionaldrawbacks.Testexecutiononclouddevicesisslowerthanwithalocaltestautomationsolution,whichispartlyduetothecommunicationbetweenthecloudprovidernetworkandyourcompanynetworkwhenrequestingandsendinglotsofdata.Thislatencycanhaveanimpactonthetestresultsandbehaviorofyourapp.Testingtheperformancewithtestautomationscriptsonamobiletestcloudisthereforenotanidealsolution.Debuggingthetestautomationscriptsontheclouddevicesisanotherissueasscriptdebuggingispossiblebutnotyetgoodenoughtoworkefficiently.Ifyou’reconsideringusingacloud-basedtestsolution,evaluateseveral

providerstoseeiftheyofferthefeaturesyouneedforyourmobileapp.Acloud-basedtestautomationapproachcanbeausefuladditiontoyourin-housetestautomationandcanmakeyourtestingworkmoreefficient.

SummaryInthesixthchapteryouhavelearnedaboutcrowdandcloudtestingservices.Bothcanbeadditionstoyourin-housemobiletestingactivitiesbutshouldneverbeusedastheonlymobiletestingsolutionforyourapp.

InthecrowdtestingsectionIexplainedthetypicalprocessofacrowdtestingcycle.Youneedtokeepinmindthatthisprocesswilltakequitesometimeforpreparationaswellasinthebuganalysisphase.Theseeffortsshouldnotbeunderestimated.Furthermore,Idescribedthedifferencesbetweenaprivateandapubliccrowdtestsession.Bothapproachesdifferalotinthedetails;however,thosedetailsareimportantinordertoavoidinfrastructure,dataprotection,andsecurityconcerns.Inthecloudtestingsection,Iexplainedthefeaturesofacloudtesting

provider.Inaddition,Idescribedwhichtesttypescanbeperformedwithinacloud.IntheprosandconssectionIoutlinedpossibleproblemswithpublicandprivatecloudsaswellastheadvantages.

Chapter7.MobileTestandLaunchStrategies

Sofaryou’velearnedalotaboutmobiletestingandthedifferenttesttechniquesandapproaches.Thischaptercoversmobiletestandlaunchstrategiesandwhatyouneedtoincludeinyourstrategies.Boththetestandlaunchstrategiesareveryimportanttoeveryproject,andyoushouldn’tunderestimatejusthowusefulitistokeepawrittenrecordofthem.InthefollowingsectionsI’llprovideyouwithsomeexamplesofmobiletestandlaunchstrategies.You’llalsofindsomequestionsthatwillhelpyoutowriteyourownstrategies.

MobileTestStrategyIngeneral,ateststrategyisadocumentthatdescribesthetestingapproachandworkinvolvedinthesoftwaredevelopmentcycle.Thisstrategycanbeusedtolettheprojectmanager,productmanager,developers,designers,andanyoneelseinvolvedinthesoftwaredevelopmentcycleknowaboutthekeyissuesofthesoftwaretestingprocess.Ateststrategyincludesthetestingobjective,testlevelsandtechniques,

resourcesrequiredtotestthesystemundertest,andthetestingenvironment.Italsodescribestheproductrisksandhowtomitigatethemforstakeholdersandcustomers.Finally,italsoincludesadefinitionoftestentryandexitcriteria.Thedefinedteststrategywillhelp,remind,andguideyousoyoudon’tforget

theimportantcomponentsandfeaturesoftheapp.Youreallyshouldtakethetimetowritedownthestepsandresourcesneededtotestthemobileapp.Furthermore,theteststrategydocumentsyourworkandendeavorswithintheproject.Writingateststrategyoncedoesn’tmeanthatit’sthensetinstoneandyoucan’tmakeanysubsequentchanges.Onthecontrary,it’simportantthatyoutalkaboutthestrategywithyourteamfromtimetotimesoyoucanadaptittoanyproductchangesorothermorerecentcircumstances.However,thereisnosinglemobileteststrategythatcanbeusedbyevery

teamorforeverymobileappasalmosteveryapphasdifferentrequirements,goals,andtargetgroupsandrunsondifferentmobileplatforms,whichmakesitimpossibletoreusetheentirestrategyineveryproject.Buttherewill,ofcourse,besomeitemsthatcanandshouldbereused.Thefollowingsectionsofthischaptershouldgiveyouanideaofhowtoshape

amobileteststrategy.Youcanusethemasastartingpointandguideforputtingtogetheryourownmobileteststrategy.

ImportantDraftingamobileteststrategydoesn’tnecessarilyinvolvewritingendlessdocumentationasyouandanyothertesterssimplywon’thavethetimeand/ortheresourcestorunthroughitall,andflexibilityisthenameofthegameinthemobiletestingbusiness.Amobileteststrategyshouldservetoguideyouandanyoneelseinvolvedintheprojectsoyoucankeeptabsontheimportantpartsofthetestingprocess.

DefineRequirementsThefirstthingyouandyourteamshoulddoisdefineyourapp’srequirementsandfeaturesattheverybeginningoftheproject.Writethemalldownanddescribethefeaturesandpossibleusescenariostogetabetterfeelingfortheappanditspotentialusers.Roughdescriptionsoftherequirementsandfeaturesareabsolutelyfineatthispointastheywillbespecifiedinmoredetailduringthedevelopmentprocess.Awrittenrecordoftheserequirementswillmakeitmucheasierforyoutoderiveyourmobiletestingstrategy.Hereisalistofsomepossiblerequirementsandfeatures:•Providearegistrationform.•Providealoginformtoaccesstheapp’scontent.•Providealogoutoption.•Implementasearchfunctionwithintheapp.•Theusershouldbeabletocreateauserprofile.•Theusershouldbeabletosharecontentwithotherusers.•Theusershouldbeabletosharecontentonsocialnetworks.•Theusershouldbeabletotakepictures.•Theappshouldbeavailableindifferentlanguages(English,German,French,andsoon).

Furthermore,youshouldknowwho’sgoingtouseyourapp.AsdescribedinChapter3,“ChallengesinMobileTesting,”youreallyneedtoknowyourtargetgroupandtheirexpectations.Gatherasmuchinformationasyoucantogainimportantinsightsintoyourcustomers’usescenarios.Here’saquickrecapofpossibleinformationaboutyourtargetgroup(thefull

listcanbefoundinChapter3):•Gender

•Monthlyincome•Educationalbackground•Location•Otherappstheyuse•Smartphonehabits•Devicestheyuse

ImportantIfyoudon’tknowanythingaboutyourtargetgroup,checkmobile-platform-specificnumbersofoperatingsystemsandhardwarespecifications.Furthermore,analyzeandgatherinformationaboutappsthataresimilartoyours.Thisisagoodstartingpointtohelpyougatherpotentialuserinformation.

Basedontherequirementsandfeaturesofyourappandknowledgeofyourtargetgroup,youcanaskspecificquestionstocollectinformationforyourtestingworkandscope:

•Isitimportanttofindcriticalbugsquickly?•Shouldtheappbetestedonlyincommonuserscenarios?•Onwhichmobileplatformsshouldtestingbeperformed?•Whatareourcustomers’carriernetworks?•Arethereanyareaswithintheappthatarelikelytochangeonaregularbasis?•Istherelease(submission)dateoftheappalreadyknown?•Istherearoadmapforfuturereleases?

Don’thesitatetoaskthesekindsofquestionsasthey’reimportant,andtheanswerswillhelpyoutodefineyournexttestingstepsandpriorities.Don’tworryifyouforgettoaskaquestionbeforeyoustartwritingtheteststrategy—it’sbettertoaskaquestionwheneveritcomesupthannotatall.Inthenextstepyoushouldcollectinformationaboutthedevelopment

environmentwithinthecompany.Itisimportanttoknowwhichtoolsareusedtosetupadevelopmentandtestpipeline.Whichcontinuousintegrationserverisbeingused?Whichtoolsareusedtobuildtheapp?Andwhichbackendtechnologiesareusedtoprocesstherequestsfromthemobileapp?Youalsoneedtoknowaboutthearchitectureoftheproductionenvironment.

Togoaboutcollectingallofthisinformation,youshouldascertaintheanswerstothefollowingquestions:

•Whatkindofsoftwaredevelopmenttoolsarealreadyavailableandinusewithinthecompany?•Isthereacommonbuildpipelinethatmustbeused?•Whichcontinuousintegrationserverisusedfortheproject?•Whichtoolsareusedtobuildthemobileapp?•Whichtechnologiesareusedtoprocessthemobilerequests?•Whatskills,suchasprogramminglanguages,areavailablewithinthemobileteam?•Howmanymobiledevicesareavailablewithinthecompanyfortesting?•Whattoolsandtechnologiesareusedintheproductionenvironment?•Howmanypeopledoweexpecttousethemobileapp?

Theanswerstothesequestionswillhelpyouchoose,forexample,thetestautomationtoolbasedonthetechnicalknowledgewithinyourteam.Theywillhelpyoutodefinethetestlevelsandtechniques,andfurthermoretheywillprovideafirstoverviewofalltechnologiesinvolvedinthemobileproject.Knowledgeofthedevelopment,test,andproductionenvironmentisveryimportantwhenitcomestocoordinatingtestingwithintheproject.

ImportantCollectingrequirementsandfeaturesisimportantasyouneedsuchusefulinformationforyourmobileteststrategy.Thiskindofinformationisagoodstartingpointforplanningyourtestingactivitiesandwillhelpyoutodefineatestingscope.

TestingScopeOnceyou’vedefinedtherequirements,youcanspecifythescopeforyourteststrategy.It’snotpossibletotestanapponeverypossiblehardwareandsoftwarecombination.Youshouldthereforereducethescopeofyourtestingeffortsandinitiallyconcentrateontheimportantpartsofthemobileapp.Therearefourwaystoreduceyourtestingscope:•Single-devicescope•Multidevicescope

•Maximumdevicescope•Use-casescope

Single-DeviceScopeThesingle-devicescopefocusesononemobiledeviceduringtesting.Thisapproachcanorwillbeusedifonlyonedeviceistobesupportedbytheapporifthereisverylittletimeavailablefortheproject.Intheeventoftimepressure,you’llprobablychooseonlythemostpopulardeviceusedbyyourtargetedcustomers.ThisdevicewillbeusedfortestingwithjustonemobilecarriernetworkandpossiblyaWi-Ficonnection.Anotherapproachcouldbetochooseadevicefromthedevicegroupthathasolderhardwarespecificationsandcouldthereforecausemoreproblemsforthedevelopersintermsofsupport.You’relikelytofindmorebugsandproblems,suchasperformanceorotherissues,withthisdevicethanwiththelatestdevice.Usingonlythesingle-devicescopecanbedangerousforthemobileapp,for

thesuccessoftheproject,orevenforthewholecompany.Itisverylikelythatyou’llmissimportantbugsthatoccuronotherdevicesandthatyourcustomerswillsubmitbadratingstotheappstores.Iftheappsupportsonlyonedevice—forexample,ifitisaninternalenterpriseapp—thisapproachcanbeused.Ontheotherhand,it’sbettertotestononlyonedevicethantoskiptheentiretestingprocess.

MultideviceScopeAsthenamesuggests,themultidevicescopefocuseseitheronseveraldevicesfromonemobileplatformoronmultiplemobileplatforms.Selecttheplatformsandtestdevicesbasedonyourtargetgroup,andthengroupthedevicesasdescribedinChapter3inthesection“MobileDeviceGroups.”Ifyoudon’thaveanyinformationaboutyourtargetgroup,usetheInternettosearchforplatform-specificnumbersandstatistics,whichwillhelpyouselectthemobileplatformanddevicestoconcentrateon.AreallyniceWebpageprovidedbyGoogleis“OurMobilePlanet”1where

youcangetinformationbasedonthecountry,age,gender,userbehavior,andbehaviorduringthecurrentyear.

1.http://think.withgoogle.com/mobileplanet/en/

MaximumDeviceScopeThemaximumdevicescopefocusesonasmanymobileplatformsanddevicesaspossible.Thisapproachcanbeusedformass-marketappsintendedtoreachas

manycustomersaspossibleallovertheworldwithnorestrictionsintermsofplatform,device,carriernetwork,ortargetgroup.Testinganappforthemassmarketisverydifficultbecausetherewillalmostalwaysbeahardwareandsoftwarecombinationthatdoesn’tworkwellwithyourapp,andit’snearlyimpossibletofindthiscombination.Inordertohandleandreducethisrisk,youneedtofindawaytotestonasmanydevicesaspossible.Thisapproachrequireslotsofresearchtogatherinformationandstatistics

aboutcurrentdeviceusage,mobileplatforms,andoperatingsystemversions.Itrequiresinformationaboutthedifferentcarriernetworksandconnectionspeedsfromaroundtheworld,andsoforth.Onceyou’vecollectedtherequiredinformation,considerusingacombination

ofin-housetesting,cloudtesting,andcrowdtestingtohandlethemass-marketsituation.Bearinmindthattestingjustwithin-houseresourcesanddeviceswillbeeithertoolimitedinscopeortooexpensive.

Use-CaseScopeBesideschoosingahardwaretestingscopetodownsizeyourtestingwork,youcanalsochooseause-casescopetolimittheworkload.Withthisapproachyoucanconcentrateoncertainpartsorfeaturesofthemobileappandleaveoutlessimportantones,suchashelptextsoredgecases.Nearlyeveryprojectisunderextremetimepressure,thusmakingitimportanttodefinewhichpartsoftheappmustbeincludedinthetestingscopeandwhichcanbeleftout.Writedownbothpartsinyourteststrategyanddescribetheusecasesthathavetobetestedandwhy.Ifthereisenoughtime,thelessimportantpartsshouldbetestedaswell.Youmaywantorneedtoincludethefollowingtestscopeinformationinyour

teststrategy:•Whichapproachshouldbeusedintheproject?•Isitpossibletocombineapproachesforcertainpartsoftheapp?•Whywasthisapproachselected?•Whataretherequiredandmainusecasesoftheapp?

DefineTestLevelsandTestTechniquesOnceyou’vedefinedtherequirementsandscope,youneedtothinkaboutthedifferenttestlevelsandtesttechniquesyouwanttouseinyourproject.KeepinmindthequalityassurancemeasuresoverviewfromChapter4,“HowtoTestMobileApps”(thesection“TraditionalTesting”),whendefiningthem.Thisoverviewwillhelpyoutoderivethetestlevelsandtechniquesforyourmobile

app.

TestLevelsAsshowninChapter5,“MobileTestAutomationandTools,”thefocusoftestlevelsfromnon-mobilesoftwaredevelopmenttendstoshiftthroughoutthemobilesoftwaredevelopmentprocess.The“mobiletestpyramid”showsthattheunittestinglevelisthesmallestpartcomparedtoend-to-endtesting,betatesting,andmanualtesting.Inmostsoftwaredevelopmentprojects,thedevelopersareresponsiblefor

writingunittests.Thisisalsothecasewithmobileappprojects.Mobiletestersareresponsibleforwritingtheend-to-endtestautomationincludingintegrationtesting.However,everyoneontheteamshouldberesponsiblefortheapp’squality—theyshouldallsupportthemobiletesterinhisorherwork.Asmentionedinseveralchaptersinthisbook,manualtestingisavery

importantpartofamobiledevelopmentprojectandformsanessentialpartofthetestlevelsinamobileproject.However,thereareotherlevelsthatareimportantformanualtesting:acceptance,alpha,andbetatesting.Useraccep-tancetestscanbeperformedtotestthemobileappagainsttheuserrequirementstoverifythattheappcoversthemall.Thisstepisusuallycarriedoutbyatester,productmanager,orthecustomer.Inmobiledevelopmentprojects,alphaandbetatestingareimportanttest

levelsthatshouldformpartofyourteststrategy.Wheneverafeatureisimplementedwithinyourapp,youshouldtestitwithpotentialcustomerstogatherearlyfeedbackaboutit.Iftestingwithpotentialcustomersisnotpossiblewithinalphatests,youcouldtrytestingthefeaturewithyourworkcolleaguestogetfeedbackfromoutsideofthedevelopmentteam.Oncethemobileapphasreachedadefinedmaturity—forexample,allofthe

featureshavebeenimplementedoronlytwobugswerefoundinthelastweek—youshouldconsiderusingbetadistributiontoolsoracrowd-basedtestingapproachtoperformbetatestswithpotentialcustomers.Thecriteriathatdeterminethatatestingphaseisfinishedandthatthenextoneisreadyforexecutionalsoneedtobedocumentedintheteststrategy.Typicalsoftwaretestlevelsthatshouldbeusedinamobiledevelopmentproj-

ectarelistedhere:•Automatedtesting•Unittesting•End-to-endtesting(includingintegrationtesting)

•Manualtesting•Acceptancetesting•Alphatesting•Betatesting

•RegressiontestingOnceyou’vedefinedthetestlevels,youshouldalsoconsiderdefininghow

intensivelyeachlevelshouldbetestedfromafunctionalandnonfunctionalpointofview.Pleasekeepinmindthatnotalltesttypeswillbeincludedateverytestlevel.Furthermore,itcanbehelpfultodefinesomemetricsthatmeasurethecurrent

stateoftheapplication.Possiblemetricsmayincludethefollowing:•Eachfeaturemusthaveatleastoneunittest.•Eachfeaturemusthaveatleastoneend-to-endtest.•Thereshouldbenowarningsorerrorsinthestaticanalysischeck.

Functionaltestingshouldincludethefollowing:•Identifythefunctionalityofthemobileapp.•Testthedifferentpartsagainstthefunctionalrequirements.•Defineandexecutethetestcases.•Createinputdatabasedonthespecifications.•Comparetheactualandtheexpectedoutputs.

Nonfunctionaltestingshouldincludethefollowingpoints:•Loadtesting•Performancetesting•Usabilitytesting•Securitytesting•Portabilitytesting•Accessibilitytesting•Internationalization/localizationtesting

Thefollowingtest-levelinformationcouldbeusedinyourteststrategy:•Whichtestlevelswillbeusedintheprojectandwhy?•Whichpartsaretobeusedforfunctionalandnonfunctionaltesting?•Defineanddescribetheautomatedtestlevels.Whichpartsneedtobeunit

testedandwhichpartswillbetestedusinganend-to-endtestautomationtool?•Defineanddescribewhenthesoftwarehasreachedacertainmaturityandcanbeusedforalphaandbetatesting.•Defineanddescribemetricsrelevanttotheproject.

TestTechniquesYoucandrawuponqualityassurancemeasures(Chapter4)todefineyourtesttechniquesandmethods.Considerusingstaticanddynamicapproachestotestyourmobileappfromdifferentpointsofview.AsdescribedinChapter4,Irecommendusingstaticcodeanalysistoolsin

yourstatictestingapproachtotestyourmobileapp’scodeforanybugsorproblems.Keepinmindthattheappcodeisnotexecutedduringstatictesting.Furthermore,alloftheprojectdocumentationshouldbereviewedforcompleteness.Withthedynamictestingapproachyoushouldusewhiteandblackboxtesting

techniquestotestyourapp.Whiteboxtestingshouldbedonebythedevelopersandcoversthefollowing:

•Statementcoverage•Pathcoverage•Branchcoverage•Decisioncoverage•Controlflowtesting•Dataflowtesting

Blackboxtestingshouldbedonebythesoftwaretestersincluding:•Equivalenceclasses•Boundaryvalueanalysis•Decisiontables•Statetransitions•Cause-effectgraph

However,thedevelopersshouldalsotestforboundaryvaluesandstatetransitionsattheunittestinglevelinordertobesurethateachunitcorrectlyhandlesthosesituations.Yourteststrategyshouldalsoincludeawrittenrecordofwhichtesttechnique

istobehandledbywhom.Besidestheaforementionedtechniques,youshouldconsiderusingexploratory

andrisk-basedtestingtoorganizeanddownsizethetestingworkwithinyourmobileteam.

ImportantDefinetestlevelsforyourmobileappbasedonitsfeaturesandrequirements.Qualityassurancemeasureswillhelpyoudefineyourtestmethodsandtechniques.

Thefollowinginformationabouttesttechniquescouldformpartofyourteststrategy:

•Whichtesttechniquewillbeusedwithyourprojectandwhy?•Defineanddescribetheorderofthetesttechniques;forexample,staticcodeanalysisanddocumentreviewaretobefollowedbywhiteboxtesting,blackboxtesting,andthenexploratorytestsessionsinvolvingthewholeteam.•Whichteammembersaretoapplythedifferenttesttechniques?•Defineanddescribethemanualtestingprocess,suchasacceptancetesting,exploratorytesting,alphaandbetatesting.•Defineanddescribethetestexitcriteriaforwhiteandblackboxtesting,forexample,80%branchcoveragewithwhiteboxtesting.

TestDataNearlyeveryappprocesses,creates,andsendsdatafromthemobileappviadatanetworkstodifferentbackendsystems.Theprocessedandtransferreddatadiffersfrommobileapptomobileappandhasdifferingrequirementsandcomplexities.Onecomponentofyourmobileteststrategyshouldbetherequiredtestdata.Itisimportanttodefinethetestdataasrealisticallyaspossiblebasedonthefeaturesandrequirementsofyourapp.Thefollowingthreepointsareanexampleofdifferenttestdatatypes:•Configurationdata:Thisdataincludesconfigurationsforthemobileapporforthebackendsystem.Itcould,forexample,includedecisionrulesforrulesenginesand/orsettingsfordatabasesandfirewalls.•Stabledata:Thisusuallyinvolvesdatawithalowrateofchangeandalongduration.Atypicalexampleofthisiscustomerinformationsuchas

usernameandpassword,orproductinformation.•Temporarydata:Thiskindofdataislikelytochangefrequently,meaningthatitcanbeusedonlyonceorwillbecreatedwhiletheappisrunning,forexample,paymentdetailsorvouchers.

Oncethetestdatarequirementsareclear,findawaytosavethetestdatasoyoucanre-createandresetthedatainadefinedstatewheneveryouneedit.Onepossiblesolutionforthisisadatabasewherethestoredinformationcanbeusedduringthedevelopmentandtestingprocess.Anotheradvantageofthisisthatyoucanalsousethedatabaseformanualandautomatedtesting.Ontheotherhand,youcanusetestdatamanagementtoolstoorganizedatawithinyourproject.Onceyou’vesetuptheresetandre-creationprocess,youshouldstartcreating

thedataassoonaspossibleasthiswillhelpyouandyourcolleaguesduringtheappdevelopmentprocess.Dependingontheapp,youmayneedlotsoftestdata.Ifthisisthecase,it

mightbeagoodideatouseageneratortocreatethedataautomatically.Ifadatageneratorisused,itisimportantthatthefunctionalityandnecessaryparametersbedocumented.Iftestdataisavailablewithinyourproject,don’tforgettoadaptittonew

featuresandchanges.Itisverylikelythatovertimefeatureswillbeimprovedandthetestdatarequirementswillchange.Yourteststrategyshouldthereforealsooutlineaprocessforupdatingthetestdata,includingresponsibilitiesandtriggersfortheupdateprocess.Itisalsorecommendedthatyoudefineastrategyforhowoutdatedtestdatashouldbearchivedinordertoreproduceincomingbugsinoldfeaturesorversionsofyourapp.Thefollowingtestdatainformationcouldformpartofyourteststrategy:•Howisthetestdatagenerated?•Whereisthetestdatastored?•Howisthedocumentationhandled,forexample,testdatatogetherwithtestresults?•Howoftenwillthetestdatabeupdated?•Whoisresponsibleforthetestdata?

SelectTargetDevicesandTestEnvironmentNowthatyou’vedescribedthefeatures,requirements,testlevel,andtechniqueaswellasthetestdata,youneedtothinkaboutthetestenvironmentandtest

targetdevicesforyourstrategy.AsyoulearnedinChapter3,groupingyourtestdevicesormobileWebbrowsersisagoodapproachtodeterminewhichdevicesshouldbeusedforthemobileapp.Creatingsuchmobiledevicegroupsrequiresinformationaboutyourtargetcustomergroupandtheirusagescenarios.Don’tforgetthetestingscopesdescribedinthischapterwhendoingso.Oncethedevicegroupsareinplace,you’llneedtoselectdevicesfromthose

groupstohavethemavailablewithinyourteam.Itisrecommendedthatyouhaveatleastonedevicefromeachgroupavailablefortesting.However,Irecommendatleastfivedevicesfromeachgroupinordertohaveabroadermixofhardwareandsoftwarecombinationswithdifferentformfactorsanddisplays.Youshouldalsokeeparecordofwhyyouchosethosetestdevices.Nowthatyouknowwhichdevicesareneededfortesting,youneedtobuyor

rentthem.Buyingallofthedevicesyouneedcanbeexpensiveandperhapsisnotanoptionduetotheproject’sbudget.Agoodapproachtosavesomemoneyisonlineauctionswhereyoucanbuyuseddevices.Inmostcasesthedevicesareingoodenoughshapeforyourtestingwork.Ifbuyingisnotanoptionatall,youcanrentthedevices.Asmentionedin

Chapter3,thereareseveralmobiledevicelabprovidersonthemarketthatwilllendyouthedevicesyouneedforasetperiod.However,checktherentalpricesandcomparethemwiththedeviceprice.Ifyouwanttorentthedevicesforaprolongedperiod,rentingwillprobablybemoreexpensivethanbuyingthedeviceinthefirstplace.AnotheralternativeisOpenDeviceLabs2wherethedevicescanbeborrowed

forfree.ChecktheOpenDeviceLabsmaptofindoneinyourarea.Ifyouwantafreedevice,youcanalsoaskaroundatyourcompanyorevenseeifsomeoneinyourfamilyhasthedeviceyouneedthatyoucanborrowforawhile.

2.http://opendevicelab.com/

Onceyou’vespecifiedyourdevicestrategy,youneedtothinkaboutthetestenvironment:thebackendsystems.Youneedtoknowthearchitectureofthebackendsystemssuchasdatabases,paymentsystems,APIs,andanyotherkindofsysteminvolvedinthemobileproject.Ifyouhavesystemsinformation,youneedtobesurethatit’salsoavailablewithinthetestenvironmentsoyoucantestasthoughyou’reintheproductionenvironment.Ifyourteststrategycontainsin-the-wildoron-the-movetests,there’sanadditionalrequirement:thetestenvironmentmustbeaccessiblefromoutsidethecompanynetwork.Thefollowinginformationaboutthetargetdevicesandthetestenvironment

couldformpartofyourteststrategy:

•Whichdeviceswillbeusedfortesting?•Whyarethosedevicesusedfortesting?•Arethedevicesavailablewithinthecompanyordoyouneedtoacquirethem?•Whatareyourreasonsforchoosingthosetestdevices?•Whataretherequirementsforthetestdevicesandthetestenvironment?•Isthereanupdatepolicyforthemobiledevices?•Whenwillnewdevicesbeintegratedintothedevelopmentandtestingprocess?•Whataretheusagescenariosofthesystem?•Whatdoesthebackendsystemconsistof?•Isthetestenvironmentsimilartotheliveenvironment?•Canthetestenvironmentbeusedfortestingpurposesfromoutsidethecompanynetwork?

Manualandin-the-WildTestingAsyouhavelearnedsofar,mobiletestingrequireslotsofmanualtestingandtestinginreal-lifeenvironments.ThinkoftheexamplewiththeskiandsnowboardappfromChapter1,“What’sSpecialaboutMobileTesting?,”wheretestingonamountainisrequiredtoseeifthemobileappactuallyworksunderrealconditions.Manualtestinginthewildisessentialforyourmobileappandrequireslotsof

planningbeforehandtoavoiduselesstestscenarioswhiletestingonthemove.Youshouldthereforetrytoidentifycommonreal-worldusagescenariosforyourmobileappanditsfeatures.Writethescenariosdowninyourteststrategyandrankthembasedonpriorityandimportancetoyourproject.Furthermore,itisrecommendedthatyoudefinetestroutes,suchasbybus,

train,car,plane,orwhilewalking.Withintheseroutes,describepossiblescenariosthatshouldbetested.Theseroutesallowyoutosimulaterealmobileuserswhiletheycommutetoworkorwhilethey’retravelingaround.Lastbutnotleast,youshoulddefinedatanetworkscenariosbasedonyour

targetgroup.Aftergatheringinformationaboutyourtargetgroup,youwillknowinwhichregionstheyliveandwhatkindofdatanetworksandspeedsareavailable,forexample,4G,3G,orEDGE.Basedonthatinformation,youcanlimitapptestingtothosenetworkspeeds,butdon’tforgettotestondifferent

networkproviders.Hereareafewexampleusagescenarios:•Testyourappbasedonitsfeatures,suchasoutsideinsunnyplacesorinsideanoffice.•Testtoseeiftheappcanbeusedindifferentweatherconditions.•Usemultipleappssuchase-mail,chat,andnewswhileyourappisrunninginthebackground.Checktoseeiftheappisinfluencedbyotherapps.

Herearesomeexamplesofroutescenarios:•Usetheappwhilecommutingtoworkbytrain,bus,orcar.•Usetheappwhilerunningandcheckthebehaviorofthesensors.•UsetheappwhilewalkingthroughacityorthecountrysideandcheckthesensorsandtheGPSorcompass.

Herearesomeexamplesofdatanetworkscenarios:•Testhowtheappworksinfastdatanetworkslike4Gor3G.•Testhowtheapphandlesthedatanetworkchangefrom4Gto3Goreven2G.•Testhowtheapphandlespacketlossoracompletelossofnetwork.

In-the-wildtestingrequireslotsofmovementandisachallengingtaskthatneedstobehandledduringtheapp’sdevelopmentprocess.Ifyoudon’thavethetimeoroptiontotestyourappinreal-worldscenarios,thinkaboutusingcrowdtestingintandemwithyourin-housetestingwork.However,remembertheprosandconsofcrowdtestingfromChapter6,“AdditionalMobileTestingMethods,”astheycanhaveanimpactonyourprojectplanning,coordination,timing,andbudget.Don’twritecomplextestcasesandscenarioswithexactstepsforin-the-wild

testing.Whileyou’reonthemove,you’reunlikelytohavealaptopwithyoutocheckthetestcases.Thisisinefficientbecauseitwilldestroythereal-lifetestingscenarios,behavior,andusersimulation.You’llprobablyhaveabagwithyoucontaininglotsofdevicesforyourin-the-wildtesting.Whenplanningsuchscenarios,keepthemshortandinformativesothattheycanbeusedwhileonthemove.Irecommendthatyouprintoutthescenariosorwritethemdownonapieceofpapersoyoucanjustreadthemandkeeptheminmind.Thefollowinginformationaboutin-the-wildtestingcouldformpartofyour

teststrategy:

•Defineanddescribetheusagescenariosofyourcustomers.•Defineanddescribethetestscenariosinthewild.•Defineanddescribethedifferentdatanetworksthatneedtobeusedfortesting.

MobileChecklistsandToursYoucanaddmobilechecklistsandmobiletourstoyourmobileteststrategy.AsdescribedinChapter4,mobilechecklistscanbeveryimportantforyourmobileappastheyhelpyoukeeparecordofthingsthatcan’tbeautomatedorarelikelytochangeonafrequentbasis.Ifyouknowtherequirementsandfeaturesofyourapp,you’llprobablyalsoknowthepartsoftheappthatneedtobetestedrepeatedly.Inthatcaseit’sgoodtoaddthosefeaturestoachecklistaspartofyourteststrategy.Basedonthefeaturesofyourapp,it’susefultodefinemobiletestingtoursto

concentrateyourtestingeffortsonspecialpartsofyourapp.CemKaner3describesatouras“...anexplorationofaproductthatisorganizedaroundatheme.”

3.http://kaner.com/?p=96

Usingtoursinyourmobiletestingworkhelpsyoutoexploreandunderstandhowthemobileappworks.Italsohelpsyoucomeupwithnewtestideaswhileyou’retestingtheapp.PleasereferbacktoChapter4foradescriptionofsometoursbackedupwithmnemonics.Herearesomeexamplesoftestingtours:•Featuretour:Exploreandtestallofthepossiblefeatureswithintheapp.•Configurationtour:Exploreandtesteverypartoftheappthatcanbeconfigured.•Gesturetour:Useeverypossiblegestureoneveryscreentoseehowtheapphandlesthedifferentinputs.•Orientationtour:Changetheorientationofeveryscreenfromportraittolandscapeandviceversatoseeifanyproblemsarise.

Iusethefollowingmobilemnemonicsinmyprojects:•FCCCUTSVIDS(http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html)fromMichaelKelly•ISLICEDUPFUN(www.kohl.ca/articles/ISLICEDUPFUN.pdf)fromJonathanKohl

Thefollowinginformationaboutmobilechecklistsandtourscouldformpartofyourteststrategy:

•Whichchecklistswillbeusedintheprojectandwhy?•Describetheusedchecklistsandtours.•Describewhenthechecklistsandtoursshouldbeusedandbywhom.

TestAutomationTestautomationcanalsoformpartofamobileteststrategy.Dependingonthemobileappanditslifecycle,youmaynotneedtoautomateit.Ifthat’sthecase,it’simportantthatyoudocumentanddescribethereasonswhytestautomationisnotnecessary.However,ifyourapprequirestestautomation,youshouldstarttothinkabout

automationandthetoolsyouwanttouseassoonaspossible.ThinkbacktothedifferentmobiletestautomationconceptsinChapter5andtheirprosandcons.Choosethetoolthatbestfitsyourcurrentprojectsituationandwhereyouhavealreadygainedsomeexperiencewithoneofthetoolsorprogrammingskills,orwhereyouhaveamobiletestingsetuporenvironmentinplace.Thiswillsaveyoulotsoftimeandmoney.Describethemobiletestautomationtoolinyourteststrategyandwhyyou

choseitforthisproject.Basedonthedescribedfeaturesandrequirements,youcandefinethepartsoftheappthatshouldbeautomatedandwhichdonotneedtobe.Duringthenextstepyoushoulddescribethedevicesonwhichtheautomated

testhastorunandinwhatenvironmentthetestsshouldbeexecuted.Yourmobileappprojectshouldfindtherightbalancebetweenrealdevices

andvirtualdevicesifyou’renotabletotesteverythingonrealdevices.Amixtureofrealandvirtualdevicescanalsobemorecost-effective.Onceyou’vedefinedthedevicesandtestautomationenvironment,I

recommendthatyoualsodefinetestsuitestogroupthedifferentautomatedtestcasesbasedoncommonfeatures,areas,andrequirementswithinyourapp.Withthehelpofthetestsuitesyoucandecidewhichtestsshouldbeexecutedwhenandhowoften.Forexample,youcandefineasmoketestsuitecontainingseveralautomatedtestsfromeachpartoftheapptomakesureacommittothecentralcoderepositorydidn’tbreakanything.Thistestsuiteshouldbesmallandruneverytimethecodechangesinordertoreceivefastfeedbackandtoswiftlyinformthedevelopersofanyproblems.

Herearesomeexamplesoftestsuites:•Smoketestsuitecontainingafewtestscriptstocheckthebasicfunctionsoftheappandtoprovidefastfeedback•Mediumtestsuiteoratestsuitecontainingonlyspecificfunctionality•Fullregressiontestsuitecontainingalltestscriptstobeexecutedonceadayorduringthenight•Testsuitescontainingonlyuserscenariossuchastheregistrationorcheckoutprocess

Testsuitescanbeagreatwaytocreateabalancebetweenfastfeedbackandbroadtestcoverage.Whenthetestsuitesareinplace,definewhentheyshouldbeexecuted;for

example,thesmoketestsuitemustberunaftereverycommitbythedeveloper.Themediumtestsuitecouldbeexecutedeverytwohours.Thefullregressiontestsuiteshouldbeexecutedonceeverynight.Youshouldalsodefinewherethetestautomationshouldbeexecuted.One

suchsolutioncanbewheretheunittestsareexecutedonthedeveloper’slocalenvironmentbeforechangesarepushedtothecentralrepository.Theunittestsandend-to-endtestscanbeexecutedontheCIserver.Thefollowinginformationabouttestautomationcouldformpartofyourtest

strategy:•IsaCIserverusedforthebuildprocess?•WhichCIserverisusedandwhy?•Whichtestenvironmentisusedtoexecutethetestautomation?•Whichtestautomationtoolsareusedintheproject?•Arethetestsexecutedonrealdevicesandvirtualdevices?•WhichdevicesmustbeconnectedtotheCIserverandwhy?•Isacloudproviderusedforthetestautomation?•Defineanddescribethetestsuitesandgroups.•Defineanddescribethedifferentbuildtriggersandexecutiontimes.•Defineanddescribewherethedifferenttestautomationsuiteswillbeexecuted,forexample,everythingonalocaldevelopermachineoronlyontheCIserver.

ProductRisks

Everyprojectisexposedtodifferentkindsofrisk.Itisimportanttoworkoutbothprojectandfeatureriskssoyoucanthendefinepossiblesolutionsforthem.Considerboththelikelihoodofriskoccurrenceandriskimpact.Iftheproductrisksareclear,youcanstartimplementingariskanalysisapproachthatcanbeusedbythewholeteamwhendefining,implementing,andtestingnewfeatures.Thefollowinginformationabouttheproductriskscouldformpartofyourtest

strategy:•Whichpartsarecriticaltothebusiness?•Whatisthelikelihoodoftheproductrisksoccurring?•Howshouldthebusiness-criticalpartsbetested?•Whatisthepotentialimpactifacriticalproblemoccurs?•Howisthefeatureriskanalysisperformed?•Isthereadisasterplaninplace?

ImportantCreatingamobileteststrategyisnoteasyasitneedstocoverlotsofmobiletestinginformation.Yourstrategymayalsoneedtobemodifiedduringthedevelopmentprocessbecauseofchangedproductfeaturesorpriorities.

MobileLaunchStrategyAsimportantasthemobileteststrategyis,it’salsoreallyimportanttowritedownamobilelaunchstrategy.Launchinganappisnoteasy,andlotsofproblemscanoccurduringandafterithasbeenlaunched.Thispartofthechapterwilldescribetheimportantpre-andpost-launchactivitiesforamobileapp.

Pre-Launch—ChecktheReleaseMaterialAsmentionedinChapter4,youshouldthinkaboutputtingtogetherareleasechecklisttomakesurethatallthereleaseinformationisavailableandinplace.Youshouldalsoperformtheupdateandinstallationtestbeforesubmittingtheapptotheappstore.Thetestsyouperformbeforecommittingtheapptotheappstoreshouldn’tbe

limitedtotheappitself.Yourlaunchstrategyshouldoutlineifandhowthebackendservicesaretestedbeforeanewrelease.

Askyourselfthequestion“DoesthenewversionoftheapprequiresomenewbackendservicesorAPIcalls?”Ifso,arethoseservicesorAPIcallsalreadyliveinthebackendsystem?Ifnot,it’sverylikelythatyourappwillberejectedbytheappstorevendor.Ifbackendservicesareavailable,checkthenewandexistingfeaturesoftheappintheproductionenvironmentonefinaltime.Whentheappisreadyforlaunch,checkthereleasenotesandfeature

descriptioninthenewappstoreinformationmaterial.Readthroughthetextsandcomparethemwiththenewandexistingfeatures.Inthereleasenotesit’simportanttobespecificaboutthenewfeatures;describethemwellandexplainthemtotheusers.Itisalsoimportanttoprovidethereleasenotesandappdescriptionineverysupportedlanguage.Don’tforgettocheckthescreenshotsoftheapp.Theyshouldbeinthesame

languageasthenotes,bethesamesizeasthepreviousscreenshots,andhavethesamestatusbariconsshowingthetime,battery,andnetworkstate.Figure7.1andFigure7.2provideexamplesofwhatnottodo:thestatusbarcontainsdifferenticonsforthesameapp.

Figure7.1Appstorescreenshot

Figure7.2Appstorescreenshotofthesameappwithdifferentstatusbarinformationandsizes

Theappstorescreenshotsneedtohavethesamestatusbarandthesamelook-and-feelinordertoportrayaprofessionalimage.Ifthenewfeaturesaredescribedbymeansofavideo,watchthevideoagain

andlistentothefeaturedescriptionandinformationitprovides.Askifthereismoremarketingmaterialavailablethatneedstobechecked.Lastbutnotleast,thereisonereallyimportantpoint:don’treleaseyourappto

yourcustomersonaFridayeveningorjustbeforeyouleavetheoffice.Inmostoftheappstoresittakessometime(uptoseveralhours)beforeyourappislistedandavailablefordownload.Iftheofficeisempty,noonecanreacttoanycriticalbugsorreleaseproblemsthatmayoccurrightaftertherelease.Someappstoresdon’thaveanappsubmissionprocessandcanthereforereact

quicklytoproblemsbyreleasingahot-fixversionofyourapprightafterthelast

release.Whenpossible,releaseyourappeitherinthemorningorontheweekendsothatyouhavesometimetoreacttocriticalissues.Thefollowinginformationaboutthereleasematerialcouldformpartofyour

launchstrategy:•Defineanduseareleasechecklist.•Performtheupdateandinstallationtestagain.•ChecktomakesureallofthebackendservicesandAPIsareavailableontheproductionenvironment.•Definewhichnewandoldfeaturesshouldbecheckedagainontheliveenvironment.•Defineanddescribehowtheappstorematerial,includingreleasenotes,screenshots,andvideos,shouldbechecked.•Defineanddescribewhentheappshouldbesubmittedtotheappstore.

Post-Release—WhatHappensafteranAppLaunch?Thepost-releasephasestartsonceyou’vereleasedyourmobileapp.Duringthisphaseyouandyourteamshouldpayattentiontoseveralpointstogetfeedbackfromyourcustomersandtohandleanyquestionsorproblemstheysendyou.Thefirstthingyoushoulddorightafterreleasingyourappistodownloadit

andinstallitfromtheappstoretomakesureit’sworkingasexpected.IftheversionisOK,youshouldarchiveitonafileservertobeabletoinstallitagainlater,forexample,toreproduceproblemsorbugsthatmaybereported.Thisstepcanalsobehandledbyyourbuildpipelinewithinthecontinuousintegrationsystem.However,thereareafewmoreissuesthatyou,yourteam,andyourcompany

needtohandle.Thefollowingsectionsdescribetheactivitiesthatcanbeperformedafterreleasingyourappinordertogetfurtherinformationaboutyourcustomersandanypotentialproblems.Itisimportanttodefineandwritedownsuchitemsinyourapplaunchstrategy.

CommunitySupportProvidingcommunityorcustomersupportforareleasedproductisessential,especiallyifyou’reofferingapaidapp.Wheneverusershaveaproblem,theyshouldbeabletocontactsomeoneatyourcompanyorthemobileteamtogetanswerstotheirquestionsorproblems.Ifnoonelooksaftertheusersorcommunities,theywillwritebadreviewsorsimplystopusingyourapp.

Youshouldthereforemakesureyoumaneverypossiblesocialmediachannelandcheckforcustomerfeedback,questions,orproblemsthatneedtobelookedinto.Itisimportanttorespondtosuchqueriestogivecustomersthefeelingthatsomeoneislisteningtothem.Itisalsopossibletoselectsomeusersandaskthemquestionsaboutthenewversionoftheapp.Thegatheredfeedbackcanthenbeusedtoimprovetheappinfuturereleases.

ImportantIfyourcompanyhasacustomersupportdepartment,Irecommendthatyouspendabitoftimewiththemtogetafeelingforcustomerneedsandproblems.

ReviewsThenextchannelyouhavetomonitorafterreleasingyourappistheappstorereviews.Readthemcarefullytogetsomefeedbackandbugreportsfromyourusers.However,Ihighlyrecommendthatyouhandlethosereviewswithcare.Therearelotsofpeopleouttherewholiketowritenegativefeedbackaboutappsthatisnottrue.Wheneverausercomplainsaboutyourapp,trytoreproducetheprobleminordertobeabletocreateabugreportaimedatfixingtheissueinasubsequentrelease.Ifyou’renotabletoreproducetheissue,trytoreplytothereviewertoaskspecificquestionsinordertogetmoreinformation.However,notallappstoresprovideareplyfunctionwithintheirreview

sectionsothatyoucantalktoyourcustomers.Iftheappstoredoesn’tprovidesuchafeature,youcanwriteyourownreview,statingthatyouarethedeveloper,tester,orproductmanageroftheappandwouldlikefurtherinformationorareofferingsomesortofsolutiontotheproblem.However,besureyou’reallowedtowritereviewswithintheappstoretoreplytopreviousreviews.Pleaserefertotheappstorereviewguidelinestomakesureyou’renotviolatinganyrules.Ifyou’reabletoreplytoreviews,usethisfunctionalitytointeractwithyourcustomersandtolearnfromthem.Ifyougetlotsofnegativefeedbackduetoamisunderstandingwithintheapp

oritsfeatures,youcanalsoprovidesomesortoftroubleshootingguideortutorialwithintheappdescription.However,ifyougetlotsofnegativefeedbackaboutyourappbecausethecustomersdidn’tunderstandthefeature,youneedtorethinkthewholeapporfeature,performadditionalusabilitytesting,andprovideanupdateassoonaspossible.

CrashReportsAnothervaluablesourceofinformationisthecrashreportsofyourapp.IfyouimplementedtoolslikeHockeyApp,4crashlytics,5orTestFlight,6youshouldcheckthemafterthereleasetoseeifthereareanyproblemswithyourapp.Noteveryproblemthatoccursduringruntimewillendupbeinganappcrash.Ifyourapphasgoodexceptionhandlinginplace,errorswillnotbeshowntotheuserbutwillbecapturedbythecrashreportingtools.Thiskindofinformationisveryimportanttohelpyouimprovetheappinfurtherreleases.

4.http://hockeyapp.net/features/5.http://try.crashlytics.com/6.www.testflightapp.com

ThetoolsprovideaWebinterfacewherethetopappcrashesareranked,grouped,andcategorized.Theyshowthetotalnumberofcrashesandthenumberofusersaffectedbytheseproblems.Nearlyeverycrashreportingtoolprovidesnicegraphsshowing,forexample,appdistributionandappcrashesovertime.Someofthecrashreportingtoolsprovideanoptiontosendfeedbackfromwithintheapptothecrashreportingbackend.Furthermore,someofthetoolsofferthird-partyintegrationforabugtrackingsystem.Ifyouhaven’timplementedacrashreportingtool,someappstore

manufacturersprovidebasiccrashreportingfunctionalitythatcanbeusedasastartingpoint.

ImportantImplementacrashreportingtoolasitwillhelpyouandyourteamtogetmoreinsightintotheproblemsandcrasheswithinyourapp.

TrackingandStatisticsTogatherinformationaboutyourcustomersandtheirusageofyourapp,youshouldimplementsomesortoftrackingmechanismtocollectimportantdata.Thiskindofinformationwillbeaggregatedbytrackingtoolstogeneratestatisticsaboutyourappandfeatureusage.Ifyourappusesatrackingmechanism,checkthestatisticsgeneratedpost-release.Dependingonthetrackingimplementation,youcangetinformationsuchas:•Mobileoperatingsystemversion•Mobiledevicemanufacturer•Devicemodel

•Displaysize•Mobilebrowserversion•Numberofpageviews•Howoftenacertainfeaturewasused•Howoftentheregistrationprocesswasaborted

Withthehelpofthosestatisticsandnumbers,trytounderstandyourusers’behaviorsoyoucantweaktheappanditsfeatures.Thefollowinglistisasamplingofsomemobiletrackingtools:

•adjust(www.adjust.com/)•appsfire(http://appsfire.com/)•AppsFlyer(www.appsflyer.com/)•Clicktale(www.clicktale.com/)•iMobiTrax(www.imobitrax.com/)•MobileAppTracking(www.mobileapptracking.com/)

Asyoucansee,it’snoteasytocreatemobiletestandlaunchstrategiesasbothrequireandcontainlotsofinformationabouttestingtopicsandpre-andpost-launchactivities.Pleasekeepinmindthatsuchstrategiesarenotsetinstone.Youshouldreworkandadaptbothstrategieswheneverchangesoccurintermsoftheproduct,risks,oranyotherpriorities.Mobiletestandlaunchstrategydocumentsareanongoingprocess,andeveryteammembershouldberesponsibleforupdatingandextendingthem.

SummaryThemaintopicofChapter7wasthemobiletestandlaunchstrategies.Everymobileteamneedstestandlaunchstrategiesinordertorememberimportanttasksbeforeandafterthereleaseoftheapp.Whenestablishingamobileteststrategy,thefollowingtopicsshouldbe

coveredanddefined:•Requirementsdefinition•Testingscope•Testlevels•Testtechniques•Testdata•Targetdevicesandenvironment

•TestautomationWiththequestionsprovidedinthispartofthechapteryouarenowabletoset

upyourownmobileteststrategyforyourapporforyourcompany.Theotherpartofthischaptercoveredthemobilelaunchstrategy.The

importanceofthereleasematerial,includingfeaturedescriptions,screenshots,andanyothermarketingmaterial,wasdescribed.Withthehelpofthequestionsprovided,youshouldfinditveryeasytocheckwhethereverythingisavailablefortherelease.Afterthereleaseoftheappitisimportanttohavecommunitysupportin

place,whereuserscanaskquestionswhentheyhaveanykindofproblemwithyourapp.Furthermore,theimportanceofcrashreports,tracking,anduserstatisticswasestablished.

Chapter8.ImportantSkillsforMobileTesters

Softwaretestersandmobiletestersinparticulararefacingmoreandmorerequirementsasmobileappsarebecomingincreasinglycomplexandthetimetomarketisgettingshorter.Mobiletestersneedtobeabletoefficientlytestcomplexmobileapplicationswithinaveryshortperiodoftimetodeliverexcellentproductstocustomers.Besidestestingknowledge,mobiletestersmusthaveseveralotherimportantskillstomanagethecomplexityofthesystemsandthehugenumberofdifferentscenarios.Thischapterisallaboutsoftwaretestingskillsandhowtoimproveyour

mobiletestingskillstobecomeabettermobiletester.

SkillSetofaMobileTesterBesidesknowledgeandskillsregardingsoftwaretestingmethods,approaches,mobileapps,anddevices,testersmusthaveasolidbasicsetofsoftskillsiftheywanttobesuccessfulinthemobiledevelopmentbusiness.Inthefollowingsectionsofthischapteryou’llreadaboutvariousskillseverysoftwaretestershouldhave.

CommunicationTheabilitytocommunicateisoneofthemostimportantskillsasoftwaretestermusthave.Softwaretestersmustbeabletodescribetheirtestingworktodifferentkindsofpeopleatdifferentlevelswithinthecompany.Theyneedtobeabletotalktodevelopers,designers,productmanagers,projectmanagers,othersoftwaretesters,andmanagementaswellastocustomers.Talkingtoothersoftwaretestersanddevelopersrequirestechnicalskillsanddetailedknowledgeofthedifferentpartsofthesoftware.Reportingbugsandshowingotherstheirmistakescanquicklyleadtonegativeemotions,whichiswhyit’simportanttoreportbugsinaclearandconcisewaywithoutbringingemotionsintotheconversation.Talkingtoproductmanagers,projectmanagers,designers,ormanagement

requirestheabilitytodescribeproblemsandbugstonontechniciansatahigherlevelinawaythatisclearandunderstandable.Inadditiontoverbalcommunication,softwaretestersneedstrongwritten

communicationskillsastheyhavetobeabletodescribeproblemsandbugsinsuchawaythateverypotentialstakeholdercanunderstandthem.

Listeningisalsoanessentialpartofcommunication,andsoftwaretestersmustbeabletolistencarefullywhenotherpeoplearetalkinganddescribingtheirthoughtsorproblems.Itisimportantthatyoudon’tinterruptotherpeoplewhenthey’retalking.Ifyouhaveanyquestions,makeanoteofthemandaskthemoncethepersonhasfinished.Tohelpyouimproveyourcommunicationskills,Irecommendthatyouread,

read,andread—notjustbooksbutblogs,newspapers,andanyotherkindofmaterialthatwillhelpyouimproveandexpandyourvocabulary,especiallyifyou’renotcommunicatinginyournativelanguage.IalsorecommendthatyouwatchmoviesorTVseriesinotherlanguagesto

bolsteryourvocabulary.Ifyouhavemoretimeavailableandwanttospendsomemoney,youcanattendlanguageandcommunicationclasses,whichareagreatwaytoimproveyourlanguageandcommunicationskills.Lastbutnotleast,Irecommendthatyougivetalksatusergroups,

conferences,orwithinyourcompany.Suchexperiencewillhaveamajorimpactonyourcommunicationskillsbecauseyougetfeedbackfromyouraudiencerightaway.Poorcommunicationgenerallyleadstodisagreementandmisunderstandings,

whichcanbeavoidedbyfollowingsomesimplerules:•Listencarefully.•Don’tinterruptotherpeoplewhilethey’respeaking.•Don’tspeaktooloudly.•Don’tspeaktooquickly.•Speakclearlyandprecisely.•Makeeyecontactwithyouraudience.•Don’tgetpersonalwhencommunicatingwithotherpeople.•Beabletocommunicateondifferentlevels,rangingfromtechnicaltonontechnicalaudiences.•Improveyourvocabularybyreadingbooks,blogs,andnewspapers.

ImportantSoftwaretestersneedtobediplomats,technicians,andpoliticiansallrolledintooneastheyhavetobeabletotalkandlistentodifferentstakeholderswithinthecompany.

CuriosityIt’shumannaturetobecurious,andsoftwaretestersneedtobecurioustoexplore,discover,andlearnnewthingsaboutthesoftwarethey’retestingandtheproductdomain.Acurioussoftwaretesterexploresthesoftwaretogetasmuchinformationaspossibleoutofthesystemtoidentifypotentialproblemsandraiseinterestingquestionsabouttheproduct.It’simportanttogobeyondtheusualsoftwaretestingapproachesandmethodstodiscovernewthings.Tobeabletodiscovernewthings,it’simportanttobeopentonew

technologiesandwillingtotrynewapproachesandmethods.Acurioussoftwaretesterdoesn’trelyonstatementsfromotherpeople;heorshequestionsthemtogainmoreinformation.Ifyou’dliketotrainyourcuriosity,Irecommendthatyoudownloadarandom

mobileapporsoftwareapplicationandstartexploringitsfeatures.Trynewapproachesandmethodswhileyou’reexploringthesoftware.Trytobreakthesystemandstartquestioningthefeatures.Makenoteofeverythingthatfeelswrongorevokessomesortofstrangereactionduringyourexplorationsoyoucanraisequestionsorpointoutanypossibleproblemswiththesoftware.

ImportantBecurious;exploreanddiscovereverypartofthesoftwaretoraiseproblemsorquestions.Don’trelyonstatementsfromotherpeople;questionthem.

CriticalThinkingAnotherreallyimportantskilleverysoftwaretestermusthaveiscriticalthinking.Withthehelpofcriticalthinkinggoodsoftwaretestersareabletoseethelargercontextofthesoftwareanditsfeatures.Theyarealsoabletobreakdownthesoftwareortherequirementsthroughanalysisandreflection.Thisisveryimportanttogainadeepunderstandingoftheproductandtofocusontherighttestingwork.ThefollowingquotefromMichaelBoltondescribescriticalthinkinginanice

way:“Criticalthinkingisthinkingaboutthinkingwiththeaimofnotgettingfooled.”1

1.www.developsense.com/presentations/2010-04-QuestioningBestPracticeMyths.pdf

It’simportanttoquestionyourownthinking,testingmethods,andapproachesaswellasyourowndecisionsandthesoftwarethatneedstobetested.Ask

yourselfthefollowingquestions:•Whatistheproblemofthisfeature/software?•Isitreallyaproblem?•Whyhaveyoutestedthisfeaturethatway?•Haveyouthoughtaboutthis?•Areyousureaboutthis?

Averygoodthree-wordcriticalthinkingheuristicfromJamesBach2isHuh?Really?So?Eachwordsuggestsaninvestigationpatternthatindicatesassumptions,sloppyinferences,andmisunderstandings:

2.www.satisfice.com/

•Huh?•Doyouunderstandwhatothersaretalkingabout?•Isitconfusing?•Isitvague?

•Really?•Isitfactuallytrue?•Whatevidencedowehaveforit?

•So?•Whydoesthismatter?•Towhomdoesitmatter?•Howmuchdoesitmatter?

Usethiscriticalthinkingheuristicinyourprojectandstartquestioningyourownworkandthemobileapp.Forfurtherinformationaboutcriticalthinking,takealookattheslidesfromMichaelBolton’scourse,CriticalThinkingforTesters.3

3.www.developsense.com/presentations/2012-11-EuroSTAR-CriticalThinkingForTesters.pdf

TenacityReportingbugsorraisingissuescanbeexhaustinganddifficult.Noteveryissuefoundbyasoftwaretesterwillbefixed.Theissuemaynotbeimportantenoughforotherteammembers,orperhapsthereisnotenoughtimetofixit.Itispartofthesoftwaretester’sjobtobetenaciousandfighttogetbugsresolved.Ifthesoftwaretesterthinksabugmaybecriticalforthecustomersorthesystem,heorsheneedstoinitiateadiscussiontodescribeandexplainwhyitneedstobefixed

inthenextrelease.Thekeywordshereare“BugAdvocacy.”TheAssociationforSoftwareTestingprovidesatrainingcourseonthisimportanttopic.4IfyouwanttogetafirstimpressionofBugAdvocacy,takealookattheslidesfromCemKaner.5

4.www.associationforsoftwaretesting.org/training/courses/bug-advocacy/5.www.kaner.com/pdfs/BugAdvocacy.pdf

Highstresslevelsarecommonbeforeareleaseandoftencausedevelopersorprojectmanagerstoneglecttheagreed-uponsoftwarequalitystandards.Insuchsituationssoftwaretestersmustbetenaciousandexplainorraisebugsoverandoveragainuntilthesoftwarequalitystandardshavebeenmet.Butbecarefulwiththisasyoumayendupbeingconsideredanuisance.Hereit’simportanttorelyonyourstrongcommunicationskills.Softwaretestershavetobetenaciouswhiletestingsoftwaresuchasmobile

apps.Dependingonthekindofappbeingtested,suchasagame,itisverylikelythatcertaingamelevelshavetobetestedoverandoveragaininordertobesurethateachlevelworksasexpected.Thiscanalsobeveryexhaustingbutrequirestenacityorsometestautomation.

ImportantBetenaciousduringtestingandduringpossiblediscussionsaboutbugsanderrorswithintheapplication.

ConstantLearnerThecurrentmobileworldandtechnologyarechangingrapidly.Tomaintainpacewiththisenvironment,softwaretestersand,inparticular,mobiletestersmustbeabletoadaptandlearnnewthingsreallyquickly.Softwaretestersneedtotakenoteofchangestakingplacearoundtheminordertoadaptandlearnnewapproaches,methods,andtechnologies.Tokeeppaceandlearnnewtechniquesandtools,softwaretesterscanread

blogsorbooksandattendconferencesandtrainingcourses.Ontheotherhand,it’simportantthatsoftwaretestersbeabletolearnduringtheirdailyjob,whiletestingsoftware,andwhileusingtoolssuchastestautomationtools.Wheneveranewtool,technique,ortechnologyentersthemarket,everysoftwaretestershouldbemotivatedtogatherinformationaboutthesenewitemsandtolearnaboutthem.

Important

Learningandimprovingpersonalskillsshouldbealifelonghabit.

CreativityAnotherimportantskillasoftwaretestershouldhaveiscreativity.Itisimportanttobeabletogeneratecreativeideastotestsoftwareinverydifferentwayssoastofindmorebugsandprovidetheteamwithusefulinformation.Thecreativityprocessstartswithdesigningthetestcasesandtestdata.Softwaretestersneedtothinkindifferentwaystofindeveryconceivableuseforapieceofsoftware.Whenthedefaulttestingapproachiscompleteandthere’ssomeprojecttime

leftfortestingactivities,Irecommendthatyoutestthesoftwareagainfromacompletelydifferentpointofview;forexample,walkthroughthebugsagaintogeneratenewtestingideas,ortalktocolleaguesorbetatesterstogetnewideasforyourtestingwork.Trytobecreativewithyourdatainputs,whenusingthenavigation,oranythingelsethatcomestomind.You’llbesurprisedabouttheresultsfromthatkindoftestingandwillnodoubtcomeacrosssomemorebugs.

ImportantMobiletestersinparticularhavetobecreativeinordertousemobiledevicesindifferentwaysbypayingattentiontoalltheinterfaces,sensors,andlocations.

CustomerFocusEverysoftwaretestershouldhaveastrongcustomerfocus.It’simportantthatsoftwaretesterstrytothinklikeacustomerinordertodeterminewhetherthesoftwarebeingtestedmeetscustomerneeds.Testersthereforeneedtohavelotsofpassionanddeterminationandbeabletoidentifystronglywithcustomers.Astrongcustomerfocusrequiresyoutobeaproductandfieldexpertwithin

yourteam.Youalsoneedtohaveanoverviewofreleasedfeaturesandfunctionalityandbeabletokeepaneyeoutforfuturereleases.Itisveryimportanttobeawareofcustomerbehaviorinordertoknowwhichfeaturesandfunctionalitytheyuse.Ifpossible,softwaretestersshouldtalktocustomerstodeterminetheirneedsandproblems.Thiscanbeachallengingjob,sosoftwaretestershavetobepatient.Whensoftwaretestershaveastrongcustomerfocus,theycancontributetheir

knowledgetoeveryphaseofthesoftwaredevelopmentprocess,whichinturnhelpstobuildbetterproducts.Tohelpimprovecustomerfocus,Irecommend

spendingacoupleofweeksworkingwithyourcustomersupportdepartmenttogetabetterfeelingforcustomerneeds.

ProgrammingandTechnicalSkillsThefactthatsoftwareproductsandmobileappsarebecomingincreasinglycomplexleadstothechallengethatmobiletestersalsoneedtohavesolidprogrammingskillsastheyhelpsoftwaretesterstounderstandthesystemundertest,tocommunicatewithdevelopersatcodelevel,toreviewcodefromdevelopersorothersoftwaretesters,andtowritetestautomationcodewhichisnowbecomingessentialineveryproject.Mobiletesterswithnoprogrammingskillsneedtotrainthemselvesbyreading

abookaboutprogramminglanguagesorpatterns,byfollowingaprogrammingtutorialontheInternet,orbyattendingaprogrammingcourse.It’salsopossibletoaskadeveloperifheorshecantrainthemobiletesterwithinaprojectorcompany.Thankstoprogrammingskills,mobiletestersareabletowritetestautomation

codefromunittoend-to-endlevel.Theyareabletoattendcodereviewstoasktechnicalquestionsandarelikelyabletowriteshellscriptsinordertoautomateeitherabuildpipelineoranyothertaskthattheteamneedstoperform.Besidescodingskills,everymobiletestermustbeabletounderstandtechnical

systemarchitecturesinordertobeabletoaskcriticalquestionsaboutthearchitectureandtoknowhowtotesteverypartofit.

ImportantEverymobiletesterneedsprogrammingskillsinordertobeabletowritetestautomationcodeandtobeabletoattendcodereviewsandtechnicaldiscussions.

HowtoImproveYourMobileTestingSkillsAsmentionedinseveralpartsofthisbook,themobileworldischangingrapidly,soyou’llneedtohoneyourskillseverydaytokeeppacewiththemobiletestingworld.Youhavetoconstantlylearnnewthingstogeneratenewtestingideas,tocollaboratewithdevelopersduringtheirworkwithprogrammingskills,andyou’llalsoneedtounderstandcustomerneeds.Inordertoimproveyourmobiletestingskillsitisimportanttohaveatleast

onemobiledeviceavailable.Inmostcasesthisisyourprivateandpersonal

device.Ifpossible,Irecommendthatyouhaveseveraldevicesavailableathomewithdifferentmobileplatformssoyoucanlearneverythingaboutthoseplatforms.Youdon’tneedtokeepbuyingthelatestdevices;youcansettleforusedphonesorevenolderversionstolearnabouteachplatform.Ifyou’renotinapositiontobuylotsofdevices,keepinmindtheOpenDeviceLabs,whereyou’reabletoborrowdifferentdevicesforfree.

LearnfromOtherAppsAveryeasywaytoimproveyourmobiletestingskillsistolearnfromotherapps.Irecommendinstallingandusingasmanyappsaspossiblefromdifferentcategorieswithindifferentappstorestoseehowtheyworkandbehave.Checkhowotherappshaveimplementedtheirnavigationandupdatemechanismandhowtheyusemobile-specificfeaturessuchasthecameraorothersensors.However,besidesusingthem,it’simportanttochecktheupdatetextsofthose

apps.Irecommendthatyouunchecktheautomaticupdatefunctionalityofallofyourappssoyoucaninstallnewversionsmanually.Beforepressingtheupdatebuttoninthedifferentmobileappstores,readtheupdatetextsandappdescriptionscarefully.Therearelotsofcompaniesordeveloperswhoarereallyprecisewhendescribingwhatthenewversionoftheappisallabout.Theydescribewhichbugsarefixedwiththenewversionandwhichnewfeatureshavebeenadded.Ifthereisabugdescriptioninanapp’supdatetext,trytoreproducethebugso

youcanseeitwithyourowneyes.Thiscanbealotoffun,butyoumayfindittakesawhiletoprovokethebug.Butthisisinitselfagreatwaytolearnlotsofnewthings.Youwillprobablygetnewtestingideas,comeacrossnewwaystouseanapp

andnewapproachestoprovokingabug,andlearnthingsthatyoumayneverhavethoughtaboutbefore.ThefollowingsectionswillprovidesomeexamplesinvolvingdifferentkindsofbugsanddescriptionsfrommobileappsIcheckfromtimetotime.

ImportantTheappscreenshotsinthe“CrashesonSpecificDevices”sectionareanonymized.AlloftheexamplesarebasedontheGooglePlaystore.However,thesamesortofbugandfeaturedescriptioncanbefoundineveryothermobileappstore.

CrashesonSpecificDevicesTheappinFigure8.1crashedwhenusingAndroidversion4.3andonsomex86devices.Ifyouwanttoreproducethiscrash,you’llneedtogetadevicewithAndroid4.3.Ifsuchadeviceisavailable,findthesectionorappviewthat’scrashing.ThismayprovetoberatherdifficultastherearelotsofdeviceswithAndroid4.3availableandeverydevicecanbehavedifferently.Ifyoumanagetofindthecrash,trytounderstandwhyithappened.Maybeit’sduetoapoorInternetconnectionorjustabadimplementation.

Figure8.1AppcrashesonAndroid4.3andx86devices

Nevertheless,asshowninFigure8.2,someappshaveproblemsonlywithacertainversionontheAndroidplatform,sotestingonseveralmobileoperatingsystemsisveryimportant.

Figure8.2Appcrashesonsomedevices

KeyboardsAsImentionedinChapter3,“ChallengesinMobileTesting,”usersareabletoreplacesystemappssuchasthekeyboardappwithathird-partysolution.ThiscanleadtovariousproblemsasshowninFigure8.3,whichcontainsabugreport.Toreproducethisissue,youneedtoinstallthethird-partykeyboardandstarthuntingthebug.

Figure8.3Appproblemsduetoalternativekeyboards

WidgetsSomemobileplatformssupporttheuseofwidgets.Whenprovidingawidget,besurethatitwon’tfreeze,crash,orconsumetoomuchbatterypowerasshowninFigure8.4.

Figure8.4Widgetconsumestoomuchbatterypowerandfreezes.

PerformanceAsIhavementionedinseveralpartsofthisbook,theloadingtimeandperformanceofanappareessentialtoasuccessfulapp.TheappdescriptioninFigure8.5providessomeinformationaboutpossibleperformanceissuesonthestatisticspage.Toreproducetheperformanceissueofthisappyouneedtogettwoverysimilardevices.Onthefirstdeviceyoushouldkeeptheoldversionoftheappwhileinstallingtheupdateontheotherdevice.Thenyoucancomparetheperformanceofthedescribedsectionaswellastheloadingtimestoseeifthereisanimprovement(seeFigure8.6).

Figure8.5Performanceissuesinsomesectionsoftheapp

Figure8.6Loadingperformanceoftheapp

LoginandPaymentIfyourappprovidesaloginfeatureormobilepaymentprocess,itiscriticalthatthosefeatureswork.Ifyouruserscan’tloginorbuyanything,youwilllosemoneyandharmyourreputation.Featurescriticaltoyourappmustworktothemaximumpossibleextent,soyouneedtobesurethatthey’rewelltestedandcoveredbytestautomation.TakealookatthescreenshotinFigure8.7,whereyoucanseethatthemobileappproviderhadanissuewithitssubscriptionmodel.

Figure8.7Checkoutproblemswithpremiumsubscription

PermissionsAsImentionedinChapter4,“HowtoTestMobileApps,”it’sveryimportantthatyouuseonlythemobileapppermissionsyoureallyneedforyourfeaturestowork.Ifyouusepermissionsthatusersdon’tunderstandorthataren’trequired(seeFigure8.8),youwillprobablygetlotsofbadreviewsinthestores,anditmayevennegativelyimpactyourapp’ssecurity.Besuretocheckthepermissionsagainbeforereleasingyourapp.

Figure8.8Usingpermissionsthataren’trequired

MobileDeviceHardwareUsageIfyourappusesmobile-device-specifichardwarefeaturessuchasthecamera,besurethatthefeatureworksonasmanydevicesaspossible.AsyoucanseeinFigure8.9,theappproviderhadanissuewiththecamerapreviewonsomedevices.Testingforhardwaresupportondifferentdevicesisagoodtaskforcrowdtesters.

Figure8.9Camerapreviewnotworkingproperly

Sincecomingupwiththeideaofcheckingappupdatetexts,IhavecontinueddoingthiseverytimebeforeIupdatetoanewerversionofanapp.Thisissometimestime-consumingandfrustratingwhenI’munabletoreproducethedescribedbug,butithelpedmealotinimprovingmymobiletestingskills.JustbyreadingthroughtheappdescriptionsIlearnedsomanynewwaysofgeneratingmobiletestingideasaswellaslotsofnewwaystoprovokebugsduringmydailytestingwork.

ObserveAnotherwaytoimproveyourmobileskillsisbyobservingotherpeople.Watchotherpeoplewhilethey’reusingtheirmobiledevices.Trytoobserveotherpeoplewhenamongthepublic,forexample,onatrain,inthesupermarket,oranywherethey’reusinganapp.Itisveryinterestingtoseehowotherpeopleuseappsintotallydifferentways,andyoucanlearnallsortsofthingsfromyourobservationsthatyoucanreusewhiletestingyourappandplanningnewfeatureswithyourteam.Itwillalsohelpyoutogeneratenewtestingideasandbearotherusagebehaviorsinmind.

ImportantDon’tbetooobviousanddon’tstartstalkingpeoplewhileobservingthem.

DuringmyobservationsIhavenoticedthatmanypeopledon’tknowthata

navigationdrawercanbeopenedbyswipingfromtheleftofthescreentotheright.Theytouchthenavigationiconinthetopleft-handcornertoopenit.Irealizedthatnoteveryoneisfamiliarwitheveryfeaturethatamobiledeviceorapphastooffer.Youshouldthereforetakethetimetoobserveotherpeopleinthewildorintestlabstoseehowtheyusetheirappsandthenapplythisknowledgeinyourmobiledevelopmentprocess.AsastartingpointIrecommendthatyouobserveeitheryourcolleaguesor

yourfamily.Thiswayyou’lllearnalot,andtheyprobablywon’tmindyouobservingthem.

TakePartinCompetitionsandTestCyclesIfyou’reinterestedincompetingwithothersoftwaretestersfromaroundtheworldandlearningfromtheirtestingexperience,Irecommendthatyoutakepartinatestingcompetition.Therearelotsofcompetitionsthroughouttheyearwheresoftwaretesterscanregistereitherasateamoralonetotestapieceofsoftwareindifferentcategories.Thegreatthingaboutcompetitionsisthatyoucanshareyourknowledgeandlearnfromothersoftwaretesters.It’salsofuntocompetewithothersoftwaretesterstoseehowgoodyourtestingskillsarecomparedtotheirs.Igenerallyattendtestingcompetitionsinordertolearn.Idon’tcareaboutmy

finalcompetitionranking;Ijustwanttolearnandimprovemytestingskills.Ireallyliketoseeothertestersduringtheirworkandpickupnewtestingideas.Hereareacoupleoftestingcompetitions:•SoftwareTestingWorldCup(www.softwaretestingworldcup.com/)•Testathon(http://testathon.co/)

AnothernicewaytoimproveyourtestingskillsandcontributetothetestingcommunityisWeekendTesting.6WeekendTestingisaplatformwheresoftwaretesterscancollaborateandlearnfromthetestingcommunity.Asthenamesuggests,WeekendTestingtakesplaceontheweekendandisallabouttestingsoftwarefromdifferentperspectivesandsharingyourtestingworkwithothersoftwaretesters.HavealookattheWeekendTestingWebsitetocheckouttheupcomingtestingdates.

6.http://weekendtesting.com/

AsmentionedinChapter6,“AdditionalMobileTestingMethods,”crowdtestingisasoftwaretestingapproachwhereyouaddsoftwaretestersfromallovertheworldtoyourin-housetestingefforts.However,crowdtestingcanserveanotherpurposeaswell:Registeringasacrowdtesterandattendingtesting

cyclesisagreatwaytolearnasyougettoseeothermobileapps,theproblemstheappproviderfaces,andthebugsthatcomeup.Furthermore,itisveryinterestingtoseeifyourbugreportsaregoodenoughtobeacceptedbythecrowdtestingproviderandmobileappprovider.

ImportantTakepartintestingcompetitions,shareyourknowledgewithothersoftwaretesters,andregisterwithcrowdtestingplatformstoseehowothermobileappswork.Remember:Whileitmaybeinterestingtoseeifyourbugreportsareacceptedbytheprovider,yourfocusshouldbeonimprovingyourmobiletestingskills.

TheMobileCommunityandtheMobileWorldAsmentionedintheprevioussection,learningfromothersoftwaretestersandmobiletestersisagreatwaytoimproveyourownmobiletestingskills.Ithereforerecommendthatyoubecomeanactivepartofthemobilecommunity,forexample,byregisteringonsoftwaretestingplatformslikeSoftwareTestingClub.7

7.www.softwaretestingclub.com/

Everysoftwaretestingplatformalsohasamobilesectionwhereyoucanexchangenewsandviewswithothermobiletesters.It’salsoveryusefultojoinmobiletestinggroupsonvarioussocialmediaplatforms.Therearealwayslotsofgreatmobiledevelopersandtesterswhowanttointeractonacertaintopicorwhohaveaproblem.Helpingsomeonewithaproblemisagreatwaytocontributetothemobilecommunity.Anddon’tbeshyaboutaskingquestionsevenifyouthinktheymightbesilly.There’snosuchthingasastupidquestion!Ifyoudon’thaveaTwitteraccountyet,Ihighlyrecommendthatyoucreate

one.EverysoftwaretestingandmobiletestingexpertusesTwittertowriteaboutnewtestingideas,blogposts,andotherimportantinformationthatisboundtohelpyouduringyourdailyworkinglife.Mostmobiletestingexpertsalsoblog,soyoushouldsubscribetotheirupdatestogetthelatestinformationfromthem.InthenextsectionIwilllistafewblogsandbooksthatmaybeofinteresttoyou.Testingeventsandconferencesareanotherwayofexchangingwiththe

mobilecommunity.Lotsofsoftwaretestingconferencesarehostedallovertheworldwheresoftwaretestingandmobiletestingexpertsmeetandsharetheirknowledgeintheformoftalksorworkshops.Ifyouhavetheopportunitytodo

so,Ihighlyrecommendthatyouattendsomeconferencesandmeetothermobiletestersinpersontotalkaboutarangeoftopics.Youcanalsochecktoseeifthereareanytestingusergroupsinyourarea;thesearegenerallyfreetoattendandagreatopportunitytomeetothersoftwaretestersnearby.Ifyouworkinthemobiletestingbusiness,youcouldstartbloggingtokeepa

recordofyourexperienceandcontributetothemobilecommunity.Youcanhelpothermobiletesterswithyouracquiredknowledgeorblogaboutyourexperienceswhilebecomingamobiletester.Besideslearningfromothermobiletesters,it’sveryimportantthatyoustay

up-to-datewithregardtothelatesttechnologiesandfeaturesofthemobileoperatingsystemsanddevices.Youneedtoknowwhenthedifferentmobiledevicemanufacturersreleasenewdevicesandnewversionsoftheiroperatingsystems.Foracompleteoverviewofsuchfeatures,Irecommendthatyouwatchthekeynotevideospostedbythemajormanufacturers.Ialsorecommendthatyouuseasmanyappsaspossiblefromarangeof

differentcategoriessoyoualwayshaveanoverviewofpossiblenewfeaturesaswellasnewwaystoimplementanduseanapp.

ValuableSourcesThissectionofthechapterprovidesyouwithsomeinterestingsoftwaretestingcommunities,books,magazines,andblogsthatyoucanusetoimproveyourknowledge.

ImportantTheselistsarebynomeanscomplete.

ConferencesThefollowingconferencesareworthgoingto:

•AgileTestingDays(www.agiletestingdays.com/)•BelgiumTestingDays(http://btdconf.com/)•DutchTestingDay(www.testdag.nl/)•EuroSTAR(www.eurostarconferences.com/)•GoogleTestAutomationConference(https://developers.google.com/google-test-automation-conference/)•Iqnite(www.iqnite-conferences.com/index.aspx)

•Let’sTest(http://lets-test.com/)•MobileAppEurope(http://mobileappeurope.com/)•Øredev(http://oredev.org/)•STAREAST(http://stareast.techwell.com/)•STARWEST(http://starwest.techwell.com/)•TestBash(www.ministryoftesting.com/training-events/testbash/)•TestExpo(http://testexpo.co.uk/)

CommunitiesThefollowingsoftwaretestingcommunitiesareworthlookinginto:

•AssociationforSoftwareTesting(www.associationforsoftwaretesting.org/)•MobileQAZone(www.mobileqazone.com/)•SoftwareTestingClub(www.softwaretestingclub.com)•TestingCircus(www.testingcircus.com/)•uTestcommunity(www.utest.com)

BooksThefollowingbooksareworthreading.Notallofthemareaboutmobiletesting,buttheyareanexcellentsourceofknowledgeforsoftwaretesters.

•AgileTesting(http://lisacrispin.com/agile-testing-book-is-now-out/)byLisaCrispinandJanetGregory•BeautifulTesting(www.amazon.com/gp/product/0596159811?tag=sw-testing-books-20)editedbyTimRileyandAdamGoucher•ExploreIt!(http://pragprog.com/book/ehxta/explore-it)byElisabethHendrickson•HowGoogleTestsSoftware(http://books.google.de/books?id=VrAx1ATf-RoC)byJamesA.Whittaker,JasonArbon,andJeffCarollo•LessonsLearnedinSoftwareTesting(www.amazon.com/gp/product/0471081124?tag=sw-testing-books-20)byCemKaner,JamesBach,andBretPettichord•SpecificationbyExample(http://specificationbyexample.com/)byGojkoAdzic•TapintoMobileApplicationTesting(https://leanpub.com/testmobileapps)byJonathanKohl

MagazinesThefollowingmagazinesprovidegreatcontentfromexpertsinvariousindustries.Thesemagazinesfocusonspecificsoftwareandmobiletestingtopics.

•AgileRecord(www.agilerecord.com/)•ProfessionalTester(www.professionaltester.com/magazine/)•Tea-timewithTesters(www.teatimewithtesters.com/)•TestingCircus(www.testingcircus.com/)

BlogsThefollowinglistcontainsgreatblogsfromsoftwaretestingexperts:

•GojkoAdzic,http://gojko.net/•JamesBach,www.satisfice.com/blog/•MichaelBolton,www.developsense.com/blog/•LisaCrispin,http://lisacrispin.com/•MartinFowler,http://martinfowler.com/•MarkusGärtner,http://blog.shino.de/•ShmuelGershon,http://testing.gershon.info/•AndyGlover,http://cartoontester.blogspot.co.uk/•AdamGoucher,http://adam.goucher.ca/•ElisabethHendrickson,http://testobsessed.com/•JimHolmes,http://frazzleddad.blogspot.com/•LenaHouser,http://trancecyberiantester.blogspot.com/•EricJacobson,www.testthisblog.com/•StephenJanaway,http://stephenjanaway.co.uk/stephenjanaway/blog/•ViktorJohansson,http://therollingtester.com/•JonathanKohl,www.kohl.ca/blog/•RobLambert,http://thesocialtester.co.uk/•AlanPage,http://angryweasel.com/blog/•HuibSchoots,www.huibschoots.nl/wordpress/•RosieSherry,www.rosiesherry.com/

Thefollowingblogshavemultiplecontributors:•http://blog.inthewildtesting.com/

•http://blog.utest.com/(uTestemployees)•http://googletesting.blogspot.de/(Googleemployees)•www.ministryoftesting.com/testing-feeds/(agreattestingfeedcollection)•http://mobileapptesting.com/•http://webapptesting.com/

Andthenthere’smyblog:•www.adventuresinqa.com

SummaryChapter8wasallaboutskillsandtherequiredskillsetofamobiletester.Hiringmobiletestersisnoteasy,becausetheyareveryrare.Ifyouhavefoundsomeonewhocouldfityourcompanyandposition,besureheorshehasthefollowingsoftskillsinordertobesuccessfulintheroleofmobiletester:

•Communication•Curiosity•Criticalthinking•Tenacity•Constantlearner•Creativity•Customerfocus•Programmingandtechnicalskills

Ifyouareamobiletesterandwanttoimproveyourtestingskills,thischapterprovidedsomesuggestions.Withlotsofexamples,thesection“LearnfromOtherApps”showedyouhowtoimproveyourtestingskillsbytryingtoreproducebugsinexistingapps.Inaddition,itisimportanttobeactiveinthemobiletestingcommunity,tolearnfromothermobiletestersandtoshareyourknowledge.Attheendofthechapteranoverviewofimportantconferences,books,blogs,andmagazineswasprovided.

Chapter9.What’sNext?AndFinalThoughts

Welcometothefinalchapterofthisbook!Thischapterdealswiththequestion“What’snext?”What’sthenextbigthingmobiletestersaregoingtohavetodealwith?What

kindsofnewtechnologyarealreadyoutthere,areontheirway,orcouldarriveinthefuture?Arethereanynewtestingchallengesthatweneedtohandle?Arethereanynewtestingtoolsandtestenvironmentsonthehorizon?Toanswerthosequestions,thefollowingsectionsofthischapterdescribe

somenewtechnologiesthatarealreadyonthemarketorduetoarriveinthenearfuture.Tokeeppaceinthefast-growingworldoftechnology,it’simportantforyoutoknowwhat’sinthepipeline.Thefollowingsectionscanbeusedforfurtherinvestigationsandresearchif

needed.Nooneisabletopredictthefuture,butI’mconvincedthatthefollowingtechnologieswillbecomemoreandmoreimportantformobiletestersinthenextcoupleofyears.

InternetofThingsTheInternetofThings(IoT)referstotheinterconnectionofuniquelyidentifiableembeddedcomputingdeviceswithintheexistingInternetinfrastructuretoofferdifferentkindsofservices.ThingsintheIoTcanincludeawidevarietyofdevicessuchashumanmedicalimplants,biochipsforanimals,andcarswithbuilt-insensorsthatcommunicatewithoneanothertoexchangeinformationaboutthecurrenttrafficsituationorprovidedriverswithcertaininformationabouttheircars.TherearealsodevicessuchaswashingorcoffeemachinesthatcanconnecttotheInternetsoyoucanmonitorthemremotely.EverythingthatcanbeassignedanIPaddressandhastheabilitytoprovideandtransferdataoveranetworkisathingintheIoT.AccordingtoastudyconductedbyGartner,1therewillbeupto26billion

devicesontheInternetofThingsby2020.Andthatfiguredoesn’tincludecomputers,tablets,orsmartphones,whichwillreachthe7.3billionmarkby2020.Ifyoucomparethosenumbers,IoTdeviceswilldwarfalltheexistingsmartdevices.ThiswillofcoursegiverisetoawholenewindustrywithpeopletryingtoconnecteverythingtotheInternet.

1.www.gartner.com/newsroom/id/2636073

HerearesomepossibleusageareasandscenariosforIoT:

•Environmentalmonitoring:Sensorscanbeusedtomonitorwaterquality,soilconditions,ortheatmosphere.•Infrastructuremanagement:Bridges,railwaytracks,andwindfarmscanbemonitored.•Energymanagement:Industrialmanufacturerscanoptimizeenergylevelsinrealtime.•Medicalandhealthcaresystems:People’shealthcanbemonitoredremotely.•Buildingandhomeautomation:Alarmandheatingsystemscanbemonitoredandmanaged.•Transportsystems:Carscancommunicatewithoneanother,suchastoavoidtraffic.

InordertostandardizetheIoT,aconsortium2ofcompanieshasbeenformedtopushalongtheIoTanddevelopdefaultcommunicationstrategies,interfaces,andprotocols.ThefollowingtwosectionsprovidesomeexamplesofcurrentIoTdevicesandscenariosfromdifferentmanufacturers.

2.www.openinterconnect.org/

ConnectedHomeMobiledevicemanufacturersGoogleandApplearecurrentlybuildingtheirfirstIoTservicesandproductswhichwillformpartoftheIoTfamilyalongsideseveralothercompanies.In2014,GoogleboughtNestLabs,3whichbuildsintelligentthermostatsandsmokealarmsforsmarthomes.ThethermostatsandsmokealarmsareconnectedtoaWi-Finetworkandcanbeaccessedfromanywhereintheworldusingacomputer,tablet,orsmartphone.

3.https://nest.com/

Googleiscurrentlydevelopingmobileappsfordifferentmobileplatformsthatgrantaccesstoconnecteddevicesaroundthehome.Theproductisintelligentasitcanlearnfromuserhabitstocontroltheheatingsystembasedonthetimeofdayandthecurrentmonthanddeterminewhetherornottheuserisathome.Userscanalsodefinevariousscenariostocontroltheirentirehomeheatingsystembasedontheirneeds.BecausethedevicesareconnectedtotheInternet,it’seasytoauto-updatethemwithnewsoftwareversionsincludingbugfixesandfeatures.ApplehasintroducedHomeKit4withiOS8,whichprovidesaframeworkto

communicateandcontrolconnecteddevicesintheuser’shome.Userswillbe

abletoautomateandcontrolIoTdeviceswiththeirvoicebyusingSiri.HomeKitwillbeabletocontroldevicessuchasthermostats,lighting,doors,andotherelementsthatcanconnecttotheInternet.AppleprovidesthedevelopmentframeworkforHomeKitandiscurrentlylookingforindustrypartnerswhowanttoimplementApple’sHomeKitaccessoryprotocoltobecontrolledbytheHomeKitapp.

4.https://developer.apple.com/homekit/

Asyoucansee,connectedhomesarealreadyavailableonthemarketandenableentirelynewwaysofinteractingwithdevicesandpartsofourdailylife.Thereareplentyofpotentialnewtestscenarios,testenvironments,andtestingchallengesthataretotallydifferentfromtraditionaltestingormobiletesting.

ImportantBesidesGoogleandApple,therearealotofothercompaniesthatareinvestinginandalreadyhavesolutionsinplaceforconnectedhomes.IoptedtowriteaboutGoogleandApplebecausetheyalsoprovideAPIsfordevelopersthatallowthemtobuildmobileapplicationsaroundtheconnectedhometechnology.

ConnectedCarConnectedcarsarethenextIoTexample.Again,GoogleandApplearealreadyontheirwaytointegratingtheAndroidandiOSmobileoperatingsystemsintocarstomakethemevenmoreintelligent.GooglehasintroducedAndroidAuto,5andApplereleasedCarPlay.6BothGoogleandApplewillprovidealightweightversionoftheirmobileoperatingsystemstoprovideuserswithfeaturestheycanusewhiledriving,suchasnavigation,music,contacts,phone,andmessages.

5.www.android.com/auto/6.www.apple.com/ios/carplay/

Besidestheirownmobileapps,bothGoogleandAppleoffertheoptiontouseinstalledthird-partyappsincars.Therearealreadylotsofcarmanufacturersthatsupportbothsystemsandletbuyerschoosethesystemtheyprefer.However,usingmobileappswithcardisplaysgivesrisetosomenew

challengesforanyoneinvolvedinthesoftwaredevelopmentprocess.Forexample,theprovidedappsandfeaturesshouldn’tdistractthedriverandneedtoofferaverysimpleuserinterfacewithlessinformationcomparedtomobileappsorWebapplications.

Thefollowingfourpointsmustbeconsideredwhendevelopingandtestingmobileappsorotherapplicationsforconnectedcars:

•Simpleinterface:Carapplicationsandinterfacesshouldnotdistractthedriver.TheUIelementsmustbeeasytousewhilethedriverisatthewheel.Traditionalinputmethodsneedtobereconsideredandshouldalsoincludevoicecontrol.•Avoiduselessfeatures:Thefeaturesetofanapprequiredwhiledrivingisprobablyalotsmallerthanforanappusedonasmartphone.Theappshouldthereforeofferfewerfeaturesonthecardisplaytopreventthedriverfrombeingoverwhelmedorfrustratedwhileatthewheel.•Third-partyappsneedguidelines:CarmanufacturersneedtoprovideanAPIforthird-partydeveloperssotheirservicescanbeintegrated.However,thisposesthechallengeofcreatingverystrictguidelinesaboutwhatisandisn’tpossible.•Testinginthecar:Developingappsforcarsischallengingenoughinitself,buttestingappsforcarsisevenmorecomplex.Itissimplynotenoughtotesttheappinalabsituationbecausecarsaregenerallyonthemove,havedifferentmanufactureryearsandmodels,andhavelotsofinterfaceswithothersystems.Electronicinterferenceinacarcanhaveahugeimpactonyourappandthewholesystem.Theprovidedappmustbesafeforthedrivertousewhileatthewheel.Lastbutnotleast,theappmustbewelltestedtoavoidanycriticalbugs.

AgoodexampleofaconnectedcarandIoTisthecarmanufacturerTesla,7whichbuildscarsthatarecompletelyconnectedtotheInternetandcanbepartiallycontrolledwiththeaidofamobileapp.Thecarreceivesautomaticupdatesthatimprovefeatures,fixbugs,andevensolveproblemswithdifferentpartsoftheengine.WiredmagazinepublishedaninterestingarticleaboutTeslaasanexampleofIoT.8

7.www.teslamotors.com/8.www.wired.com/2014/02/teslas-air-fix-best-example-yet-internet-things/

Asyoucansee,thetwoexamples—connectedhomeandcar—representnewchallengesfortheentiresoftwaredevelopmentprocess.Fromatestingpointofviewinparticular,thesenewtechnologiesrequiredifferenttestingmethodsaswellasnewtestingenvironments,newtestingdevices,andcompletelynewtestingscenarios.

Wearables

Wearabletechnologyisarapidlygrowingfieldthat’sexpectedtogrowexponentiallyoverthenextfewyears.Therearelotsofnewandinnovativeformfactorsfordevicesthatcanbewornondifferentpartsofthebody.Thesenewformfactorsposenewchallengesforcompaniesastheylooktofindsmartwaystomaketheirproductfunctional,usable,andlovablefortheircustomers.Thesameappliestodevelopersandsoftwaretesterswhoneedtorethinktheirworkandthewaytheydevelopandtestsuchproducts.Wearablesgenerallyinvolvesmartwatches,smartglasses,andfitnesswristbands.

SmartWatchesandFitnessWristbandsSmartwatchesandfitnesswristbandsareextensionstomobiledevicesthatsendandreceiveinformationsuchasmessages,news,incomingcalls,andhealthstatustoandfrommobiledevices.Togetthelatestinformationfromanapp,usersnolongerneedtotaketheirdeviceoutoftheirpocket.Mostdevicescanbecontrolledbytheuser’svoiceorasmalltouchscreen.However,asmartwatchorfitnesswristbandisessentiallyuselesswithoutamobiledevicetointeractwith.Theusabilityanddesignofsmartwatchesandfitnesswristbandsneedtobe

thoroughlytestedandchecked.Designingsoftwareforreallysmallscreensisnoteasy,whichiswhydesignersandUXexpertsneedtorethinktheirconceptsinordertobuildniceproductsthatwearablesuserswillloveanduse.JonathanKohlwroteanexcellentarticleabouthislessonslearnedwhendesigningproductsforsmartwatchesandwearables.9

9.www.kohl.ca/2014/lessons-learned-when-designing-products-for-smartwatches-wearables/

Ifyouhavetheopportunitytotestwearables,especiallysmartwatchesorfitnesswristbands,youshouldkeepaneyeonthelook-and-feelofthedevicetogetherwiththesoftwareundertest.Thisalsoincludestestingthedesignandusability.Whendoingso,youshouldaskyourselfthefollowingquestionstogeneratevaluablefeedbackabouttheproduct:

•Isthedevicenicetowear?•Doestheappmakesenseonthewearabledevice?•Arethefeatureseasytouseandhelpful?•Docertainpartsofthedevicegetinthewaywhileyou’reonthemoveandwhileyou’reusingthesoftware?•Howcantheuserinteractwiththesmartwatchorfitnesswristband?

Thelook,feel,design,andusabilityarethemainsuccessfactorswhenitcomestowearabletechnology.Ifawearabledevicedoesn’tfeelgood,userswill

notbuyorwearit.Fromatechnologicalpointofview,testingsmartwatcheshassomeadditional

challengescomparedtomobileappsanddevices.Thefactthatsmartwatchesareextensionstomobiledevicesrequirestestingofthewearabledevicetogetherwiththesoftwaretoseehowbothcommunicatewithmobiledevicesinordertoreceiveandtransferdata.Thisscenarioisn’tsomethingyoucanautomate.I’msureyou’realreadyawareofmobiledevicefragmentation,butthisiscompoundedbythefactthatsmartwatchesandfitnesswristbandsneedtoworkcorrectlywithinadifferentsetofuniquedailyuserscenarios,andallthosescenariosrequireextensivein-the-wildtesting.Suchtestinginareal-lifeenvironmentwillplayanessentialpartinthesuccessofsoftwareforsmartwatches.GoogleintroducedAndroidWear10in2014tokickoffthewearabledevice

era.AppleintroducedtheAppleWatch11inSeptember2014andstartedsellingitinearly2015.IfyousearchtheInternetforsmartwatchesandfitnesswristbands,you’llcomeacrossvariousdevicemanufacturersandallthedifferentdevicesonthemarket.10.www.android.com/wear/11.www.apple.com/watch/

Forinformationaboutbuildingsoftwareproductsforwearabledevices,checkoutthePebbledeveloper,12theGooglewearable,13andtheAppleWatch14featurepages.12.https://developer.getpebble.com/13.https://developer.android.com/training/building-wearables.html14.www.apple.com/watch/features/

SmartGlassesGoogleGlassisanotherwearablethatwasintroducedbyGoogle.GoogleGlass15includesalmostthesamehardwareasamobiledeviceextendedwithanopticalhead-mounteddisplay(OHMD)tohavethecontentandinformationdirectlyinfrontofyoureyes.Theglassesareequippedwithlotsofsensorsandacameratointeractwithyoursurroundings.Youcancontrolthiswearabledevicewithyourvoiceorbyusingthetouchpadonthesideoftheframe.15.www.google.com/glass/start/

TheexplorereditionofGoogleGlasshasbeenonsaleinsomecountriessince2014,butlotsofcountriesandcompanieshaveexpressedprivacyconcernssinceitsintroductionin2012becausethedeviceisabletorecordpeopleinpublic

withouttheirpermission.Furthermore,thereareconcernsabouttheproductintermsofcorporatesecretsandsafetyconsiderationswhileusingitindifferentscenarios,suchaswhiledrivingacarorridingamotorbike.However,Googleprovidedanewwayofusingmobiletechnologiesandset

newstandardsandinnovationsintheworldofwearabledevices,eventhoughtheproductisn’treadyforthemassmarketyet.GoogleGlassisagreatexampleofthedirectiontechnologywilltakeoverthecomingyears.Ifyouhavetheopportunitytodevelopandtestsoftwareforsmartglasses,

don’tforgettorunthroughthelistofquestionsapplicabletosmartwatches.You’llalsoneedtorethinkyourtestingapproachforthisdevice.

HealthAppsAnotherinterestingandgrowingmarketismobilehealthapps.“ThenumberofmHealthappspublishedonthetwoleadingplatforms,iOSandAndroid,havemorethandoubledinonly2.5yearstoreachmorethan100,000apps(Q12014)”and“...willreach$26billioninrevenuesby2017,...”asquotedinthecurrentmHealthAppDeveloperEconomics16report.ThishugeincreaseshowsthatmHealthappswillbeontheriseinthenearfuture.16.http://mhealtheconomics.com/mhealth-developer-economics-report/

ThetopfourmHealthappsare:1.Fitnessapps(30%)2.Medicalreferenceapps(16%)3.Well-beingapps(15%)4.Nutritionapps(8%)

TheremainderofthemHealthappsaredistributedamongdifferentcategoriessuchasmedicalconditionmanagement,diagnostics,compliance,reminders,alerts,andmonitoring.Healthappsinvolvetheuseofmobiledevicesorwearablestomonitorthe

humanbodyforcurrentbloodpressure,pulse,heartrate,sleeppatterns,calorieconsumption,orcurrentspeedwhilerunning.ThehugenumberofmHealthappsonthetwoleadingplatforms,iOSandAndroid,hasconvincedAppleandGoogletoinvestinanddevelopmHealthAPIsandappsfortheirmobileplatforms.AppleintroducedHealth17foritscustomersandHealthKit18foritsdevelopersatthesametimeasitrolledoutiOS8.GoogleintroducedGoogleFit19in2014.ThefactthatbothAppleandGoogleareenteringthehealthmarketwillleadtoseveralnewdevicesandappsbeingrolledoutinthenearfuture.

17.www.apple.com/ios/whats-new/health/18.https://developer.apple.com/healthkit/19.https://developers.google.com/fit/

Asthenumbersshow,mostoftheseappshelpcustomerstotracktheirfitnessordietaryhabits,buthealthappsalsoposehighriskstocustomers.Appsthatmanageinsulindosesforpatientswithdiabetescouldhavedisastrousconsequencesifabugoccurs.Thisriskraisesanimportantquestion:Canwetrusthealthcareapps?MedicaldevicesaregenerallyregulatedbytheUnitedStatesFoodandDrug

Administration(FDA),butthisisnotthecasewitheverymobilehealthapp.ExpertsfromtheNewEnglandJournalofMedicine20saythattheFDAdoesn’thaveenoughresourcestoregulateallthehealthappsavailableinthedifferentmobileappstores.AnotherchallengethatisnearlyimpossiblefortheFDAandappproviderstohandleisallthemobileoperatingsystemupdatesprovidedbythedifferentvendors.Eachmobileplatformreceivesmorethanoneortwoupdatesayear,andeachoperatingsystemupdatemustbecompliantwiththeFDAregulations.20.www.nejm.org/doi/full/10.1056/NEJMhle1403384

Sotheanswertothepreviousquestionisno.Wecan’ttrusthealthappsiftheyaren’tregulatedbyanyinstitutionbecausewecan’tbesurethatthedelivereddataiscorrectandfreeofmistakes.Ifyouhavetheopportunitytotestmobilehealthandfitnessapps,pleasebear

thefollowingpointsinmind:•GetinformationfromtheFDAandothermedicalinstitutionswithregardtoregulationsandhealthcareworkflows.•Theprovideddatamustbecorrectinordertoprotecthumanlife.•Datasecurityisaveryimportantaspectduetotheprivacyofaperson’sstateofhealth.•Mobilehealthappsmusthaveexcellentusabilityinordertocoverthetargetgroup’sneeds.•Geolocationdatamustbecorrectforfitnesstrackers.

Besidesthat,allofthemobiletestingknowledgeyouhaveacquiredinthisbookalsoappliestohealthandfitnessapps.Ifyou’dliketofindoutmoreaboutmHealth,visitthemHealthNews21or

mobileHealthEconomics22WebsitesorhavealookatthemHealthAppDeveloperEconomics2014study.23

21.www.mhealthnews.com/22.http://mhealtheconomics.com/23.http://mhealtheconomics.com/mhealth-developer-economics-report/

FinalThoughtsThisisthelastsectionofmybookaboutmobiletesting,andI’dliketoprovideyouwithsomefinalthoughts.Duringthecourseofthisbookyou’velearnedagreatdealaboutmobiledevices,mobileapps,mobileusers,andthetoolsthatareimportantwhenitcomestomobiletesting.Ialsohopeyou’velearnedfrommyideasandexperiencesduringmytimeasamobiletester.Thisbookisdesignedtohelpyouinyourdailylifeasamobiletester,mobile

developer,orproductmanagerbygivingyoutheimpetustogeneratenewtestingideasandtryoutnewmobiletestingapproaches.Itshouldalsoserveasabasisfordevelopingyourowntestingideasandapproaches,whilealsohelpingyoutoextendyourknowledgelevel.Asyouhaveseen,themobiledevelopmentandtestingbusinessisrapidly

changingwithlotsofnewtechnologyenteringthemarketeverydayandplentymoreinthepipeline.Thisiswhyit’simportantthatyoustayup-to-date,strivetolearnconstantly,andadaptyourskillstotheever-changingworldoftechnology.

FiveKeySuccessFactorsToroundthingsoff,I’dliketoprovideyouwithmyfivekeysuccessfactorsforbecomingasuccessfulmobiletester.

SuccessFactor1:HaveHighExpectationsMobileusershavehighexpectations,andyoushouldalsohaveveryhighexpectationswhenitcomestomobileappsandtheirusability,performance,andfeatureset.Bearinmindthatmobilecustomerswilluninstallyourappveryquicklyifthey’renothappywithit,andthey’llprobablysubmitabadreviewtotheappstore.It’sthereforeimportantthatyoualwayskeepyourcustomersinmind,encouragegoodusabilityandperformance,andmakesurethatalltheimportantbugsgetfixed.KeeptheKIFSUprincipleinmindandlistentoyourcustomers’needs.

SuccessFactor2:BeanExpertonMobileDevicesAsuccessfulmobiletesterneedstobeanexpertonmobiledevices.Itisessentialtoknowallthedifferenthardwareandsoftwarefeaturesofmobiledevicesfromthevariousplatforms.Thisknowledgewillhelpyoutokeeplotsofdifferenttest

scenariosinmindduringyourdailybusiness.Ifyou’reabletodoso,buymobiledevicesusingthedifferentplatformssoyoucanstayup-to-date.Ifbuyingisnotanoption,trytorentthemfromamobiledevicelab.Youshouldalsosubscribetovarioustechnologyblogsandnewspagesin

ordertogetthelatestnewsaboutmobileoperatingsystemsandmobiledevices.Irecommendthatyouwatchthekeynotevideosofthemajormobilemanufacturerstofindoutthelatestinformationaboutthedifferentplatforms.

SuccessFactor3:BeontheMoveOneofthemostimportantpointsyoushouldbearinmindwhentestingamobileappistobeonthemovewhileyou’retesting.Yourcustomersusetheirmobileappsinmanydifferentscenarios,locations,anddatanetworks.Therefore,it’sessentialthatyoutestyourappinseveraldatanetworkswithdifferentnetworkspeedstoreplicatereal-lifescenarios.Whiletestingyourappinthewild,you’lldoubtlesscomeacrosslotsofdifferentproblemsthatwouldprobablynevershowupintheoffice.Whentestinganapponthemove,therewillbelotsofinterferencesthatcouldhaveanimpactonyourappwhenusingthevarioussensorsandinterfacesamobiledevicehasonoffer.Sogrababag,fillitwithmobiledevices,andstarttestinginthewildright

now!

SuccessFactor4:BuildUpYourProgrammingSkillsMobiletestersneedtobeabletowritetestautomationcode.Ifyoudon’thaveanyprogrammingskillsrightnow,doyourbesttogetintoprogrammingsoyoucanwritereliableandrobusttestautomationscriptsforyourmobileapp.Programmingskillswillalsohelpyousupportthedeveloperswiththeirregressiontests,andyou’llbeabletocommunicateanddiscusstheapp’scodewiththedevelopers.Ifyourprogrammingskillsneedbrushingup,now’sthetimetoreadsomeprogramminglanguagebooksorrunthroughsomeonlinetutorials.

SuccessFactor5:BeaConstantLearnerThefinalsuccessfactorinbecomingabettermobiletesteristobeaconstantlearner.Thisdoesn’tjustapplytomobiletesters;itshouldbethecaseforanyoneinvolvedintheITbusiness.Thetechnologiesusedtobuildcomplexsystemsincludingmobileappsarechangingconstantly.Furthermore,newwaysofusingandcommunicatingwithnewtechnologiesareontheirway,andit’simportantforyoutofindoutaboutthemassoonaspossible.

Besideslearningnewtechnologies,youshouldalsoworkonimprovingyourtestingskills.Greatwaystodothisincludereadinglotsofblogsandbooks,attendingconferences,andtakingpartincompetitionstolearnfromothermobiletestersandshareexperienceswiththem.Thiswillhelpyoutoimproveyourtestingideas,approaches,andskills.Don’tshyawayfromtryingoutnewthings—makemistakesandlearnfrom

them.

SummaryThelastchapterofthisbookcoveredthetopic“What’snext?”Whataretheupcomingtechnologytrendssoftwaretestershavetodealwith?Idescribedfivepossibletechnologytrendsthatarealreadyonthemarketorontheirway.Thefivetechnologiesare

•InternetofThings•Connectedhomes•Connectedcars•Wearables•Healthapps

Inthe“FinalThoughts”sectionIoutlinedfivekeysuccessfactorstobecomeasuccessfulmobiletester.Thosesuccessfactorsare

•Havehighexpectations.•Beanexpertonmobiledevices.•Beonthemove.•Buildupyourprogrammingskills.•Beaconstantlearner.

That’sit;thankyouverymuchforreadingmybook.Ihopeyoulearnedalotofnewthingsandgotnewideasforyourdailylifeasamobiletester.Happymobiletesting!

Index

0Gnetworks,121Gnetworks,122Gnetworks,12–133Gnetworks,134Gnetworks,135Gmobilestandard,1399tests,145

Accelerationsensor,testing,38AccessibilityguidelinesAndroid,62iOS,62WC3initiative,62WindowsPhone,63

AccessibilitytestingAndroid,62auditoryimpairment,61cognitiveimpairment,61–62iOS,62overview,60physicalimpairment,61visualimpairment,60–61WC3initiative,62WindowsPhone,63

Actualresult,inbugreports,98AdvancedUserInteractionsAPI,125Advertisements,selling,24AgileRecord,197AgileTesting,196AgileTestingDays,195AmazonAppstoreGuidelines,26Ambientlightsensor,testing,37

AMPS(AdvancedMobilePhoneService),12AMTS(AdvancedMobileTelephoneService),12Androiddesignguide,18guidelines,81

Androidappstestautomation.SeeTestautomationtools,Android.testingcriteria,103

AndroidAssetPackagingTool,66AndroidAuto,201AndroidEmulator,51Androidoperatingsystemaccessibilityguidelines,62accessibilitytesting,62AppQualityAlliancesite,103pseudo-randomtestevents,66

Androidsmartphonebatterystatus,displaying,18docksection,18generatingpseudo-randomtestevents,66interrupttesting,66–67introductionof,14vs.iOS,17–19JouleUnitenergyprofilingtool,65Monkeytool,66networkstrength,displaying,18notifications,displaying,18pinningapps,18statusbar,18stresstesting,66–67time,displaying,18userinterface,18

AndroidWebbrowservs.otherbrowsers,48Appdevelopmentdevelopmentguidelines,2–3

KIFSU(KeepItFastSimpleUsable)principle,3KISS(KeepItSimple,Stupid)principle,3rejection,reasonsfor,18testing.SeeMobiletesting.

AppQualityAlliancesite,103AppstoresAppleAppStore,25BlackBerryWorld,26GooglePlayStore,25listof,25MicrosoftStore,26numberofavailableapps,25rejectingapps,reasonsfor,18reviewandpublishingguidelines,26Samsung,25uploadingappsto,26forWindowsPhone,26

Appaloosa,138AppBlade,138AppiumforAndroid,127–128foriOS,134nativeobjectrecognition,111onlineresources,111,134

Applause,138,145AppleAppStore,25AppleAppStoreGuidelines,26AppleWatch,204“ApplyingtheSFDPOTHeuristictoMobileTesting,”88Appsdeveloping.SeeAppdevelopment.health,205–207mHealth,205–207numberavailable,2,25pinning,18

preinstalled,testing,44reasonsfordeleting,2system,testing,44testing.SeeMobiletesting.uploadingtoappstores,26WindowsPhone,downloading,26

Apps,Androidtestautomation.SeeTestautomationtools,Android.testingcriteria,103

Apps,iOStestautomation.SeeTestautomationtools,iOS.testingcriteria,103

Apps,typesofhybrid,20–21native,19–20Web,22–23

appsfire,177AppsFlyer,177AppThwack,36,149Appurify,149AssociationforSoftwareTesting,196Auditoryimpairment,accessibilitytesting,61Automatedtesting.SeeTestautomation.

Bach,James,88,90Bamboo,137Batteryconsumptiontools,64–66Batterylife,savingwithproximitysensor,37–38Batterystatus,displaying,18Batteryusage,testingbatteryconsumptiontools,64–66Charlestool,63Fiddlertool,64fullychargedbattery,63–64Instrumentsenergyusagefunction,65

JouleUnitenergyprofilingtool,65lowbattery,64overheating,64usagestatistics,64–66

BeautifulTesting,196BelgiumTestingDays,195BetabyCrashlytics,138Betadistributiontools,138–139BirdFlight,138BlackBoxSoftwareTesting,55Blackboxtesting,54–55,164BlackBerryguidelines,81relativemarketshare,14,16

BlackBerryWebbrowservs.otherbrowsers,48BlackBerryWorld,26BlackBerryWorldGuidelines,26Blindness.SeeVisualimpairment.Blink,47Blogs,196–197BooksaboutSQLite,75Booksandpublications.SeealsoOnlineresources.AgileRecord,197AgileTesting,196BeautifulTesting,196ExploreIt!,196HowGoogleTestsSoftware,196TheInmatesAreRunningtheAsylum,30LessonsLearnedinSoftwareTesting,196MobileUsability,57ProfessionalTester,197SpecificationbyExample,196TapintoMobileApplicationTesting,92,196Tea-timewithTesters,197TestingCircus,197

Bots,137Boundaryvaluetesting,55Browserversion,inbugreports,99BugAdvocacy,183Bugcategory,inbugreports,101BugID,inbugreports,97Bugreportsbepolite,102keepitsimple,102mindmaps,examples,94–96overview,93,96–97remaincalm,102

Bugreports,contentsactualresult,98browserversion,99bugcategory,101bugID,97description,97expectedresult,98language,100logfiles,101–102mobiledevice,99mobileplatform,99mobile-device-specificinformation,99networkconditionandenvironment,100operatingsystem,99reproducible,99screenshotorvideo,101severity,101softwarebuildversion,100stepstoreproduce,98testdata,100testerwhofoundthebug,102work-around,98

Bugs,findingandprotectingfrom,53–55

Buildbot,137Businessmodelschoosing,24–25freemium,23–24in-apppurchases,24paid,24sellingadvertisements,24transaction,24

CalabashforAndroid,125–126foriOS,131–132nativeobjectrecognition,111

Camerasmobiletesting,43–44resolutionandimagesize,43–44testing,43–44

Capacitivetouchscreens,41Captureandreplaytools,111–112CarPlay,201CDMA(CodeDivisionMultipleAccess),12Cellphonesgenerationsof,12–13heads,12historyof,4–5transceivers,12

Charlesbatterytool,63Checklists.SeealsoGuidelines;Mindmaps;Mnemonics.genericmobiletesting,85–87mobileteststrategy,169–170overview,84

Checkstyletool,54ChromeWebbrowservs.otherbrowsers,48CIsystems,137Clicktale,177

Cloudtesting,147–152CloudMonkey,36,149Cognitiveimpairment,accessibilitytesting,61–62Cohn,Mike,105,106Colordeficiencies.SeeVisualimpairment.Color-blindness.SeeVisualimpairment.Communication,testerskill,179–181Communities,196Communitysupport,174–175Conferences,195–196Configurationdata,mobileteststrategy,165Connectedcars,201–203Connectedhomes,200–201Constantlearning,testerskill,183–184Cooper,Alan,30Coordinate-basedrecognitiontools,110COPFLUNGGUN(Communication,Orientation,Platform,Function,Location,Userscenarios,Notifications,Guidelines,Gesture,Updates,Network),92Crashreports,176Crashesonspecificdevices,testerskillsimprovement,187–192Creativity,testerskill,184Criticalthinking,testerskill,181–182Crowdtesting,141–147Crowdtestingproviders,145crowdsourcedtesting,145Crowdsourcing,141CruiseControl,137CSS3,22CucumberforAndroid,125–126Curiosity,testerskill,181Customerfocus,testerskill,184–185Customers.SeealsoUsers.gettinginformationabout,29–32invitingtousabilitytestingsessions,31–32representingwithpersonas,30–32

satisfying,checklistfor,32

Databases,testing,75Dateformats,testing,46Deafness.SeeAuditoryimpairment.Decisiontables,testing,55Deletingapps,reasonsfor,2Description,inbugreports,97Developingapps.SeeAppdevelopment.Devicemovement,detecting,38Deviceorientation,detecting,38Docksection,AndroidandiOSdevices,18DolphinWebbrowservs.otherbrowsers,48Doubletapgesture,42Downloadingresources.SeeOnlineresources.Draggesture,42Dumbphones,5DutchTestingDays,195Dynamictesting,53–55

EDGE(EnhancedDataratesforGlobalEvolution),13eggPlantforAndroid,129imagerecognition,110OCR/textrecognition,111

EmulatorsAndroidEmulator,51handlingfragmentation,35mobiledevicetestclouds,35prosandcons,115–116vs.realdevices,51–52vs.simulators,51–52

EnergyUsageInstrument,65–66EspressoforAndroid,128

nativeobjectrecognition,111ETSI(EuropeanTelecommunicationsStandardsInstitute)standards,82EuroSTAR,195Expectedresult,inbugreports,98ExperitestforAndroid,129foriOS,134mobilecloudtesting,149

ExploreIt!,196ExtSolo,122–123

FCCCUTSVIDS(Featuretour,Complexitytour,Claimstour,Configurationtour,Usertour,Testabilitytour,Scenariotour,Variabilitytour,Interoperabilitytour,Datatour,Structuretour),89–90,170Featurephones,5Fiddlerbatterytool,64FindBugstool,54FirefoxMobilebrowservs.otherbrowsers,48Fitnesswristbands,203–2045Gmobilestandard,134Gnetworks,13Fragmentation,definition,5Fragmentation,handlingemulators,35groupingtestdevices,33–34mobilecloudtesting,35–36mobiledevicelabs,34–36prioritizingmobiledevicegroups,33–34privacy,35rentingtestdevices,34–36security,35simulators,35

Frank,134Freemiumbusinessmodel,23–24Fullychargedbattery,testing,63–64

Functionaltesting,56,162

Gatheringcustomerinformation,30Gecko,47GeneralPacketRadioService(GPRS),13Generationsofmobiledevices,11–12Geographicalregions,adaptingto.SeeL10n(localization).GesturesupportforAndroid,125Gesturesdetecting,40testing,41–42

GherkinforAndroid,126GHUnit,134GlobalAppTesting,145Google,OurMobilePlanet,34GoogleBestPractices,57GoogleGlass,204–205GoogleInstrumentationTestRunner,128GooglePlaynativeAppBetaTesting,138GooglePlayStore,25GooglePlaystoreGuidelines,26GoogleTestAutomationConference,195Googlewearables,204GPRS(GeneralPacketRadioService),13GPScoordinates,detecting,39–40GPSlocation,settingforAndroid,122Groupingtestdevices,33–34GSM(GlobalSystemforMobileCommunications),12Guidelines.SeealsoChecklists;Mindmaps;Mnemonics.AmazonAppstoreGuidelines,26Android,81appdevelopment,2–3appstores,reviewandpublishing,26AppleAppStoreGuidelines,26AppleiOSapproval,81

BlackBerry,81BlackBerryWorldGuidelines,26forcorporatelook-and-feel,59GooglePlaystoreGuidelines,26platform-specificmobiletesting,81–82testautomation,114–115testingconnectedcars,202WindowsPhone,81

Guidelines,accessibilitytestingforAndroid,62foriOS,62WC3initiative,62forWindowsPhone,63

Gyroscopesensor,testing,38

Hardwareusagetesting,192Heads,cellphones,12Healthapps,205–207Hearingimpairment.SeeAuditoryimpairment.Help.SeeBooksandpublications;Onlineresources.HockeyApp,138HockeyKit,139HomeKit,201HowGoogleTestsSoftware,196Howe,Jee,141HTCDreamsmartphone,14HTML5,22Humiditysensor,testing,39Hybridapps,20–21

ISLICEDUPFUN(Inputs,Store,Location,Interactions/interruptions,Communication,Ergonomics,Data,Usability,Platform,Function,Userscenarios,Network),90–92,170I18n(internationalization),45–46IEEE(InstituteofElectricalandElectronicsEngineers),82Ignite,195

Imagerecognitiontools,109iMobiTrax,177In-apppurchases,24TheInmatesAreRunningtheAsylum,30Inputs,Store,Location,Interactions/interruptions,Communication,Ergonomics,Data,Usability,Platform,Function,Userscenarios,Network(ISLICEDUPFUN),90–92,170Installationtesting,71–72Instruments,energyusagefunction,65Internationalization(I18n),45–46Internetconnections,testing,4InternetExplorerMobileWebbrowservs.otherbrowsers,48InternetofThings(IoT).SeeIoT(InternetofThings).Interrupttesting,66–68In-the-wildtesting,167–169iOSaccessibilitytesting,62approvalguidelines,81designguide,18designresources,81generatingpseudo-randomtestevents,67simulators,52

iOSappstestautomation.SeeTestautomation,iOS.testingcriteria,103

iOSsmartphonevs.Android,17–19batterystatus,displaying,18docksection,18generatingpseudo-randomtestevents,67Instrumentsenergyusagefunction,65interrupttesting,67–68loggingenergyusage,onlineresource,66networkstrength,displaying,18notifications,displaying,18

pinningapps,18statusbar,18stresstesting,67–68time,displaying,18UIAutoMonkeytool,67–68userinterface,18

ios-driver,132–133IoT(InternetofThings)AndroidAuto,201AppleCarPlay,201AppleHomeKit,201connectedcar,201–203connectedhome,200–201mobiletesting,199–203NestLabs,200overview,199–200

ISTQBclass,55

JamoSolutionsAndroid,129iOS,134

Janky,137Javastaticanalysistools,54JavaScript,22Jenkins,137Johnson,KarenNicolemindmaps,93–96mnemonics,88

JouleUnit,energyprofilingtool,65JSONWireProtocolcompatibilitywithSelendroid,124andios-driver,132–133

JUnittool,54

Kaner,Cem,183

Kelly,Mike,89Keyboardinput,testing,44Keyboards,testing,187–188KeynoteforAndroid,129foriOS,134mobilecloudtesting,149

KeynoteMobileTesting,36KIF(KeepItFunctional),133KIFSU(KeepItFastSimpleUsable)principle,3KISS(KeepItSimple,Stupid)principle,3,102Kiwi,134Kohl,Jonathan,90,92

L10n(localization),45–46Languageinbugreports,100switching,forAndroid,122

Languagesadaptingto.SeeI18n(internationalization);L10n(localization).fallback,46

Launchstrategy.SeeMobilelaunchstrategy.LessonsLearnedinSoftwareTesting,196Let’sTest,195Linttool,54Localstoragetesting,76Localization(L10n),45–46Locationsensor,testing,39–40Logfilesinbugreports,101–102checking,82–83

LoggingenergyusageiniOS,66Logintask,testing,190Longtouchgesture,41Lowbattery,testing,64

LTE(LongTermEvolution),13

Magazines.SeeBooksandpublications.Magneticfields,detecting,38–39Magneticsensor,testing,38–39Manualtesting,mobileteststrategy,162,167–169Maximumdevicescope,160Metrics,mobileteststrategy,162mHealthapps,testing,205–207Microphones,testing,42–43MicrosoftStore,26Mindmaps,93,94–96MinistryofTesting,93Mnemonics.SeealsoChecklists;Guidelines;Mindmaps.COPFLUNGGUN(Communication,Orientation,Platform,Function,Location,Userscenarios,Notifications,Guidelines,Gesture,Updates,Network),92FCCCUTSVIDS(Featuretour,Complexitytour,Claimstour,Configurationtour,Usertour,Testabilitytour,Scenariotour,Variabilitytour,Interoperabilitytour,Datatour,Structuretour),89–90ISLICEDUPFUN(Inputs,Store,Location,Interactions/interruptions,Communication,Ergonomics,Data,Usability,Platform,Function,Userscenarios,Network),90–92mobileteststrategy,170overview,87SFDPOT(Structure,Function,Data,Platform,Operations,Time),88

Mob4Hire,145Mobile,definition,11MobileAppEurope,195MobileCheatSheet,79Mobilecloudtestinghandlingfragmentation,35–36mobiledevicetestclouds,35–36

Mobilecloudtestingproviders,149–150Mobilecommunity,testerskillsimprovement,194–195Mobiledevice,inbugreports,99

Mobiledevicelabs,34–36Mobiledevicetestcloudsemulators,35groupingtestdevices,33–34mobilecloudtesting,35–36mobiledevicelabs,34–36prioritizingmobiledevicegroups,33–34privacy,35rentingtestdevices,34–36security,35simulators,35

Mobiledevices.SeealsoSmartphones;specificbrands.acquiringfortesting,6–8cellphones,4–5dumbphones,5featurephones,5fragmentation,5rentingfortesting,7selectingfortesting,6

MobileFrameworksComparisonChart,21MobileLabs,36Mobilelaunchstrategycheckreleasematerial,172–174communitysupport,174–175crashreports,176post-release,174pre-launch,172–174reviews,175statistics,176–177timingoftherelease,173tracking,176–177

Mobilenetworks,historyofAMPS(AdvancedMobilePhoneService),12AMTS(AdvancedMobileTelephoneService),12CDMA(CodeDivisionMultipleAccess),12

EDGE(EnhancedDataratesforGlobalEvolution),13generations,11–12GPRS(GeneralPacketRadioService),13GSM(GlobalSystemforMobileCommunications),12LTE(LongTermEvolution),13MobileRadioTelephoneSystem,12MTD(MobileTelephonySystemD),12MTS(MobileTelephoneService),12NMT(NordicMobileTelephone),12OLT(OffentligLandmobilTelefoni),12overview,11–14speed,11–13UMTS(UniversalMobileTelecommunicationsSystem),13WiMAX(WorldwideInteroperabilityforMicrowaveAccess),13

MobilePersonas,30Mobileplatform,inbugreports,99MobileQAZone,196MobileRadioTelephoneSystem,12Mobilereleasecycles,6–8MobileSecurityTesting,79MobileTelephoneService(MTS),12MobileTelephonySystemD(MTD),12Mobileteststrategyautomatedtesting,162blackboxtesting,164configurationdata,165definingrequirements,156–158FCCCUTSVIDS,170functionaltesting,162ISLICEDUPFUN,170in-the-wildtesting,167–169manualtesting,162,167–169mnemonics,170mobilechecklists,169–170mobiletours,169–170

nonfunctionaltesting,162–163overview,155–156possiblemetrics,162productrisks,172regressiontesting,162stabledata,165targetdevices,selecting,166–167targetgroupinformation,gathering,157–158temporarydata,165testautomation,170–171testdata,164–166testenvironment,selecting,166–167testlevels,161–163testtechniques,163–164usagescenarios,168whiteboxtesting,163

Mobileteststrategy,scopeoftestingmaximumdevicescope,160multidevicescope,159–160single-devicescope,159use-casescope,160–161

Mobiletesting.SeealsoQualityassurancemeasures;specifictests.acquiringdevicesfor,6–8vs.automatedtesting,52–53batteries.SeeBatteryusage,testing.becreative,83boundaryvaluetesting,55cameras,43–44checkinglogfiles,82–83CI(continuousintegration),136–137cloudtesting,147–152connectedcar,201–203forconnectedcars,202connectedhome,200–201crowdtesting,141–147

databases,75decisiontables,55documentingresults.SeeBugreports.ETSI(EuropeanTelecommunicationsStandardsInstitute)standards,82functionaltesting,56generatingpseudo-randomtesteventsforiOSsmartphone,67genericchecklistfor,85–87gestures,41–42hardwareusage,192IEEE(InstituteofElectricalandElectronicsEngineers),82installationtesting,71–72interrupttesting,66–68IoT(InternetofThings),199–203keyboardinput,44keyboards,187–188localstoragetesting,76logintask,190mHealthapps,205–207microphones,42–43mindmaps,93,94–96onmultipleplatforms,187–192vs.othertechnologies,1–2,8paymenttask,190forpeoplewithdisabilities.SeeAccessibilitytesting.performance.SeePerformancetesting.permissions,190–191platform-specificguidelines,81–82preinstalledapps,44pseudo-randomtesteventsforAndroid,66qualityofInternetconnections,4inreallifeenvironments,3–4remotelocations,4rentingdevicesfor,7security.SeeSecuritytesting.selectingdevicesfor,6

smartglasses,204–205soundinputs,42–43standardsconformance,82standbytesting,71stresstesting,66–68systemapps,44TestingMap,93touchscreen,41–42uninstalltesting,72userinterface.SeeUsabilitytesting.voicecommands,42–43W3C(WorldWideWebConsortium),82wearables,203–204weatherconditions,4widgets,187–189

Mobiletesting,mnemonics.SeealsoChecklists;Guidelines;Mindmaps.COPFLUNGGUN(Communication,Orientation,Platform,Function,Location,Userscenarios,Notifications,Guidelines,Gesture,Updates,Network),92FCCCUTSVIDS(Featuretour,Complexitytour,Claimstour,Configurationtour,Usertour,Testabilitytour,Scenariotour,Variabilitytour,Interoperabilitytour,Datatour,Structuretour),89–90ISLICEDUPFUN(Inputs,Store,Location,Interactions/interruptions,Communication,Ergonomics,Data,Usability,Platform,Function,Userscenarios,Network),90–92overview,87SFDPOT(Structure,Function,Data,Platform,Operations,Time),88

Mobiletesting,sensorsandinterfacesaccelerationsensor,38ambientlightsensor,37changesindevicemovement,detecting,38deviceorientation,detecting,38gestures,detecting,40GPScoordinates,detecting,39–40gyroscopesensor,38

humiditysensor,39locationsensor,39–40magneticfields,detecting,38–39magneticsensor,38–39possibletests,40–41pressuresensor,39proximitysensor,37–38screenbrightness,37temperaturesensor,39touchlesssensor,40

Mobiletesting,updatetestingadbtool,74–75forAndroidapps,74–75foriOSapps,73–74overview,72typicalscenarios,73

MobileTools,79Mobiletours,169–170Mobiletrackingtools,177MobileUsability,57MobileWebbrowserscomparisonof,48layoutengines,47

MobileAppTracking,177Mobile-device-specificinformation,inbugreports,99Mobilityanddatanetworks,3–4Monkeytool,66MonkeyTalkforAndroid,129coordinate-basedrecognition,110foriOS,134OCR/textrecognition,111

Moolya,92MTD(MobileTelephonySystemD),12MTS(MobileTelephoneService),12

Multidevicescope,159–160Multitouchgesture,42

Nativeapps,description,19–20NativeObjectRecognition,111Neotys,149NestLabs,200Networkconditionandenvironment,inbugreports,100Networkstrength,displaying,1899tests,145NMT(NordicMobileTelephone),12Nonfunctionaltesting,162–163Notifications,displaying,18

OCMock,134OCR/textrecognition,110–111OLT(OffentligLandmobilTelefoni),121Gnetworks,12Onlineresources.SeealsoBooksandpublications.5Gmobilestandard,1399tests,145adjust,177AmazonAppstoreGuidelines,26Androiddesignguide,18AndroidEmulator,51Androidguidelines,81AppQualityAlliancesite,103apprejection,reasonsfor,18appstorereviewandpublishingguidelines,26Appaloosa,138AppBlade,138Appium,134Applause,138,145AppleAppStore,25AppleAppStoreGuidelines,26

AppleiOSapprovalguidelines,81AppleiOSdesignresources,81AppleWatch,204“ApplyingtheSFDPOTHeuristictoMobileTesting,”88appsfire,177AppsFlyer,177AppThwack,36,149Appurify,149Bamboo,137batteryusage,65–66BetabyCrashlytics,138betadistributiontools,138–139BirdFlight,138BlackBerryguidelines,81BlackBerryWorld,26BlackBerryWorldGuidelines,26Blink,47BooksaboutSQLite,75Bots,137Buildbot,137CDMA(CodeDivisionMultipleAccess),12CIsystems,137Clicktale,177CloudMonkey,36,149crowdtestingproviders,145crowdsourcedtesting,145CruiseControl,137CSS3,22EDGE(EnhancedDataratesforGlobalEvolution),13eggPlant,129EnergyUsageInstrument,65–66Espresso,128Experitest,129,134,149Frank,134gatheringcustomerinformation,30

Gecko,47GHUnit,134GlobalAppTesting,145GoogleBestPractices,57GoogleInstrumentationTestRunner,128GooglePlaynativeAppBetaTesting,138GooglePlayStore,25GooglePlaystoreGuidelines,26Googlewearables,204GPRS(GeneralPacketRadioService),13GSM(GlobalSystemforMobileCommunications),12HockeyApp,138HockeyKit,139HTML5,22iMobiTrax,177iOSdesignguide,18iOSsimulators,52ios-driver,133JamoSolutions,129,134Janky,137JavaScript,22Jenkins,137Keynote,129,134,149KeynoteMobileTesting,36Kiwi,134loggingenergyusageiniOS,66MicrosoftStore,26Mob4Hire,145mobilecloudtestingproviders,149–150mobiledevicetestcloud,36MobileFrameworksComparisonChart,21MobileLabs,36MobilePersonas,30mobiletrackingtools,177mobileusability,57

mobileusabilitydesignprinciples,57mobileuserinterfacedesignprinciples,57MobileAppTracking,177MonkeyTalk,129,134Neotys,149OCMock,134OpenDeviceLab,36operatingsystemmarketshares,16OurMobilePlanet,34passbrains,145Pebble,204PerfectoMobile,36,129,134,150PhoneGapframework,20platform-specificguidelines,81–82Presto,47Ranorex,129,134,150Roboelectric,129Robotiumprojectpages,122Robotiumwikipage,122SauceLabs,36,150Siesta,129SilkMobile,129,134SOASTA,129,134Specta,134Subliminal,134TeamCity,137TenKodEZTestApp,129TestBirds,145TestChameleon,150testCloud,145Testdroid,150TestFlight,139testingcriteriaforAndroidandiOS,103testingtools,54Testmunk,36,150

TestObject,36,129,150TestPlus,145TouchTest,36TravisCI,137Trident,47UIAutomator,129UMTS(UniversalMobileTelecommunicationsSystem),13usabilityheuristics,57usabilityprinciplesandtechniques,57usabilitytesting,57wearables,204Webbrowserlayoutengines,47WebKit,47widgets,18WiMAX(WorldwideInteroperabilityforMicrowaveAccess),13WindowsPhoneguidelines,81XamarinTestCloud,36,150XCTest,134Zucchini,134

Onlineresources,fortestersAgileTestingDays,195AssociationforSoftwareTesting,196BelgiumTestingDays,195blogs,196–197booksandpublications,196–197communities,196conferences,195–196DutchTestingDays,195EuroSTAR,195GoogleTestAutomationConference,195Ignite,195Let’sTest,195MobileAppEurope,195MobileQAZone,196Øredev,195

SoftwareTestingClub,196STAREAST,196STARWEST,196TestBash,196TestExpo,196TestingCircus,196uTestcommunity,196

OpenDeviceLab,36OperaMobileWebbrowservs.otherbrowsers,48Operatingsystemsthebigtwo,16–19.SeealsoAndroid;iOS.inbugreports,99relativemarketshares,16

Øredev,195OurMobilePlanet,34Overheating,batteries,64OWASP(OpenWebApplicationSecurityProject),77OWASPMobileSecurityPageMobileCheatSheet,79MobileSecurityTesting,79MobileTools,79Top10MobileRisks,80

Paidbusinessmodel,24passbrains,145Paymenttask,testing,190Pebble,204PerfectoMobileforAndroid,129coordinate-basedrecognition,110foriOS,134mobilecloudtestingproviders,150mobiledevicetestcloud,36

Performancetestingcriticalareas,69

mobileapps,70–71overview,68–69reproducingissues,189–190

Permissions,testing,190–191Personas,30–32PhoneGapframework,20Physicalimpairment,accessibilitytesting,61Pinchopen/closegestures,42Pinningapps,18Platform-specificguidelinesfortesting,81–82PMDtool,54Post-release,mobilelaunchstrategy,174Preinstalledapps,testing,44Pre-launch,mobilelaunchstrategy,172–174Pressuresensor,testing,39Presto,47Prioritizingmobiledevicegroups,33–34Privacy,35Processfocus,53–55Productfocus,53–55Productrisks,mobileteststrategy,172ProfessionalTester,197Programminglanguage,choosing,foriOStests,132–133Programmingskills,testerskill,185Proximitysensor,testing,37–38Pyramid.SeeTestautomationpyramid.

Qualityassurancemeasures.SeealsoMobiletesting.blackboxtesting,54–55dynamictesting,53–55findingbugs,53–55Javastaticanalysistools,54processfocus,53–55processfocusphase,54productfocus,53–55

protectingfrombugs,53–55statictesting,53–55whiteboxtesting,54

RanorexforAndroid,129foriOS,134mobilecloudtestingproviders,150

Realdevices,prosandconsoftesting,116–117Reda,Renas,122Regressiontesting,mobileteststrategy,162Rejectingapps,reasonsfor,18Releasecycles.SeeMobilereleasecycles.Releasematerial,checking,172–174Rentingdevicesfortesting,7Rentingtestdevices,34–36Reproducible,inbugreports,99Requirementsdefinition,mobileteststrategy,156–158ResearchinMotion,14Resistivetouchscreens,41Reviewandpublishingguidelines,appstores,26Reviews,mobilelaunchstrategy,175Roboelectric,129RobotiumforAndroid,121–122nativeobjectrecognition,111OCR/textrecognition,111recordingtestscripts,122–123

RobotiumExtensions,122–123Robotiumprojectpages,122RobotiumRecorder,122–123Robotiumwikipage,122Rotategesture,42Rotatingtouchscreens,42RoutineBot,110

SafariWebbrowservs.otherbrowsers,48Samsungappstore,25SauceLabs,36,150Scopeoftestingmaximumdevicescope,160multidevicescope,159–160single-devicescope,159use-casescope,160–161

Screenbrightness,testing,37Screens.SeeTouchscreens.Screenshotorvideo,inbugreports,101Securityhandlingfragmentation,35mobiledevicetestclouds,35

Securitytestingcachestorage,78client-sidevalidation,78commonappsecuritymistakes,77–79encryption,78fuzzing,77overview,76–77penetrationtesting,77securityrequirementsanalysis,79–80summary,80–81testingtools,77Top10MobileRisks,80unencryptedcommunications,78unintendedlocalstorage,78unnecessaryapppermissions,78

SeeTest,111SelendroidforAndroid,124–125nativeobjectrecognition,111

Selenium,111Sellingadvertisements,24

Sensors,testing.SeeMobiletesting,sensorsandinterfaces.Severityoferror,inbugreports,101SFDPOT(Structure,Function,Data,Platform,Operations,Time),88Sherry,Rosie,93–96Siesta,129Sikuli,110SilkMobileforAndroid,129foriOS,134

Simulatorsvs.emulators,51–52handlingfragmentation,35iOS,52mobiledevicetestclouds,35prosandcons,115–116vs.realdevices,51–52

Single-devicescope,159Smartglasses,204–205Smartwatches,203–204Smartphones.SeealsoMobiledevices;specificbrands.commonfeatures,5externalcomponents,15–16hardwarecomponents,15historyof,14–16marketpenetration,5post-2007,14–16

SOASTAforAndroid,129foriOS,134

Softwarebuildversion,inbugreports,100SoftwareTestingClub,196Soundinputs,testing,42–43SpecificationbyExample,196Specta,134Speedofmobilenetworks,historyof,11–13

Spoon,123–124Square,123,133Stabledata,mobileteststrategy,165Standardsconformance,testing,82Standbytesting,71STAREAST,196STARWEST,196Staticanalysistools,54Statictesting,53–55Statistics,mobilelaunchstrategy,176–177Statusbar,18Stepstoreproduce,inbugreports,98Strategyforproductlaunch.SeeMobilelaunchstrategy.fortesting.SeeMobileteststrategy.

Stresstesting,66–68Structure,Function,Data,Platform,Operations,Time(SFDPOT),88Subliminal,134Swipegesture,41Systemapps,testing,44

Tapgesture,41TapintoMobileApplicationTesting,92,196Targetdeviceselection,mobileteststrategy,166–167Targetgroupinformationgathering,mobileteststrategy,157–158TeamCity,137Tea-timewithTesters,197Technicalskills,testerskill,185Temperaturesensor,testing,39Temporarydata,mobileteststrategy,165Tenacity,testerskill,183TenKodEZTestAppforAndroid,129nativeobjectrecognition,111

Testautomation

cloud-based,115–116,152emulators,prosandcons,115–116goalsfor,113–114guidelinesfor,114–115vs.manualtesting,52–53mobileteststrategy,162,170–171vs.mobiletesting,52–53realdevices,prosandcons,116–117selectingtasksfor,113–115simulators,prosandcons,115–116

Testautomationpyramiddefaultconfiguration,105–106flippedconfiguration,106–108mobiletestconfiguration,108–109

TestautomationtoolsAppium,111betadistribution,138–139Calabash,111captureandreplay,111–112coordinate-basedrecognition,110eggPlant,110,111Espresso,111imagerecognition,109MonkeyTalk,110,111NativeObjectRecognition,111OCR/textrecognition,110–111overview,109PerfectoMobile,110recommendations,112–113Robotium,111RoutineBot,110SeeTest,111selectioncriteria,118–120Selendroid,111Selenium,111

Sikuli,110TenKodEZTestApp,111TestObject,110,111

Testautomationtools,AndroidAdvancedUserInteractionsAPI,125Appium,127–128automatingUIacceptancetests,125CalabashforAndroid,125–126Cucumber,125–126eggPlant,129Espresso,128Experitest,129ExtSolo,122–123gesturesupport,125Gherkin,126GoogleInstrumentationTestRunner,128JamoSolutions,129Keynote,129MonkeyTalk,129overview,120–121PerfectoMobile,129Ranorex,129recommendations,129–130recordingtestscripts,122–123Roboelectric,129Robotium,121–122RobotiumExtensions,122–123RobotiumRecorder,122–123Selendroid,124–125settingGPSlocation,122Siesta,129SilkMobile,129SOASTA,129Spoon,123–124switchingcurrentlanguage,122

TenKodEZTestApp,129TestObject,129togglingWi-Fionandoff,122UIAutomator,129UIAutomatorViewer,121

Testautomationtools,iOSAppium,134CalabashforiOS,131–132choosingaprogramminglanguage,132–133Experitest,134Frank,134GHUnit,134ios-driver,132–133JamoSolutions,134Keynote,134KIF(KeepItFunctional),133Kiwi,134MonkeyTalk,134OCMock,134PerfectoMobile,134Ranorex,134SilkMobile,134SOASTA,134Specta,134Subliminal,134UIAutomation,129–130XCTest,134Zucchini,134

Testdatainbugreports,100mobileteststrategy,164–166

Testenvironmentselection,mobileteststrategy,166–167Testlevels,mobileteststrategy,161–163Testscripts,recording,122–123Teststrategy.SeeMobileteststrategy.

Testtechniques,mobileteststrategy,163–164TestBash,196TestBirds,145TestChameleon,150testCloud,145Testdroid,150Testersblogs,196–197booksandpublications,196–197communities,196conferences,195–196namedinbugreports,102successfactors,207–209

Testers,requiredskillscommunication,179–181constantlearning,183–184creativity,184criticalthinking,181–182curiosity,181customerfocus,184–185programmingskills,185technicalskills,185tenacity,183

Testers,skillsimprovementcrashesonspecificdevices,187–192engagewiththemobilecommunity,194–195learnfromotherapps,186observeotherpeople,192–193participateintestingcompetitions,193–194Twitter,194

TestExpo,196TestFlight,139Testingapps.SeeMobiletesting.TestingCircus(community),196TestingCircus(magazine),197

Testingcompetitions,testerskillsimprovement,193–194TestingcriteriaforAndroidandiOS,103TestingMap,93Testingtoolsadjust,177appsfire,177AppsFlyer,177forautomation.SeeTestautomationtools.Checkstyle,54Clicktale,177FindBugs,54iMobiTrax,177JUnit,54Lint,54formobiletracking,177MobileAppTracking,177PMD,54staticanalysis,54TestNG,54whiteboxtesting,54XCTest,54

Testmunk,36,150TestNGtool,54TestObjectforAndroid,129imagerecognition,110mobilecloudtestingproviders,150mobiledevicetestcloud,36OCR/textrecognition,111

TestPlus,1453Gnetworks,13Time,displaying,18Timingoftherelease,173T-MobileG1smartphone,14Tools.SeeOnlineresources;Testingtools;specifictools.

Top10MobileRisks,80Touchgestures,41–42Touchlesssensor,testing,40Touchscreenscapacitive,41effectsofweatherconditions,42mobiletesting,41–42resistive,41rotating,42testing,41–42touchgestures,41–42typesof,41

TouchTest,36Tracking,mobilelaunchstrategy,176–177Trackingtools,177Transactionbusinessmodel,24Transceivers,cellphones,12TravisCI,137Trident,47Twitter,testerskillsimprovement,1942Gnetworks,12–13

UIacceptancetests,automating,125UIAutomation,129–130UIAutomator,129UIAutomatorViewer,121UMTS(UniversalMobileTelecommunicationsSystem),13Uninstalltesting,72Updatetestingadbtool,74–75forAndroidapps,74–75foriOSapps,73–74overview,72typicalscenarios,73

Uploadingappstoappstores,26

Usabilityheuristics,57TheInmatesAreRunningtheAsylum,30principlesandtechniques,57

Usabilitytestingallowmistakes,58consistency,59corporateidentityguidelinesforlook-and-feel,59details,checking,58documentyourobservations,59–60invitingcustomersto,31–32lessismore,57navigation,checking,58overview,56self-explanatoryapps,57usabilityanddesignpatterns,58verifytext,59workflows,checking,58

Usabilitytestingsessions,31–32Usagescenarios,mobileteststrategy,168Usagestatistics,batterylife,64–66Use-casescope,160–161UserinterfaceAndroidvs.iOS,18components,viewingpropertiesof,121designprinciplesformobiledevices,57testing.SeeMobiletesting,sensorsandinterfaces.

Users,expectations.SeealsoCustomers.overview,2–3priorityappcharacteristics,2reasonsfordeletingapps,2

uTestcommunity,196

Visualimpairment,accessibilitytesting,60–61Voicecommands,testing,42–43

W3C(WorldWideWebConsortium)accessibilityguidelinesinitiative,62conformancetesting,82

Wearablesfitnesswristbands,203–204GoogleGlass,204–205mobiletesting,203–204overview,203smartglasses,204–205smartwatches,203–204

Weatherconditionseffectsontouchscreens,42testing,4

Webapps,description,22–23Webbrowserlayoutengines,47Webbrowsers.SeeMobileWebbrowsers.WebKit,47WeekendTesting,193–194Whiteboxtestingmobileteststrategy,163toolsfor,54

WidgetsiOSvs.Android,18testing,187–189

Wi-Fi,togglingonandoff,122WiMAX(WorldwideInteroperabilityforMicrowaveAccess),13WindowsPhoneaccessibilityguidelines,63accessibilitytesting,63appstore,26relativemarketshare,16

WindowsPhoneapps,downloading,26Work-around,inbugreports,98WorldwideInteroperabilityforMicrowaveAccess(WiMAX),13

XamarinTestCloud,36,150XCTest,134XCTesttool,54

0Gnetworks,12Zucchini,134

CodeSnippets