IBM Mobile Quality Assurance - Open Beta Study Group Session 1

© 2013 IBM Corporation

IBM Mobile Quality Assurance Continuously deliver quality mobile apps – MQA Study Group - 25 October 2013

Host: Roger Snook, WorldWide Enablement Leader: Mobile, SOA & Design +1.703.943.1170, [email protected]

© 2013 IBM Corporation2 #IBMMobile

Please note – This discussion is about an Open Beta

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarksin a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.


Mobile is primary91% of mobile users keep their device within arm’s reach 100% of the time

Mobile is primary91% of mobile users keep their device within arm’s reach 100% of the time

Insights from mobile data provide new

opportunities 75% of mobile shoppers

take action after receiving a location based messages

Insights from mobile data provide new

opportunities 75% of mobile shoppers

take action after receiving a location based messages

Mobile is about transacting96% year to year increase in mobile cyber

Monday sales between 2012 and 2011

Mobile is about transacting96% year to year increase in mobile cyber

Monday sales between 2012 and 2011

Mobile must create a continuous brand

experience90% of users use multiple screens as channels come

together to create integrated experiences

Mobile must create a continuous brand

experience90% of users use multiple screens as channels come

together to create integrated experiences

Mobile enables the Internet of Things

Global Machine-to-machine connections will increase

from 2 billion in 2011 to 18 billion at the end of 2022

Mobile enables the Internet of Things

Global Machine-to-machine connections will increase

from 2 billion in 2011 to 18 billion at the end of 2022

A business in motion must be MobileFirst


35% of developers are currently targeting tablets, with more than 90% plan to develop for tablets in the near future.

Nearly 90% of developers are currently extending enterprise apps to mobile or plan to in the near future.

In the next 12 months 77% of developers will

be focused on Hybrid application development.

More than 200m people updated to iOS7 in the first week

Source: Evans Data Mobile Developer Survey Mobile Development Report 2012 Volume

Source: Business Insider (September 2012)

25+ Almost all expect to deploy more than 25 mobility

applications in the next two years

29% of mobile developers currently focus on Hybrid

app development.

Since apps just don’t fall out of the sky.....Mobile application development is a top priority for businesses!


Why mobile development is different

• Smaller screens pulling content from more sources: need to choreograph content from multiple repositories and applications , both on premises and in the cloud

• Greater challenges of application governance, distribution, and version management

• AppStore Terms of Service present challenges for managing B2C apps

Development is different.Mobile apps are different.• They are more strategic – Greater urgency and

immediacy of engagement

• They are context-aware –know where you are and what you are doing

• They run on unstable networks–interruption is the norm, not the exception

• Smaller footprint – More challenging to make compelling and easy to use

• Always on – An opportunity to deliver greater value

Management is different. Security is different.

• Faster development cycles

• More devices to support, each with its own set of capabilities, OS and UI behaviors

• More development approaches to choose from – Web, HTML, Native

• More third-party and open-source tools, frameworks and libraries to choose from

• Consumerization of Technology is changing the definition of mobile app quality

• Greater risks of exposing applications and data on small, light and always on portable devices

• Greater authentication challenges associated with content mashups

• AppStore Terms of Service can limit security options


Scan & Certify

Instrument

Test

IntegrateObtain Insight

Manage

Deploy

Design & Develop

A mobile enterprise lifecycle has many “success disciplines”


Organizations that effectively leverage software development outperform their competitors... yet few are able to deliver it effectively

86%

of companies believe software delivery is important or critical

25%

leverage software delivery effectively today

But only…

Source: “The Software Edge: How effective software development drives competitive advantage,” IBM Institute of Business Value, March 2013

69%

outperformthose who don’t

of those wholeverage software

delivery today


Industry Solutions

IBM & Partner Applications

BankingBanking InsuranceInsurance TransportTransport TelecomTelecom GovernmentGovernmentRetailRetail HealthcareHealthcare AutomotiveAutomotive

Application & Data Platform

Str

ateg

y &

Des

ign

Ser

vice

s

Cloud & Managed Services

Devices Network Servers

Develo

pm

ent &

Inte

gratio

n S

ervices

ManagementManagement AnalyticsAnalyticsSecuritySecurity

IBM MobileFirst offering portfolio


Scan & Certify

Instrument

Test

IntegrateObtain Insight

Manage

Deploy

Design & Develop

Mobile enterprise lifecycle: Focus on Mobile Quality


Mobile quality has a tremendous business impact: Quality vs Time

2 star rating30% loss of subscribers2 star rating30% loss of subscribers

$10B of eBay sales done thru Mobile

Financial exposure due to security breach

The same percentage of mobile developers who claim Testing and Quality Assurance is in

their top 5 concerns…

Also claim they are challenged with

developing applications as quickly as management

wants

SD Times: “More than half of organizations are

building mobile applications”

http://www.sdtimes.com/link/36553


100,000+ different devices for Android devices only, 500+ brands Doesn’t account for OS versions, carriers or locations Doesn’t account for iOS, Windows Phone or Blackberry

Mobile application quality is more complex


Mobile quality presents unique needsConsumerization of Technology is Changing the Definition of Quality

Usability and fit-to-purpose are important aspects of perceived quality Quality of Experience is defined as “the overall acceptability of an application or service,

as perceived subjectively by the end user.”

Time to Feedback Mobile engagement requires a lifecycle focus on “time to feedback” Shorter iterations, and more frequent delivery Majority of the testing performed by developers

Focus is on reducing time to feedback vs. bug count


Mobile Quality automation fundamental to a DevOps LifecycleContinuously deliver high-quality mobile applications (HTML5, hybrid and native) and support multiple

mobile operating environments and devices with the simplicity of a single, shared code base

Systems ofInteraction

DevOps LifecycleOperations/ProductionDevelopment/TestCustomers Business Owners

Continuous Innovation, Feedback and Improvements

Ec

os

ys

tem

Be

st P

rac

tice

s

Plan and Measure Develop and Test Release and Deploy

Monitor and Optimize

IBM Mobile Quality Assurance

Worklight

Automate and orchestrate the delivery pipeline for IT infrastructure and the mobile app, reducing cycle times to respond faster to feedback

Judge mobile application quality and customer readiness across multiple dimensions, to deliver apps that better meet customer expectations

Rational Test Workbench


Continuously deliver high quality mobile applicationsWhat’s New

• End user feedback and quality metrics available at every stage of development

• App quality assurance in a fragmented and complex environment

Features/Business Value

• Evidence-based prioritization – enable business and IT to collaborate on mobile strategy and user experience

• Over the air app distribution – get the latest in the hands of testers as soon as it is available

• Frictionless bug reporting – spend every minute on testing latest and greatest builds, not the hassles

• In-app crash reporting – rapid understanding of why an app fails

• Sentiment analysis – mine app ratings and reviews to extract actionable feedback before they go viral(coming soon)

Client Benefits• Improve mobile app quality with holistic visibility into user

experience, including bugs, crashes, feedback, and user sentiment

• Proactively manage quality risks through rapid assessment of issues and reduced cycle time

Updated xx-xx-12

Introducing IBM Mobile Quality Assurance (beta)

”What you are talking about doing is fantastic. You have a

very powerful approach here and it will drive a lot of what people will want to do in the market”

IBM Mobile Quality Assurance (beta)


Streamline the feedback loop into development, before and after release – five use cases (aligned with this Study Group)

Developer Tester UsersDistribute to testers and users

1

Report Bugs 2

Devices Report Crashes / Logs 3

Provide Feedback 4

Analyze AppStore5


Tester End UsersDeveloper

Over the air build distribution1 In app bug reporting 2

Crash log reporting

4

In app user feedback 3

LOB/Digital Marketer

Sentiment Analysis5

Builds

User Feedback Crash logs

BugsBugs vs. Crashes

Quality Dashboard with Sentiment Analysis

Mobile Quality Assurance features that support the Use Cases

Coming soon


Helping the software team get smarter about their mobile apps

Tester End UsersDeveloperLOB/Digital Marketer

“When someone leaves a negative review in the app store, it scars your app for life, you can’t respond to it, and you can’t learn more about the problem in order to fix it quickly.”

- Mobile Orchard

• How effective our mobile app is at engaging customers?

• How can I stay on top of current quality metrics, and turn them into an actionable strategy?

• How could I get visibility to issues that customers experience in production so that I can address them quickly?

• How can I get more people to test the apps so that we can cover different devices, network carriers, and OS versions?

• How can I get the latest and greatest build without wasting my time to track that down?

• How can I quickly submit bug reports with screenshots and device details right from the application?

• How can I provide direct and constructive feedback?


Over the air app distribution

Developer uploads new build

Identifies test devices

IBM MQA notifies testers

Testers download new version through the app or thru IBM MQA

Enables developers to distribute new builds to test devices


In app bug reporting

When bugs are identified, user simply shakes their device

1. IBM MQA is activated

2. Bug details are reported in structured manner to RTC

Bug details include•Device maker & model•OS & version•Carrier & connection speed•Battery life & resolution•Repro steps & screenshot

Enables developers to receive bug reports


Direct user feedback

Developers can customize the feedback workflow

When user has feedback, they trigger a feedback widget

IBM Mobile Quality Assurance is activated

Free-form feedback is shared directly to IBM MQA

Enables developers to connect with users


Crash log reporting

When your app crashes, IBM MQA logs the event

IBM MQA captures the crash details•Device maker & model•OS version•Carrier & connectivity•Battery life•Stack trace•Logs

IBM MQA aggregates all crashes into an easy-to-use reporting UI

Enables developers to know when and why crashes occur


IBM Mobile Quality Assurance


Sentiment Analysis

Mine app ratings and reviews to extract actionable feedback

IBM MQA allows users to see•Review stats on their mobile apps•Detailed attribute scores•Cumulative sentiment score and trends

Enables line of business to mine user sentiment

Coming soon – Not yet in Open Beta


Evidence-based prioritization

Collaborate on mobile strategy and end user experience

IBM MQA delivers•Tracking of app installations •Cumulative usage sessions•Crash report tracking over time

Enables prioritization of action on mobile app development


Your Study Group Action Items for Today:

1.Take Brief Survey – Your feedback helps! (Coming soon to your inbox)

2.Try It: MQA Open Betahttp://www.quality4mobile.com NOTE: This is the current URL and may

change in the future. To be sure you can access this, use the landing page.

Your Study Group Action Items for Today:

1.Take Brief Survey – Your feedback helps! (Coming soon to your inbox)

2.Try It: MQA Open Betahttp://www.quality4mobile.com NOTE: This is the current URL and may

change in the future. To be sure you can access this, use the landing page.


1Tools and Technologies for MobileFirst Developers: http://www.ibm.com/developerworks/mobile/

23

Ways to get started with IBM MobileFirst

Take part in the IBM Mobile Quality Assurance open beta: http://ibm.co/174C6ug

Read the CTO Mobile Frontier Blog: https://www.ibm.com/developerworks/community/blogs/mobileblog

Our Next Session: IBM Mobile Quality Assurance Study Group Session 2: Viewing crashes and bug reports Nov 1, 2013 9:00 AMhttps://events.na.collabserv.com/register.php?id=44a9a6a4dd&l=en-US

Further Study: Mobile Learning Circle (coming soon)Mobile Testing Product Usage Learning Roadmap (coming soon)



Legal Disclaimer

© IBM Corporation 2013. All Rights Reserved.The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:UNIX is a registered trademark of The Open Group in the United States and other countries.If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.

IBM Mobile Quality Assurance - Open Beta Study Group Session 1

Technology

mobile users

mobile data

aa mobile

mobile shoppers

mobile insights

primary mobile

mobilefirst mobile

mobile application development