Appium & Perfecto: A Perfect Match Max Skybin, Director of Sales Engineering Nick Sanjines, Senior Sales Engineer
Appium & Perfecto: A Perfect Match
Max Skybin, Director of Sales EngineeringNick Sanjines, Senior Sales Engineer
AGENDA
• Digital transformation context• Key enablers of digital quality
• What is Appium?• Appium’s strengths and limitations
• Who is Perfecto and how we work with Appium• How we improve and build on Appium’s core strengths
• Benefits of using Perfecto & Appium together• Demo
© 2016, Perfecto Mobile Ltd. All Rights Reserved.
05/01/2023 3© 2015,Perfecto Mobile Ltd. All Rights Reserved.
3
- Physical experience and interaction gives way to Digital
- Digital is an Omni-Channel strategy
- Mobile is the first screen among connected consumers
MobileSocial
Enterprise End User
Web
Internet of Things
The Digital Transformation
DIGITAL PLATFORMS ARE ALL MERGING INTO ONE EXPERIENCE
05/01/2023 4© 2015, Perfecto Mobile Ltd. All Rights Reserved.
One User Experience Across All Digital Platforms
Source: Google
Network QualityCellular, Wi-Fi
Interacting with sensors
Peak time -Millions of transactions Multiple devices
Specific profiles
USERS DEMAND EXCEPTIONAL DIGITAL EXPERIENCE REGARDLESS OF CONDITIONS
Varied Device Conditions
Background Apps Location
Immediate, reliable, secure 24/7 access to mobile devices and web-browsers across all
required platforms
Unattended parallel automation covering full
scope of functional and non-functional requirements
Real-time on-demand reporting with high-level view of full digital quality as well as
detailed drill-down into specific test results
KEY ENABLERS FOR EXCEPTIONAL DIGITAL QUALITY
© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Enterprise-GradeDevice Lab
RobustAutomation
ComprehensiveReporting
Reduce device management spend and enable high levels of
automated testing
Avoid manual testing spend without compromising quality
Quickly identify quality issues earlier in SDLC, prioritise, triage and fix defects across web and mobile
WHAT IS APPIUM?
Appium is a tool for automating testing of applications on mobile devices. It was created as an open-source alternative to existing commercial tools.
05/01/2023 7© 2016, Perfecto Mobile Ltd. All Rights Reserved.
APPIUM’S BENEFITS
• Cross-platform• Write tests against multiple platforms (iOS, Android, Windows), using the same API
• Non-intrusive• Doesn’t require the app to be modified or recompiled• Doesn’t require access to application’s source code• Test what you ship to end-users
• Open• Automate using any language/framework• Based on industry-standard Selenium Web Driver• Open source project supported by community
05/01/2023 8© 2016, Perfecto Mobile Ltd. All Rights Reserved.
APPIUM’S LIMITATIONS
• Open Source Support Model• No official support when something goes wrong• Tends to take 1-2 months to support a new iOS version after GA release
• No OS-level or environmental controls• Testing outside the application context is impossible, which prevents automation of full
digital experience (2 factor authentication, widgets, interaction with other apps)• No environment controls are available to change GPS, network conditions, etc.• Object-level recognition only, no ability to visually validate user experience
• Unreliable automation• Defects in Appium/Selenium cause test failures• Parallel execution difficult to configure and sustain
• Negative productivity impact• No reporting capabilities – you have to build your own reporting• Lack of IDE plug-ins slows down script creation
05/01/2023 9© 2016, Perfecto Mobile Ltd. All Rights Reserved.
IMPACT: APPIUM DOESN’T ADDRESS THE FUNDAMENTAL PROBLEM OF PROVIDING AN ENTERPRISE-GRADE LAB
10
Real end user conditions
Complex hardware & software setup
Secure environments
On-demand access to mobile devices, browsers & wearables
24x7 continuous support
iOS 10 Beta 6/13
iOS G.A.9/13
10/11 - Appiumsupport for iOS 10
120 days without iOS 10 support!54% user adoption of iOS 10 during Appium support gap
IMPACT: LACK OF TIMELY OS SUPPORT IN APPIUM LEADS TO RISK OF RELEASING UNTESTED APPLICATIONS DURING WIDE OS ADOPTION
Appi
um C
anno
t Tes
tAp
pium
Can
Te
stIMPACT: LACK OF OS-LEVEL CONTROLS AND VISUAL VALIDATIONS LIMITS THE AMOUNT OF AUTOMATION COVERAGE
05/01/2023 12© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Verify the user is able to enter only valid postal code • Verify the user is able to enter only valid post code
by checking input exists in DOM
• Verify the continue button is visible so the user can proceed to the next screen
• Verify the screen renders within acceptable time under WiFi, LTE and 3G connectivity
• Validate that when invalid postal code is entered, error message is colored red
IMPACT: POOR REPORTING MAKES TEST RESULT ANALYSIS AND DEFECT ISOLATION A PAINFUL EXERCISE
05/01/2023 13© 2016, Perfecto Mobile Ltd. All Rights Reserved.
WHO IS PERFECTO?
A Cloud Based Continuous Quality
Lab enabling application testing and
monitoring on real devices in real
end user conditions, for enterprises
adopting digital transformation and
agile delivery processes
INDUSTRY LEADERSHIP
PERFECTO OFFERS THE TESTING PLATFORM FOR COMPLETE DIGITAL COVERAGE
Digital Test Suite Reporting for Agile Teams
One ScriptFor All Digital Platforms
One Lab – Web & MobileOne Secure Cloud
Manual & Automated Testing
GLOBAL, ENTERPRISE-GRADE DIGITAL LAB
05/01/2023 17© 2015, Perfecto Mobile Ltd. All Rights Reserved.
OPEN AND INTEGRATED
18
Open & Integrated
XcodeSelenium
HOW PERFECTO EXTENDS APPIUM
ACHIEVE HIGH %OF AUTOMATION
SYSTEM LEVEL
CONTROL
IMAGE INJECTION
OCR & IMAGE OBJECT
RECOGNITION
RICH REPORTS
MIMIC REAL USER ENVIRONMENT
GPS INJECTION
NETWORK VIRTUALIZATI
ON
PERSONA-BASED
TESTING
HYBRID CLOUDGLOBAL
COVERAGE
GET ENTERPRISE
GRADE
ENTERPRISE SECURITY
24X7 SUPPORT
SCALE & FLEXIBLE
ROLES & PERMISSION
S
MAKING OPEN-SOURCE
ENTERPRISE READY
OPEN SOURCE
IOS & ANDROID
NATIVE/DOM OBJECT
RECOGNITION
CI SUPPORT
BENEFITS OF USING APPIUM WITH PERFECTO
• Reliable unattended parallel automation• Reuse the same Appium scripts and skill-set• Fixed Appium/Selenium defects that cause failed executions fixed• Unlimited parallel execution, no setup required
• OS-level and environmental controls• You can interact with anything on the screen testing real-user interactions• Test pop-ups, incoming calls, SMS, interruptions, etc• Full ability to test a variety of real user conditions
• Accelerate tester and developer productivity• Integrated reporting engine provides screenshots, videos, script and device
analytics, and more accelerating defect triage and giving executive visibility• IDE plug-ins for popular IDEs accelerate script development and debugging
• Enterprise Support Model• 24x7 support, global, secure, audited environments• OS Beta support. Same-day support for GA versions.
05/01/2023 20© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Appi
um w
ith P
erfe
ctoA
ppiu
m O
nly
IMPACT: ACHIEVE COMPLETE TESTING COVERAGE WITH VISUAL ANALYSIS, OS-LEVEL TESTING, AND REAL END-USER CONDITIONS
05/01/2023 21© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Verify the user is able to enter only valid postal code • Verify the user is able to enter only valid post code
by checking input exists in DOM
• Verify the continue button is visible so the user can proceed to the next screen
• Verify the screen renders within acceptable time under WiFi, LTE and 3G connectivity
• Validate that when invalid postal code is entered, error message is colored red
IMPACT: UNDERSTAND THE IMPACT OF REAL-WORLD CONDITIONS ON APPLICATION PERFORMANCE WITH PERSONA-BASED TESTING
22
Marketing / Business Business-Engineering alignment Automated lab configuration Communicate Risk in business terms
((( )))Name: SaraLocation: LondonAge: 25Device: NewSensitivity to UX: HighBehavior: Fashion magazine designer
Engineering
Name: SaraLocation: New YorkAge: 25Device: NewSensitivity to UX: HighBehavior: Fashion magazine designer
Data?
Test Cases
Name: SaraLocation: New YorkDevice: iPhone 7, iOS 10Carrier: BTNetwork: 3GBackground Apps: Instagram
IMPACT: UNDERSTAND HIGH-LEVEL DIGITAL QUALITY TRENDS AND QUICKLY NARROW DOWN PROBLEMATIC AREAS
05/01/2023 23© 2016, Perfecto Mobile Ltd. All Rights Reserved.
IMPACT: ACCELERATE DEFECT RESOLUTION BY ZOOMING INTO DETAILS OF EACH FAILED TEST EXECUTION WITH DEVICE VITALS, SCREENSHOTS AND VIDEOS
05/01/2023 24© 2016, Perfecto Mobile Ltd. All Rights Reserved.
PERFECTO PLATFORM
IN-HOUSE OR3RD PARTY LAB
APPIUM & SELENIUM
PERFECTO ENTERPRISE EXTENSIONS
AUTOMATION FRAMEWORK
AUTOMATION SCRIPTS
CONTINUOUS INTEGRATION
< 50% AUTOMATION
90%+ AUTOMATION
ACHIEVE HIGH BUSINESS VALUE
WITH PERFECTO & APPIUM
05/01/2023 26© 2016, Perfecto Mobile Ltd. All Rights Reserved.
DEMO