CI for iOS: do more while you sleep Kevin Munc – @muncman Method Up LLC [MU]
May 12, 2015
CI for iOS:do more while you sleep
Kevin Munc – @muncmanMethod Up LLC
[MU]
CI?
For iOS?
Pain
Pain & Suffering
It can be done
• Start with the basics
• Xcode and the command line
• Version control and CI server
• Script it, run it, schedule it
Tip #1
Jenkins
• http://jenkins-ci.org/
• Mac installer or via homebrew
• Many alternatives
• Many plugins
Tip #1.1
How many Simulators does it take to break a build?
A CI server gives you:
A CI server gives you:
Automation
A CI server gives you:
Automation
Feedback
A CI server gives you:
Automation
FeedbackWonderful!
A CI server gives you:
Automation
FeedbackWonderful!
Peace o
f mind
(for bette
r naps)
Requirement: OS X
Requirement:
• macminicolo.net
• hosted-ci.com
• cisimple.com
• macincloud.com
• hostmyapple.com
• macminivault.com
• xcloud.me
OS X
Requirement:
• macminicolo.net
• hosted-ci.com
• cisimple.com
• macincloud.com
• hostmyapple.com
• macminivault.com
• xcloud.me
OS X
Or schedule CI to run at night,
on your own Mac,
while you sleep...
Securitysudo /usr/sbin/DevToolsSecurity --enable
Security
sudo xcodebuild -license
Credentials
• The CI server needs access to your repository
• SSH key
• The CI server needs Keychain credentials if you want to sign your builds
• Developer certificate and profiles
Keychain
• Export and Import using Keychain Access app
• Keys and Certificates
• Or use the command line:• sudo security import /path/to/distribution.cer -k /Library/Keychains/
System.keychain
• sudo security import /path/to/distribution.p12 -k /Library/Keychains/System.keychain
• Don’t forget about provisioning profiles
• Troubleshooting: http://tinyurl.com/iOSDevToolTroubles
Tip #2
Avoid this error from Xcode
RunPlatformUnitTests: warning: Skipping tests;
the iPhoneSimulator platform does not currently support application-hosted tests (TEST_HOST set).
The culprit
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Tools/RunPlatformUnitTests
The workaround(s)
Xcode Plugin
• https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin
• Specify provisioning profile
• Keychain access
• Signed IPA
Xcode Plugin
• Recommended xcodebuild arguments:
• GCC_SYMBOLS_PRIVATE_EXTERN=NO
• COPY_PHASE_STRIP=NO
• Allows test bundle to link with Release build symbols
Signing with the Plugin
Signing with the Plugin
Signing with the Plugin
Tip #3
Prefer Scripts over CI Plugins
• More flexibility
• More resilient to Apple’s changes
• More power to adapt and expand (specificity)
• Less coupling to the specific CI server type
• You can version control the settings easier
xcodebuild
xcodebuild \ clean build
Tip #4
Be specific with xcodebuild options
xcodebuild
xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
xcodebuild
xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
xcodebuild
xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
xcodebuild
xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
xcodebuild
xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
xcodebuild
xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
xcodebuild
xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
xcodebuild
xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build
Tip #5
Specify Alternate Output Locations
• Don’t rely on Xcode’s cryptic locations
• This will make your scripts easier to manage
# Build location for Instruments test runs. CI_BUILD_DIR=/tmp/SOLID_AUTOMATIONmkdir -p $CI_BUILD_DIR
xcodebuild \ ... \ CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \ clean build
# Build location for Instruments test runs. CI_BUILD_DIR=/tmp/SOLID_AUTOMATIONmkdir -p $CI_BUILD_DIR
xcodebuild \ ... \ CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \ clean build
# Build location for CI test runs. CI_BUILD_DIR=/tmp/SOLID_AUTOMATIONmkdir -p $CI_BUILD_DIR
xcodebuild \ ... \ CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \ clean build
Demo
Tip #6
Ensure Return Codes
• If the tool you are invoking in your script doesn’t return an exit code indicating pass/fail, it’s up to you.
• Unix-friendly 0 (success) or 1 (failure)
Tip #7
Test against multiple SDKs
-sdk iphonesimulator5.0-sdk iphonesimulator5.1-sdk iphonesimulator6.0
Other Tools to Know About
• xcode-select (manage xcode path)
• Overridden by DEVELOPER_DIR
• xcrun (find and run dev tools)
• ibtool (for Interface Builder files)
• /Applications/Xcode.app/Contents/Developer/usr/bin/
• Variables in Xcode
• “Build Settings Reference”
OCUnit
Other Related Tools
Other Related Tools• OCMock
• OCMockito
• OCHamcrest
• Expecta
• TinyMock
• LRMocky
• OCUnit2JUnit
• OCUnitReport
• Kicker
• xcodebuild-rb
• xcodearchive
• XcodeTest
• AutoBuild
• xcodeproj (CocoaPods)
• Circle?
• Simon?
• ios-maven-plugin
• Ceedling
• XcodeCoverage
• JMRTestTools
• Nocilla
• OHHTTPStubs
OCUnit Alternatives
• Kiwi
• Cedar
• OCDSpec
• C++ Automated Test Cases in Headers (CATCH)
• Objective-Shoulda
• Specta
Tip #8
OCUnit2JUnit for Test Results
• ocunit2junit.rb
• https://github.com/ciryon/OCUnit2JUnit
• Converts output to JUnit format for easier transformation to HTML
UIAutomation
Other UIAutomation Tools
• tunup_js
• jasmine-iphone
• uiautomation-jasmine-iphone
• Bwoken
• Zucchini Framework
• WaxSim
• ios-sim
• ios-sim-locale
• iphonesim
• ui-auto-monkey
• ui-screen-shooter
• uiautomation-rb?
UIAutomation Alternatives
• Frank
• (iCuke)
• KIF
• Calabash
• UISpec
• Bromine
• Objective C Slim (ocslim)
• AutomationKit
• Sikuli
• MonkeyTalk (formerly FoneMonkey)
• Plus other commercial tools, such as Telerik Test Studio for iOS
Tip #9
PListBuddy is Your Buddy
/usr/libexec/PlistBuddy
Tip #10
Always quit the simulator
• killall
• killall -m -KILL "iPhone Simulator"
• AppleScript
• osascript -e 'tell app "iPhone Simulator" to quit'
Tip #11
Use Jonathan Penn’s AutomationExample
https://github.com/jonathanpenn/AutomationExample
Demo
More?
Code Coverage
“Risk coverage”
Coverage Tools
• gcov
• GCC’s coverage tool
• GUIs for gcov
• LCOV (HTML)
• CoverStory (Mac app)
• ocov?
Tip #12
gcovr for Coverage Automation
Produces Cobertura-formatted output from gcov data
Tip #12.1
Enable Test Coverage
Static Analysis
• Clang
• http://clang-analyzer.llvm.org/
• scan-build
Tip #13
Symbolic links can be your ally
Avoid scan-build Painsudo ln -s \
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/
libarclite_iphonesimulator.a \/usr/lib/arc/libarclite_iphonesimulator.a
What about test build deployment?
Tip #14
Use TestFlight
• testflightapp.com
• OTA deployment
• SDK for more features
• Free!!
TestFlight
• Web UI
• Desktop App
• REST API
• Jenkins Plugin
TestFlight Alternatives
• HockeyApp
• http://hockeyapp.net
• Also has a Jenkins plugin (forked)
• BetaBuilder for iOS
• http://www.hanchorllc.com/betabuilder-for-ios/
• More players in this space, esp. for enterprises...
TestFlight Config
• API Token and Team Token
• Need a signed IPA
• They get an email with a link for OTA installation!
Tip #15
Protect Your Tokens
• API Token & Team Token
• Keep them out of scripts
• Instead, define them in Jenkins (and leverage Jenkins security)
• 'Configure System'
• 'Global properties' section
• Or use the plugin (same parameters)
TestFlight Plugin
TestFlight Script
Deployment
Deployment
Q: Once you have a nightly job to build, unit test, sign an IPA and deploy it to QA,
what’s next?
Q: Once you have a nightly job to build, unit test, sign an IPA and deploy it to QA,
what’s next?
A: Sleeping easy.
Documentation
• Appledoc
• Doxygen
• HeaderDoc
DocSets
Appledoc
Doxygen
Tip #16
Don’t stop there!
Other Goodies
Other Goodies
• Ensure your site or API is up
• Scan your code for TODOs and FIXMEs
• Get trend reports for lines of code (SLOCCount w/ sloc2html)
• Scan for duplicated blocks of code (CPD, Simian, etc.)
• Use agvtool (Apple-Generic Versioning Tool) to increment your build number
Still Missing
• Cyclomatic Complexity
• Coding convention/Style checker
• Code Formatter
• Uncrustify, UniversalIndentGUI
• UML Generation
• via CLI, as opposed to OmniGraffle (AppleScript?)
• ER Diagram from Core Data schema
Tip #n
Sources• Slides: http://www.slideshare.net/muncman
• http://blog.carbonfive.com/category/mobile/
• http://www.stewgleadow.com/
• http://lifeandcode.net/
• http://longweekendmobile.com/blog/
• http://longweekendmobile.com/2011/04/17/xcode4-running-application-tests-from-the-command-line-in-ios/
• http://qualitycoding.org
• et cetera
Thank you!
• CodeMash organizers
• CodeMash sponsors
• and YOU!
[MU]
Thank you!
• CodeMash organizers
• CodeMash sponsors
• and YOU!
Questions?
[MU]