Integrating Mobile Applications - Contrasting the Browser with Native OS Apps
Cary FitzGerald
Browser vs. Native
• The Web is more than Browsers
– AJAX & REST transcends browsers
– JSON is the standard data exchange format
• Real Time communication is more than Chat
• The challenge and opportunity is to find unique ways to leverage WebRTC
6
Native Apps
• Compiled, Packaged app
• True Native or Hybrid
• Deployed separate from use
• Not just Phones and Tablets– Augmented / Virtual Reality
– Embedded Systems
– Desktop Apps
– Servers
7
Browsers
PROS
• Leverage existing expertise
• Develop once, use everywhere (in theory)
• Simple API
• Simple deployment
CONS
• Design Complexity
• Limited API
• iOS not supported
• Windows Phone not supported
• IE not supported
8
Native Aps
Pros
• Background operation
• Optimized UI and I18N
• Preapproved access to resources
• Easy access to device features
Cons
• Many languages to support
• Little consistency
• More complex deployment
• Mostly DIY*
9
When to use a Browser
• Secondary Feature of Larger App
• Very Simple App
• Minimally Viable Product
10
Primary Interaction is Real Time
• Communication while in the background
• Offline notifications
• Switch Camera (front/back)
11
Augmented Reality
• Great opportunity for unique interactions
• Expert assistance overlay
• Live Games
14
Managing multiple platforms
• Just bite the bullet
• Translation
• Hybrid
• Cross-compile with Xamarin
16
Acision at-a-glance
Heritage & history
20 year history
270 customers globally
2 trillion messages/ year
Global presence & reach
35+ offices across 6
continents
850+ employees
100+ support team
1st to launch commercial
SMSC
Cloud & virtualized
services
Multi-Channel messaging
+ WebRTC
Innovation
Data and insights
Carrier grade security
Operational Excellence
24/7 Support
Android
• Supported in Google Chrome, Mozilla Firefox, and Ericsson Bowser* for Android
• Mobile browser support trails desktop browser support
• WebRTC in WebViewssupported in Android Lollypop
iOS
• Supported in Ericsson Bowser*for iOS
WebRTC in browsers on mobile devices
* Ericsson Bowser is not yet (and might never be) a platform for consumers to use
Blackberry 10
• Natively supported within the OS
– Google Chrome rendering engine in built into BB10
Windows Mobile
• No support today
WebRTC in browsers on mobile devices
WebRTC in browsers on mobile devices
• Support for WebRTC in mobile browsers is inconsistent across the various platforms
• Even on Android it lags behind what is available in desktop browsers
• Other features needed by many apps (for example, mobile push notifications) still require native apps
For the rest of this presentation I will only consider Android and iOS
Google Media Engine (GME)
• Liberal open-source license
• Used within Google Chrome and parts used within Mozilla Firefox
• G.711 and Opus audio
• VP8 video
• DataChannel support
• Widely used today
OpenWebRTC (from Ericsson)
• Liberal open-source license (BSD)
• Used within Ericsson Bowser
• G.711 and Opus audio
• H.264* and VP8 video
• No DataChannel support (yet)
• Not widely used today
WebRTC libraries for mobile devices
* Does not confer a license to use H.264
WebRTC libraries for mobile devices
• The GME is the best known and widest used library today
– The only option if you want to use DataChannels right now
• OpenWebRTC is worth watching
– The fact that it contains an H.264 implementation doesn’t mean developers can just use it
For the rest of this presentation I will only consider the GME
Android
• Click-2-call style apps can be used in Google Chrome on Android– You can’t receive sessions when the browser is closed
– You can’t write WebRTC apps that require mobile push notifications
• Some hybrid apps (combination of native code and WebViews using WebRTC) are possible on Android Lollypop
WebRTC Architecture
RTCWeb
G.711/OPUS Codec
NetEQ for Voice
Echo Canceller /Noise Reduction
Voice Engine
H.264/VP8 Codec
Video Jitter Buffer
Image enhancements
Video Engine
SRTP
Multiplexing
P2PSTUN + TURN + ICE
Transport
Audio Capture/Render Video Capture Network I/O
Session management / Abstract signalling (Session)
WebRTC C++ API (PeerConnection)
WebRTC API
You
r we
bap
p #1
You
r we
bap
p #2
You
r we
bap
p #3. . . The web
Your browser
Based on the diagram from http://www.webrtc.org/reference/architecture
WebRTC C++ API
• The C++ API documentation at http://www.webrtc.org/is very basic
– There is a close mapping between the functions defined in the W3C WebRTC draft (for JavaScript) and the C++ API
– You can infer the C++ API behaviour by interpreting the W3C draft
• Getting the WebRTC C++ API working on a native mobile device is not easy
The media engine isn’t enough
• Even once you’ve gotten the media engine working you need more to establish your sessions– Discovering peers and exchanging media capabilities and
connection details needs to happen before the media session can be established
• You need to consider user management and provisioning too
There are many developer toolkits and SDK available to help you here (each with slightly different benefits) – take advantage of them
Consider using OAuth2 if you can
Web Service
OAuth2Provider
Authentication Code
Au
then
tica
tio
n C
od
e
Acc
ess
Toke
n
Req
ues
t Id
enti
ty Iden
tity
Identity
1
2
4
5
6
3
7
8
Consider using OAuth2 if you can
UA
SignallingServer
Web Service
5: Shared secret
Communication linknot required
You don’t have tocreate or manageaccounts on theSignalling Server
Android ExampleAcisionSdkConfiguration config =
new AcisionSdkConfiguration("MYAPIKEY ", "user", "password");
config.setApplicationActivity(this);
AcisionSdk acisionSdk = new AcisionSdk(config, new AcisionSdkCallbacks() {
@Override
public void onConnected(AcisionSdk acisionSdk) {
session = acisionSdk.getWebrtc().connectTo("+441632960000",
new WebrtcSessionCallbacks() {
@Override
public void onConnected() {
session.setLocalView(localVideoView);
session.setRemoteView(remoteVideoView);
}
}, new WebrtcOptions());
}
});
iOS ExampleACSAcisionSDKConfiguration *config =
[ACSAcisionSDKConfiguration acisionSDKConfigurationWithAPIKey:@"MYAPIKEY”
username:@"user" password:@"password"];
self.acisionSDK = [ACSAcisionSDK acisionSDKWithConfig:config delegate:self];
- (void)onConnected:(ACSAcisionSDK *)acisionSDK {
ACSAcisionSDKWebRTCOptions* options =
[[ACSAcisionSDKWebRTCOptions alloc] init];
self.currentSession =
[self.acisionSDK.webrtc connectTo: @"+441632960000”,
delegate:self options:options];
}
-(void)onConnected {
self.currentSession.localView = self.localView;
self.currentSession.remoteView = self.remoteView;
}
JavaScript Example
var lib = new AcisionSDK("MYAPIKEY", {
onConnected: function() {
var session = lib.webrtc.connect("+441632960000");
session.remoteVideoElement = document.getElementById("vidRemote");
session.remoteAudioElement = document.getElementById("audRemote");
}
}, {
username: "user",
password: "password"
}
);