Application Development for Mobile and Ubiquitous ...ts2/admuc/lecture1415/7. Web-based Mobi… · Ubiquitous Computing 7. Web-based Mobile Applications Dr. Ing. Thomas Springer Technische

Department of Computer Science Institute for System Architecture, Chair for Computer Networks

Application Development for Mobile and Ubiquitous Computing

7. Web-based Mobile Applications

Dr. Ing. Thomas Springer Technische Universität Dresden

Chair of Computer Networks

Lecture Structure

Dr. Thomas Springer Application Development - 7. Web-based Mobile Applications 2

Table of Contents

§  Motivation and Challenges §  Main Principles

•  Responsive Web design •  Mobile First •  Design patterns

§  Further Features in HTML5 §  Server-side adaptation

3 Dr. Thomas Springer Application Development - 7. Web-based Mobile Applications

Technologies for Web Applications

4

§  Upcoming HTML5 and CSS 3 •  advanced interface styling •  plugin-less multimedia and computing capabilities •  layout transformation for multiple screen sizes •  offline application support •  alternative media files

•  quickly start new HTML5 project: www.initializr.com

Dr. Thomas Springer Application Development - 7. Web-based Mobile Applications

Challenges of Mobile Web Design

§  Dependency to network connection §  Heterogeneous device support (desktop, tablets, smartphones)

•  Slower network and CPUs •  Screen size can no longer be fixed •  Native look and feel •  Touch/gesture input •  Access to native functions (e.g. location or camera)

Ø  Flexible adaptation of information/media towards different

devices and networks required

http://line25.com/articles/showcase-of-outstanding-responsive-web-designs 5 Dr. Thomas Springer Application Development - 7. Web-based Mobile Applications

Responsive Web Design

§  Main adaptation problem: interface scaling over different device screens •  inofficial default screen width for PCs: 1024 px •  screen width range on mobile devices: 320 - 2048 px •  classical approach: different versions of web site/app

(e.g. mobile.example.com)

§  Responsive Web Design •  principle described by Ethan Marcotte •  single HTML page markup

(avoid separate mobile version) •  adapts to screen size through CSS •  main concepts:

o  Fluid grids o  Flexible images o  CSS Media Queries

http://en.wikipedia.org/wiki/File:Boston_Globe_responsive_website.jpg


Fluid Grids

§  page-layout based on grid system •  maximal layout width as starting point •  layout defined by columns with fixed width / margins

§  translate fixed grid sizes to proportional values •  % or em (relative font size) for block elements and font sizes •  value for each element proportional to parent (context):

element size / parent size = relative size

960px69px

900px

12px

566px 331px

Article566 ÷ 900 = 62.8888889%

.article {width: 62.8888889%;float: left;}

Sidebar

.sidebar {width: 36.7777778%;float: right;}

331 ÷ 900 = 36.7777778%

Banner 900 ÷ 960 = 0.9375.banner {width: 93.75%;}

#page {width: 90%;} (90% of browser window -‐> equals to 960px)

7

960 Grid System (960.gs) 1200px Grid System (1200px.com) Blueprint (blueprintcss.org)

Dr. Thomas Springer Application Development - 7. Web-based Mobile Applications

Flexible Images

§  large images don't automatically resize •  break layout if larger than parent element

§  simple solution: •  img {max-width: 100%;} •  force image to match width of its container •  resized proportionally in most browsers (keeps ratio) •  also for other elements (video, object, embed)

§  low quality resizing in older Windows browsers •  Workaround: img { -ms-interpolation-mode: bicubic; } •  works in IE7 or higher


Multi-resolution images

§  Problem: images don't scale across current screen resolutions (72 - 320 dpi) •  recent problem due to high res (retina) screen on mobile devices •  many mobile devices need higher resolution images than PCs •  network bandwidth needs to be considered

§  Solution: <picture> element •  currently developed by W3C, not implemented yet •  similar to <video> •  multiple sources bound to media attributes (e.g. min-width) •  src-set attribute contains list of image files bound to device-pixel-ratios •  device-pixel-ratio: ratio between physical pixels and pixels on device,

given in 1x, 1.5x, 2x iPhone: 320 physical px and 320 browser px => 1x re9na iPhone: 640 physical px and 320 browser px => 2x

<picture alt="Descrip9on of image subject"> <source media="(min-‐width: 18em)" srcset="med.jpg 1x, med-‐highres.jpg 2x"> //small layout <source media="(min-‐width: 45em)" srcset="large.jpg 1x, large-‐highres.jpg 2x"> // larger layout <img src="small.jpg" alt="Descrip9on of image subject."> //fallback for older browsers </picture>


Multi-resolution images

§  <picture> not yet in browsers, but possible through Javascript •  e.g. picturefill (http://scottjehl.github.com/picturefill/) •  emulates <picture> element programmatically •  uses <div> and data-* attributes

<div data-‐picture data-‐alt="A giant stone face at The Bayon temple in Angkor Thom, Cambodia"> <div data-‐src="small.jpg"></div> <div data-‐src="medium.jpg" data-‐media="(min-‐width: 400px)"></div> <div data-‐src="large-‐highres.jpg" data-‐media="(min-‐width: 800px) and (min-‐device-‐pixel-‐ra9o: 2.0)"></div> <div data-‐src="extralarge.jpg" data-‐media="(min-‐width: 1000px)"></div> <noscript> <img src="external/imgs/small.jpg" alt="A giant stone face at The Bayon temple in Angkor Thom, Cambodia"> </noscript> </div>


CSS Media Queries

§  Conditionally apply styles based on browser attributes •  multiple CSS styles for the same elements

§  Mainly: style content based on the screen size •  define multiple page layouts with CSS on same HTML markup •  define breakpoints for different layouts (e.g. screen width) •  show/hide/resize/move elements depending on layout

§  List of example sites: http://mediaqueri.es

Internet Images SRL, http://interim.it


CSS Media Queries

§  media types: •  all | aural | braille | handheld | print | projection | screen | tv

§  limit scope of CSS properties by media type / feature: •  <link rel="stylesheet" media="only screen and (color)“

href="example.css" /> •  @media screen and (max-‐width: 600px) { ... }

§  any combination of single media type and chained media features (and operator) •  @media screen and (min-‐device-‐width: 480px) and (orientation: landscape) •  @media screen and (max-‐width: 1200px) and (min-‐resolution: 260dpi) and

(aspect-‐ratio: 1/1)

§  media features: •  width | min-‐width | max-‐width | height | min-‐height | ... •  device-‐width | min-‐device-‐width | max-‐device-‐width | device-‐height | ... •  aspect-‐ratio | min-‐aspect-‐ratio | max-‐aspect-‐ratio •  resolution | min-‐resolution | max-‐resolution •  orientation | ...

stylesheet loading

inside stylesheet


Multi-Device Layout Patterns

§  Luke Wroblewski has compiled list of popular responsive layouts: §  Creation based on the principles of responsive web design §  See examples at: http://www.lukew.com/ff/entry.asp?1514

Column Drop

Tiny Tweaks

Off Canvas 13 Dr. Thomas Springer Application Development - 7. Web-based Mobile Applications

Mostly Fluid

Layout Shifter

Example - Mostly Fluid

320px - Phone

1024px - large Tablet, Desktop

768px - large Tablet portrait


480px - small Tablet, Phone landscape

Example - Mostly Fluid

§  Mobile first - base CSS (320px): §  2nd media query (768px):

§  1st media query (480px): §  3rd media query (1000px+):

body { margin: 10px 0; } header, nav, article, footer { width: 100%; } nav ul li { width: 100%; }

@media (min-‐width: 480px) { nav ul li { float: left; width: 25%; /* 4 Items */ } article div.list div { float: left; width: 50%; } }

@media (min-‐width: 768px) { body { margin: 20px 0; } header { float: left; } nav { float: right; width: auto; } }

@media (min-‐width: 1000px) { article { float: right; width: 75%; } aside { float: left; width: 25%; } }


Mobile First

§  mobile web is the future market •  "Mobile phones will overtake PCs as the most common Web

access devices worldwide by 2013" - Gartner •  optimize content for mobile also benefits desktop

o  Focus on most important content, tune content for limited resources •  Mobile First approach promoted by Luke Wroblewski

§  Principle: start with mobile first •  small screen

o  condensed navigation / content (reduce to most important parts) o  responsive layout

•  slow network o  reduce requests and file sizes o  image sprites, single CSS/JS files, minfied (tools like Google Minify) o  ApplicationCache, localStorage o  CSS 3 / <canvas> / SVG instead of raster images

•  mobile device usage o  optimize for one-hand navigation / partial attention of user


Mobile First - Innovation

§  touch input •  touch target sizes:

o  recommended: 9mm/34px o  minimum: 7mm/26px

•  visual size is 60-100% of target size •  use multi-touch input:

o  Tap | Double Tap | Drag | Flick | Pinch | Spread | Rotate •  interface paradigms: drag to reveal, drag to refresh, ...

§  device features •  location

o  adapt content to user location •  compass

o  augmented reality based on user's location and direction •  orientation (portrait, landscape)

o  optimize layout (navigation, content) to device orientation •  camera

o  use images as input / output


Mobile Enablers in HTML5

§  based on XHTML 1.1 and HTML 4 §  authored by W3C and WHATWG §  new elements and attributes §  new technologies, modularized under sections:

•  promoted under HTML5, but most are separate standards

§  more capabilities for web applications §  reduce necessity of proprietary plugins

•  plugins often not available on mobile devices

Semantics Multimedia

Offline & Storage 3D, Graphics & Effects

Device Access Performance & Integration

Connectivity CSS 3.0


Semantics

§  semantic elements: •  <article> •  <footer> •  <header> •  <nav> •  <section>

§  add structure to HMTL markup

§  microdata: •  add extra semantic information to

HTML markup (e.g. phone number, address, name)

•  itemscope defines vocabulary o  schema.org, Rich Snippets, WHATWG

•  itemprop o  assign word of vocabulary to element

(e.g. itemprop=„name“> John Doe)


Multimedia

§  <video> and <audio> tag: §  play video / audio files without plugin

•  control via browser agent or Javascript •  multiple file sources possible (codec alternatives) •  attributes: autoplay, controls, muted, ... •  varying codec support across browsers •  video: theora / h.264 / webM •  audio: ogg / wav / mp3 / webM / aac

<video id="myVideo"> //alternative sources <source src="http://media.w3.org/2010/05/sintel/trailer.mp4" type='video/mp4; codecs="avc1, mp4a"'> <source src="http://media.w3.org/2010/05/sintel/trailer.ogv" type='video/ogg; codecs="theora, vorbis"'> Your user agent does not support the HTML5 Video element. //fallback text </video> <button type="button" onclick="vid_play_pause()">Play/Pause</button> //Javascript video API <script> function vid_play_pause() { var myVideo = document.getElementById("myVideo"); if (myVideo.paused) {myVideo.play();} else {myVideo.pause();} } </script>


Device Access

§  access device sensors and data sources: •  Location, camera, microphone, screen orientation, motion /

acceleration, contacts, calendar §  still early development

•  currently only few APIs: o  Javascript Geolocation API o  deviceorientation, devicemotion DOM events

§  mobile frameworks (e.g. Phonegap) also offer similar device access

§  example: Geolocation API •  provides highlevel interface to location information through

Javascript •  adds navigator.geolocation element to DOM •  User can allow/disallow location access! •  http://caniuse.com/#feat=geolocation


// test if Geolocation is supported if (typeof navigator.geolocation === 'undefined'){ alert("Geolocation not supported by your web browser."); } else {geo = navigator.geolocation;} // get Position if (geo) {geo.getCurrentPosition(successFunction, errorFunction); } // read position object and display lat./long. function successFunction(position) { var lat = position.coords.latitude; var long = position.coords.longitude; var acc = position.coords.accuracy; var speed = position.coords.speed; var heading = position.coords.heading; alert('Your latitude is :'+lat+' and longitude is '+long); } // display error message function errorFunction(position) { alert('Error!'); }

Device Access

§  currently supported in •  IE 9+ •  Firefox 15+ •  Chrome 22+ •  Safari 5.1+ •  iOS 3.2 •  Android 2.1


Connectivity

§  WebSockets: •  bidirectional communication channel over TCP •  WebSocket API + WebSocket •  optimized, standardized version of Comet •  long-lived, open connection allows server push instead of polling

•  protocol: o  IETF RFC 6455 o  open WebSocket through HTTP upgrade request

•  API (Javascript): o  create new WebSocket object: new WebSocket(url, protocols) o  send(data), close() methods o  socket events: onopen, onclose, onmessage, onerror


Performance & Integration

§  Web Workers: •  client background Javascript tasks •  long-running, multi-threaded, computationally expensive •  independent from user-interface tasks •  no access to DOM, uses message-passing

§  two types: •  dedicated: linked to creator script •  shared: accessible by all scripts from same origin/domain

§  XMLHTTPRequest 2: •  additional functionality to XHR 1 •  tracking data transmission progress through progress event •  binary data uploads (not directly possible with XHR 1) •  cross-domain AJAX requests


Offline & Storage

§  ApplicationCache: •  control caching of user agent (browser) •  instruct with cache manifest file •  list files in CACHE, NETWORK (online only) and FALLBACK

sections •  Javascript API to force cache updates

§  Web Storage: •  store named key/value pairs locally •  Javascript API •  value can be any data type supported by Javascript •  size limitation per origin/domain (5-10MB)

§  two storage types: •  localStorage: no expiration date for stored data •  sessionStorage: data flushed after session


Offline & Storage

§  IndexedDB: •  noSQL transactional local storage •  persist (complex) Javascript objects as key/value pair •  indexing of value properties •  no hard size limit (user permission needed for 50MB+)

§  asynchronous Javascript API: •  transaction requests, retrieve data via callbacks •  synchronous API for Web Workers planned

§  File API: •  access local files in Javascript •  user has to select files through <input> element or drag and drop •  read single File, directory (FileList) or binary data (Blob) •  uses FileReader interface •  reference local files through Blob URIs


CSS 3.0

§  more advanced interface styling: •  rounded borders •  shadows •  color alpha channel (RGBA) •  multiple backgrounds

§  transitions and animations: •  timed transition between two CSS styles •  animations: complex transitions over multiple styles

§  web fonts: •  Embed font types with @font-face •  Embedded Open Type (.eot), TrueType (.ttf), OpenType (.otf)

§  reduce necessity for plugins (Flash, ...) and images (e.g. for rounded borders)


HTML5 – Current State

§  HTML5 •  planned "Candidate Recommendation" status by end of 2014 •  most standards further in the future (Working Draft)

§  use of features depend on two factors: •  implementation across different browsers •  standardized interface between implementations

§  www.caniuse.com: •  check browser support for each technology

http://caniuse.com/#feat=websockets


Mobile Web Application Development

§  Responsive CSS Frameworks: •  Bootstrap (twitter.github.com/bootstrap) •  jQuery UI (jqueryui.com) •  Foundation (foundation.zurb.com) •  many more...

§  Mobile Web Application Frameworks: •  Sencha Touch (www.sencha.com/products/touch) •  jQuery Mobile (jquerymobile.com) •  M-Project (the-m-project.org) •  LungoJS (www.lungojs.com) •  Joshfire (framework.joshfire.com)

§  Testing / Debugging •  Ripple (Chrome extension, ripple.tinyhippos.com) •  Adobe Edge Inspect (html.adobe.com/edge/inspect)


Sencha Touch 2

§  MVC Javascript framework for mobile applications §  ability to package as native iOS/Android app §  platform support:

•  iOS •  Android •  Blackberry

§  layout structure/design optimized for iOS

§  provides class system based on ExtJS •  Pure Javascript framework for cross-plattform applications

§  application built of: •  Models •  Views •  Controllers •  Stores (model persistence) •  Profiles/Themes (device classes/UI)


Sencha Touch 2 - Concepts

§  Components (visual interface classes) •  interface widgets (subclass of Ext.Component) •  nested inside Containers or floating •  e.g. Panels, Forms, DataViews, Lists, Maps, ...

§  Layouts (sizing and positioning of Components) •  defined in Containers •  flexible vertical, horizontal grid | card layout •  docking

§  Events •  on changes to Components •  custom event firing / listeners •  event scoping / buffering

§  Environment Detection •  Platform (Ext.os.name) •  Browser (Ext.browser.name) •  Browser Features (Canvas, CSS3, Geolocation, Touch, ...)

var panel = Ext.create('Ext.Panel', { html: 'This is my panel‘ }); Ext.Viewport.add(panel);

some Layouts:


Sencha Touch 2 - Touch Events

§  Sencha Touch supports a variety of touch events

•  used through event listeners •  listeners called in order they were added

§  supported events: •  touchstart •  touchmove •  touchend •  dragstart •  dragdragend •  tap •  singletap •  doubletap •  longpress •  swipe •  pinch (on iOS and Android 3+) •  rotate (on iOS and Android 3+)

// event listener set with .on() myBuZon.on('tap', func9on(event) { alert("User tapped this buZon."); }); // Pinch to zoom: // pinchstart fired once myElement.on('pinchstart', func9on(event) { this.baseWidth = this.getWidth(); this.baseHeight = this.getHeight(); }); // pinch fired con9nuously when there is pinching myElement.on('pinch', func9on(event) { var scale = event.scale; this.setWidth(this.baseWidth * scale); this.setHeight(this.baseHeight * scale); }); // pinchend fired once myElement.on('pinchend', func9on(event) { alert("Finished pinch-‐to-‐zoom"); });


Sencha Touch 2 - Look and Feel

§  Sencha app look depends on CSS styling and Component layout

•  e.g. iOS blue navigation bar vs. Android black tab bar

§  Styling in CSS files = Themes §  Platform-specific coloring/styling provided

with preshipped Themes •  sencha-touch.css •  apple.css •  android.css •  bb6.css

§  Component layout customized for each platform through Profiles

•  e.g. Phone and Tablet profile

§  Further customization necessary to support platform-specific layout!

•  e.g. iOS back-button vs. Android hardware button

// switch Theme depending on pla?orm Ext.setup({ onReady:func9on(){ if(Ext.is.Android){ Ext.get("myStylesheet").dom.href = "android.css"; } else if(Ext.is.iOS){ Ext.get("myStylesheet").dom.href = "apple.css"; } } }); // switch View depending on Profile // On a phone, fill complete viewport Ext.define('view.phone.Form', { extend: 'view.Form', config: { 9tle: 'Phone-‐specific version‘ } }); // On a tablet, use modal with fixed size Ext.define('view.tablet.Form', { extend: 'view.Form', config: { 9tle: 'Tablet-‐specific version', modal: true, height: 505, width: 480, centered: true } });


Web Application Deployment

§  in browser (all platforms): •  accessible through URL / browser bookmark •  run inside device browser app •  offline available when using ApplicationCache

§  on "home screen" (most platforms): •  create bookmark on device home screen (next to native apps) •  on iOS:

o  opens in fullscreen browser: <meta name="apple-‐mobile-‐web-‐app-‐capable" content="yes" />

o  additional splash screen: <link rel="apple-‐touch-‐startup-‐image" href="./startup.png" />

•  on Android and iOS: o  custom icon:

<link rel="apple-‐touch-‐icon" href="./apple-‐touch-‐icon.png" />

§  native bundle (most platforms): •  bundle web application in native app through mobile application framework

(e.g. PhoneGap) •  application is run in fullscreen WebView •  more functionality through mobile application framework (e.g. PhoneGap)


Server-side adaptation

§  Opera Mini •  requested web page rendered on proxy server •  rendered page compacted to binary Opera Binary Markup

Language (OBML) and send to client •  reduces download sizes by up to 90%

•  limitations: o  only onLoad-events (server-side Javascript) are fully supported o  on client, only user-input events (onclick, onchange, ...) supported o  limited Ajax support -> page only changes if user clicks something

Opera Mini(Java ME)

Opera Mini Server

Web Server1

Web Server2

Web Servern

...

HTMLOBML

12

3

45

6


Server-side adaptation

§  Amazon Silk •  browser on Kindle® devices •  browser subsystems (fetching, Javascript, rendering) also

implemented in Amazon Web Services cloud (EC2) •  dynamically split which subsystems executed in EC2

and on device: •  cloud advantages:

o  computationally expensive tasks (rendering, scripts) done in cloud o  faster fetch of referenced files (CSS, Javascript, images, ...) o  predictive caching, through all Silk users o  optimized image compression

(since Silk knows browser/device) Dynamic Split


Summary

§  Adaptive Web applications for mobile devices

§  Major principles: •  Responsive Web design

Ø  Scalable layout and images Ø  Alternative layout and content based on Media Queries

•  Mobile Device Layout Patterns Ø  Adapt layout for different devices/view sizes

•  Mobile First Ø  optimize content for mobile -> also benefits desktop

§  HTML5 provides key enablers for mobile •  Device access, CSS3, Multimedia, Offline and Storage,… •  Extended browser capabilities

§  Content adaptation in the infrastructure Ø  Reduce data volume for transfer Ø  Support server-side processing/rendering


Some further readings

§  HTML5 Technologies Specifications: •  HTML5: http://dev.w3.org/html5/spec/single-page.html •  CSS: http://www.w3.org/Style/CSS/Overview.en.html •  Microdata: http://www.w3.org/TR/html5/microdata.html •  SVG 2: https://svgwg.org/svg2-draft/ •  WebGL: https://www.khronos.org/registry/webgl/specs/1.0/ •  Device Access: http://www.w3.org/2009/dap/ •  WebSockets: http://www.w3.org/TR/websockets/ •  Web Workers: http://www.w3.org/TR/workers/ •  XMLHttpRequest 2: http://www.w3.org/TR/XMLHttpRequest/ •  ApplicationCache: http://www.w3.org/TR/offline-webapps/ •  Web Storage: http://www.w3.org/TR/webstorage/ •  Indexed Database: http://www.w3.org/TR/IndexedDB/ •  File API: http://www.w3.org/TR/FileAPI/

§  Crowd-Docs of Web Standards: www1.webplatform.org §  Ethan Marcotte: Responsive Web Design, A Book Apart, 2011 §  Luke Wroblewski: Mobile First, A Book Apart, 2011 §  Luke Wroblewski: Multi-Device Layout Patterns, http://

www.lukew.com/ff/entry.asp?1514 38 Dr. Thomas Springer Application Development - 7. Web-based Mobile Applications

Application Development for Mobile and Ubiquitous ...ts2/admuc/lecture1415/7. Web-based Mobi… · Ubiquitous Computing 7. Web-based Mobile Applications Dr. Ing. Thomas Springer Technische

Documents