Persistent Offline Storage White

JavaScript Persistence & Offline StorageDec 9 2009

Alexei White (Foresee Results - Nitobi)http://ambiguiti.es

@AlexeiRWhite

What is Persistence / Storage?

• Data ‘locker’ that persists between page loads, and/or browser sessions.

• Client-side data storage vs Server-side storage.

• Browsers & JavaScript engines typically do not have file I/O capabilities.

• Many reasons to want to keep data local.

Why Persistent Storage?

• Provide sandbox mode for online documents. (eg Word vs Google Docs)

• Persist trivial data like layout and preferences.• Eliminate unnecessary server requests.• Provide powerful data lookup capabilities

(SQL) to JavaScript apps.• Browser crash fail-safe / offline mode.• Cacheing scripts.

Offline Storage Use Case

Isn’t That What Cookies are For?

• Very limited storage capacity (4000 bytes per)• Limited # of cookies. (~20 max per domain,

~300 cookies max overall)• Sent to server on every request.• Useful storage needs to be in 100k+ range

contiguous.

The Flavors of JS Persistence

Storage Flavors

window.name

• Very old string property of the browser’s window object.

• Used primarily for setting hyperlink targets.• Persistent across pages and domains.• Can be used before domready event.

window.name

• Combine with JSON lib or native code for reading and writing:

// Write a structure to window.name:var myObj = { “a”:”hello world”, “b”:324234 };window.name = JSON.stringify(myObj);

// Read it back:var myObj;try {

myObj = JSON.parse(window.name);} catch(e) {}

window.name• Completely insecure.– Same Origin Policy does not apply.– Data stored in plain text within the DOM.– Can be read outside your website.

• Tends to degrade browser performance a bit.– 2MB upper limit FF or starts to get slow.– 4MB IE same.

• Not Crash-safe.– Wont be restored even if tabs are restored after crash.– Not written to disk.

• Great place to cache trivial data or communicate inter-app.

Flash Local Shared Object

• Key/Value pair storage object built into Flash player 6+.

• Up to 100K free. Afterward, dialogue appears asking user for permission to save more.

• Supported by virtually all browsers via ExternalInterface:– IE5+– Netscape 8+– Firefox 1+– Safari 1.3+– Opera 9+

Flash Local Shared Object

• Step 1: Create a new Flash movie:import flash.external.ExternalInterface;

// Saves data to datastorefunction saveData(store, key, value) {

sharedObject = SharedObject.getLocal(store);sharedObject.data[key] = value;sharedObject.flush();

}

// Retrieves datafunction loadDate(store, key) {

return SharedObject.getLocal(store).data[key];}

ExternalInterface.addCallback(“saveData”, this.saveData);ExternalInterface.addCallback(“loadData”, this.loadData);

Flash Local Shared Object

• Step 2: Embed the movie (manually or via SwfObject)

<object classid=“bla” codebase=“bla” height=“1” width=“1” id=“storageMovie”>

<param name=“movie” value=“sharedobject.swf” /><param name=“allowScriptAccess” value=“always” /><embed src=“shredobject.swf” allowScriptAccess=“always”

width=“1” height=“1” name=“storageMovie” /></object>

Flash Local Shared Object

• Step 3: Communicate via ExternalInterface// Save$(‘storageMovie’).saveData(store,key,value);

// Loadvar result = $(‘storageMovie’).loadData(store, key);

Flash Local Shared Object

• Gotchas:– Flash movie must be on visible portion of screen

at least once before accessing via ExternalInterface.

– If it’s not working:• Check that the movie did load.• It’s on visible portion of screen.• document.getElementById(movieId) returns an object.• allowScriptAccess = “always”

The Lowest Common Denominator

• Flash Local Shared Object– ExternalInterface supported by all browsers.– Provides persistent offline + crash-safe support.– Has wide adoption on web.– Provides decent (~100K+) storage capacity*.– Not wiped when user clears browser cache.

* When combined with GZIP.

Silverlight IsolatedStorage

• Silverlight has same functionality.

• More capacity (1024 KB vs 100 KB).• Binary storage format• Streamwriters / readers• Compression

http://www.shinedraw.com/data-handling/silverlight-vs-flash-local-storage/

System.IO.IsolatedStorage.IsolatedStorageFile

Silverlight IsolatedStorage

http://www.shinedraw.com/data-handling/silverlight-vs-flash-local-storage/

using System.IO.IsolatedStorage.IsolatedStorageFile;

IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication();   if(store.FileExists(FILENAME)){

      isfs = new IsolatedStorageFileStream(FILENAME, FileMode.OpenOrCreate, store);      StreamReader streamReader = new StreamReader(isfs);      string s;      while ((s = streamReader.ReadLine()) != null)          TextArea.Text += (s + '\n');      streamReader.Close();  }    // save data to local storage  IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication();  IsolatedStorageFileStream isfs = new IsolatedStorageFileStream(FILENAME, FileMode.OpenOrCreate, store);  StreamWriter streamWriter = new StreamWriter(isfs);  streamWriter.Write("New Text");  streamWriter.Flush();  streamWriter.Close();

HTML5 Overview

• Next big update to HTML standard.• AKA “Web Applications 1.0”.• In “Last Call” state with WHATWG.• Not necessarily the future of web*.• New stuff:– Canvas tag– Rich media (video)– Storage– Doc editing– Drag and Drop

- Cross-doc messaging- History management- MIME and protocol reg.

HTML5 Features In Jeopardy

• Canvas tag– Microsoft has VML & possibly SVG in IE9

• DOM WebDB (SQL based storage)– No for IE (lots of reasons why)– Probably not for Mozilla

• Media embedding– No for IE *– Confusion over codecs (H.264 / Ogg Theora)– Concern over Patent litigation.

HTML5 Storage Overview

• Two kinds of storage proposed:– Key/value pair (hash) storage• Create expando properties on storage object.• Similar to Shared Local Object in flash.

– WebDB (SQLite based)• Create and populate databases right in the browser• Perform SQL queries on the fly.• Based on SQLite.

HTML5 DOM Storage

• Hash-based storage of strings (not objects).• 10 MB per zone.• Two kinds:– sessionStorage:• Die when browser closes• Not shared between tabs

– localStorage• Crash-safe• Shared BW tabs / windows and sessions – but not

zones.

HTML5 DOM Storage

• Reading and writing:

• Won’t work:

// Writing datasessionStorage.myKey = “Hello”;window.sessionStorage[“myKey”] = “Hello”;

window.localStorage.myKey = “Hello”;localStorage[“myKey”] = “Hello”;

sessionStorage.myObj = {a:true, b:false};console.log(sessionStorage.myObj)// “[Object object]”

HTML5 DOM Storage

• Events (onstorage, onstoragecommit*):function handleStorageEvent(e) {

// These attributes only appear in Safarivar str = “Key: “ + e.key + “\n”;str += “Original Value: “ + e.oldValue + “\n”;str += “New Value: “ + e.newValue;alert(str);

}

// Bindings, IE then W3Cif (document.attachEvent)

document.attachEvent(“onstorage”, handleStorageEvent);else

document.body.addEventListener(“storage”, handleStorageEvent);

*IE8 only

HTML5 DOM Storage

• Clearing:

• Iterating:

• Deleting:

sessionStorage.clear();

for (var I = 0; I < sessionStorage.length; I++) {console.log(sessionStorage[sessionStorage.key(I)]);

}

sessionStorage.removeItem(“mykey”);

HTML5 WebDB

• Will be supported in Safari 3.1+, Opera 9.?, Chrome.

• Around 5MB per domain.• Accessible across tabs and windows.• Basic SQLite SQL language features and types

supported.• Schema versioning supported.• Transactions supported.• Tables must be created on first run.

HTML5 Database

• Detection:

• Error trapping:

if (window.openDatabase) {// …

}

try {// DB operations go here

} catch(e) {if (e == 2) {

// Version mismatch}

}

HTML5 Database

• Simple DB Access:

• Other parameters:

var database = openDatabase("Database Name", "Database Version");database.executeSql("SELECT * FROM test", function(result1) {   // do something with the results   database.executeSql("DROP TABLE test", function(result2) {     // do some more stuff     alert("My second database query finished executing!");   });});

openDatabase(shortName, version, displayName, maxSize)

HTML5 Database

• Parameterized queries:

• Transactions:

transaction.executeSql(“INSERT INTO products (name, price) VALUES (?, ?);’, [“Book”, 2.01]);

myDB.transaction(function(transaction) {

// Create the products tabletransaction.executeSQL(‘CREATE TABLE products (id INTEGER

NOT NULL PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL DEFAULT “Widget”, price REAL NOT NULL DEFAULT 9.99);’, []);

transaction.executeSql(“INSERT INTO products (name, price) VALUES (?, ?);’, [“Book”, 2.01]);

}}

HTML5 Database

• Selects:db.transaction( function (transaction) { var query="SELECT * FROM products”; transaction.executeSql(query, [], handleProductsQuery, errorHandler);

}, transactionErrorCallback);

function handleProductsQuery(transaction, resultSet) {for (var I = 0; I < resultSet.length; I++) {

var row = resultSet.rows.item(i);console.log(row[‘name’] + “: “ + row[‘price’]);

}}

http://developer.apple.com/safari/library/documentation/iPhone/Conceptual/SafariJSDatabaseGuide/UsingtheJavascriptDatabase/UsingtheJavascriptDatabase.html#//apple_ref/doc/uid/TP40007256-CH3-SW1

HTML5 Database

• Safari DB Inspector

Alternatives to HTML5 WebDB

• TrimQuery - http://code.google.com/p/trimpath/wiki/TrimQuery

• CouchDB– Apache-project for RESTful JSON-based data queries.– Can solve the backend/frontend interop problem by

being available to both via HTTP requests.

• Google Gears SQLite DB. Same Interface. *• Hash-based data storage using DOM storage or

other mechanism.

* Discontinued by Google

Questions?

• Slide deck at http://ambiguiti.es