The Future of Couchbase Mobile: Couchbase Connect 2014

The Future Of Couchbase LiteJens Alfke — Mobile Architect, Couchbase

• 2011:

- Port of CouchDB to iOS/Android

• 2012:

- TouchDB investigation

• 2013:

- Couchbase Lite and Sync Gateway in development

• 2014:

- May: Couchbase Lite 1.0, Sync Gateway 1.0

- August: 1.0.1

- September: 1.0.2

The Story So Far

2

• 2015

- …?

The Story Continues

3

• 2015

- Continued maintenance releases

- Major feature release

- …?

The Story Continues

4

• ForestDB storage engine

• Enhanced query APIs

• Database encryption

Some Upcoming Couchbase Lite Features

5

Disclaimer

6

This presentation contains certain forward-looking information and statements pertaining to

unreleased (“vaporware”) software features. The use of any of the words “expect”, “promise”,

“really soon now” and similar expressions are intended to identify forward-looking information or

statements. The forward-looking information and statements included in this presentation are not

guarantees of future availability and should not be unduly relied upon. Such information and statements

involve known and unknown unknowns and other factors that may cause actual product features to differ

materially from those anticipated in such forward-looking information or statements. No other warranty

expressed or implied. Performance may settle during shipment and handling. Void where prohibited by law. Do

not use while operating a motor vehicle or heavy equipment. Color variations are part of the natural character of

this feature, not defects. Simulated picture. Professional developer on closed track. Do not code this at home.

Formatted to fit your screen. Some equipment shown is optional. Discontinue product development if burning or

itching occur. Prize not redeemable for cash value. No user-serviceable parts inside. Substantial penalty for early

withdrawal. To be used as a supplementary restraint system only. Always fasten your safety belt. Subject to change

without notice. Times approximate. If condition persists, consult your physician. Not rated by the Motion Picture Association

of America. Call for nutritional information. Any resemblance to real persons, living or dead is purely coincidental. Limited time

offer, call sales dept. now to ensure prompt delivery. No passes accepted for this engagement. This is not an offer to buy securities.

Processed at a facility that also processes peanuts, tree nuts & soy. Not returnable if seal is broken. Post office will not deliver

without postage. Not tested on laboratory animals. Do not remove tag under penalty of law. Employees and their families and

friends are not eligible. All models over 18 years of age. May be too intense for some viewers. Use only as directed; intentional

misuse by deliberately concentrating and inhaling APIs can be harmful or fatal. Fingers crossed behind back. Do not taunt Happy

Fun Feature.

Moving to the ForestDB Storage Engine

• Couchbase Lite currently uses SQLite for storage

- Ubiquitously available, highly reliable

• But SQLite has overhead we don’t need

- SQL processing, query planner, data constraints…

- Document vs. relational impedance mismatch

Why ForestDB?

8

• New storage engine developed at Couchbase

- For use by server and mobile

• Flat, ordered key-value blob store

- Same family as Berkeley DB, Tokyo Cabinet, LevelDB…

• Highly Optimized

- Hierarchical B+tree-Based Trie index

- Append-only log-based file format

• Watch video of the ForestDB talk from this morning

- “Preview: A Next Generation Storage Engine For NoSQL Database Systems”

What Is ForestDB?

9

• Faster

- HB+-Trie index is more compact

- More efficient write-ahead log

- Document revisions are grouped together

• Smaller Footprint

- Buffer cache shared between threads

• Higher Availability

- Writers never block readers

- Automatic background compaction

Couchbase Lite On ForestDB

11

• CBForest is a layer between ForestDB and CBLite

- Cross-platform C++

- https://github.com/couchbaselabs/cbforest

• Higher-level OO API

• Implements some CBLite features:

- Document revision trees

- Map/reduce indexes

- JSON-compatible key collation

CBForest

12

• Implemented in CBLite for iOS/Mac

- ‘forestdb’ branch

- github.com/couchbase/couchbase-lite-ios/tree/forestdb

• 2x – 5x performance of 1.0

• Coming to other platforms too; no ship date yet

Current Status

13

Performance

14

0

1000

2000

3000

4000

Populating DB Indexing (compound keys) Query with grouping

Docum

ents

/sec

SQLite (CBL 1.0.1) With ForestDB

Data set: iTunes library, 12189 documents(higher is better)

PERFORMANCE!!!

15

Enhanced Querying

• Map/Reduce is

- Flexible

- Efficient

- Scalable

• but not

- Intuitive

- Full-featured

Map/Reduce

17

• Couchbase is working on a new query language

• Still in development

• Initial implementation is server-focused

• Watch videos of earlier talks:

- “A N1QL For Every Query”

- “Native APIs For Querying Couchbase Server With N1QL”

N1QL: A New Query Language

18

• Cocoa — CoreData

- NSPredicate, NSSortDescriptor, etc…

• .NET — LINQ (Language-INtegrated Query)

- IQueryable, etc…

• My investigation focuses on Cocoa

Existing Platform Query APIs

19

• Picks subset of rows using arbitrary criteria

• Supports NSPredicate’s simple query language

• Evaluated on every row; O(n) performance

- Not a substitute for proper map/reduce design!

Filtering (WHERE)

20

CBLQuery* query = [view createQuery];query.endKey = @1950;query.postFilter = [NSPredicate predicateWithFormat:

@"value.rating >= 90 or(value.inColor and value.rating >= 70)"];

• Allows post-processing of order of rows

Sorting (ORDER BY)

21

query.sortDescriptors = @[[NSSortDescriptor sortDescriptorWithKey: @"value.rating"

ascending: NO],[NSSortDescriptor sortDescriptorWithKey: @"value.title"

ascending: NO],];

Key-paths relative

to CBLQueryRow

• These enhancements will be in version 1.0.3of Couchbase Lite for iOS

- Currently on branch release/1.0.3

- Release date Oct 17

Surprise!

22

• Automatically generates map function

• Automatically configures CBLQuery parameters

• Variables can be substituted at query time

Future Query Planner

23

CBLQueryPlanner Example

24

CBLQueryPlanner* planner = [[CBLQueryPlanner alloc]initWithView: view

select: @[@"title", @"body", @"author", @"date"]where: @"type == 'post' and title beginswith[c] $PREFIX

and tags contains $TAG"orderBy: @[[NSSortDescriptor

sortDescriptorWithKey: @"date" ascending: YES]]error: &error];

CBLQueryEnumerator* e = [query run: &error];for (CBLQueryRow* row in e) {

...}

CBLQuery* query = [planner createQueryWithContext: @{@"PREFIX": @"Announcing", @"TAG":@"mobile"}];

Database Encryption

• Encrypts the entire local database file

- Using SQLCipher

- AES-256 — current best practice for file encryption

Database Encryption

32

• Does not provide network encryption

- SSL already takes care of that

• Does not provide end-to-end document encryption

Database Encryption

33

• If you are required to

- Healthcare (HIPAA), banking, etc.

• If you don’t trust the device’s security

- User may not have set a passcode

- Device may have an old, less-secure OS version

- User may be targeted by the NSA, SVR, Illuminati…

Why to use database encryption

34

• Poor user experience

- Users dislike typing long passphrases

- Lost passphrase may mean lost data, support calls

• (Slight) drop in database performance

- Around 5-15%

• (Slight) growth in app binary size

- Due to having to embed custom SQLite library, ~1MB

• Device may actually be secure enough already

- iOS Security White Paper

- Android Security Overview

Why not to use database encryption

35

• Create a key

- Ask the user for a passphrase

- Or generate 32 securely random bytes

- SecRandomCopyBytes

- java.security.SecureRandom

• Register the key with the Manager

Using Database Encryption

36

• Passphrase:

- User must remember it (or write it on a sticky note)

- and type it in at launch time

• Raw key:

- Store it “somewhere secure”

- i.e. more secure than the file itself

- A regular iOS Keychain item is not more secure

- …but a TouchID-protected Keychain item is

Storing Keys

37

Stuff I Ran Out Of Time To Talk About

Subsetting a server database

Remote (server-side) queries

Improved model objects

Check it out:

forestdb

feature/query

feature/encryptio

n

github.com/couchbase/couchbase-lite-ios/tree/{

The End … For NowAny questions?