Building Powerful Mac + iOS Apps with Couchbase Lite and Feather: Couchbase Connect 2015

Build offline first withCouchbase & Feather

Matias Piipari (@mz2)CEO & Co-founder, Manuscriptsapp.comCTO, Papersapp.com

Manuscripts.appA Couchbase Lite powered writing tool

for complex documents.

Word processor & Couchbase?

http://your-data

Web frontend

Manuscript contents Author name / affiliation data Citation data

Feather (Couchbase Lite model)

Web model (e.g. PouchDB)

Mac app UI

Sync and share your manuscripts

You own and control your dataEnd-user is our customer, not part of the product.

Individual Individual + co-author / supervisor

Group of collaborators

50+ model entity types

~30K documents + attachments.Compresses to 60+ MB of pre-bundled content

Crowdsourced data

Cross-platform data access

Why Couchbase Lite?

1. Serve the data and its change feed over HTTP.

2. Focus on delivering a great offline experience.

3. Object-document mapping and attachments.

4. Sync is about conflict resolution and model robustness.

5. Sync is available peer-to-peer, between local databases.

6. Gives us + the end user control over data.

7. Cross-platform (Core Data).

8. Low memory footprint, fast start-up doesn't hurt on desktop.

9. Runtime performance drawbacks less of a concern on desktop.

10. Source code is readable.

❤Couchbase is

a great open source project. ❤

"Why does everything have to work so differently

in Couchbase?"— Anonymous Cocoa developer.

What is different?

• Data modeling.

• Database querying.

• Consistency.

• Data validation.

• Performance bottlenecks.

• [...]

Speed up common tasks.Impose structure and conventions.

Feather: utilities and conventions.• Partition data to multiple databases (speed + storage size).

• Manage repositories, or 'collections', of model objects.

• Mix-ins for unrelated model entities to share behaviour.

• Pre-compute data shipped in the app.

• Support nested object collections / trees in a document.

• Help with making the data model scriptable.

Contents database Authors database Bibliography database

Sections repository

User visible database package A

(~/Documents/thesis.manuscript)

Paragraphs repository Styles repository

Model object controllers

Section model Paragraph modelParagraph style

Model object

Margin

Citation repository

Citation

Cite “Piipari et al (2014)”

Cite “Hubbard et al (2011)”Embedded objects

Shared database package(~/Library/Application Support/Manuscripts)

Styles

User visible database package A

(~/Documents/conf-abstract.manuscript)Contents database Authors database Bibliography

database Styles

Styles databaseTemplates databaseFunders database

✨ Model objects and model object controllers ✨

• Model is dumb.

• Object controller for CRUD.

• XsController is implictly repository for models of type X (can be overriden).

• Observer callbacks for didAddX, didUpdateX, didDeleteX given.

• Properties which are prefixed cached are… cached.

• Properties which are prefixed embedded contain… embedded objects.

• Principle: if you're going to fail, detect it during startup.

Embedded objects• Nested in the containing object's JSON representation.

• Nested collections supported (arrays, sets, dictionaries).

• A nested object itself nests objects or collections of objects.

• Propagates changes to containing top-level model object.

Bibliography database

Database package

Citation repository

Citation

Cite “Piipari et al (2014)”

Cite “Hubbard et al (2011)”

Embedded object collections// Exhibit A: citation

@interface MPCitation : MPManagedObject

/** Array of MPCitationItem objects. */@property (readwrite) NSArray *embeddedCitationItems;

@end

@interface MPCitationItem : MPEmbeddedObject

@property (readonly, strong) MPBibliographyEntry *bibliographyEntry;

@end

Querying data/** Query the view with the given keys, * and return managed object representations. * * @param view name of the view queried. * @param an array of keys, or nil * @return * */- (NSArray *)objectsMatchingQueriedView:(NSString *)view keys:(NSArray *)keys;

- (NSArray *)contributorsInRole:(NSString *)role { return [self objectsMatchingQueriedView:@"contributorsByRole" keys:@[role]];}

Respond to object graph changes• Model objects and repositories observe for add / update / delete.

• A callback naming convention for notification handlers.

• Automated cache busting.- (NSArray *)borderStyles { if (!_cachedBorderStyles) { _cachedBorderStyles = [self stylesOfClass:MPBorderStyle.class]; }

return _cachedBorderStyles;}

Cascading object look-ups1. Get data from a database package.

2. If you found it, get it.

3. If you didn't find it, fall back to shared database package.

• Use-case: cascading style system.

• A more complex case: effective properties (follow a tree).

• Visitors: deep save a subgraph of objects.

Built-in support for tracking contributorship"Get authors contributing to paragraph X?""Get content contributed to by Y""Get me the author who most recently contributed to figure Z?"

Pasteboard support• NSPasteboard writing and reading support.

• An 'object reference': a fully qualified identifier for object.

• Uniquely identifies object also when multiple DBs with same document revision are open.

• Handy for supporting drag & drop.

HTTP listener• Easy API for spinning up the HTTP listener.

• Try a port, back off, then try again a few more times before failing.

/** If callback is not implemented, listener is added. * If callback is implemented, YES return value causes a listener to be created, * NO leads to it being omitted. * * Listener is by default not started for XPC services, * command line tools and when the package controller * has not been set to synchronize peerlessly (`-synchronizesPeerlessly`). */ - (BOOL)packageControllerRequiresListener:(MPDatabasePackageController *)packageController;

State initialisation & placeholder1. Bundle data by loading a JSON file

(MPManuscriptTemplatesController --> manuscript-templates.json)

2. Bundle data by pulling from a bundled CBL database

(MPManuscriptTemplatesController --> manuscript-templates.cblite).

3. Bundle data by copying a bundled CBL database in.

• Initial state (e.g. required documents at expected state).

• Placeholder content: e.g. on-boarding content.

- (id)ensureInitialStateInitialized { return [self ensurePrimaryManuscriptInitialized];}

Saving• 'session' property: track which party created a certain

revision.

• 'contributor': documents have an array of ordered contributor IDs.

• e.g. distinguish changes between different devices / writers.

Model mixins+ (void)implementProtocol:(Protocol *)protocol andProtocolsMatching:(MPAdoptedProtocolPatternBlock)patternBlock overloadMethods:(BOOL)overloadMethods;

+ (void)initialize { if (self == [MPDraft class]) [self implementProtocol:@protocol(MPDeliverable) overloadMethods:NO];}

@protocol MPDeliverable <NSObject>

@optional // methods optional to avoid compiler errors with Objective-Mixin

@property (readwrite, strong) NSDate *deadline;@property (readwrite, strong) MPContributor *responsibleContributor;@property (readwrite) MPImportance importance;@property (readonly) NSString *humanReadableImportance;

@property (readwrite, strong) MPLabel *statusLabel;

@property (readwrite) MPProgressLevel progressLevel;@property (readonly) NSString *humanReadableProgressLevel;

@property (readonly, strong) NSArray *requirements;

@property (readonly) CGFloat progress;

- (void)addRequirement:(MPRequirement *)requirement;- (BOOL)removeRequirement:(MPRequirement *)requirement;

- (MPRequirementEvaluation)meetsRequirement:(MPRequirement *)req;

@end

Stapler: a utility for bundling data from JSON to a CBL 'database package'

Input:

• A series of JSON files describing documents to load.

• Optional: path to attachment metadata.

Output: a Feather database package.

Stapler --target ./csl/bundles \--objectType MPBundle \--source ./csl/csl-data.json \--attachments ./csl/csl-paths.json

feather-grep: a Node.js commandline helper for grepping Feather data.

➜ Projects feather-grep -o MPManuscriptCategory -k name,desc,objectType '/Users/mz2/Library/Application Support/Manuscripts'

'MPManuscriptCategory:F9EC50FB-7E8D-4D47-BBEC-7A4E709B9306' : '1-1db4dade18b4d106d779cb640113c0bb'

{ name: 'Book',

desc: 'Books & Book chapters',

objectType: 'MPManuscriptCategory' }

'MPManuscriptCategory:714D3A21-C0F9-46A9-80D8-13796818098B' : '1-985e08c3be9829986719241f03eda379'

{ name: 'Dissertation',

desc: 'Master\'s & PhD theses, and other dissertations.',

objectType: 'MPManuscriptCategory' }

Model scriptability (JS / AppleScript)

• A scripting category for model and repository classes.

• A 1000+ line .sdef XML with mostly silent error handling --> nightmare.

• Hence wrote MPScriptingDefinitionManager: a runtime .sdef validator.

Thank you!github.com/mpapp/Feather.git

[email protected] (@mz2)