Developer friendly open data

Developer FriendlyOpen Data

Albert O’Connor @amjoconnColin Bell @collectdevCA

{ lessons from the Internet }

{ APIs }

{ platform for productivity }

{ lessons from the Internet }

“The Internet is becoming the town square for the global village of

tomorrow.”

Bill Gates

The Web

The Web

The Internet

SSH

FTP

Skype EmailIRC BitTorrent

DNS

Online Gaming

GopherBulletin Board Systems

How did the Web come to dominate?

Clay Shirky observed that the Web won because it is

evolutionary

Source: http://www.shirky.com/writings/evolve.html [2013]

Not because it was perfectly designed or technically

superior

Weakly Defined

Solutions that produce partial results when partially implemented can

succeed.

Constantly Changing

What is, is wrong, no evolving system is ever perfectly in sync with the

challenges it faces.

Evolution is cleverer than you

The ability to understand what is missing at any given moment does not mean

that one person or a small central group can design a better system in the long

haul.

HTML5

Open Data

{ application programmer interfaces — APIs }

What application?

UI or GUIGraphical User Interface

Data User Interfaceor

Developer User Interface

DUI

DUIerrAPI

File Based Open Datahttp://www.regionofwaterloo.ca/opendatadownloads/DoorsOpendataset.xls

Save file on disk

Parse the file

Store as Intermediate format

Load entire dataset into app

Index

Query

Present Result

App

Data loader

API Open Datahttp://api.uwaterloo.ca/public/v1/?key=FFF&service=Weather&output=json

{} Json

Convert some data types

Present ResultApp

LanguageSpecificLibrary

When and how often do I load the data?

Do I always have to resubmit my App?

Why API?

1 - Monitoring

2 - Metrics

Why API?

1 - Monitoring / Audit ➡ Who, What, When?

2 - Metrics / Analytics ➡ How?

{ platform for productivity }

Effectiveness + Efficiency

Efficient (adj.) - performing or functioning in the best possible manner with the least waste of time and effort.

Source: http://dictionary.reference.com [2013]

Effective (adj.) - adequate to accomplish a purpose; producing the intended or expected result.

Effectiveness + Efficiency

{} JSON

Convert some data types

Present Result

App

LanguageSpecificLibrary

API Open Data

Save file on disk

Parse the file

Store as Intermediate format

Load entire dataset into app

Index

Query

Present Result

App

Data loader

File Based Open Data

{} JSON

Convert some data types

Present Result

App

LanguageSpecificLibrary

API Open Data

Save file on disk

Parse the file

Store as Intermediate format

Load entire dataset into app

Index

Query

Present Result

App

Data loader

File Based Open Data

Data Access

File Based

botheffective

Data Access

API

Effectiveness + Efficiency

{ file based works }

{} JSON

Convert some data types

Present Result

App

LanguageSpecificLibrary

API Open Data

Save file on disk

Parse the file

Store as Intermediate format

Load entire dataset into app

Index

Query

Present Result

App

Data loader

File Based Open Data

7 steps

File Based

<less

efficient

3 steps

API

Effectiveness + Efficiency

{} JSON

Convert some data types

Present Result

App

LanguageSpecificLibrary

API Open Data

Save file on disk

Parse the file

Store as Intermediate format

Load entire dataset into app

Index

Query

Present Result

App

Data loader

File Based Open Data

7 steps

File Based

<less

efficient

3 steps

API

Effectiveness + Efficiency

- duplication- silos- complexity- blind to use

- on-demand- direct- benign- visible

Effectiveness + Efficiency = Productivity

Productivity is the effective use of innovation and resources to increase the value-added content of products and services. It is the true source of competitive advantage that creates long term economic viability and a better standard of living for all.

Source: http://hkpc.org => http://bit.ly/ZoivM7 [2013]

Effectiveness + Efficiency = Productivity

Innovation + Resourcescombine to make

Products + Services

Effectiveness + Efficiency = Productivity

Innovation + Resourcescombine to make

Products + Services

Adding value makes us more efficient!

Effectiveness + Efficiency = Productivity

Innovation + Resourcescombine to make

Products + Services

Adding value makes us more efficient!Effectiveness and Efficiency make us Productive.

{ make all the things! }

{ make all the things! }

... but you can’t ...

{ make all the things! }

... but you can? ...

{ make all the things! }

... but you can! ...we

Rewind... Evolutionary

partial results when partially implemented can succeed

no evolving system is ever perfect

no single entity can meet all user needs and account for all niches

Embrace the reality, empower the toolmakers

We can all innovate to add value together!

Define data entities (nouns) ... see EA[1]

Make data accessible so Developers are Efficient

Developers inside and outside your organization can design + build services with the same data

[1]: http://en.wikipedia.org/wiki/Enterprise_architecture [2013]

Empower Developers within your Organization across Organizational Boundaries

Improve your Organization’s Productivity

Steadily evolve your APIs and begin using them

yourself.

Improve your Productivity

Design for ‘maker’* Users

https://www.gov.uk/designprinciples

* Toolmakers... aka. Developers

Design for AccessAccess by Design (AbD)

http://www.ipc.on.ca/images/Resources/accessbydesign_7fundamentalprinciples.pdf

Proactive, not Reactive

Access Embedded into Design

Openness and Transparency = Accountability

Fosters Collaboration

Enhances Efficiency

Makes Access Truly Accessible

Increases Quality of Information

{ provide friendly access }

Empower the ‘maker’ users to help you.

Make friends with developers!

{ provide open access }

Do your part to help improve productivity!

Empower yourself and others to provide better (more valuable) service.

{ further reading }

https://www.gov.uk/service-manual/making-software/apis.htmlhttp://www.w3.org/Provider/Style/URI.htmlhttp://www.shirky.com/writings/evolve.html

http://en.wikipedia.org/wiki/Enterprise_architecture

Developer FriendlyOpen Data

Albert O’Connor @amjoconnColin Bell @collectdevCA