7.2.2. Introduction to the SDMX-JSON Draft Format - OECD

2013-09-05 SDMX Technical Working Group 1

Introduction to the

SDMX-JSON Draft Format

Sami Airo & Olav ten Bosch

Capacity Building Session – Technical Stream

SDMX Global Conference 2013

OECD

2

Background: JSON

“JSON (JavaScript Object Notation) is a lightweight data-interchange format.”

✓ Text based

✓ Supports Unicode

✓ Documented (www.json.org)

✓ Standardized (RFC 4627)

✓ Language independent (over 30 from ASP to VB)

✓ Supported by tools (all browsers have it built-in)

✓ Used in production (Google, Facebook, Twitter etc.)

3

Background: JSON and SDMX

4

Requirements: Conceptual Design

Developers will be able to write software that requests responses in JSON from SDMX RESTful API and

processes the data and metadata in the JSON response in order to produce useful data visualizations on the

Web.

Database Web

Service

Web

browser

GET request

Response in JSON

Java

Script

5

Requirements: For the Web

For Slow Networks For Mobile Devices

For Diverse Developers For Diverse Users

© S

as

ch

a P

oh

fle

pp

© Z

are

k

6

Requirements: Features

• Must-have

–Conforms to JSON specification RFC 4627.

–Supports the SDMX 2.1 information model.

–Supports the full functionality in the SDMX 2.1 RESTful API for data.

–Supports many different types of visualizations.

–Streamable on the server.

• Linear (the more/less, the better)

–Easy: minimal amount of client code required.

–Fast: few roundtrips, small messages, and fast parsing.

7

Implementation: Basic Message

- Header is for technical information: prepared, sender, ...

- Errors is for error messages.

- Structure contains metadata: dimensions and attributes.

- DataSets contains the actual data.

- Examples: flat grouped

{

“header”: { ... },

“errors”: { ... },

“structure”: { ... },

“dataSets”: [ ... ]

}

8

Implementation: Data + Structure

• Format combines both structure (e.g. dimensions and attributes) and data (observation and attribute values) in the same message.

• Server will join information from different parts of the SDMX IM (e.g. names for codes) and merge it into one message.

• This avoids multiple requests from the client which saves time and simplifies client code.

• Client does not need to process different parts of the SDMX IM and their relationships.

9

Implementation: Metadata Encoding

• Because dimensions and attributes typically take on a limited number of different values we can store them as integers in the dataset and keep the actual values as part of the structure.

• This includes both coded and uncoded attributes.

• The integers map directly to the array indices in the values fields.

• Statistical package R uses a similar method for storing factor (or categorical) variables.

10

Implementation: SDMX Features

• Supports both flat data sets and data sets with series grouping. Also supports optional grouping of common dimension and attribute values.

• Supports multiple data sets with different actions (update, append, delete, informational).

• Supports hyperlinks to other parts of the IM (e.g. data flows).

• Supports optional annotations.

• Includes human readable names and descriptions for data sets, dimension, attributes etc.

• Supports roles for dimensions/attributes (time etc.)

11

Demos

Live versions of demos are available at: http://goo.gl/Af21z5

12

Potential Usage Scenarios

13

More Information

• Github Repository

–SDMX-JSON Draft Documentation

–SDMX-JSON Sample Files

–SDMX 2.1 RESTful API tutorial

• Discussion Group

–SDMX-JSON Google Group

14

That’s all, Thank You!

© 2010 Alec Perkins