Peter Spicer - Agile approaches to BI - FutureData 2017

UNLOCK THE FUTURE

TM

Gateway – Transforming Academic ResearchSydney | October 2017

2

Who are we?

• Established in 2005 as SIRCA Technology, rebranded to RoZetta Technology in 2015.

• Commercialisation entity of SIRCA Ltd.

• We provide platforms and solutions for clients, specialisingin Big Data, Analytics and Cloud based application development.

• We have a global partnership with Thomson Reuters servicing 650 global banks and hedge funds with our tick history product which covers more than 10billion trades per day, from over 400 stock exchanges for the last 20 years.

• Established in 1997, SIRCA is a membership based organisation representing prominent universities in Australia and New Zealand (33 universities).

• SIRCA provides a common infrastructure to researchers, comprising technology and data, so that researchers can share the benefits of a common technology based data platform.

• The economics of common infrastructure, makes it possible to create tools which otherwise would have been unviable or unsustainable for a single university/researcher to undertake.

• Provides a channel to a university resource pool which RoZettacan draw upon in new and emerging fields of technology and business domains.

3

Legacy PortalAcademic Portal providing pre-canned access to data with a download capability

4

Launch Video

New SIRCA Gateway delivers value at multiple levels

5

Combining Compute Power and Analytics

6

User Example: Joining 2015 Full Orderbook Data with Time & Sales Data

7

8

Architecture

9

Databricks

10

Databricks Notebook

11

Transition Enablement for Academic UsersThe transition for our academic users has been significant and we have provided a set of tools to help the adoption of our platform:

TRAINING VIDEOS WEBINARS NOTEBOOKS DATA

DEFINITIONS FAQs

12

Challenges and Lessons LearntAs a result there are a number of challenges and lessons learnt from this project:

Ingesting data as early as

possible

Work closely with your data

analysts

Development and production of Databricks

environments should have the same

features enabled

Consider how you will manage

changes to dataset

schemas

Pre-purchase reserved instances

Spark does not guarantee the

order of ingested data

13

Features Important to Our Users

Choice of programming language to

execute analytics

Visualise results

Collaboratewith

nominated colleagues

Schedule jobs

Share analysis

Publish dashboards

Teachingaid in

classrooms

UNLOCK THE FUTURE

TM

Contact:

Peter SpicerChief Technology [email protected]

15

Video

https://www.youtube.com/watch?v=7WhQThzJbK0