Working with real world data

Working with real data

1

Payam Barnaghi

Centre for Communication Systems Research (CCSR)

Faculty of Engineering and Physical Sciences

University of Surrey

Guildford, United Kingdom

2

Things, Data, and lots of it

image courtesy: Smarter Data - I.03_C by Gwen Vanhee

Data is not what we want or is it?

What we need are insights and actionable-knowledge

Diffusion of innovation

image source: Wikipedia

IoT

Problem #1

Data: We seem to have lots of it…

Real World Data: it is always difficult to get (silos, format, privacy, business interests or lack of interest!...)

Problem #2

Data: interoperability and metadata frameworks…

Real World Data: there are solutions for service based (Restful) access, meta-data/semantic representation frameworks (W3C SSN, HyperCat,…) but none of them are widely adapted.

Problem #3

Data: quality, reliability…

Real World Data: data can be noisy, crowed source data can be inaccurate, contradictory, delay in accessing/processing the data…

Problem #4

Data: having too much data and using analytics tools alone won’t solve the problem…

Real World Data: in addition to the HPC issues, we need new methods/solutions that can provide real-time analysis of dynamic, variable quality and multi-modal streams…

Problem #5

Data: abstraction, discovering the associations…

Real World Data: co-occurrence vs. causation; we need hypothesis, background knowledge,…After all data is not what we are really after…

We need more linked open data

(near) real-time linked open data

Streams

Sometimes it’s even better if we have:

(near) real-time linked open data

Streams+

meta-data (semantic annotations)+

Adaptable and scalable analytics tools+

Sufficient background knowledge

or even better than that if we have:

Data analytics

14

Data:

DataData

Domain

KnowledgeDomain

Knowledge

Social

systemsSocial

systemsInteractionsInteractionsOpen

InterfacesOpen

Interfaces

Ambient

IntelligenceAmbient

IntelligenceQuality and

TrustQuality and

Trust

Privacy and

SecurityPrivacy and

Security

Open DataOpen Data

15

Challenges and opportunities

− Providing infrastructure − Publishing, sharing, and access solutions on a global scale− Heterogeneity and interoperability at different layers− Indexing, query and discovery (data and resources)− Aggregation, integration and fusion− Trust, privacy and security− Data analytics and creating actionable knowledge

− Integration into services and applications in e-health, the public sector, retail, manufacturing and personalised apps.− Mobile apps, location-based services, monitoring control etc.

− New business models

− Thank you.

− EU FP7 CityPulse Project:

http://www.ict-citypulse.eu/

@ictcitypulse

[email protected]