Software-Defined Approach for QoS and Data Quality in Multi-Tenant Clouds

Software-Defined Approach forQoS and Data Quality in

Multi-Tenant Clouds

The 2nd Portugal|UT Austin summer school in systems and networking and EMJD-DC spring event 2016June 3, 2016. Costa da Caparica, Portugal.

Pradeeban Kathiravelu

+x*

, Luís Veiga

+x

x

Distributed Systems Group, INESC-ID Lisboa

+

Instituto Superior Técnico, Universidade de Lisboa, Portugal

*

Emory University, Atlanta, GA, USA.

1

❖ Tenants assume various roles in the enterprise

data center networks.

➢ Differentiated Quality of Service (QoS).

➢ Data quality assurance.

➢ Isolation guarantees.2Int

roduc

tion

❖ Efficiency and Accuracy

3

WARNING: Work-in-Progress

4

WARNING: Work-in-Progress

PhD Start: September, 2014Expected Defence: September, 2018

5

❖ Data storage and execution

6

❖ Distributed

vs.

❖ Centralized

7

❖ Distributed

Horizontal scalability

8

❖ Centralized

Management efficiency.

Accuracy.

9

❖ Distributed

Software-Defined Networking

❖ Centralized

10

Software-Defined Networking

Systems

11

❖ Global view of the entire data center network

❖ A logically centralized controller

12

13

Our Approach

❖ Software-Defined Networking (SDN)

❖ SDN Northbound - Message-Oriented Middleware

❖ Scalable and Very Large Scale Data and Deployments

❖ Modular Architecture -> Reusable14

15

1. Distributed Execution: QoS and Data Quality

2. SDN Systems: SDN with Message-Oriented Middleware

3. SDN Simulations and Emulations16

Contributions &

Research Directions =>

1. Distributed Execution: QoS and Data Quality

2. SDN Systems: SDN with Message-Oriented Middleware

3. SDN Simulations and Emulations17

Contributions &

Research Directions =>

18

∂u∂u Multi-Tenanted Framework:Distributed Near Duplicate Detection for Big Data

23rd International Conference on Cooperative Information Systems (CoopIS 2015)28-30 October 2015, Rhodes, Greece.

CoopIS 2015

Pradeeban Kathiravelu, Helena Galhardas, Luís Veiga

INESC-ID / Instituto Superior Técnico, Universidade de Lisboa, Portugal

[1.1]

19

20

21

MEDIator: A Data Sharing Synchronization Platform for Heterogeneous Medical Image Archives

[1.2]

1. Distributed Execution: QoS and Data Quality

2. SDN Systems: SDN with Message-Oriented

Middleware

3. SDN Simulations and Emulations 22

Contributions & Research Directions =>

23

CHIEF: Controller Farm for Clouds of Software-Defined Community Networks

[2.1]

24

Building Blocks of Mayan:Componentizing the

eScience Workflows ThroughSoftware-Defined Service Composition

23rd IEEE International Conference on Web ServicesJune 27 - July 2, 2016, San Francisco, USA

IEEE ICWS 2016

Pradeeban Kathiravelu

+

, Tihana Galinac Grbac

*

, Luís Veiga

+

+

INESC-ID / Instituto Superior Técnico, Universidade de Lisboa, Portugal

*

University of Rijeka, Croatia

[2.2]

❖ Adaptive execution of scientific workflows

❖ More efficient and diverse service composition

❖ Very large-scale and reliable.

➢ Find and consume the current best-fit

➢ Among the multiple implementations or

deployments of the same service. 25

❖ Each execution component in the workflow:

Building Block of Mayan

26

27

28

29

<Service3,(<Service1, Input1>, <Service2, Input2>)>

30

Speed up of a data warehousing workflow:❖ Near duplicate detection ->

Data cleaning -> Write to the warehouse

❖ Data sets from TCIA* and synthetic data* Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M, Tarbox L, Prior F. The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository, Journal of Digital Imaging, Volume 26, Number 6, December, 2013, pp 1045-1057.

31

32

33

Cassowary: Middleware Platform for Context-Aware Smart Buildings with Software-Defined Sensor Networks

[2.3]

1. Distributed Execution: QoS and Data Quality

2. SDN Systems: SDN with Message-Oriented Middleware

3. SDN Simulations and Emulations

34

Contributions & Research Directions =>

35

SENDIM for Incremental Development of Cloud Networks: Simulation, Emulation & Deployment Integration Middleware

[3.1]

Summary

❖ SDN and Message-Oriented Middleware

❖ Quality of Service and Data Quality

❖ Modular Architecture -> Reusability

❖ Building Blocks of Mayan36

Conclusion

❖ We can scale smarter

37

Conclusion

❖ We can scale smarter

❖ We can manage better

38

Conclusion

❖ We can scale smarter

❖ We can manage better

❖ We can do more39

Recent Publications

40

[1] Distributed Execution: QoS and Data Quality

1. Kathiravelu, P., Galhardas, H., & Veiga, L. (2015). ∂u∂u Multi-Tenanted Framework: Distributed Near Duplicate

Detection for Big Data. In 23 rd International Conference on Cooperative Information Systems (CoopIS 2015)

(Acceptance Rate: 20%). On the Move to Meaningful Internet Systems: OTM 2015 Conferences, pp. 237-256.

Springer International Publishing, Oct. 2015. http://doi.org/10.1007/978-3-319-26148-5_14

2. Kathiravelu, P. & Sharma, A. (2015). MEDIator: A Data Sharing Synchronization Platform for Heterogeneous

Medical Image Archives. In Workshop on Connected Health at Big Data Era (BigCHat'15) , co-located with 21 st

ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2015). Aug. 2015. ACM. 6 pages. http:

//doi.org/10.13140/RG.2.1.3709.424841

[2] SDN Systems: SDN with Message-Oriented Middleware

1. Kathiravelu, P., Grbac, T.G., & Veiga, L. (2016). Building Blocks of Mayan: Componentizing the eScience

Workflows Through Software-Defined Service Composition. In 23rd IEEE International Conference on Web

Services (ICWS 2016). June 2016. 8 pages. To appear.

2. Kathiravelu, P. & Veiga, L. (2016). CHIEF: Controller Farm for Clouds of Software-Defined Community

Networks. In 3rd IEEE International Symposium on Software Defined Systems (SDS-2016). Apr. 2016. pp. 1 – 6.

42

3. Kathiravelu, P., Sharifi, L., & Veiga, L. (2015). Cassowary: Middleware Platform for Context-Aware Smart

Buildings with Software-Defined Sensor Networks. In 2nd Workshop on Middleware for Context-Aware

Applications in the IoT (M4IOT 2015), co-located with ACM/USENIX/IFIP Middleware 2015. Dec. 2015. ACM.

pp. 1 – 6. http://doi.org/10.1145/2836127.2836132

4. Kathiravelu, P., Grbac, T.G, & Veiga, L. (2016). A FIRM Approach to Software-Defined Service Composition. In

MIPRO 2016 - 39th International Convention on Telecommunications & Information (CTI).

43

[3] SDN Simulations and Emulations

1. Kathiravelu, P. & Veiga, L. (2016). SENDIM for Incremental Development of Cloud Networks: Simulation,

Emulation & Deployment Integration Middleware. In IEEE International Conference on Cloud Engineering

(IC2E 2016). Apr. 2016. pp. 143 – 146.

44

Doctoral Symposiums

1. Kathiravelu, P. (2016). Software-Defined Networking-Based Enhancements to Data Quality and QoS in Multi-

Tenanted Data Center Clouds. In IEEE International Conference on Cloud Engineering (IC2E 2016) Doctoral

Symposium. Apr. 2016. pp. 201 – 203.

45

Acknowledgements

❖ Erasmus Mundus Joint Doctorate in Distributed Computing (EMJD-DC)

❖ Google Summer of Code

❖ Prof. Ashish Sharma, Emory University, USA

❖ Prof. Helena Galhardas, IST, Portugal

❖ Prof. Tihana Galinac Grbac, University of Rijeka, Croatia

❖ COST action 1304 Autonomous Control for a Reliable Internet of Services (ACROSS)

❖ The OpenDaylight Foundation

❖ Ed Warnicke, Cisco, OpenDaylight TSC46

Thank you!

47

To discuss more about OpenDaylight:OpenDaylight Lisboa Meetups. :)

http://www.meetup.com/OpenDaylight-Lisbon/

48

Announcement ;)

Questions?

❖ We can scale smarter

❖ We can manage better

❖ We can do more

[email protected]