KUKDM’2011, Zakopane Semantic Based Storage QoS Management Methodology Renata Słota, Darin Nikolow, Jacek Kitowski Institute of Computer Science AGH-UST,

KUKDM’2011, Zakopane

Semantic Based Storage QoS Management Methodology

Renata Słota, Darin Nikolow, Jacek KitowskiInstitute of Computer Science AGH-UST, Krakow, Poland

ACK Cyfronet AGH, Krakow, Poland

Research supported by MNiSW grant nr N N516 405535

KUKDM’2011, Zakopane

Outline

Introduction

Common mass storage system model and ontology

Storage system performance monitoring and estimation

Use cases

Summary

KUKDM’2011, Zakopane

Introduction There are applications (called data intensive applications), which:

– use data storage systems intensively,

– Have constantly growing demands concerning capacity and storage system efficiency.

Example of applications dealing with huge amounts of data

– Scientific applications (simulations, out-of-core computations, HEP experiments),

– Backup & restore, archiving, disaster recovery.

The performance of the data intensive applications depend on the performance of the underlying storage system

Applications running in distributed environments need some QoS (Quality of Service) concerning data access

KUKDM’2011, Zakopane

Introduction (cont.) Storage QoS metrics include

– data access latency

– data access bandwidth

– storage space

– data availability

Diversity of storage systems

Hierarchical Storage Management (HSM) systems

Disk arrays

Tiered storage

Problem of efficient storage performance utilization while respecting the storage QoS constrains

Storage performance monitoring and data transfer scheduling are necessary

KUKDM’2011, Zakopane

Goal of research

The subject of this research is the development of semantic-based storage management methodology allowing to achieve QoS for storage performance metrics.

As part of this research the following has been done:

• Development of common mass storage system model (C2SM) and relevant ontology

• Development of storage performance monitoring sensors and estimators

• Implementation of two use cases demonstarting our approach

The research has been done within the OntoStor project http://www.icsr.agh.edu.pl/ontostor

KUKDM’2011, Zakopane

C2SM

Is a common mass storage system model, which can be used to describe the state of a storage system

Consists of

set of well defined storage performance related parameters

Algorithms specifying the storage system behaviour

• Is based on the Common Information Model – CIM

• Is used in our methodology for unifying the description of storage performance parameters of heterogenious storage systems

KUKDM’2011, Zakopane

C2SM class diagram

KUKDM’2011, Zakopane

OntoStor ontology

• Developed based on C2SM

• Has been created semi-automatically using the ‘cim2owl’ tool, which has been developed at the DCS AGH

• The ontology is used in our methodology to find storage resources using semantic queries

•

KUKDM’2011, Zakopane

OntoStor ontology diagram

KUKDM’2011, Zakopane

Storage System Performance Monitoring

• Two components have been defined in our methodology: – sensors – obtaining storage

performance parameters,• Storage system dependent

– estimators – estimating the future storage performance based on the data from the sensors.

• Simulational, rule-based, statistical

• Three types of systems supported – local disk, disk array, HSM system

Testbed

KUKDM’2011, Zakopane

OntoStor Portal

KUKDM’2011, Zakopane

Disk array monitoring

KUKDM’2011, Zakopane

Monitoring of loaded disk array

KUKDM’2011, Zakopane

Estimation test

KUKDM’2011, Zakopane

Use cases

Two use cases have been implemented using the proposed methodology

Data access optimization with replication

Finding the best location for newly created replica for write access

Selecting the best existing replica for read access Components used:

Sensors, estimators, monitoring system, C2SM SLA (Service Level Agreement) monitoring

Components used: Sensors, monitoring system, OntoStor ontlogy, QoS metrics

ontology, C2SM

KUKDM’2011, Zakopane

SLA monitoring

1. QoS metric limits

2. How to obtain values of QoS metrics

3. How to monitor performance parameters

4. Description of parameters

KUKDM’2011, Zakopane

Summary

• Semantic-based storage management methodology based on monitoring and estimation of storage performance has been presented

• Common mass storage model and its relevant ontology has been proposed

• A set of sensors and estimators has been implemented

• Two use cases have been implemented using the proposed methodology

• The proposed methodology has been used in FiVO/QStorMan.