Promise Resource Reservation Capacity Management Requirements and Implementation Overview 18 May 2015 Peter Lee, ClearPath Networks.

Promise Resource ReservationCapacity Management

Requirements and Implementation Overview

18 May 2015Peter Lee, ClearPath Networks

Promise - Project Overview

• TSC approved on December 4, 2014 as a Requirements project

– http://wiki.opnfv.org/promise

• ETSI NFV Architectural Framework Alignment

– http://docbox.etsi.org/ISG/NFV/Open/Published/gs_NFV002v010201p - Architectural Framework.pdf

• Requirements documentation delivered on April 2015

– Reached “Stable” documentation release• https://gerrit.opnfv.org/gerrit/#/c/385 /

– ETSI and OPNFV cross-analysis/alignment established with ETSI NFV IFA005• ETSI NFV IFA005 will be made public soon

• Reference Implementation commenced on March 2015

– Comprehensive review of current OpenStack NBI APIs– YANG data model schemas modeling VIM using stormforge

• http://github.com/stormstack/stormforge 2

Promise – Committers & Contributors

• Committers:

– Arturo Martin De Nicolas (Ericsson: [email protected])

– Ashiq Khan (DOCOMO: [email protected])– Carlos Goncalves (NEC: [email protected])– Dirk Kutscher (NEC: [email protected])– Hironobu Maeda (ClearPath: [email protected])– Peter Lee (ClearPath: [email protected])– Mario Cho (Open Frontier Lab.: [email protected])– Ravi Chunduru (ClearPath: [email protected])– Ryota Mibu (NEC: [email protected])– Serge Manning (Sprint: [email protected])– Siva Busa (ClearPath: [email protected])

• Contributors:

– Zhiqi Wang(Huawei: [email protected])– Susana Sabater (Vodafone: [email protected])

3

Promise – Use Cases

• Resource Reservation

– Immediate and Future resource reservation– Co-existing reserved and unreserved resource allocations

• Capacity Management

– Discovery/Monitoring/Notification of capacity availability

4

Promise – Resource Reservation

5

Name Type Description

Start Timestamp Start time for consumption of the reserved resources

End Timestamp End time for consumption of the reserved resources

Expiry TimestampIf not all reserved resources are allocated between start_time and expiry, the VIM shall release the corresponding resources

Amount NumberAmount of the resources per resource element type (i.e. compute/network/storage) that need to be reserved

Zone Identifier The zone where the resources need(s) to be reserved

Attributes ListAttributes of the resources to be reserved such as DPDK support, hypervisor, network link bandwidth, affinity rules, etc.

Resources ListIdentifiers of existing resource elements to be reserved (such as images, flavors, virtual containers, networks, physical machines, etc.)

Information Elements

Promise – Resource Reservation

6

Promise – Capacity Management

7

Information Elements

Name Type Description

Notification IdentifierIdentifier issued by the VIM for the capacity change event notification

Zone Identifier Identifier of the zone where capacity has changed

Used/Reserved/Total Capacity

List

Used, reserved and total capacity information regarding the resource items subscribed for notification for which capacity change event occurred

Name Type Description

Zone Identifier Identifier of the zone where capacity is requested

Attributes ListAttributes of resource items to be notified regarding capacity change events

Resources List

Identifiers of existing resource items to be queried regarding capacity info (such as images, flavors, virtual containers, networks, physical machines, etc.)

Promise – Capacity Management

8

Promise – Implementation Plan

• Reference Implementation already commenced on March 2015

– Conducted comprehensive review of current OpenStack NBI APIs– YANG data model schemas modeling VIM using stormforge

• http://github.com/stormstack/stormforge

• Phase 1 – Develop a “Shim-layer” to OpenStack

– Complete YANG modeling of Virtual Infrastructure Resource Elements– Auto-generate NBI for NFVO and VNFM consumption– Utilize existing OpenStack NBI from Resource and Capacity Managers– Stand-alone implementation to validate message flows and logical enforcement

• Phase 2 – Native Integration into OpenStack Internal Components

– Expose additional data models currently not visible via existing OpenStack NBI– Tighter integration with real-time capacity status synchronization– Utilize native OpenStack controller components (such as MQ, DB, etc.)– Upstream integration as OpenStack native project (e.g. Blazar)

9

Promise – Phase 1 Implementation Architecture

10

Promise – YANG Schema for Resource Reservation

11

module opnfv-promise-models { prefix opm;

import storm-common-models { prefix scm; } import complex-types { prefix ct; }

feature resource-reservation; ct:complex-type ResourceReservation { ct:extends scm:ResourceElement; description "Contains the capacities of various resource services being reserved along with any resource elements needed to be available at the time of allocation(s)."; reference "OPNFV-PROMISE, Section 3.4.1"; leaf start { type ct:date-and-time; } leaf end { type ct:date-and-time; } leaf expiry { description "Duration in seconds from start when unallocated reserved resources will be released back into the pool"; type number; units "seconds"; } leaf zone { type instance-identifier { ct:instance-type scm:AvailabilityZone; } } container capacity { uses scm:compute-capacity; uses scm:networking-capcity; uses scm:storage-capacity; } leaf-list resources { description "Reference to a collection of existing resource elements required by this reservation. It can contain any instance derived from ResourceElement, such as ServerInstances or even other ResourceReservations. If the ResourceReservation request is accepted, the ResourceElement(s) listed here will be placed into 'protected' mode as to prevent accidental delete."; type instance-identifier { ct:instance-type scm:ResourceElement; } // following 'must' statement applies to each element must "boolean(/provider/elements/*[@id=id])" { error-message "One or more of the ResourceElement(s) does not exist in the provider to be reserved";

} }

leaf provider { if-feature multi-provider; config false; description "Reference to a specified existing provider from which this reservation will be drawn if used in the context of multi-provider environment."; type instance-identifier { ct:instance-type scm:ResourceProvider; require-instance true; } }

container remaining { config false; description "Provides visibility into total remaining capacity for this reservation based on allocations that took effect utilizing this reservation ID as a reference.";

uses scm:compute-capacity; uses scm:networking-capcity; uses scm:storage-capacity; }

leaf-list allocations { config false; description "Reference to a collection of consumed allocations referencing this reservation."; type instance-identifier { ct:instance-type ResourceAllocation; } } }

Introduction to stormforgeModel-driven Software Architecture

Provides inheritance-derived multi-layer abstractionfor flexible cross-platform integration

stormforge introductionWhat is it?

• A Model-driven Architecture (MDA) software framework for building schema defined software modules

Why was it created?

• To serve as a rapid prototyping framework for model driven software architecture

• To support OPNFV Promise implementation for abstracting ResourceElement(s) of the virtualized infrastructure for state management of capacity and reservations for future usage

How does it work?

• It uses Metaprogramming techniques to process YANG schema (RFC 6020) files as a software language

• It uses Creational Patterns to dynamically construct hierarchy of runtime class objects based on YANG data models

What’s the current status?

• It’s early stage with working compilers and generators

stormforge design objectives

14

Enable Rapid Prototyping by Non-Developers• Interpreting YANG schema as program data allows anyone to express

data models and configuration hierarchy as a simple text file and have it come to life for API interaction without writing code

Simplify Integration Across Systems• Let the framework do the challenging work of interfacing and

interacting between systems using data model driven auto-generated interfaces

• Enables exchange of software modules via import/export across remote systems

• It’s all just JavaScript, write it once and let it run anywhere (including the browser)

Application is King

• Every new module defined is an active asset that can be imported and used by other modules

• Every new module is a library, a server-side service, and a client-side utility all-in-one

stormforge framework

15

Schema

• provides definitions for generating the data models

Model/View/Controller

• decouples logic layers

Data Store

• handles record instances of data models

Events/Messages

• enable reactive control logic integration

Interface Layer

• auto-generates data model derived interfaces

Import/Export

• facilitates exchange of data models

stormforge core components

16

meta-class• The core base class that houses the meta-data and enables Class definitions to be

mutable

yang-compiler• Parses YANG schema text and produces runtime JS class object hierarchy (composed

of meta-class)

• Implemented based on YANG version 1.0 specifications (RFC 6020)

• Provides compile, generate, import, and export facilities (can be used as a server-side API service)

storm-compiler• Extensions beyond YANG version 1.0 to introduce complex-types (RFC 6095)

• Enhanced resolvers to infuse with data-storm objects providing type validators and model relationships

yang-storm• Defines a collection of common data models

• Serves as a DataStore for housing schemas, modules, models, controllers, views, and instantiated records

stormforge compiler features

17

Parser parse YANG schema files and generate runtime JavaScript meta-class semantic tree hierarchy

Map/Reduce traversal of the parser output to dynamically resolve YANG statement extensions and transform nodes in the tree as well as collapsing them into a final output module

Import/Export

capabilities to load modules using customizable importers based on regular expressions and custom import routines. Ability to serialize module meta data into JSON format that is portable across systems. Also exports serialized JS functions as part of export meta data.

Runtime Generation

allows compiler to directly create a live JS class object definition so that it can be instantiated via new keyword and used immediately

Dynamic Extensions

enable compiler to be configured with alternative resolver functions to change the behavior of produced output

stormforge usage examples

18

Prototyping a new module1. Write a new YANG schema definition file, such as hello-world.yang

2. stormforge run -p 5000 hello-world

3. curl http://localhost:5000

Validating an external module• stormforge validate http://intercloud.net/hello-world.yang

• stormforge validate http://intercloud.net/hello-world.json

• stormforge run -p 5000 http://intercloud.net/hello-world.json

Building/Signing/Publishing a new module1. stormforge build hello-world

2. stormforge sign -k <keyfile> hello-world

3. stormforge publish --registry=<url> hello-world

The steps above can apply for external sources as well

stormforge seeking contributors!

19

• Other OPNFV projects can benefit from using stormforge

– MOVI, Parser, Resource Scheduler, Doctor, Copper, etc.

• Any projects dealing with interface abstraction layer can take advantage of the framework to build/test/deploy data model driven components

• For Promise, we are seeking additional contributors to help with the YANG schema modeling of the Resource Elements as well as developing the NBI/SBI auto-generation component

Please contact [email protected] if interested in using stormforge for your project(s) as well as contributing to the development of stormforge!

Promise and stormforge

20

Any Questions?

The team will also be available to meet with any interested folks (for using/contributing) during the Summit on Tuesday

and Wednesday.

Please schedule a session with one of us and we’ll coordinate a time with you for deeper-dive discussions.

Thank you!