Summit 16: Open-O Mini-Summit - VF Event Streaming Project Proposal

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

VF (Virtual Functions) Event Streaming (VES) Project Proposal

June 13th, 2016

For Questions, Please contact:

Alok Gupta

+1 732-420-7007; [email protected]

1

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

VES Project Proposal

2

Objective

Enable significant reduction in effort to develop and integrate VNF telemetry-

related data into automated VNF management systems, by promoting convergence to a common event stream format and collection system.

Background

Telemetry data formats and semantics / expected actions by management systems vary widely

For Fault Events, vendors use SNMP, 3GPP Corba, MTOSI, OSSJ etc, and semantics can differ (e.g. Critical Severity as “1” or “5”)

For Measurement events (KPI/KCI), vendors deliver CSV or XML based files, with varying internal data formats.

This variance complicates NF integration into management systems

Scope/Deliverables

A VNF Event Stream (VES) Common Event Data Model

Consisting of a Header and an Event Specific Block, with additional Name/Value fields for extensibility

OPNFV support for the VES Common Event Data Model

A VES Agent that can collect the VNF Event Stream data from the VNF and deliver it to the VES Collector

A VES Collector that can consume the VNF Event Stream data, and store the data in a database backend

VES plugins for integration with OpenStack infrastructure services such as Monasca and Vitrage

Documentation and test support (e.g. reference event data)

A plan for upstreaming the project code

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property. 3

Problem Statement / Fault Management (FM) Interface Comparison Categories

SNMP 3GPP Corba MTOSI via JMS MTOSI via REST over

HTTPS (JSON)

General Attributes

- There are 3 current version of SNMP. - Devices are not compliant in all 3 versions. - SNMP V3 handling requires special adapter due to increased security - SNMP is connectionless, except for SNMPv3 - No assurance of delivery - No resynch functionality

- Corba is connection oriented - assurance of delivery - resynch functionality - Corba has sets of actions - Corba payload provides more details in less traffic volume

- JMS is Connection oriented - assurance of delivery - resynch functionality - MOTSI has a sets of actions - MTOSI payload provides more details in less traffic volume

-service providing organized access to stateless resources -connectionless -stateless

Availability Agents and processes are widely available Reasonable Support Mixed Mixed Reuseability Low High High High Exapndability Low High High High Authenticaltion-Security SNMP V3 over SSL High-SecurityIRP High - SSL High – SSL Platform Dependency Dependent Dependent Independent Independent

SOA Interface and Data event No Yes Yes Yes Implementation Vendor Dependednt Vendor Independent Vendor Independent Vendor Independent CLI Command Standard Standard Standard Standard Payload High Constant constant Constant Multiple Device Support No Yes Yes Yes Multi-Vendor Support No Yes Yes Yes

Sensititivity to new Alarm Data High Low Low Low

point-to-Point/Multipoint p-t-p p-t-p

deploy muiltple pub-sub to avoid bottleneck Pub-Sub

latency <1ms <10ms <10ms Server/Code deployment Difficult Java driven Independent interfacing to diffenet Apps Difficult Easy Easy Output SNMP 3GPP XML/JSON XML/JSON Cost of Wonership adapter 49 7 2 line of codes 500000 6500 800 Equipment Type Supported 50 20 5

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Common Event Data Model: Fault, Heartbeat, Performance, Syslogs, Other, Usage, Signaling, Security, others

Common Header and Domain Specific Event

Additional name value pair fields for extensibility

Profile Driven Event Data Needed at time of VNF Instantiated and Data Profile Support to change event streaming by VNF and Collector/Listener:

Collector Address: IP, Port

Authentication Data: User Name/Password

Data Profile: Event Type (Fault (Custom (Full, c1, c2..) or Programmable))

VES – Common Event Data Model - Dynamic

Pre-Defined: No API

Overhead

Defined Real Time via API

Header

DomainVNF AttributeTiming

Fault

SeveritySourceSpecific Problem..

Name/Vlaue

Measurement

KPIKCIVF Scaling…....

Name/Vlaue

SysLogs

FacilityDataProc…...

Name/Vlaue

MobileFlow

RptEndPtOthEndPtFlow…...

Name/Vlaue

Other

Name/Vlaue

Usage, Config...

…..Other: Signaling, Internal ...TBD

Name/Vlaue

4

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

In scope:

• Green/solid interfaces

• Agent/Collector interface support library and reference implementation

• OpenStack component plugins

Agent code is embedded in VNFCs, Virtualization Hosts, and Bare Metal Hosts

Agents / VNFCs can also deliver directly to OpenStack Plugins (not shown)

VES Architectural View

OpenStack

Application Controller

SDNC

Bare Metal Host

Virtualization Host(s)

VES Collector

VES Agent VNFC VES Agent

Plugins

Monasca Vitrage

Congress etc

Collector can integrate with

other OpenStack Collectors

Other Management Functions

Message Bus

Direct (REST)

5

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

VES and Other OPNFV Projects

VES is a standardized event framework that can be used by Open Stack, OPNFV projects

OPNFV projects that potentially benefit from the VES project Fault Management (Doctor)

Virtualized Infrastructure Deployment Policies (Copper)

High Availability for OPNFV (Availability)

Data Collection for Failure Prediction (Prediction)

Audit (Inspector)

Fault localization (RCA, Pinpoint )

Service Function Chaining (sfc)

Moon Security Management

OpenStack projects that potentially benefit from the VES project Monasca: a monitoring-agent plugin for VES can provide a new/converged source for Monasca events

Vitrage: similar to Monasca can provide new data for the Graph DB of Vitrage

Congress: Congress data source drivers can integrate additional data from Monasca, Vitrage, or directly

6

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

1. Use cases that illustrate the value, reuse and increased efficiency/cost reduction

2. Software code behind:

Application Interface Document

Library Capabilities

Demo Reference VNF

Event Library Integration

AT&T’s Operational Experience using the Proposed Framework

7

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Use Case

Results Once development is done for the first VNF, the subsequent VNF can be supported via Configuration changes

and/or Policy Identification and Enforcement, resulting in over 80% cost and time savings

Use Case and Results

Collector Event

Processor Correlation

Policy Enforcement/Automation

Data Bus

Fault Event / Clear

Ticketing System

Scenario 3: High CPU Utilization Published an event for service orchestrator to create a New VF

Scenario 2: Payload Processor Failure Publish an event to Restart VM

Scenario 1: False Alarm

Scenario 2: VM Failure

Scenario 3: Scaling Alert Scenario 1: Failure Event, Notification

PM (Threshold

Crossing Alert)

Ticket Generated, Updated and Closed

8

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Thank You

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property. 9

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Hosted in the AT&T Research github site: http://github.com/att

• evel-library

• evel-test-collector

• evel-reporting-reference-vnf

To be contributed to the VES project under the Apache 2.0 license

ECOMP Vendor Event Listener code on Github

10

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Purpose

A test harness to play the role of the ECOMP Vendor Event Listener in environments where access to ECOMP isn’t available

Capabilities

Catches JSON events

Checks the events are valid against the defined schema

Checks the authentication credentials provided

“Pretty prints” the event for desk-checking

Use-cases

Testing VNFs during development by vendors

Familiarization with the API

Packaging

GitHub repository with detailed documentation

Source code tree for development, running in Windows environment etc (Python)

VES Collector

11

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Purpose

A simple VNF which provides a Web UI to a local database of event definitions

Capabilities

The user can define arbitrarily complex events

The user can generate single-shot and repeated (scheduled) events

Use-cases

Testing ECOMP

Familiarization with API

Mocking-up events as part of designing a new VNF

Packaging

GitHub repository with detailed documentation

Source code tree for development (Python, Django, SQLite)

Reference VF

12

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Reference VF

13

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Latest Python 2.7.x

https://www.python.org/downloads/

Check the option “Add Python to path”

But you may have to edit the Path environment variable to “Apply” the changes – or just reboot!

Django

pip install Django

JSON Schema

pip install jsonschema

Initialize the VNF database for first use

…\go-scripts\initialize-webserver.bat

Start the app

Test collector …\go-scripts\go-collector.bat

Backend …\go-scripts\go-backend.bat

Webserver …\go-scripts\go-webserver.bat

Ref VF Event Demo

14

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Point your favorite web-browser at:

127.0.0.1:8000

Generate an event

Fault

Measurement

Mobile Flow

Syslog

Check that the Test Collector reports its safe receipt…

…and try to work out why not if it doesn’t!

Ref VF Event Demo – Contd.

15

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Purpose

A C library to simplify adding ECOMP event reporting to VNFs

Capabilities

Handles all the JSON encoding

Handles all the HTTP(S) transport

Provides an asynchronous API to the VNF

Generates syslog events for delivery failures etc

Use-cases

VNF development

Familiarization with the API if implementing an independent event-reporting mechanism

Packaging

Source code tree for development

gcc toolchain

VF Event Library

16

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Handles all the JSON encoding

Trivial JSON encoder “hand-built” into library

Offloads “non-application” fields (e.g. VM Name and UUID) from the VNF application code

Optional fields are omitted where possible

Handles all the HTTP(S) transport

Uses cURL library - https://curl.haxx.se/libcurl/

Provides an asynchronous API to the VNF

Implements a ring-buffer between the application API and the HTTP client to the JSON API

Offloads waiting for responses

Generates syslog events for delivery failures etc

Last-ditch record of what was going on for post-mortem analysis

VF Event Library Capabilities

17

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

Initialize

Per client process

Must be single-threaded at this point

Generate events

Create an event (Measurement, Fault or Report)

Provide mandatory scalar event parameters to factory function

Add parameters

Optional scalars

Mandatory vectors

Hand-off to the library

Application code relinquishes all responsibility (e.g. freeing memory) at this point

Shutdown

Swift and brutal!

The ring-buffer is not depleted

Cleans up all memory & the cURL library

VF Event Library Usage

18

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

VF Event Library Initialize

19

/***************************************************************************/

/* Initialize the EVEL interface. */

/***************************************************************************/

if (evel_initialize(api_fqdn,

api_port,

api_path,

api_topic,

api_secure,

"Alice",

"This isn't very secure!",

EVEL_SOURCE_VIRTUAL_MACHINE,

"EVEL demo client",

verbose_mode))

{

fprintf(stderr, "Failed to initialize the EVEL library!!!");

exit(-1);

}

else

{

EVEL_INFO("Initialization completed");

}

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

VF Event Library Create Event

20

/***************************************************************************/

/* Create a new Fault event and set optional parameters. */

/***************************************************************************/

fault = evel_new_fault("My alarm condition",

"It broke very badly",

EVEL_PRIORITY_NORMAL,

EVEL_SEVERITY_MAJOR);

if (fault != NULL)

{

evel_fault_type_set(fault, "Bad things happen...");

evel_fault_interface_set(fault, "My Interface Card");

evel_fault_addl_info_add(fault, "name1", "value1");

evel_fault_addl_info_add(fault, "name2", "value2");

/***********************************************************************/

/* Post the event to the Vendor Event Listener. */

/***********************************************************************/

evel_rc = evel_post_event((EVENT_HEADER *)fault);

if (evel_rc != EVEL_SUCCESS)

{

EVEL_ERROR("Post failed %d (%s)", evel_rc, evel_error_string());

}

}

else

{

EVEL_ERROR("New fault failed");

}

© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

VF Event Library Termination

21

/***************************************************************************/

/* Shutdown the library. */

/***************************************************************************/

evel_terminate();