Aruba VAN SDN Controller 2.8 Administrator Guideh20628.SDN architecture separates the network control plane from the forwarding hardware on network devices. Control can then be centralized,

Aruba VAN SDN Controller 2.8Administrator Guide

Part Number: 5200-3100Published: March 2017Edition: 1

© 2017 Hewlett Packard Enterprise Development LP

NoticesThe information contained herein is subject to change without notice. The only warranties for Hewlett PackardEnterprise products and services are set forth in the express warranty statements accompanying such productsand services. Nothing herein should be construed as constituting an additional warranty. Hewlett PackardEnterprise shall not be liable for technical or editorial errors or omissions contained herein.

Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, orcopying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer SoftwareDocumentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor'sstandard commercial license.

Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprisehas no control over and is not responsible for information outside the Hewlett Packard Enterprise website.

Aruba VAN SDN Controller license textThe Aruba VAN SDN Controller license text is in /opt/sdn/legal/EULA.txt. The Aruba VAN SDN Controllerincorporates materials from several Open Source software projects. Therefore, the use of these materials by theAruba VAN SDN Controller is governed by different Open Source licenses. Refer to /opt/sdn/legal/HP-SDN-CONTROLLER-OPENSOURCE-LIST.pdf for a complete list of the materials used.

Open Source SoftwareFor information on licenses for the open source software used by the Aruba VAN SDN Controller, see the ArubaVAN SDN Controller Open Source and Third-Party Software License Agreements.

This product includes code licensed under the GNU General Public License, the GNU Lesser General PublicLicense, and/or certain other open source licenses. A complete machine-readable copy of the source codecorresponding to such code is available upon request. This offer is valid to anyone in receipt of this informationand shall expire three years following the date of the final distribution of this product version by Hewlett PackardEnterprise. For information about acquiring the open source code for the Aruba VAN SDN Controller, contactHewlett Packard Enterprise Support, listing the product name and version information for which the source codeis being requested. See Support and other resources for information about contacting Support. Because suchinformation can become outdated quickly, Hewlett Packard Enterprise does not publish mailing addresses andtelephone numbers for open source queries. Available source code distribution methods include networktransmission of the source code and sending the source code on physical media to a mailing address. Physicalmedia distribution might require a fee to cover the media and mailing costs.

The Aruba VAN SDN Controller includes both proprietary software that is closed source in addition tothe open source software listed in the Aruba VAN SDN Controller Open Source and Third-PartySoftware License Agreements. In response to queries to Hewlett Packard Enterprise for source codeon the Aruba VAN SDN Controller, Hewlett Packard Enterprise distributes the source code for opensource software only. Hewlett Packard Enterprise does not distribute source code for closed sourcesoftware.

AcknowledgmentsIntel®, Itanium®, Pentium®, Intel Inside®, and the Intel Inside logo are trademarks of Intel Corporation in the UnitedStates and other countries.

Microsoft® and Windows® are either registered trademarks or trademarks of Microsoft Corporation in the UnitedStates and/or other countries.

Adobe® and Acrobat® are trademarks of Adobe Systems Incorporated.

Java® and Oracle® are registered trademarks of Oracle and/or its affiliates.

UNIX® is a registered trademark of The Open Group.

Chapter 1 Introduction.....................................................................................8About the Aruba VAN SDN Controller .......................................................................................................8The Aruba SDN ecosystem....................................................................................................................... 8SDN Controller applications and the App Store.......................................................................................10Hewlett Packard Enterprise SDN information library............................................................................... 10Switch and OpenFlow requirements........................................................................................................ 10

Chapter 2 Understanding the controller architecture.................................12SDN controller architecture......................................................................................................................12List of controller embedded applications..................................................................................................13OpenFlow Link Discovery........................................................................................................................ 13OpenFlow Node Discovery ..................................................................................................................... 14Path diagnostics.......................................................................................................................................15Topology Manager................................................................................................................................... 15Topology Viewer.......................................................................................................................................15Path Daemon........................................................................................................................................... 15

Chapter 3 Using the SDN controller UI.........................................................18Starting the SDN controller console UI.................................................................................................... 18Changing the language displayed in the user interface........................................................................... 19The SDN controller user interface............................................................................................................20Changing column widths in the UI........................................................................................................... 21SDN User window....................................................................................................................................21Navigation menu...................................................................................................................................... 23Alerts........................................................................................................................................................25Applications..............................................................................................................................................30Understanding application states and OSGi artifacts.............................................................................. 35Configuration components....................................................................................................................... 37Audit log................................................................................................................................................... 45Licenses...................................................................................................................................................48Team........................................................................................................................................................ 49Support logs.............................................................................................................................................50Packet listeners........................................................................................................................................53OpenFlow Monitor....................................................................................................................................54OpenFlow topology ................................................................................................................................. 58OpenFlow Trace log.................................................................................................................................67OpenFlow Classes ..................................................................................................................................74

Chapter 4 Hybrid mode for controlling packet forwarding........................ 78Overview.................................................................................................................................................. 78Learning more about hybrid mode........................................................................................................... 78Viewing and changing the hybrid mode configuration............................................................................. 78Coordinating controller hybrid mode and OpenFlow switch settings....................................................... 80Controller packet-forwarding when hybrid mode is disabled................................................................... 82Controller packet forwarding when hybrid mode is enabled.................................................................... 83

Contents

Contents 4

Chapter 5 License Registration and Activation...........................................85Overview of the license registration and activation process.................................................................... 85License types, usage, and expiration.......................................................................................................85Preparing for license registration............................................................................................................. 86Registering and activating a license........................................................................................................ 87Registering your license and obtaining a license key.............................................................................. 87Activating a license on the controller....................................................................................................... 91Managing licenses................................................................................................................................... 94Using evaluation licenses...................................................................................................................... 100

Chapter 6 Configuring for High Availability...............................................101High Availability best practices...............................................................................................................101About teaming for High Availability........................................................................................................ 101Requirements for controller teams.........................................................................................................102Requirements for configuration, licensing, or application changes to controller teams......................... 103Team status............................................................................................................................................103Controller status ....................................................................................................................................103Manually synchronizing Cassandra database notes using the nodetool repair utility............................104Configuring controllers to use the same local NTP servers................................................................... 105Viewing your team configuration using the UI........................................................................................106Methods for configuring HA teaming......................................................................................................108Defining inputs for teaming in a configuration file.................................................................................. 109Using a Python script from a controller to configure a team...................................................................111

Chapter 7 Security........................................................................................113SDN Controller authentication ...............................................................................................................113Changing the default controller keystore and truststore to use CA signed certificates.......................... 113SDN Controller keystore and truststore locations and passwords ........................................................ 115Encryption.............................................................................................................................................. 115Built-in OpenFlow controller................................................................................................................... 116REST authentication.............................................................................................................................. 117Controller code verification.................................................................................................................... 124Revoking Trust ......................................................................................................................................125SDN administrative REST API ..............................................................................................................126Virgo admin UI access via localhost only...............................................................................................127Virgo console access disabled by default.............................................................................................. 127JMX console enabled for local access only........................................................................................... 127Creating the Cassandra keystore and truststore................................................................................... 128Cassandra keystore and truststore locations and passwords ...............................................................129Security procedure.................................................................................................................................129Security best practices...........................................................................................................................130

Chapter 8 Configuring OpenFlow instances............................................. 132Configuring OpenFlow Instances with Multiple VLANs .........................................................................132Configuring OpenFlow Instances with Single VLAN Identifier............................................................... 132Configuring OpenFlow instances to enable MAC group matching........................................................ 136

Chapter 9 Backing up and restoring ......................................................... 140Backing up and restoring Best Practices............................................................................................... 140Backing up a controller ......................................................................................................................... 140

Contents 5

Restoring a controller from a backup..................................................................................................... 143Distributed (team) backing up and restoring ......................................................................................... 147Backing up and restoring the Keystone configuration and database.....................................................148

Chapter 10 Metrics....................................................................................... 149Viewing metric data................................................................................................................................149Viewing all controller JVM metrics......................................................................................................... 157Viewing current metric data using a JMX client..................................................................................... 158Generating a controller support report................................................................................................... 160

Chapter 11 Troubleshooting........................................................................164REST API request returns HTTP code 401........................................................................................... 164Controller not listening on port TCP/8443..............................................................................................165Packets not received at the end point....................................................................................................165Session expired message in the UI....................................................................................................... 165Error running the config_sdn.py script with date/time/NTP option......................................................... 166Licensing................................................................................................................................................166Applications that use the Cassandra database are experiencing failures............................................. 167Controller support log fills disk space, contains multiple “Too many open files” messages...................168Application management errors............................................................................................................. 168OpenFlow errors.................................................................................................................................... 172Troubleshooting teamed environments..................................................................................................173

Chapter 12 Websites.................................................................................... 178

Chapter 13 Support and other resources.................................................. 179Accessing Hewlett Packard Enterprise Support.................................................................................... 179Accessing updates.................................................................................................................................179Customer self repair...............................................................................................................................179Remote support..................................................................................................................................... 180Warranty information..............................................................................................................................180Regulatory information...........................................................................................................................180Documentation feedback....................................................................................................................... 181

curl commands.............................................................................................182About the curl commands...................................................................................................................... 182Getting an authorization token using a curl command...........................................................................183Export audit log data as a CSV file using curl commands..................................................................... 183Licensing actions using curl commands................................................................................................ 183Application manager actions using curl commands...............................................................................186Viewing metric data using curl commands.............................................................................................193Managing SNMP keys .......................................................................................................................... 193Managing NETCONF keys ................................................................................................................... 195Team configuration using curl commands..............................................................................................196

Scripts........................................................................................................... 216Restoring a controller.............................................................................................................................216Backing up a controller team................................................................................................................. 220Restoring a controller team ...................................................................................................................227

6 Aruba VAN SDN Controller 2.8 Administrator Guide

Using an external policy manager.............................................................. 234

Performance testing.....................................................................................235

Examples of Metrics.....................................................................................236

Contents 7

This document describes the configuration and management of the Aruba VAN SDN Controller in standalone andteam modes.

About the Aruba VAN SDN ControllerThe Aruba VAN SDN Controller provides a unified control point in an OpenFlow-enabled network, simplifyingmanagement, provisioning, and orchestration and enabling delivery of a new generation of application-basednetwork services.

In the Aruba Software Defined Networking (SDN) architecture, the control and data planes of the network aredecoupled from each other, centralizing network intelligence and abstracting the underlying network infrastructurefrom applications. Controller software manages forwarding behavior for physical and virtual switches under itscontrol via the industry-standard OpenFlow protocol. Network ports, links, and topologies are all directly visible,enabling centralized policy administration and more effective path selection based on a dynamic, global view ofthe network. This dramatically simplifies the orchestration of multi-tenant environments and the enforcement ofnetwork policy for both mobile clients and servers.

The Aruba VAN SDN Controller is designed to operate in a variety of computing environments, including campus,data center, service provider, private cloud, and public cloud. The Aruba VAN SDN Controller features:

• An enterprise-class platform for the delivery of a broad range of network innovations• An extensible, scalable, and resilient controller architecture• Compliance with OpenFlow 1.0 and 1.3 protocols• Support for Hewlett Packard Enterprise and H3C OpenFlow-enabled switches• Secure authentication using a local or remote Keystone server• Controller teaming for distributed platform High Availability (HA)• Embedded applications that provide common network services• Open APIs enable SDN application developers to deliver innovative solutions that dynamically link business

requirements to network infrastructure using either custom Java programs or general-purpose RESTful controlinterfaces, including functions to extend the controller REST API and UI.

• Integration with HPE Intelligent Management Center (IMC). HPE IMC provides full controller application lifecycle management and monitoring, enhanced reporting and SDN network visualization.

The Aruba SDN ecosystemSDN architecture separates the network control plane from the forwarding hardware on network devices. Controlcan then be centralized, while forwarding remains distributed. SDN is based on OpenFlow, which is a standards-based protocol allowing for a centralized-control plane in a separate device (the controller).

OpenFlow is managed by the Open Networking Foundation (ONF). By separating the control plane from theforwarding plane, SDN makes it possible for the network status and capabilities to be exposed directly to thebusiness service layer, so that business systems can request services from the network directly. SDN applicationsthus provide higher level application direction to the SDN controller. And freed from the control function, theforwarding plane can then provide optimized packet processing at very high speeds.

The Aruba VAN SDN Controller is the central building block of the Aruba SDN ecosystem and creates a platformfor application development.

Chapter 1Introduction


The Aruba SDN ecosystem includes the following:

• Infrastructure.

The infrastructure layer is made up of network devices, typically but not exclusively routers and switches. Thedevices are OpenFlow-enabled. An OpenFlow switch consists of one or more flow tables and a group table,which perform packet lookups and forwarding and provide an OpenFlow channel to the Aruba VAN SDNController. The switch communicates with the controller and the controller manages the switch via theOpenFlow protocol. Hewlett Packard Enterprise has more than 50 switch models that are OpenFlow-enabled.

• Control.

Aruba VAN SDN Controller provides centralized control and automation for an SDN network. The controllercontrols policy and forwarding decisions, which are communicated to the OpenFlow-enabled switches in thedata center or campus network. A variety of Hewlett Packard Enterprise and third-party SDN applications canleverage the controller to automatically deliver the necessary business and network service levels.

• Applications.

Hewlett Packard Enterprise and third-party SDN applications provide a true end-to-end service level fornetwork performance, quality of service, and security, which can be tuned to an applications’ needs. Forexample, SDN applications can inspect flows, or perform other network control functions via the Aruba VANSDN Controller. Aruba SDN applications include: Aruba Network Protector SDN Application, Aruba NetworkOptimizer SDN Application and Aruba Network Visualizer SDN Application.The extensibility and open APIs ofthe Aruba VAN SDN Controller allows new applications to be created that make requests of the underlyingnetwork, without the need to physically uproot or re-configure the underlying infrastructure. Northbound APIsutilize the REST architecture and provide easy access to applications that are integrated directly in thecontroller or off the controller. Native APIs, provided in Java, deliver support to Network Control applicationsthat are integrated directly in the controller.

• Management.

The HPE Intelligent Management Center (IMC) VAN SDN Manager software integrates with HPE IMC toprovide administrators with a single interface to manage both the traditional network and the SDN. The IMCVAN SDN Manager Software monitors and manages all three layers of the SDN architecture: infrastructure,control, and application, providing comprehensive management—including fault, configuration, accounting,monitoring, and security for the controller and OpenFlow infrastructure. IMC provides full controller applicationlife cycle management and monitoring, reporting of network service status and OpenFlow-related information,and SDN network visualization.In addition, the Aruba VAN SDN Controller provides REST and Java APIs thatenable applications to interact with the controller to receive alerts, to get information about the network,devices, and controller, and to perform various network management tasks.

Chapter 1 Introduction 9

SDN Controller applications and the App StoreThe Aruba VAN SDN Controller includes a default set of core network service applications that are installed asmodules with the controller. These embedded applications provide services such as authentication, datapersistence, logging and alerts. For details, see Understanding the controller architecture on page 12.

The Aruba VAN SDN Controller also provides a platform for developing and deploying SDN applications. Severalapplications have been developed by Aruba including Aruba Network Protector SDN Application, Aruba NetworkOptimizer SDN Application, and Aruba Network Visualizer SDN Application. There are also SDN applicationsdeveloped by third-party partners. In addition, you can develop your own SDN applications.

Aruba’s SDN applications as well as third-party SDN applications are available through the Hewlett PackardEnterprise SDN App Store. Access the App Store at www.hpe.com/networking/sdnappstore.

The Aruba VAN SDN Controller includes an SDK providing the tools needed to develop applications for thecontroller. The SDK includes documentation for both the Java and REST APIs as well as all of the jar filesnecessary during compilation. A sample application is also included along with API specifications. For details onhow to develop applications for the controller, see the Aruba VAN SDN Controller Programming Guide.

Hewlett Packard Enterprise offers an SDN developer community, as well as forums, events, and other services, tohelp developers and partners build and sell SDN applications.

Hewlett Packard Enterprise SDN information libraryThe following information is provided for the Aruba VAN SDN Controller:

• Aruba VAN SDN Controller Release Notes• Aruba VAN SDN Controller Installation Guide• Aruba VAN SDN Controller Administrator Guide• Aruba VAN SDN Controller and Applications Support Matrix• Aruba VAN SDN Controller Programming Guide• Aruba VAN SDN Controller REST API Reference• Aruba VAN SDN Controller Troubleshooting Guide• Aruba VAN SDN Controller Open Source and Third-Party Software License Agreements

The most recent versions of these documents are in the Hewlett Packard Enterprise SDN information library atthe following website: www.hpe.com/info/sdn/infolib.

Switch and OpenFlow requirementsSwitch and OpenFlow requirements for use with the controller:

• The controller must be connected to a network that includes one or more switches configured to runOpenFlow.

• Plan and implement the switch OpenFlow configurations before connecting the controller to the network.• OpenFlow switches in the network must be configured to allow control by the Aruba VAN SDN Controller. In a

controller domain, including a switch that does not support OpenFlow or allow control by another Aruba VANSDN Controller creates separate clusters of OpenFlow networks.

• Create a separate VLAN for an OpenFlow control plane. Running the OpenFlow control mode on a specifiedswitch VLAN disrupts the traffic on that VLAN until the controller configures the required flow rules in theswitch using the OpenFlow controller API. For information on configuring OpenFlow, see the latest OpenFlowAdministration Guide for your switch.

• Do not connect OpenFlow switches in a controller domain in a loop topology with switches outside the domain.


http://www.hpe.com/networking/sdnappstore

http://www.hpe.com/info/sdn/infolib

Allowing such connections can create broadcast loops inside the OpenFlow network. For more onpacket-forwarding decisions, see Hybrid mode for controlling packet forwarding on page78.

• Set the controller to hybrid mode true (the default) in order to support IPv6 traffic running in the data plane ofan OpenFlow network. In this state the controller is not aware of the IPv6 traffic. However, with hybrid modeset to false (all packets sent to the controller), the controller drops IPv6 packets, and they do not reach theirdestinations.

For information about supported network switches, OpenFlow versions, and switch configuration requirements,see the Aruba VAN SDN Controller and Applications Support Matrix.

Chapter 1 Introduction 11

SDN controller architectureThe Aruba VAN SDN Controller software is built upon a Linux OS, Java 1.8, and OSGI (Virgo stack and Equinoxframework) and uses an Apache Cassandra distributed post-relational database.

Keystone is an external service that provides authentication and high level authorization services. It supportstoken-based authentication

REST API and GUI framework are used by SDN application developers for building applications (RESTful webservices and web based UIs).

Figure 1: Aruba VAN SDN Controller software stack

The following gives a short description of the controller components:

• The controller Application Manager enables installing, upgrading, enabling (starting), disabling (stopping), anduninstalling SDN applications on the controller.

• The Audit Log records events related to activities, operations, and configuration changes initiated by anauthorized user. The Audit Log is managed by the controller Audit Log service.

• The Alert Log records information about events that affect controller operation, and in some cases indicate thatsome action is needed to correct a condition. Alerts are managed by the controller Alert service.

• Client Mapper Service combines information known about a network client by the controller, such as host IPaddress, host MAC addresses, and the connected datapath and port, with information about the network clientknown by an outside policy manager, such as the Aruba ClearPass policy manager, to provide informationabout network clients, including user information, device information, and location information. This informationis available via the REST API only.

• The controller uses the embedded applications Topology Manager and Topology Viewer to collect and displayinformation about the OpenFlow network.

• The controller provides a framework to back up and restore controller and application state in a backup file.The backup file can be copied and stored for later use. The stored backup file can be uploaded to thecontroller.

• The Distributed Coordination Framework is one of the high-availability features of the controller. It provides theinfrastructure for controller-to-controller communication and coordination of state information for controllers in acontroller team.

Chapter 2Understanding the controller architecture


• The controller can be configured in a team. The teaming services of the controller keep the runtime state ofeach controller in the team (active, unreachable, or suspended) up to date and is used by other parts of thecontroller for functions related to high-availability.

• The Device Drivers model the capabilities of the devices and provide APIs for interacting with different devicetypes.

• The controller uses the embedded applications OpenFlow Link Discovery and OpenFlow Node Discovery todiscover information about the OpenFlow network.

• The OpenFlow controller (also called the core controller) handles the connections from OpenFlow devices andprovides the means for upper layers of software to interact with those devices.

The Aruba VAN SDN Controller includes a default set of core network service applications that are installed withthe controller (see List of controller embedded applications.

List of controller embedded applicationsThe Aruba VAN SDN Controller includes a default set of core network service applications that are installed asmodules on the controller. The following applications are embedded in the controller and are installed when youinstall the controller:

• OpenFlow Link Discovery• OpenFlow Node Discovery• Path Daemon• Path Diagnostics• Topology Manager• Topology Viewer

OpenFlow Link DiscoveryThe OpenFlow Link Discovery application is the default OpenFlow link supplier application that is installed withthe controller. This application implements the com.hp.sdn.supplier.LinkSuppliersBroker interface anduses LinkSupplierService and LinkService APIs to create and maintain link information for OpenFlowdatapaths that register with the controller.

The OpenFlow Link Discovery application does the following:

• Discovers the following types of links:

◦ direct links◦ multi-hop links

• Listens to device and interface events and registers with the ControllerService API to send OpenFlowpackets to datapaths.

• If the OpenFlowLinkDiscoveryComponent configuration has age.multihop.links=true, theOpenFlow Link Discovery application periodically injects link-discovery packets into the controlled network torefresh the multihop links. Any multihop links that are not refreshed at the interval configured for themultihop.poll.interval key are considered to be invalid and are removed from the link table.Additionally, network events such as a port going down or a device status change causes relevant links to beremoved from the link table, and causes discovery packets to be sent to all datapaths that are in a ready state.

How the OpenFlow Link Discovery application determines the link type1. The application injects two controller-generated link-discovery packets to each port in an OpenFlow instance.

These packets have the same Ethernet type (0x8999), but are sent to different destination MAC addresses:

• The link-local MAC address to discover direct links:01:08:c2:00:00:0e• The multicast MAC address to discover multihop links:01:1B:78:E9:7B:CD

2. The application evaluates the hybrid mode configuration of the controller:

Chapter 2 Understanding the controller architecture 13

• If the ControllerManager configuration has hybrid.mode=true, the application installs a flow rule onevery OpenFlow device to steal controller-generated link discovery packets. Packets that match this flowrule are forwarded to the controller from the OpenFlow instance and port on which they were received.

• If the ControllerManager configuration has hybrid.mode=false, all packets are forwarded to thecontroller by default. Therefore the application does not install flow rules on the OpenFlow devices.

3. The application listens for link-discovery packets.The application determines the link type by examining thelink-discovery packets sent to the controller:The link type is derived from the destination MAC address of thepacket (direct or multi-hop). If a link is direct, it will be discovered as both direct and multi-hop from thereporting OpenFlow instance, but the type direct has precedence over the type multi-hop, so the link isrecorded as direct.

Characteristics of a controller-generated link-discovery packetA controller-generated link-discovery packet:

• Uses a non-standard protocol, BDDP, which uses a payload format similar to LLDP.• Is sent to either a link-local MAC address (to discover direct links) or a multicast MAC address (to discover

multihop links).• Contains the source device and port that introduced the packet to the controlled network.• Has the source device ID embedded in its payload. The destination device can be derived from the

PACKET_IN message. This design enables the OpenFlow Link Discovery application to populate the link tablewith information it learns from received packets.

• Is used only for discovering links, so it is received from the device by the controller via a PACKET_INmessage, but is not returned to the device via a PACKET_OUT message.

OpenFlow Node DiscoveryThe OpenFlow Node Discovery application is the default OpenFlow node supplier application that is installed withthe controller. This application implements the com.hp.sdn.supplier.NodeSuppliersBroker interface anduses NodeSupplierService and NodeService APIs to create and maintain node information for OpenFlowdatapaths that register with the controller.

The OpenFlow Node Discovery application uses the following process to create and maintain node information inthe node table:

1. The application evaluates the hybrid mode configuration of the controller:

• If the ControllerManager configuration has hybrid.mode=true the OpenFlow Node Discovery applicationinstalls flow rules that instruct OpenFlow datapaths to copy ARP packets or DHCP packets to thecontroller.By default in hybrid mode, IP packets are not sent to controller. The OpenFlow Node Discoveryapplication does not install flow rules that instruct OpenFlow datapaths to send IP packets because doingso would drastically reduce network performance by overwhelming the control plane. However, theOpenFlow Node Discovery application can listen for IP packets that other applications have instructedOpenFlow datapaths to send to the controller.

• If the ControllerManager configuration has hybrid.mode=false, all packets are forwarded to thecontroller by default. Therefore the application does not install flow rules on the OpenFlow devices.

2. The application ignores PACKET_IN messages from ports that identified by the Topology Manager as beingpart of the infrastructure, and listens for PACKET_IN messages that contain the ARP or DHCP protocols. If theOfIpDiscoveryComponent configuration has learn.ip=true, the application also listens for PACKET_INmessages that contain the IP protocol.Because these PACKET_IN messages are copies of packets that havealready been forwarded to the controller, the controller does not send corresponding PACKET_OUT messagesfor these PACKET_IN messages.


3. If learn.ip=true, based upon the information supplied by these copied ARP, DHCP, and IP packets, theOpenFlow Node Discovery application registers as a node supplier and supplies updates to the node table.

4. The controller administrator can configure the timeout value for nodes discovered by each protocol by settingthe value of the age key of the configurable component for that protocol. When the timeout value is reached,OpenFlow Node Discovery application removes the node from the node table.

Path diagnosticsThe Path Diagnostics application determines and verifies the path taken by trace packets from a source host to adestination host. The application finds an existing flow that matches the description of the trace packet, clones itwith higher priority, and adds an action to instruct the selected datapath to send this packet back to the controllerfor status tally.

The Path Diagnostics application is available when the Controller Manager configuration hashybrid.mode=false only.

Topology ManagerThe Topology Manager provides topology information about the control domain and facilitates shortest pathtraversals through the control domain.

The Topology Manager:

• Indicates whether a connection point is part of Infrastructure or is connected to an end host• Indicates whether ingress broadcast traffic can be allowed through a specified connection point• Determines if a path exists between two datapaths• Identifies the shortest path between two datapaths based on hop count or link edge weight• Provides enumeration of the grouping of datapaths into clusters of strongly connected nodes• For a given datapath, provides information about the cluster to which the datapath belongs• Provides information about number of datapaths, number of links, and number of clusters in the current

topology• Creates the clusters and broadcast tree to avoid loops and broadcast storms.• Provides notifications to subscribed applications about changes in its broadcast tree and cluster. Applications

that subscribe to these notifications can use the information to respond to changes in topology.

Topology ViewerThe Topology Viewer application creates and updates a network graph for visualizing the network the controllerdiscovers. This graph is displayed on the OpenFlow Topology screen.

The Topology Viewer uses the services of the Topology Manager and the Link Manager.

Path DaemonPath Daemon is a path-paving application that listens for all ARP and IPv4 PACKET_IN messages—flow missesthat arrive at the controller—and attempts to install or modify flows on datapaths along the forwarding path toensure that such packets get forwarded at line-rate.

Path Daemon operates only when the entire network is controlled by the controller (ControllerManagerconfiguration has hybrid.mode=false and there are no uncontrolled devices).

By default, Path Daemon is responsible for Layer-2 forwarding only.

Each PACKET_IN message processed by Path Daemon results in a PACKET_OUT message and possibly a flow-mod message getting pushed to one or more controlled devices.

By default, the Path Daemon application installs flows that attempt to forward traffic using the following:


• MAC address and incoming port for ARP PACKET_IN messages• IPv4 address and incoming port for IPv4 PACKET_IN messages

Specifically, the flows will match all packets that enter a specific switch on a specific port and they will match onlypackets with the source MAC or IPv4 address and destination MAC or IPv4 address from the PACKET_INmessage. Any packets that match the flow will be forwarded by the datapath to the most optimal destination port—determined by Path Daemon—for the packet to reach its intended destination.

Operational notes

The Path Daemon:

• Does not handle multicast or broadcast traffic• Does not configure the reverse path along with the forward path• Drops packets from sources that the controller has not learned• Floods packets when their destinations are not known• Does not support fast-failover• Performance is topology-dependent and can degrade when the network contains more than approximately 200

nodes.

Match fields used by Path Daemon

The Path Daemon uses the following match fields for flow-mod message. These match fields have been chosenso that the flow entries are added to hardware tables in ArubaOS-based, ProVision-based, and Comware-basedswitches.

• Ether type: OFPXMT_OFB_ETH_TYPE• Source MAC or IP address: OFPXMT_OFB_ETH_SRC or OFPXMT_OFB_IPV4_SRC• Destination MAC or IP address: OFPXMT_OFB_ETH_DST or OFPXMT_OFB_IPV4_DST• Input port: OFPXMT_OFB_IN_PORT

Detailed operation

The Path Daemon does the following:

1. Registers with the controller as a Director. Directors are allowed to send a packet out.2. Registers for ARP packets and IPv4 packets.3. Registers for Port Status Down messages. When such messages are received, Path Daemon removes all

flows configured for the impacted port, thereby causing the PACKET_IN messages to again come to thecontroller.

4. Uses the Node Manager to get the end hosts corresponding to the source and destination MAC addresses andthe datapaths to which these hosts are connected.


5. Uses the Path Selection manager to get the end-to-end shortest path between the source and destinationhosts.

6. Uses the controller to push flow-mod messages to the datapaths. The flowchart below provides more details ofits operation.

Figure 2: Path Daemon flowchart


The SDN controller provides a console UI you can use as follows:

• View information such as alerts and logs and view OpenFlow information such as data flow details, topology ofdiscovered switches and end nodes including shortest path and view OpenFlow classes that applications haveregistered.

• Perform actions such as acknowledging an alert, adding or enabling an application, exporting log data andentering licensing information.

• Configure SDN controller components such as setting key values for alert policies.

The SDN controller also provides REST APIs you can use to program or configure the controller and developapplications to run on the controller. For details on how to use the REST APIs and how to develop applications,see the Aruba VAN SDN Controller Programming Guide and Aruba VAN SDN Controller REST API Reference.

This chapter includes details on the following:

Starting the SDN controller console UI Licenses on page 48

The SDN controller user interface Team

SDN User window on page 21 Support logs on page 50

Navigation menu on page 23 Packet listeners on page 53

Alerts on page 25 OpenFlow Monitor on page 54

Applications on page 30 OpenFlow topology on page 58

Configuration components on page 37 OpenFlow Trace log on page 67

Audit log on page 45 OpenFlow Classes on page 74

Changing the language displayed in the user interface on page 19

Starting the SDN controller console UIAccess the SDN controller from a Chrome or Firefox supported browser. A message will be displayed indicating ifyou are using an unsupported browser such as Internet Explorer.

Procedure

1. Using a supported browser, access the controller UI:

https://<SDN_Controller_Address>:8443where <SDN_Controller_Address> is the IP address for your controller.

The URI is case sensitive. For example: https://192.0.2.1:8443.2. Enter the User Name and Password credentials, then select Login. For example:

Default user name: sdn

Chapter 3Using the SDN controller UI


3. Once you log in, the main controller screen is displayed. For more information about the controller console UI,see The SDN controller user interface.

The Keystone default timeout is 1 hour. If it is more than 1 hour since you logged in a message indicating that thesession has expired is displayed. You must reload the page and log in again. For details on changing theKeystone timeout value, see Session expired message in the UI.

Default domain name, user name, and passwordDefault domain name: sdnDefault user name: sdnDefault password: skyline

Changing the language displayed in the user interface

How the controller determines what language to displayTo determine what language to display, the controller uses the value of the LANG environment variable in theUbuntu /etc/default/locale file at the time the controller starts.

Applications installed on the controller might not include multiple language support, so user interface componentsprovided by applications might not be in the same language as the controller components.

Supported languages

Language LANG entry in the /etc/default/locale file

English LANG=en_US.UTF-8

Chinese LANG=zh_CN.UTF-8

Japanese LANG=ja_JP.UTF-8

Default languageThe default language is English (en_US.UTF-8).

The controller user interface displays the default language in the following circumstances:

• There is no /etc/default/locale file.• The LANG environment variable is set to a value that the controller does not support.

Chapter 3 Using the SDN controller UI 19

Changing the LANG environment variableTo change the locale:

Procedure

1. Log in to Ubuntu on the system on which the controller is installed.2. Open the /etc/default/locale file for editing:

~$ sudo vi /etc/default/locale3. Change the entry for LANG to one of the supported values, then save and close the file.4. Enter the following command to ensure the setting is applied to the user that logs into the controller:

$ su username -For example, for the sdn user, enter the following command:$ su sdn -

5. Verify that the locale has changed by entering the following command to display the current locale:$ locale

6. Restart the controller:a. Close any instance of the web interface in which the controller might be running.b. Restart the controller:

~$ sudo service sdnc restart

The SDN controller user interfaceDescriptions for common areas, icons, and controls on the UI screen are listed after the image.

Figure 3: Screen areas and menus

• Banner: Identifies the user interface. Contains the alert notification counter and links to the navigation menu,alert information, and the SDN User window.

• Alert notification counter: Displays the current number of active alerts. Clicking this icon displays the Alerts asof Today dialog box.

• SDN User window: Enables you to log out of the controller, link to external websites, change the theme for thecontroller, and identify the version of controller software currently in use.

• Navigation menu: The primary menu for navigating to controller and application resources. Contains thecontroller navigation tree, labeled General, and can contain additional navigation trees for installed


applications that integrate with the controller UI. Can be displayed as a pane (as shown) or as a window thatoverlays the controller screen (see Expanding or collapsing the navigation menu).

• Navigation tree: Used to select the controller or application screen to display in the details pane. General isthe controller navigation tree. Navigation trees for installed applications are displayed below or to the right ofthe General navigation tree.

• Details pane: Displays the detailed interface for the controller or application resource selected in theNavigation menu.

• Pagination control: Can appear on screens that have lists of items. Use these controls to view the listings pageby page.

• Listing control: Can appear on screens that have lists of items. Use these controls to select the number ofitems to display in a single view. The Auto option displays all items in a single screen. For listings exceedingthe length of the screen, you can use the scroll bar on the right side of the screen.

UI top banner components

Screencomponent

Description

SDN Controllermenu expansionicon

Expands or collapses the navigation menu as an overlay window.

critical alert Expands or collapses the controller Alerts as of today window. The number next to theicon is the alert notification counter, which provides a count of the current active alerts.

sdn user window

icon

Expands or collapses the UserSDN window.

Changing column widths in the UITo change the column widths, drag the column head borders.

For example:

Procedure

1. To narrow the Severity column width, click the border to the left of Date/Time and drag it to the left.2. To change the width of the navigation menu pane, click and drag the divider between the menu pane and the

details pane.

SDN User windowThe SDN User window displays as an overlay on the controller screen.


User window screen detailsFigure 4: SDN user window

Screencomponent

Description

Log out Logs the user out of the controller.

Change SDN UserPassword

Change the SDN user password.

Links: Links to websites outside of the controller:SDN Information LibraryLinks to theinformation library on the Hewlett Packard Enterprise Software-Defined Networkingwebsite. The Hewlett Packard Enterprise Information Library for SDN provides links to thetechnical documentation for the Aruba VAN SDN Controller and the HP SDN applications.The Hewlett Packard Enterprise Software-Defined Networking website provides factsheets, case studies, white papers, product summaries, technical and businessdocumentation, and other information to help you identify SDN solutions for your businessneeds.SDN CommunityLinks to the Hewlett Packard Enterprise SDN communitydiscussion forum website within the HP Enterprise Business Community. This site offersresources such as:

• SDN discussion boards• SDN development information• An SDN knowledge base

Set Theme: Changes the theme for the controller UI:DayWhen selected, plain text is black and thebackground is white.NightWhen selected, plain text is white and the background is black.

Table Continued


Screencomponent

Description

SDN ControllerVersion:

Displays the version of the controller software that is running on this system.

collapse icon Collapses the window.

Expanding or collapsing the SDN User window

To expand the SDN User window, from the top banner, click sdn user window icon .

To collapse the SDN User window, do one of the following:

Procedure

1. In the SDN User window, click the collapse icon .

2. From the top banner, click the sdn user window icon .

Changing the SDN user password

Procedure

1. Expand the SDN User window.2. Select Change SDN User Password.

The SDN user password you can change on this screen is the Keystone user password not theHPE Linux operating system password.

3. In the Change SDN User Password dialog box, enter the Old Password, New Password, and Re-enter NewPassword and click Apply. Or click Cancel to exit without changing the password.

Changing the background and text colorsThe background and text colors are part of the theme of the controller UI.

Procedure

1. Expand the SDN User window.2. In Set Theme:, select one of the following options:

a. Dayb. Night

Logging out of the controller

Procedure

1. From the SDN User window, select Log out.

Navigation menu


About the navigation menuThe navigation menu is the primary menu for navigating to controller resources. The resources included with thecontroller are described in this document. Applications installed on controller might add resources to this menu.

Displays as a pane or an overlay window

You can display the navigation menu in the following ways:

• As a pane on the left side of the controller browser window.• As a window that overlays part of the main screen of the controller browser window.

Contains one or more navigation trees

The navigation menu contains the General controller navigation tree and can contain additional navigation treesfor installed applications that integrate with the controller UI.

Expanding or collapsing the navigation menuThe navigation menu is displayed as a navigation pane by default. You can display the navigation menu as apane on the controller screen or as a window that overlays the controller screen.

Expanding or collapsing the navigation menu as an overlay window

To display the navigation menu as an overlay window, from the top banner of the controller screen, click nav

menu .

To collapse the navigation window, do one of the following:

Procedure

1. In the window, click the collapse icon .2. From the top banner, click Aruba VAN SDN Controller.

Expanding or collapsing the navigation menu as a window pane

To expand or collapse the navigation menu as a window pane, click the following icon:

• When the navigation menu is expanded as a window pane, the icon is located on the right side of the menu.• When the navigation menu is collapsed, the icon is located in the left margin of the controller screen.

Navigation menu screen details

Screen component Description

General Displays the navigation tree for the resources that are provided with the controller. Bydefault, the General controller navigation tree is expanded and the Alerts screen isselected and displayed. To display the screen for another resource, select theresource in the navigation tree.

Alerts Displays the Alerts screen. This screen provides details on alerts and allows you toacknowledge alerts and unacknowledge alerts back to an active state. For moreinformation, see Alerts on page 25.

Table Continued



Applications Displays the Applications screen. This screen provides details on your controllerapplications and allows you to add, upgrade, uninstall, enable and disable theseapplications. For more information, see Applications on page 30.

Configuration Displays the Configurations screen. This screen lists the configurable components ofthe controller and allows you to modify key values. For more information, see Configuration components on page 37.

Audit Log Displays the Audit Log screen. This screen displays audit log records related toactivities, operations and configuration changes initiated by an authorized user, suchas, installing an application. For more information, see Audit log on page 45.

Licenses Displays the Licenses screen. This screen provides details on licenses and allows youto enter a license. For more information, see Licenses on page 48.

Team Displays the Team screen. For more information, see Team.

Support Logs Displays the Support Logs screen. This screen displays support log records of internalcontroller operations that can be used by a support engineer for troubleshooting anSDN installation. For more information, see Support logs on page 50.

OpenFlow Monitor Displays the OpenFlow Monitor screen. This screen lists the Data Path IDs anddescriptive information for the active switches. For more information, see OpenFlowMonitor on page 54.

OpenFlow Topology Displays the OpenFlow Topology. Displays a topology of discovered switches and endnodes in the controller domain. For more information, see OpenFlow topology onpage 58.

OpenFlow Trace Displays the OpenFlow Trace screen. OpenFlow conversations are captured inmessages to and from the controller and the OpenFlow devices it manages anddisplayed on this screen. For more information, see OpenFlow Trace log on page67.

OpenFlow Classes Displays the OpenFlow Classes screen. This screen shows the OpenFlow classesthat applications have registered with the controller. For more information, see OpenFlow Classes on page 74.

Packet Listeners Displays the Packet Listeners screen. This screen displays details on the packetlisteners that are currently running on the controllers. For more information, see Packet listeners on page 53.

Other navigation menuitems

May include additional navigation trees for installed applications that integrate with thecontroller UI.

AlertsAlerts give notification of events that affect controller operation, and in some cases indicate that some action isneeded to correct a condition.

When controllers are operating in a team, alerts generated by any team member are visible in the Alerts screenfor all active team members.


By default, alerts are in an unacknowledged, active state. An alert must be in an active state to appear in thefollowing places:

• The alert notification counter• The Alerts as of today window

Alerts screen detailsFigure 5: Display the learn.ip option


Refresh Updates the alerts displayed on the screen. The controller does not update thedisplay as new alerts are generated. Use this action to refresh the display.

Acknowledge Changes the selected alert to an acknowledged state. The controller displays thealert in gray text. Use this action to indicate that you have read the alert.

UnAcknowledge Changes the selected alert to an active, unacknowledged state.

Alert text color Indicates the state of the alert:

• The controller displays active, unacknowledged alerts the alert in the textcolor corresponding to the controller theme. For example, when the controllertheme is daylight, the active alerts appear in black text.

• The controller displays the selected alert in blue text. Click an alert to select it.• The controller displays acknowledged alerts in gray text.

SeverityIndicates the severity of the alert. informational icon warning icon critical icon

Date/Time Indicates the date and time the alert was generated.

Description Describes the alert in human readable text.

Table Continued



Origin Indicates which component or application generated the alert.

Topic Indicates the category for this alert. Multiple origins can contribute alerts to thesame topic.

Controller ID Identifies the controller that generated the alert. The controller is represented asa hexadecimal number. When you use controller teaming, this ID enables you toidentify which controller in the team generated the alert.

Alert notification counterThe alert notification counter is displayed in the top banner and appears on all controller screens.

Figure 6: Alert notification counter

This counter indicates the number of active alerts:

• The controller increments this counter when each new alert occurs.• The controller decrements this counter when you acknowledge an alert or when the controller deletes an alert

according to the alert policies set for aging out alerts.

Viewing the ten most severe recent active alertsThe Alerts as of today window shows up to 10 alerts ranked by severity (highest to lowest) and then by date andtime (newest to oldest).

Procedure

1. In the top banner, click the critical icon .


The Alerts as of today window is displayed:

2. To close the window, do one of the following:a. To close the window and display the Alerts screen, click All.

b. At the bottom of the window, click the collapse icon .

c. In the top banner, click either the alert counter number or critical icon .

Acknowledging an alertTo acknowledge an alert from the Alerts as of today window:

Procedure

1. Click the alert to select it.2. Click Acknowledge.

The result is that the alert is removed from the Alerts as of today window, the alert is displayed in gray text onthe Alerts screen, and the alert notification counter is decremented by one.

To acknowledge an alert from the Alerts screen:

1. Click the alert to select it.2. Click Acknowledge.

The result is that the alert is displayed in gray text on the Alerts screen, and the alert notification counter isdecremented by one.

Deleting an alertYou can acknowledge an individual alert, but you cannot clear or delete the alert.


The controller deletes alerts according to the configured alert age-out policy. To configure the age-out policy, see Configuring how alerts age out.

Configuring how alerts age out

Procedure

1. On the Configurations screen in the System tab, select thecom.hp.sdn.adm.alert.impl.AlertManager component.:

Figure 7: Select the AlertManager component

2. Click Modify.

The Modify System Configuration dialog box is displayed for thecom.hp.sdn.adm.alert.impl.AlertManager component:

3. Change the values for the keys (see Key values that control how alerts age out on page 29).4. Click Apply.

Key values that control how alerts age outYou can configure the following key values for alerts to control how alert data ages out.


Key Description

trim.alert.age Specifies the number of days an alert remains in persistent storage and remainsdisplayed on the Alerts screen.

Data type A number from 1 through 31

Default value 14

trim.enabled When true, specifies that the controller deletes alerts that have exceeded thetrim.alert.age limit.

Default value true

trim.frequency Specifies how often, in hours, the controller is to delete alerts that have exceededthe trim.alert.age limit.


Default value 24

Example Enter 8 to specify that the controller delete aged-out alerts everyeight hours.

Applications

About the application managerThe Application Manager is a component on the controller that supports default and add-on network services, andenables installing, upgrading, enabling (starting), disabling (stopping), and uninstalling SDN applications.

When controllers are operating in a team, actions performed on one controller are propagated to the othercontrollers in the team. Actions you select in the Applications screen for one controller, such as Install, Enable,and Disable, are propagated to the other controllers.

See also:

• Obtaining applications from the Hewlett Packard Enterprise SDN App Store on page 32• Adding or upgrading an application on page 32• Disabling (stopping) or enabling (starting) an application• Uninstalling an application on page 34• Understanding application states and OSGi artifacts

Prerequisites for installing an applicationAny application to be installed on the controller must meet the following requirements:

• It must be in a zip format.• The zip file must be accessible from the browser UI's file manager (or downloadable from the App Store).• It must contain an application descriptor file with key value pairs of the attributes associated with the

application, including all mandatory attributes.• If jar signing checking is turned on, the application zip files must be signed as well.


Applications you purchase from Hewlett Packard Enterprise or the Hewlett Packard Enterprise SDN App Storemeet these requirements. For information about developing applications that meet these requirements, see theAruba VAN SDN Controller Programming Guide

Applications screen detailsFigure 8: Applications screen details


Refresh Reloads the view.

New Installs an application on the controller.

Upgrade Installs an upgrade to an application that has already been installed on thecontroller.

Uninstall Removes an application from the controller.

Enable Starts or allows an application to continue operations on the controller.

Disable Stops or prevents an application from operating on the controller.

Launch Network Protector If you have the Aruba Network Protector SDN Application installed, this buttonwill be enabled to allow you to launch the Aruba Network Protector application.

Table Continued



Name The name of the application.The following core embedded applications thatcome with the controller are listed by default on the Applications screen:

• Path Diagnostics• OpenFlow Link Discovery• OpenFlow Node Discovery• Path Daemon

These are the only embedded applications you are allowed to manage using theUI. Other embedded applications are not listed because they should not bedisabled or uninstalled. For information about embedded applications, see Understanding the controller architecture on page 12.If you have installedany of Aruba’s SDN applications, such as Aruba Network Protector, or any third-party SDN applications these applications will also be listed.

Version The version number of the application.

State The most common states are active, staged and disabled.

AppStore PurchasedApplications

The name and version number of SDN applications purchase from the HewlettPackard Enterprise SDN App Store.

Launch AppStore Launches the Hewlett Packard Enterprise SDN App Store website.

Obtaining applications from the Hewlett Packard Enterprise SDN App StoreFrom the App Store, you can purchase and download applications for your controller.

In the case of a web proxy, you need a proxy configuration to connect to the App Store portal. To set the proxy, inthe /etc/init/sdnc.conf file, replace c1, c2, c3, and team with the controller IP addresses and the team IPaddress (when applicable):

env JAVA_OPTS="-Xms512m -Xmx4096m -XX:MaxPermSize=512m -Dhttps.proxyHost=my-web-proxy -Dhttps.proxyPort=my-web-proxy-port -Dhttp.nonProxyHosts=127.0.0.1|localhost|c1|c2|c3|team" Command Example

env JAVA_OPTS="-Xms512m -Xmx4096m -XX:MaxPermSize=512m -Dhttps.proxyHost=web-proxy.test.com -Dhttps.proxyPort=8088 -Dhttp.nonProxyHosts=127.0.0.1|localhost|11.152.121.172|11.152.126.13|11.152.127.5|11.152.123.6"You must update the /etc/init/sdnc.conf file on each controller and then restart the controllers for thesechanges to take effect.

If you are downloading a signed application from the App Store, the JAR signing requires a trusted certificate inthe sdnjar_trust.jks file even if the certificate is trusted in the JAVA cacerts keystore. For details, see Adding certificates to the jar-signing truststore on page 125

Adding or upgrading an applicationAny application in the proper format can be added to the controller (see About the application manager on page30).

To use curl commands and the REST APIs to complete this task, see curl commands on page 182.


Use this procedure to install either a new application or a new version of an existing application on the controllerusing the UI.

Procedure

1. Do one of the following:

• To install a new application, click New.• To upgrade to a new version of an existing application, select the application from the Name list and click

Upgrade.2. Click Browse to navigate to the location of the application zip file and select the file.3. Click Upload to upload the file.4. Wait for Completed to appear. For example:

5. Click Deploy.

The new application then appears by name on the Applications screen as ACTIVE.

After you complete this procedure:

1. The application is started and in an active state.2. If the controller is in a controller team, the controller propagates the application to all the controllers in the team

automatically.

Disabling (stopping) or enabling (starting) an applicationThis procedure temporarily stops an active application from servicing requests, but retains the application on thesystem. The application remains present on the system and can be restarted when needed. (The application doesnot automatically restart when the controller restarts.)


To disable an application using the UI:

Procedure

1. In the Applications screen, select the application you want to stop.2. Click Disable to display the Disable Application dialog box.


3. In the Disable Application dialog box, click Disable.

The Disable Application dialog box closes and the application state is changed to DISABLED.

To enable an application using the UI:

1. In the Applications screen, select the application you want to enable.2. Click Enable to display the Enable Application dialog box.

3. Click the Enable button to activate the application. The application starts or resumes operation and theapplication state is changed to ACTIVE.

Uninstalling an applicationThis procedure completely removes an application from the controller. To later restore the removed application,see Adding or upgrading an application on page 32.


Procedure

1. Use the following procedure to uninstall an application using the UI.2. In the Applications screen, select the application you want to uninstall.3. Click Uninstall.


4. Click the Uninstall button to remove (delete) the application.

Understanding application states and OSGi artifactsIn the default state, or when an application has been started, it is in the ACTIVE state and is servicing requests.Application states include the following:

Table 1: Application States

State Description

ACTIVE The application is running and servicing requests.

STAGED A new application has been downloaded to the controller and is ready to be installed.

UPGRADE_STAGED

A new version of an existing running application has been downloaded to the controller andthe new version is ready to be installed (upgrade/downgrade).

INSTALLING A transitive state indicating a new application is in the process of being installed.

UPGRADING A transitive state indicating the existing application is being stopped and a new version ofthe application is being installed.

CANCELING A transitive state indicating a non-installed version of an application is being deleted fromthe controller.

DISABLING A transitive state indicating the application is in the process of being disabled (stopping).

DISABLED The application is disabled (stopped). A disabled application is not automatically startedwhen the controller restarted.

ENABLING A transitive state indicating the application is being started.

UNINSTALLING A transitive state indication an application is being stopped and completely removed fromthe controller.

RESOLVED The application is stopped and not servicing requests. An application can only be in thisstate when it is stopped externally to the SDN Controller (e.g. the virgo console).


Table 2: Error condition management

State Description

NEW > STAGED NEW > UPGRADE-STAGED If an error condition occurs when staging theapplication, then it actually does not exist. (Errorconditions in this stage clean up after themselves.)

STAGED > ACTIVE If an OSGi deployment exception is encountered, theapplication is moved to DISABLED if it fails to deployas it is. If a File I/O or URI exception is encountered,the application remains in the installing state.

UPGRADE-STAGED > ACTIVE If an exception is encountered (OSGi deployment, FileI/O, or URI), rollback attempt is made, as listed below.(Depending on the original exception, not all optionsmay be possible).

1. Calls AppStore.deleteStore on the upgradedversion of the application.

2. Attempts to redeploy the original version of theapplication.

ANY STATE > UNINSTALLED If any exception is encountered, the applicationremains in UNINSTALLING state

ANY STATE > DISABLED If an exception is encountered, remains in DISABLINGstate.

DISABLED > ENABLED If an OSGi deployment exception is encountered, theapplication is moved to the DISABLED state if it fails todeploy as it is. If any other exception is encountered(file I/O or URI), the application remains in theENABLING state.

To access the link to the OSGi artifacts for an application, click on the bullet for the application in the web GUI.For example, clicking on the bullet for the Path Diagnostics application displays the link to identity of theassociated OSGi artifacts:

Figure 9: Links to OSGi artifacts associated with individual applications


Configuration componentsThe Configurations screen enables access to the configurable components in the controller which are used tomanage the controller and application features. Some examples of when you might want to make configurationchanges include:

• Specify an NTP server or date and time on the controller system using the NTP component or specify a staticIP address using the Network component.

• Specify hybrid mode for controlling packet forwarding by configuring the ControllerManager component.• Define how long to keep alerts and how to age out alerts by configuring the AlertManager component.• Define how audit log data ages out by configuring the AuditLogManager component.

Adding or removing an SDN application might add or remove additional configuration components. However,direct addition or removal of configuration components is not supported.

When controllers are operating in a team, configuration changes on one active controller propagateto the other active controllers in the team.

See also:

• Using configuration component keys on page 37• Modifying a component configuration• Modifying NTP server or date and time• Modifying Network settings• Modifying Logger settings

Using configuration component keysEach configuration component contains one or more component keys, each of which identify a configurableproperty of the component.

Information about each component key includes the current value, the default value, and a brief description.Where applicable, the range of suggested values is also included.

You can find information about each component key on the Configurations screen of the controller UI. Thecontroller Configs REST API is also available for configuring components, excluding Network and NTPcomponents, and the REST API for Logger configuration can only be done for each individual module (such ashp.sdn.event) not groups of modules.

Inappropriate changes to key values can result in severely degraded system performance. For thisreason, Hewlett Packard Enterprise strongly recommends that managing the default key values bedone only by experienced network administrators and programmers who have a strongunderstanding of SDN controller systems.

Configurations screen detailsOn the Configurations screen, the configuration components are accessed from the following four tabs:

• Basic provides access to tracing, topology discovery and flow priority configuration (see BasicConfigurations view).

• Advanced provides access to timing, performance tuning and debugging configuration (see AdvancedConfigurations view).

• System provides access to platform specific configuration (see System Configurations view).• Apps provides access to configuration components for installed SDN applications (see Apps Configurations

view).

The controls on these screens are the same.



Modify Select a component and then click Modify to open the ModifyConfiguration dialog box for the selected component.

expand icon Click to display a list of the configurable keys for the component. Thedisplay for each key includes the current value, the default value, anda brief description. Where applicable, the range of suggested values isalso included.

collapse icon Click to hide the key and value information for the component.

Component Lists the basic configuration components. The components aredescribed in the following sections.

Basic Configurations viewFigure 10: Basic Configurations view

Components in the Basic configurations tab:

• com.hp.sdn.ctl.of.impl.ControllerManager

The ControllerManager component provides parameters used in the implementation of the OpenFlow protocol.You can configure parameters such as hybrid.mode, keystore, keystore.password, truststore,truststore.password.

• com.hp.sdn.ctl.of.impl.TraceManager

The TraceManager controls OpenFlow trace duration.

◦ Use the record.duration key to specify how long a trace is to run after it starts.• com.hp.sdn.ctl.path.impl.PathDaemon

The PathDaemon component provides parameters used to perform L2 path calculations based on IPv4addresses for IPv4 packets or MAC addresses for ARP packets. You can set the following flow timeoutparameters:

◦ Use the idle.timeout key (default 60 seconds) to configure the idle timeout value for each flow-mod.The idle timeout value specifies how long the flow-mod will remain in the device if the flow-mod is notactively being used.

◦ Use the hard.timeout key (default 0, which implies infinite timeout) to configure the hard timeout valuefor each flow-mod. The hard timeout value specifies how long the flow-mod will remain in the datapath(regardless of usage).

• com.hp.sdn.disco.of.node.impl.OfArpDiscoveryComponent


The OpenFlow ARP discovery component of the OpenFlow Node Discovery application is used for topologyhost discovery via ARP protocol.

◦ Use the arp.age key to configure the node timeout values.◦ The listener.altitude key changes the altitude of the OfArpDiscoveryComponent component. For

more information, see Packet listeners on page 53.• com.hp.sdn.disco.of.node.impl.OfDhcpDiscoveryComponent

The OpenFlow DHCP discovery component of the OpenFlow Node Discovery application is used for topologyhost discovery via DHCP protocol.

◦ Use the dhcp.age key to configure the node timeout values.◦ The listener.altitude key changes the altitude of the OfDhcpDiscoveryComponent component.

For more information, see Packet listeners on page 53.• com.hp.sdn.disco.of.node.impl.OfIpDiscoveryComponent

The OpenFlow IP discovery component of the OpenFlow Node Discovery application is used for topology hostdiscovery via IP Protocol.

◦ Use the ip.age key to configure the node timeout values.◦ The learn.ip key indicates whether the controller should discover nodes from all IP packets it receives.◦ The listener.altitude key changes the altitude of the OfIpDiscoveryComponent component. See

Packet listeners on page 53.

Advanced Configurations viewFigure 11: Advanced Configurations view

Components in the Advanced Configurations view.

• com.hp.sdn.adm.mgr.impl.hpws.HpwsInstallManager

The HpwsInstallManager component provides a service for installing applications from the Hewlett PackardEnterprise SDN App Store, a remote web service.

• com.hp.sdn.api.impl.AlertPostManager

The AlertPostManager component uses the HTTP(s) protocol to send alert data as a JSON string to registeredalert topic listeners.

• com.hp.sdn.cms.impl.ClientMapperServiceProvider

The ClienMapperServiceProvider component provides information about a client by combining informationfrom Aruba ClearPass log on and log off events and location information from the controller about the switchand its port connected to the client.

• com.hp.sdn.ctl.diag.impl.PathDiagnosticComponent


The PathDiagnosticComponent provides the ability to send out a diagnostic packet on one switch and receiveit on the next. You can use it to trace a path for debugging link failures in your network.

• com.hp.sdn.disco.of.link.impl.OpenflowLinkDiscoveryComponent

The OpenflowLinkDiscoveryComponent transmits link discovery packets to the attached Openflow devices,listens to the responses, and populates the Link Service cache with the results.

◦ Use the age.multihop.links key to configure the OpenFlow Link Discovery application to removemultihop links from the link table if the link is not re-discovered in two poll intervals.

◦ Use the multihop.poll.interval key to configure the polling interval, in seconds, for multihop links.• com.hp.sdn.misc.AdminRestComponent

The AdminRestComponent provides parameters for internal communication between SDN components andthe Admin REST API of the controller.

• com.hp.sdn.misc.ServiceRestComponent

The ServiceRestComponent provides parameters for internal communication between SDN components andthe SDN controller Northbound REST API.

• com.hp.sdn.node.impl.NodeCacheComponent

The NodeCacheComponent component serves as an in memory cache of the nodes known to the controller. Itprovides add, update, remove, and get methods for its nodes.

◦ The cache.size key allows you to specify a maximum number of nodes that can be stored by theNodeManager. The default value is 20,000.

• com.hp.sdn.rs.RestPerfProvider

The RestPerfProvider component reports performance data for the REST API. You can configure theperf.profile key.

System Configurations viewFigure 12: System Configurations view

Components in the System Configurations view.

• NTP

Configure NTP server or set a specific date and time for the controller system. For details, see Modifying NTPserver or date and time.

• Network


Configure networking (Static IP address or DHCP) for the controller system. For details, see ModifyingNetwork settings for the eth0 interface.

• Loggers

Configure logging levels (ALL, TRACE, DEBUG, INFO, WARN, ERROR, OFF). For details, see ModifyingLogger settings.

• com.hp.sdn.adm.alert.impl.AlertManager

The AlertManager component controls the quantity of alert data present on the system by periodically checkingfor alert data to be deleted based on the configured age-out policy. For more information about alert logpolicies, see Configuring how alerts age out.

• com.hp.sdn.adm.auditlog.impl.AuditLogManager

The AuditLogManager component controls the quantity of audit log data present on the system by periodicallychecking for audit log data to be deleted based on the configured age-out policy. For more information aboutaudit log policies, see Configuring how audit log data ages out.

• com.hp.sdn.adm.auth.impl.AuthenticationManager

The AuthenticationManager component provides for the authentication of external users to the controller andbetween the controller and the Keystone server.

• com.hp.sdn.adm.health.impl.HealthManager

The HealthManager component is the Application/Component Health Monitor parameters.You must configurethe autoShutdown.properties exactly as it is done in the sample file. The possible health status arecritical, unhealthy, healthy, or hung.

• com.hp.sdn.adm.log.impl.LogManager

The LogManager component controls the number of log message rows displayed in the Support Logs display.For more information on support log queue size, see Configuring the support log queue size.

• com.hp.sdn.adm.metric.impl.MetricManagerComponent

The MetricManagerComponent determines how measurement data is maintained by the controller. Thecontroller includes a metering framework that internal components and installed applications can use to collectvarious types of data. (Data can be persisted on the controller from sources external to the controller.) Anymetric created with the framework might optionally be persisted over time or directed to the controller JMXfacility for viewing. Data persisted over time can be viewed using the controller REST API, while data sent toJMX can be viewed using JConsole or another JMX client. The MetricManagerComponent permits changingdefault values for certain aspects of the metering framework operation, such as how long the controller shouldretain persisted data, at what time of day persisted data that is too old should be trimmed, and how oftenpersisted metric values should be saved to disk. (This value can be overridden for any metric when the metricis created).

• com.hp.sdn.dvc.impl.DeviceManager

The DeviceManager component serves as an in memory cache for the persistent devices known to thecontroller. It holds information about those devices and whether they are currently connected to the controller.It provides add, update, remove, and get methods for its devices.

• com.hp.sdn.link.impl.LinkServiceComponent

The LinkServiceComponent controls the Link Manager service, which serves as an in memory cache of thelinks known to the controller. It provides add, update, remove, and get methods for its links.

◦ The cache.size configurable parameter allows you to specify a maximum number of links that can bestored by the Link Manager. Default is 20,000.

• com.hp.teaming.imple.CassandraProcessManager

The CassandraProcessManager component controls configuration parameters of the Cassandra database.• com.hp.sdn.teaming.impl.TeamConfigurationManager

The TeamConfigurationManager component manages the configuration of team communication. When one ofthe components’s keys is modified, the administrator must wait for the new value to be forwarded to the other


members of the team (this can be confirmed by making sure the change appears in the UI of each controller),and then ALL controllers must be restarted.

Apps Configurations viewFigure 13: Apps Configurations view

If you have other SDN applications installed, configurable components for these applications are listed in theApps Configurations view. For example in the screen shown above thecom.hp.mvisor.adm.topo.impl.NetworkVisualizerTopologyManager component for the Aruba NetworkVisualizer SDN Application is listed in the Apps Configurations view.

For details on configuring these SDN application specific components see the documentation for the SDNapplication.

Modifying a component configuration

Procedure

1. On the Configurations screen, select the tab that contains the component you want to modify (Basic,Advanced, System, or Apps).

2. Select the component you want to modify.3. Click Modify.

A Modify Configuration dialog box is displayed for the component you selected. For example:

4. Enter new values for each of the keys you want to modify.5. Do one of the following:


a. To save your changes and close the dialog box, click Apply.b. To close the dialog box without saving changes, click Cancel.

Modifying NTP server or date and timeBest practices

• HPE recommends that you use an NTP server rather than setting date and time because if you changenetwork settings, the date/time will be reset to current date/time.

• If the controller is in a team and you want to change the NTP server or date and time, the other two controllersin the team must have the same NTP server or date and time. You can make the change on an individualcontroller but will see a message reminding you to check that the NTP server or date and time is the same onthe other controllers in the team.

Modify NTP server or date and time

Procedure

1. On the Configurations screen in the System tab, select the NTP component.You can use the expand icon to view the NTP information currently configured

2. Click Modify.The Modify System Configuration dialog box is displayed for the NTP component. For example:

3. Select either NTP Server or Date/Time and make the following configuration changes:a. Select NTP Server to configure an NTP server for use by the controller system. Enter either the server IP

address or server name. You may only enter one server.b. Select Date/Time to configure the date and time to set for the controller system and click in the Select a

Date field.

The calendar view is displayed. You can select Now to use the current date and time or you can select adate on the calendar and enter the time in hours and minutes. Then click Done.

c. Click Apply to save your changes.d. To close the dialog box without saving changes, click Cancel.


If you clicked Apply, a confirmation window is displayed showing a message saying that you will be loggedout of the UI and will need to log back in for authentication. If the controller is in a team, the message willalso remind you to check that all controllers in the team have the same NTP server or date and time. ClickYes in the confirmation window to save your changes.

Modifying Network settings for the eth0 interface

Prerequisites

If the controller is in a team you must first disband the team before modifying the network settings. You canconfigure Hostname, IP Address and Type (Static or DHCP) of network connection. The configuration is for theeth0 interface only.

Modify network settings

Procedure

1. On the Configurations screen in the System tab, select Network.

You can use the expand icon to view the network information currently configured.2. Click Modify.

The Modify System Configuration dialog box is displayed for the Network component. For example:

3. Enter new values for Host Name, IP Address, Type, and other fields as required.

No spaces are allowed in the Host Name field. If the controller is in a team, you must first disband the teambefore modifying the network settings. If you are configuring a static IP address, you must enter values forGateway, Netmask, and Primary DNS fields; the Secondary DNS field is optional.

4. Click Apply to save your changes.

To close the dialog box without saving changes, click Cancel.

If you clicked Apply, a confirmation window is displayed showing a message that the controller automaticallyreboots.

5. Click Yes in the confirmation window to save your changes.


After applying the change, you will be disconnected from the UI and will need to wait for the controller to restartbefore logging back in.

Modifying Logger settingsFor troubleshooting you may want to increase the logging level to generate more information in the log file for usein debugging a problem.

Procedure

1. On the Configurations screen in the System tab, select Loggers.

You can use the expand icon to view the logging levels currently configured.2. Click Modify.

The Modify System Configuration dialog box is displayed for the Loggers component. For configurationpurposes the loggers are grouped into categories listed as keys you can modify. For example:

You can configure the logging level for each of the logger keys listed.

The log levels from most verbose to least verbose are: ALL, TRACE, DEBUG, INFO, WARN, ERROR, OFF. Ifthe controller is restarted or if the virtual machine is rebooted, the log levels for all loggers revert back to INFO.

Setting all loggers to a high verbose level of logging is not recommended because it can lead to ashortage of system storage space very quickly.

3. Click Apply to save your changes.

To close the dialog box without saving changes, click Cancel.

Audit log


About the audit logThe audit log is available through both the controller GUI and the REST API, and records events related toactivities, operations, and configuration changes initiated by an authorized user. This includes activities such as:

• Installing an application (or starting, stopping, uninstalling an application)• Modifying the configuration of a controller component• Installing a license• Forming a controller team

When controllers are operating in a team, the audit log shows events for all controllers in the team.

See also:

• Deleting an audit log entry on page 47• Configuring how audit log data ages out• Exporting and archiving audit log data on page 48

Audit log screen detailsFigure 14: Viewing the Audit Log


Refresh Updates the log entries displayed on the screen. The controller does not updatethe display as new entries are generated. Use this action to refresh the display.

User The user that performed the operation that triggered the log entry

Occurred A time stamp (in UTC format) indicating when the controller created the log entry.

Activity The type of activity that triggered the creation of the log entry.

Data Detailed information about the log entry.

Table Continued



Origin The application or controller component that generated the log entry.

Controller ID A hexadecimal number that identifies controller that generated the log entry.When you use controller teaming, this ID enables you to identify which controllerin the team generated the alert.

Deleting an audit log entryYou cannot delete or modify a log entry. The controller deletes entries according to the configured audit logpolicies. To configure the audit log policies, see Configuring how audit log data ages out.

Configuring how audit log data ages outYou can configure the following key values for the audit log to control how audit log data ages out. To set thesekey values, configure the com.hp.sdn.adm.auditlog.impl.AuditLogManager component using theConfigurations screen.

Key DefaultValue

Description

trim.auditlog.age 365 Specifies the number of days to retain a log entry. Use this key toimplement your record retention policy.

Data type A number from 31 through 1825.

trim.enabled true true Specifies that the controller deletes log entries that haveexceeded the trim.auditlog.age limit.

false Specifies that the controller does not delete log entries thathave exceeded the trim.auditlog.age limit.

trim.frequency 24 Specifies how often, in hours, the controller is to delete log entries thathave exceeded the trim.alert.age limit.


Example Enter 24 to specify that the controller delete aged-out logentries every 24 hours (once per day).

To configure how audit log data ages out:

Procedure

1. On the Configurations screen in the System tab, select thecom.hp.sdn.adm.auditlog.impl.AuditLogManager component.

2. Click Modify.

The Modify System Configuration dialog box is displayed for thecom.hp.sdn.adm.auditlog.impl.AuditLogManager component.


3. Change the values for the keys (these keys are described in the table above).4. Click Apply .

Figure 15: The AuditLogManager Configuration Component Controls Audit Log Policy

Exporting and archiving audit log dataTo retain log records for longer than the trim.auditlog.age limit, you must export the audit log from thecontroller to a file before the trim.auditlog.age limit is reached. Exporting audit log data does not remove itfrom persistent storage.

To export the audit log, you must use the REST APIs since this action cannot be performed in the UI.

For example, you can use the curl command at Export audit log data as a CSV file using curl commands onpage 183.

LicensesA license is required for the controller. In addition, SDN applications can require licenses that are separate fromthe license for the controller. For information on installing, activating, uninstalling or transferring licenses, see License Registration and Activation on page 85.

Licenses screen detailsThe Licenses screen displays the controller Install ID, and is used to activate new licenses, and deactivateinstalled licenses (for transfer to another installation). Licenses screen example:



Refresh Updates the screen with the latest license information.

Add Adds and activates the specified license key on this controller.

Deactivate Deactivates the selected license.

Copy Uninstall Key When a license is deactivated, an uninstall key is assigned for license transferpurposes and you can copy this uninstall key by selecting this button, see Transferring licenses on page 94.

Install ID Contains the installation identifier for this controller.

Serial# A sequence/serial/record number given for that license across all licensesgenerated for that install ID. The HPE My Networking Portal assigns the serialnumber while generating license records.

Product Name of the application or product for which the license is generated.

Licensed For License metric name. For example, nodes and HA nodes.

Qty Quantity of the Licensed For metric based license.

Type Type of license. For example, PRODUCTION, DEMO, or EVAL.

Status ACTIVE, EXPIRED, or DEACTIVATED

Expire By Date and time when the license Licensed For expires.

Uninstall Key When a license is deactivated, an uninstall key is assigned for license transferpurposes, see Transferring licenses on page 94.

Installing, activating, uninstalling, or transferring licensesFor information about installing, activating, uninstalling, and transferring licenses, see License Registration andActivation on page 85.

TeamThe Team screen displays team and region configuration information including:

• Team status (top banner)• Team configuration and controller status (top section)• Region configuration (middle section)• Device owners (bottom section)


For details on viewing information on the Team screen, see Viewing your team configuration using the UI. Fordetails on configuring High Availability (HA) and teaming, see Configuring for High Availability on page 101.

Figure 16: Team Screen

Support logs

About support logsThe support logs maintain an internal record of events of interest from the operations of an active SDN controller.This information is the type of data a support engineer would request when troubleshooting an SDN installation.

The log file is configured to have a maximum size of 10 MB and to keep 4 previous versions. The log.log file isthe primary log for controller information. With 4 full previous versions and the one active log, the core controllerlogging will consume at most 50 MB of disk space. The controller allows up to five support logs; one active andfour in storage:

• Support logs are stored in the controller /var/log/sdn/virgo/logs directory.• When the current log reaches 10 MB, the controller copies the log to storage and starts a new log.• When the log file has rolled over four times, the controller purges the oldest log file when it needs to roll over

again. The core controller has at most 5 log files.• Support logs can be exported to a file.

In a controller team environment:

• Each controller maintains its own support logs.• Changing the support log queue size on any controller propagates to all active controllers in the team.• The Export action gathers the set of support log file data from all active controllers in the team, and stores the

data as a single compressed archive.

See also:

• Configuring the support log queue size• Exporting the support logs on page 53


Support logs screen detailsFigure 17: Selecting the Support Logs screen


Refresh Displays a listing of the most recent log messages, as determined by thecurrently configured queue size. For example, with a queue size of 100, Refreshlists the 100 most recent log messages.

Export Gathers the set of support log file data from the standalone controller or all activecontrollers in the team, and stores the data as a single compressed archive.



Level The severity level for the entry. The logging levels are hierarchical. Messagesare logged with the lowest logging level and above. The lowest level is TRACE,which results in all messages being logged for the selected logger when theTRACE logging level is specified. INFO is the logging level In the defaultconfiguration. Severity levels are:

Value Description

ERROR Indicates a problem to investigate. The problem could causefunctional or performance issues with application.

WARN Indicates a problem that you might want to investigate. Theproblem could be an early indication of issues that could latercause an error.

INFO Indicates a normal operational event that requires no action.

DEBUG Indicates an informational event that is most useful fordebugging applications.

TRACE Indicates an informational event that is most useful fordebugging applications. Often used to show programexecution details when DEBUG-level events do not provideenough information

Using the Virgo Administrator console, you can dynamically change the logginglevel for a component that is writing to the support log. For example, you canenable the DEBUG level logging for just the NodeManager configurationcomponent.You can also dynamically change the logging level by using theREST API. See Aruba VAN SDN Controller REST API Reference

Logger The module or feature that triggered the logging condition.

Thread The thread that caused the logging condition to occur.

Message Describes the details of the logging condition.

Data Detailed information about the log entry.

Controller ID A hexadecimal number that identifies controller that generated the log entry.When you use controller teaming, this ID enables you to identify which controllerin the team generated the alert.

Configuring the support log queue sizeThe default queue size is 100 lines.

Procedure

1. On the Configurations screen in the System tab, select the com.hp.sdn.adm.log.impl.LogManagercomponent.

2. Click Modify.


The Modify System Configuration dialog box is displayed for thecom.hp.sdn.adm.log.impl.LogManager component.

3. Change the value for the max.display.rows key.4. Click Apply.

Figure 18: Configurations screen with LogManager component keys

Exporting the support logsThe Export operation:

1. Gathers the set of support log file data from the controller, or in a team environment, all active controllers in theteam, and stores the data as a single compressed archive file:sdn-all-logs.zip

2. Downloads the archive file from the controller to the default download directory specified by your browser. Forexample, in Ubuntu installations, this is usually the Downloads directory.

3. Click Export. The following menu appears in the lower-left corner of the controller console:

Figure 19: Completion of the export operation

4. When the download completes, you can either resume interaction with the controller or examine the log byselecting an item from the menu, such as:

• Open a window showing the new log zip file.• Set the default operation to always open the directory containing the log zip file.• Show the log zip file in the default directory for receiving downloads.

The actions resulting from these choices depend on the browser and operating system, not on thecontroller.

Packet listenersThe controller applications (and SDN applications) register packet listeners with the controller. The order ofprocessing an incoming packet is determined by the roles (Advisor, then Director, then Observer), and thenaltitudes within a role (in decreasing value, with 0 the lowest altitude). An incoming packet (PacketListenerRole) is


wrapped in a Message Context (which also holds a Packet-Out reply) which is passed to each packet listener inturn.

Packet listeners display detailsThe packet listeners screen displays the packet listeners that are currently running on the controller.

Figure 20: Selecting the Packet listeners screen


Refresh Refreshes the information on the screen.

PacketListener Role The PacketListener Role is one of the following:ADVISORExamines the incomingpacket. Might add processing hints to the message context, but does not modifythe packet out message.DIRECTORProcesses the packet. Might add actions orinstructions to the packet-out message. Can instruct the controller to block thepacket, or to send the packet out.OBSERVERA passive observer who mightexamine the incoming packet and any packet-out response.Packets are given topacket listeners with role of ADVISOR first, DIRECTOR second, and OBSERVERthird. Every packet listener is guaranteed to see the packet-in message.Depending on the action taken by higher altitude Directors, a lower altitudeDirector might be too late to influence the packet processing.

Altitude The weight or priority this packet listener should have relative to other packetlisteners that have the same role. The controller gives packet listeners withhigher numbers priority over packet listeners with lower numbers.

Average (ms) The average time, in milliseconds, that the packet listener spent processing apacket.

# Samples The number of packets processed by that packet listener since the packetlistener registered.

OpenFlow MonitorThe SDN controller UI includes several screens providing information on OpenFlow enabled switches:

• OpenFlow Monitor on page 54• OpenFlow topology on page 58• OpenFlow Trace log on page 67• OpenFlow Classes on page 74


When the controller is active in an OpenFlow domain, the OpenFlow Monitor enables tracking of switch trafficsummaries, packet traffic per port, and applied flow rules for switches detected in the controller domain.

For a graphical view of Data Path ID assignments to individual OpenFlow switches, see OpenFlow topology onpage 58.

OpenFlow Monitor screen detailsThe main display lists the Data Path IDs and descriptive information for the active switches and the options forviewing traffic information. To view information about a specific device, click the Data Path ID for that device andthen select one of these tabs for the view you want to display: Summary, Ports, Flows, Groups.

Click Refresh to update the display for Topology changes, such as a newly discovered OpenFlow device or theloss of a device that has been disconnected.

Figure 21: The Main OpenFlow Monitor screen


Refresh Updates the information displayed on the screen.

Summary

tabDisplays the Summary for data path view for the selected data path.

Ports

tabDisplays the Ports for data path view for the selected data path.

Flows

tabDisplays the Flows for data path view for the selected data path.

Groups

tabDisplays the Groups for data path view for the selected data path.

Data Path ID Identifies a detected OpenFlow switch. The OpenFlow data path identification foreach detected OpenFlow switch. This ID also appears in the representation ofthe switch in the OpenFlow Topology screen.

Address Identifies the IP address associated with an OpenFlow data path instance.

Negotiated Version The version of OpenFlow in use with the corresponding data path.

Table Continued



Manufacturer Manufacturer of the device.

H/W Version Hardware version of the device.

S/W Version Software version on the device.

Serial Number Serial number on the device.

Summary for data path viewFigure 22: Summary view for a specific OpenFlow device

The OpenFlow Monitor > Summary view includes the following details related to the selected device:

• Manufacturer• Hardware and software version• Serial number and device description of the device• Device identification (Data Path ID) and IP address• TCP port on the device• Negotiated OpenFlow version (latest OpenFlow version common to both the controller and the switch)• OpenFlow table and buffer information• OpenFlow capabilities on the device

Ports for data path viewThe OpenFlow Monitor > Ports view includes information on the ports used for OpenFlow traffic on the selecteddevice.

Figure 23: Ports view for a specific OpenFlow device


Flows for data path viewThe OpenFlow Monitor > Flows view shows current flows on the selected OpenFlow device. For a given flow,traffic meeting the requirements specified in the "Matches" field is directed as specified in the corresponding"Actions/Instructions" field.

Beginning with version 2.8 of the Aruba VAN SDN Controller, the VAN OpenFlow flow table presentation has beenenhanced to group a datapath's flows into expandable rows for each flow table ID. Each flow table ID row displaysa Flow Count and Table Name, and can be expanded to view detailed information for the flows installed to thattable ID. Additionally, each individual flow can be further expanded to view detailed information for that flow. TheOpenFlow flow table for a datapath can be viewed by navigating to OpenFlow Monitor, selecting a specificdatapath, then selecting Flows.

Figure 24: Flows view for a specific OpenFlow device

The Table ID applies to OpenFlow 1.3 and later, but not to OpenFlow 1.0.


Groups for data path viewThe OpenFlow Monitor > Groups view provides information on group actions, if any, defined for the device. Thegroup actions can assign more specific forwarding actions.

Figure 25: Groups view for a specific OpenFlow device

OpenFlow topologyThe OpenFlow Topology screen displays a topology of discovered switches and end nodes in the controllerdomain. You can view and change the graphical view of the network, as well as compute the broadcast tree toavoid loops and broadcast storms. The shortest path is computed using a Dijkstra graph search algorithm. TheOpenFlow topology screen:

• Displays a topology of discovered switches and end nodes.• Identifies the ports discovered on a given switch. Interface name and OpenFlow numbers are displayed• Identifies the shortest path between two nodes.• Provides node identification options (such as MAC or IP address label).• Provides a view of switch port identifiers, active flow rules, and a tool for testing flow rule options.

Beginning with version 2.8 of the Aruba VAN SDN Controller, the VAN OpenFlow Topology page defaults to"Collapse All", which means that end-host nodes will be "collapsed" into the OpenFlow device to which they areattached. "Collapse All" can be toggled using the "Collapse All" option on the View pull-down menu (located at thetop of the OpenFlow Topology page), or using the 'c' hotkey. In previous VAN releases, the OpenFlow Topologypage defaulted to "non-collapsed".

Also beginning with version 2.8 of the Aruba VAN SDN Controller, the VAN OpenFlow Topology page auto-refreshtimer has been extended to 5 minutes. Auto-refresh can be enabled or disabled using the Auto Refresh option onthe View pull-down menu (located at the top of the OpenFlow Topology page). Regardless of the auto-refreshsetting, the topology can always be refreshed using the Reload button.

In a topology where two or more controlled switches connect to the same uncontrolled switch, thecontroller will not learn the location of hosts directly connected to the uncontrolled switch.

Do not configure a looped topology in the network between the OpenFlow and non-OpenFlowportions of your network unless you enable Spanning Tree Protocol on the non OpenFlow devicesoperating in the network.

See also:


• Displaying the network Topology on page 59• Using keyboard shortcuts to change the display on page 59• Changing the topology display using the View menu• Viewing the shortest path between two nodes• Viewing flow details for selected nodes on page 66• Viewing details on packet selection criteria for a data flow on page 67

Displaying the network TopologyThe OpenFlow Topology screen includes the switches and end-nodes in the controller domain.

Figure 26: Topology viewer

The topology legend is show in the top right corner:

• Switch — shown in light green• Collapsed Switch — shown in dark green• End Host — shown in orange

Using keyboard shortcuts to change the display

Use the question mark


To use the keyboard shortcut keys you must first click somewhere in the topology view to bring it into focus andthen you can select a shortcut key. An outline around the topology indicates it is in focus.

Figure 27: Keyboard shortcuts

See also:

• Changing switch and host node labeling• Using the mouse to change the topology display on page 61• Viewing node tooltips

Changing switch and host node labelingYou can change how nodes are labeled in the topology using keyboard shortcuts.

To turn on or off ALL node labels, enter the keyboard shortcut L.

To change the host node labeling in the topology, enter the keyboard shortcut H and the display will cycle throughthe different node labels each time you enter H.

Host end-nodes can be labelled with one of the following:

• IP Address(default)

• MAC Address• No Label


To change the switch node labeling in the topology, enter the keyboard shortcut N and the display will cyclethrough the different switch labels each time you enter N.

Switches can be labelled with one of the following:

• System name(default, if the switch does not contain a system name then IP address is shown instead)

• IP address• DatapathId• No label

For example, to change the default display showing System name labels to show the IP addresses of the switchnodes, click anywhere in the topology display, then press N. The switch IP addresses appear as labels in thetopology diagram:

Figure 28: Switch IP address labeling

Press N again to display the switch datapath IDs as labels in the topology diagram:

Figure 29: Switch datapath IDs as labels

Press N again to display the unlabeled switch view.

And press N again to return to the System Name switch labels.

Using the mouse to change the topology displayZoom in or out in the topology by using the scroll wheel on the mouse.

To drag the topology to a desired location, place the cursor in the topology and hold the mouse button down whiledragging to move the topology.

Select or deselect a switch or end-node host by clicking the node.


Viewing node tooltipsYou can view node tooltips by hovering the mouse-over a node in the topology. Or you can press O to toggle onand off tooltips.

Mouse-over the switch to display datapath information.

Mouse over the host to display end-node information.


Changing the topology display using the View menuYou can use the View menu to change the topology display.

Figure 30: Topology View Menu

See also:

• Using Search• Viewing port labels on switches• Viewing details on page 65• Using tools on page 65• Using pin, Collapse All, Auto Refresh and Reload on page 65

Using SearchYou can search the topology based on various criteria by using one of the following methods:

• Search using View > Search. Or press the F shortcut key to open the Search dialog box. This search is basedon any one of the criteria Switch IP, Datapath ID, Host IP or Host MAC.

• Enter the search criteria in the Search (regex) box and click Search. This search is across all of the textincluding Switch IP address, Datapath ID, System name, End host IP address and MAC address.

Search using the Search menu:

Procedure

1. Select View > Search. Or press the F shortcut key to open the Search dialog box.

The Search dialog box is displayed. When the Search dialog box is opened, if one or more nodes arecollapsed or highlight path is enabled, all will be cleared during the search. After the search dialog is closed,the state of collapse and highlighting will be returned.

2. From the drop down list, select one of the search criteria Switch IP, Datapath ID, Host IP, or Host MAC, andthen enter the search value.


3. Click Search. If any match is found, the border changes to green. If no match is found, the border changes tored.

4. To close the dialog box, click Close or click Reset to clear the search value and reset the topology view.

Search using Search (regex):

1. Enter the value you want to search on in the Search(regex) field located in the top right of the topology view.You can enter a regular expression for more complex searches.

For an exact match, $ should be appended at the end. For example, if there are IP addresses like 10.10.10.10,10.10.10.100, 10.10.10.101, 10.10.10.102, etc. and if you want to search for only 10.10.10.10, you should putthe search string as 10.10.10.10$.

2. Click Search or press Enter.

Viewing port labels on switchesYou can view port labels on the links between switches and between switches and end nodes. Port labels can beinterface name or OpenFlow numbers.

Select View > Ports to display port labels on switches. Press the P shortcut key to toggle between displayingOpenFlow port number or port name. Toggling only works when the Ports menu option is selected.


Viewing detailsYou can view details for a switch by selecting View > Details. For more information, see Viewing flow details forselected nodes on page 66.

Using toolsAfter specifying a source and destination data flow you can view details on the packet selection criteria byselecting View > Tools. For more information, see Viewing details on packet selection criteria for a data flowon page 67.

Using pin, Collapse All, Auto Refresh and ReloadPin

To pin or unpin the switches and end nodes, press X or select View > Pin All.

When you enable Pin All nodes and Auto Refresh, if any topology updates occur (such as an end host moved orwas removed), then Pin All is automatically removed to update the topology and then once the topology isupdated, the topology is pinned back. However if you had any customized view then that may be changed duringthat time. If the number of nodes increases by more than 500 (both switch and end hosts), Pin All will be enabledand disabled automatically. Once node count comes down to less than 500, Pin All will be re-enabled.

Collapse All

Collapse the topology display to show only the number of end nodes connected to each switch, instead ofshowing all end nodes (the default) which can present a cluttered display where a large number of end nodes areconnected to the OpenFlow switches. To collapse or expand end-nodes for a particular switch, double-click theselected switch.

To collapse all end nodes, select View > Collapse All.

Auto Refresh

To automatically refresh the topology, select View > Auto Refresh.

Reload

To reload the whole topology, click the Reload button in the top right of the topology view. When the topology isreloaded, Highlight, collapse, Collapse All, selection, and node labels will be reset.

Viewing the shortest path between two nodesYou can view the shortest path between two nodes as follows:

Procedure

1. Select the source node and click Src or press S.2. Select the destination node and click Dst or press D.

The controller displays the path between the two nodes as a line, see Viewing the shortest path between twonodes).


Features like Collapse all, collapse a single node and highlight a particular node using Ctrl click are not allowedwhen a path is selected.

Figure 31: Locating the shortest path between two nodes

To exchange source and destination nodes, press A.

To clear the source and destination flags as well as clearing the path, press Z.

Follow FlowThe Follow Flow option is enabled only when the controller is in pure OpenFow mode, where hybrid.mode isset to false. The Follow Flow option is disabled when the controller is in hybrid mode.

When the controller is in pure OpenFlow mode, select Path > Follow Flow.

Highlight flowThe Highlight option is enabled only when a path is selected (either Shortest Path or in Follow Flow mode).Highlight path is cleared when you toggle between Shortest Path or Follow Flow. For example, with Shortest Pathis enabled you select Highlight path, then you select Follow Flow, the Highlight path will be cleared and you haveto select Highlight path again for Follow Flow.

Select Path > Highlight.

Viewing flow details for selected nodesThe Switch Details window displays flow details.


Select a switch node and then select View > Details or press I to display the Switch Details screen.

Figure 32: Flow details for the selected source-destination end nodes

Viewing details on packet selection criteria for a data flowFor a source-destination data flow you can view details on the packet selection criteria used.

Select View > Tools to display the Packet Selection dialog box or press T. The display is read only.

The Abstract Packet window displays selection criteria for packets moving between the Source-Destination nodepair. MAC addresses and IP addresses are shown based on the source and destination nodes selected.

Figure 33: Searching for flows for specific packet types

OpenFlow Trace logThis troubleshooting tool logs OpenFlow conversations captured in messages to and from the controller and theOpenFlow devices it manages.

You can export the captured messages in the trace log to a CSV (Comma-Separated Values) file that can beopened by applications such as Excel that are designed to accommodate this file type. This enables you to createa filter to display only the messages from the specific data paths you want to examine.


About the OpenFlow Trace logThe number of events that can be held in the trace log is limited by system memory. For this reason, HewlettPackard Enterprise recommends that you export to a remote storage location any trace log content you want toretain, and to clear the controller trace log whenever its content is not needed on the controller itself.

See also:

• Starting, stopping, or clearing OpenFlow trace on page 69• Displaying trace event details• Exporting the OpenFlow Trace log on page 70• Filtering the OpenFlow trace log in a CSV file• Changing the OpenFlow trace interval

OpenFlow Trace screen detailsFigure 34: Example of OpenFlow Trace Default Display


Start trace icon Starts trace logging. In the default configuration, the trace stops after ten seconds havepassed. (To change the trace interval, see Changing the OpenFlow trace interval.)

Stop trace icon Stops trace logging before the end of the configured trace interval.Trace logging stopsautomatically at the end of the configured trace interval.Multiple consecutive traces canbe held in the trace log. To add additional trace results, start another trace.

Clear trace screen

icon

Clears (resets) the current trace log. To preserve the contents of the trace log beforeclearing it, see Exporting the OpenFlow Trace log on page 70.

magnifying glass icon Displays details of the selected trace event.

Table Continued



Export Copies the trace log into a CSV (comma-separated values) file. See see Exporting theOpenFlow Trace log on page 70.

Time The time the message event was generated.

Event The event type. For example:

CkPt Indicates a check point in the trace log, such as the starting or stopping of atrace operation.

Rx Indicates an OpenFlow message received by the controller (from a datapath).

Tx Indicates an OpenFlow message sent from the controller (to a datapath).

Data Path ID The Data Path ID of the data path associated with the event.

Message The trace message.

Starting, stopping, or clearing OpenFlow traceUse the buttons above the Time field to control trace operations (see OpenFlow Trace screen details).

Displaying trace event details

Procedure

1. Select the event you want to examine.

Figure 35: Selecting an event in the OpenFlow Trace log

2. Click


The Event Detail dialog box is displayed

Figure 36: Displaying event details

3. To close the Event Detail window, click Close.

Exporting the OpenFlow Trace logExporting an OpenFlow Trace Log places the trace content in a CSV file that is stored in the default downloadsfolder specified in your web browser settings.

For more information about CSV files, see RFC 4180.

This section shows how to export and access OpenFlow Trace Log files using Google Chrome. Youmight experience different results than shown here, depending on your web browser and itsconfiguration.

Procedure

1. Click Export. This action places the trace log contents into a CSV file in the default downloads folder in thesystem on which the controller is running. Check your web browser for an indication that the file has beencreated.

2. To display and filter the CSV file content, see Filtering the OpenFlow trace log in a CSV file.


Filtering the OpenFlow trace log in a CSV file

Procedure

1. Open the CSV file in the default folder. For example, using Google Chrome, open the menu adjacent to the filename (of-trace.csv) and select Show in folder.

Figure 37: Accessing the stored CSV file

2. In the resulting folder listing, locate the of-trace.csv file and open it using an application, such as MicrosoftExcel, that enables you to read the log messages and configure a filter. For example, to investigate themessages collected for data path 00.00.00.00.00.00.00.02:

3. Select the DPID (Data Path ID) column.

Figure 38: DPID column


4. Set the filter.

Figure 39: Setting the filter

5. Apply the filter by checking the box for data path 00.00.00.00.00.00.00.02.

Figure 40: Applying the filter


In the resulting display, only the data filtered to data path 00:00:00:00:00:00:00:02 appears.

Figure 41: Filtered trace log

Changing the OpenFlow trace intervalThe default trace interval is ten seconds.

Procedure

1. From the navigation menu, select Configurations. Then select the Basic tab.2. Select the com.hp.sdn.ctl.of.impl.TraceManager component.3. Click Modify.

The Modify Basic Configuration dialog box is displayed for thecom.hp.sdn.ctl.of.impl.TraceManager component.

4. Change the value for the record.duration key.5. Click Apply to set the new time span for active trace recording, and return to the OpenFlow Trace screen.


OpenFlow ClassesThe OpenFlow Classes screen shows the OpenFlow classes that applications have registered with the controller.

About OpenFlow classesWhen multiple applications share the same resource the flow tables of OpenFlow switches how can their prioritiesrelative to each other be determined and how can their actions be coordinated? If flow table modification prioritiesare directly coded into each application, applications can end up directly competing with other applications for thehighest priorities, which can result in conflicts in general network traffic control and unintended results when youimplement a solution that has multiple SDN applications attempting to act on the same packets. In addition, manyenvironments make it difficult to trace the origin of flow modification requests installed in switches.

The Aruba VAN SDN Controller uses OpenFlow classes to dynamically manage the priorities of the OpenFlowrules being deployed to the network, thus enabling applications to execute their business logic in a more orderlyfashion.

1. For each class of flow modification message the application can send, the application must register anOpenFlow class with the controller. The OpenFlow class must specify the types of match fields, types ofactions, and (optionally) the relative position (higher than or lower than) for this class with respect to other flowclasses.

2. The controller adds a unique base cookie to be used with each future flow modification to be validated againstthis OpenFlow class, and assigns an actual priority for the OpenFlow class. This actual priority is based on thelogical priorities of all of the OpenFlow classes of all the applications that are registered with the controller.

3. When the application sends a flow modification message, it must set the match and action to be the samefields as specified in the OpenFlow class and, instead of providing an actual priority, the application sets thelogical priority as assigned by the flow class, and a cookie that is derived from the base cookie of theOpenFlow class.

4. Before sending the flow table modification message to the switch, the controller evaluates the requested flowmodification against the registered OpenFlow classes and replaces the logical priority provided by theapplication with an actual priority.

In addition to enabling the controller to manage priorities for multiple applications, OpenFlow classes enable thecontroller to validate flow modifications an application makes against a set of expected flow modification requests.This capability means that the behavior of an application must match the intent that the application disclosedwhen it registered with the controller:

• The flow match must contain exactly the fields and field types that were disclosed when the applicationregistered with the controller. The controller validates field types but not field value.

• The action or instruction must fall into the category that was disclosed during registration.An action is classifiedinto one of the following categories:

FORWARDDROPPROCESSSTEALCOPY

• The upper 16 bits of the flow modification cookie must match the upper 16 bits of the base cookie that wasissued during registration.


OpenFlow classes screen detailsThe OpenFlow Classes screen displays the OpenFlow classes that are currently registered with the controller.

Figure 42: Example of OpenFlow classes screen


Refresh Refreshes the list.

Flow Class ID The symbolic name for the flow class. The prefix identifies the application thatregistered the class; the suffix uniquely identifies the class.

Priority The actual priority the controller assigns to flows of this class.

Cookie The base value of the cookie assigned to this OpenFlow class. The applicationthat registered this class must use this base cookie when constructing flows thatbelong to this class.

Match Fields The types of match fields that are expected to be specified in flows that belong tothis class.

Actions The general category of the action or instruction a flow that belongs to this classis expected to include. For a list of categories, see About OpenFlow classes onpage 74.

Description Short description of what the OpenFlow class does. The application describesthe OpenFlow class when it registers the class with the controller.

Controller enforcement levels for OpenFlow classesThe following table lists the enforcement levels that the controller can use for applications that send flows toswitches.


Enforcement level Description

none The controller does not manage flow modification priorities or validate flowmodification requests:

• Applications that do not register OpenFlow classes with the controller arepermitted to send flow modifications to switches.

• The controller does not validate flow modifications, even for applications thatregister OpenFlow classes with the controller.

• The controller does not replace logical priorities with actual priorities for flowmodification requests from any applications.

weak (Default) The controller manages flow modification priorities and validates flowmodification requests for applications that register OpenFlow classes:

• Applications that do not register OpenFlow classes with the controller arepermitted to send flow modifications to switches.

• The controller validates flow modifications from registered applications against theOpenFlow classes that are registered.

• The controller replaces logical priorities with actual priorities for registeredapplications only.

strict The controller manages all flow modification priorities and validates all flowmodification requests:

• Applications that do not register OpenFlow classes with the controller are notpermitted to send flow modifications to switches.

• The controller validates all flow modifications against the OpenFlow classes thatare registered.

• The controller replaces logical priorities with actual priorities for all applications.

Changing the enforcement levels for OpenFlow classesTo change the enforcement level the controller applies to applications sending flows to switches, change the valuefor the flow.mod.enforcement key of the com.hp.sdn.ctl.of.impl.ControllerManager component.

Procedure

1. From the navigation menu, select Configurations. Then select the Basic tab.2. Select the com.hp.sdn.ctl.of.impl.ControllerManager component.3. Click Modify.4. The Modify Basic Configuration dialog box is displayed for the

com.hp.sdn.ctl.of.impl.ControllerManager component.5. Change the value for the flow.mod.enforcement key.6. Click Apply.


For information about the enforcement levels the controller can apply, see Controller enforcement levels forOpenFlow classes on page 75.

Figure 43: Configurations screen with Controller Manager component keys


OverviewThe hybrid mode setting determines which packet-forwarding decisions are made by controlled OpenFlowswitches and which of these decisions are made by the controller itself.

• If hybrid mode is enabled (the default setting), the controller delegates normal packet forwarding to thecontrolled switches, but overrides these switches for non-standard packet-forwarding decisions required byinstalled applications for specific packet types. In this mode the controller relies on the controlled switches toresolve loops and determine forwarding paths by using traditional networking mechanisms (such as STP).

• If hybrid mode is disabled, the controller makes the forwarding decisions for all packets in the OpenFlow-controlled network. In this state, the controller resolves network loops and determines forwarding paths.

Managing hybrid mode includes the following:

• Viewing and changing the hybrid mode configuration• Coordinating controller hybrid mode and OpenFlow switch settings on page 80

In all cases, the controller only monitors or directs packets within OpenFlow instances. The controller cannotdirect or monitor packets outside of OpenFlow instances.

For information on supported network switches, see the Aruba VAN SDN Controller and Applications SupportMatrix.

Learning more about hybrid modeFor more on hybrid mode as it relates to OpenFlow, see the latest OpenFlow Switch Specification on the OpenNetworking Foundation website.

For a list of Hewlett Packard Enterprise switches that support OpenFlow operation, see the latest edition of theAruba VAN SDN Controller and Applications Support Matrix.

Viewing and changing the hybrid mode configurationTo view or change the hybrid mode setting:

Chapter 4Hybrid mode for controlling packet forwarding


Procedure

1. In the Controller UI, from the navigation menu, select Configurations. Then select the Basic tab.2. Select the com.hp.sdn.ctl.of.impl.ControllerManager component. Click to show the configurable

keys for this component and view the current value for hybrid.mode.

Figure 44: Open the Controller Manager component

3. Continue with the following steps if you want to change the setting.4. Click Modify.

The Modify Basic Configuration dialog box is displayed for thecom.hp.sdn.ctl.of.impl.ControllerManager component.

Figure 45: Select the hybrid.mode field

Chapter 4 Hybrid mode for controlling packet forwarding 79

5. Set hybrid.mode to one of the following:a. true: (the default): enables hybrid mode. The controller makes packet-forwarding decisions only as

required by installed applications.b. false: disables hybrid mode. The controller makes all forwarding decisions. (Release 2.0 of the Aruba VAN

SDN Controller operates only in this mode – pure OpenFlow mode).6. Restart the controller. In a controller team environment, restart all controllers in the team.

In a controller team environment, a configuration change on one controller typically propagates to the othercontrollers on the team. However, to implement a hybrid mode configuration change, restart all the controllersin the team. Make certain that the change has propagated to all members in the team before restarting thecontrollers:

a. Close any instance of the web interface in which the controller might be running.b. At the Linux command prompt (sudo), restart the controller:

~$ sudo service sdnc restart

You can also use the REST API to set or reset hybrid mode. See the "configs REST API" section inthe

Aruba VAN SDN Controller REST API Reference

.

Coordinating controller hybrid mode and OpenFlow switchsettings

Supporting hybrid mode on OpenFlow switchesThe OpenFlow configuration on individual Hewlett Packard Enterprise switches must support the controller hybridmode setting. The following table shows the correspondence between the hybrid mode configuration on thecontroller and the per-instance passive/active configuration on Hewlett Packard Enterprise OpenFlow switches.

Table 3: Hybrid mode support on ProVision switches

Hybrid Mode Settings ProVision OpenFlow Instance Configuration

Enabled (true) passive

Disabled (false) active

For more information on the specific switch, how to configure passive/active mode, and how these switchesbehave if they lose their control-plane connection to the controller, see the OpenFlow documentation.

For a list of switches that are supported in Hybrid and pure OpenFlow mode, see Aruba VAN SDN Controller andApplications Support Matrix.

Configuring controller settings to support hybrid modeNetwork-related settings on the controller must agree with the controlled switches. Failure to achieve agreementbetween the controller’s network-related settings and the settings in the controlled switches may result inunpredictable network behavior. The following table lists the specific network-related controller settings thatshould agree with managed switches.


For information on limitations in OpenFlow table support, see the Aruba VAN SDN Controller and ApplicationsSupport Matrix.

Table 4: Controller settings to support hybrid mode

Controller Configurations Component Key Comments

com.hp.sdn.ctl.of.impl.ControllerManager hybrid.mode Set to true or false.

com.hp.sdn.disco.of.link.impl.OpenflowLinkDiscoveryComponent

age.multihop.links Set this value to the

refresh rate formultihop links. Afaster refresh rate willintroduce more link-discovery packets intothe network. A slowerrefresh rate willrespond more slowlyto a topology change.

com.hp.sdn.disco.of.node.impl.OfArpDiscoveryComponent

arp.age To support ARP-based host discovery,change this setting inthe controller to begreater than or equalto the “ip arp-age”setting on controlledswitches.

com.hp.sdn.disco.of.node.impl.OfDhcpDiscoveryComponent

dhcp.age To support DHCP-based host discovery,change this setting inthe controller to begreater than or equalto the timeout value ofthe DHCP server(s)on your network.

To view or reconfigure any of the above controller configuration components:

Procedure

1. In the controller UI, select Configurations.a. Select the Basic tab to view or modify the following components:

• com.hp.sdn.ctl.of.impl.ControllerManager• com.hp.sdn.disco.of.node.impl.OfArpDiscoveryComponent• com.hp.sdn.disco.of.node.impl.OfDhcpDiscoveryComponent

b. Select the Advanced tab to view or modify the following component:

• com.hp.sdn.disco.of.link.impl.OpenflowLinkDiscoveryComponent2. Click the Modify button. In the Modify Configuration dialog box you can view the current setting for each key

for the component and make changes.3. Click Apply to save the changes.

For more information on using the Configurations screen, see Configurations screen details.


LimitationsFor information on limitations in OpenFlow table support, see the Aruba VAN SDN Controller and ApplicationsSupport Matrix.

OpenFlow 1.0 is the default version of OpenFlow for Hewlett Packard Enterprise ProVision switches. OpenFlowdoes not allow the controller to optimize flow location in hardware tables. For concerns about line-rate data planeperformance, configure all managed switches to use OpenFlow 1.3. Failure to properly configure the switch in thisway may cause packet loss or other problems associated with high switch CPU utilization.

Uncontrolled switches in an OpenFlow Hybrid network are not visible to or controlled by the Aruba VAN SDNController. Uncontrolled switches are either controlled by another controller (outside the team) or not controlled atall (traditional networking). Traffic by such switches is independently managed.

The Aruba VAN SDN Controller Path Diagnostic Tool is useful only when hybrid mode is disabled. When hybridmode is enabled, the controller does not monitor or direct all flows in the network. As a result, the path diagnostictool (PathDiagnosticManager) does not have visibility into all flows on the network, and should not be used.

Controller packet-forwarding when hybrid mode isdisabledFigure 46: Controller operation with hybrid mode disabled

When hybrid mode is disabled (set to "false"), the controller examines and directs the packets in all flows for thegiven OpenFlow instance. The controller forwarding decisions for flows in a given instance are based on therequirements of the installed applications. The forwarding decision is communicated to controlled switchesthrough OpenFlow. In instances where the controller has not provided the switch with a rule for how to forward apacket type, the switch sends the packet to the controller and waits for the controller to provide forwardinginstructions.


Hybrid mode is commonly disabled in networks that are either used for experimental OpenFlow work (such asdeveloping a controller application) or for networks that are completely new and designed to be fully controlled byOpenFlow.

Controller packet forwarding when hybrid mode is enabledFigure 47: Controller operation with hybrid mode enabled

When hybrid mode is enabled (the default), the specific packet types for which the controller monitors andoverrides switch forwarding rules depends on the applications installed and running in the controller. That is, thecontroller overrides normal packet forwarding rules in the OpenFlow switch with application-specific forwardingrules, such as:

• copying ARP request/reply and DHCP offer/ACK packets to the controller so that it can discover end-hosts• stealing BDDP packets to the controller so that it can discover inter-switch links• changing the priority on Microsoft Lync packets to improve instant messaging speed• monitoring DNS requests to detect dangerous end-host behavior

Packets in flows that the controller does not examine or direct are forwarded through normal switching operationswithout controller intervention.


Hewlett Packard Enterprise recommends that hybrid mode be enabled when controlling traditional,established networks where applications-related traffic is responsible for only a subset of the overalltraffic load on the network. Hybrid mode is commonly enabled in established networks where newapplications are installed and running on the controller, creating a need to override normal switchingbehavior for specific flows.


A license is required for the controller. In addition, SDN applications can require licenses that are separate fromthe license for the controller. Typically, you must have both a license for the controller and a license for eachapplication. For Aruba SDN applications, you register the license, obtain the license key, and activate the licenseon the controller using the same methods you use to register and activate controller licenses. For informationabout obtaining license keys for an application, see the Administrator Guide for the application.

For information on the different types of licenses and how to maintain license registration and activation after acontroller software update, see License types, usage, and expiration.

Overview of the license registration and activationprocessAfter you have downloaded and installed the controller software, as described in the Aruba VAN SDN ControllerInstallation Guide, you can begin the license registration and activation process.

Evaluation licenses are available, for details see Using evaluation licenses.

.

The basic steps are:

Procedure

1. Preparing for license registration on page 86:a. Prerequisites for license registration on page 86b. Identifying the Install ID displayed in the controller UI on page 86

2. Registering and activating a license on page 87a. Registering your license and obtaining a license key using the My Networking portalb. Activating a license on the controller

3. Managing licenses on page 944. Transferring licenses on page 94

a. Deactivating licenses to prepare for transferb. Transferring licenses to a new platformc. Use new license keys to activate the licenses on the target controller.

License types, usage, and expirationThe following licenses are available for the Aruba VAN SDN Controller:

• Aruba VAN SDN Ctrl Base SW w/ 50–node E-LTU—The base license for the controller.• Aruba VAN SDN Ctrl 50–node E-LTU—Provides an additional 50–node license.• Aruba VAN SDN Ctrl HA E-LTU—Enables the controller to form a team for high availability. The following

guidelines apply:

◦ The number of team members for an Aruba VAN SDN Controller team is three.◦ When forming a team, only one Aruba VAN SDN Controller base license is required, along with at least two

High Availability licenses, all on the same Master controller. Once a team is formed, Add Nodes licensescan be added to the team leader for increased support. In addition, you must:

Chapter 5License Registration and Activation

Chapter 5 License Registration and Activation 85

– Use non-previously licensed controller installations to form the team.– Use a new hardware platform (or Virtual Machine) with a new installation of the Aruba VAN SDN

Controller.– Run the same software version on all controllers.

• Application Licenses

—Licenses for SDN applications. For more information, refer to the administrator guide for the specificapplication.

Preparing for license registration

Prerequisites for license registrationBefore beginning the license registration and activation process, you must do the following:

Procedure

1. Obtain a Hewlett Packard Enterprise My Networking portal user account.2. Obtain the order number or product registration ID, and e-mail address from your Aruba VAN SDN Controller

license order confirmation.3. Install the Aruba VAN SDN Controller software and have the controller running, as described in the Aruba VAN

SDN Controller Installation Guide.

Identifying the Install ID displayed in the controller UIEach controller installation generates a unique Install ID that is used for licensing activities.

To view the Install ID using the UI, select Licences from the navigation menu. In the Licenses screen, the InstallID appears before the list licenses.


Figure 48: Default License GUI


Registering and activating a licenseUsing your Install ID, you must now register your license on the My Networking portal. Doing this results in alicense key, which enables you to activate the license on the controller.

If you are registering licenses in addition to the base controller license, Hewlett Packard Enterpriserecommends you do so in the following order:

1. Register the base controller license.2. Register any Add Nodes licenses, and then activate the last license key generated.3. Register any High Availability licenses, and then activate the last license key generated.4. Register any application licenses you have acquired.

Registering your license and obtaining a license keyPrerequisites

To register your license and obtain a license key:

Procedure

1. Log on to the My Networking portal at http://www.hpe.com/networking/mynetworking.2. Select My Licenses.3. In the Order number or Registration ID field, enter your order number or registration ID and then click

Next.a. If you enter a registration ID, go to step 5.b. If you enter an order number, the Email field appears.

4. In the Email field, enter either the “Ship to” or “Sold to” e-mail address listed in your sales order confirmation,and then click Next.

A license selection screen appears, as shown below.

Figure 49: Selecting licenses

5. Select the license type, enter the quantity to be registered to your Install ID, and then click Next.


http://www.hpe.com/networking/mynetworking

• For an Aruba VAN SDN Ctrl Base SW w/ 50–node E-LTU license, the quantity must be 1.• For Aruba VAN SDN Ctrl 50–node E-LTU or Aruba VAN SDN Ctrl HA E-LTU licenses,

quantity is the number of licenses to be installed with a single Install ID.• For information on using this process for an application license, see the administrator guide

for that application.

The registration details screen appears, as shown below.

Figure 50: Entering the install ID

6. In the Install ID field, enter your Install ID number. (See Identifying the Install ID displayed in thecontroller UI on page 86.

7. Optional: Enter a Friendly name and Customer notes for this license.8. Click Next.

The end user software license agreement screen appears.9. To continue after reading the license agreement, select I accept all of the above terms, and then click

Finish.


The confirmation screen appears, as shown below.

Figure 51: Reviewing your registration

10. Review your license registration details, and record the License key listed.11. Optional: To download the license key file, click Save as, and then save it to your local hard drive.12. Optional: To e-mail the registration details:

a. Enter one or more e-mail addresses, separated by a comma or semi-colon in the field provided.b. Optional: Enter Comments about this license.c. Click Send email.

13. Optional: If you want to register additional licenses for this order:a. Click Register more for this order to return to the license selection screen shown in Transferring

licenses to a new platform.b. Repeat steps 5 through 13 until you have registered all licenses.

Viewing your license information

Procedure

1. Log on to the My Networking portal at http://www.hpe.com/networking/mynetworking2. Select My Licenses.3. Click View Licenses to see a screen similar to the following:



4. To view the information for the license you just loaded, click the Select button for that license.


You will then see a screen similar to the following:

Figure 52: Viewing your license and other information

5. Record the license key in the above screen for use when you activate the license on the controller.

Activating a license on the controllerUsing your license key, you must now activate a license on the controller, completing the license registration andactivation process.

Procedure

1. If your previous curl session has closed or timed out, re-enter the authentication command to obtain a newtoken.

2. Activate the license on the controller:

curl [options] -H "X-Auth-Token:token" \-d license_key \https://controller_ip:8443/sdn/v2.0/licenses


a. Replace token with the token you obtained using the authentication command.b. Replace license_key with the key obtained in Registering your license and obtaining a license key. You

can view the key by logging on to the My Network portal and selecting My Licenses, as shown in Viewingyour license and other information.

c. Replace controller_ip with your controller IP address.

If you are installing a High Availability license, enter the IP address of the lead controller.

The installed license information appears in JSON format, as shown below. See: example

Installed license output

{ "license" : { "install_id" : 1249679, "serial_no" : 13, "license_metric" : "HA Controller", "product" : "HP VAN SDN Ctrl Base", "metric_qty" : 500, "license_type" : "PRODUCTION", "base_license" : false, "creation_date" : "2013-09-06T00:26:52.248+0000", "activated_date" : "2013-09-06T00:26:52.248+0000", "expiry_date" : "2014-01-14T00:26:52.248+0000", "license_status" : "ACTIVE" }}

Adding and activating a license using the controller UIUse the following procedure to add and activate a license using the controller UI.

Procedure

1. In the controller UI, from the navigation menu, select Licenses.2. On the Licenses screen, enter the license key you acquired in Registering your license and obtaining a

license key in the text box next to the Add button.


3. Enter the key in the field to enable the Add button.

Figure 53: Enter the License Key

4. To activate the license, click the Add button.

The active license is displayed in the table, below the Install ID, and the Add button is no longer available.

Figure 54: Active License Displayed on License screen

Activating a license using a scriptAs an alternative to using the controller UI to activate the license, you can use a post install configuration scriptrun locally on the controller. For information on other post install configuration options using the script, see theAruba VAN SDN Controller Installation Guide chapter on post install configuration.

The post install configuration Python script is run on the local controller machine. The script is /opt/sdn/scripts/postinstall/config_sdn.py. You can run it interactively or you can use a configuration file toenter the inputs for activating the license. The default configuration file is /opt/sdn/scripts/ansible/config_sdn.conf or you can create a custom configuration file for use with the script.

To run the config_sdn.py script to activate the controller license:

Procedure

1. To use the script, first ssh to the controller system. For example, ssh [email protected], and then enter the sshpassword.

2. You can run the script either interactively or with a configuration file as follows:a. If you are running the script interactively without a configuration file, enter the option for add license on the

command line:


python config_sdn.py -LYou are prompted to enter the license key.

b. If you are using the configuration file to enter inputs for the script, edit the config_sdn.conf file or createa custom configuration file.

c. In the [general] section enter the controller IP, user name and (optionally) the password.

If you do not enter a password in the configuration file, you are prompted to enter the it when you run thescript.

d. In the [scripts] section, set addlicense=true.e. In the [addlicense] section, enter the license key, and then run the script with the command: python

config_sdn.py -f ../ansible/config_sdn.conf (or whatever the custom configuration file nameis).

3. Respond to any prompts as the script runs.

Managing licenses

Transferring licensesYou can transfer a license from one controller to another. To do so, you must first deactivate all licenses from thecontroller. Keeping a license on one controller while transferring one or more other licenses from the samecontroller to another controller is not permitted.

When upgrading, no special effort is required to preserve the licenses. Note that the license transfermechanism is only required when you want to switch the controller currently running hardware. Youmust install the controller on the new hardware and transfer the licenses to that new hardware beforeretiring the old hardware.

Prerequisites for transferring licensesBefore you transfer licenses, you must first:

Procedure

1. Deactivate all licenses, as described in Deactivating licenses to prepare for transfer.2. Obtain an Install ID for each destination controller, as described in Identifying the Install ID displayed in the

controller UI.

Deactivating licenses to prepare for transferWhen you deactivate a license to prepare for transfer, the controller generates an Uninstall Key for that license,which you will need when you transfer the license. Be prepared to record the Uninstall Key for each license youdeactivate. The Uninstall Key is a long text string. For example:

AE2RCLT7CJMDI-MAGAQHS2NBTOB-6VM4QKEQ4HAEZ-3AY4QELRPG4AA-3EMHQELRPGAYQ


Procedure

1. In the controller UI, from the navigation menu, select Licenses.2. Select the license to deactivate to prepare for transfer.

Figure 55: License screen with Deactivate button highlighted

3. Click Deactivate.4. Click OK when the deactivation prompt appears:

You will see an Uninstall key displayed for that license. Copy the Uninstall key for that license to the clipboardby clicking Copy Uninstall Key.

Figure 56: License Deactivation Prompt

5. Repeat the preceding steps for each of the remaining licenses on the controller.

Transferring licenses to a new platformAfter you have deactivated all of the licenses for a controller, you can transfer them to another controller.

Procedure

1. Log on to the My Networking portal at http://www.hpe.com/networking/mynetworking.2. From the My Licenses section, select Transfer licenses to a new platform.3. In the Search field, enter the Install ID for the controller from which you deactivated the license, and then click

Search.



The transfer license screen displays a list of associated licenses, as shown below.

Figure 57: Selecting licenses to transfer

4. Click the Select icon next to the license to be transferred.


The license details screen appears, as shown below.

Figure 58: Reviewing details before transfer

5. Verify that this is the license you want to transfer, and then click Next.

The target Install ID screen is displayed as shown below.

Figure 59: Target Install ID screen


In the Target Install ID field, enter the Install ID of the controller to which you want to transfer the license andthen click Next.

Figure 60: Entering target install and uninstall IDs

6. In the screen above, do the following:a. In each Uninstall field, enter a license uninstall key. (For more on acquiring uninstall keys, see

Deactivating licenses to prepare for transfer.)

For the transfer process to succeed, enter an Uninstall value for every registered license.

b. Click the Transfer button in the lower-right corner of the screen.


New license registration information displays on the license transfer confirmation screen and license detailsscreen, as shown below.

Figure 61: Viewing license transfer confirmation and details screens

7. Review the confirmation screen details.8. For each license you are transferring, record the new license key so that it will be available when you add and

activate the license on the new controller.9. Optional: To e-mail transferred license details:

a. Enter one or more e-mail addresses, separated by a comma or semi-colon in the field provided.b. Optional: Enter Comments about this license transfer.c. Click Send email.


The license screen displays the status of the original licenses as Transferred, and the new Install IDs as Active,as shown below.

Figure 62: Review transferred license status screens

To register the transferred licenses on the new controller, see Activating a license on the controller.

Using evaluation licensesProcedure

1. Install the Aruba VAN SDN Controller and install all the Aruba SDN applications you would like to evaluate. Ifyou are using the Hewlett Packard Enterprise SDN App Store, install the Trial Mode SDN applications.

2. Go to the My Networking portal (MNP) at http://www.hpe.com/networking/mynetworking).3. Under Licenses, select Register License.4. From the menu on the right of the screen, select SDN Evaluation Licenses.5. Enter the Aruba VAN SDN Controller installation identifier (install-id).

The My Networking portal generates every evaluation license possible for that install-id.6. Apply the relevant evaluation licenses to the controller and applications.



Standalone controller operation provides management for the OpenFlow switches in a network. However, it doesnot provide high availability (HA), with the result that a controller failure leaves the network in an unmanagedstate. Configuring a team of controllers and one or more corresponding controller regions creates a highavailability network with failover capability, resulting in a continuously managed network in the event that acontroller in the team goes down.

You can view your team configuration using the UI, see Viewing your team configuration using the UI.

High Availability best practices• Ensure the team and region configuration meets all of the requirements for teaming. For details, see

Requirements for controller teams on page 102.• Ensure that IP routing configuration in the controller domain enables the controller team IP address to be

reached from all areas of the domain.• If any application installed on the controller uses the Cassandra database, run the Cassandra nodetool

repair command every 10 days. For details, see Manually synchronizing Cassandra database nodesusing nodetool repair utility.

• Before you create the team, configure NTP such that all the controllers that will be in the team use the samelocal NTP server. Using the same NTP server for all controllers helps to ensure that the controller clocksremain synchronized. Keeping the system clocks synchronized is especially important for applications that usethe Cassandra database. For details, see Configuring controllers to use the same local NTP servers.

• Either use the same Keystone server for authentication for all three controllers in the team, or use a localKeystone server for each controller in the team. For details on security, see Security on page 113.

• To use TLS connections for communications between the switch and the built-in OpenFlow controller moduleof the Aruba VAN SDN Controller, Hewlett Packard Enterprise recommends that all controller and devicecertificates be signed by the same CA. For information about configuring TLS on a switch, see thedocumentation for the switch.

• If you are configuring web proxy server settings, ensure that you include the team IP address and the IPaddresses for all three controllers in the team in the /etc/init/sdnc.conf file. After editing thesdnc.conf file all three controllers will need to be restarted. For details, see Obtaining applications fromthe Hewlett Packard Enterprise SDN App Store on page 32.

About teaming for High Availability• Each controller belonging to a team is a team member. To centralize team management and control, one

controller is elected by the team as the team leader.• Teaming is configured on one controller and is automatically propagated to the other controllers in the team,

regardless of which controller becomes the team leader.• After a team is configured, any configuration changes will propagate to each controller.• If the team leader goes down, another active controller becomes the team leader. If a team leader that failed

recovers and rejoins the team, it rejoins the team as a team member and does not resume team leadership.• Each controller in the team has its own IP address, which is the IP address of the machine on which the

controller is installed. In addition, the administrator configures a separate IP address—called the team IPaddress—to represent the team as a whole. The team IP address is active on the team leader. If the currentteam leader goes down, the failover process includes keeping the team IP address active on the new teamleader.

• For the controllers in a team to remain active, they must be part of the team quorum. To be part of a teamquorum, a controller must be connected to at least one other team member that has a status of active or

Chapter 6Configuring for High Availability

Chapter 6 Configuring for High Availability 101

initializing. If one controller in the team goes offline, controller operations can continue. However, if twocontrollers in a team fail, the third controller does not operate as a standalone controller. Instead, the thirdcontroller loses its membership in the team quorum, and the controller status is changed to suspended.

• A region groups devices together with their controllers. A region must have three controllers which must bespecified in priority order for all devices within the region (master, primary slave, secondary slave).

• Putting the region configuration in place for a controller team ensures seamless failover and failback amongthe configured controllers for the specified network devices in a region. When a controller experiences a fault,the region configuration ensures that a slave controller immediately assumes the master role over the groupsof network devices for which the failed controller was master. Once the failed controller recovers and rejoinsthe team, the rejoining controller takes back the role for which it was configured with respect to the networkdevices. For details on failover and failback, see Failover behavior within a region on page 205 and Failback behavior within a region on page 207.

• Devices in a region can be expressed as a list of individual IPv4 addresses, a list of IPv4 ranges, or acombination of both. Devices included in a region can connect to the region’s controllers. OpenFlow 1.3devices must be configured with the IPs of all three controllers in a team. This allows one of those controllersto assert itself as the master of a given device. The device then automatically assigns a role of slave to theother two configured controllers. This ensures the master knows of all the events happening on the devicewhile the slaves are kept up to date on a subset of events.

• Applications are stopped when there is a change in the teaming status for a given controller. For example:

◦ When the controller transitions from stand-alone to a member of a team, all applications are stopped priorto the creation of the team, and then restarted after the team is formed.

◦ When a teamed controller detects that it is no longer part of the quorum, all applications are stopped untilthe controller detects it has rejoined the quorum.

The team status can be: active, unreachable, or unknown (for details, see Team status on page 103). You canview team status from the Team screen in the controller UI, see Viewing team status on page 106.

The controller status can be: initializing, active, suspended, or unreachable (for details, see Controller status onpage 103. You can view controller status from the Team screen in the controller UI, see Viewing teamconfiguration and controller status.

You can view region configuration from the Team screen in the controller UI, see Viewing region configuration.

Requirements for controller teams• Controller team operation requires the High Availability Add Controller license (HP VAN SDN Ctrl HA E-LTU).• A team must consist of three controllers.• A controller can be part of one team only.• Each controller must be able to communicate with all controllers on TCP ports 5700, 7001, and 8443.• All controllers in a team must be running the same software version.• The administrator must create the teams and regions.• When controllers are in a team, the use of regions is required.• A team requires one IP address for each controller, plus one IP address that represents the team as a whole.• Only OpenFlow 1.3 network devices are supported:

◦ Each network device must be configured to connect to the IP addresses of all the controllers in the team,but not to the team IP address.

◦ For the controller to recognize the device, when you configure regions on the controller, you must alsoexplicitly add the network device to a region.

• After a team is created, controllers adopt the data of one of the members of the team. Hewlett PackardEnterprise recommends that you minimize the amount of data to be copied by configuring teams soon after thecontrollers are installed and started.


Requirements for configuration, licensing, or applicationchanges to controller teamsAll controllers in a teamed environment must be active before you can make configuration, licensing, orapplication changes, or changes to regions. Changes attempted when a team member is initializing ordisconnected are not guaranteed to be consistent. If you attempt to make a change while a controller is not in anactive state:

• If you are using the user interface, the change is blocked.• If you are using using the REST API, the change is blocked and the controller returns a Forbidden response

(error code 403) and an UnsafeConfigurationException exception.

Team statusYou can view your team status from the top banner of the controller UI, see Viewing team status on page 106.

The team states are:

• activeAll 3 controllers are actively operating. A prerequisite for this state is that all controllers are able tocommunicate with each other.

• unreachableAny single controller of a team is not able to communicate with the rest of the team members. Thisstatus occurs because of either of the following possible situations:

◦ The sdnc service stopped working or a controller is in the process of rebooting.◦ A network partition occurred and a controller in a team has become separated from the other team

members.• unknownA team status cannot be determined because of either of the following possible situations:

◦ A communication failure with a REST service component occurred.◦ A network failure caused the controller to be suspended.

Controller statusAruba VAN SDN Controller groups components into categories and those component groups are initialized inorder as the startup sequencer moves through the stages in the startup sequence. During the operational phase,the OpenFlow port is opened. Core services are the first to be initialized and they are always active. SystemInformation Service is part of the core services and thus the controller status cannot be determined or reporteduntil the core services complete the initialization phase.

For more information, see Error log for team configuration on page 200, Team alias node on page 202, Failover behavior within a region on page 205, Failback behavior within a region on page 207.

The controller states are:

• initializingThe sequencer completed the startup sequence through the team stage and the controller is part of the teamquorum (connected to at least one other active controller in the team), but has not yet deployed and initializedthe operational group services. At this point, the OpenFlow port is not open.

• activeThe sequencer completed all stages of the startup sequence and the OpenFlow port is open. If the controller isa member of a team, it is part of the team quorum (connected to at least one other team member that hasstarted its teaming services).

• suspendedThe sequencer completed all stages in the suspend sequence. The sequencer initiates the suspend sequencewhen a monitored core service reports an unhealthy status or a teamed controller loses its membership in the


team quorum. Core services are started. Teaming services are started but are waiting until the controller canbecome a member of a team quorum. The OpenFlow port is closed.

• unreachableA controller sees a remote controller as unreachable if the connection to the remote controller is broken. Acontroller never sees itself as unreachable.

If an application reports an unhealthy status, an alert is generated but the controller remains in the active state.

If two controllers in a team fail, the third controller does not operate as a standalone controller. Instead, the thirdcontroller loses its membership in the team quorum, and the sequencer initiates the suspend sequence.

You can view your controller status from the top section of the Team screen in the UI, see Viewing teamconfiguration and controller status.

Manually synchronizing Cassandra database notes usingthe nodetool repair utilityThe Cassandra nodetool repair utility corrects inconsistencies among instances of the Cassandra databasesuch that all nodes have the same and current data. It is recommended that the nodetool repair utility be run onall cluster nodes periodically once a week. Beginning with version 2.8 of the Aruba VAN SDN Controller, this hasbeen automated.

As part of HP VAN SDN Controller install, a script (/opt/sdn/admin/casMaint.sh) is installed andautomatically scheduled to execute every Sunday at 2:00 AM by default. The script works only on the currentTeam Leader of a cluster, although it is executed on all members of the cluster as part of the cron job. The scriptruns nodetool repair on all active Cassandra nodes that are part of the cluster. The script logs the results ofit's execution in a log file located at /var/log/sdn/cassandra/maint.log.

Changing the default script execution time

The Cassandra maintenance script runs every Sunday at 2:00 AM by default. The script is scheduled via cron.

An example of crontab format with commented fields is shown next. This example shows the Cassandramaintenance script scheduled at 2:00 AM every Sunday:

#Minute

Hour

Day ofMonth

Month Day ofWeek

Command

#(0-59)

(0-23)

(1-31) (1-12 orJan-Dec)

(0-6 orSun-Sat)

0 2 * * 0 /opt/sdn/admin/casMaint.sh -r

To change the default schedule, log in to the controllers as user sdn and execute the following commands:

• Save the current cron tab settings to a temp file: crontab -u sdn -l > /tmp/cron.tmp• Edit the saved temp file (/tmp/cron.tmp) and change the schedule by editing the line that has

casMaint.sh• Update the crontab by running: crontab -u sdn /tmp/cron.tmp

Guidelines for running the nodetool repair utility• Run the utility on each server in the controller team.• Schedule regular repair operations for one server in the controller team at a time.• Schedule regular repair operations once every 10 days.• Disk activity increases during repair operations, so schedule repair operations during low-usage hours.


Running the Cassandra nodetool repair command

Prerequisites

The commands in this procedure are run from the command prompt on the Linux system on which the controlleris installed.

Procedure

1. Confirm that the Cassandra database is online:

~$ /opt/sdn/cassandra/bin/nodetool status

If you see the following message, the Cassandra database is not online:

Failed to connect to '127.0.0.1:7199': Connection refused2. If the Cassandra database is not online, you must restart the controller to restart the Cassandra instance on

that controller:a. Close any instance of the web interface connected to the controller to be restarted.b. Restart the controller:

~$ sudo service sdnc restart3. Repair and synchronize the database by entering the following command on each controller:

~$ /opt/sdn/cassandra/bin/nodetool repairYou can perform this step while the system, controller, and applications are running.

Configuring controllers to use the same local NTP serversUsing the same NTP servers for all controllers you plan to include in a team helps to ensure that the controllerclocks remain synchronized. Keeping the system clocks synchronized is especially important for applications thatuse the Cassandra database. The controllers in the team must be configured with one or more NTP servers.

Before you create the controller team, configure all the controllers you plan to include in a team to use the samelocal NTP servers. You can specify an NTP server for each controller using the controller UI (see Modifying NTPserver or date and time) or using the manual steps below.

Obtain the IP addresses of the local NTP servers for your site and ensure that these local NTP servers are theonly NTP servers configured for each controller you plan to include in the team.

Procedure

1. Log in to the Linux system on which the controller is installed.2. Verify that NTP is configured on the system:

~$ ntpdc -c peersIf a list of servers is displayed, one or more NTP servers are configured on the system.

3. Edit the /etc/ntp.conf file to remove the entries for servers other than the local NTP server and replacethose entries with an entry for the local NTP server:a. Remove (or enclose in comments by prepending the # character) all lines that start with the word server.b. Add the following line for each local NTP server, where local-ntp-ip is the IP address of that local NTP

server:

server local-ntp-ip iburstc. Save and close the file.

4. Restart the NTP service:~$ sudo service ntp restart

5. Verify that the local NTP servers are the only NTP servers configured on the system:


~$ ntpdc -c peers

NTP is configured correctly if the IP addresses for the local NTP servers are the only entries displayed.

Viewing your team configuration using the UIYou can view your team and region configuration from the SDN Controller UI’s Team screen. To access the Teamscreen, click Team in the controller UI navigation pane. The Team screen is read-only and includes:

• Team status (top banner)• Team configuration and controller status (top section)• Region configuration (middle section)• Device Owners (bottom section)

Figure 63: Viewing your team status and configuration

Viewing team statusYou can view your team status from the top banner of the controller UIscreen. The team status indicator refreshesdynamically to immediately notify you of important team status changes, such as when a 3-node team changes toa 2-node team.

The team status banner displays one of the following team status messages:

• ACTIVE

All 3 controllers are actively operating (for example, all controllers are able to communicate with each other) soa healthy team status message is displayed:

• UNREACHABLE

Any single controller of a team is not able to communicate with the rest of the team members so a degradedstatus massage is displayed:

Unreachable statusoccurs for either of the following reasons:


◦ The sdnc service stopped working or a controller has been rebooted.◦ A network partition occurred and a controller in a team has become separated from the other team

members.• UNKNOWN

A team status cannot be determined and an unknown team status message is displayed:

Unknown status occurs for either of the following reasons:

◦ A communication failure with a REST service component occurred.◦ A network failure caused the controller to be suspended.

Viewing team configuration and controller statusFrom the Team screen, the team configuration and controller status (top section) displays the following fields:

• IP: IP address of the controller• Role: Member or leader. A team can only have one leader and at most three controllers.• Status: The status for each controller in the team, which can be one of the following (for details see

Controller status on page 103):

◦ initializing◦ active◦ suspended◦ unreachable

• Version: The build version of the controller software running on the controller.• CDV: Core data version. This field is incremented every time the controller experiences a change in

configuration. This field is used to determine which controller to synchronize with when a controller joins acluster.

• CDV Timestamp: Date and time at which the controller experienced its last change in configuration.

Viewing region configurationFrom the Team screen, the region configuration (middle section) displays the following fields:

• Name: Name of the region.• Controller by Priority: First controller (in bold) is the master controller for the region. The master controller

handles the flows and packet-ins. The following controllers (grayed out) are slaves. If the highest prioritycontroller is unavailable, HA fails-over to the next highest priority device in the region configuration for thatdevice.

• Region UID: Identifies a region; used by certain REST API commands.

To view a region’s details, select the desired region. In this example, Region123 is selected:

The region’s details include the following fields:

• Ranges: The configured ranges (IP ranges).

:• Devices: List of IP addresses one-by-one.

One of these must be present, or both may be used.


Viewing devices, datapaths, and debug logsThe Device Owners portion of the Team screen (bottom section) displays the following fields:

• Device: The device IP• Owner Controller: The controller that is considered the current owner of the device (for example, the

configured OpenFlow master)• Region: The region that the device belongs to• Datapath ID: Datapath identifier for the device. A device can have multiple datapaths.• Datapath ready: Default flows have been pushed

To view a device’s debug log for support purposes, select the desired device. In this example, the debug log fordevice 12.255.125.29 is shown:

Methods for configuring HA teamingThere are a number of different ways for you to configure HA teaming. For the first method (using a script), aconfiguration file is required, see Defining inputs for teaming in a configuration file on page 109.

For details on viewing teaming information in the controller UI, see Viewing your teamconfiguration using the UI.

Chose the method that best meets your needs.

• Using a Python script from a controller to configure a team


Use a Python script executed locally on a controller to configure a team of three controllers. Typically thescripts are used to configure a single team. The script uses a configuration file to define the inputs forconfiguring the team. The script can be used to do the following:

◦ Create a team◦ Disband a team◦ Create region(s)◦ Delete region(s)◦ Add device to a region◦ Remove device from a region

• Using curl commands for REST APIs to configure a team, see Team configuration using curl commands onpage 196.

Defining inputs for teaming in a configuration fileThe first step in configuring teaming using a Python script is to define the inputs for teaming in a configuration file.

The default configuration file is build_team.conf. The file is on the controller system at /opt/sdn/scripts/teaming/build_team.conf. If you cannot access the /opt/sdn path, you need to enter sudo su first inorder to see this directory path.

You can edit this file to provide the input for team configuration or you can create a custom configuration file with adifferent name but the same format and in the same directory.

You can create any number of configuration files.

The following is an example of a build_team.conf file.

# build_team.conf

User=<sdn_controller_username>Password=<sdn_controller_password>Sleep_Time=20

Team_IP=##.##.##.###IP1=##.##.##.###IP2=##.##.##.###IP3=##.##.##.###

Controller_IPs=$IP1,$IP2,$IP3#Controller_IPs=##.##.##.###,##.##.##.###,##.##.##.###

Region1_Name=RED#Region1_Prioritized_Controller_IPs=##.##.##.###,##.##.##.###,##.##.##.###Region1_Prioritized_Controller_IPs=$IP1,$IP2,$IP3Region1_Device_Ranges=##.##.##.##-##,##.##.##.##-##,##.##.##.#-#Region1_Device_IPs=##.##.##.###

Region2_Name=BLUE#Region2_Prioritized_Controller_IPs=##.##.##.###,##.##.##.###,##.##.##.###Region2_Prioritized_Controller_IPs=$IP2,$IP3,$IP1Region2_Device_Ranges=##.##.##.##-##,##.##.##.##-##,##.##.##.#-#Region2_Device_IPs=##.##.##.##,##.##.##.###,##.##.##.#

Region3_Name=GREEN#Region3_Prioritized_Controller_IPs=##.##.##.###,##.##.##.###,##.##.##.###Region3_Prioritized_Controller_IPs=$IP3,$IP1,$IP2Region3_Device_Ranges=##.##.##.##-##,##.##.##.##-##,##.##.##.#-#Region3_Device_IPs=##.##.##.##,##.##.##.##,##.##.##.#


Edit the configuration file (for example vi build_team.conf) or create a custom configuration file to includethe following inputs about the team configuration. You cannot change the parameter name, such as User orTeam_IP. Some entries are optional as noted in the following table.

You can create multiple configuration files, each for use with a different team configuration. Then youcan use the same script (but with a different configuration file) to create another team.

Parameter Value

User (optional) The default login user name is sdn. Enter a different login username if a user name other than sdn has been set for the controllers. Orleave the user name blank in the configuration file if you want the user tobe prompted to enter it when they run the script. All controllers in theteam you are configuring must have the same user name and password.

Password (optional) The default login password is skyline. Enter a different loginpassword if a different one has been set for the controllers. Or leave thepassword blank if you want the user to be prompted to enter it when theyrun the script. All controllers in the team you are configuring must havethe same user name and password.

Sleep_Time (required) The default is 30 seconds. You can change this value if youwant to allow more time for the controller to propagate the changes toother controllers. You may need to do this if, for example, your network isslow. If you leave this blank, the default 30 seconds is used.

Team_IP (required) The IP address to be assigned to the team. The team IPaddress is different from the individual controller IP addresses. It is usedas a virtual address for connecting to the team leader.

IP1, IP2, IP3 Physical IP addresses for the three controllers in the team. IP1, IP2, andIP3 are optional because alternately you can enter the IP addressesunder Controller_IPs instead of as IP1, IP2, and IP3.If you enter an IPaddress for IP1, IP2, and IP3 you can use the notation $IP1, $IP2, and$IP3 in the Controller_IPs and Region parameters that follow. Whenrunning the team configuration script, it will check to verify that the IPaddresses are valid.For requirements on the IP address you enter forteam configuration, see High Availability best practices on page 101.

Controller_IPs (required) The order of the controllers in the team from master to primaryslave and secondary slave. You can either enter the full IP addressesseparated by commas or you can use the notation $IPx where x is thenumber from the values you defined above. For example $IP1, $IP2,$IP3.

Region1_name (required if specifying regions) User-defined name for the first region.You can specify 1, 2, or 3 regions in a team, or you can have aconfiguration with no regions.

Table Continued


Region1_Prioritized_Controller_IPs (required if specifying regions) The prioritized order of the controllers inthis region from first to last. You can either enter the full IP addressesseparated by commas or you can use the notation $IPx where x is thenumber from the values you defined above. For example $IP1, $IP2,$IP3.

Region1_Device_Ranges (optional but either range or device IPs are required if defining devices)The ranges for devices that you want to be configured in Region1. Youcan either define devices by range or by individual IP address.Forrequirements on the device IPs you enter for team configuration, see High Availability best practices on page 101.

Region1_Device_IPs (optional but either range or device IP is required if defining devices) Thedevice IPs configured in Region1. You can either define devices by rangeor by individual IP address.The devices must be configured with the IPsof all three controllers in a team.

Region2, Region3 (optional) Enter inputs for other regions you want to define. You canspecify 1, 2, or 3 regions in a team, or no regions. If you don’t want todefine Region2 or Region3 then leave all items for that region blank.

Using a Python script from a controller to configure ateamYou can configure teaming using a Python script (conf_team.py) and a configuration file on any activecontroller you plan to include in the team. For details on the teaming requirements, see High Availability bestpractices on page 101.

From a controller, the script allows the following teaming configuration actions: create a team, create regionswithin a team, add devices to a region, disband a team, delete regions within a team, remove devices from aregion.

To use the script you must ssh to one of the controllers you plan to include in the team. For example, [email protected] (then enter the ssh password). You do not need to ssh to the other two controllers in the team.

The Python script conf_team.py is run from the controller you ssh into. Edit a configuration file on that samecontroller to define the inputs you want to use when running the script. The default configuration file isbuild_team.conf. The file is on the controller system at /opt/sdn/scripts/teaming/build_team.conf. (The /opt/sdn path may not be visible to the ssh user. The user must be part of the sdngroup to see the directory structure). You can edit this file or you can create a custom configuration file with adifferent name, but the same format and in the same directory.

For details on using the configuration file, see Defining inputs for teaming in a configuration file on page 109.

Once you have all the inputs you want to use for team configuration in the configuration file, run the script. Byrunning the script, /opt/sdn/scripts/teaming/conf_team.py you can access all the team configurationactions.

To run the script:

Procedure

1. Issue the command to run the script:python conf_team.py [config file name] [-v]


The [config file name] default is build_team.conf. If you don’t specify a configuration file, the script usesthe build_team.conf file in the same directory as the script. If you have created a custom configuration file,enter that configuration file name.

Note that -v is optional. If you want to run the script in debug mode, add -v at the end of the command.2. If prompted, enter user and password. The three controllers must have the same user name and password.3. Check for messages or errors and respond to any prompts.4. Select X/x to exit the script.


The Aruba VAN SDN Controller communicates with different components, both internal and external to thecontroller, via secure channels. This section documents these channels, their defaults, and how to configure themin a deployment environment.

SDN Controller authenticationThe SDN Controller identifies itself via Public-Key Infrastructure (PKI) for its communication with externalsubsystems and other controllers. It uses a Java keystore and truststore to keep its private key and public keyrespectively. These keys can be used for confidential and trusted communication with clients and keystone. ForREST APIs, the controller uses bearer token authentication to authenticate the client. The client must present avalid token via the X-Auth-Header to authenticate itself with the controller. Since this means of tokenauthentication are bearer tokens, use PKI to ensure trusted communication with keystone and clients, and toavoid unauthorized use of tokens. Make sure that the certificates that you use for both keystone and the controllerare part of a valid trust chain. Token authentication is discussed further under SDN Controller keystore andtruststore locations and passwords on page 115.

The controller ships with a self-signed certificate. Therefore, it is recommended that the self-signed certificate bereplaced by a certificate signed by a reputable Certificate Authority (CA). If you choose to replace the self-signedcertificates with CA signed equivalents, see Changing the default controller keystore and truststore to useCA signed certificates. Also, the default password for the keystore and truststore should be changed as well.

Enable (2-way SSL) mutually trusted PKI communication to require both the controller and keystone to presentvalid certificates before starting the communication.

Hewlett Packard Enterprise strongly recommends that you change all default credentials to notexpose any access to the controller. This includes changing the values from the defaults for thecontroller password, keystore and truststore passwords, the keystore admin token, and the controllerservice token (see Security procedure).

Changing the default controller keystore and truststore touse CA signed certificates

In a teamed environment, unique certificates are generated for each controller in the team. You mustrepeat the following procedure for each controller in the team.

To create a CA-signed keystore and truststore, as the SDN user (for example, sudo - sdn):

Procedure

1. From the Configurations screen of the controller UI:a. In the Advanced tab of the Configurations screen, select each of the following components and change

the value of the selfsigned key to false:

com.hp.sdn.api.impl.AlertPostManagercom.hp.sdn.misc.AdminRestComponentcom.hp.sdn.misc.ServiceRestComponent

b. Select the com.hp.sdn.adm.mgr.impl.hpws.HpwsInstallManager configurable component andensure that the following keys have the values indicated in the following table:

Chapter 7Security

Chapter 7 Security 113

Key Value

keystore /opt/sdn/admin/keystore

keystore.password password is not displayed(ENC())

selfsigned falsetruststore /opt/sdn/admin/truststore

truststore.password password is not displayed (ENC())

For more information about changing controller configurable components, see Modifying a componentconfiguration.

2. Log in to the system running the SDN Controller as the sdn user and stop the controller.sudo service sdna stop

3. Back up your default /opt/sdn/admin/keystore and /opt/sdn/admin/truststore to a safelocation.

A Java keytool is used to create the new keystore and CSR . This tool can be found at /opt/sdn/openjdk8-jre/bin/keytool.

4. As the sdn user, create a new keystore:

cd /opt/sdn/admin

rm keystore truststore

/opt/sdn/openjdk8-jre/bin/keytool -genkey -alias serverKey -keyalg rsa -keysize2048 -keystore keystore

5. To support teaming, specify an IP address as the common name when configuring your server for the firstand last name question.

6. Generate a CSR (Certificate Signing Request) for signing:

/opt/sdn/openjdk8-jre/bin/keytool -keystore keystore -certreq -alias serverKey -keyalg rsa -file sdn-server.csr

7. Send the sdn-server.csr to a CA to be signed.

The CA authenticates you and returns a signed certificate and its CA certificate chain. We assume the signedcertificate from the CA is named signed.cer and the CA's certificate is root.cer. If root.cer is fromyour own internal CA, then you need to import root.cer into your browser as an authority.

8. Import the signed certificates into your keystore and truststore:a. Import the root.cer certificate into your keystore and truststore:

/opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore keystore-file root.cer -alias CARoot

/opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystoretruststore -file root.cer -alias CARoot

b. Import the root-int.cer certificate into your keystore and truststore:

/opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore keystore-file root-int.cer -alias CARoot

/opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystoretruststore -file root-int.cer -alias CARoot

9. If you do not have an intermediate root-int.cer file, use a different alias such as CARootInt. Forexample:


/opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore keystore -file root-int.cer -alias CARootInt

/opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore truststore-file root-int.cer -alias CARootInt

10. Replace your self-signed certificate in your serverKey entry with the signed certificate from your CAsigned.cer:

/opt/sdn/openjdk8-jre/bin/keytool -importcert -keystore keystore -filesigned.cer -alias serverKey

11. Add the certificate from your CA to Linux trusted certs using root. For example:

root@sdnctl1:/opt/sdn/admin# cp cacert.pem /usr/local/share/ca-certificates/cacert.crt

root@sdnctl1:/opt/sdn/admin# update-ca-certificates

The following is an example of what you will see displayed during this process:

Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done Running hooks in /etc/ca-certificates/update.d... Adding debian:cacert.pem done. done. root@sdnctl1:/opt/sdn/admin#

12. Start the controller:

sudo service sdna start

SDN Controller keystore and truststore locations andpasswordsThe SDN Controller keystore and truststore are referenced by the following configurable components:

• com.hp.sdn.api.impl.AlertPostManager• com.hp.sdn.misc.AdminRestComponent• com.hp.sdn.misc.ServiceRestComponent• com.hp.sdn.adm.mgr.impl.hpws.HpwsInstallManagerThe values for keystore and keystore.password contain the keystore location and encrypted keystorepassword respectively.

The values for truststore and truststore.password contain the truststore location and encryptedtruststore password respectively.

The keystore and truststore location and password for each component must match the configured keystore andtruststore location and password.

EncryptionSensitive information such as tokens and passwords are stored encrypted on the SDN Controller. However, toencrypt and decrypt these properties, the controller requires a master key that is passed into the controller upstartscript via an environment variable. To change the default master key (recommended):

Procedure

1. Stop the following services:


sudo service sdnc stop

sudo service sdna stop2. Change the default master key:

sudo /opt/sdn/admin/sdnpass old_master_key new_master_key

Built-in OpenFlow controllerThe Aruba VAN SDN Controller has a built-in OpenFlow controller for controller-to-switch communications. TheOpenFlow controller component relies on PKI to establish mutual trust (2-way SSL) between itself and theOpenFlow switches that it manages. To establish TLS connections for controller-to-switch OpenFlowcommunications, Hewlett Packard Enterprise recommends the following:

• Use different store names for the built-in OpenFlow controller keystore and truststore than used for the ArubaVAN SDN Controller keystore and truststore.

• Use the same CA (certificate authority) to sign the controller and all device certificates.

For information about configuring TLS, see the latest HPE OpenFlow Administrator Guide for your switch.

Creating a keystore and truststore for OpenFlow switch communicationThe process for creating the OpenFlow keystore and truststore is similar to the steps outlined under Changingthe default controller keystore and truststore to use CA signed certificates.

Built-in OpenFlow controller keystore and truststore locations andpasswordsThe Aruba VAN SDN Controller has a built-in OpenFlow controller for controller-to-switch communications. Theconfigurations for the built-in OpenFlow controller keystore and truststore are located in thecom.hp.sdn.ctl.of.impl.ControllerManager component. The keystore and keystore.passwordkeys store the location of the keystore and the password of the keystore respectively. Similarly, the truststoreand truststore.password keys store the location of the truststore and the password of the truststorerespectively.

You can configure the com.hp.sdn.ctl.of.impl.ControllerManager component in the Configurationsscreen Basic tab (screen example is shown below). A controller restart is required if these configurations arechanged.

The path to the keystore or truststore location must be specified as a relative path from the /opt/sdn/virgodirectory. For example, to specify a location of /opt/sdn/config/of.jks enter the following:


../config/of.jksFigure 64: Components that reference OpenFlow keystore and truststore

REST authenticationThe Aruba VAN SDN Controller relies on token-based authentication to authenticate most of its REST APIs. AllREST APIs except the /auth and /rsdoc APIs require an authentication token embedded in an X-Auth-Tokenheader to be included with each REST request. The /auth API allows you to obtain a token, while the /rsdocAPI provides REST API documentation information about the controller’s REST API.

The Aruba VAN SDN Controller REST API documentation is accessible from a web browser and inPDF format in the Hewlett Packard Enterprise Information Library for SDN. The RSdoc for yourcontroller can also contain REST API documentation added by applications installed on thecontroller. Although the RSdoc API explorer interacts directly with the controller REST API, RSdoc isnot intended as a management or configuration interface. Use caution when using the Try it out!button for POST or PUT methods because this action can result in changes to your current controllerenvironment.

Requests to the controller using the POST method of the cms/client/event resource can be authenticatedusing client certificate-based authentication instead of token-based authentication. For details on the ClientMapper Service that allows integration with an external policy manager such as Aruba ClearPass Policy Manager,see Using an external policy manager on page 234.

When you use certificate-based authentication, the controller authenticates the REST API request by verifying thecertificate presented by the client when the SSL connection is established against the client certificate in thecontroller truststore.

Requirements for controller environment when using certificate-based authentication

For the controller or for each controller in the team these requirements must be met:

• The certificate for the client, an external policy manager such as Aruba ClearPass, must be imported in to thetruststore of the controller.

• The issuer CN (common name) of the certificate must be entered for the value of the clearpass.cert.cnkey of the com.hp.sdn.cms.impl.ClientMapperServiceProvider controller configurable component.


• Controller hybrid mode must be enabled (set to true).• The value of the com.hp.sdn.cms.impl.ClientMapperServiceProvider controller configurable

component key clearpass.integration.enabled must be true.

Requirements for the REST API request when using certificate-based authentication

• Certificate-based authentication can only be used for cms/client/event POST requests.• The request must not include an X-Auth-Token in the request header.• The URI for the request must use port 8445. For example:

POST https://CONTROLLER_IP:8445/sdn/v2.0/cms/client/event where CONTROLLER_IP is the IP address of the controller

OpenStack Keystone used for user and token managementThe SDN Controller uses Openstack Keystone as an identity management for managing users, generatingtokens, as well as token validation.

The controller supports Keystone releases supporting the 2.0 REST API from Folsom up to the Juno release. Itsupports the following token authentication providers:

• UUID – 32 character string (All Keystone releases)• PKI – CMS message containing service catalog, user roles, and metadata (Grizzly and later)• PKIZ – ZLIB compressed PKI token (Juno and later)

The controller is configured by default to auto-detect the token provider. It can also be forced to use a specificprovider.

The auto detection logic determines that any token longer than 32 characters is PKI or PKIZ. Distinguishingbetween PKI and PKIZ is accomplished by detecting the PKIZ prefix which is prepended to PKIZ compressedtokens.

UUID AuthenticationThe UUID authentication follows this process:

Procedure

1. The controller, upon receiving the username/password pair for a user, sends the pair along with a tenant/project to the Keystone Identity Management service.

2. Keystone, upon receiving the username/password pair:

• Checks if the username/password is valid for the requested user and tenant/project• If the username/password/tenant combination is valid:

◦ Generates a UUID token◦ Stores the UUID token in its backend◦ Sends a copy of the UUID token back to the controller

3. The controller caches the token and returns a copy to its client.4. The controller’s client uses this token on each API request to the controller.5. Upon each user request, the controller sends this UUID back to Keystone for validation.6. Keystone returns success or failure status to the controller.7. On success Keystone grants access to its client to the API call, otherwise it would fail the call with an

authorization failure message.

This design requires every API request to call in to Keystone for validation. This approach does not scale well asthe number of API calls increases. The PKI authentication mechanism addresses this issue by using a private/


public key pair to produce a CMS message which can be verified by an endpoint without checking with Keystonefor every API request.

PKI AuthenticationThe PKI authentication provider was introduced in the Grizzly release of Keystone. To use PKI tokens, keys andcertificates need to be generated. In Grizzly, this is done by using the keystone-manage pki_setup command.Keystone becomes a (CA) by signing user tokens.

The authentication process is as follows:

Procedure

1. At startup, the controller:a. Starts a task that periodically downloads the CA and signing certificates from the Keystone server

(configurable using the PKICertsDownloadHour item).b. Starts a task that periodically downloads the token revocation list from the Keystone server (configurable

using the RevListPollPeriod item).

item)2. The controller, upon receiving the username/password pair for a user, sends the pair along with a tenant/

project to the Keystone Identity Management service.3. Keystone, upon receiving the username/password pair:

a. Checks whether the username/password is valid for the requested user and tenant/project.b. If the username/password/tenant combination is valid:

• Keystone builds a JSON message using:

◦ Service catalog details◦ User role◦ Metadata

• Produces a CMS message signing it using the private key.• Strips the header and footer and then produces a URL safe base64 encoded token.• Returns the token to the controller.

c. The controller caches the token and returns a copy to its client.d. The controller’s client uses this token on each API request to the controller.e. Upon each user request, the controller validates this token by:

• Checking whether the token is in its cache, if it generated the token.• Checking whether the token is valid using signature verification with the signing certificate, if the token is

not in its cache and not on the revocation.• If the received token is compressed, the controller decompresses it before checking the signature.

f. The revocation list is periodically retrieved from the Keystone server and is used to determine whether atoken is revoked.

g. The periodic certificate download results in the CA and signing certificates to be updated daily.

Local vs Remote KeystoneBy default the Keystone server is assumed to be installed on the same machine (localhost) as the controller. Aremote Keystone server can be specified using the ServerVIP configuration key in theAuthenticationManager component.

Keystone controller configurationThe following Keystone controller configuration is set in the controller UI Configurations screen in the Systemtab under the com.hp.sdn.adm.auth.impl.AuthenticationManager component. The keys are describedas follows:


AdminToken – Keystone admin token.

ConnPoolEvictPeriod – Keystone idle connection clean-up cycle in milliseconds. Minimum is 100.

ConnPoolMaxActive – Keystone maximum active connections. Minimum is 1.

ConnPoolMaxIdle – Keystone maximum idle connections. Minimum is 1.

ConnPoolMinIdleTime – Keystone minimum idle connection time in milliseconds. Minimum is 1000.

ConnSSLClientAuth – Keystone mutual authentication using TLS.

ConnTimeout – Keystone connection timeout in milliseconds. Minimum is 0.

Keystore – Keystone keystore location.

KeystorePass – Keystone keystore password.

MaxCachedTokens – Maximum number of cached tokens. Minimum is 0.

PKICertsDownloadHour – Hour in a 24 hour day (0-23) when PKI certificates download form the Keystoreserver occur.

PKICertsPath – Keystone PKI (signing and CA) certificates location.

RevListPollPeriod – Keystone PKI revocation list poll interval in seconds.

ServerPort – Keystone server port.

ServerVIP – Keystone server virtual IP.

ServiceRole – Role for shared secret.

ServiceTenant – Tenant (project) for shared secret.

ServiceToken – Shared secret for internal requests.

ServiceTokenTimeout – Timeout for shared secret, 0 for never. Minimum is 1.

ServiceUser – User for shared secret.

Tenant – Keystone tenant (only a single tenant is supported).

TokenProvider – Keystone token provider (Auto-Detect | PKI | PKIZ | UUID)

Truststore – Keystone truststore location.

TruststorePass – Keystone truststore password.

UserRole – Keystone user role (only a single role is supported. Only a user having this role is allowed access tothe controller.

For information on Keystone, see the OpenStack Keystone documentation at http://docs.openstack.org/developer/keystone/.

SecuritySince tokens for either providers (UUID, PKI, or PKIZ) are bearer tokens, they should be protected by usingmutually authenticated TLS. This can be accomplished by using valid PKI transport configuration as described in Changing the default controller keystore and truststore to use CA signed certificates:

The controller must have:

• The valid trusted CA signed identity and CA certificates configured in the controller keystore• The CA certificate must be configured in the controller truststore• The authentication manager ConnSSLClientAuth should be set to true to enforce mutual authentication

Keystone should be configured to:


http://docs.openstack.org/developer/keystone/

http://docs.openstack.org/developer/keystone/

• Use valid trusted CA signed identity certificate• Configure trusted CA to be used to validate client certificates• Require SSL• Require the client certificate to be valid

Role-Based Access Control (RBAC)Aruba VAN SDN Controller supports limited RBAC (Role Based Access Control). The SDN Controller currentlyenforces a single role which has access to all controller features. By default, the single role is sdn-admin. Theauthenticated user must have this role in order to be granted access to the controller. You must ensure thatKeystone is configured to grant this role.

The applications installed on the SDN Controller can enforce RBAC to meet their security requirements.

Assigning a user to a roleTo assign a user the sdn-admin role and give the user access to the desired SDN Controller:

Procedure

1. Create a tenant (the example creates a test tenant):

curl -H "X-Auth-Token:ADMIN" -H "Content-Type: application/json" -d '{"tenant": {"enabled": true, "name": "test-tenant", "description": "Test Tenant"}}' http://<controller-ip>:35357/v2.0/tenants

2. List tenants:

curl -H "X-Auth-Token:ADMIN" http://<controller-ip>:35357/v2.0/tenants3. Create a user:

curl -H "X-Auth-Token:ADMIN" -H "Content-Type: application/json" -d '{"user": {"email": "[email protected]", "password": "somepass", "enabled": true, "name": "test-user", "tenantId": "2c851897a09f483fa452e2de11511f71"}}' http://<controller-ip>:35357/v2.0/users

4. List users:

curl -H "X-Auth-Token:ADMIN" http://<controller-ip>:35357/v2.0/users5. Create a role:

curl -H "X-Auth-Token:ADMIN" -H "Content-Type: application/json" -d '{"role": {"name": "test-role"}}' http://<controller-ip>:35357/v2.0/OS-KSADM/roles

6. List roles:

curl -H "X-Auth-Token:ADMIN" http://<controller-ip>:35357/v2.0/OS-KSADM/roles7. Assign a user to a role:

curl -X PUT -H "X-Auth-Token:ADMIN"; http://<controller-ip>:35357/v2.0/tenants/<tenant-id>/users/<user-id>/roles/OS-KSADM/<role-id>

8. List roles for a user for a given tenant:

curl -X GET -H "X-Auth-Token:ADMIN" http://<controller-ip>/v2.0/tenants/<tenant-id>/users/<user-id>/roles

Example

1. List tenants


root@sdnctl1:/var# curl -H "X-Auth-Token:ADMIN" http://192.168.4.61:35357/v2.0/tenants | python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed100 243 100 243 0 0 38786 0 --:--:-- --:--:-- --:--:-- 40500{ "tenants": [ { "description": "", "enabled": true, "id": "575d62cc28bc403c97409072ba6536d3", "name": "sdn" }, { "description": "Test Tenant", "enabled": true, "id": "fb2f0c68d410440baf67ba134733dbdb", "name": "test-tenant" } ], "tenants_links": []}2. Create a user as part of sdn tenant

root@sdnctl1:/var# curl -H "X-Auth-Token:ADMIN" -H "Content-Type: application/json" -d '{"user": {"email":"[email protected]", "password": "somepass", "enabled": true, "name": "test-user", "tenantId":"575d62cc28bc403c97409072ba6536d3"}}' http://192.168.4.61:35357/v2.0/users{"user": {"username": "test-user", "name": "test-user", "id": "867e7e2e88644e73a4eee25e4b80c303", "enabled": true, "email": "[email protected]", "tenantId": "575d62cc28bc403c97409072ba6536d3"}}root@sdnctl1:/var# curl -H "X-Auth-Token:ADMIN" http://192.168.4.61:35357/v2.0/users | python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed100 297 100 297 0 0 36232 0 --:--:-- --:--:-- --:--:-- 37125{ "users": [ { "enabled": true, "id": "4a4a30ce53b743798dd79d98f5ab7daf", "name": "sdn", "username": "sdn" }, { "email": "[email protected]", "enabled": true, "id": "867e7e2e88644e73a4eee25e4b80c303", "name": "test-user", "tenantId": "575d62cc28bc403c97409072ba6536d3", "username": "test-user" } ]}3. List roles

root@sdnctl1:/var# curl -H "X-Auth-Token:ADMIN" http://192.168.4.61:35357/v2.0/OS-KSADM/roles | python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current


Dload Upload Total Spent Left Speed100 342 100 342 0 0 58481 0 --:--:-- --:--:-- --:--:-- 68400{ "roles": [ { "description": "Default role for project membership", "enabled": "True", "id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_" }, { "id": "1719c0d3b647488da8ca7ff6a1d0288b", "name": "sdn-user" }, { "id": "c105e3dc4a484f5582563e28f7483edc", "name": "sdn-admin" }, { "id": "67eb2907e94d43f7b3e2518485e20bbc", "name": "test-role" } ]}4. Assign user to sdn-admin role for the sdn tenant

root@sdnctl1:/var# curl -X PUT -H "X-Auth-Token:ADMIN" http://192.168.4.61:35357/v2.0/tenants/575d62cc28bc403c97409072ba6536d3/users/867e7e2e88644e73a4eee25e4b80c303/roles/OS-KSADM/c105e3dc4a484f5582563e28f7483edc{"role": {"id": "c105e3dc4a484f5582563e28f7483edc", "name": "sdn-admin"}}5. Verify which roles this user has for the sdn tenant

root@sdnctl1:/var# curl -X GET -H "X-Auth-Token:ADMIN" http://192.168.4.61:35357/v2.0/tenants/575d62cc28bc403c97409072ba6536d3/users/867e7e2e88644e73a4eee25e4b80c303/roles | python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed100 213 100 213 0 0 11715 0 --:--:-- --:--:-- --:--:-- 12529{ "roles": [ { "description": "Default role for project membership", "enabled": "True", "id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_" }, { "id": "c105e3dc4a484f5582563e28f7483edc", "name": "sdn-admin" } ]}

API access requires authenticationTo authenticate, one needs to present username , domain, and password to the /auth API as below (using curlas an example):

curl -sk -H 'Content-Type:application/json' -d '{"login":{"user":"sdn","password":"skyline","domain":"sdn"}}'https://<controller-ip>:8443/sdn/v2.0/auth


Credential information (user name, password, domain, and authentication tokens) used in curlcommands might be saved in the command history. For security reasons, Hewlett PackardEnterprise recommends that you disable command history prior to executing commands containingcredential information.

The above call returns this example JSON data structure that includes the authentication token,which, by default, expires in one hour:

{ "record": { "domainId": "62e312edff47413fad7e1d7fa6ac7bc7", "domainName": "sdn", "expiration": 1377917359000, "expirationDate": "2013-08-30 19-49-19 -0700", "token": "54a6f80a9ae243db89bfa05de4ced51d", "userId": "bca3dea8a28b457e99e899ae16b79634", "userName": "sdn" "roles":["sdn-user","sdn-admin"], }}

Please guard this token information, as it can be used as an API key to gain access to your controllerREST APIs.

To gain access to the REST API, include the token in the X-Auth-Token header as in the following curl example:

curl -sk -H "X-Auth-Token:54a6f80a9ae243db89bfa05de4ced51d" https://<controller-ip>:8443/sdn/v2.0/systemsOne can continue using the same token for different SDN Controller APIs within the default one hour period sincetoken creation. If desired, one can change this default one hour timeout in the /etc/keystone/keystone.conf file. (See the OpenStack Keystone Administration Guide for more information).

Service and admin tokensThe Service token is used for internal communication between controllers and is not exposed to the user. TheAdmin token is used for communication between controller and the Keystone server and is not exposed to theuser.

The values for these tokens can be seen in the controller UI in the Configurations screen in the System tabunder the com.hp.sdn.adm.auth.impl.AuthenticationManager component. All controllers in a teammust have the same Service token to communicate successfully. For the Admin token, both the controller tokenvalue and the Openstack Keystone admin_token in the directory /etc/keystone/keystone.conf must match forsuccessful authentication.

Controller code verificationAll controller code is signed by Hewlett Packard Enterprise. Validating the certificate via jarsigner should return aHewlett Packard Enterprise X.509 certificate similar to the following:

X.509, CN=Hewlett-Packard, OU=HPGlobal, OU=Digital ID Class 3 - Java ObjectSigning, O=Hewlett-Packard, L=Andover, ST=Massachusetts, C=US[certificate is valid from 11/14/12 4:00 PM to 11/15/14 3:59 PM]


X.509, CN=VeriSign Class 3 Code Signing 2010 CA, OU=Terms of use at https://www.verisign.com/rpa (c)10, OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US[certificate is valid from 2/7/10 4:00 PM to 2/7/20 3:59 PM][CertPath not validated: null]If a controller jar or war file is tampered with, the jar verification fails, and the container does not start up.

If an application is not signed by Hewlett Packard Enterprise, or has its certificate trusted by the controller (seesection below), the application is not allowed to run on the controller.

Adding certificates to the jar-signing truststoreTo deploy other signed applications onto the controller, use the Java keytool to import the public certificate thatwas used to sign the application jars and/or zips into the controller jar-signing truststore (/opt/sdn/admin/sdnjar_trust.jks):

/opt/sdn/openjdk8-jre/bin/keytool -importcert -keystore /opt/sdn/admin/sdnjar_trust.jks -file signed_app.cer -alias mysignedcertThe controller needs to be restarted for the new truststore to take effect.

Running the SDN controller without Jar-Signing validationThe SDN Controller enforces jar/zip-signing validation by default. For an experimental or developmentenvironment, where unsigned applications need to be deployed, jar/zip-signing validation can be turned offaltogether:

Procedure

1. Stop the SDN Controller:

sudo service sdnc stop2. Modify the /opt/sdn/virgo/bin/dmk.sh script to add the following option to the list of JMX_OPTS:

-Dsdn.signedJar=none

For example:

cd $KERNEL_HOME; exec $JAVA_EXECUTABLE \$JAVA_OPTS \$DEBUG_OPTS \$JMX_OPTS \-XX:+HeapDumpOnOutOfMemoryError \-XX:ErrorFile=$KERNEL_HOME/serviceability/error.log \ -XX:HeapDumpPath=$KERNEL_HOME/serviceability/heap_dump.hprof \-Dsdn.signedJar=none \-Djava.security.auth.login.config=$AUTH_LOGIN \-Dorg.eclipse.virgo.kernel.authentication.file=$AUTH_FILE \

3. Start the SDN Controller:

sudo service sdnc startTo enable jar/zip-signing validation, remove the line containing the -Dsdn.signedJar=none option fromthe /opt/sdn/virgo/bin/dmk.sh script and restart the controller.

Revoking Trust


Revoking trust via truststoreThe controller components rely on the public certificates in the respective truststore to establish trust with a givenidentity. Therefore, revoking trust from a client with a given public certificate amounts to removing its certificatefrom the respective truststore. To remove a given certificate from the truststore:

• List the certificates in your truststore:/opt/sdn/openjdk8-jre/bin/keytool –list –v -keystoretruststore [-storepass password]

• Delete certificate from truststore:/opt/sdn/openjdk8-jre/bin/keytool –delete –alias cert-aliastruststore

Revoking trust via CRLFor the controller’s REST API, a CRL (Certificate Revocation List) might also be specified to allow blacklisting ofcertain clients. This is done by modifying the /opt/sdn/virgo/configuration/tomcat-server.xml file toinclude the CRL file location in the SSL connector:

<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"ciphers="TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA"maxThreads="150" scheme="https" secure="true"clientAuth="false" sslEnabledProtocols="TLSv1,TLSv1.1,TLSv1.2"keystoreFile="../admin/keystore"keystorePass="skyline"/>For the change to take effect, restart the controller.

SDN administrative REST APIThe main SDN Controller daemon (sdnc) is accompanied by an ancillary daemon process (sdna), which runsunder user sdnadmin in order to grant it access to some elevated privileges.

The administrative REST API can be used to securely perform various management functions in a privilegedcontext. It would be undesirable for the main SDN Controller process to possess those privileges as it might behosting execution of third-party code.

The SDN Administrator daemon can be accessed via the REST API via HTTPS on port 8081. The access issecured through either token-based authentication or basic authentication, against the locally running Keystoneserver, which is the same as the main SDN Controller REST API.

The following set of features are accessible through the administrative REST API:

• SDN Controller daemon (sdnc) stop/start/restart• Adding/removing the team leader IP alias (required only when in team mode)• Configure iptables rules to protect team communication

If the iptables rule programming for Cassandra fails, the Cassandra server will not come up. Inprevious releases, the server would come up regardless of the iptables rule programming.

• Downloading the ZIP bundle of log files• Uploading upgrade Debian bundles and installing/removing Debian packages• Uploading upgrade ZIP bundles and executing upgrade commands• System reboot

The install process adds a number of sudoers entries for the sdnadmin user. These are as follows:

• /sbin/ifconfig• /sbin/reboot• /sbin/iptables


• /usr/bin/service• /usr/bin/at• /usr/bin/dpkg• /usr/sbin/arp• /usr/bin/arpingAll, or any, of the above entries can be blocked or removed from the sudoers configuration. The /sbin/ifconfig entry is only required when running in teamed mode. Otherwise the controller cannot migrate the teamIP address from node to node as team leader changes. The /sbin/iptables is also required in teamed mode tosecure team communication.

The sdna process must be present and active for the SDN controller to function. The sdnc process will not startwithout sdna.

Virgo admin UI access via localhost onlyYou can access the Virgo admin UI by using a remote browser at https://<ip address>:8443/admin. Thisshould not be used under normal circumstances, but can be useful for debugging purposes.

To change the credentials of this console, get root console access to the machine(s) running the Aruba VAN SDNController and edit the following file:

/opt/sdn/virgo/configuration/org.eclipse.virgo.kernel.users.propertiesThis file includes the following two entries:

user.admin=sdnrole.admin=adminwhere role.admin defines the user and user.admin defines the password. This file needs to be owned byuser:sdn, group:sdn. Changes to this file require a restart of the controller to recognize the new credentials.

To disable access to the Virgo Admin UI, either remove the following file or move it to a safe location outside thepickup directory.

Virgo console access disabled by defaultThe Virgo console is disabled by default as it is not security hardened. If you choose to enable it for debuggingpurposes, make sure you are in a trusted environment and disable it as soon as possible.

To access the Virgo Admin WEB UI (GUI), copy theorg.eclipse.virgo.management.console_3.6.2.RELEASE.jar file from the /opt/sdn/admindirectory to the /opt/sdn/virgo/pickup directory. You must either be the sdn user on the SDN controller tocopy the file or change the file ownership to sdn once it is copied.

JMX console enabled for local access onlyThe JMX console is only enabled for local access. This is used by the controller for metering and can also beused for debugging. The JMX console is not security hardened and should be enabled for remote access only intrusted environments.

To enable JMX console remote access, edit /opt/sdn/virgo/bin/dmk.sh. The following line determineswhether JMX allows remote access or not, in this case indicating local only access.

-Dcom.sun.management.jmxremote.local.only=true \Any changes to this file require a controller restart to recognize the change.


Creating the Cassandra keystore and truststoreTo create the keystore and truststore:

Procedure

1. Log in to the system running the SDN Controller and stop the controller.2. As the sdn user (for example, su - sdn or sudo -i -u sdn), back up your default /opt/sdn/

cassandra/conf/.keystore and /opt/sdn/cassandra/conf/.truststore to a safe location.3. Create a new keystore using the following commands (note the default password here is skyline):

cd /opt/sdn/cassandra/conf

rm .keystore .truststore

/opt/sdn/openjdk8-jre/bin/keytool -genkey -alias serverKey -keyalg rsa -keysize2048 -keystore .keystore

You must specify a fully-qualified domain for your server for the "first and last name" question assome CAs, such as VeriSign, expect it.

4. Generate a CSR (Certificate Signing Request) for signing:

/opt/sdn/openjdk8-jre/bin/keytool -keystore .keystore -certreq -alias serverKey-keyalg rsa -file sdn-server.csr

5. Send the sdn-server.csr to a CA to be signed.

The CA authenticates you and returns a signed certificate and its CA certificate chain. This procedureassumes that the signed certificate from the CA is named signed.cer and the CA's certificate isroot.cer.

6. Import the signed root certificate into your keystores:

In a team, you must add the certificate (and chain) from each other controller tothe .truststore of all the other members on the team.

/opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore .keystore-file root.cer -alias CARoot

/opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore .truststore -file root.cer -alias CARoot

7. Replace your self-signed certificate in your serverKey entry with the signed certificate from your CA(signed.cer):

/opt/sdn/openjdk8-jre/bin/keytool -importcert -keystore .keystore -filesigned.cer -alias serverKey

8. Log in to the controller UI:

https://controller_ip:8443/sdn/ui9. Select Configurations on the left navigation pane, select the System tab and then select the

com.hp.sdn.teaming.impl.CassandraProcessManager component.10. Select Modify on the top. When the Modify System Configuration dialog box opens, update the location and

password of the new keystore. Click Apply.11. Restart the controller.


Cassandra keystore and truststore locations andpasswordsThe Cassandra keystore and truststore are referenced by thecom.hp.sdn.teaming.impl.CassandraProcessManager component. To change the passwordskeystore.password and truststore.password of this component:

Procedure

1. From the controller UI, select Configurations, then select the System tab.2. Select the com.hp.sdn.teaming.impl.CassandraProcessManager component.3. Select Modify.

The stores are located in /opt/sdn/cassandra/config/.keystore and /opt/sdn/cassandra/config/.truststore. Only a trusted authority should sign the certificates. You must install each of thecertificates in the .truststore file of all of the nodes in the team.

Security procedureHewlett Packard Enterprise strongly recommends that you change all default credentials to preventexposure of access to the controller.

Change the values from the defaults for the following:

• Keystore password• Truststore password• Admin Token• Service Token• Jarsigning

Procedure

1. Log into http://<controller_IP>:/8443/sdu/ui as the sdn user.2. From the controller UI, select Configurations. Then select the System tab.3. Select the component com.hp.sdn.adm.auth.impl.AuthenticationManager.4. Select Modify and change the default values for the following keys to the newly chosen credentials:

• Set the AdminToken key to the newly chosen Keystone (authentication) admin token.• Set the ServiceToken to the newly chosen internal communication secret.• Set the KeystorePass to the value that you will be using to secure the SSL Keystore.• Set the TruststorePass to the value that you will be using to secure the SSL Truststore.

5. Specific to Keystone, set the ConnSSLClientAuth key to true.6. Update the Keystone admin token in the file etc/keystone/keystone.conf.

a. Change the admin token from the default admin_token=ADMIN to admin_token=<newAdminToken>Where the <newAdminToken> is the newly chosen AdminToken value you entered in step 4. If the line iscommented out, remove the # sign in front of the line.

b. Restart the Keystone service (sudo service keystone restart).7. Update the keystore password to match the newly chosen KeystorePass value you entered in step 4 using

the following:

/opt/sdn/openjdk8-jre/bin/keytool -storepasswd -storepass <OldKeystorePass> -new<newKeystorePass> -keystore /opt/sdn/admin/keystore


8. Update the keystore’s internal serverkey to match the newly chosen KeystorePass value you entered instep 4 using the following:

/opt/sdn/openjdk8-jre/bin/keytool -keypasswd -alias serverkey -storepass<newKeystorePass> -keypass <oldKeystorePass> -new <newKeystorePass> -keystore /opt/sdn/admin/keystore

9. Update the truststore password to match the newly chosen TruststorePass value you entered in step 4using the following:

/opt/sdn/openjdk8-jre/bin/keytool -storepasswd -storepass <oldKeystorePass> -new<newKeystorePass> -keystore /opt/sdn/admin/truststore

10. Update the jar signing keystore password (named sdnjar_trust.jks):a. Use the keytool command to assign a new password. The default or old keystore password is skyline.

/opt/sdn/openjdk8-jre/bin/keytool -storepasswd -storepass <oldKeystorePass> -new <newpass4sign> -keystore /opt/sdn/admin/sdnjar_trust.jks Where<newpass4sign> is a newly chosen password. This new password does not have to match the others.

b. Update the dmk.sh to provide the new password as an environment variable for the running controller.

• Navigate to the /opt/sdn/virgo/bin directory as the sdn user.• Open the dmk.sh file to edit.• In the dmk.sh file, find the line containing XX:HeapDumpPath....• After the XX:HeapDumpPath... line, add a new line-Dsdn.trustpass=<newpass4sign>• Save the dmk.sh file.

c. Restart the sdnc service (sudo service sdnc restart) for the modified password to be read by thecontroller.

When you have completed security configuration, restart Keystone service and restart the controller. Be sure toremove the visible passwords from the shell history.

Security best practicesObserving these rules can help to prevent unauthorized access to the controller:

• Do not enable shell history on your controller.• Do not allow other users besides sdn, sdnadmin and the Linux user to have access to your controller system.• Do not store your authentication token in plain text, such as a non-encrypted cookie.• Do not use self-signed certificates in a production environment.• Do not alter contents under /opt/sdn/Cassandra and /opt/sdn/Hazelcast.• To prevent authentication tokens from being stolen:

◦ Always log out of the UI and close the web page, when you are done using it.◦ Never leave a browser window open and unattended when you are accessing the UI.◦ Never let someone who does not have access rights to the controller ‘look over your shoulder’ while

accessing the UI.◦ Make sure Keystone is configured to expire tokens after a short period of time (a common industry practice

is 20 minutes).• Do not delete any iptables with the name hazelcast, cassandra-default, or cassadra-team, or any

rules with the following ports: 5700, 7000, 7001, 7199, 9160.• Do not manually override the iptables rules to allow or deny ports 5700, 7000, 7001, 7199, and 9160.

Rules created for a team of 3 controllers on page 131 displays the rules created for a team of 3 controllers(1.2.1.1, 1.2.1.2, 1.2.1.3) when running the sudo iptables -nL command:


Rules created for a team of 3 controllers

# sudo iptables -nLChain INPUT (policy ACCEPT)target prot opt source destination cassandra-team all -- 0.0.0.0/0 0.0.0.0/0 cassandra-default all -- 0.0.0.0/0 0.0.0.0/0 hazelcast all -- 0.0.0.0/0 0.0.0.0/0

Chain FORWARD (policy ACCEPT)target prot opt source destination

Chain OUTPUT (policy ACCEPT)target prot opt source destination cassandra-team all -- 0.0.0.0/0 0.0.0.0/0 cassandra-default all -- 0.0.0.0/0 0.0.0.0/0 hazelcast all -- 0.0.0.0/0 0.0.0.0/0

Chain cassandra-default (2 references)target prot opt source destination ACCEPT tcp -- 127.0.0.1 127.0.0.1 tcp dpt:7001ACCEPT tcp -- 127.0.0.1 127.0.0.1 tcp dpt:9160ACCEPT tcp -- 127.0.0.1 127.0.0.1 tcp dpt:7199DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:7199DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:9160DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:7000DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:7001

Chain cassandra-team (2 references)target prot opt source destination ACCEPT tcp -- 1.2.1.2 1.2.1.3 tcp dpt:7001ACCEPT tcp -- 1.2.1.3 1.2.1.2 tcp dpt:7001ACCEPT tcp -- 1.2.1.1 1.2.1.3 tcp dpt:7001ACCEPT tcp -- 1.2.1.3 1.2.1.1 tcp dpt:7001ACCEPT tcp -- 1.2.1.3 1.2.1.3 tcp dpt:7001

Chain hazelcast (2 references)target prot opt source destination ACCEPT tcp -- 1.2.1.2 1.2.1.3 tcp dpt:5700ACCEPT tcp -- 1.2.1.3 1.2.1.2 tcp dpt:5700ACCEPT tcp -- 1.2.1.1 1.2.1.3 tcp dpt:5700ACCEPT tcp -- 1.2.1.3 1.2.1.1 tcp dpt:5700DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:5700


Configuring OpenFlow Instances with Multiple VLANsHewlett Packard Enterprise recommends that the OpenFlow instance VLAN membership be identical throughoutthe controlled network topology. If an OpenFlow instance contains a set of VLANs on one switch, thenneighboring switches should also have an OpenFlow instance with the same set of VLANs. For networktopologies that contain both ProVision and Comware OpenFlow instances, configure the ProVision aggregatemode and Comware Openflow instances to contain all VLANs.

You should untag inter-switch links in the OpenFlow instance VLANs. Alternatively, you can tag these links in theOpenFlow instance VLANs. Default the untagged traffic to one of the OpenFlow instance VLANs.

Hewlett Packard Enterprise recommends the following mixed ProVision/Comware topology switch configuration:

• ProVision – ProVision

Instance: Aggregate OpenFlow instance on all switches or virtualized only instances on all switchesInterface:Any valid configuration

• ProVision – Comware

Instance: ProVision Aggregate, Comware group = allInterface: Comware interswitch link port PVID inOpenFlow instance

• Comware – Comware

Instance: Consistent OpenFlow instance VLAN configuration throughout topologyInterface: Interswitch link portPVID in OpenFlow VLAN

Configuring OpenFlow Instances with Single VLANIdentifierIn a topology that contains both Comware and ProVision switches, if the ProVision switch OpenFlow instance isconfigured for a single VLAN, configure the Comware OpenFlow instance to a single VLAN as well.

In some cases, when the ports of the inter-switch link between the ProVision and Comware devices areconfigured as tagged-only, the link from the Comware device to the ProVision device may not be discovered. Inorder for links to be correctly discovered in these configurations, configure the controller to insert a VLAN tagwhen injecting link discovery packets on the Comware device. Use the device REST API to configure theappropriate linkDiscoveryVlan that matches the VLAN identifier configured in the Comware device OpenFlowinstance.

Configuring linkDiscoveryVlan to discover all links from the Comware devices

In this example, the topology consists of a ProVision 3800 on port 6 connected to a Comware 5500HI on portGigabitEthernet1/0/14:

Chapter 8Configuring OpenFlow instances


The ProVision 3800 has the following configuration:

openflow controller-id 1 ip 172.17.8.129 controller-interface vlan 800 instance "1" member vlan 40 controller-id 1 version 1.3 enable exit enable exitvlan 1 name "DEFAULT_VLAN" no untagged 6,25,48 untagged 1-5,7-24,26-47,49-52 ip address dhcp-bootp exitvlan 40 name "VLAN40" tagged 6 no ip address exitThe Comware 5500HI has the following configuration:

interface GigabitEthernet1/0/14 port link-mode bridge port link-type trunk undo port trunk permit vlan 1 port trunk permit vlan 40#openflow instance 1 controller 1 address ip 172.17.8.129 classification vlan 40 loosen#

Chapter 8 Configuring OpenFlow instances 133

Initially, the REST API reports the following devices:

Initially, the REST API reports only one link; the link between the Comware 5500Hl and the ProVision 3800:

The Rest API reports the link discovered when the controller injected link discovery packets to the ProVision 3800OpenFlow instance. The ProVision switches insert a VLAN tag in packets injected by the controller when theegress ports are configured as tagged only. The 5500 port receives the properly tagged packet and forwards it tothe controller, resulting in the discovered link.

The corresponding link from the 3800 to the 5500 is missing. The controller-injected link discovery packets sent toport GigabitEthernet/1/0/14 on 5500 OpenFlow instance (the device with DPID 00:01:cc:3e:5f:6a:d3:80) were nottagged, therefore they were dropped by the receiving 3800 whose port 6 is configured for tagged traffic.

Solution

To ensure link discovery packets are tagged when sent out by tagged Comware device ports, configure the devicelinkDiscoverVlan value for the Comware device with the tagged-only port using the device REST API. For


example, in the above topology, configure the 5500 device to set the linkDiscoveryVLAN to 40 (the VLANconfigured in the OpenFlow instances):


The REST API query now reports that both links are discovered:

When you configure a linkDiscoveryVlan for a device, the controller will always insert a 802.1Qheader with the configured VLAN on each link discovery packet sent to all ports of the device,regardless of the actual port configurations, as the controller does not have any knowledge of theport configuration.

Configuring OpenFlow instances to enable MAC groupmatching

MAC group matchingBy supporting MAC group matching and MAC group tables, an OpenFlow instance can store flow entries thatmatch MAC groups instead individual MAC IDs, freeing up space in the policy (TCAM) table of the physical switchfor other flow entries. The list of the MAC addresses in the MAC group are defined in other tables (40 for sourceand 41 for destination).

For example, to use MAC addresses to block access to the network for 10 devices owned by a particular group ofusers:

• Without support for MAC group matching, you would have to create a flow entry in the policy table of theswitch for each MAC address, for a total of 10 flow entries.

• With support for MAC group matching and MAC group tables, you can create a MAC group, add the MACaddresses to the MAC group, and then create a single flow entry in the policy table to match that MAC groupID. The list of the MAC addresses in the MAC group are defined in other tables (40 for source and 41 fordestination).

For more information about programming flow entries for OpenFlow instances, see the following documents:


• Aruba VAN SDN Controller Programming Guide• Aruba VAN SDN Controller REST API Reference

Switches that support MAC group tables and MAC group matchingBeginning with version 2.6, the Aruba VAN SDN Controller supports the use of MAC groups and MAC tables forOpenFlow v1.3 instances on ProVision-based switches running release K/KA/KB/WB 15.18 or later with thefollowing restrictions:

• Support is provided for V2 and V3 modules only. MAC group tables are not supported when the switch is in V1module compatible mode.

• The 2920 switch does not support destination MAC address matching or destination MAC group matching.

Configuration rules for OpenFlow instances and MAC groups• By default, MAC group tables are disabled. You must enable MAC group tables to use them.• MAC group matching is supported in the standard match mode default pipeline only.• Exactly one OpenFlow instance per physical switch can be enabled with MAC group match support.• All OpenFlow instances on the physical switch must be disabled before you can enable the MAC group

feature.• MAC group tables do not support counters or idle-timeout metering. For example, if you want to count the

number packets that match a MAC group, you must create a separate flow entry.• The policy table cannot match on a source MAC and destination MAC address separately when MAC group

tables are part of the pipeline.

Enabling or disabling MAC group matching on an OpenFlow instance

Prerequisites (SDN controller 2.8)

The default domain and user settings are sdn. The default password setting is skyline.


Procedure

1. Install and start three standalone controllers in the network. (See the latest Aruba VAN SDN ControllerInstallation Guide.)

2. Optional: To improve security, you can change the username and password from the default settings on eachof the standalone controllers in step 1.

3. Select any one of the controllers to use for configuring the team.4. On the selected controller, acquire an Authentication Token. Use the following curl command, with the

controller IP address, to acquire the token:

curl --noproxy controller_ip> -X POST --fail-ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/auth"-H "Content-Type: application/json" --data-binary '{"login":{"domain": "<domain>","user": "<user>","password": "<password>"}}'In this example, the default domain, user name, and password are used.


curl --noproxy 192.15.135.187 -X POST --fail -ksSfL --url "https://192.15.135.187:8443/sdn/v2.0/auth" -H "Content-Type: application/json" --data-binary '{"login": {"domain": "sdn","user": "sdn","password": "skyline"}}'The command generates the authentication token 1759f214479e4ffd9504acb42123ef40.

{"record":{"token":"1759f214479e4ffd9504acb42123ef40","expiration":1381982391381982399000,"expirationDate":"2014-10-16 20-59-59 -0700","userId":"b00cb0e94c9441d58011f980cf9635ae","userName":"sdn","domainId":"a6701f6593d84fa5b8f23f9ab4ed69db","domainName":"sdn"}}

5. Determine the team configuration parameters:

Parameter Value

Team IP Address The team IP address is different from the individualcontroller IP addresses. It is used as a virtual address forconnecting to the team leader.

Controller IP Address IP address of a team member.

When the virtual address is programmed on the team leader, gratuitous ARP is sent out. Thevirtual address moves from one controller to another in the team as the leader changes. If anysecurity features are configured to prevent such a move, they should be reconfigured to allow themovement of team IP Address such that it remains reachable for the rest of the network.

Enabling MAC groups

Procedure

1. To enable source MAC groups, enter the following command, where instance-name is the name of theOpenFlow instance for which you are enabling MAC groups:

openflow instance instance-name src-mac-grp-table.2. To enable destination MAC groups, enter the following command, where instance-name is the name of the

OpenFlow instance for which you are enabling MAC groups:

openflow instance instance-name dst-mac-grp-table.

Disabling MAC groups

Procedure

1. To disable source MAC groups, enter the following command, where instance-name is the name of theOpenFlow instance for which you are disabling MAC groups:

2. openflow instance instance-name no src-mac-grp-table.3. To disable destination MAC groups, enter the following command, where instance-name is the name of the

OpenFlow instance for which you are disabling MAC groups:4. openflow instance instance-name no dst-mac-grp-table.

Disabling MAC groups

Procedure

1. To disable source MAC groups, enter the following command, where instance-name is the name of theOpenFlow instance for which you are disabling MAC groups:


openflow instance instance-name no src-mac-grp-table.2. To disable destination MAC groups, enter the following command, where instance-name is the name of the

OpenFlow instance for which you are disabling MAC groups:

openflow instance instance-name no dst-mac-grp-table.


This chapter describes controller backup and restore actions using curl commands. For information about theREST APIs related to backup and restore, see /restore and /backup in the RSdoc facility on the controller.Using a Google Chrome browser window on the controller, enter:

https://system_ip_address:8443/api

Backing up and restoring Best Practices• You cannot use RSdoc to download or upload files.• Only one backup, restore, upload, or download operation can be active at any time on a given controller or

controller team. Parallel operations are not supported.• Use standard VM tools (such as Snapshot or Clone) to back up and restore the entire controller image.• Use standard Linux server-based tools (such as rsync, LVM snapshot, and Amanda/Zmanda) to back up and

restore the controller on bare metal.• If you change the name of any of the following files, the controller backup will not be able to backup the

renamed file. So before the backup you should manually back up the renamed files. Then after a restore, stopthe controller service, copy the renamed files to the appropriate location and restart the controller.

◦ /opt/sdn/admin/keystore◦ /opt/sdn/admin/truststore◦ /opt/sdn/admin/sdnjar_trust.jks

Backing up a controllerA controller backup takes a snapshot of the controller state, and includes the following in a single file:

• Controller databases• License compliance history and metrics log data• In a teaming environment, the teaming configuration• User repository folder (for user-installed applications)• Controller configuration folder• Application data for applications that have implemented backup/restore functionality.

The controller does not back up the sdnjar_trust.jks file or existing keystore and truststore filesin the /opt/sdn/admin directory. If you have CA signed certificates or you have added third partyapplications to the controller, you must back these files up separately and recover them afterrestoration.

For backing up and restoring, Hewlett Packard Enterprise recommends that you use off-the-shelf solutions suchas rsync or Amanda. If your applications use the controller's backup and restore service and you are usingCassandra, then consider the following:

Chapter 9Backing up and restoring


• To back up Cassandra, set the backupLockSeconds configurable Cassandra lock timer parameter to thesize of the data being backed up, the default is 10 minutes. Depending on the size of the backup, Cassandracould be locked for up to 10 minutes.

• If the applications that are using Cassandra encounter a lock timed out failure during the backup, increase thetimer. The failure will generate an error message Cassandra lock timed out before backup wasfinished. in the /var/log/sdn/virgo/logs/log.log file.

• To configure the backupLockSeconds parameter:

1. Login to the GUI.2. Click Configurations.3. From the list of configurations, click com.hp.sdn.teaming.impl.CassandraProcessManager.4. Click Modify.

See Configuration components on page 37.

Examples of curl commands in this guide use the --noproxy option, which is appropriate where execution ofcurl commands do not need a proxy to access controllers. If your network is set up such that a proxy is needed toaccess controllers, use the --proxy option. For details on curl proxy options, visit http://curl.haxx.se/docs/manpage.html.

Backup operationA controller backup includes the controller configuration and databases in one *.zip file.

• Backups run in the background, and, except for locking the Cassandra database to prevent writes, do notinterrupt system operation.

• Whether operating in a team or operating in standalone mode, each controller is backed-up as a singlesystem.

• When the controller is deployed in a VM, standard VM backup/restore tools (such as Snapshot or Clone) canbe used.

• When the controller is deployed on bare metal, standard Linux server-based backup/restore tools (such asrsync, LVM snapshot, and Amanda/Zmanda) can be used.

• To complete a teamed backup, no controller can be in a failed state. (A controller team must have threecontrollers.)

• On any controller or controller team, only one operation can run at any given time (backup, restore, upload, ordownload). Also, starting a new backup while another backup is being downloaded creates an error conditionand halts the new backup.

• Only authenticated users are allowed to create and restore backups. In some cases the domain name is alsorequired.

The default domain name is sdn. The default username is also sdn. The default password isskyline.

The controller does not save a non-default domain, user name, or pass-word across a backup.Changing these settings to non-default values and later backing up the controller, resets thesesettings to their defaults in the backup file. Later restoring the backup to the controller resets thedomain, user name, and password to their default settings in the controller.

For backup and restore of the Keystone configuration and database, see Backing up and restoring theKeystone configuration and database on page 148.

• If uploading a backup fails, then no backup version remains on the system.• Starting a new backup replaces any earlier backup remaining in the controller. If a backup is being downloaded

when a new backup is started, the new backup halts.• Metering time-series data is not encompassed by the controller backup process. There can be a large amount

of data, possibly tens of gigabytes in size, which is keyed to time. Not only is the time series data impractical toback up because of its size, but upon restoring it there is a likelihood that some of the restored data will not be

Chapter 9 Backing up and restoring 141

http://curl.haxx.se/docs/manpage.html


usable because it will be older than the sliding window of time that metrics are retained for on the controller.However, there is one metering file that is backed up and restored. It contains a mapping of metric descriptorinformation (such as the ID of the application that created a metric and the metric's primary tag, secondary tag,and name) to the UID that was assigned to each metric. When a restore is performed, this file is restored, andany existing metering time-series data is deleted because it might not match the restored file. The mappingsthat are restored might, depending upon time elapsed since the backup was taken, be used to assign thesame UID to a metric created following the restore (and subsequent controller restart) that was assigned to themetric before the backup was taken. This provides continuity for a metric across the time spanned betweenbackup and restore because all data for the metric is keyed to the same UID. Thus, while time-series data frombefore the restore was not retained during the restore, UIDs used to key time-series data that was exported toexternal tools or storage before the restore will continue to be used for the same metrics.

Backing up a controller

Procedure

1. 1. Acquire the authentication token for the controller backup:

curl --noproxy controller_ip -X POST --fail -ksSfL --url "https://controller_ip:8443/sdn/v2.0/auth" -H "Content-Type: application/json" --data-binary '{"login": {"domain": "domain","user": "user","password": "password"}}'

Credential information (user name, password, domain, and authentication tokens) used in curlcommands might be saved in the command history. For security reasons, Hewlett PackardEnterprise recommends that you disable command history prior to executing commandscontaining credential information.

2. If needed, increase the Cassandra backupLockSeconds configuration using the GUI. See Configurationcomponents on page 37.

3. Acquire the controller uid:

curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -ksSfL --request GET--url "https://controller_ip:8443/sdn/v2.0/systems"

4. Set the IP address of the controller using the following curl command:

curl --noproxy controller_ip> --header "X-Auth-Token:auth_token" --fail -ksSfL --request PUT "https://controller_ip:8443/sdn/v2.0/systems/controller_uid" --data-binary '{"system":{"ip":"controller_ip"}}'

5. Perform the actual backup using the following curl command:

curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -ksS --request POST --url "https://controller_ip:8443/sdn/v2.0/backup"


6. Get the checksum to verify the backup file has not been corrupted.

curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -ksS --request GET --url "https://controller_ip:8443/sdn/v2.0/backup/checksum"

7. Check on the status of a backup.

curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -ksSfL --request GET--url "https://controller_ip:8443/sdn/v2.0/backup/status"

Downloading a backup from the controller to another locationThe backup file should be downloaded to a secure location. Choose the correct name now; you cannot renamethe files later or you will get a file corruption error when you attempt to upload it for a restore.

The file name must begin with sdn_controller_backup.

Procedure

1. Download the Backup.zip File:

curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -ksSfL --request GET --url "https://system_ip:8443/sdn/v2.0/backup>path-and-file-name.zip"

Recommended backup practices• Do not run backup while making configuration changes. Instead, run the backup after completing configuration

changes. Otherwise, an inconsistent system state could result with a subsequent restore.• Always back up all of the controllers in a team after a configuration change. Just backing up a subset of the

controllers is not sufficient.• Back up all of controllers in a team at approximately the same time. (Team backups can be in sequence or in

parallel). Do not allow days to pass in-between backups of different controllers in the same team.• If any controller in a team fails to complete the backup, start the backup over for all members of the controller

team.• A completed backup should be downloaded from the controller to another location for safekeeping. Include the

IP address in the backup filename, so you can easily determine which backup belongs to which controller in ateam. Recommended file naming is:

◦ sdn_controller_backup_ip-address.zip• Store the backup files you take off each controller in the team together, so they can easily be retrieved for a

future restore.

Restoring a controller from a backupProcedure

1. Uninstall the controllers to be restored. If this is a rollback to a previous state, uninstall all controllers.2. Set CTL_RESTORE_INSTALL_MODE=True in the ~/.sdn_install_options file in the home directory.


a. If this file is not present in the directory, create it with the CTL_RESTORE_INSTALL_MODE entry.b. If the file is already present, ensure that it includes the CTL_RESTORE_INSTALL_MODE entry.

This entry directs the installer to perform the necessary changes to direct the controller to start in recovery/restore mode, during which OpenFlow activity is suspended for the subject controller.

3. Re-install the failed controllers, making sure to use the same IP address configuration. During the re-installation, log messages similar to the following appear in the Audit Log:

root@mak:~/dev/controller/dist# sudo dpkg -i hp-sdn-ctl_1.11_amd64.debSelecting previously unselected package hp-sdn-ctl.(Reading database ... 212350 files and directories currently installed.)Unpacking hp-sdn-ctl (from hp-sdn-ctl_1.11_amd64.deb) ...Setup has detected a compatible jre-headless - 1.7.0_25 Creating system group 'sdn'... ...done. Creating system user 'sdn'......done. Creating system user 'sdnadmin'......done. Configuring PostgreSQL database...* Restarting PostgreSQL 9.1 database server [ OK ]...done.Adding SDN-related items to Keystone... keystone stop/waitingkeystone start/running, process 11514...done.Setting up hp-sdn-ctl (1.11) ... Certificate was added to keystoreCTL_RESTORE_INSTALL_MODE option is set SDN controller will be started in restore modesdna start/running, process 11633 sdnc start/running, process 11636Processing triggers for ureadahead ...

Do not re-install any applications before you complete the restore process. The restoration addsdata from the backup file into the current database contents. If you re-install applications thatare part of the controller backup, then those applications might end up with duplicate orconflicting entries in their database. If required, only re-install applications after you havecompleted all steps of the restore process.

4. Edit the /etc/sudoers file to add password-less access for the sdn user to run the required restore scripts:a. Open the /etc/sudoers file with this command: sudo /usr/sbin/visudob. At the end of the file, add the following line:

sdn ALL=(sdnadmin) NOPASSWD: /opt/sdn/cassandra/bin/caServer.shc. Write out the file by entering CTRL-O, then enter CTRL-X to exit. If there are parse errors when you try to

exit, the visudo program displays a warning and a syntax error. Enter e at the prompt to edit the file tocorrect the error.

Do not exit the file until all parse errors are corrected. If the /etc/sudoers file containsparse errors, no users can use sudo to execute commands.

5. Edit the /opt/sdn/backup/Restore.sh file:


a. On or near line 62 of the script, locate the command and change it to the following:

sudo -u sdn rm -rf "$OPT_ROOT/cassandra/conf/"

rm -rf "$OPT_ROOT/cassandra/conf/"b. On or near line 72 of the script, locate the command and change it to the following:

sudo -u sdnadmin rm -rf "$OPT_ROOT/cassandra/commitlog/"

rm -rf "$OPT_ROOT/cassandra/commitlog/"c. On or near line 79 of the script, locate the command and change it to the following:

sudo -u sdn rm -rf "/var/lib/sdn/cassandra/"

rm -rf "/var/lib/sdn/cassandra/"

The following listing shows the corrected code snippets:

59 #pick the cassandra zip file and unpack60 restore_log "Deleting cassandra configuration"61 if [ -d "$OPT_ROOT/cassandra/conf" ]; then62 rm -rf "$OPT_ROOT/cassandra/conf/"63 check_and_exit $?64 fi

...69 # 2. Clear all files in /var/lib/cassandra/commitlog70 restore_log "Deleting the commitlog directory"71 if [ -d "$OPT_ROOT/cassandra/commitlog/" ]; then72 rm -rf "$OPT_ROOT/cassandra/commitlog/"73 check_and_exit $?74 fi

...

77 restore_log "Deleting cassandra data directory"78 if [ -d "/var/lib/sdn/cassandra/" ]; then79 rm -rf "/var/lib/sdn/cassandra/"80 check_and_exit $?81 fi

6. Acquire the authentication token for the system restore:

curl --noproxy controller_ip -X POST --fail -ksSfL --url "https://controller_ip:8443/sdn/v2.0/auth" -H "Content-Type: application/json" --data-binary '{"login": {"domain": "domain","user": "user","password": "password"}}'

Credential information (user name, password, domain, and authentication tokens) used in curlcommands might be saved in the command history. For security reasons, Hewlett PackardEnterprise recommends that you disable command history prior to executing commandscontaining credential information.


7. Acquire the controller uid:

curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -ksSfL --request GET --url "https://controller_ip:8443/sdn/v2.0/systems"

8. Use the following curl command to set the IP address:

curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -ksSfL --request PUT "https://controller_ip:8443/sdn/v2.0/systems/controller_uid" --data-binary '{"system":{"ip":"controller_ip"}}'

9. Perform a single controller restore onto each controller needing restoration.a. Upload the backup files that will be restored:

curl --noproxy controller_ip -X POST --fail -ksSfL --url "https://controller_ip:8443/sdn/v2.0/restore/backup" -H "X-Auth-Token:auth_token" --data-binary @path-and-file-name.zip"where path-and-file-name is the full path to the file and the filename. The filename MUST match the nameyou used during the backup.

b. Initiate the restore:

curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -ksS --request POST --url "https://controller_ip:8443/sdn/v2.0/restore"

For a controller team, wait for HA synchronization to complete to all the controllers and wait for the team tobecome connected. The team can take a few minutes to come back up. Be sure to verify that team statushas all controllers as active and one of the team members is a leader.

curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -ksSfL --request GET --url "https://controller_ip:8443/sdn/v2.0/systems" • If less than a quorum of controllers are restored, then those controllers are updated to the latest state of

the running team via HA synchronization. (A quorum is n/2+1 where n is the total number of controllers ina team. In a three-controller team, a quorum is two controllers.)

• If the entire team is restored, then each controller is reset to the previous backed-up state.10. After the controller restore is complete, change the value of CTL_RESTORE_INSTALL_MODE to false in

the ~/.sdn_install_options file on each controller so that it does not impact a future installation. This isbecause a future installation of the controller might not involve starting in recovery mode.

It is possible to query the restore status by using the get command at v2.0/restore/status. Since therestore is not hitless, the REST query fails until the controller has successfully restarted.

To restore a controller team, restore each controller as a standalone controller. See Distributed(team) backing up and restoring on page 147.


Attempting to restore a backup taken on any release prior to version 2.6 will not complete.

11. If you have files that were manually backed up prior to restoration, such as truststore or keystore files withCA signed certificates or certificates in the sdnjar_trust.jks file, do the following:a. Stop the controller.b. Copy the backed-up files to their original locations.c. Start the controller.

Restore operationTo restore a controller from a backup, it is necessary to re-install the controller.

• During a user-initiated restore operation, the controller GUI is not accessible and the controller is not fullyfunctional until the restore is complete.

• In a controller team environment each active controller is restored as a single system.• If a backed-up controller in a team fails, use single-system restore to restore the controller. The HA

synchronization updates the controller to the latest version.• The controller blocks traffic over OpenFlow ports during a restore operation.

The controller ceases to operate during a restore operation.

System restore requirementsA system backup can be restored only to a system having the following:

• The same controller version that existed at the time the backup was taken.• The same network settings (IP address) as were present at the backup.• The same license ID as was in effect when the controller was installed.

If you have modified any environment—specific settings in files such as /opt/sdn/virgo/options or /etc/init/sdnc.conf, ensure that the appropriate changes are made to these filesafter you re-install the controller and before you start the restore. For example, the network interfacethat the Virgo service uses (default: eth0) might be eth1 or another setting on some systems.

Distributed (team) backing up and restoringIn a team environment, all team members must successfully complete the backup.

A team backup consists of using the single-system backup process. All controllers in the team must be active,and all of the backups in the team should be done either serially at approximately the same time, or in parallel. Tocomplete a teamed backup, no controller can be in a failed state. (A controller team must have three controllers.)In a team environment, all team members must successfully complete the backup for the backup to besuccessful.

A team restore consists of using the single-system restore process on each controller in the team. Like backups,a system restore in a team should be done either serially at approximately the same time, or in parallel. Two


controllers must be up and running before either one can become active. If your team has one or more nodes,make sure that all controllers are up and Cassandra is running, as follows:

./opt/sdn/cassandra/bin/nodetool ring |grep -c 'Up' This command must return 3. You must login to each controller in sequence and run the following command, nomatter how many controllers were actually restored:

./opt/sdn/cassandra/bin/nodetool repair

Do not attempt to run this command at the same time on different nodes. It must run to completionon one node before you run it on another node. This command may have a significant impact on diskand network I/O across all controllers in the team and it may take some time.

When restoring a team, be sure to re-install all of the controllers, before initiating the actual restoreon any of the controllers.

Also, if backing up the team controllers was done serially, then the restore of the team controllersshould be done in reverse order.

A controller that fails a restore operation is not allowed to rejoin the team, and must be re-added as a newcontroller.

Backing up and restoring the Keystone configuration anddatabaseBackup/Restore for the Keystone configuration and database are separate actions from the controller Backup/Restore. The backup/restore does not backup any Keystone related configuration/credentials therefore anychanges made to Keystone will be lost after the restore.

These instructions apply to the default local Keystone instance (Keystone 2012.2) as specified in theAruba VAN SDN Controller Installation Guide. If you are using a different Keystone installation,please follow the OpenStack instructions for backup/restore of the Keystone instance specific to yourinstallation. For OpenStack documentation, visit http://docs.openstack.org.


http://docs.openstack.org/

Viewing metric dataMetric data created by the controller and applications can assist you when you are troubleshooting issues with thecontroller or network.

The curl commands in this section interact with the metrics/apps REST API to display information about metricdata collected by the controller. These commands must be used on individual controllers; they do not returninformation from controller teams.

About metricsMetrics are created by controller components or applications and can be of several different types, such as timers,counters, gauges, and histograms.

How metric values are processedWhen the controller component or application creates a metric, it specifies what to do with the resulting metricvalues. Metric values can be processed in one or more of the following ways:

• Persisted to disk as a time-series of values• Exposed for inspection by the user via JMX• Retained in memory for internal use by the component or application

Metric identifiersWhen a metric is created, it is associated with the following text strings that, taken as a combination, uniquelyidentify the metric:

Application ID (REQUIRED) Identifies the application creating the metric. For example, the application ID forthe controller is com.hp.sdn.

metric name (REQUIRED) Describes the metric. This name is provided by the application that creates themetric. For example, an application that creates metrics to represent the characteristics of trafficon device ports might use names such as:

framesTransmittedTxFramesframesReceivedRcvFrames

primary tagsecondary tag

Additional description of the metric, such as a category or a device. Primary and secondary tagsare optional, but if a secondary tag is provided, a primary tag must be provided. These tags areprovided by the application or component that creates the metric.

For example, an application that creates metrics to represent the characteristics of traffic ondevice ports might use a primary tag to identify a device and a secondary tag to identify specificports on the device. The resulting metrics are structured so that for each device (primary tag)there are multiple ports (secondary tags) that each have several metrics associated with them.

Chapter 10Metrics

Chapter 10 Metrics 149

Viewing the application IDs for applications that have persisted metrics todiskUse the application ID as input for metrics commands that require an application ID in the URL.

Procedure

1. List the application IDs for all applications (including those that are part of the controller itself) that havepersisted metric data to disk:

curl --noproxy controller_ip -X GET \--header "X-Auth-Token:token" --fail -ksSfL \--url "https://controller_ip:8443/sdn/v2.0/metrics/apps"

Command example

curl --noproxy 10.1.1.9 -X GET \--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \--url "https://10.1.1.9:8443/sdn/v2.0/metrics/apps" Command output

In the above example, only the base controller itself has persisted metric data to disk. The application id is:com.hp.sdn.

The next example shows the JSON output as returned by the curl command:

{"apps":[{"app_id":"com.hp.sdn","app_name":"HP VAN SDN Controller"}]}The next example shows the JSON output formatted for readability:

{ "apps":[ { "app_id":"com.hp.sdn", "app_name":"HP VAN SDN Controller" } ]}

Viewing the metrics persisted by an application

Procedure

1. To list all of the metrics available for an application, use the following curl command:

curl --noproxy controller_ip -X GET \--header "X-Auth-Token:token" --fail -ksSfL \--url "https://controller_ip:8443/sdn/v2.0/metrics/apps/app_id \?name=name&primary_tag=primary_tag&secondary_tag=secondary_tag"


Optional parameters

Name and value pairs after the ? character, separated by the & character, are optional parameters:

name The name of the metric. The controller lists only the metrics that have a name that matchesthe specified value. If you do not specify a name, the controller lists all metrics.

primary_tag The name of the primary tag. The controller lists only the metrics that have a primary tag thatmatches the specified value.

secondary_tag The name of the primary tag. The controller lists only the metrics that have a secondary tagthat matches the specified value.

Usage

• If you specify more than one parameter, the controller lists only those metrics that match all the parametersyou specify.

• If you do not specify a value for a parameter, the controller does not filter the results based on that parameter.

Command example

curl --noproxy 10.1.1.9 -X GET \--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \--url "https://10.1.1.9:8443/sdn/v2.0/metrics/apps/com.hp.sdn" Command output

This example shows a partial listing of the output from the example command.

The uid for a metric is the unique identifier assigned to the metric on the controller.

{ "metrics":[ { "app_id":"com.hp.sdn", "type":"RATIO_GAUGE", "name":"cpuLoadSystem", "description":"The recent CPU usage of the system.", "primary_tag":"jvm", "secondary_tag":"operatingSystem", "jmx":false, "persistence":true, "summary_interval":"ONE", "uid":"42f65cd8-03c3-4cad-9788-012d513e3c0f" }, { "app_id":"com.hp.sdn", "type":"GAUGE","name":"committedBytes", "description":"The amount of non-heap memory in bytes committed for the JVM to use.", "primary_tag":"jvm", "secondary_tag":"memoryNonHeap", "jmx":false,


"persistence":true, "summary_interval":"ONE", "uid":"b82f5b00-0373-4a23-b5a8-bbda7eec44cb" }, { "app_id":"com.hp.sdn", "type":"GAUGE","name":"countTerminated", "description":"Number of JVM threads that had exited.", "primary_tag":"jvm", "secondary_tag":"threads", "jmx":false, "persistence":true, "summary_interval":"ONE", "uid":"0e9fe62e-01fd-42e9-88a6-f92021a5e786" },

.

.

. { "app_id":"com.hp.sdn", "type":"ROLLING_COUNTER", "name":"count", "description":"The number of JVM garbage collections actions during the sampling interval.", "primary_tag":"jvm", "secondary_tag":"garbageCollection", "jmx":false, "persistence":true, "summary_interval":"ONE", "uid":"d62c49d4-46b3-4c2c-be60-24f6fa6c6bf6" },...

Viewing the primary tags for metrics persisted by an application

Procedure

1. To list the primary tags associated with the metrics persisted by a specific application, use the following curlcommand:

curl --noproxy controller_ip -X GET \--header "X-Auth-Token:token" --fail -ksSfL \--url "https://controller_ip:8443/sdn/v2.0/metrics/apps/app_id/primaries\?name=metric_name"

Optional parameters


name The name of the metric.


Usage

If you do not specify a value for a parameter, the controller does not filter the results based on that parameter.

Command example

curl --noproxy 10.1.1.9 -X GET \--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \--url "https://10.1.1.9:8443/sdn/v2.0/metrics/apps/com.hp.sdn/primaries" Command output

The only primary tag associated with the controller itself in this example is jvm.

{ "primaries":[ "jvm" ]}

Viewing the secondary tags for metrics persisted by an application

Procedure

1. To list the the secondary tags associated with the metrics persisted by a specific application, use the followingcurl command:

curl --noproxy controller_ip -X GET \--header "X-Auth-Token:token" --fail -ksSfL \--url "https://controller_ip:8443/sdn/v2.0/metrics/apps/app_id/secondaries\?name=name&primary_tag=primary_tag"

Optional parameters


name The name of the metric. The controller lists only the metrics that have a name that matches thespecified value.


Usage



Command example

curl --noproxy 10.1.1.9 -X GET \--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \


--url "https://10.1.1.9:8443/sdn/v2.0/metrics/apps/com.hp.sdn/secondaries" Command output

{ "secondaries":[ "nioDirectMemory", "operatingSystem", "threads", "garbageCollection", "memoryNonHeap", "memoryHeap", "memoryTotal", "nioMappedMemory" ]}

Viewing the names of metrics persisted by an application

Procedure

1. To list the names of the metrics persisted by a specific application, use the following curl command:

curl --noproxy controller_ip -X GET \--header "X-Auth-Token:token" --fail -ksSfL \--url "https://controller_ip:8443/sdn/v2.0/metrics/apps/app_id/names\?primary_tag=primary_tag&secondary_tag=secondary_tag"

Optional parameters



secondary_tag The name of the primary tag. The controller lists only the metrics that have a secondary tagthat matches the specified value.

Usage



Command example

curl --noproxy 10.1.1.9 -X GET \--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \--url "https://10.1.1.9:8443/sdn/v2.0/metrics/apps/com.hp.sdn/names" Command output


This example shows a partial listing of the output from the example command.

{ "names":[ "averageBufferUsedBytes", "countDeadlocked", "bufferCapacityBytes", "count", "countNew", "bufferCount", "countWaiting", "fileDescriptorsOpen", "uptimeMs", "countTerminated", "elapsedMs", "countTimedWaiting", "countDaemon",... "countBlocked", "committedBytes" ]}

Viewing information about a persisted metric identified by its UID

Procedure

1. To list information about a specific metric, which is identified by its UID (unique ID), use the following curlcommand:

curl --noproxy controller_ip -X GET \--header "X-Auth-Token:token" --fail -ksSfL \--url "https://controller_ip:8443/sdn/v2.0/metrics/metric_uid"

Command example

curl --noproxy 10.1.1.9 -X GET \--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \--url "https://10.1.1.9:8443/sdn/v2.0/metrics/431b746e-e62e-4874-a801-b1438eaac635" Command output

{ "metric":{ "app_id":"com.hp.sdn", "type":"GAUGE", "name":"usedBytes", "description":"The amount of heap memory currently being used by the JVM in bytes.", "primary_tag":"jvm", "secondary_tag":"memoryHeap", "jmx":false,


"persistence":true, "summary_interval":"ONE", "uid":"431b746e-e62e-4874-a801-b1438eaac635" }}

Viewing the time-series values for a persisted metric identified by its UID

Procedure

1. To list the time-series values for a specific metric, which is identified by its UID (unique ID), use the followingcurl command:

curl --noproxy controller_ip -X GET \--header "X-Auth-Token:token" --fail -ksSfL \--url "https://controller_ip:8443/sdn/v2.0/metrics/metric_uid/values\?start=start&end=end&interval=interval" Optional parameters


start The earliest time to query for time-series data. The date and time in must be in the format yyyy-mm-dd+hh:mm. If you specify an end but do not specify a start, the value used for start is thetime of the oldest instance of the metric that is within the configured age-out time.

end The most recent time to query for time-series data. The date and time in must be in the formatyyyy-mm-dd+hh:mm. If you specify a start but do not specify an end, the value used for end isthe time you enter the command.

interval The interval each returned data point is to represent. This parameter is required if you specify thestart or the end parameter. The interval you specify must be equal to or greater than the intervalat which the metric’s time-series data is persisted. Choose one of the following values: 1, 5, 15, 30,60, day, and all, where the numeric values represent the number of minutes, day represents a 24-hour period, and all represents the summarized values over the lifetime of the metric datacollection.

Usage

• If you do not specify parameters, only the last persisted time-series value for the metric is returned.• Specifying an interval summarizes the data, and enables you to quickly look for anomalous data. For example,

instead of viewing 1440 discrete minute-by-minute values for a specific metric over the course of a day, youcan request that each returned value represent 60 minutes so that the number of returned values is moremanageable and can be more easily inspected.

• Summarizing the returned data for a period typically results in larger values for the metrics themselves andlarger values for the milliseconds spanned. The computation of the values returned over longer intervalsdepends upon the type of metric.

• For example, gauge values are averaged over the data points encompassed in the summary, but countervalues are summed over the summary interval.

Command example

curl --noproxy 10.1.1.9 -X GET \


--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \--url "https://10.1.1.9:8443/sdn/v2.0/metrics/431b746e-e62e-4874-a801-b1438eaac635?start=2014-09-23+18:00&interval=5" Command output

{ "metric_values":{ "uid":"431b746e-e62e-4874-a801-b1438eaac635", "type":"GAUGE", "datapoint_count":6, "datapoints":[ { "update_time":"Tue Sep 23 18:03:55 PDT 2014", "milliseconds_span":300000, "last":3.274097568E8 }, { "update_time":"Tue Sep 23 18:08:55 PDT 2014", "milliseconds_span":300000, "last":3.133927072E8 }, { "update_time":"Tue Sep 23 18:13:55 PDT 2014", "milliseconds_span":300000, "last":2.154562624E8 }, { "update_time":"Tue Sep 23 18:18:55 PDT 2014", "milliseconds_span":300000, "last":4.192128832E8 }, { "update_time":"Tue Sep 23 18:23:55 PDT 2014", "milliseconds_span":300000, "last":3.864813136E8 }, { "update_time":"Tue Sep 23 18:27:55 PDT 2014", "milliseconds_span":240000, "last":3.847236E8 } ] }}

Viewing all controller JVM metricsMany metrics are not persisted to disk as time-series data, often because they do not change over time.

To view all metrics that are tracked by the controller with regard to the JVM, including those that are also persistedas time-series data:


Procedure

1. Create a controller support report.2. View the data in the support report under the jvm-metrics ID.jvm-metrics ID.

Viewing current metric data using a JMX clientProcedure

1. You can use JConsole or another JMX client to connect to the controller's JMX server to view metric data as itis updated in real time.

Metrics that are viewable using a JMX clientThe jmx field of the MetricDescriptor used to create the TimeStampedMetric determines whether or notthe metric can be viewed using a JMX client.

The content exposed for each TimeStampedMetric is contingent on the type of TimeStampedMetric, buttypically the most current values used by the TimeStampedMetric are visible as they are updated by thecreator of the TimeStampedMetric.

PrerequisitesFor JMX clients to connect to the JMX server of the controller, the following conditions must be true:

Procedure

1. The JMX client must be installed on the same system as the controller.2. No JMX clients are included with the controller or are among the prerequisites for installing it; they must be

installed separately. For example, to use the JConsole JMX client, theopenjdk-7-jdkpackage must be installed on the same system as the controller.

3. The controller must be configured to permit local JMX access.

For more information about using JConsole and configuring local JMX access, see the JConsole documentationprovided by Oracle at:

http://docs.oracle.com/javase/7/docs/technotes/guides/management/jconsole.html

Connecting to the JMX server using the JConsole JMX client

Procedure

1. Start the JConsole JMX client.2. From the New Connection screen, select Local Process.


http://docs.oracle.com/javase/7/docs/technotes/guides/management/jconsole.html

For an example, see below.

Figure 65: JConsole new connection

3. Choose a local connection to the JMX server instance and click Connect.

After successfully connecting to that JMX server instance, a screen similar to the screen shown below isdisplayed.

Figure 66: JConsole window

Viewing metrics using JConsole JMX

Procedure

1. To display the metrics for an application, expand the application folder in the left pane:a. To view metrics for the Aruba VAN SDN Controller and its embedded applications, expand the folder named

Aruba VAN SDN Controller.b. To view metrics for an application installed on the controller, expand the folder for the application.


The figure below shows an example of the expanded Aruba VAN SDN Controller application folder with themetrics displayed.

Figure 67: JConsole window displaying Aruba VAN SDN Controller metrics

2. Expand the metric you want to view.

The name displayed for each metric is a combination of the application ID, metric name, and—if present—theprimary and secondary tags that were specified during the creation of its MetricDescriptor. Thiscombination is unique among all TimeStampedMetric instances monitored for a specific application.

3. Select Attributes to display the metric attributes that are exposed via JMX.

For persisted TimeStampedMetric instances, MsSpanned indicates the number of milliseconds that haveaccrued during the persistence interval that is underway but not yet complete. The figure shows an example ofdisplaying the attributes for a metric.

Figure 68: JConsole window displaying details about a metric

Generating a controller support reportThe controller support report provides data, such as JVM metrics, that can be useful when troubleshooting.


Procedure

1. To create a controller support report, use the following curl command:

curl --noproxy controller_ip -X GET \--header "X-Auth-Token:token" --fail -ksSfL \--url "https://controller_ip:8443/sdn/v2.0/support?id=id-string&fields=fields-string"

Optional parameters


id The list of contributors to include in the returned support report data. For example, to include only itemsreported by the controller JVM metrics, specify the following string:

id=jvm-metrics

fields The set of fields, separated by commas, to be returned in the report. For example, to include only thetitle and content fields, specify the following string:

fields=title,content

If you specify more than one parameter, the support report includes only those items that match all theparameters you specify.

If you do not specify a value for a parameter, the support report includes all items for that parameter.

Command example

curl --noproxy 10.1.1.9 -X GET \--header "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" --fail -ksSfL \--url "https://10.1.1.9:8443/sdn/v2.0/support" Command output

The following example contains a partial listing of the support report returned by the previous command:

{ "support_report":[ { "title":"Alert Framework", "id":"alert", "content":[ "Alert-Topics: licensing", "Alert-Count: 7", "Data Retention Age Out: 14 days", "Data Trim Interval: 24 hours", "Data Trim Enabled: true", "Last trim conducted at: Mon Sep 22 19:15:20 PDT 2014" ]}, { "title":"Alert Topic Listener", "id":"alert_listener",


"content":[ "No registered alert topic listeners" ]}, { "title":"App Manager", "id":"app-management", "content":["Installed Applications: 8", "Path Diagnostics, Version: 2.5.0.SNAPSHOT, State: ACTIVE", "Link Manager, Version: 2.5.0.SNAPSHOT, State: ACTIVE", "Node Manager, Version: 2.5.0.SNAPSHOT, State: ACTIVE", "OpenFlow Link Discovery, Version: 2.5.0.SNAPSHOT, State: ACTIVE", "OpenFlow Node Discovery, Version: 2.5.0.SNAPSHOT, State: ACTIVE", "Path Daemon, Version: 2.5.0.SNAPSHOT, State: ACTIVE", "Topology Manager, Version: 2.5.0.SNAPSHOT, State: ACTIVE", "Topology Viewer, Version: 2.5.0.SNAPSHOT, State: ACTIVE" ]},... { "title":"JVM Metrics", "id":"jvm-metrics", "content":[ "Metric count: 44", "Last update time: Wed, 24 Sep 2014 01:31:55 GMT", "Uptime: 10,037 minute(s)", "Memory", " Total", " Initial: 548,288 kB", " Committed: 740,032 kB", " Maximum: 4,301,824 kB", " Used: 477,308 kB", " Heap", " Initial: 524,288 kB", " Committed: 655,360 kB", " Maximum: 3,728,384 kB", " Used: 393,227 kB", " Usage: 10.547 %", " Non-Heap", " Initial: 24,000 kB", " Committed: 84,672 kB", " Maximum: 573,440 kB", " Used: 84,081 kB", " Usage: 14.663 %", "NIO Buffer Memory", " Direct", " Capacity: 0 bytes", " Used: 0 bytes", " Buffers: 0", " Mapped", " Capacity: 0 bytes", " Used: 0 bytes", " Buffers: 0", "Garbage Collection (last 1 minute(s))", " Executions: 0", " Elapsed time: 0 ms", "Threads", " Total count: 122",


" By Type", " Daemon: 65", " Non-daemon: 57", " By State", " Blocked: 0", " Deadlocked: 0", " New: 0", " Runnable: 7", " Terminated: 0", " Timed waiting: 24", " Waiting: 91", "Operating System", " CPU Usage", " System: 0.079 %", " JVM: 0.058 %", " File Descriptors", " Maximum: 8,192", " Open: 214", " Usage: 2.612 %" ]},...


REST API request returns HTTP code 401Symptom

Getting unauthorized HTTP code 401 for REST calls to the controller.

Cause

The role-based access control (RBAC) role is not authorized with the token. Aruba VAN SDN Controller 2.5enforces a single role. By default the single role is sdn-admin. A user must have this role configured on theKeystone server for the domain (tenant) that the user belongs to.

Action

Procedure

1. Configure a user with the sdn-admin role on the Keystone server for the domain (tenant) that the userbelongs to. The domain name and role configured for the user on the controller must match the domain nameand role configured for that user in Keystone.a. Create a tenant:

curl –H "X-Auth-Token:ADMIN" –H "Contant-Type: application/json" –d '{"tenant": {"enabled": true, "name": "test-tenant", "description": "Test Tenant"}}' http://<controller-ip>:35357/v2.0/tenants

b. List tenants:

curl –H "X-Auth-Token:ADMIN" http://<controller-ip>:35357/v2.0/tenantsc. Create a user:

curl –H "X-Auth-Token:ADMIN" –H "Contant-Type: application/json" –d '{"user": {"email": "[email protected]", "password": "somepass", "enabled": true, "name": "test-user", "tenantId": "2c851897a09f483fa452e2de11511f71"}}' http://<controller-ip>:35357/v2.0/users

d. List users:

curl –H "X-Auth-Token:ADMIN" http://<controller-ip>:35357/v2.0/userse. Create a role:

curl –H "X-Auth-Token:ADMIN" –H "Contant-Type: application/json" –d '{"role": {"name": "test-role"}}' http:// <controller-ip>:35357/v2.0/OS-KSADM/roles

f. List users:

curl –H "X-Auth-Token:ADMIN" http://<controller-ip>:35357/v2.0/roles

Chapter 11Troubleshooting


g. Assign role:

curl –X PUT –H "X-Auth-Token:ADMIN" http://<controller-ip>:35357/v2.0/tenants/<tenant-id>/users/<user-id>/roles/OS-KSADM/<role-id>

h. List roles for a user:

curl –H "X-Auth-Token:ADMIN" http://<controller-ip>:35357/v2.0/OS-KSADM/roles/<user-id>

2. Use the curl command to request authentication using the default username and password. You mustinclude the keyword domain and the default domain name value, in this example sdn, as follows:

curl -sk -H 'Content-Type:application/json' -d '{"login":{"user":"sdn","password":"skyline","domain":"sdn"}}' https://<controller-ip>:8443/sdn/v2.0/auth)

Controller not listening on port TCP/8443Symptom

The SDN controller installed successfully but it is not listening on port TCP/8443.

Cause

You are running a version of Linux that is not supported.

Action

Procedure

1. Install the controller on Linux Ubuntu version 14.04 LTS 64-bit server.

Packets not received at the end pointSymptom

HTTP traffic is not received at the end point.

Cause

In some situations, a switch might not forward HTTP traffic.

Action

Procedure

1. Check the switch functionality and compatibility with your setup. In some cases, the firmware of the switchdoes not forward HTTP traffic if there is a copy for it.

2. Check the switch version and possible CPE fixes. In firmware 15.16, if there is an Openflow flow-mod with acopy action for HTTP traffic, it will not forward the HTTP traffic. This issue was fixed in a CPE branch forfirmware 15.16.

Session expired message in the UISymptom

You see a message in the controller UI that your login session has expired.

Chapter 11 Troubleshooting 165

Cause

It has been longer than the defined Keystone session timeout since you logged in so your session has expired.The default Keystone login session timeout is 1 hour.

Action

Procedure

1. Log on to the controller as sudo user.2. Open the /etc/keystone/keystone.conf file for editing.3. Locate the line: #expiration=3600.4. Modify that line by removing the comment (hash tag) and change the value to something larger.5. Save the file and exit the editor.6. Restart the Keystone server using this command:

service keystone restart7. Logout from the UI and log back in; the session timeout is increased to the new value.

Error running the config_sdn.py script with date/time/NTPoptionSymptom

You run the post install configuration script with the option to change the date and time or NTP server (pythonconfig_sdn.py –d) and after entering the new date and time or NTP server information, the script will try toaccess the file /etc/net.conf but the script fails and you see a file permission denied error message.

Cause

Making changes to the date and time or NTP server information using the GUI will change the permission of thefile /etc/net.conf.

After that, using the post install script to change date and time or NTP server information (pythonconfig_sdn.py –d) will fail because the script will try to access the /etc/net.conf file which has had thepermission changed.

Action

Procedure

1. Do one of the following:

• Once you use the controller GUI to change date and time or NTP server, continue to make any subsequentchanges to date and time or NTP server using the GUI rather than the script.

• Or you can reset the permissions on the /etc/ntp.conf file with the command: sudo chmod 644ntp.conf.

Licensing

Redeem quantity error

Symptom

You see an error message that your license has a maximum redeem quantity.


Cause

You specified a license quantity that exceeds what your license type supports.

Action

Procedure

1. Return to the My Network portal license selection screen.2. Enter the correct quantity in the Redeem column for your license type:

a. For an Aruba VAN SDN Ctrl Base SW w/ 50–node E-LTU license, the quantity must be 1.b. For Aruba VAN SDN Ctrl 50–node E-LTU or Aruba VAN SDN Ctrl HA E-LTU licenses, the quantity can be

any quantity on your sales order.

Install ID format errors

Symptom

You see an error message that your Install ID format is invalid.

Cause

You entered an invalid Install ID, or have not entered an Install ID.

Action

Procedure

1. Carefully check your Install ID.2. Return to the license registration details screen and enter a valid value in the Install ID field.

Install ID errors

Symptom

You see an error message that your Install ID is required.

Cause

The Install ID has not been entered in the portal during the registration process.

Action

Procedure

1. See the Aruba VAN SDN Controller Administrator Guide for instructions on how to enter your Install ID.

Applications that use the Cassandra database areexperiencing failuresSymptom

Applications that use the Cassandra database are experiencing failures, and there are log entries that indicateproblems connecting to the Cassandra database.


Action

Procedure

1. Log in as the sdnadmin user to Linux on the server on which the controller is installed.2. Ensure that the Cassandra database is online:

~$ /opt/sdn/cassandra/bin/nodetool status

If you see the following message, the Cassandra database is not online:

Failed to connect to '127.0.0.1:7199': Connection refused3. If the Cassandra database is not online, you must restart the controller to restart the Cassandra instance on

that controller:a. Close any browser window in which the controller might be running.b. Restart the controller:

~$ sudo service sdnc restart4. If the Cassandra database stops repeatedly, determine why by doing the following:

a. Export the logs files.b. In the sdn-all-logs.zip file, check the /var/log/sdn/cassandra/system.log file for possible

errors.

Controller support log fills disk space, contains multiple“Too many open files” messagesSymptom

The controller support log exceeds its configured maximum size and potentially consumes all available disk spaceon the system on which the controller is installed. The log contains multiple instances of the following message:

Unable to accept incoming connection: java.io.IOException: Too many open files

Cause

The system on which the controller is installed has run out of file descriptors, either because it controls too manydevices, links, and hosts, or some other process on the system has consumed a large number of file descriptors.

Action

Procedure

1. Ensure that the system on which the controller is installed conforms to the recommended hardwarerequirements for the number of devices, links, and hosts. For hardware recommendations, see the Aruba VANSDN Controller and Applications Support Matrix.

2. Take one or more of following actions:a. Form a controller team and distributing ownership of the switches in the network across the team members

such that each controller in the team controls one third of the switches in the network.b. Increase the system resources, such as the number of file descriptors, on the system on which the

controller is installed.c. Install the controller on a larger system, such as one that conforms to the hardware recommendations

appropriate to the size of the network deployment.

Application management errors


Application not starting and in disabled state

Symptom

An application cannot start and is automatically moved to the DISABLED state.

Cause

The Application Management framework detected a failure to start an application in the OSGi runtimeenvironment.

Action

Procedure

1. Correct the OSGi runtime conditions.2. Enable the application via the GUI or a REST call.

Application in transitive state

Symptom

An application is in a transitive state.

Cause

An unexpected error condition occurred when manipulating an application (such as file I/O exception or missingfiles).

Action

Procedure

1. Examine the log files for exceptions, determine the source of error and correct.2. Uninstall the application. An application can only be uninstalled when it is trapped in a transitive state.3. Upload the application.4. Install the application.

Application management exceptions

Getting IllegalStateException: HTTP code 500

Symptom

Getting IllegalStateException, HTTP code: 500.

Cause

Pushing a meter or group mod to a connected switch from a controller that is not an owner will not return theproper error message if the switch returns an error.

Pushing flows, groups, or meters via a northbound REST API to any controller in the team is supported even ifthat controller is not the master of the given device. In that case, the controller will delegate the request to thecontroller who is the master of the switch and the master controller will handle the request. If the switch returns anerror from the given request (for example, the meters table is full), the switch responds to the owning controllerwith a proper Openflow error. But when that error is sent back to the requesting controller, it is not parsed properlyand results in an error.


Action

Procedure

1. If you are pushing a meter or group mod to a connected switch from a controller, you must make that controllerthe master in order to handle the errors correctly.

Getting UnsafeConfigurationException, HTTP code: 403

Symptom

Getting UnsafeConfigurationException, HTTP code: 403. Applies to application, license, and configurationchanges.

Cause

One member of a team is not active. All team members in a teamed environment must be active before you canmake configuration, licensing, or application changes, or changes to regions. Otherwise, the configuration is notguaranteed to be synchronized with all the members of the team.

Action

Procedure

1. Indicates that the operation cannot be completed.

Getting ApplicationDisableException, HTTP code: 500

Symptom

Getting ApplicationDisableException, HTTP code: 500.

Cause

Occurs when an application status is STAGED or UPGRADE_STAGED, or that something has gone wrong asspecified in error message.

Action

Procedure

1. Indicates that an application cannot be disabled.

Getting ApplicationEnableException, HTTP code: 500

Symptom

Getting ApplicationEnableException, HTTP code: 500.

Cause

Occurs when an application status is not DISABLED, or that something has gone wrong as specified in the errormessage.


Action

Procedure

1. Indicates that an application cannot be enabled.

Getting ApplicationInstallException, HTTP code: 500

Symptom

Getting ApplicationInstallException, HTTP code: 500.

Cause

Occurs when an application status is not STAGED, or that something has gone wrong as specified in the errormessage.

Action

Procedure

1. Indicates that an application cannot be installed.

Getting ApplicationUpgradeException, HTTP code: 500

Symptom

Getting ApplicationUpgradeException, HTTP code: 500.

Cause

Occurs when an application status is not UPGRADE_STAGED, or that something has gone wrong as specified inthe error message.

Action

Procedure

1. Indicates that an application cannot be upgraded

Getting ApplicationUninstallException, HTTP code: 500

Symptom

Getting ApplicationUninstallException, HTTP code: 500.

Cause

Occurs when something has gone wrong as specified in the error message.

Action

Procedure

1. Indicates that an application cannot be uninstalled.

Getting ApplicationUploadException, HTTP code: 500

Symptom

Getting ApplicationUploadException, HTTP code: 500.


Cause

Occurs when an I/O error occurs while uploading the application to the controller.

Action

Procedure

1. Indicates that an application cannot be uploaded.

Getting ApplicationValidationException, HTTP code: 400

Symptom

Getting ApplicationValidationException, HTTP code: 400.

Cause

Occurs when the file format or contents is invalid, or when the signed jar verification failed (if enabled).

Action

Procedure

1. Indicates that an application zip file fails validation.

OpenFlow errors

Host location not learned by controller

Symptom

The host is not present in the node database maintained by the controller. The REST/Java API that gets the nodeinformation is missing on that host.

Cause

The ARP/DHCP/IP (if ip.learn=true in pureof mode) packets might not have reached the controller.

Action

Procedure

1. Make the host trigger the ARP/DHCP/IP packets.2. Turn on the OpenFlow trace in the controller.3. Check that those packets are reaching the controller using the PACKET_IN messages.

Switches constantly being disconnected and reconnected

Symptom

The switches are constantly being disconnected and reconnected.

Cause

The pure OpenFlow loop topology requires packets to be flooded across the network. This can result in a lot ofnetwork traffic and interfere with the switches' ability to send echo packets.


Action

Procedure

1. When running a looped topology in pure OpenFlow mode, be aware of how much traffic is being flooded.Some virtual switch implementations might get overwhelmed by the amount of traffic being generated.

Unexpected network or service problems in hybrid mode

Symptom

Unexpected network or service problems.

Cause

You changed the hybrid.mode configuration of the controller without restarting and disabling the controller, thenre-enabling each controlled OpenFlow instance in the OpenFlow switches.

Action

Procedure

1. Change the hybrid.mode configuration. See the Aruba VAN SDN Controller Administrator Guide forinformation about changing the hybrid.mode configuration.

Troubleshooting teamed environmentsControllers dropped from team or unable to form team

Symptom

A group of SDN controllers fail to form a team, or one or more controllers are dropped from the configured team.

Cause

The system clocks on one or more controllers are not synchronized with each other.

Action

Procedure

1. Synchronize all systems (even after a power cycle).2. If the time is incorrect on boot up and restart the sdnc/sdna service.3. If you are installing the controller on a Virtual Machine (VM), you must synchronize the host hypervisor with the

NTP and NTP client running on the hypervisor host.

Because there can be differences in the time reported by different NTP servers, Hewlett Packard Enterpriserecommends that servers in a team be configured to use a centralized NTP daemon. If the servers for thecontrollers in the team are configured such that they connect to different NTP servers, change theconfigurations of the servers to use a centralized NTP daemon.

Teaming framework does not run

Symptom

The teaming subsystem does not run.


Cause

The Iptable rule programming for the teaming framework (Hazelcast) failed.

Action

Procedure

1. Enter:sudo iptables -D OUTPUT -p tcp --destination-port 5700 -j DROP

2. Enter:sudo iptables -D INPUT -p tcp --destination-port 5700 -j DROP

3. Restart the controller.

Controller becomes suspended

Symptom

The controller is suspended and the RSdoc and UI are unavailable.

Cause

A controller transitions to the suspended state when quorum is lost or core services report critical health. When ateam is partially created, a controller might never get quorum, and thus remain suspended, because teamcreation failed in other controllers.

Action

Procedure

1. Authenticate using the curl command:

curl --noproxy [IP_ADDRESS] -X POST --fail -ksSfL --url "https://[IP_ADDRESS]:8443/sdn/v2.0/auth" --header "Content-Type: application/json" --data-binary '{"login":{"user":"sdn","password":"skyline","domain":"sdn"}}'For example:

curl --noproxy 1192.0.123.5 -X POST --fail -ksSfL --url "https://192.0.123.5:8443/sdn/v2.0/auth" --header "Content-Type: application/json" --data-binary '{"login":{"user":"sdn","password":"skyline","domain":"sdn"}}'The curl command in this example generated the following response:

{"record":{"token":"2ee24d3a87c345e98deaeaed4bf770ef","expiration":1416343000000,"expirationDate":"2014-11-18 12-36-40 -0800","userId":"06338e545b9a4f04b145a5ad7c541254","userName":"sdn","domainId":"1685f85a515e4983b4ea64fff9917ecc","domainName":"sdn"}}

2. Check systems status using the curl command:

curl --noproxy [IP_ADDRESS] --header "X-Auth-Token:[AUTHENTICATION_TOKEN]" --fail -ksS -L -f --request GET --url "https://[IP_ADDRESS]:8443/sdn/v2.0/systems"


For example:

curl --noproxy 192.0.123.5 --header "X-Auth-Token:2ee24d3a87c345e98deaeaed4bf770ef" --fail -ksS -L -f --request GET --url "https://192.0.123.5:8443/sdn/v2.0/systems"The curl command in this example generated the following response:

{"systems":[{"uid":"0046a56e-a65e-4960-9d9e-f1820a285e53","version":"2.5.2.0496","ip":"192.0.123.5","role":"member","core_data_version":16,"core_data_version_timestamp":"2014-11-18T19:30:37.587Z","status":"suspended","self":true},{"uid":"1ac1c1e7-1a6e-4401-ad3e-206028d57db8","version":"2.5.2.0496","ip":"192.0.127.34","role":"member","core_data_version":16,"core_data_version_timestamp":"2014-11-18T19:30:38.480Z","status":"unreachable"},{"uid":"2533a1df-cced-44f9-b9be-f6e3851da261","version":"2.5.2.0496","ip":"192.0.125.176","role":"member","core_data_version":16,"core_data_version_timestamp":"2014-11-18T19:30:38.933Z","status":"unreachable"}]}

3. Check alerts.a. To get all alerts, use thecurl command:

curl --noproxy [IP_ADDRESS] --header "X-Auth-Token:[AUTHENTICATION_TOKEN]" --fail -ksS -L -f --request GET --url "https://[IP_ADDRESS]:8443/sdn/v2.0/alerts"

b. To get alerts from a time interval, use the following curl command:

curl --noproxy [IP_ADDRESS] --header "X-Auth-Token:[AUTHENTICATION_TOKEN]" --fail -ksS -L -f --request GET --url "https://[IP_ADDRESS]:8443/sdn/v2.0/alerts?start=XXXX-XX-XXTXX:XX:XX.XXXZ&end=XXXX-XX-XXTXX:XX:XX.XXXZ"For example:

curl --noproxy 192.0.123.5 --header "X-Auth-Token:2ee24d3a87c345e98deaeaed4bf770ef" --fail -ksS -L -f --request GET --url "https://192.0.123.5:8443/sdn/v2.0/alerts?start=2014-11-18T19:30:15.000Z&end=2014-11-18T19:30:17.000Z"The curl command in the preceding example generated the following response:

{"alerts":[{"uid":"9ddf298e-a747-409c-b021-2610d6bf3e85","system_uid":"0046a56e-a65e-4960-9d9e-f1820a285e53","topic":"HealthMonitor","org":"HealthMonitor","ts":"2014-11-18T19:30:16.641Z","sev":"CRITICAL","state":true,"desc":"Health Monitor com.hp.sdn.adm.system.impl.QuorumRegistar changed state to CRITICAL > reason: No quorum"},{"uid":"0e8deb7d-edcf-4000-a01d-db9bbabcc337","system_uid":"1ac1c1e7-1a6e-4401-ad3e-206028d57db8","topic":"HealthMonitor","org":"HealthMonitor","ts":"2014-11-18T19:30:16.905Z","sev":"CRITICAL","state":true,"desc":"Health Monitor com.hp.sdn.adm.system.impl.QuorumRegistar changed state to CRITICAL > reason: No quorum"}]}


Unable to create team

Symptom

A group of SDN controllers fail to create a team.

Cause

The DNS server configured on each controller is unreachable.

Action

Procedure

1. Resolve the DNS server reachability issue.2. If the DNS server cannot be reached, add each team member to the /etc/hosts file on each controller.

Controller and application data differs among controllers in a team

Symptom

Controllers and applications in a team do not display the same data, or data appears to be out of synchronizationbetween controllers in a team.

Cause

The system clocks for the controllers in the team are not synchronized.

For controllers that are configured to use NTP (Network Time Protocol) but connect to different NTP servers, theserver clocks can drift over time, resulting in a significant relative difference between clock times. Cassandraserializes write operations by timestamp, so differences between system clock times can result in inconsistentdata across those systems.

Action

Procedure

1. Ensure that all the servers on which the controllers in the team are installed are configured to use NTP.2. Ensure that the servers are configured to use a centralized NTP daemon.

Application data is not synchronized after a controller rejoins the team

Symptom

Data displayed in the user interfaces and the output of programmatic interfaces of applications that useCassandra differs among the controllers in a team after a controller rejoins the team after a long outage (such asmore than 1 hour).

Controller data and data for applications that do not use the Cassandra database are not affected.

Cause

After a Cassandra instance is offline for longer than is supported for automatic synchronization by the Cassandradatabase, the Cassandra databases in a controller team are not synchronized automatically.


Action

Procedure

1. Stop the controller that was offline, remove that Cassandra instance from the cluster, delete all theCassandra data on the controller, and then restart the controller:

2. Log in to Linux on the server on which the controller with the incorrect data is installed. Typically, this is thecontroller that came back online after a long outage.

3. Start a root user command shell:

~$ sudo su4. Stop the controller and administrative services:

~# service sdnc stop

~# service sdna stop5. Log in to Linux on a server on which a controller with the correct data installed.6. Start a root user command shell:

~$ sudo su7. Navigate to the /opt/sdn/cassandra/bin directory and enter the following command:

/opt/sdn/cassandra/bin# ./nodetool status8. Copy the Host ID of the Cassandra instance that has a state of DN in the output of the command you entered

in the previous step. For example, the last entry in the following output is the Cassandra instance for thecontroller that was stopped:

Datacenter: datacenter1=======================Status=Up/Down|/ State=Normal/Leaving/Joining/Moving-- Addres Load Tokens Owns (effective) Host ID RackUN 192.0.2.142 1.23 GB 1 100.0% 4245b8ab-6c3c-4755-bb28-90850d3a4a24 rack1UN 192.0.2.140 1.23 GB 1 100.0% c172bbe2-799c-4adf-bd38-690dfa75ac79 rack1DN 192.0.2.141 310.11 MB 1 100.0% 26999328-abec-4d80-a689-eb8b1f7f89d1 rack1

9. Remove the Cassandra instance from the cluster by entering the ./nodetool removenode HostIDcommand, where HostID is the identifier of the Cassandra instance to remove. For example:

/opt/sdn/cassandra/bin# ./nodetool removenode 26999328-abec-4d80-a689-eb8b1f7f89d1

10. Log out of the server:

/opt/sdn/cassandra/bin# exit11. From the Linux prompt of the server for the controller you stopped, delete the Cassandra data:

/opt/sdn/cassandra/data# rm -rf * /opt/sdn/cassandra/commitlog# rm */opt/sdn/cassandra/saved_caches# rm *

12. Restart the controller and Cassandra database, then log out of the server:

/opt/sdn/cassandra/bin# service sdnc start/opt/sdn/cassandra/bin# exit


SDN websites

Hewlett Packard Enterprise NetworkingInformation Library for SDN

www.hpe.com/info/sdn/infolib

Hewlett Packard Enterprise Software-DefinedNetworking website

www.hpe.com/info/sdn

Hewlett Packard Enterprise SDN communitydiscussion forum

community.arubanetworks.com/t5/Software-Defined-Networking-SDN/bd-p/SDN

Hewlett Packard Enterprise SDN App Store www.hpe.com/networking/sdnappstore

Hewlett Packard Enterprise Open Sourcedownload website

www.hpe.com/software/opensource

Networking Websites

Hewlett Packard Enterprise Networking Information Library www.hpe.com/networking/resourcefinder

Hewlett Packard Enterprise Networking Software www.hpe.com/networking/software

Hewlett Packard Enterprise Networking website www.hpe.com/info/networking

Hewlett Packard Enterprise My Networking website www.hpe.com/networking/support

Hewlett Packard Enterprise My Networking Portal www.hpe.com/networking/mynetworking

Hewlett Packard Enterprise Networking Warranty www.hpe.com/networking/warranty

General websites

Hewlett Packard Enterprise Information Library www.hpe.com/info/EIL

For additional websites, see Support and other resources.

Chapter 12Websites


http://www.hpe.com/info/sdn/infolib

http://www.hpe.com/info/sdn

http://community.arubanetworks.com/t5/Software-Defined-Networking-SDN/bd-p/SDN

http://community.arubanetworks.com/t5/Software-Defined-Networking-SDN/bd-p/SDN

http://www.hpe.com/networking/sdnappstore

http://www.hpe.com/software/opensource

http://www.hpe.com/networking/resourcefinder

http://www.hpe.com/networking/software

http://www.hp.com/info/networking

http://www.hpe.com/networking/support


http://www.hpe.com/networking/warranty

http://www.hpe.com/info/EIL

Accessing Hewlett Packard Enterprise Support• For live assistance, go to the Contact Hewlett Packard Enterprise Worldwide website:

http://www.hpe.com/assistance• To access documentation and support services, go to the Hewlett Packard Enterprise Support Center website:

http://www.hpe.com/support/hpesc

Information to collect

• Technical support registration number (if applicable)• Product name, model or version, and serial number• Operating system name and version• Firmware version• Error messages• Product-specific reports and logs• Add-on products or components• Third-party products or components

Accessing updates• Some software products provide a mechanism for accessing software updates through the product interface.

Review your product documentation to identify the recommended software update method.• To download product updates:

Hewlett Packard Enterprise Support Center www.hpe.com/support/hpesc

Hewlett Packard Enterprise Support Center: Softwaredownloads

www.hpe.com/support/downloads

Software Depot www.hpe.com/support/softwaredepot• To subscribe to eNewsletters and alerts:

www.hpe.com/support/e-updates• To view and update your entitlements, and to link your contracts and warranties with your profile, go to the

Hewlett Packard Enterprise Support Center More Information on Access to Support Materials page:

www.hpe.com/support/AccessToSupportMaterials

Access to some updates might require product entitlement when accessed through the HewlettPackard Enterprise Support Center. You must have an HPE Passport set up with relevantentitlements.

Customer self repairHewlett Packard Enterprise customer self repair (CSR) programs allow you to repair your product. If a CSR partneeds to be replaced, it will be shipped directly to you so that you can install it at your convenience. Some partsdo not qualify for CSR. Your Hewlett Packard Enterprise authorized service provider will determine whether arepair can be accomplished by CSR.

Chapter 13Support and other resources

Chapter 13 Support and other resources 179

http://www.hpe.com/assistance



http://www.hpe.com/support/downloads

http://www.hpe.com/support/softwaredepot

http://www.hpe.com/support/e-updates

http://www.hpe.com/support/AccessToSupportMaterials

For more information about CSR, contact your local service provider or go to the CSR website:

http://www.hpe.com/support/selfrepair

Remote supportRemote support is available with supported devices as part of your warranty or contractual support agreement. Itprovides intelligent event diagnosis, and automatic, secure submission of hardware event notifications to HewlettPackard Enterprise, which will initiate a fast and accurate resolution based on your product's service level.Hewlett Packard Enterprise strongly recommends that you register your device for remote support.

If your product includes additional remote support details, use search to locate that information.

Remote support and Proactive Care information

HPE Get Connected www.hpe.com/services/getconnected

HPE Proactive Care services www.hpe.com/services/proactivecare

HPE Proactive Care service: Supportedproducts list

www.hpe.com/services/proactivecaresupportedproducts

HPE Proactive Care advanced service:Supported products list

www.hpe.com/services/proactivecareadvancedsupportedproducts

Proactive Care customer information

Proactive Care central www.hpe.com/services/proactivecarecentral

Proactive Care service activation www.hpe.com/services/proactivecarecentralgetstarted

Warranty informationTo view the warranty for your product, see the Safety and Compliance Information for Server, Storage, Power,Networking, and Rack Products document, available at the Hewlett Packard Enterprise Support Center:

www.hpe.com/support/Safety-Compliance-EnterpriseProducts

Additional warranty information

HPE ProLiant and x86 Servers and Options www.hpe.com/support/ProLiantServers-Warranties

HPE Enterprise Servers www.hpe.com/support/EnterpriseServers-Warranties

HPE Storage Products www.hpe.com/support/Storage-Warranties

HPE Networking Products www.hpe.com/support/Networking-Warranties

Regulatory informationTo view the regulatory information for your product, view the Safety and Compliance Information for Server,Storage, Power, Networking, and Rack Products, available at the Hewlett Packard Enterprise Support Center:

www.hpe.com/support/Safety-Compliance-EnterpriseProducts

Additional regulatory information

Hewlett Packard Enterprise is committed to providing our customers with information about the chemicalsubstances in our products as needed to comply with legal requirements such as REACH (Regulation EC No1907/2006 of the European Parliament and the Council). A chemical information report for this product can befound at:


http://www.hpe.com/support/selfrepair

http://www.hpe.com/services/getconnected

http://www.hpe.com/services/proactivecare

http://www.hpe.com/services/proactivecaresupportedproducts

http://www.hpe.com/services/proactivecareadvancedsupportedproducts

http://www.hpe.com/services/proactivecareadvancedsupportedproducts

http://www.hpe.com/services/proactivecarecentral

http://www.hpe.com/services/proactivecarecentralgetstarted

http://www.hpe.com/support/Safety-Compliance-EnterpriseProducts

http://www.hpe.com/support/ProLiantServers-Warranties

http://www.hpe.com/support/EnterpriseServers-Warranties

http://www.hpe.com/support/Storage-Warranties

http://www.hpe.com/support/Networking-Warranties

http://www.hpe.com/support/Safety-Compliance-EnterpriseProducts

www.hpe.com/info/reach

For Hewlett Packard Enterprise product environmental and safety information and compliance data, includingRoHS and REACH, see:

www.hpe.com/info/ecodata

For Hewlett Packard Enterprise environmental information, including company programs, product recycling, andenergy efficiency, see:

www.hpe.com/info/environment

Documentation feedbackHewlett Packard Enterprise is committed to providing documentation that meets your needs. To help us improvethe documentation, send any errors, suggestions, or comments to Documentation Feedback([email protected]). When submitting your feedback, include the document title, part number, edition,and publication date located on the front cover of the document. For online help content, include the productname, product version, help edition, and publication date located on the legal notices page.

Chapter 13 Support and other resources 181

http://www.hpe.com/info/reach

http://www.hpe.com/info/ecodata

http://www.hpe.com/info/environment

mailto:[email protected]

curl commandsThe Aruba VAN SDN Controller provides a RESTful web service API. There are several tools available foraccessing RESTful web service APIs, one of which is curl. This appendix shows some examples of accessing thecontroller's RESTul web service API with curl. For details on installing the curl application, see http://curl.haxx.se/download.html.

The curl application has many options, which are described in detail in the curl manual (run "curl --manual") andat http://curl.haxx.se/docs/manpage.html. The examples in this appendix use minimal options and assume anon-scripted, command line mode of execution and no conflicts with a web proxy. Additional options can be usedto customize your experience for your environment.

The -k option should only be used when issuing the request against an Aruba VAN SDN Controllerwith a self-signed certificate, which is installed by default. If a CA signed certificate is installed, the -k option should not be used. See http://curl.haxx.se/docs/sslcerts.html for further details.


Examples of curl commands in this guide use the --noproxy option, which is appropriate whereexecution of curl commands does not need a proxy to access controllers. If your network is set upsuch that a proxy is needed to access controllers, use the "--proxy" option. For details on curl proxyoptions, visit http://curl.haxx.se/docs/manpage.html.

The following sections describe some typical curl commands:

• Export audit log data as a CSV file using curl commands on page 183• Licensing actions using curl commands on page 183• Application manager actions using curl commands on page 186• Viewing metric data using curl commands on page 193• Team configuration using curl commands on page 196

About the curl commands• The backslash (\) character at the end of the line indicates that the command continues on the next line. In the

Bash shell, which you use to enter curl commands, a backslash character that is followed by the newlinecharacter is removed from the input stream automatically such that the command is processed as if it wereentered on a single line.

• When using a command in Linux, ensure that you replace any curly or smart quotation marks (“ “) with straightquotation marks (").

• Examples of curl commands in this document use the --noproxy option, which is appropriate whereexecution of curl commands does not need a proxy to access controllers. If your network is set up such that aproxy is needed to access controllers, use the --proxy option.

• Examples of curl commands in this document use the default user name and password for the controller. Yourcontroller user names and passwords might have been changed.

Appendix Acurl commands


http://curl.haxx.se/download.html

http://curl.haxx.se/download.html


http://curl.haxx.se/docs/sslcerts.html


For information about curl proxy options, see the man pages for curl.

Getting an authorization token using a curl commandProcedure

1. To get an authorization token using curl execute the following command:

curl --noproxy controller_ip -X POST --fail -ksSfL --url "https://controller_ip:8443/sdn/v2.0/auth" -H "Content-Type: application/json" --data-binary '{"login":{"domain": "domain","user": "user","password": "password"}}'

The output of the curl command contains the token.2. Without including the quotation marks, copy the value displayed for token.

Example

Example input:

$ curl --noproxy 10.1.1.10 -X POST --fail -ksSfL \--url "https://10.1.1.10:8443/sdn/v2.0/auth" \-H "Content-Type: application/json" \--data-binary \'{"login":{"user":"sdn","password":"skyline","domain":"sdn"}}'

Example output:

{"record":{"token":"237c78769e464508895f51410881ef26",”expiration":1368749850000,"expirationDate":"2013–05–16 17–17–30 —0700","userID":"a4fc1cecad844bc280953f983bbdcc26","userName":"sdn","domainID":"ba4e20f1c232401e8f75e9f318c0ae8a","domainName":"sdn"}}

Export audit log data as a CSV file using curl commandsTo export the audit log use the following command:

curl [options] -H "X-Auth-Token: token"\-H "Accept-Type: application/zip" \https://controller_ip:8443/sdn/v2.0/auditlog \-o zip-file-nameTo acquire the token for the command, see Getting an authorization token using a curl command.

For example, to export the current content in the controller audit log in a file named auditlogExport.CSVinside a zip file:

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \-H "Accept-Type: application/zip" \https://10.0.1.42:8443/sdn/v2.0/auditlog \-o auditlogExport.zip

Licensing actions using curl commands

Appendix A curl commands 183

Obtaining an install IDTo acquire the token for the command, see Getting an authorization token using a curl command.

Procedure

1. To obtain an Install ID, use the following command to obtain the SDN controller-assigned install_id value. :

curl [options] -H "X-Auth-Token:token" \https://controller_ip:8443/sdn/v2.0/licenses/installid • Replace token with the token created in step 1.• Replace controller_ip with your controller IP address.


A numerical install_id appears. For example: 12496792. Record your install_id for use in the next part of the license registration process.

Activating a license on the controllerUsing your license key, you must now activate a license on the controller, completing the license registration andactivation process.

Procedure

1. If your previous curl session has closed or timed out, re-enter the authentication command to obtain a newtoken.

2. Activate the license on the controller:

curl [options] -H "X-Auth-Token:token" \-d license_key \https://controller_ip:8443/sdn/v2.0/licenses a. Replace token with the token you obtained using the authentication command.b. Replace license_key with the key obtained in Registering your license and obtaining a license key. You

can view the key by logging on to the My Network portal and selecting My Licenses, as shown in Viewingyour license and other information.

c. Replace controller_ip with your controller IP address.


The installed license information appears in JSON format, as shown below. See: example

Installed license output

{ "license" : { "install_id" : 1249679, "serial_no" : 13, "license_metric" : "HA Controller",


"product" : "HP VAN SDN Ctrl Base", "metric_qty" : 500, "license_type" : "PRODUCTION", "base_license" : false, "creation_date" : "2013-09-06T00:26:52.248+0000", "activated_date" : "2013-09-06T00:26:52.248+0000", "expiry_date" : "2014-01-14T00:26:52.248+0000", "license_status" : "ACTIVE" }}

Uninstalling licenses to prepare for transfer

Prerequisites

To uninstall licenses, see Installing, activating, uninstalling, or transferring licenses on page 49.

Procedure

1. Obtain information about all installed licenses on your controller:

curl [options] -H "X-Auth-Token:token" \https://controller_ip:8443/sdn/v2.0/licenses a. Replace token with the token created in step 1.b. Replace controller_ip with your controller IP address.

If you are uninstalling a High Availability license, enter the IP address of the lead controller.

The installed license information appears in JSON format, as shown below.

All installed licenses output

{ "licenses" : [{ "install_id" : 12491640, "serial_no" : 12, "license_metric" : "Controller Node", "product" : "HP VAN SDN Ctrl Base", "metric_qty" : 52, "license_type" : "PRODUCTION", "base_license" : true, "creation_date" : "2013-09-06T00:26:52.248+0000", "activated_date" : "2013-09-06T00:26:52.248+0000", "expiry_date" : "2014-01-14T00:26:52.248+0000", "license_status" : "ACTIVE" },{} "licenses" : { "install_id" : 12491640, "serial_no" : 13, "license_metric" : "HA Controller", "product" : "HP VAN SDN Ctrl Base", "metric_qty" : 500,


"license_type" : "PRODUCTION", "base_license" : false, "creation_date" : "2013-09-06T00:26:52.248+0000", "activated_date" : "2013-09-06T00:26:52.248+0000", "expiry_date" : "2014-01-14T00:26:52.248+0000", "license_status" : "ACTIVE" }]}

2. Record each serial_no value.3. Uninstall or deactivate each active license on your controller:

curl [options] -H "X-Auth-Token:token"-d deactivate \https://controller_ip:8443/sdn/v2.0/licenses/serial_number/action a. Replace token with the token you obtained using the authentication command.b. Replace controller_ip with your controller IP address.


c. Replace serial_number with the serial number of the license you want to deactivate. You can view the keyby logging on to the My Network portal and selecting My Licenses, as shown in The license uninstallkey appears in JSON format, as shown below.License uninstall key output

{ "license" : { "install_id" : 1249679, "serial_no" : 13, "license_metric" : "HA Controller", "product" : "HP VAN SDN Ctrl Base", "metric_qty" : 500, "license_type" : "PRODUCTION", "base_license" : false, "creation_date" : "2013-09-06T00:26:52.248+0000", "activated_date" : "2013-09-06T00:26:52.248+0000", "expiry_date" : "2014-01-14T00:26:52.248+0000", "license_uninstall_key" : "MYOCD9JMCRRRM-IRTEQ2QUNBYCB-6Q6CJIEIJFKIQ-VAI2QUJBYC433" "license_status" : "INACTIVE" }}

4. Record your license_uninstall_key. Security token obtained from output, for example:

The license_uninstall_key obtained from the example in the previous step is:

MYOCD9JMCRRRM-IRTEQ2QUNBYCB-6Q6CJIEIJFKIQ-VAI2QUJBYC433

Application manager actions using curl commandscurl commands can be used to perform actions on embedded or installed SDN applications.


Listing applications

Procedure

1. Form

curl [options] -H "X-Auth-Token:token" \https://controller_ip:8443/sdn/v2.0/apps

2. Example of listing applications

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \https://10.0.1.42:8443/sdn/v2.0/apps

3. Example output:

{ "apps": [ { "action": "NONE", "catalog_id": "", "deployed": "2014-06-18T19:22:49.536Z", "desc": "Path Diagnostic Utility", "download_url": "", "name": "Path Diagnostics", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.hp.sdn.ctl.diag", "vendor": "Hewlett-Packard", "version": "2.3.5.6370" }, { "action": "NONE", "catalog_id": "", "deployed": "2014-06-18T19:22:50.890Z", "desc": "Link Management", "download_url": "", "name": "Link Manager", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.hp.sdn.ctl.linkdisco", "vendor": "Hewlett-Packard", "version": "2.3.5.6370" } ] }


Listing information about an application

Procedure

1. Form

curl [options] -H "X-Auth-Token:token" \https://controller_ip:8443/sdn/v2.0/apps/app_id

2. Example

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \https://10.0.1.42:8443/sdn/v2.0/apps/com.hp.sdn.ctl.diag

3. Example output

{ "app": { "action": "NONE", "catalog_id": "", "deployed": "2014-06-18T19:22:49.536Z", "desc": "Path Diagnostic Utility", "download_url": "", "name": "Path Diagnostics", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.hp.sdn.ctl.diag", "vendor": "Hewlett-Packard", "version": "2.3.5.6370" } }

Getting application health status

Procedure

1. The HEAD command on health status returns only the response code rather than the entire message formanagement-type clients that want to poll for health status. Returns HTTP status as follows:

200 for healthy290 for unhealthy295 for critical

2. Form

curl [options] -H "X-Auth-Token:" -w %{http_code} \-X HEAD https://controller_ip:8443/sdn/v2.0/apps/app_id/health


3. Example

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" -w %{http_code} \-X HEAD https://10.0.1.42:8443/sdn/v2.0/apps/com.hp.sdn.ctl.diag/health

4. Example output

200

Uploading an application (new or upgrade)

Procedure

1. Form

curl [options] -H "X-Auth-Token:token" \-X POST https://controller_ip:8443/sdn/v2.0/apps/ \--data-binary @<full_path_to_app_zip>

2. Example

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \-X POST https://10.0.1.42:8443/sdn/v2.0/apps/ \--data-binary @/home/hummer/dev/flare/dist/testApps/geewiz-apps-1.0.0.zip

3. Example output (new)

{ "app": { "action": "NONE", "catalog_id": "", "deployed": "1970-01-01T00:00:00.000Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "", "state": "STAGED", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "1.0.0" } }

4. Example output (upgrade)

{ "app": { "action": "NONE", "catalog_id": "", "deployed": "2014-06-18T23:04:25.955Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "",


"state": "UPGRADE_STAGED", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "2.0.0" } }

Installing a new application

Procedure

1. Form

curl [options] -H "X-Auth-Token:" \-X POST https://controller_ip:8443/sdn/v2.0/apps/app_id/action \-d install

2. Example

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \-X POST https://10.0.1.42:8443/sdn/v2.0/apps/com.geewiz/action \-d install

3. Example output

{ "app": { "action": "NONE", "catalog_id": "", "deployed": "2014-06-18T21:46:39.845Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "1.0.0" } }

Upgrading an application

Procedure

1. Form

curl [options] -H "X-Auth-Token:token" \-X POST https://controller_ip:8443/sdn/v2.0/apps/app_id/action \


-d upgrade

2. Example

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \-X POST https://10.0.1.42:8443/sdn/v2.0/apps/com.geewiz/action \ -d upgrade

3. Example output


Disabling an application

Procedure

1. Form

curl [options] -H "X-Auth-Token:token" \-X POST https://controller_ip:8443/sdn/v2.0/apps/app_id/action \-d disable

2. Example

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \-X POST https://10.0.1.42:8443/sdn/v2.0/apps/com.geewiz/action \-d disable

3. Example output

{ "app": { "action": "NONE", "catalog_id": "", "deployed": "2014-06-18T23:04:25.955Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "",


"state": "DISABLED", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "2.0.0" } }

Enabling an application

Procedure

1. Form

curl [options] -H "X-Auth-Token:token" \-X POST https://controller_ip:8443/sdn/v2.0/apps/app_id/action \-d enable

2. Example

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \-X POST https://10.0.1.42:8443/sdn/v2.0/apps/com.geewiz/action \-d enable

3. Example output


Removing a staged application

Procedure

1. This curl request is used to remove a newly uploaded application before it is installed or upgraded. It has nooutput.

2. Form

curl [options] -H "X-Auth-Token:token" \-X POST https://controller_ip:8443/dn/v2.0/apps/app_id/action \


-d cancel

3. Example

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \-X POST https://10.0.1.42:8443/sdn/v2.0/apps/com.geewiz/action \-d cancel

Deleting an application

Procedure

1. This curl request is used to shutdown and completely remove all application versions. It has no output.2. Form

curl [options] -H "X-Auth-Token:token" \-X DELETE https://controller_ip:8443/sdn/v2.0/apps/app_id

3. Example

curl -ksS -H "X-Auth-Token:3d61f0d3e61349359e6dbd82ec02c113" \-X DELETE https://10.0.1.42:8443/sdn/v2.0/apps/com.geewiz

Viewing metric data using curl commandsYou can use curl commands view the metric data persisted by the controller components and installedapplications. For more information, see Metrics on page 149.

Managing SNMP keysNetwork management systems can use SNMP (Simple Network Management Protocol) to monitor network-attached devices for conditions that require administrative attention.

As part of the switch identification process, the controller might use SNMP to obtain more specific informationabout the switch. SNMP-enabled switches can be configured with community names to provide more secureaccess.

The Key Service component of the controller allows network administrators to configure SNMP security keys sothat the controller can communicate with those switches using the secure key.

Getting the SNMP keys

Procedure

1. To get the SNMP keys known to the controller, enter the following curl command:

curl --header "X-Auth-Token: token" -sX GET http://controller_ip:8080/sdn/v2.0/net/keys/SNMP Sample command

curl --header "X-Auth-Token: 131eaa225ece4293bcebfd7f8e3cffd0" -sX GET http://localhost:8080/sdn/v2.0/net/keys/SNMP


Sample Response

{ "SNMP": [ "Default SNMP key", "Test Key1", "Test Key2", "Test Key3" ]}

Adding SNMP keys

Procedure

1. To add one or more SNMP keys to the controller:

curl --header "Content-Type: application/json" --header "X-Auth-Token:token" -sX POST -d @inputfile http://controller_ip/sdn/v2.0/net/keys Where inputfile is a file in the local directory that contains the key information in JSON format.

Sample command

curl --header "Content-Type: application/json" --header "X-Auth-Token:18433683aaea4e0782fa632ee9f04953" -sX POST -d @inputfile http://localhost:8080/sdn/v2.0/net/keysSample content of inputfile for an SNMP v1 key{"key": { "type": "SNMP", "description": "v1Key", "snmp-config": { "snmpversion":"v1", "readcommunityname": "public", "writecommunityname": "public" } } }Sample content of inputfile for an SNMP v3 key{"key": { "type": "SNMP", "description": "authPriv", "snmp-config":{ "snmpversion": "v3", "username": "authPriv", "authorization": { "type":"SHA","password":"MySHApassword" }, "privacy": { "type":"AES","password":"MyAESpassword" } } } }

Deleting an SNMP keyTo delete an SNMP key from the controller, enter the following curl command:

curl --header "X-Auth-Token: token" -sX DELETE http://controller_ip:8080/sdn/v2.0/net/keys/SNMP/description/description Where description matches the description for the key you want to delete.

Sample command

curl --header "X-Auth-Token: 131eaa225ece4293bcebfd7f8e3cffd0" -sX DELETE http://localhost:8080/sdn/v2.0/net/keys/SNMP/description/"authPriv"


Managing NETCONF keysSome devices provide support NETCONF (Network Configuration Protocol).

The Key Service component of the controller allows network administrators to configure NETCONF security keysso that the controller can communicate with those switches using the secure key.

Getting the NETCONF keys

Procedure

1. To get the NETCONF keys known to the controller:

curl --header "X-Auth-Token: token" -sX GET http://controller_ip:8080/sdn/v2.0/net/keys/NETCONF Sample command

curl --header "X-Auth-Token: 131eaa225ece4293bcebfd7f8e3cffd0" -sX GET http://localhost:8080/sdn/v2.0/net/keys/NETCONFSample Response

{ "NETCONF": [ "Default Netconf key" ]}

Adding NETCONF keys

Procedure

1. To add one or more NETCONF keys to the controller:

curl --header "Content-Type: application/json" --header "X-Auth-Token:token" -sX POST -d @inputfile http://controller_ip/sdn/v2.0/net/keys Where inputfile is a file in the local directory that contains the key information in JSON format.

Sample command

curl --header "Content-Type: application/json" --header "X-Auth-Token:18433683aaea4e0782fa632ee9f04953" -sX POST -d @inputfile http://localhost:8080/sdn/v2.0/net/keysSample content of inputfile for a NETCONF key{"key": { "type": "NETCONF", "description": "SampleNetconfKey" } }

Deleting a NETCONF keyTo delete a NETCONF key from the controller, enter the following curl command:

curl --header "X-Auth-Token: token" -sX DELETE http://controller_ip:8080/sdn/v2.0/net/keys/NETCONF/description/description


Where description matches the description for the key you want to delete.

Sample command

curl --header "X-Auth-Token: 131eaa225ece4293bcebfd7f8e3cffd0" -sX DELETE http://localhost:8080/sdn/v2.0/net/keys/SNMP/description/"SampleNetconfKey"

Team configuration using curl commandsThe following curl commands can be used to configure teaming.

Examples of curl commands in this section use the --noproxy option, which is appropriate whereexecution of curl commands does not need a proxy to access controllers. If your network is set upsuch that a proxy is needed to access controllers, use the --proxy option. For details on curl proxyoptions, visit http://curl.haxx.se/docs/manpage.html.

Creating a team using curlBefore creating a team ensure that your environment meets the requirements for teaming, see Requirements forcontroller teams on page 102. And for each controller that will become a member of the team, configure NTP touse a centralized NTP daemon, see Configuring controllers to use the same local NTP servers.

Considerations when a controller team is formed using RESTThe default configuration of the SDN Controller is the system’s eth0 interface. When a controller team is formedvia REST with the team IP Address, an alias in the controller elected as leader will be configured automatically bythe system and will attach to the eth0 interface by default. If the SDN Controller has multiple Ethernet interfaces,a different interface can be required for the team IP Address. In this case the configuration /etc/sdn/admin/options might be changed using vim or emacs to reflect the desired configuration.

sdncontroller:/opt/sdn/admin# cat optionsexport ADMIN_OPTS="-Dcom.hp.sdn.admin.interface=eth0" Once the change has been made, the SDNA service must be restarted as shown with the following command:

sdncontroller:/opt/sdn/admin#sudo service sdna restart

sdna stop/waitingThis change must be made for every active controller within the team and does not require that the team bedeleted via REST.

To view the team IP Address designation from the SDN Controller console or SSH session, use the ifconfigcommand:

sdncontroller:$ ifconfig eth0 Link encap:Ethernet HWaddr ac:16:2d:9a:62:60 inet addr:172.17.3.17 Bcast:172.17.15.255 Mask:255.255.240.0 inet6 addr: fe80::ae16:2dff:fe9a:6260/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1151070 errors:0 dropped:284 overruns:0 frame:0 TX packets:1134356 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:684988786 (684.9 MB) TX bytes:882495744 (882.4 MB) Memory:f7f80000-f8000000 eth0:0 Link encap:Ethernet HWaddr ac:16:2d:9a:62:60



inet addr:172.17.3.41 Bcast:172.17.15.255 Mask:255.255.240.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Memory:f7f80000-f8000000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:116581 errors:0 dropped:0 overruns:0 frame:0 TX packets:116581 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:32894518 (32.8 MB) TX bytes:32894518 (32.8 MB)

Configuring a controller team using curlThis section describes configuring a controller team using curl commands.

Team authentication is managed with iptables. The team communication channel is not encrypted. If youdeploy the team in a highly secure environment, you can always use standard Linux IPSec functionality to encrypttraffic between the team members.

You will need to setup bi-directional rules on each controller for communication with each of the other twocontrollers using the IP addresses specified during the team creation. For simplicity, you can configure host tohost IPsec and secure all traffic between the controllers. However, all external communication channels otherthan teaming are already secured with SSL. If you only want to secure the team communication channel, the rulesshould be created to only encrypt port 5700.

Do not use the team leader IP address in any of the IPsec rules.

For more information on how to configure IPsec for Ubuntu Linux:

https://help.ubuntu.com/community/IPSecHowTo

http://manpages.ubuntu.com/manpages/precise/man5/ipsec.conf.5.html

http://manpages.ubuntu.com/manpages/precise/man5/ipsec.secrets.5.html

Prerequisites (SDN controller 2.8)

The default domain and user settings are sdn. The default password setting is skyline.


Procedure

1. Install and start three standalone controllers in the network. (See the latest Aruba VAN SDN ControllerInstallation Guide.)

2. Optional: To improve security, you can change the username and password from the default settings on eachof the standalone controllers in step 1.

3. Select any one of the controllers to use for configuring the team.4. On the selected controller, acquire an Authentication Token. Use the following curl command, with the

controller IP address, to acquire the token:


https://help.ubuntu.com/community/IPSecHowTo

http://manpages.ubuntu.com/manpages/precise/man5/ipsec.conf.5.html

http://manpages.ubuntu.com/manpages/precise/man5/ipsec.secrets.5.html

curl --noproxy controller_ip> -X POST --fail-ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/auth"-H "Content-Type: application/json" --data-binary '{"login":{"domain": "<domain>","user": "<user>","password": "<password>"}}'In this example, the default domain, user name, and password are used.

curl --noproxy 192.15.135.187 -X POST --fail -ksSfL --url "https://192.15.135.187:8443/sdn/v2.0/auth" -H "Content-Type: application/json" --data-binary '{"login": {"domain": "sdn","user": "sdn","password": "skyline"}}'The command generates the authentication token 1759f214479e4ffd9504acb42123ef40.

{"record":{"token":"1759f214479e4ffd9504acb42123ef40","expiration":1381982391381982399000,"expirationDate":"2014-10-16 20-59-59 -0700","userId":"b00cb0e94c9441d58011f980cf9635ae","userName":"sdn","domainId":"a6701f6593d84fa5b8f23f9ab4ed69db","domainName":"sdn"}}

5. Determine the team configuration parameters:

Parameter Value

Team IP Address The team IP address is different from the individualcontroller IP addresses. It is used as a virtual address forconnecting to the team leader.

Controller IP Address IP address of a team member.

When the virtual address is programmed on the team leader, gratuitous ARP is sent out. Thevirtual address moves from one controller to another in the team as the leader changes. If anysecurity features are configured to prevent such a move, they should be reconfigured to allow themovement of team IP Address such that it remains reachable for the rest of the network.

Configuration procedure

Procedure

1. Select any active controller to initially configure the team.2. Enter the following curl command:

curl -m 240 --noproxy member-1-ip --header X-Auth-Token:auth_token --fail -ksS --request POST --url https://member-1-ip:8443/sdn/v2.0/team --data-binary '{"team":{"ip":"team-ip","members":[{"ip":"member-1-ip"},{"ip":"member-2-ip"},{"ip":"member-3-ip"}]}}'

The member-1-ip should be the IP address of the controller chosen to configure the team.

After executing the command in step 2, the team elects a team leader. The team leader then configures allteam members and normal controller operation begins in the domain. The team creation command does notblock until the team creation is complete. You will need to check the status of the system to verify on eachcontroller that the team was successfully created.

The team create POST command may now take up to 4 minutes to complete. You will need to make sure theread timeout for the client request is increased accordingly.

Configuration example


This example shows a team of controllers configured with the following team member values:

• Team IP Address: 192.0.2.100• Member IP Addresses:

◦ 192.0.2.119◦ 192.0.2.125◦ 192.0.2.127

• Domain: sdn (the default domain name)• Username: myname• Password: mypass

It is not mandatory that the team IP address be in the same subnet as the member IP addresses.Other IP aliases can be used if the appropriate IP routes are present for the addresses to bereachable and usable.

The following steps create and enable the team:

1. Acquire the authentication token:

curl --noproxy 192.0.2.119 -X POST --fail -ksSfL --url "https://192.0.2.119:8443/sdn/v2.0/auth" -H "Content-Type: application/json" --data-binary '{"login": {"domain": "sdn","user": "myname","password": "mypass"}}'Command response

{"record":{"token":"10f728e477cb4612b07069f339d0ca29","expiration":1381119301000,"expirationDate":2013-12-06 21-15-01-0700","userId":"51802e12d16345fe9a4389290c1a04e2","username":"sdn","domainId":"d45eca9bde1b4dc78bd7dff69ee9440d","domainName":"sdn"}}

2. Configure the controller team by using the team values and token from step 1:

curl -m 240 --noproxy 192.0.2.119 --header X-Auth-Token: 10f728e477cb4612b07069f339d0ca29 --fail -ksS --request POST --url https://192.0.2.119:8443/sdn/v2.0/team --data-binary '{"team":{"ip":"192.0.2.100","members":[{"ip":"192.0.2.119"},{"ip":"192.0.2.125"},{"ip":"192.0.2.127"}]}}'

Since team creation is asynchronous, the response is always 202 unless the team configuration (JSON) is notvalid or there is a problem configuring the local controller. Possible codes are:

202 Accepted400 Bad request401 Unauthorized503 Service unavailable

In case the team is not created in a quorum or if the team is partially created, an alert is posted.

Example of an alert for a team partially created

Team partially created: [Successes: 192.168.1.1, 192.168.1.2], [Failures: 192.168.1.3]The alert does not include the error description, however, the error description is added to the log files. Reviewthe log files to get the cause of the partial team creation.

Example of an alert for a team creation that failed in a quorum


Team could not be created on a quorumIf a team cannot be created in a quorum, delete the team and create it again.

Error log for team configurationTable 5: Error log for team configuration

Log message Description

Build version not consistent on all the systems. Not all systems on the team have the same controllerbuild version. Update the team as needed to have thesame build version.

Invalid configuration. The team configuration JSON is not valid.

Local member must be part of the team configuration. If the members list from the JSON configuration doesnot include the system where the team is being created(The local system).

Team size must be greater than zero. Aruba VAN SDN Controller 2.5 only supports a team of3 controllers.

A team has already been created. Teaming is already running on the system.

Team could not be created on a quorum. Team configuration has failed on a majority of systems.For example, a team of three systems has experiencedfailures on two systems.

Team could not be deleted on a quorum. A team delete has failed on a quorum number ofsystems.

Team not configured on this system. An attempt has been made on a standalone controllerto disband a team.

Programming team alias ip-address failed. See Team alias node on page 202.

Unprogramming team alias ip-address failed. See Team alias node on page 202.

Table Continued


Log message Description

Recovering from Partial Team Creation • In case the team is not successfully created in allcontrollers, it is not possible to fix the failedcontrollers without disbanding the team. To recoverfrom this failure it is recommended to disband theteam, fix the problem in the controllers where thecreate operation failed, and try again.

• Team configuration can also fail if the iptablesrules for Hazelcast or Cassandra are notsuccessfully programmed on the Linux OS.

Recovering from Partial Team Deletion If the team was not successfully disbanded in allcontrollers, the failed controllers might go tosuspended mode because they might not have quorum– they won’t be able to connect to those controllerswhere the operation was a success. To recover fromthis failure it is recommended to disband the team oneach failed controller so configuration files areremoved and so the controllers transition to standalonemode. Or you may need to reinstall the controller.

Table 6: Success log

Message Description

Team created. The controller has completed all required steps toconfigure itself as part of the team.

Team created with the following configuration: [TeamIP: <team ip>, [Members<member list>].

The controller has completed all required steps toconfigure itself as part of the team. The log entryincludes the team configuration provided by the user.

Team disbanded. The controller has completed all required steps toconfigure itself as a standalone controller.

Programmed Team alias: <team ip>. The controller has configured the team IP address asan alias. This results in the election of a leader.

Unprogrammed Team alias: <team ip>. The controller has removed the team IP address fromthe list of aliases. This results in the election of aleader.


Table 7: Team IP error log

Message Description

Exception while checking alias: <team ip>,<exception>

The controller is unable to verify whether the team IPaddress has been configured as an alias.

Team alias: <team ip> already programmed Represents an invalid state. The controller is trying toconfigure the team IP address when it has beenalready configured.

Exception while programming alias: <team ip>,<exception>

The controller was unable to configure the team IPaddress as an alias.

Exception while unprogramming alias: <team ip>,<exception>

The controller was unable to remove the team IPaddress from the aliases.

Team alias nodeAn IP Address (North-Bound IP) alias is created on the node that is elected as team leader to allow a controllerteam to be accessible with a single IP Address no matter which controller is the leader. This IP Address isprovided as part of the team configuration when creating a team. If the elected node stops being the team leader,the team IP Address must be removed from the interface because this address must be reassigned to the actualteam leader. This must be done as the sdn user. If assigning or removing an alias fails, one of the followingmessages appears in the Alert log:

• Programming team alias ip-address failed• Unprogramming team alias ip-address failed

In either of these instances, the condition is logged and the team continues to operate. In this case you canmanually program the team alias.

You can configure the alias as follows:

sudo ifconfig <alias interface> <alias IP> netmask <net mask address> upExample command

sudo ifconfig eth0:0 192.0.123.5 netmask 255.255.255.0 upYou can disable the alias as follows:

sudo ifconfig <alias interface> <alias IP> netmask <net mask address> down Example command

sudo ifconfig eth0:0 192.0.123.5 netmask 255.255.255.0 down

Cassandra database maintenance in a teamSome applications may choose to use the Cassandra database to store persistent data. For these applications,there is an extra maintenance step that you must run once every ten days to help maintain the performance of thedatabase and the consistency across the team:

1. Make sure all controllers are up and Cassandra is running:

/opt/sdn/cassandra/bin/nodetool ring |grep -c 'Up'


This command must return 3.2. Login to each controller in sequence and run the following command. The command must be run as the sdn

user.

/opt/sdn/cassandra/bin/nodetool repair

Do not attempt to run this command at the same time on different controllers. It must run tocompletion on one controller before you run it on another. The command may have a significanteffect on disk and network I/O across all controllers in the team and it may take some time. Runthis command during a low usage window.

Disbanding a team using curlDisbanding a team returns the teamed controllers to standalone operation. This action initiates the team delete.The REST call might return before the delete has completed. You must check the system to see the running stateof the system.

Procedure

1. Before disbanding a team, delete the region configuration for that team. (See Removing a region using curlon page 215.)

2. Acquire an authentication token for the team leader. (See step 4 of Prerequisites)3. Using the token acquired in the step 2, disband the team:

curl --noproxy team-ip --header "X-Auth-Token:auth_token" --fail-ksSfL --request DELETE --url https://member-ip:8443/sdn/v2.0/team The deletion of the team can take up to 4 minutes to complete. Increase the read timeout for the client requestaccordingly.

For the controller to be fully operational in standalone after a team is disbanded, you must completely re-installthe controller on each node.

Since team deletion is asynchronous, the response is always 202 unless there is a problem configuring the localcontroller as standalone. Possible codes are:

202 Accepted400 Bad request401 Unauthorized503 Service unavailable

If you get a returned error of 500, it might be that all regions have not been removed.

In case the team is not deleted in a quorum or if the team is partially deleted, an alert is posted:

Team partially deleted: [Successes: 192.168.1.1, 192.168.1.2], [Failures: 192.168.1.3]The alert does not include the error description, however, the error description is added to the log files. Reviewthe log files to get the cause of the partial team deletion.

To recover from this failure, Hewlett Packard Enterprise recommends that you delete the team on each failedcontroller, which will remove the configuration files and transition the controllers to standalone mode.

Example of an alert for a team deletion that failed in a quorum


Team could not be deleted on a quorumIf a team cannot be deleted in a quorum, delete the failed controllers individually.

Viewing the team configuration using curlTo view your team configuration, do as follows:

Procedure

1. Acquire an authentication token for the team leader. (See step 4 of Prerequisites)2. Using the token acquired in Step 1, view the team configuration as follows:

curl --noproxy team-ip --header "X-Auth-Token:auth_token" --fail -ksSfL --request GET --url https://team-ip:8443/sdn/v2.0/team

curl --noproxy 192.0.2.100--header "X-Auth-Token:<auth_token>" --fail -ksSfL --request GET --url https://192.0.2.100:8443/sdn/v2.0/team

{ "team": { "ip": "192.0.2.100", "revision":0 "members": [ { "ip": "192.0.2.119" }, { "ip": "192.0.2.125" }, { "ip": "192.0.2.127" } ]}}

Creating regions using curlTo support High Availability (HA) for controllers to OpenFlow switches, create region configurations in thecontrollers using the REST APIs provided by the Device Owner Service.

A region groups devices together with their controllers. Every region has a unique identifier (UID) assigned uponcreation. Some REST commands will require that UID to manage the region. A region must have three controllerswhich must be specified in priority order for all devices within the region (master, primary slave, secondary slave).Devices in a region can be expressed as a list of individual IPv4 addresses, a list of IPv4 ranges, or a combinationof both.

The Device Owner Service provides high availability between devices and controllers and ensures the availabilityof a controller to the devices. The Device Owner Service also provides a measure of security; only devicesexplicitly included in a region can connect to the region’s controllers; thus if no regions are defined for the teamedcontrollers, then no devices will be able to connect to the controllers.

Putting the region configurations in place for a controller team ensures seamless failover and failback among theconfigured controllers for the specified network devices in a region. That is, when a controller experiences a fault,the Device Owner Service ensures that a slave controller immediately assumes the master role over the group ofnetwork devices for which the failed controller was master. Once the failed controller recovers and rejoins theteam, the Device Owner Service ensures restoration of this controller’s role; that is, the rejoining controller takesback the role for which it was configured with respect to the other network devices. If the controller was configured


to operate as the master in a region, then it would be restored to the master role. If it was configured to operate inthe slave role, it would resume operation in the slave role.

Once the region definition(s) are in place, the Device Owner Service ensures that a master controller is alwaysavailable to the respective network element(s) even if the configured master fails or there is a disruption of thecommunication channel between the controller and the network device(s).

• IPv6 addresses occurring in any region field are not supported and will cause the region to beignored.

• All region configuration operations (create, update, refresh, and delete) using the RESTAPI require that every controller specified in the team, including the master controller and all slavecontrollers, be in an active state. If any controller in the region is in a suspended orunreacheable state, then the region configuration operations are disallowed.

Regions and device ownershipRegions in standalone environments

In a standalone environment, regions are not enforced, and devices are owned by the standalone controller.

Regions in teamed environments

In a teamed environment, assigning devices and controllers to regions can ensure controller availability to deviceswhen one of the following situations occur:

• A single controller fails.• A single controller is disconnected from the other controllers in the team.• A single device is disconnected from one or two controllers in the team.

Each region is configured with a prioritized list of controllers, which are used by the Device Owner Service toassign specific roles to each controller:

master The highest-priority controller is the configured master controller. The controller that has the role ofmaster for a region is also considered the owner of the devices in the region. For a given device, onlythe controller that currently has the role of master controller can write to or modify the device. Thecontroller that has the role of master at any given time might not be the configured master controller.For example, the configured master controller might be offline.

slave Controllers with this role can read the configurations of the network devices that are managed by theregion, but cannot write or modify those configurations. The slave controllers, in priority order, are theconfigured primary slave and the configured secondary slave. Controllers with a configured role of slavecan be assigned the role of master controller temporarily, such as when the configured master controlleris offline or a device has lost connection to both the configured master controller and the primary slavecontroller.

Failover behavior within a regionDevice Owner Service triggers the failover operation in two cases:

• Controller failure

: The Device Owner Service detects a controller failure in a team through notifications from the teamingsubsystem. If Device Owner Service determines that the failed controller instance was a master for anydevices within a region, it immediately elects an appropriate backup (slave) controller to assume the masterrole over the affected devices.

• Device disconnect

: The Device Owner Service instance in a controller is notified of a communication failure with networkdevice(s) through the Controller Service notifications. It instantly communicates with all Device Owner Serviceinstances in the team to determine if the network device(s) in question are still connected to any of the backup


(slave) controllers within the team. If that is the case, it elects one of the slaves to assume the master role overthe affected network device(s).The first slave will be chosen as master if it still has connectivity with thedevice(s), and the second slave will be chosen as master if neither the configured master or first slave haveconnectivity with the device(s).

In this example, the master controller 172.17.6.70 fails. Although it is still in the region, it is unavailable to thedevices for which it is the configured master and is no longer the master controller. The primary slave controller172.17.6.71 becomes the master controller. All 5 devices 192.168.1.101–105 now belong to the failovermaster controller 172.17.6.71.

Example command

curl --noproxy teamIp --header "X-Auth-Token:token" -ksS --request GET --url https://teamIp:8443/sdn/v2.0/owners/region-uid/state?dataPathDetails=yes Example output

{ "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"Region-Red", "prioritizedControllerIps":[ "172.17.6.70", "172.17.6.71", "172.17.6.72" ], "devices":[ { "deviceIp":"192.168.1.101", "owningControllerIp":"172.17.6.71", "dataPaths":[ { "dpid":"00:01:44:31:92:5c:af:86", "owningControllerIp":"172.17.6.71" } ] }, { "deviceIp":"192.168.1.102", "owningControllerIp":"172.17.6.71", "dataPaths":[ { "dpid":"00:1e:c8:cb:b8:dd:f0:c0", "owningControllerIp":"172.17.6.71" } ] }, { "deviceIp":"192.168.1.103", "owningControllerIp":"172.17.6.71", "dataPaths":[ { "dpid":"00:1e:f0:92:1c:21:af:00", "owningControllerIp":"172.17.6.71" } ] }, { "deviceIp":"192.168.1.104", "owningControllerIp":"172.17.6.71", "dataPaths":[


{ "dpid":"00:01:cc:3e:5f:6b:19:00", "owningControllerIp":"172.17.6.71" } ] }, { "deviceIp":"192.168.1.105", "owningControllerIp":"172.17.6.71", "dataPaths":[ { "dpid":"00:1e:00:9c:02:e0:e4:00", "owningControllerIp":"172.17.6.71" } ] } ] } ]}

Failback behavior within a regionWhen the configured master recovers from a failure and rejoins the team, or when the connection from thedisconnected device(s) with the original master is resumed, Device Owner Service initiates a failback operation inwhich the master role is restored to the configured master as defined in the region definition.

In this example, the configured master controller 172.17.6.70 recovers from failure and is active again. Thecontroller 172.17.6.71 returns back to being a primary slave and all 5 devices 192.168.1.101–105 belong tothe configured master controller 172.17.6.70.

Example command

curl --noproxy teamIp --header "X-Auth-Token:token" -ksS --request GET --url https://teamIp>:8443/sdn/v2.0/owners/region-uid/state?dataPathDetails=yes Example output

{ "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"Region-Red", "prioritizedControllerIps":[ "172.17.6.70", "172.17.6.71", "172.17.6.72" ], "devices":[ { "deviceIp":"192.168.1.101", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:01:44:31:92:5c:af:86", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.102", "owningControllerIp":"172.17.6.70",


"dataPaths":[ { "dpid":"00:1e:c8:cb:b8:dd:f0:c0", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.103", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:1e:f0:92:1c:21:af:00", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.104", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:01:cc:3e:5f:6b:19:00", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.105", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:1e:00:9c:02:e0:e4:00", "owningControllerIp":"172.17.6.70" } ] } ] } ]}

Adding a region using curlThis POST command adds a region to those configured on the controller and propagates the modifications to eachcontroller in the team. All controllers configured for the team must be available for such a configuration change tobe permitted.

In this example, we are adding a region with UID 713def9a-4f96-485f-990c-8924bc06c8d8 that has threecontrollers: 172.17.6.70 (master), 172.17.6.71 (primary slave), 172.17.6.72 (secondary slave). Twodevices are part of the region: 192.168.1.101 and 192.168.1.10. The devices IP range includes192.168.1.103, 192.168.1.104, 192.168.1.105Example command

curl --noproxy controllerIp --header "X-Auth-Token:token" --header "Content-Type:application/json" -ksS --request POST --url https://controllerIp:8443/sdn/v2.0/owners --data-binary '{"region": {"name": "Region-Red", "prioritizedControllerIps": ["172.17.6.70", "172.17.6.71", "172.17.6.72"], "deviceIps":["192.168.1.101", "192.168.1.102"],"deviceIpRanges": ["192.168.1.103-105"]}}'


Example output

{ "region":{

"uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"Region-Red", "prioritizedControllerIps":[ "172.17.6.70", "172.17.6.71", "172.17.6.72" ], "deviceIps":[ "192.168.1.101", "192.168.1.102" ], "deviceIpRanges":[ "192.168.1.103-105" ] }}If deviceIps or deviceIpRanges are not in numeric order, they are reordered in the response and insubsequent GET calls of the configuration.

Adding a device to a region using curlThis POST command adds a device with the specified IP address to the region with the specified UID andpropagates the modifications to each controller in the team. A deviceIp query parameter must be specified. Youcan only add one IP address at a time in order to know which device IP address is involved when an errorcondition is encountered. The new device IP address will be added to the deviceIps list, unless it alreadyappeared in the deviceIps list or was encompassed by one of the ranges listed among the deviceIpRanges,in which case no change will be made. All controllers configured for the team must be available for such aconfiguration change to be permitted.

In this example, for region UID 713def9a-4f96-485f-990c-8924bc06c8d8, device 192.168.1.103 isadded to the region.

Example command

curl --noproxy teamIp --header "X-Auth-Token:token" -ksS --request POST --url https://teamIp:8443/sdn/v2.0/owners/region_uid/devices?deviceIp=192.168.1.103 Example output

{"result":"Device IP address 192.168.1.103 now exists in the region with UID '713def9a-4f96-485f-990c-8924bc06c8d8'"}To check if your device was added to the region, check the Team screen in the controller UI or see Getting theconfiguration of a specific region using curl on page 210.

Getting the configuration of all regions using curlThis GET command retrieves the configuration of all regions. The regions configuration may have been modifiedsince controller startup to reflect the dynamic addition or removal of regions or devices within specific regions.

In this example, there were no changes since controller startup and the configuration is the same is in Adding aregion using curl on page 208.

Example command


curl --noproxy teamIp --header "X-Auth-Token:token" -ksS --request GET --url https://teamIp:8443/sdn/v2.0/owners Example output

{ "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"Region-Red", "prioritizedControllerIps":[ "172.17.6.70", "172.17.6.71", "172.17.6.72" ], "deviceIps":[ "192.168.1.101", "192.168.1.102" ], "deviceIpRanges":[ "192.168.1.103-105" ] } ]}

Getting the configuration of a specific region using curlThis GET command retrieves the configuration of the specified region. The regions configuration may have beenmodified since controller startup to reflect the dynamic addition or removal of regions or devices within specificregions.

In this example, there were no changes to the region of interest since controller startup and the configuration isthe same is in Adding a region using curl on page 208.

Example command

curl --noproxy teamIp --header "X-Auth-Token:token" -ksS --request GET --url https://teamIp:8443/sdn/v2.0/owners/region_uid Example output

{ "region":{ "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"Region-Red", "prioritizedControllerIps":[ "172.17.6.70", "172.17.6.71", "172.17.6.72" ], "deviceIps":[ "192.168.1.101", "192.168.1.102" ], "deviceIpRanges":[ "192.168.1.103-105" ] }}


Determining whether a controller owns a specific device using curlThe HEAD command returns a status (essentially a Boolean value) indicating whether the local controller owns thedevice corresponding to the specified IP address; this is decided based on the controller's locally cached data,which is guaranteed to be consistent with the receipt of DeviceOwnerEvents events. You must specify thedeviceIp query parameter. The command returns one of the following statuses:

• 204: The local controller owns the device.• 404: The local controller does not own the device.

Example command

curl --noproxy controllerIp -X HEAD --header "X-Auth-Token:token" -IL --insecure --url https://controllerIp:8443/sdn/v2.0/owners?deviceIp=192.168.1.102 Example output

HTTP/1.1 204 No ContentServer: Apache-Coyote/1.1X-FRAME-OPTIONS: denyAccess-Control-Allow-Origin: *Access-Control-Allow-Methods: GET, POST, PUT, HEAD, PATCHAccess-Control-Allow-Headers: Content-Type, Accept, X-Auth-TokenDate: Wed, 11 Mar 2015 18:34:07 GMT

Getting the owning controller and devices for a region using curlThis GET command retrieves the devices in the region with the specified UID that the specified controller currentlyowns. You can use an optional controllerIp parameter to specify the controller IP address of interest; ifno such address is specified, the local controller IP is used.

In this example, for region UID 713def9a-4f96-485f-990c-8924bc06c8d8 and controller 172.17.6.70(master), the devices are 192.168.1.101, 192.168.1.102, 192.168.1.103, 192.168.1.104, and192.168.1.105.

Regardless of how many IPs are configured for the devices in the region, this GET command indicates only thosethat are actually active and owned by the specified controller.

Example command

curl --noproxy controllerIp --header "X-Auth-Token:token" -ksS --request GET --url https://controllerIp>:8443/sdn/v2.0/owners/region_uid/devices Example output

{ "ownership":{ "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"Region-Red", "owningControllerIp":"172.17.6.70", "devices":[ { "deviceIp":"192.168.1.101" }, { "deviceIp":"192.168.1.102" }, { "deviceIp":"192.168.1.103" },


{ "deviceIp":"192.168.1.104" }, { "deviceIp":"192.168.1.105" } ] }}

Getting the status of a specific region using curlThis GET command retrieves the current status of the region with the specified UID, including its configureddevices and the controller that currently owns each device.

In this example, for region UID 713def9a-4f96-485f-990c-8924bc06c8d8, the command output shows thestatus of all the devices, including their datapaths. In this example all devices are owned by the master controller172.17.6.70.

Example command

curl --noproxy teamIp --header "X-Auth-Token:token" -ksS --request GET --url https://teamIp:8443/sdn/v2.0/owners/region_uid/state?dataPathDetails=yes Example output

{ "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"Region-Red", "prioritizedControllerIps":[ "172.17.6.70", "172.17.6.71", "172.17.6.72" ], "devices":[ { "deviceIp":"192.168.1.101", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:01:44:31:92:5c:af:86", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.102", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:1e:c8:cb:b8:dd:f0:c0", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.103", "owningControllerIp":"172.17.6.70", "dataPaths":[ {


"dpid":"00:1e:f0:92:1c:21:af:00", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.104", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:01:cc:3e:5f:6b:19:00", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.105", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:1e:00:9c:02:e0:e4:00", "owningControllerIp":"172.17.6.70" } ] } ] } ]}

Getting the status of all regions using curlThis GET command retrieves the current status of all regions, including their configured devices and the controllerthat currently owns each device. You can filter the returned content of this command in order to get the currentstatus for a specified device, allowing you to determine which region the device is configured in addition to whichcontroller owns the device.

You can also specify the optional dataPathDetails parameter to enhance the output for devices to includeDPID information consisting of each DPID associated with the device and the controller that currently owns eachDPID. You can specify this parameter independently of the deviceIp parameter, and the resulting output issimilar to that when the optional dataPathDetails parameter is specified for the GET /owners/{region_uid}/state command.

In this example, there is only one region therefore, the command output is the same as the Getting the status ofa specific region using curl on page 212.

Example command

curl --noproxy teamIp --header "X -Auth-Token:token" -ksS --request GET --url https://teamIp:8443/sdn/v2.0/owners/state?dataPathDetails=yes Example output

{ "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"Region-Red", "prioritizedControllerIps":[ "172.17.6.70", "172.17.6.71", "172.17.6.72"


], "devices":[ { "deviceIp":"192.168.1.101", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:01:44:31:92:5c:af:86", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.102", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:1e:c8:cb:b8:dd:f0:c0", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.103", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:1e:f0:92:1c:21:af:00", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.104", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:01:cc:3e:5f:6b:19:00", "owningControllerIp":"172.17.6.70" } ] }, { "deviceIp":"192.168.1.105", "owningControllerIp":"172.17.6.70", "dataPaths":[ { "dpid":"00:1e:00:9c:02:e0:e4:00", "owningControllerIp":"172.17.6.70" } ] } ] } ]}

Removing a device from a region using curlThis DELETE command removes a device with the specified IP address from the region with the specified UIDand propagates the modifications to each controller in the team. A deviceIp query parameter must be specified.You can only remove one IP address at a time in order to know which device IP address is involved when an errorcondition is encountered. If the deleted IP address appears among the deviceIps addresses, it will be removed


from the list of IPs, and if the deleted IP address was encompassed by one of the ranges listed among thedeviceIpRanges, the range will be transformed into one or more ranges and/or individual IP addresses toreflect the removal of the deleted device IP address and the original range will be removed. All controllersconfigured for the team must be available for such a configuration change to be permitted.

In this example, for region UID 713def9a-4f96-485f-990c-8924bc06c8d8, device 192.168.1.103 isremoved.

Example command

curl --noproxy teamIp --header "X-Auth-Token:token" -ksS --request DELETE --url https://teamIp:8443/sdn/v2.0/owners/region_uid/devices?deviceIp=192.168.1.103 Example output

{"result":"Device IP address 192.168.1.103 no longer exists in the region with UID '713def9a-4f96-485f-990c-8924bc06c8d8'"}To check if your device was removed from the region, check the Team screen in the controller UI or see Gettingthe configuration of a specific region using curl on page 210.

Removing a region using curlThis DELETE command removes the region with the specified UID and propagates the modifications to eachcontroller in the team. All controllers configured for the team must be available for such a configuration change tobe permitted.

In this example, region UID 713def9a-4f96-485f-990c-8924bc06c8d8 is deleted. Since only one regionwas configured, there are no regions configured for the team at this point. Therefore, no devices will be allowed toconnect to the team until a new region is defined that encompasses the desired devices.

Example command

curl --noproxy teamIp --header "X-Auth-Token:token" -ksS --request DELETE --url https://teamIp:8443/sdn/v2.0/owners/region_uid Example output

{"result":"Region with UID 713def9a-4f96-485f-990c-8924bc06c8d8 no longer exists"}


Scripts

Restoring a controllerThe Restore.sh script restores a controller from a backup file. This script must have permissions set to 770 andbe owned by the sdn user and sdn group:

#!/bin/bash

readonly OPT_ROOT="/opt/sdn"readonly VAR_LIB_SDN="/var/lib/sdn"readonly backupDir=${OPT_ROOT}"/backup"readonly targetDir=${backupDir}"/tmp/com.hp.sdn.adm.backup.impl.BackupRestoreLegacyManager"readonly configDir=${OPT_ROOT}"/config/"readonly repoDir=${OPT_ROOT}"/virgo/repository/usr"readonly backupFile=${backupDir}"/sdn_controller_backup*.zip"readonly LOG_FILE=${backupDir}"/restore.log"readonly INFO_FILE=${backupDir}"/info.bin"readonly metricsDir=${OPT_ROOT}"/virgo/metrics"WAIT_FOR_STOP=120

function restore_log { typeset script_name=${0##*/}

typeset DATE_FORMAT=${DATE_FORMAT:-"+%b %e %H:%M:%S"} typeset LOG_PREFIX="$(whoami)@$(hostname)"

echo "$(date "$DATE_FORMAT") $LOG_PREFIX $script_name[$$]: $*" >> $LOG_FILE}

# For restore, clean virgo runtime environmentfunction clean_virgo_runtime { export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 $OPT_ROOT/virgo/bin/startup.sh -clean -noStart}

function get_sdnc_pid { echo $(ps -ef | grep -w "/bin/bash" | grep -w "/opt/sdn/admin/sdnc.sh" | head -n 1 | awk '{print $2}')}

is_sdnc_running(){ pid='get_sdnc_pid' [ "x" = "x$pid" ] && return 1 || return 0

Appendix BScripts


}

function restoreCassandraData { # 1. shutdown the node - done by stopping sdnc /opt/sdn/cassandra/bin/caServer.sh status if [ $? -eq 0 ]; then restore_log "Cassandra is still running, attempting stop..." sudo -u sdnadmin /opt/sdn/cassandra/bin/caServer.sh stop check_stop_and_exit $? fi

#pick the cassandra zip file and unpack restore_log "Deleting cassandra configuration" if [ -d "$OPT_ROOT/cassandra/conf" ]; then rm -rf "$OPT_ROOT/cassandra/conf/" check_and_exit $? fi

restore_log "Unzipping the cassandra configuration" unzip -o $2 -d "$OPT_ROOT/cassandra/conf/" # 2. Clear all files in /var/lib/cassandra/commitlog restore_log "Deleting the commitlog directory" if [ -d "$OPT_ROOT/cassandra/commitlog/" ]; then rm -rf "$OPT_ROOT/cassandra/commitlog/" check_and_exit $? fi

# 3. Delete system and other data directories restore_log "Deleting cassandra data directory" if [ -d "/var/lib/sdn/cassandra/" ]; then rm -rf "/var/lib/sdn/cassandra/" check_and_exit $? fi

# 4. pick the cassandra zip file and unpack restore_log "Unzipping the cassandra data directory" unzip -o $1 -d "/var/lib/sdn/cassandra/" chmod -R g=u /var/lib/sdn/cassandra }

function restoreTeamConfig { # Restore the teaming config unzip -o $1 -d "$VAR_LIB_SDN"}

function check_stop_and_exit { OUT=$1 if [[ $OUT -ne 0 && $OUT -ne 1 ]]; then restore_log "Stopping Cassandra failed and Restore failed : $OUT" rm $INFO_FILE exit 1 fi }

function check_and_exit { OUT=$1

Appendix B Scripts 217

if [ $OUT -ne 0 ]; then restore_log "Restore failed:$OUT" rm $INFO_FILE exit 1 fi}

function restorePostGre { restore_log "Restoring postgre database..."

# unzip the zip of postgre unzip -o postgreSQLdata.zip -d $targetDir

# delete the sdndb first and re-create it dropdb sdndb createdb -O sdn sdndb

# remove the extra things that pg_dump back up sed -i '/REVOKE ALL ON SCHEMA public FROM PUBLIC/d' backupPG.sql sed -i '/REVOKE ALL ON SCHEMA public FROM postgres/d' backupPG.sql sed -i '/GRANT ALL ON SCHEMA public TO postgres/d' backupPG.sql sed -i '/GRANT ALL ON SCHEMA public TO PUBLIC/d' backupPG.sql

# this assumes that sdndb database already exists psql sdndb < backupPG.sql check_and_exit $?}

function restoreLicenseLogs { if [ -f licenselog.zip ] then restore_log "Restoring license history logs..."

# unzip the license logs unzip -o licenselog.zip -d /var/log/sdn/virgo/logs check_and_exit $? fi}

function restoreMetricsData { if [ -f metricsData.zip ] then restore_log "Restoring metrics data..."

# wipe out existing contents rm -rf $metricsDir/*

# unzip the metrics data unzip -o metricsData.zip -d "$metricsDir" check_and_exit $? fi}

function command_exists { command -v $1 &> /dev/null;}


function wait_for_sdnc_stop { restore_log "Waiting for SDNC to stop..." pid='get_sdnc_pid' if [[ -z $pid ]]; then return 0; fi

for tries in 'seq $WAIT_FOR_STOP'; do sleep 1 is_sdnc_running || return 0 done

return 1}

# check if unzip is present or notif command_exists unzip; then :else restore_log "Unzip is not installed" restore_log "Please install unzip utility and try again" check_and_exit 1fi

wait_for_sdnc_stopcheck_and_exit $?

# extract the backup archive contentcd $targetDir

for file in 'ls -a *.*' do case $file in config.zip) restore_log "Restoring config files..." unzip -o $file -d $configDir check_and_exit $? ;; teamConfig.zip) restore_log "Restoring teaming Config files..." restoreTeamConfig $file check_and_exit $? ;; userrepo.zip) restore_log "Restoring user repository..." rm -rf $repoDir/* unzip -o $file -d $repoDir check_and_exit $? ;; cassandradata.zip) restore_log "Restoring cassandra data files..." restoreCassandraData $file cassandraconfig.zip check_and_exit $? ;; *) ;; esac done

# restore postgre sql


restorePostGre

# restore licensing compliance history logsrestoreLicenseLogs

# restore metrics datarestoreMetricsData

# clean up virgo runtime environmentclean_virgo_runtime

# create links to /var/log/sdn/virgo[ ! -L /opt/sdn/virgo/serviceability ] && ln -s /var/log/sdn/virgo /opt/sdn/virgo/serviceability

# Change permissions in case the user IDs have changed since the backup.sudo chown -R sdn:sdn /opt/sdn /var/lib/sdn /var/log/sdn/virgo/logs/license-history.log

restore_log "Turning off the restore mode..."

# delete the restore.indicator file[ -f /opt/sdn/backup/restore.indicator ] && rm /opt/sdn/backup/restore.indicator

restore_log "Restore done..."

Backing up a controller teamBecause the scripts in this appendix cross page boundaries, be careful to avoid including the pagenumber when copying a script. Copying a script one page at a time can prevent inclusion of pagenumbers.

#!/bin/bash #------------------------------------------------------------------------------- # Copyright 2013 Hewlett Packard Co., All Rights Reserved. #-------------------------------------------------------------------------------## Backup a Team #-------------------------------------------------------------------------------export BACKUP_DIR="/opt/sdn/backup"export BACKUP_TEAM_DIR="/opt/sdn/team_backup"export TEAM_BACKUP_STATUS_FILE="$BACKUP_TEAM_DIR/teamBackup_status" export TEAM_BACKUP_LOGFILE="$BACKUP_TEAM_DIR/teamBackup_log.log" export BACKUP_WAIT_COUNT=200 # this * 10 = seconds to wait for backup to finishexport B_PID=$$trap "exit 1" TERM#==============================================================================# F U N C T I O N S #==============================================================================#------------------------------------------------------------------------------ # Function validateTeamLead ( ) # Validates configured node IP against the team leader IP.#------------------------------------------------------------------------------function validateTeamLead {leaderIp='ifconfig|grep -o $leaderIp'if [ "$leaderIp" == "" ]; then


teamBackup_log "Run this script from the team lead node."exitBackup 1fiteamBackup_log "Leader node IP $leaderIp is correctly configured." }#------------------------------------------------------------------------------ # Function validateTeamBackupStatus ( )# Checks if a new backup can be started.#------------------------------------------------------------------------------function validateTeamBackupStatus {TEAM_BACKUP_ON="backup_in_progress=true"# Check if any backup is going on now. if [ -e "$TEAM_BACKUP_STATUS_FILE" ]; then teamBackup_log "Backup status file $TEAM_BACKUP_STATUS_FILE exists." backupStatus='cat $TEAM_BACKUP_STATUS_FILE'if [ "$backupStatus" == "$TEAM_BACKUP_ON" ]; thenteamBackup_log "Backup already in progress, aborting new backup..."

exitBackup 1fifirm -rf $BACKUP_TEAM_DIRmkdir $BACKUP_TEAM_DIRchmod 777 $BACKUP_TEAM_DIR echo $TEAM_BACKUP_ON>$TEAM_BACKUP_STATUS_FILEteamBackup_log "No backup is currently in progress. A new backup can start." }#------------------------------------------------------------------------------ # Function backupNode ( <nodeIndex> )# Backs up a node.#------------------------------------------------------------------------------function backupNode { local nodeIndex=$1local backupToken=${nodeAuth[$nodeIndex]}local backupIp=${ipArr[$nodeIndex]}local backupUUID=${nodeUUID[$nodeIndex]} backupURL="https://$backupIp:8443/sdn/v2.0/backuppost $backupIp $backupToken "$backupURL"if [ $errorCode -ne 0 ]; thenteamBackup_log "Failed to start backup for $backupIp."exitBackup 1fiif [ "$sessionID" == "" ]; thenteamBackup_log "Failed to start backup on $backupIp."exitBackup 1 fiecho $sessionID}#------------------------------------------------------------------------------# Function downloadBackupSet ( <nodeIndex> )# Downloads the backup file from each node to the team leader node, verifying the checkum. #------------------------------------------------------------------------------function downloadBackupSet {local nodeIndex=$1 local backupAuth=${nodeAuth[$nodeIndex]}local backupIp=${ipArr[$nodeIndex]}local backupUUID=${nodeUUID[$nodeIndex]} local fileName=""if [ "$backupIp" == "$leaderIp" ]; thenfileName="$BACKUP_TEAM_DIR/sdn_controller_backup_$backupIp.Leader.zip"elsefileName="$BACKUP_TEAM_DIR/sdn_controller_backup_$backupIp.zip" fi


backupUrl="https://$backupIp:8443/sdn/v2.0/backup"'get $backupIp $backupAuth $backupUrl > $fileName'expected='get $nodeIP "v2.0/backup/checksum"'actual=$(sha256sum "$fileName" | cut -d ' ' -f1)if [ "$expected" != "$actual" ]; thenecho "Checksum failure: expected $expected but got $actual."exitBackup 1fiteamBackup_log "Successfully copied backup MD5 file from $backupIp."}

}#------------------------------------------------------------------------------# Function verifyBackupStatus ( <nodeIndex> )# Verifies the success of the backup.#------------------------------------------------------------------------------ function verifyBackupStatus { local nodeIndex=$1 local backupIP=${ipArr[$nodeIndex]}local backupUrl="https://$backupIP:8443/sdn/v2.0/backup/status"backupStatus[$nodeIndex]='get $backupIP ${nodeAuth[$nodeIndex]} $backupUrl'if [ "${backupStatus[$nodeIndex]}" == "SUCCESS" ]; thenteamBackup_log "Backup completed successfully on $backupIP."let "backup_complete = $backup_complete - 1"return fi}

#------------------------------------------------------------------------------# Function teamBackupZip ( )# Creates a single zip for all the team backup data.#------------------------------------------------------------------------------ function teamBackupZip { teamZip='date|tr ' ' '_'|tr ':' '_'' teamZip="$BACKUP_TEAM_DIR/sdn_team_backup_$teamZip.zip"rm -rf $BACKUP_TEAM_DIR/sdn_team_backup* $TEAM_BACKUP_STATUS_FILEzip -r $teamZip $BACKUP_TEAM_DIR/rm -rf $BACKUP_TEAM_DIR/sdn_controller_backup* }#------------------------------------------------------------------------------# Function remoteBackupFileCopy ( )# Copies the team backup zip to the specified remote location.#------------------------------------------------------------------------------function remoteBackupFileCopy {if [ "$remotePath" == "" ]; thenteamBackup_log "Team backup data was not copied to the remote location."returnfiteamBackup_log "Copying team backup to the remote location $remotePath..." scp $BACKUP_TEAM_DIR/sdn_team_backup* $remotePath}#------------------------------------------------------------------------------# Function getSysInfo ( <authToken> )# Gets the SysInformation for the running node.#------------------------------------------------------------------------------function getSysInfo { local leadAuth=$1 local sysUrl="https://localhost:8443/sdn/v2.0/systems" for i in {1..5}; do sysInfo='get localhost $leadAuth "$sysUrl"'if [ $errorCode -ne 0 ]; thenteamBackup_log "Failed to retrieve the system information."exitBackup 1


fi[ "$sysInfo" != "" ] && breaksleep 5doneif [ "$sysInfo" == "" ]; thenteamBackup_log "Failed to retrieve the system information."exitBackup 1fi}#------------------------------------------------------------------------------# Function extractRole_NodeIP ( <systemInfo>) # Extracts IP and role for all the nodes in a team. #------------------------------------------------------------------------------function extractRole_NodeIP {sysinfo=$1 ipArr=($(echo $sysinfo|tr -d '"'| tr -d '['|tr -d ']'| sed -e 's/\,/\n/g'| grep -w "ip"| cut -d ':' -f2-)) roleArr=($(echo $sysinfo|tr -d '"'| tr -d '['|tr -d ']'| sed -e 's/\,/\n/g'| grep -w "role"| cut -d ':' -f2-))numNodes=${#ipArr[@]} teamBackup_log "Number of nodes in the team is $numNodes."for (( i=0; i<=$numNodes; i++ )); doif [ "${roleArr[$i]}" == "leader" ]; thenleaderIp=${ipArr[$i]} teamBackup_log "The team leader is $leaderIp."breakfidone}#------------------------------------------------------------------------------# Function teamBackup_log ( <message> )# Writes messages to the log for the team backup operation.#------------------------------------------------------------------------------function teamBackup_log { msg="$1"echo "$msg" |tee -a $TEAM_BACKUP_LOGFILE}#------------------------------------------------------------------------------# Function exitBackup ( <exitStatus> )# Exits the backup.#------------------------------------------------------------------------------function exitBackup {[ $1 -ne 0 ] && teamBackup_log "Stopping backup/restore with errors."rm -rf $TEAM_BACKUP_STATUS_FILE kill -s TERM $B_PIDexit $1 }#------------------------------------------------------------------------------# Function get ( <ipAddr> <authToken> <url> )# Performs a GET.#------------------------------------------------------------------------------function get {local getIP=$1local getToken=$2local getUrl=$3 local attempts=0 while [ $attempts -lt 5 ]; docurl --noproxy $getIP --header "X-Auth-Token:$getToken" \--fail -ksS -L -f --request GET --url "$getUrl" errorCode=$?let "attempts = $attempts + 1"if [ 35 -eq $errorCode ]; thenteamBackup_log "SSL error on GET of $getUrl, retrying..."continue;


fibreak;done}#------------------------------------------------------------------------------# Function post ( <ipAddr> <authToken> <url> <data>)# Performs a POST of the specified data.#------------------------------------------------------------------------------function post { local postIP=$1local postToken=$2 local postUrl=$3local postData=$4 local attempts=0 while [ $attempts -lt 5 ]; do postRes='curl --noproxy $postIP --header "X-Auth-Token:$postToken" \--fail -ksS --request POST --url "$postUrl" --data-binary "$postData"'errorCode=$? let "attempts = $attempts + 1" if [ 35 -eq $errorCode ]; thenteamBackup_log "SSL error on POST to $postUrl, retrying..."continue;fi

break;doneecho $postRes}#------------------------------------------------------------------------------# Function put ( <ipAddr> <authToken> <url> <data> )# Performs a PUT of the specified data. #------------------------------------------------------------------------------function put { local putIP=$1 local putToken=$2 local putUrl=$3local putData=$4local attempts=0while [ $attempts -lt 5 ]; doputRes='curl --noproxy $putIP --header "X-Auth-Token:$putToken" \ --fail -ksS -L -f --request PUT "$putUrl" --data-binary "$putData"'errorCode=$? let "attempts = $attempts + 1"if [ 35 -eq $errorCode ]; thenteamBackup_log "SSL error on POST to $putUrl, retrying"continue;fibreak;doneecho $putRes}#------------------------------------------------------------------------------# Function extractJSONString ( <json> <fieldName> ) # Extracts the Json value corresponding to the field name. #------------------------------------------------------------------------------function extractJSONString {json=$1 field=$2json='echo $json|tr -d '"'| sed -e 's/\,\|{/\n/g'|grep -w "$field"| \cut -d ':' -f2-' echo $json}#------------------------------------------------------------------------------# Function getAuthToken ( <ipAddr> )


# Log-in and get the UID. #------------------------------------------------------------------------------function getAuthToken { local nodeIP=$1url="https://$nodeIP:8443/sdn/v2.0/auth"login="{\"login\": { \"domain\": \"$domain\",\"user\": \"$user\",\"password\": \"$pass\" }}"# Attempt to authenticate and extract token if successful. auth=$(curl --noproxy $nodeIP -X POST --fail -ksSfL --url "$url" \-H "Content-Type: application/json" --data-binary "$login" 2>&1)if [ $? -ne 0 ]; thenteamBackup_log "Unable to authenticate as user $user in $domain domain."exitBackup 1 fiauthToken='extractJSONString "$auth" "token" | sed '/^$/d''if [ $restore_mode -ne 1 ] && [ "$authToken" == "" ]; then teamBackup_log "Failed to get the authentication token."exitBackup 1fiecho $authToken}#==============================================================================# M A I N#==============================================================================restore_mode=0# Check for zip package.command -v zip &> /dev/null if [ $? -ne 0 ]; thenecho "The zip package must be installed to use this script."exit 1fi# Check the user specified script parameters.if [ $# -lt 2 ]; then echo "Usage : backupTeam <user> <domain> [<user@ip:path>]"echo " <user> - user name to access the controller" echo " <domain> - domain of the controller" echo " [<user@ip:path>] - remote location to store backup file" echo " user - the login name for the system"echo " ip - the ip address of the system" echo " path - where to copy the file to on the remote system" exit 1fivalidateTeamBackupStatususer="$1"echo -n "Enter Controller Password: " read -s pass echodomain="$2"remotePath=$3errorCode=0# Get the authentication token for the local controller. leaderAuth='getAuthToken localhost'# Get the system Information for the local controller. getSysInfo $leaderAuth # Get the set of team IPs and their associated team roles. extractRole_NodeIP $sysInfo (validateTeamLead)# Initiate a backup on each node. for (( i=0; i<$numNodes; i++ )); do


nodeAuth[$i]='getAuthToken ${ipArr[$i]}'uuidURL="https://${ipArr[$i]}:8443/sdn/v2.0/systems"nodeUUID[$i]='get ${ipArr[$i]} ${nodeAuth[$i]} "$uuidURL?ip=${ipArr[$i]}"' nodeUUID[$i]='extractJSONString "${nodeUUID[$i]}" "uid" | sed '/^$/d'' if [ "${ipArr[$i]}" == "$leaderIp" ]; then# Skip the leader backup backup, since it will be done last. leaderIndex=$i continuefibackupNode $iteamBackup_log "Started backup on ${ipArr[$i]}."

done# Verify the status of the backup on each node. backup_complete=$numNodes waitTime=$(($BACKUP_WAIT_COUNT*10/60))for (( k=0; k<$BACKUP_WAIT_COUNT; k++ )); do if [ $backup_complete -le 1 ]; thenteamBackup_log "Backup on all member nodes completed successfully." breakfisleep 10 for (( i=0; i<$numNodes; i++ )); do # Skip the leader node check, since it will be done last. [ "${ipArr[$i]}" == "$leaderIp" ] && continue# Backup already completed for this node, so continue. [ "${backupStatus[$i]}" == "SUCCESS" ] && continueverifyBackupStatus $i donedoneif [ $backup_complete -gt 1 ]; then teamBackup_log "Backup of all member nodes took longer than $waitTime min. Aborting backup..." teamBackup_log "To increase backup wait time, change BACKUP_WAIT_COUNT in the script." exitBackup 1fi# Last, backup the leader node to avoid synchronization issues on a restore. backupNode $leaderIndexteamBackup_log "Started backup on leader ${ipArr[$leaderIndex]}."backup_complete=1

# Verify the backup on the leader node.for (( k=0; k<$BACKUP_WAIT_COUNT; k++ )); dosleep 10verifyBackupStatus $leaderIndexif [ $backup_complete -le 0 ]; then teamBackup_log "Backup on the leader node completed successfully."breakfidoneif [ $backup_complete -gt 0 ]; thenteamBackup_log "Backup of the leader node took longer than $waitTime min. Aborting backup..." teamBackup_log "To increase backup wait time, change BACKUP_WAIT_COUNT in the script."exitBackup 1 fi# Copy all the backup files from each node in the team onto the leader node.for (( i=0; i<$numNodes; i++ )); dodownloadBackupSet $i done# Create one zip for entire team and copy it to the specified remote location. teamBackupZip


remoteBackupFileCopyechoteamBackup_log "The team was backed up successfully."exitBackup 0

Restoring a controller teamBefore running this script, re-install the controller. Otherwise an Error 404 condition results and thecontroller is not restored. See Restoring a controller from a backup.

Because the scripts in this appendix cross page boundaries, be careful to avoid including the pagenumber when copying a script. Copying a script one page at a time can prevent inclusion of pagenumbers.

#!/bin/bash#-------------------------------------------------------------------------------# Copyright 2013 Hewlett Packard Co., All Rights Reserved. #-------------------------------------------------------------------------------## Restore a Team #-------------------------------------------------------------------------------export BACKUP_DIR="/opt/sdn/backup" export BACKUP_TEAM_DIR="/opt/sdn/team_backup" export RESTORE_TEAM_DIR="/opt/sdn/team_restore"export TEAM_BACKUP_STATUS_FILE="$RESTORE_TEAM_DIR/teamRestore_status" export TEAM_BACKUP_LOGFILE="$RESTORE_TEAM_DIR/teamRestore_log.log" export RESTORE_BACKUP_FILESET="$RESTORE_TEAM_DIR/opt/sdn/team_backup" export B_PID=$$ trap "exit 1" TERM #==============================================================================# F U N C T I O N S#==============================================================================#------------------------------------------------------------------------------# Function extract_zip_and_ip ( ) # Extracts the team backup zip and the backed up IP addresses.#------------------------------------------------------------------------------function extract_zip_and_ip { unzip -o "$RESTORE_TEAM_DIR/sdn_team_backup*" -d $RESTORE_TEAM_DIRif [ $? -ne 0 ]; then teamBackup_log "Failed to unzip the team backup file." exitBackup 1fiteamBackup_log "Extracted the team backup file successfully."rm -rf "$RESTORE_TEAM_DIR/sdn_team_backup*" backupIp=($(ls $RESTORE_BACKUP_FILESET | grep "zip$" | sed "s/.zip//" | \ sed "s/.Leader//" | sed "s/sdn_controller_backup_//")) numBackup=${#backupIp[@]} teamBackup_log "Found $numBackup backup file sets in the team backup file." }#------------------------------------------------------------------------------# Function create_restoreDir ( ) # Creates the team restore directory.#------------------------------------------------------------------------------function create_restoreDir { rm -rf $RESTORE_TEAM_DIR mkdir $RESTORE_TEAM_DIRchmod 777 $RESTORE_TEAM_DIR }#------------------------------------------------------------------------------# Function validate_my_Ip ( ) # Validates the configured node IP against the backed up IP addresses.


#------------------------------------------------------------------------------function validate_my_Ip { for (( v=0; v<numBackup; v++ )); do myip='ifconfig|grep -o "${backupIp[$v]}"'if [ "$myip" != "" ]; then teamBackup_log "IP $myip is a valid member of the team." returnfidoneteamBackup_log "IP $myip is not a valid member of the team, exiting." exitBackup 1 }#------------------------------------------------------------------------------# Function upload_backup_file ( <systemIp> <systemUUID> <authToken> <zipFile> ) # Uploads backup file to the specific nodes of the team.#------------------------------------------------------------------------------function upload_backup_file { local sysIp=$1 local sysUUID=$2local sysAuth=$3local uploadUrl="https://$sysIp:8443/sdn/v2.0/restore/backup" local zipFile=$4if [ ! -f $zipFile ]; thenteamBackup_log "File $zipFile does not exist."exitBackup 1 ficurl --noproxy $sysIp -X POST --fail -ksSfL --url $uploadUrl \ -H "X-Auth-Token:$sysAuth"\--data-binary @$zipFileif [ $? -ne 0 ]; thenteamBackup_log "Failed to upload backup $zipFile to $sysIp."exitBackup 1 fiteamBackup_log "Backup $zipFile uploaded successfully to $sysIp."}#------------------------------------------------------------------------------# Function restore_node ( <systemIp> <systemUUID> <authToken> ) # Restores a particular node. #------------------------------------------------------------------------------function restore_node { local sysIp=$1local sysUUID=$2 local sysAuth=$3local restoreUrl="https://$sysIp:8443/sdn/v2.0/restore"# Set the IP first. Ignore errors since this only works for standalone.put $sysIp $sysAuth "https://$sysIp:8443/sdn/v2.0/systems/$sysUUID" \"{\"system\":{\"ip\":\"$sysIp\"}}" > /dev/null 2>&1 restoreSession='post $sysIp $sysAuth $restoreUrl ' if [ $errorCode -ne 0 ]; thenteamBackup_log "Failed to start restore on node $sysIp."exitBackup 1fiteamBackup_log "Started restore on node $sysIp." }#------------------------------------------------------------------------------# Function validate_node_status ( )# Validates node status after the restore. #------------------------------------------------------------------------------function validate_node_status { local sysIp=$1 # Wait for the restore to complete. local sysUrl="https://$sysIp:8443/sdn/v2.0/systems"for (( k=0; k<100; k++ )); do sleep 30


authToken='getAuthToken $sysIp'[ "$authToken" == "" ] && continue# Try to contact the system.data='get $sysIp $authToken "$sysUr?ip=$sysIp"' [ "$data" == "" ] && continueteamBackup_log "Node:$sysIp came up successfully." && return doneteamBackup_log "Node:$sysIP failed to come up." exitBackup 1 }#------------------------------------------------------------------------------ # Function restore_nodes ( <ipAddrArray> ) # Restores only the specified node(s). #------------------------------------------------------------------------------function restore_nodes {local leaderindex=-1 local restoreIpArr=("$@")local numNodes=${#restoreIpArr[@]} for (( i=0; i<$numNodes; i++ )); do # Get the auth token for a specific node. restoreAuth[$i]='getAuthToken ${restoreIpArr[$i]}' if [ "${restoreAuth[$i]}" == "" ]; then teamBackup_log "Failed to get the auth Token for ${restoreIpArr[$i]}, can't start restore."exitBackup 1 fiuuidURL="https://${restoreIpArr[$i]}:8443/sdn/v2.0/systems"restoreUUID[$i]='get ${restoreIpArr[$i]} ${restoreAuth[$i]} "$uuidURL"' if [ "${restoreUUID[$i]}" == "" ]; then teamBackup_log "Failed to get the UUID for ${restoreIpArr[$i]}, can't start restore."exitBackup 1 firestoreUUID[$i]='extractJSONString "${restoreUUID[$i]}" "uid" | sed '/^$/d'' teamBackup_log "UUID for ${restoreIpArr[$i]} is ${restoreUUID[$i]}" # Upload the backup files to a specific node. local ipFileName="sdn_controller_backup_${restoreIpArr[$i]}*.zip" local zipFile='ls $RESTORE_BACKUP_FILESET/$ipFileName'upload_backup_file ${restoreIpArr[$i]} ${restoreUUID[$i]} \ ${restoreAuth[$i]} $zipFile# Check if this is the leader node from the backup set. local leaderZip='echo $zipFile|grep "Leader"' [ "$leaderZip" != "" ] && leaderIndex=$Idone # Start restore in the leader node first before all the other nodes. if [ $leaderIndex -ne -1 ]; then restore_node ${restoreIpArr[$leaderIndex]} ${restoreUUID[$leaderIndex]} \${restoreAuth[$leaderIndex]} fi# Verify the leader node is up after the restore. validate_node_status ${restoreIpArr[$leaderIndex]} # Continue restore on the remaining nodes. for (( i=0; i<$numNodes; i++ )); do # Skip the leader node; it's already done.[ $i -eq $leaderIndex ] && continue # Restore the specified node.restore_node ${restoreIpArr[$i]} ${restoreUUID[$i]} ${restoreAuth[$i]} donesleep 200 # Validate that the restored nodes are up. for (( n=0; n<$numNodes; n++ )); do # Skip the leader node; it's already done. [ $n -eq $leaderIndex ] && continuevalidate_node_status ${restoreIpArr[$n]}


done}#------------------------------------------------------------------------------ # Function teamBackup_log ( <message> )# Writes messages to the log for the team backup operation. #------------------------------------------------------------------------------function teamBackup_log {msg="$1"echo "$msg" |tee -a $TEAM_BACKUP_LOGFILE}#------------------------------------------------------------------------------# Function exitBackup ( <exitStatus> )# Exits the backup.#------------------------------------------------------------------------------function exitBackup {[ $1 -ne 0 ] && teamBackup_log "Stopping backup/restore with errors."rm -rf $TEAM_BACKUP_STATUS_FILEkill -s TERM $B_PID exit $1}#------------------------------------------------------------------------------# Function get ( <ipAddr> <authToken> <url> ) # Performs a GET.#------------------------------------------------------------------------------function get {local getIP=$1local getToken=$2local getUrl=$3 local attempts=0 while [ $attempts -lt 5 ]; docurl --noproxy $getIP --header "X-Auth-Token:$getToken" \ --fail -ksS -L -f --request GET --url "$getUrl"errorCode=$?let "attempts = $attempts + 1" if [ 35 -eq $errorCode ]; thenteamBackup_log "SSL error on GET of $getUrl, retrying..."continue; fibreak; done}#------------------------------------------------------------------------------# Function post ( <ipAddr> <authToken> <url> <data>)# Performs a POST of the specified data. #------------------------------------------------------------------------------function post { local postIP=$1 local postToken=$2local postUrl=$3 local postData=$4local attempts=0while [ $attempts -lt 5 ]; dopostRes='curl --noproxy $postIP --header "X-Auth-Token:$postToken" \--fail -ksS --request POST --url "$postUrl" --data-binary "$postData"'errorCode=$? let "attempts = $attempts + 1"if [ 35 -eq $errorCode ]; thenteamBackup_log "SSL error on POST to $postUrl, retrying..." continue;fibreak;doneecho $postRes


}#------------------------------------------------------------------------------# Function put ( <ipAddr> <authToken> <url> <data> )# Performs a PUT of the specified data.#------------------------------------------------------------------------------function put {local putIP=$1local putToken=$2local putUrl=$3local putData=$4local attempts=0 while [ $attempts -lt 5 ]; do putRes='curl --noproxy $putIP --header "X-Auth-Token:$putToken" \--fail -ksS -L -f --request PUT "$putUrl" --data-binary "$putData"'errorCode=$?let "attempts = $attempts + 1"if [ 35 -eq $errorCode ]; thenteamBackup_log "SSL error on POST to $putUrl, retrying" continue;fibreak;doneecho $putRes}#------------------------------------------------------------------------------# Function extractJSONString ( <json> <fieldName> ) # Extracts the Json value corresponding to the field name. #------------------------------------------------------------------------------function extractJSONString {json=$1field=$2json='echo $json|tr -d '"'| sed -e 's/\,\|{/\n/g'|grep -w "$field"| \cut -d ':' -f2-' echo $json }#------------------------------------------------------------------------------# Function getAuthToken ( <ipAddr> ) # Log-in and get the UID.#------------------------------------------------------------------------------function getAuthToken {local nodeIP=$1 url="https://$nodeIP:8443/sdn/v2.0/auth"login="{\"login\": {\"domain\": \"$domain\",\"user\": \"$user\", \"password\": \"$pass\"}}}" # Attempt to authenticate and extract token if successful. auth=$(curl --noproxy $nodeIP -X POST --fail -ksSfL --url "$url" \-H "Content-Type: application/json" --data-binary "$login" 2>&1)if [ $? -ne 0 ]; thenteamBackup_log "Unable to authenticate as user $user in $domain domain."exitBackup 1fiauthToken='extractJSONString "$auth" "token" | sed '/^$/d'' if [ $restore_mode -ne 1 ] && [ "$authToken" == "" ]; thenteamBackup_log "Failed to get the authentication token." exitBackup 1fiecho $authToken}


#==============================================================================# M A I N#==============================================================================restore_mode=1 selective_restore=0# Check for unzip package.command -v unzip &> /dev/null if [ $? -ne 0 ]; then echo "The unzip package must be installed to use this script."exit 1 fi# Check the user specified script parameters. if [ $# -lt 3 ]; thenecho "Usage : restoreTeam <user> <domain> [<ip1> <ip2> ...] <user@IP:path>"echo " <user> - user name to access the controller"echo " <domain> - domain of the controller"echo " [<ip1> <ip2> ...] - ip(s) of node(s) to be restored; if none are specified all nodes are restored"echo " <user@IP:path> - remote location to retrieve backup file" echo " user - the login name for the system" echo " ip - the ip address of the system" echo " path - where to copy the file from on the remote system"exit 1ficreate_restoreDiruser="$1"echo -n "Enter Controller Password: "read -s passechodomain="$2"file="" if [ $# -eq 3 ]; thenteamBackup_log "Starting the team restore. This will restore all the nodes in a team."file=$3elseteamBackup_log "Starting selective restore on specified IPs. This restore will happen only on the specified nodes." count=0selective_restore=1for ip in "$@"; dorestoreIp[$count]=$iplet "count = $count + 1"donefileIndex=$(($# - 1))file=${restoreIp[$fileIndex]} && unset restoreIp[$fileIndex] fi# Upload the team backup file from the user specified location.scp $file $RESTORE_TEAM_DIRif [ $? -ne 0 ]; thenteamBackup_log "Failed to upload team backup file to the node."exitBackup 1fi# Unzip the team backup file.extract_zip_and_ip# Validate the IP address of the node.validate_my_Ip# Restore the node(s).if [ $selective_restore -eq 1 ]; thenrestore_nodes ${restoreIp[@]}elserestore_nodes ${backupIp[@]}fiecho


teamBackup_log "The team was restored successfully."exitBackup 0


Using an external policy managerBy integrating the controller with an external policy manager such as Aruba ClearPass Policy Manager, you canget information about a client device based on its activity in the network. Aruba ClearPass Policy manager canpush information about a client device to any other server using its REST API. A client device could be a laptop,desktop, any wireless device, or any server that is connected to the network. Events are generated based on auser login, logout or rejected event.

The Client Mapper Service is an internal controller application that combines information known about a networkclient by the controller, such as host IP address, host MAC addresses, and the connected datapath and port, withinformation about the network client known by an outside policy manager, such as the Aruba ClearPass PolicyManager, to provide information about network clients, including user information, device information, and locationinformation. External SDN applications can use the information about a client and perform appropriate actions.Currently the information is available on the controller via the REST API only. For REST API details, see theAruba VAN SDN Controller REST API Reference.

To integrate the controller with Aruba ClearPass Policy Manager, you must configure ClearPass Policy Managerto use the controller as the external server. You must also configure ClearPass Policy Manager to use thecontroller’s Client Mapper Service POST REST API to post user events. For every event posted to the controllerby the Aruba ClearPass Policy Manager, the Client Mapper Service posts a corresponding ClientEvent to thecontroller. For details on configuring ClearPass Policy Manager to meet these requirements, see the ArubaClearPass Policy Manager documentation.

Authentication of Client Mapper Service related REST API requests can be either token-based authentication orclient certificate-based authentication. For details, see REST authentication on page 117.

Appendix CUsing an external policy manager


Performance testingMeasuring flows (packets) per second

For measuring flows-per-second for performance testing, disable the additional processing required by learn.ipkey of the com.hp.sdn.disco.of.node.OfIpDiscoveryComponent component by setting the value of thekey to false.

Procedure

1. From the navigation menu, select Configurations.2. In the Basic tab, select the com.hp.sdn.disco.of.node.OfIpDiscoveryComponent component.3. Click Modify.

Figure 69: Display the learn.ip option

4. For the learn.ip key, enter false in the Value box.5. Click Apply to set the new learn.ip configuration and close the window.

When flow measurement tasks are complete, set the learn.ip key to true (its default value). Flowmeasurement results can vary based on the type of server used for the controller and on the serverconfiguration.

Appendix DPerformance testing

Appendix D Performance testing 235

Examples of MetricsThe SDN controller has a subsystem for tracking metric values over time. Metric values are held as a time serieswhich becomes available to the user via JMX or may be persisted to disk. For metric values that are persisted todisk the time-series values for each individual metric may be persisted at intervals of 1, 5, or 15 minutes; thecomponent or application creating the metric may choose to use the default persistence. Each persisted valuerepresents the value of the metric over the elapsed interval represented by the metric. Persisted metric values areretained on the controller disk for the period of time configured via the metric manager configuration. Persistedmetric values may be extracted from the controller using a series of REST calls.

Persisted value

Consider the value of a counter that incremented from 100 to 145 during a 5 minute interval. By taking the valueat the end of the interval (145), less the value at the start of the interval (100) the value persisted for the intervalwould be 45.

The amount of time during the interval over which the metric value was accrued is also persisted.

If the counter value was accrued over only the last 3 minutes of the 5 minute interval, then the normalized rate ofaccrual over the interval would be 15 counts per minute or 75 counts for the entire 5 minute interval.

The user can inspect the most recent value of the exposed metric using any JMX client (e.g. jconsole). Thevalues exposed will show the absolute value of each such metric. Metric values that are retained only in memoryand not exposed to the user via persistence and/or JMX may be used internally by a component or application.

The metric REST API consists of several distinct commands that may be used in combination to determine whichmetrics have persisted time-series values resident on the controller, what each metric represents, and to retrievetime series values for specific metrics of interest. Note this API operates only on a single controller at a time; it is

Appendix EExamples of Metrics


not team-aware and does not return values that span a controller team. The metrics applications commands willdisplay the application IDs – including the controller itself – that have persisted metrics to disk.

Figure 70: Metrics options

Displaying the application IDs

The metrics/apps command will display the application IDs for applications – including the controller itself – thathave persisted metrics to disk. In this example, only the base controller itself has persisted metrics to disk.

Curl equivalent command

curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/metrics/apps"Curl output

{"apps":[{"app_id":"com.hp.sdn","app_name":"HP VAN SDN Controller"}]}

Appendix E Examples of Metrics 237

Result

The application ID for the controller is com.hp.sdn. It can be plugged into other metric REST API calls thatrequire an{app_id} value in their URL.

All metrics available

The following command lists all of the metrics available for a specific application (the controller itself in thisexample); because of the number of metrics available the output is cut off.

This output describes the metrics; it does not represent the time-series values for them.


curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/metrics/apps/<app_id>"Curl output for app_id=com.hp.sdn


{"metrics":[{"app_id":"com.hp.sdn","type":"RATIO_GAUGE","name":"cpuLoadSystem","description":"The recent CPU usage of the system.","primary_tag":"jvm","secondary_tag":"operatingSystem","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"42f65cd8-03c3-4cad-9788-012d513e3c0f"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"committedBytes","description":"The amount of non-heap memory in bytes that is committed (guaranteed) for the JVM to use.","primary_tag":"jvm","secondary_tag":"memoryNonHeap","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"b82f5b00-0373-4a23-b5a8-bbda7eec44cb"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countTerminated","description":"Number of JVM threads that had exited.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"0e9fe62e-01fd-42e9-88a6-f92021a5e786"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"uptimeMs","description":"The uptime of the JVM in milliseconds.","primary_tag":"jvm","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"93920384-c0e4-4b4c-987e-79a690a541af"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"committedBytes","description":"The amount of heap memory in bytes that is committed (guaranteed) for the JVM to use.","primary_tag":"jvm","secondary_tag":"memoryHeap","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"4cb1a4be-3a8d-4f69-a626-2c5ae134c7e3"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"usedBytes","description":"The total amount of memory currently being used by the JVM in bytes.","primary_tag":"jvm","secondary_tag":"memoryTotal","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"fe56c3fd-9994-4ec4-af59-121357359d58"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"usedBytes","description":"The amount of heap memory currently being used by the JVM in bytes.","primary_tag":"jvm","secondary_tag":"memoryHeap","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"431b746e-e62e-4874-a801-b1438eaac635"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"usedBytes","description":"The amount of non-heap memory currently being used by the JVM in bytes.","primary_tag":"jvm","secondary_tag":"memoryNonHeap","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"afa9a4b2-856a-4f69-8abf-a4775fd0f2e7"},{"app_id":"com.hp.sdn","type":"ROLLING_COUNTER","name":"count","description":"The number of garbage collections undertaken by the JVM during the sampling interval.","primary_tag":"jvm","secondary_tag":"garbageCollection","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"d62c49d4-46b3-4c2c-be60-24f6fa6c6bf6"},{"app_id":"com.hp.sdn","type":"RATIO_GAUGE","name":"usage","description":"The ratio of non-heap memory currently being used by the JVM to maximum non-heap memory requested by the JVM.","primary_tag":"jvm","secondary_tag":"memoryNonHeap","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"fafefbb9-e953-492e-9ddf-db76961f8958"},{"app_id":"com.hp.sdn","type":"RATIO_GAUGE","name":"averageBufferUsedBytes","description":"The average bytes used in each mapped memory buffer associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioMappedMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"25a38f14-1ea3-4bc8-9588-fe4c53a587dd"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"bufferUsedBytes","description":"The total amount of mapped buffer memory that the JVM is using in bytes.",


"primary_tag":"jvm","secondary_tag":"nioMappedMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"c715e388-87dc-4f05-a430-c3c39e73615e"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"bufferUsedBytes","description":"The total amount of direct buffer memory that the JVM is using in bytes.","primary_tag":"jvm","secondary_tag":"nioDirectMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"f7301f57-e013-477c-af8f-cf3dae376232"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"bufferCapacityBytes","description":"The total capacity in bytes of direct buffer memory associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioDirectMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"fbcebed0-21a2-4b8a-8e68-0351cddefee8"},{"app_id":"com.hp.sdn","type":"RATIO_GAUGE","name":"averageBufferCapacityBytes","description":"The average capacity in bytes of each mapped memory buffer associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioMappedMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"42aee5ea-934d-4816-a1fd-f2f8f99e5160"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countRunnable","description":"Number of JVM threads that were executing.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"b3acc121-e892-4759-90c5-63969b0fcfb7"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countTimedWaiting","description":"Number of JVM threads waiting up to a specified period for another thread to perform a particular action.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"9c6928b5-9ca3-4e9b-956f-177625ea2b8a"},{"app_id":"com.hp.sdn","type":"RATIO_GAUGE","name":"fileDescriptorsUsage","description":"The ratio of file descriptors open on the operating system to the maximum supported file descriptors.","primary_tag":"jvm","secondary_tag":"operatingSystem","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"466e03fa-a9b7-4ba5-b7cd-d697dba9b1be"},{"app_id":"com.hp.sdn","type":"RATIO_GAUGE","name":"averageBufferCapacityBytes","description":"The average capacity in bytes of each direct memory buffer associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioDirectMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"393bc882-0fc4-4eea-9aaa-97acd716a0c5"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countBlocked","description":"Number of JVM threads that were blocked.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"f0be8e6d-a456-4b8c-83cb-d156629b9aec"},{"app_id":"com.hp.sdn","type":"ROLLING_COUNTER","name":"elapsedMs","description":"The number of milliseconds for which garbage collection was undertaken by the JVM during the sampling interval.","primary_tag":"jvm","secondary_tag":"garbageCollection","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"b550925d-c8d9-4396-a3ab-ec37d498fd8f"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countDeadlocked","description":"Number of JVM threads that were in deadlock.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"505b577c-8606-4172-92d5-90b26ac2be85"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countWaiting","description":"Number of JVM threads waiting indefinitely for another thread to perform a particular action.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"cb00d09b-1d9c-4f96-a5c9-8a6cd06ad5bd"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"bufferCount","description":"The number of direct memory buffers associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioDirectMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"cdcbb352-d001-4be0-a7fd-


c6d614c75f1a"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countNew","description":"Number of JVM threads that had not yet started.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"0e67f839-48b6-4270-9d83-ac7828c742f6"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countDaemon","description":"Number of JVM threads that were live daemon threads.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"7328dd03-57fd-4baa-b741-25dab77446fc"},{"app_id":"com.hp.sdn","type":"RATIO_GAUGE","name":"cpuLoadJvm","description":"The recent CPU usage of the JVM process.","primary_tag":"jvm","secondary_tag":"operatingSystem","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"cc82ee87-80eb-417b-8d41-9010fbf24b63"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"bufferCapacityBytes","description":"The total capacity in bytes of mapped buffer memory associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioMappedMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"6c08d248-e6dd-4d96-b3de-9ee6c32825d3"},{"app_id":"com.hp.sdn","type":"RATIO_GAUGE","name":"usage","description":"The ratio of heap memory currently being used by the JVM to maximum heap memory requested by the JVM.","primary_tag":"jvm","secondary_tag":"memoryHeap","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"bc729aec-4bc4-453e-8e9b-6c2fff5eeaef"},{"app_id":"com.hp.sdn","type":"RATIO_GAUGE","name":"averageBufferUsedBytes","description":"The average bytes used in each direct memory buffer associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioDirectMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"0cac91dd-4f53-4002-8333-d2a104362bd3"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"fileDescriptorsOpen","description":"The number of file descriptors open on the operating system.","primary_tag":"jvm","secondary_tag":"operatingSystem","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"60bcbb25-7689-484b-bbc9-b403a1ce9b56"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countTotal","description":"Total (daemon and non-daemon) number of live JVM threads.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"fc9d9166-f525-4b7a-93a7-903278075f31"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"bufferCount","description":"The number of mapped memory buffers associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioMappedMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"dbe9e2fc-f5a5-42d7-a4e9-45bb6c5d0d8d"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"countNonDaemon","description":"Number of JVM threads that were live non-daemonthreads.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"e13bce02-45f0-47e3-b38e-284d30bc84af"},{"app_id":"com.hp.sdn","type":"GAUGE","name":"committedBytes","description":"The total amount of memory in bytes that is committed (guaranteed) for the JVM to use.","primary_tag":"jvm","secondary_tag":"memoryTotal","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"8c6e1603-46a8-4cbb-a2e5-225372d2e36a"}]}


Result For each metric listed, one can see the its type, its associated application ID, its name, its primary andsecondary tags, whether it is persisted, whether it is exposed via JMX, and its summary interval. Also displayedfor each metric is the unique ID (uid) assigned to the metric on the controller. Other metric REST API calls can beused to view specific subsets of this data.


Lists primary tags associated with a specific application


curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/metrics/apps/<app_id>/primaries"Curl output for app_id=com.hp.sdn

{"primaries":["jvm"]}

Result

The only primary tag associated with the controller is jvm.

Secondary tags associated with a specific application

The secondary tags associated with a specific application are listed using the following REST call.


curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -


ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/metrics/apps/<app_id>/secondaries"Curl output for app_id=com.hp.sdn

{"secondaries":["nioDirectMemory","operatingSystem","threads","garbageCollection","memoryNonHeap","memoryHeap","memoryTotal","nioMappedMemory"]}

Result

Several secondary tags are associated with the primary tag jvm along with several subcategories of jvm metric:memoryHeap metrics and threads metrics, among others.

Metric names associated with a specific application

Metric names associated with a specific application are displayed using the following call.



curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/metrics/apps/<app_id>/names"Curl output for app_id=com.hp.sdn

{"names":["averageBufferUsedBytes","countDeadlocked","bufferCapacityBytes","count","countNew","bufferCount","countWaiting","fileDescriptorsOpen","uptimeMs","cou ntTerminated","elapsedMs","countTimedWaiting","countDaemon","countBlocked","fileDescriptorsUsage","averageBuff erCapacityBytes","cpuLoadSystem","countTotal","bufferUsedBytes","usedBytes","usage","countNonDaemon","countRu nnable","cpuLoadJvm","committedBytes"]}


Result

Optional query parameters are provided in each of the calls. To see the metric names for specific primary tagsand optionally secondary tags, you may be specific in the call so that only the applicable metric names aredisplayed.

Primary and secondary tags


curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -


ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/ metrics/apps/<app_id>/names?primary_tag=<primary_tag>, secondary_tag=<secondary_tag>"Curl output for app_id=com.hp.sdn

primary_tag=jvm&secondary_tag=memoryHeap): {"names":["usedBytes","usage","committedBytes"]}

Result

Metric names are specific to JVM heap memory. The UID can be obtained once the specific metric of interest isidentified via the earlier call. Optional query parameters to filter the output and list the metrics associated with anapplication ID may be employed.

Filter primary, secondary and metric name

Filter for a primary tag of “jvm”, a secondary tag of “memoryHeap”, and a metric name of “usedBytes”.



curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/metrics/apps/ <app_id>?primary_tag=jvm&secondary_tag=memoryHeap&name=usedBytes"Curl output for app_id=com.hp.sdn, primary_tag=jvm, secondary_tag=memoryHeap, name=usedBytes

{"metrics":[{"app_id":"com.hp.sdn","type":"GAUGE","name":"usedBytes","description":"The amount of heap memory currently being used by the JVM in bytes.","primary_tag":"jvm","secondary_tag":"memoryHeap","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"431b746e-e62e-4874-a801-b1438eaac635"}]}


Result

Detailed information about the metric can be retrieved using a specific metric UID. This same information isencompassed by the previous metric listing.

Time-series data for a metric

The following example shows time-series data for a metric. The time for which the actual metric value wasaccrued is shown in each 18 interval in number of milliseconds spanned. The “ast” value is the metric value forthe indicated time, spanning the indicated number of milliseconds ending at that time.



curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/metrics/<metric_uid>/ values?start=<start_time>&interval=<interval_value>"Curl output

for app_id=com.hp.sdn, metric_uid=431b746e-e62e–4874–a801–b1438eaac635,start=2014–09–23+18:00,interval=1

{"metric_values":{"uid":"431b746e-e62e-4874-a801-b1438eaac635","type":"GAUGE","datapoint_count":25,"datapoints":[{"update_time":"Tue Sep 23 17:59:55 PDT 2014","milliseconds_span":60000,"last":3.22526704E8},{"update_time":"Tue Sep 23 18:00:55 PDT 2014","milliseconds_span":60000,"last":3.24059976E8},{"update_time":"Tue Sep 23 18:01:55 PDT 2014","milliseconds_span":60001,"last":3.28183496E8},{"update_time":"Tue Sep 23 18:02:55 PDT 2014","milliseconds_span":60000,"last":3.28906008E8},{"update_time":"Tue Sep 23 18:03:55 PDT 2014","milliseconds_span":59999,"last":3.333726E8},{"update_time":"Tue Sep 23 18:04:55 PDT 2014","milliseconds_span":60001,"last":3.3423956E8},{"update_time":"Tue Sep 23 18:05:55 PDT 2014","milliseconds_span":59999,"last":3.39215208E8},{"update_time":"Tue Sep 23 18:06:55 PDT 2014","milliseconds_span":60001,"last":3.40427184E8},{"update_time":"Tue Sep 23 18:07:55 PDT 2014","milliseconds_span":60000,"last":3.44840904E8},{"update_time":"Tue Sep 23 18:08:55 PDT 2014","milliseconds_span":59999,"last":2.0824068E8},{"update_time":"Tue Sep 23 18:09:55 PDT 2014","milliseconds_span":60000,"last":2.09534728E8},{"update_time":"Tue Sep 23 18:10:55 PDT 2014","milliseconds_span":60000,"last":2.13733336E8},{"update_time":"Tue Sep 23 18:11:55 PDT 2014","milliseconds_span":60000,"last":2.15092288E8},{"update_time":"Tue Sep 23 18:12:55 PDT 2014","milliseconds_span":60000,"last":2.18786248E8},{"update_time":"Tue Sep 23 18:13:55 PDT 2014","milliseconds_span":60000,"last":2.20134712E8},{"update_time":"Tue Sep 23 18:14:55 PDT 2014","milliseconds_span":60000,"last":3.73961104E8},{"update_time":"Tue Sep 23 18:15:55 PDT 2014","milliseconds_span":60000,"last":4.2576752E8},{"update_time":"Tue Sep 23 18:16:55 PDT 2014","milliseconds_span":60000,"last":4.28963472E8},{"update_time":"Tue Sep 23 18:17:55 PDT 2014","milliseconds_span":60000,"last":4.32102736E8},{"update_time":"Tue Sep 23 18:18:55 PDT 2014","milliseconds_span":60000,"last":4.35269584E8},{"update_time":"Tue Sep 23 18:19:55 PDT 2014","milliseconds_span":60000,"last":4.40140248E8},{"update_time":"Tue Sep 23 18:20:55 PDT 2014","milliseconds_span":60001,"last":4.47377528E8},{"update_time":"Tue Sep 23 18:21:55 PDT 2014","milliseconds_span":59999,"last":4.50576336E8},{"update_time":"Tue Sep 23 18:22:55 PDT 2014","milliseconds_span":60001,"last":


2.19738976E8},{"update_time":"Tue Sep 23 18:23:55 PDT 2014","milliseconds_span":59999,"last":3.7457348E8}]}}

Result

The computation of values returned over longer intervals depends upon the type of metric. Gauge values asshown in this example are averaged over the data points encompassed in the summary. Counter values aresummed over the summary interval in which histogram values are combined.


Returned data for a period results

Summarizing the returned data for a period results in (generally) larger values for the metrics themselves andlarger values for the milliseconds spanned.


curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/metrics/<metric_uid>/ values?start=<start_time>&interval=<interval_value>"Curl output

for app_id=com.hp.sdn, metric_uid=431b746e-e62e–4874–a801–b1438eaac635,start=2014–09–23+18:00,interval=5

{"metric_values":{"uid":"431b746e-e62e-4874-a801-b1438eaac635","type":"GAUGE","datapoint_count":6,"datapoints":[{"update_time":"Tue Sep 23 18:03:55 PDT 2014","milliseconds_span":300000,"last":3.274097568E8},{"update_time":"Tue Sep 23 18:08:55 PDT 2014","milliseconds_span":300000,"last":3.133927072E8},{"update_time":"Tue Sep 23 18:13:55 PDT 2014","milliseconds_span":300000,"last":2.154562624E8},{"update_time":"Tue Sep 23 18:18:55 PDT 2014","milliseconds_span":300000,"last":4.192128832E8},{"update_time":"Tue Sep 23 18:23:55 PDT 2014","milliseconds_span":300000,"last":3.864813136E8},{"update_time":"Tue Sep 23 18:27:55 PDT 2014","milliseconds_span":240000,"last":3.847236E8}]}}


Result

Support report

This report may be generated via the following REST API.



curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail -ksSfL --url "https://<controller_ip>:8443/sdn/v2.0/support"Curl output controller_ip:8443/sdn/v2.0/support

{"support_report":[{"title":"Alert Framework","id":"alert","content":["Alert-Topics: licensing","Alert-Count: 7","Data Retention Age Out: 14 days","Data Trim Interval: 24 hours","Data Trim Enabled: true","Last trim conducted at: Mon Sep 22 19:15:20 PDT 2014"]},{"title":"Alert Topic Listener","id":"alert_listener","content":["No registered alert topic listeners"]},{"title":"App Manager","id":"app-management","content":["Installed Applications: 8","Path Diagnostics, Version: 2.5.0.SNAPSHOT, State: ACTIVE","Link Manager, Version: 2.5.0.SNAPSHOT, State: ACTIVE","Node Manager, Version: 2.5.0.SNAPSHOT, State: ACTIVE","OpenFlow Link Discovery, Version: 2.5.0.SNAPSHOT, State: ACTIVE","OpenFlow Node Discovery, Version: 2.5.0.SNAPSHOT, State: ACTIVE","Path Daemon, Version: 2.5.0.SNAPSHOT, State: ACTIVE","Topology Manager, Version: 2.5.0.SNAPSHOT, State: ACTIVE","Topology Viewer, Version: 2.5.0.SNAPSHOT, State: ACTIVE"]},{"title":"Audit Log Framework","id":"audit_log","content":["Audit Log Count: 0","Data Retention Age Out: 365 days","Data Trim Interval: 24 hours","Data Trim Enabled: true","Last trim conducted at: Mon Sep 22 19:15:20 PDT 2014"]},{"title":"Server Environment","id":"env","content":["OS architecture: amd64","OS Name: Linux","OS Version: 3.5.0-52-generic","Java Vendor: Oracle Corporation","Java Version: 24.65-b04","Java Name: OpenJDK 64-Bit Server VM","Available processors (cores): 4","Max Heap: 3817865216 [3641Mb]","Heap: 671088640 [640Mb]","Heap used: 405144704 [386Mb]","Start Date: Tue Sep 16 19:14:57 PDT 2014","UpTime: 6 Days, 23 Hours","HP VAN SDN Controller Version: 2.5.0.0482"]},{"title":"JVM Metrics","id":"jvm-metrics","content":["Metric count: 44","Last update time: Wed, 24 Sep 2014 01:31:55 GMT","Uptime: 10,037 minute(s)","Memory"," Total"," Initial: 548,288 kB"," Committed: 740,032 kB"," Maximum: 4,301,824 kB"," Used: 477,308 kB"," Heap"," Initial: 524,288 kB"," Committed: 655,360 kB"," Maximum: 3,728,384 kB"," Used: 393,227 kB"," Usage: 10.547 %"," Non-Heap"," Initial: 24,000 kB"," Committed: 84,672 kB"," Maximum: 573,440 kB"," Used: 84,081 kB"," Usage: 14.663 %","NIO Buffer Memory"," Direct"," Capacity: 0 bytes"," Used: 0 bytes"," Buffers: 0"," Mapped"," Capacity: 0 bytes"," Used: 0 bytes"," Buffers: 0","Garbage Collection (last 1 minute(s))"," Executions: 0"," Elapsed time: 0 ms",


"Threads"," Total count: 122"," By Type"," Daemon: 65"," Non-daemon: 57"," By State"," Blocked: 0"," Deadlocked: 0"," New: 0"," Runnable: 7"," Terminated: 0"," Timed waiting: 24"," Waiting: 91","Operating System"," CPU Usage"," System: 0.079 %"," JVM: 0.058 %"," File Descriptors"," Maximum: 8,192"," Open: 214"," Usage: 2.612 %"]},{"title":"Licensing","id":"licensing","content":["Number of licenses Found: None"]}]}


Result

All of the metrics tracked by the controller with regard to the JVM, including those that are also persisted as time-series data because they do vary throughout the JVM’ lifetime, are available in the controller support report.

The controller support report offers various information, such as the number of installed applications, configurationdata, and the number of alerts and audit logs in the database.


Aruba VAN SDN Controller 2.8 Administrator Guideh20628.SDN architecture separates the network control plane from the forwarding hardware on network devices. Control can then be centralized,

Documents