Cisco Cloud Object Storage Release 2.1.2 User Guide · Contents vi Cisco Cloud Object Storage Release 2.1.2 User Guide Replicating Objects During Swift Write Operations A-15 APPENDIX

Cisco Cloud Object Storage Release 2.1.2 User Guide

April 17, 2015

Cisco Systems, Inc.www.cisco.com

Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco website at www.cisco.com/go/offices.

http://www.cisco.com

http://www.cisco.com/go/offices

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

Cisco Cloud Object Storage Release 2.1.2 User Guide

© 2014-2015 Cisco Systems, Inc. All rights reserved.

http://www.cisco.com/go/trademarks

C O N T E N T S

iii

Preface vii

Audience vii

Document Organization vii

Document Conventions viii

Related Publications ix

Obtaining Documentation and Submitting a Service Request ix

C H A P T E R 1 Overview 1-1

Product Description 1-1

COS and MOS 1-1

Components 1-1

Networks 1-2

Platform and Applications Manager (PAM) 1-2

COS Nodes 1-2

COS Cluster 1-3

Object Store Metadata 1-3

Hardware Platforms 1-3

Features 1-3

Overview 1-4

COS Service Manager GUI 1-4

High Availability (HA) 1-5

Swauth API 1-5

Swift Object Store API 1-6

Object Store Metadata Resiliency 1-7

Object Data Resiliency 1-7

Service Load Balancing 1-7

CLI Utilities 1-8

Prerequisites 1-8

Restrictions and Limitations 1-8

iiiCisco Cloud Object Storage Release 2.1.2 User Guide

Contents

C H A P T E R 2 Deploying COS 2-1

Pre-Deployment Tasks 2-1

COS Node Hardware Installation 2-1

PAM Installation and Configuration 2-2

Setting Up the External DNS Server 2-3

Editing NTP Server Information 2-6

Viewing DNS Server Information 2-7

Configuring COS 2-7

Configuring the Service Instance Template 2-8

Configuring IP Pools 2-9

Editing or Deleting IP Pools 2-9

Configuring a COS Cluster 2-10

COS Node Initial Configuration 2-10

Adding a COS Node to a COS Cluster 2-12

Configuring a COS Node 2-12

Configuring the Service Interface of a COS Node 2-13

Configuring Cassandra Database Maintenance 2-14

C H A P T E R 3 System Monitoring 3-1

COS Application Status Monitoring 3-1

COS Node Status Monitoring 3-2

Viewing COS Node Status 3-2

AIC Client Monitoring 3-3

Storage Monitoring 3-3

Interface Monitoring 3-3

Services Monitoring 3-3

COS Management Events 3-3

Viewing COS Node Management Events 3-3

Service Events 3-4

COS AIC Client Events 3-5

COS AIC Server Events 3-6

COS Node Platform Monitoring with SNMP 3-6

Overview 3-6

Installation 3-6

Configuration 3-7

MIB Extensions 3-7

Extensions to HOST-RESOURCES-MIB::hrDeviceTable 3-7

Extensions to HOST-RESOURCES-MIB::hrDiskStorageTable 3-8

ivCisco Cloud Object Storage Release 2.1.2 User Guide

Contents

Extensions to HOST-RESOURCES-MIB::hrStorageTable 3-8

Extensions to UCD-SNMP-MIB::dskTable 3-9

Monitored Items 3-9

A P P E N D I X A Reference Information A-1

COS Service Model A-1

Using the COS Service Manager GUI A-2

Before You Begin A-2

Logging In A-3

Using the Home Page A-3

Nodes Allocated A-4

Alarm Summary A-5

Service Summary A-5

Using the Navigation Panel A-7

Selecting the View A-7

Home Button A-7

Infrastructure A-8

Service Domain Objects A-8

Services A-8

Monitoring A-8

Working with Tables A-9

Adding an Entry A-9

Editing an Entry A-9

Deleting an Entry A-10

Refreshing the Table A-10

Sorting Entries A-10

Changing the Order of Columns A-10

Filtering Contents A-10

Managing GUI Users A-11

COS Network Ports and Services A-11

COS Maintenance A-12

COS Node Reboot A-12

Behavior of COS Services on COS Node Boot A-13

COS Service Reliability A-13

COS Node Disks A-14

COS Services A-14

COS Node Interfaces A-14

Server Reachability A-14

COS Node Hard Drive Replacement A-15

vCisco Cloud Object Storage Release 2.1.2 User Guide

Contents

Replicating Objects During Swift Write Operations A-15

A P P E N D I X B Troubleshooting B-1

Viewing the Statuses of Primary System Services B-1

Service High-Availability B-1

Troubleshooting Swift and Swauth API Errors B-1

Unresponsive COS Service Manager GUI B-2

Viewing COS AIC Logs B-3

A P P E N D I X C COS Tunables C-1

Debug Tunables C-1

Info Tunables C-2

Configuration Tunables C-5

A P P E N D I X D COS Command Line Utilities D-1

Hardware Prerequisites D-1

Installing the CLI Utilities D-1

Using the CLI Utilities D-3

cos swauth D-3

cos-swift D-5

viCisco Cloud Object Storage Release 2.1.2 User Guide

Preface

This preface describes who should read the Cisco Cloud Object Storage Release 2.1.2 User Guide, how it is organized, and its document conventions. It contains the following sections:

• Audience

• Document Organization

• Document Conventions

• Related Publications

• Obtaining Documentation and Submitting a Service Request

AudienceThis guide is for the networking professional managing the Cisco Cloud Object Storage (COS) product. Before using this guide, you should have experience working with Linux platforms, and be familiar with the concepts and terminology of Ethernet, local area networking, clustering and high-availability, and network services like DNS and NTP.

Document OrganizationThis document contains the following chapters and appendices:

Chapters or Appendices Descriptions

Chapter 1, “Overview” Describes the COS, its components, features, and prerequisites for deployment.

Chapter 2, “Deploying COS” Gives procedures for deploying the COS components.

Chapter 3, “System Monitoring” Provides tools and methods for monitoring the COS system and components.

Appendix A, “Reference Information” Describes the COS GUI, additional features, maintenance details, and tools for managing the COS system.

Appendix B, “Troubleshooting” Provides information and procedures for troubleshooting COS problems.

viiCisco Cloud Object Storage Release 2.1.2 User Guide

PrefaceDocument Conventions

Document ConventionsThis document uses the following conventions:

Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the manual.

Tip Means the following information will help you solve a problem. The tips information might not be troubleshooting or even an action, but could be useful information, similar to a Timesaver.

Caution Means reader be careful. In this situation, you might perform an action that could result in equipment damage or loss of data.

Timesaver Means the described action saves time. You can save time by performing the action described in the paragraph.

Appendix C, “COS Tunables” Describes the Debug, Info, and Configuration tunables available in COS.

Appendix D, “COS Command Line Utilities”

Provides instructions for installing and using the COS CLI utilities.

Chapters or Appendices Descriptions

Convention Indication

bold font Commands and keywords and user-entered text appear in bold font.

italic font Document titles, new or emphasized terms, and arguments for which you supply values are in italic font.

[ ] Elements in square brackets are optional.

{x | y | z } Required alternative keywords are grouped in braces and separated by vertical bars.

[ x | y | z ] Optional alternative keywords are grouped in brackets and separated by vertical bars.

string A nonquoted set of characters. Do not use quotation marks around the string or the string will include the quotation marks.

courier font Terminal sessions and information the system displays appear in courier font.

< > Nonprinting characters such as passwords are in angle brackets.

[ ] Default responses to system prompts are in square brackets.

!, # An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line.

viiiCisco Cloud Object Storage Release 2.1.2 User Guide

PrefaceRelated Publications

Warning IMPORTANT SAFETY INSTRUCTIONS

This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents. Use the statement number provided at the end of each warning to locate its translation in the translated safety warnings that accompanied this device.

SAVE THESE INSTRUCTIONS

Warning Statements using this symbol are provided for additional information and to comply with regulatory and customer requirements.

Related PublicationsRefer to the following documents for additional information about COS 2.1.2:

• Release Notes for Cisco Cloud Object Storage Release 2.1.2

• Cisco Content Delivery Engine 205/220/250/420/460/470 Hardware Installation Guide

• Cisco Cloud Object Storage Release 2.1.1 API Guide

• Cisco Media Origination System Release 2.3 User Guide

• Open Source Used in Cisco Cloud Object Storage Release 2.1.2

Obtaining Documentation and Submitting a Service RequestFor information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What’s New in Cisco Product Documentation at: http://www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html.

Subscribe to What’s New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation as an RSS feed and delivers content directly to your desktop using a reader application. The RSS feeds are a free service.

ixCisco Cloud Object Storage Release 2.1.2 User Guide

http://www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html

PrefaceObtaining Documentation and Submitting a Service Request

xCisco Cloud Object Storage Release 2.1.2 User Guide

C

C H A P T E R 1
Overview
Product DescriptionThe Cisco Cloud Object Storage (COS) provides distributed, resilient, high-performance storage and retrieval of binary large object (blob) data. The primary interface for managing COS content is the OpenStack Swift API, with enhancements that improve the quality of service when accessing large media objects.

Object storage is distributed across a cluster of hardware systems, or nodes. The storage cluster is resilient against hard drive failure within a node and against node failure within the cluster. Nodes may be added to or removed from the cluster as needed to provide for changes in cluster capacity. To administer the cluster, COS includes an HTTP-based cluster-management API.

COS also includes an authentication and authorization service that implements the OpenStack Swauth API.

COS and MOSCOS is designed to integrate transparently with the Cisco Media Origination System (MOS), which is designed for highly optimized ingest and storage. MOS uses a hierarchical storage design that supports huge content libraries while simplifying content storage management. Its distributed architecture can separate ingest and storage from streaming, allowing each function to be scaled independently as needed to dynamically increase network ingest and storage resources. For more information, see the Cisco Media Origination System Release 2.3 User Guide.

ComponentsCOS has a number of subsystems.

• Networks — Interfaces are grouped into distinct networks to isolate management functions from high-volume data traffic.

• Clusters and Nodes — COS services are provided by a cluster of nodes, with both the cluster and the individual nodes as distinctly manageable components.

• Object Metadata Store — The metadata for the cluster is stored in a high-performance distributed NoSQL database hosted on the COS nodes in a cluster.

• Platform and Applications Manager (PAM) — COS components are managed using services running on the Platform and Applications Management system (PAM).

1-1isco Cloud Object Storage Release 2.1.2 User Guide

Chapter 1 Overview Components

• Hardware Platforms — COS software is deployed on Cisco Content Delivery Engine (CDE) platforms.

Each of these components and subsystems are described further in the following sections.

NetworksCOS divides network interfaces into two groups: the management network and the data network. The management network is used for monitoring and managing the COS cluster and individual COS nodes. The data network is used by client applications to interact with the COS authentication and authorization services, and the COS object storage services. The client applications use Swauth API to interact with the COS authentication and authorization services, and the Swift API to interact with the COS object storage services.

In customer installations, COS management network traffic can be routed with management traffic meant for non-COS systems. When routing COS data network traffic, however, it must be ensured that high-volume data traffic does not congest networks used to manage COS and other systems.

For this reason, each network adapter in a COS node is assigned to either the management network or the data network. Typically, high-bandwidth network adapters are assigned to the data network. One or two lower-bandwidth adapters are usually configured for use on the management network.

Platform and Applications Manager (PAM)The PAM is a virtual instance dedicated to the installation, configuration, monitoring and recovery of other COS components. It acts as the management machine for the COS system and runs the following services and applications:

• The Platform Manager – It facilitates setting up of networks and configuration of external DNS servers, and hosts the NTP Server

• The Document Server, including MongoDB and Redis

• The COS Service Manager

• The COS Application Instance Controller (AIC), which facilitates management of COS appliances

For more information on PAM and its components, see the Cisco Media Origination System Release 2.3 User Guide.

COS NodesThe COS software runs on a collection of computing systems, called nodes, that are connected via the management and data networks. Currently, there are two types of COS nodes: the cluster controller that runs the PAM software and the COS AIC software, and the storage nodes.

The storage nodes host software that manages object-store and authentication and authorization service metadata, stores and retrieves object contents, and communicates with the cluster controller. COS storage nodes may be added or removed without disrupting COS service availability. Adding nodes is a way of elastically increasing the storage and bandwidth capacity of the COS cluster.

1-2Cisco Cloud Object Storage Release 2.1.2 User Guide

Chapter 1 Overview Features

The COS node software includes a customized Linux distribution, currently based on CentOS 6. This provides the basic framework for the other software applications and modules that run on the node. Each node runs a number of daemons in the Linux user-space. These include daemons to coordinate service log files, to communicate with the cluster controller, to provide a distributed database for object-store metadata, and to communicate with the modules running in the kernel.

Kernel modules are used to provide real-time management of node hardware resources, to provide the distributed, resilient content-store used for object-store data, and to provide the Swift and Swauth API support via the data network. While the management-network interfaces communicate directly with the user-space daemons, the data-network interfaces communicate directly with the kernel modules.

COS ClusterCOS services are provided by software running on a set of nodes called a COS cluster that is connected by data and management networks. Each COS cluster has a single fully-qualified domain name (FQDN) that is used by client applications to access COS services. A COS cluster also has a number of configuration parameters that define the cluster's behavior. Some of these parameters include the Swift and Swauth API constraints, the IP address pools used to assign IP addresses to individual node network adapters, and the IP address of the PAM configuration document server. For a detailed description of the configuration parameters, see Chapter 2, “Deploying COS”.

Object Store MetadataCOS object store metadata and Swauth service data are stored in the high-performance, resilient NoSQL Cassandra database. The cosd daemon running on each COS node acts as the Cassandra client and implements the schema for Swift and Swauth metadata documents stored in Cassandra. Each COS storage node runs an instance of the Cassandra server, so metadata storage capacity increases linearly along with content storage capacity as COS storage nodes are added to the cluster.

Hardware PlatformsCOS software can be deployed on the Cisco Content Delivery Engine (CDE) platforms CDE460 and CDE470. For information about the CDE460 and CDE470 platforms, see the Cisco Content Delivery Engine 205/220/250/420/460/470 Hardware Installation Guide.

Features• Overview, page 1-4

• COS Service Manager GUI, page 1-4

• High Availability (HA), page 1-5

• COS Service Manager GUI, page 1-4

• Swift Object Store API, page 1-6

• Object Store Metadata Resiliency, page 1-7

• Object Data Resiliency, page 1-7

• Service Load Balancing, page 1-7


http://www.cisco.com/c/en/us/td/docs/video/cds/cde/cde205_220_420/installation/guide/cde205_220_420_hig.html



• CLI Utilities, page 1-8

OverviewThe table below provides an overview of the COS features.

COS Service Manager GUIThe COS Service Manager GUI enables you to quickly and easily configure the COS infrastructure, service domain objects, and services. The GUI also provides valuable monitoring functions, including alarm and alarm history display, system and service diagnostics, event display, log analysis and display, and configuration display.

Table 1-1 Overview of COS Features

Feature Set Features

COS Service Manager GUI • Lets you quickly and easily access many COS deployment and monitoring functions

High Availability (HA) • COS now supports HA as implemented in MOS

Swauth API • Simple Auth Service API for authentication of Swift operations

• Based on Swauth Open-Source Middleware API

• Used to manage accounts, users and account service endpoints

Swift Object Store API • An implementation of a subset of the continually evolving OpenStack Swift API

• Command executions are authenticated using auth tokens provided by Swauth service

• Used to create and manage containers and objects for persistent storage in a COS cluster

Object Store Metadata Resiliency • Metadata resiliency is provided by a distributed and replicated Cassandra document database

• Each COS node participates in the persistence of a subset of the Cassandra database

• Manual administrative intervention required upon node failure

Object Store Data Resiliency • Data is resilient to both hard drive and COS node failures

• Local COS node data resiliency provided by local software RAID

• COS cluster data resiliency provided by object replication

Service Load Balancing • COS cluster load balancing is provided by DNS round-robin of a FQDN to multiple physical IPv4 addresses hosted by COS nodes

• Optimal load balancing is provided by extensions to the Swift API through the implementation of HTTP redirect



For information on launching an navigating the GUI, the COS Service Manager GUI section of Appendix A, “Reference Information”.

High Availability (HA)The PAM has two classes of components for HA:

• Third party components, such as ZooKeeper, MongoDB, and Redis, use their own proprietary clustering and redundancy schemes.

• Cisco components, such as the COS Service Manager GUI and DocServer, use ZooKeeper for leader election.

In an HA environment, multiple PAM VMs provide redundancy for the applications that run on the PAM. HA requires at least three PAM VMs, because applications such as ZooKeeper, MongoDB, and Redis require at least three components to form a working cluster.

Many of these applications also require a majority in order to form a quorum. That is, a cluster of three components can recover from the failure of a single component, because there are still two components to form a majority. But if two components fail, the single remaining component is not a majority, and the cluster cannot recover until one of the failed components recovers.

Therefore, we recommend that you configure more than three PAM VMs to ensure recovery in the event of multiple failures and to support high performance, especially the sharing of databases and other applications.

Swauth APICOS includes a basic authentication service that can be used when COS is not installed along with other OpenStack services such as the Keystone Identity service. The API for the COS authentication service is derived from the OpenStack Swauth middleware component API. The authentication service API provides the following functions for managing accounts, users, and service endpoints:

• Listing Accounts

• Retrieving Account Details

• Creating an Account

• Deleting an Account

• Creating or Updating a User

• Retrieving User Details

• Deleting a User

• Creating or Updating Account Service Endpoints

• Getting an Authentication Token

For a detailed description of these functions, see the Cisco Cloud Object Storage Release 2.1.1 API Guide.



Swift Object Store APIThe COS object storage API is based on the OpenStack Swift API. It is implemented as a set of Representational State Transfer (ReSTful) web services. All account, container, and object operations can be performed with standard HTTP calls. The requests are directed to the host and URL described in the X-Storage-Url HTTP header that is part of the response to a successful request for an authentication token.

The COS object storage API defines restrictions on HTTP requests. These restrictions, borrowed from the Swift API, are listed in the table below.

Also, the container and object names must be UTF-8 encoded and then URL-encoded before inclusion in the HTTP request line.

Note All the length restrictions are enforced against the URL-encoded request line.

The COS object store API provides the following functions, some of which provide extended functionality beyond the standard SWIFT API defined by OpenStack:

• Listing Containers

• Listing Objects

• Creating a Container

• Deleting a Container

• Retrieving an Object

• Creating or Updating An Object

• Deleting an Object

• Creating or Updating Container Metadata

• Retrieving Container Metadata

• Deleting Container Metadata

• Retrieving Object Metadata

For a detailed description of these functions, see the Cisco Cloud Object Storage Release 2.1.1 API Guide.

Table 1-2 COS API Restrictions

Constraint Value

Maximum # of HTTP Headers per request 90

Maximum length of all HTTP Headers 4096 bytes

Maximum length per HTTP request line 8192 bytes

Maximum length of container name 256 bytes

Maximum length of object name 1024 bytes



Object Store Metadata ResiliencyCOS stores metadata for Swift and Swauth accounts, users, containers, and objects as documents in a Cassandra database instance. Cassandra is a distributed document store. In a typical multi-node Cassandra cluster, no single node persists (saves) to local disk a copy of the entire database. Instead, each Cassandra cluster node locally persists a subset of the database. To ensure resiliency of the data in case of node failure, Cassandra has configuration options to specify the number of document replicas to maintain on separate cluster nodes.

For metadata resiliency in COS, each COS cluster node participates in the Cassandra cluster, and each COS node locally persists a part of the Cassandra database. The database cluster is automatically configured to create document replicas to be resilient to a single node failure.

If a second node fails before full metadata resiliency is restored or before full content resiliency is restored, there is a risk of data loss.

Object Data ResiliencyThe COS stores Swift object data to local disk or to external facing hard drives. To maintain data resiliency in the event of a loss of a local hard drive, the COS system, by default, uses erasure coding techniques (software RAID) to stripe the object data across multiple local drives. The default scheme for erasure coding is to maintain two parity blocks for twelve data blocks. This ensures that data is resilient to the loss of two local hard drives. In the event of a hard drive loss, the COS system immediately begins to recover the data from the lost drive and replace the missing data blocks on surviving hard drives. The execution of this recovery process, including the duration, is scheduled with low priority, and depends on the availability of system resources and available disk space.

For data resiliency in the event of a COS node failure, the COS cluster can be configured to maintain copies of object data on one or more additional COS nodes. Recommended practice is to configure the COS cluster to maintain at least two copies of object data for resiliency. When configured for multiple object copies, the COS cluster automatically attempts to create the configured object copy count within the cluster in the event of a COS node failure, without manual administrative action. As soon as the COS cluster detects a node failure, the cluster begins to create additional copies of objects that were maintained on the failed node. Upon restoring the failed COS node, the COS cluster purges unnecessary copies to recover disk space.

Service Load BalancingThe COS cluster is composed of COS nodes. Each node has limited CPU, network, and disk resources. To ensure the best performance and quality of service, the workloads of the Swift and Swauth operations must be distributed among the COS nodes. The recommended solution for service load balancing is to use a DNS system to round-robin clients to different physical IP addresses hosted by the various COS nodes. While not a perfect solution, such a DNS round-robin should provide sufficient distribution of workloads across the COS nodes.

In addition to using DNS to distribute workload, the COS Swift implementation supports intelligently redirecting a Swift client to an optimal location for Swift object create and read operations using standard HTTP redirect semantics. Given a Swift client that supports HTTP redirect semantics, the client can provide an X-Follow-Redirect: true HTTP header in the HTTP PUT and GET requests for Swift object create and read operations. In the event that a more optimal location should be used for the operation, the COS node will respond with an HTTP 307 (temporary redirect) status, indicating to the client where the operation should be requested.


Chapter 1 Overview Prerequisites

For Swift object read operations, COS provides two levels of service and transfer profile: best-effort and committed rate. These levels of service contribute to service load balancing. COS provides extensions to Swift object read that allows the client to request a guaranteed and committed transfer rate as the data is sent from the COS node. A COS node can reject a read request if the client has requested a committed rate transfer, and the COS node does not have sufficient resources available to satisfy the client's request. If a client does not request a committed rate transfer, the COS node will attempt to satisfy the request with the system resources available, and at a priority lower than that of any in-progress committed rate requests. For more information, see the Cisco Cloud Object Storage Release 2.1.1 API Guide.

CLI UtilitiesCOS provides the following command line utilities for use on Linux:

• cos-swift — provides command-line access to the Swift API.

• cos-swauth — provides command-line access to the Swauth API.

Note These utilities do not work between two COS nodes or between a COS node and a local node, as the HTTP request will be refused.

For more information on the COS command line utilities, see the COS Command Line Utilities section of Appendix A, “Reference Information”.

PrerequisitesThe COS management and configuration operations require specific hardware components for deployment. For more information on the hardware requirements, see the MOS 2.3 User Guide.

The COS system is most effective in engineered networks, with separate routes for management and data flow. In designing and provisioning networks, capacity for the high data-network throughput for the expected application of COS must be ensured. Also, the high data traffic generated by the COS systems must not interfere with the management network segment or other important network segments.

Restrictions and Limitations• The COS does not support IPv6.

• The OpenStack Swift and Swauth APIs continue to evolve. COS does not currently implement all the Swift or Swauth API functions. For a list of supported functions, see Swift Object Store API and Swauth API in this chapter.

• Secure Sockets Layer (SSL) or other means for providing session security and encryption are not supported with the Swift and Swauth APIs.

• See the Release Notes for Cisco Cloud Object Storage Release 2.1.2 for open caveats and known issues related to this release of the COS software.


C

C H A P T E R 2
Deploying COS
To deploy the COS 2.1.2 software, perform the following tasks in order:

1. Pre-Deployment Tasks

2. Setting Up the External DNS Server

3. Configuring COS

Pre-Deployment TasksBefore deploying the COS, you must complete the following preliminary tasks:

Step 1 Prepare your COS VMware datacenter topologies and networks. The COS requires the following networks:

• Management — This is the primary COS network that connects all of the components.

• Cache Fill — This is the data network. It connects the COS Appliances.

Step 2 Download the COS Service Manager Open Virtual Appliance (OVA) file and load the image into a repository (HTTP server) that is accessible by vCenter.

Step 3 Determine the blade and VM layout that you will use, and your management IP address allocation scheme for the network interfaces.

Step 4 Install the external DNS and NTP servers.

• Determine the origin service FQDN and prepare the downstream client (client-facing) DNS servers to point to the COS Appliance dataplane (primary interface) IP addresses.

COS Node Hardware InstallationCOS software can currently be deployed on the Cisco Content Delivery Engine (CDE) platforms CDE460 and CDE470. For information about installing the CDE460 and CDE470 platforms, see the Cisco Content Delivery Engine 205/220/250/420/460/470 Hardware Installation Guide.



Chapter 2 Deploying COS Pre-Deployment Tasks

Upgrading to COS 2.1.2If upgrading to COS 2.1.2 from an earlier COS release, perform the following steps on each COS node:

Step 1 Log in to the COS node as the root user.

Step 2 Copy the COS 2.1.2 software ISO image to /root on each target COS node.

Step 3 Mount the ISO image to /mnt/cdrom.

[root@cos-node ~]# mount -o loop /root/cos_repo-2.1.2.iso /mnt/cdrom

Step 4 Prepare the YUM software repository using the following sequence of commands:

[root@cos-node ~]# rm -f /etc/yum.repos.d/*.repo[root@cos-node ~]# /mnt/cdrom/local_repo_setup[root@cos-node ~]# yum clean all

Step 5 Upgrade the COS software using the following sequence of commands:

[root@cos-node ~]# yum -y groupinstall cos cos_support cos_test[root@cos-node ~]# yum -y upgrade[root@cos-node ~]# reboot

The system reboots and returns to service running the upgraded COS software version.

PAM Installation and ConfigurationTo deploy the PAM from the vCenter, do the following:

Step 1 Log into the vCenter.

Step 2 Click File > Deploy OVF Template.

Step 3 Choose the cos-mgmt ova file and click Next.

Step 4 Verify the OVF template details and click Next.

Step 5 Accept the license agreement and click Next.

Step 6 Enter a name and select a location for the deployed template. Then, click Next.

Step 7 Choose the UCS on which the PAM is to be deployed and click Next.

Step 8 Do not change the disk format. Click Next.

Step 9 In the Network Mapping screen, choose VM Network from the Destination Networks drop-down list for source network Network for adapter 1. Click Next.

Step 10 In the Networking section, enter the following information:

• Hostname — Local hostname of the PAM. Do not enter the fully qualified domain name (FQDN). For example, if the FQDN is pam.cisco.com, enter only pam.

• Domain — The domain name used by the PAM, such as testdns.com (required).

• IP Address — Primary management IP address to which the MOS can SSH (required). This is the IP address for the Management network for the PAM and for configuring Ethernet port eth0. The management IP address must be accessible from a management PC (for example, from a PC that is running vSphere).

• Subnet Mask — Default subnet mask for the PAM (required).


Chapter 2 Deploying COS Pre-Deployment Tasks

• Gateway — Default gateway IP address for the PAM (required).

Step 11 In the Additional Domain Name Servers section, enter the following information:

• DNS Server — Name or IP address of the DNS server used by the PAM to forward queries that it could not resolve (that is, the DNS forwarder). Enter names for all of the DNS servers used by the PAM.

Step 12 In the Additional Network Time Protocol Servers section, enter the following information:

• NTP Server — Name or IP address of the external Network Time Protocol (NTP) server used by the PAM to synchronize its clock. Enter names for all of the NTP servers used by the PAM.

You must enter at least one NTP server.

Step 13 In the External DNS Credentials section, enter the following information:

• DNS Server — Name or IP address of the external DNS server used by the PAM to forward queries that it could not resolve.

• Transaction Signature Key — Transaction Signature (TSIG) key that is configured on the external DNS server.

• Transaction Signature Algorithm — Name of the TSIG algorithm that is configured on the external DNS server.

Note The administrator must ensure that the DNS zones are configured in the external DNS server.

Step 14 Click Next, then click Finish to deploy the OVA and create the PAM VM. The progress bar shows the status of the deployment.

Step 15 After the deployment is complete, map the adapter to the correct network, and power on the PAM VM.

Step 16 Verify connectivity and open an SSH session into the PAM VM.

Upgrading the PAM COS ControllerWhen upgrading to COS 2.1.2 from an earlier COS release, perform the following steps to upgrade the PAM COS controller:

Step 1 Log on to the PAM COS controller as the root user.

Step 2 Copy the cos-aic rpm to be upgraded to the PAM root directory.

Step 3 Run the command supervisorctl stop cos-controller.

Step 4 Get the current cos-controller rpm using the command rpm-qa|grep cos-aic.

Step 5 Remove the current rpm using rpm –e current cos-aic.rpm.

Step 6 Install the new rpm using the command rpm –ivh new cos-aic.rpm.

Step 7 Start the cos-controller again using supervisorctl start cos-controller.


Chapter 2 Deploying COS Setting Up the External DNS Server

Downgrading the PAM COS ControllerWhen downgrading from COS 2.1.2 to an earlier COS release, perform the following steps to downgrade the PAM COS controller:

Step 1 Log in to the PAM Console as the root user.

Step 2 Copy the cos-aic rpm to be downgraded to the PAM root directory.

Step 3 Run the command supervisorctl stop cos-controller.

Step 4 Get the current cos-controller rpm using the command rpm-qa|grep cos-aic.

Step 5 Remove the current rpm using rpm –e current cos-aic.rpm.

Step 6 Install the new rpm using the command rpm –ivh new cos-aic.rpm.

Step 7 Start the cos-controller again using supervisorctl start cos-controller.

Note The RPM current cos-aic.rpm is the output of the previous command rpm-qa|grep cos-aic, which is currently running on the system. The RPM cos-aic.rpm is the new RPM to be installed on the PAM.

Setting Up the External DNS ServerThe administrator is responsible for creating the transaction signature (TSIG) key for the TSIG algorithm on the external DNS server. Valid TSIG algorithms are hmac-md5, hmac-sha1, hmac-sha224, hmac-sha256, hmac-sha384, and hmac-sha512. The following procedure shows how to create the TSIG key for the hmac-md5 algorithm.

Step 1 Generate the TSIG key by entering the following command:

dnssec-keygen -a HMAC-MD5 -b 128 -n HOST testdns.com.

where:

• HMAC-MD5 is the TSIG algorithm.

• 128 is the number of bits in the key.

• testdns.com. is the name of the key. The name of the key, the domain name of the PAM, and the DNS zone in the external DNS server should all be the same (in this example, testdns.com).

• The command must end with a period (.), which is required when generating the key.

Sample output:

dnssec-keygen -a HMAC-MD5 -b 128 -n HOST testdns.com.

Ktestdns.com.+157+05519

This command creates a key file and a private key file.

• Sample .key file:

Ktestdns.com.+157+05519.key

testdns.com. IN KEY 512 3 157 ujLdXfCZenQZQKZlFy42fw==

• Sample .private key file:



Ktestdns.com.+157+05519.private

\Private-key-format: v1.3

Algorithm: 157 (HMAC_MD5)

Key: ujLdXfCZenQZQKZlFy42fw==

Bits: AAA=

Created: 20140325141250

Publish: 20140325141250

Activate: 20140325141250

Step 2 Create the key file (in this example, testdns.com.key) in the /etc/ directory.

Sample key file:

testdns.com.key

key testdns.com. {

algorithm hmac-md5;

secret “ujLdXfCZenQZQKZlFy42fw==“;

};

where:

• hmac-md5 is the TSIG algorithm.

• ujLdXfCZenQZQKZlFy42fw== is the TSIG key.

Step 3 Add the key file path to the /etc/named.conf file by inserting the following line:

include “/etc/testdns.com.key“;

Step 4 Configure the DNS zones (used when deploying the PAM) on the external DNS server as shown in the following examples.

a. Update the /etc/named.conf file with the details of the DNS zone (in this example, testdns.com) and reverse zone, for all interfaces.

In the following sample information:

– The DNS zone is related to the 172.20.216.xx subnet, which can be the Management interface.

– For the Data In and Data Out interfaces, similar information related to the reverse zone must be added.

– Data In is related to the 15.1.1.x subnet.

– Data Out is related to the 25.1.1.x subnet.

– testdns.com. is the key name.

DNS Zone Detailszone testdns.com IN {type master;file “slaves/db.testdns.com“;allow-update { key “testdns.com.“; };notify yes;};

Reverse Zone Details (Management Interface)zone 216.20.172.IN-ADDR.ARPA IN {



type master;file “slaves/db.216.20.172“;allow-update { key “testdns.com.“; };notify yes;};

Reverse Zone Details (Data In Interface)zone 1.1.15.IN-ADDR.ARPA IN {type master;file “slaves/db.1.1.15“;allow-update { key “testdns.com.“; };notify yes;};

Reverse Zone Details (Data Out Interface)zone 1.1.25.IN-ADDR.ARPA IN {type master;file “slaves/db.1.1.25“;allow-update { key “testdns.com.“; };notify yes;};

b. Create db.* files in the /var/named/slaves directory in the external DNS server for the DNS zone and reverse zone, for all interfaces.

Sample db File for DNS Zone (db.testdns.com)@ 86400 IN SOA dns pam 2014031001 3600 1800 604800 86400

@ 86400 IN NS dns

dhcp 86400 IN CNAME dns

dns 86400 IN A pam_ip_address

ntp 86400 IN CNAME dns

Sample db File for Reverse Zone of Management Interface (db.216.20.172)$ORIGIN .

$TTL 86400 ; 1 day

216.20.172.IN-ADDR.ARPA IN SOA dns.testdns.com. pam.testdns.com. (

2012071021 ; serial

3600 ; refresh (1 hour)

1800 ; retry (30 minutes)

604800 ; expire (1 week)

86400 ; minimum (1 day)

)

NS dns.testdns.com.

$ORIGIN 216.20.172.IN-ADDR.ARPA.

$TTL 7200 ; 2 hours

xx PTR dns.testdns.com. //xx = last two digits of the PAM IP


Chapter 2 Deploying COS Editing NTP Server Information

Sample db File for Reverse Zone for Data In Interface (db.1.1.15)

$ORIGIN .

$TTL 86400 ; 1 day


2012071021 ; serial





)

NS dns.testdns.com.


$TTL 7200 ; 2 hours

1 PTR dns.testdns.com.

Sample db File for Reverse Zone for Data Out Interface (db.1.1.15)

$ORIGIN .

$TTL 86400 ; 1 day


2012071021 ; serial





)

NS dns.testdns.com.


$TTL 7200 ; 2 hours

1 PTR dns.testdns.com.

Step 5 Make sure that the PAM and the external DNS server are in sync for the time and date.

Editing NTP Server Information

Note NTP servers are configured when the PAM is deployed. You cannot add or delete NTP servers using this procedure.

To edit NTP server information using the COS Service Manager GUI, follow these steps:


Chapter 2 Deploying COS Viewing DNS Server Information

Step 1 Log in to the COS Service Manager GUI as described in Appendix A, “Reference Information”.

Step 2 Go to Infrastructure > Platform Services.

In the NTP Servers table, the following information is displayed for each NTP server:

• Name (required)

– The name of the NTP server is a string of up to 30 characters. Acceptable characters include uppercase and lowercase letters, numbers, periods (.), dashes (-), and underscores (_).

– The name must not begin with a period (.), and it is not case-sensitive.

• Description — The description is a string of up to 100 characters, and can include uppercase or lowercase letters, numbers, and any special characters.

• Region — The region in which the NTP server resides.

• Servers — The hostname or IPv4 address of the NTP server.

Step 3 Edit the Description, Region, and Servers fields of an NTP server as required.

Viewing DNS Server Information

Note DNS servers are configured when the PAM is deployed.

To view DNS server information using the COS Service Manager GUI, follow these steps:


Step 2 Go to Infrastructure > Platform Services.

In the DNS Servers table, the following information is displayed for each DNS server:

• Name (required)

– The name of the DNS server is a string of up to 35 characters. Acceptable characters include uppercase and lowercase letters, numbers, periods (.), dashes (-), and underscores (_).

– The name cannot begin with a period (.), and it is not case-sensitive.

• Description — The description is a string of up to 100 characters, and can include uppercase or lowercase letters, numbers, and any special characters.

• Region — Region in which the DNS server resides.

• IPv4 Address — IPv4 address of the DNS server.

• TSIG — Transaction Signature (TSIG) of the DNS server.

• TSIG Algorithm — Name of the TSIG algorithm used by the DNS server.

Configuring COSTo configure COS using the COS Service Manager GUI, follow these steps:



Chapter 2 Deploying COS Configuring COS

Step 2 Go to Infrastructure > Regions & Zones.

A zone is a set of Cloud platform components (compute, network, storage, and security) that are fate-shared. The zone can be mapped to the underlying Cloud platform provider, such as a datacenter in vCenter, an Availability Zone, or any other combination of fate-shared Cloud resource topologies. Each zone is associated with one Cloud Controller.

A region is made up of one or more zones. It is an abstract representation of the underlying Cloud platform. A region can be associated with a geographical region, one or more datacenters, or a service area. Release 2.1.2 of the COS software supports only one region.

Step 3 Expand a region to display the zones associated with that region.

Step 4 Click Add Row to add zones to the region, as needed.

Step 5 Go to Infrastructure > Networks to define IP pools for COS Node cache fill interfaces.

Step 6 On the COS node, execute /opt/cisco/cos/config/cosinit and configure the IP pool name, DocServer IP and Port (which is the COS Service Manager management IP). This will register the COS node with the COS Service Manager.

Step 7 Go to Infrastructure > COS Nodes and configure Cluster Name by choosing cluster.ca.01 from the drop-down list.

Step 8 Ensure that the Admin State for the node is set to Inservice. Click Save.

Step 9 Go to Services > Overview and confirm that you can see the registered COS Node.

Configuring the Service Instance TemplateTo configure the COS service instance template using the COS Service Manager GUI, do the following:


Step 2 Go to Services > Cloud Object Storage. The Service Summary page opens, displaying a list of COS service instances.

Step 3 Click the COS service instance to be configured. The Service Definition tab for that instance opens, containing the General and Service Endpoints areas.

Step 4 The fields of the General area are listed below. You can edit them appropriately.

• Title — The title of the COS service. The default title is An unused COS Service.

• Description — A brief description of the COS service. The default description is COS Service.

• Admin State — The administrative state of the COS service. From the drop-down list, you can choose to Enable or Disable the service.

• Service Template — Currently, this read-only field identifies the default service template, Cisco Object Store (COS) Service.

Step 5 The Service Endpoints area currently lists only one service endpoint. The following parameters are displayed for the service endpoint and can be edited appropriately:

• Name — The name of the service endpoint is set to the default value ce1 and cannot be changed.

• Description — A brief description of the service endpoint.

• Region — The region to which the service endpoint belongs.

• Min Nodes — The minimum number of nodes that must be associated the service endpoint.

• Desired Nodes — The desired number of nodes for the service endpoint.



• Max Nodes — The maximum number of nodes that can be associated with the service endpoint.

• Max Storage — This field is to be ignored for Release 2.0.1.

• Cluster Name — From the drop-down list, you can currently choose only the default COS cluster cluster.ca.01.

• Auth Profile — The default value is auth-1.

• State — The state of the service endpoint. From the drop-down list, choose Disabled or Enabled.

Step 6 For your changes to take effect, click Save. To discard your changes, click Cancel.

Configuring IP PoolsBefore installing COS nodes, you must configure IP pools using the COS Service Manager GUI.

To configure an IP pool, follow these steps:


Step 2 Go to Infrastructure > IP Pools and Networks.

Step 3 To add a new IP pool, click IP Address Pools > Add Row.

Step 4 In the Name field, enter a unique name for the IP pool. In the Description field, enter a brief description for the IP pool.

Step 5 To associate an IP address range with the IP pool, click IP Address Ranges > Add Row.

Step 6 Enter appropriate values in the fields described below:

Note If you attempt to initialize a COS node using an IP pool with an insufficient number of available IP addresses, the initialization fails and an Event is generated. For more information on this class of events, see “COS AIC Server Events”.


Note After you have associated an IP pool with a COS node, do not edit or delete the pool before dissociating the pool from the node.

Table 2-1 IP Address Ranges Fields

Field Description

Range Start The first IP address of the range.

Range End The last IP address of the range.

Netmask The netmask for the range.

Gateway The Gateway for the range.



Editing or Deleting IP PoolsTo edit or delete configured IP pools using the COS Service Manager GUI, follow these steps:


Step 2 Go to Infrastructure > IP Pools and Networks.

Step 3 Before editing or deleting an IP pool, ensure that it is not serving any COS nodes.

a. Dissociate any COS nodes associated with the IP pool to be edited or deleted, and link these nodes to another IP pool.

Step 4 Check the box against the name of the IP pool to be edited or deleted.

Step 5 Click Edit or Delete.

Configuring a COS ClusterAs part of the COS installation, a single default node cluster is created. The cluster can be configured using the COS Service Manager GUI.

To view or edit the settings of a COS node cluster, follow these steps:


Step 2 Go to Infrastructure > COS Clusters.The default COS node cluster is displayed.

Step 3 Edit the fields described below based on your deployment.


COS Node Initial ConfigurationCOS nodes ship with a pre-installed image. While provisioning a COS node on a network, you can upgrade the node to a different image.

After the image is installed and the node rebooted, you will be prompted to start configuration by running the cosinit script, which must be run to perform the initial setup and add the node to the COS management system.

• Enter yes to run cosinit immediately.

• If you wish to configure the node later, enter no. You can run the cosinit later using the following command: run /opt/cisco/cos/config/cosinit

The cosinit script can be run with the options shown below:

cosinit [–skipnw] [–input <configFile>] [–help]

Table 2-2 COS Node Cluster Fields

Field Description

Authentication FQDN The FQDN for COS authentication requests.

Storage FQDN Currently, this must be the same as the Authentication FQDN.



• [–skipnw] — Skip network configurations

• [–input <configFile>] — Specify an input file for configuration

• [–help] — Displays the usage

Examples:

/opt/cisco/cos/config/cosinit -skipnw

This is used to configure IP pool and DocServer, and register the cos node into COS Service Manager.

/opt/cisco/cos/config/cosinit -input configFile

This enables cosinit to run with a config file. All the parameters will be read from the config file.

Sample configFileInterfaceConfig : skip

(Optional. If the above line is added, management network configuration will be skipped.)

IpAddress : 172.22.99.218NetMask : 255.255.254.0Broadcast : 172.22.99.255DefaultGateway : 172.22.98.1HostName : CDE460-218PartNum : CDE460-4R1DocServerHost : 172.22.116.35DocServerPort : 5087BaudRate : 9600eth4 : 4eth5 : 4eth6 : 4eth7 : 4

Executing the cosinit script completes the following tasks:

• Configuring the management interface — ethX, IP, netmask, broadcast.

• Configuring the default gateway.

• Configuring the hostname.

• Mapping the network interfaces to IP addresses in the network.

• Configuring Doc Server IP address and Port.

• Recording appliance model name — CDE 470-4R2 or CDE 460-4R1.

• Configuring the baud rate by reading the value in install_baud_rate. If the install_baud_rate file is not found under the /root directory, the operator is prompted to enter a baud rate during cosinit execution or specify the value in the configFile.

When the cosinit script is successfully executed,

• /etc/sysconfig/network-scripts/ifcfg-eth0 is generated and the management interface configuration is saved in it.

• /etc/sysconfig/network is generated, and the hostname and default gateway info are saved in it.

• /etc/hosts is generated and the host info is saved in it.

• Network is restarted.

• /tmp/.cosnodeinit file is created with the configuration specified in cosinit.



• The baud rate is updated in the /boot/grub/grub.conf file.

• The COS AIC client and other applications are started.

• The COS AIC client reads the cosnodeinit generated by cosinit.

• Using the Doc Server IP address and port mentioned in cosnodeinit, the AIC client connects to the Doc Server.

• The AIC client sends the cosannounce to the Doc Server. cosannounce is a partially populated smcosnode document.

The COS node is now registered with the COS management system. You can configure the node using the COS Manager GUI, add it to a COS cluster, and make it operational.

Adding a COS Node to a COS ClusterAfter a COS node is installed and initialized, it must be added to a COS cluster to begin servicing COS requests.

To add a node to the COS cluster using the COS Service Manager GUI, do the following:


Step 2 Go to Infrastructure > COS Nodes.

Step 3 Check the box against the name of the node you want to edit and click Edit.

Step 4 From the Cluster Name drop-down list, choose the default COS cluster, cluster.ca.01.


When a COS node is added to a COS cluster, the following files are written on to the node:

• setupfile — This file holds the primary configuration data for the COS node and is required by CServer.

• RemoteServers — This file holds the service interface IP addresses for all of the COS nodes in the cluster.

• SubnetTable — This file holds the IP, Netmask, Gateway and Network data for each service interface of a COS node.

• cosd.conf — This file holds the configuration for the cosd service.

• cassandra.yaml — This file, located in /etc/cassandra/conf/, is responsible for the configuration of the Cassandra service.

When the node Admin State is set to Inservice, all enabled service interfaces of that node are written to the DNS (internal or external).

Note To avoid possible bootstrapping issues with the Cassandra service, be sure to provide (or use a script to provide) a delay of at least two minutes between adding two nodes in sequence to the cluster.

If the replication factor is too high, use the following script on any one node in the cluster to adjust it:

sh /opt/cisco/cos-aic-client/cassandra/cassandra-adjust-replication.sh { 1 | 2 | 3 }

For the final argument, use 1 for one node, 2 for two nodes, and 3 for three or more nodes.



Configuring a COS NodeAfter a COS Node is installed and initialized, you can modify its configuration parameters using the COS Service Manager.

To edit the settings of a functioning COS node using the COS Service Manager GUI, follow these steps:




Step 4 Edit the fields described below based on your requirements.

Note The Admin State of a COS node must be set to Maintenance before removing the node from a COS cluster.

Step 5 To modify the service interfaces of the node, follow the procedure “Configuring the Service Interface of a COS Node”.


Configuring the Service Interface of a COS NodeTo modify the settings of a service interface of a functioning COS node using the COS Service Manager GUI, follow these steps:




Step 4 Check the box against the name of service interface you wish to edit and click Edit.

Table 2-3 COS Node Fields

Field Description

Description A brief description of the node.

Model The part number of the node.

Zone The zone to which the node belongs.

Cluster Name The COS cluster to which the node belongs.

Admin State Indicates whether the node is Inservice or under Maintenance.



Step 5 Edit the fields described below based on your requirements.


Configuring Cassandra Database MaintenanceThe Cassandra database requires periodic maintenance with an anti-entropy repair that must be manually configured to execute every two days on each COS node. For more information on the Cassandra repair process, see the Cassandra 2.1 documentation available at:

www.datastax.com/documentation/cassandra/2.1/

To execute the periodic repair, we recommend configuring a CRON job on each node, and scheduling the CRON jobs on the nodes to begin at different times so as to avoid the repair running concurrently on more than one node at a time. As a reference, we recommend providing 15 minutes between scheduling of the repair operation on each node.

The command syntax for the repair is:

/usr/bin/nodetool repair -par -inc

Table 2-4 Service Interface Fields

Field Description

IP Pool The IP pool from which this interface will be assigned an IP address. You may associate a different IP pool with this interface only when the node Admin State is set to Maintenance.

Enabled From the drop-down list, choose True or False to enable or disable the service instance, respectively.




C

C H A P T E R 3
System Monitoring
The COS Service can be monitored through the following:

• COS Application Status Monitoring

• COS Node Status Monitoring

• COS Management Events

COS Application Status MonitoringThe COS service is implemented through one or more application instances, with each application instance serving a service instance, or an endpoint within a service instance. COS Release 2.1.2 supports only one application instance, namely, the Application Instance Controller (AIC). This model, in turn, allows for only one service instance, one service endpoint, one asset workflow template, and a single COS cluster.

The application status is reported by the AIC to the COS Service Manager. This status is accessible in the Service Summary pane of the Home page of the COS Service Manager GUI.

For each endpoint, the endpoint details, status as per the resource or capacity service level agreement (SLA), and Management events are reported.

The resource SLA includes a constraint based on the minimum, desired, and maximum number of COS nodes that can be in service.

• If none of the COS nodes is in service, the node status is set to critical.

• If the number of COS nodes in service is less than the minimum number of nodes, or greater than the maximum number of nodes, in the SLA, the node status is set to warning.

• If the number of COS nodes in service is within the SLA range, the node status is set to normal.

• If the COS Service is not properly configured, or if it is set to an inactive (maintenance) state, the node status and other SLA parameters, such as bandwidth and session statuses, is set to unavailable as part of the Application Status message.

In addition to the node status, the storage status is reported. Currently, the storage status is not calculated as per the SLA, but instead, is defined according to the following criteria:

• If the cumulative available storage on all nodes is 70% or less of the cumulative total storage on all nodes, the storage status is set to normal.

• If the cumulative available storage on all nodes is greater than 70% but no more than 90% of the cumulative total storage on all nodes, the storage status is set to warning.


Chapter 3 System Monitoring COS Node Status Monitoring

• If the cumulative available storage on all nodes is more than 90% of the cumulative total storage on all nodes, the storage status is set to critical.

The session and bandwidth status are also reported as part of the application status.

The session status is determined by analyzing the aggregate HTTP traffic for incoming and outgoing sessions separately. Based on the analysis, a status (unavailable, normal, warning, critical) is assigned to both the incoming and the outgoing sessions. The more severe of the two statuses is reported as the session status.

Similarly, the bandwidth status is determined by analyzing the ingress and egress traffic separately. Both the ingress and egress traffic are further resolved into committed external bandwidth (associated with CBR streaming), best effort bandwidth, and internal CServer traffic.

The bandwidth usage of each component is calculated as a 15 minute moving average of 90 samples (or the number available, if less than 90) taken at 10-second intervals. The sum of the committed external bandwidth usage and the best effort bandwidth usage is measured against the total capacity, and a status is assigned to both the ingress and the egress traffic. The more severe of the two statuses is reported as the bandwidth status.

Alarms reporting in the GUI is not supported in this release.

COS Node Status MonitoringA COS node is in service if the associated service instance and asset workflow template are enabled. The status of each node that is in service and part of a COS cluster is displayed in the COS Service Manager GUI. This status is updated once every 30 seconds.

The COS AIC client on each COS node posts a document containing the status information for the node to the DocServer once every 10 seconds.

The COS AIC, once every 30 seconds, parses the data in this document and renders the data in a format suitable for display in the COS Service Manager GUI. If the AIC cannot find status information for a node, it waits for 30 seconds. If the information is not received within this waiting period, AIC reports that the node is not functional.

The status of each node reflects the status of the node’s network interfaces, disks, and services.

• If either all the interfaces of the node are not functional, the node status is reported as critical.

• If all the disks of the node are not functional, the node status is reported as critical.

• If a service is down, the node status is reported as critical.

The issue is described in detail in the Fault Detail field.

Viewing COS Node StatusTo view the status of a COS node using the COS Service Manager GUI, do the following:


Step 2 Go to Services > Cloud Object Storage.

Step 3 Click on the Overview tab.


Chapter 3 System Monitoring COS Management Events

AIC Client MonitoringThe COS AIC client running on a COS node periodically monitors the disks, interfaces, and services (processes) of that node and posts the data to the DocServer as a COS-specific document.

The AIC client begins the monitoring activity when a node is configured and added to a COS cluster. As long as the node is running and is part of a COS cluster, monitoring occurs once every 10 seconds.

Storage Monitoring

The AIC client can monitor and report storage (disk) state and statistics only if the CServer is running on the node.

The following information is reported for each disk:

• Disk name

• Bytes read

• Bytes written

• Requests

• State

• S.M.A.R.T. Status

The client also reports the total storage space on all disks and the total storage space currently in use.

Interface Monitoring

For each interface, the AIC client reports the interface state and the transmit and receive statistics. The client can monitor and report the state and statistics of CServer interfaces only when the CServer is running on the node.

Services Monitoring

The AIC client monitors the following services:

• Cisco Cache Server (CServer)

• Cisco Cloud Object Storage Daemon (cosd)

• Cassandra Server

• NTP Daemon

• SNMP Daemon

COS Management EventsThe COS management events indicate the current state of COS services and the health of COS nodes.

Viewing COS Node Management EventsThe management events for a COS node are displayed in the COS Service Manager GUI.



To see the management events for a COS node, follow these steps:


Step 2 Go to Monitoring > Events.

All the events for the node are listed with the oldest event first.

Events belong to one of the following levels of severity:

• Info – The event represents information only and does not require operator intervention.

• Warning – The event represents an issue that is possibly transitory and the operator should investigate the cause.

• Critical - The event represents an issue from which the node may not recover without operator intervention, and the operator must act immediately because the issue may cause service outage.

Service EventsThe COS service events are listed in the table below.

Table 3-1 COS Service Events

Event Name Description Severity Event TypeEvent Subtype

Service.Activated COS service activated by enabling Service Instance

Info Config Config

Service.Deactivated COS service deactivated by disabling Service Instance

Critical Config Config

Service.Deactivated COS Service Instance deleted from DocServer


Service.Activated COS service activated by enabling Asset Workflow Template

Info Config Config

Service.Modified COS service modified — Asset Redundancy Policy Reference changed in AWT

Info Config Config

Service.Deactivated Asset Workflow Template disabled or deleted from DocServer

Info Config Config

ServiceInstance.NotFound

COS AIC could not fetch Service Instance object from DocServer


CaptureEndpoint.NotFound

COS AIC could not fetch Capture Endpoint object from DocServer


COSNodeCluster.NotFound

COS AIC could not fetch Node Cluster object from DocServer


AssetWorkflowTemplate.NotFound

COS AIC could not fetch Asset Workflow Template object from DocServer


COSNode.Down COS node has not reported status for last 30 seconds

Critical Node Accessibility



COS AIC Client EventsCOS AIC client generates events pertaining to storage (disk), network (interface), and service (process) on each node. These events are generated only if the AIC client monitoring is enabled. For more information on this monitoring activity, see “AIC Client Monitoring”.

The events generated by a COS AIC client are listed in the table below.

• An event is generated for every change in the state of operation of a disk, interface, or process.

• Although events are generated only on a state change, monitoring is done every 10 seconds, so events can be generated as often as every 10 seconds.

Service.StorageStatus SLA status for storage is warning Warning Application SLA

Service.StorageStatus SLA Status for Storage is critical Critical Application SLA

Service.SessionStatus SLA status for sessions is warning Warning Application SLA

Service.SessionStatus SLA status for sessions is critical Critical Application SLA

Service.BandwidthStatus SLA status for sessions is warning Warning Application SLA

Service.BandwidthStatus SLA status for sessions is critical Critical Application SLA

COS-AIC Started COS AIC has started Info Application Health

COS-AIC Terminated COS AIC is exiting Critical Application Health

COS-AIC Terminated COS AIC is exiting after receiving SIGINT

Critical Application Health

COS-AIC Terminated COS AIC is exiting after receiving SIGTERM


COS-AIC Terminated An uncaught exception was found; the program will end


Table 3-1 COS Service Events

Event Name Description Severity Event TypeEvent Subtype

Table 3-2 COS AIC Client Events

Event Name Description Severity Event Type Event Subtype

CosNodeDiskDown Disk disk_name down Warning COS-Node Health

CosNodeDiskUp Disk disk_name up Info COS-Node Health

CosNodeInterfaceDown Interface if_name down Warning COS-Node Health

CosNodeInterfaceUp Interface if_name up Info COS-Node Health

CosNodeServiceDown Service svc_name down Warning COS-Node Health

CosNodeServiceUp Service svc_name up Info COS-Node Health


Chapter 3 System Monitoring COS Node Platform Monitoring with SNMP

COS AIC Server EventsThe COS AIC server generates the events listed in the table below.

COS Node Platform Monitoring with SNMP

OverviewEach COS node provide hardware statistics, events, and alerts using Simple Network Management Protocol (SNMP). In addition to hardware platform information, some basic state for the COS node disk and file system usage is also available with SNMP.

Each COS node executes an instance of the Net-SNMP snmpd service as an SNMP agent. An instance of the Net-SNMP snmptrapd service also executes on each COS node, and can be customized for environment-specific configuration for the handling of traps and notifications generated by the snmpd service.

For the CDE-460 and CDE470 hardware platforms, Super Micro Computer’s SuperDoctor 5 service is leveraged to provide hardware status and monitoring. The SuperDoctor 5 utility integrates with SNMP and provides platform information through SNMP Management Information Bases (MIBs) defined by Super Micro Computer.

InstallationThe SNMP services for COS nodes are provided by the following RPMs:

• net-snmp

• net-snmp-libs

• net-snmp-utils

• SuperDoctor

• cos_snmp

These RPMs come pre-installed on each COS node, and are preconfigured with a basic configuration for read-only access to the SNMP service.

Table 3-3 COS AIC Server Events

Event Name Description Severity Event Type Event Subtype

AddCosNode New COS node added Info COS-Node Accessibility

DeleteCosNode COS node deleted Warning COS-Node Accessibility

CosNodeInterfaceError No available IP addresses in IP pool

Critical COS-Node Accessibility

CosUpdatedActiveIpPool

An active IP pool was edited

Critical COS-Node Accessibility



ConfigurationBasic configuration for the Net-SNMP services (snmpd and snmptrapd) and the SuperDoctor 5 utility is provided by the cos_snmp RPM. The snmpd and snmptrapd service configuration can be customized to accommodate customer-specific environments by manually editing the /etc/snmp/snmpd.conf and /etc/snmp/snmptrapd.conf configuration files, respectively.

Note Customization made to these files will be replaced with an update to the cos_snmp RPM. The cos_snmp RPM will back up configuration files in the /etc/snmp directory upon upgrade.

MIB ExtensionsNet-SNMP is distributed with support for various MIBs that provide OID instances for some hardware and Linux platform information. The definitions for MIBs distributed with Net-SNMP service are stored in /usr/share/snmp/mibs on each COS node. For more information about the standard MIBs provided by Net-SNMP, see:

http://www.net-snmp.org/docs/mibs/

Definitions for MIBs distributed with the SuperDoctor 5 service are stored in /opt/Supermicro/SuperDoctor5/mibs on each COS node. For more information about the SuperDoctor 5 service, see:

http://www.supermicro.com/solutions/SMS_SD5.cfm

Extensions have been added to Net-SNMP to add CServer disk and storage information to the following tables:

• HOST-RESOURCES-MIB::hrDeviceTable

• HOST-RESOURCES-MIB::hrDiskStorageTable

• HOST-RESOURCES-MIB::hrStorageTable

• UCD-SNMP-MIB::dskTable

Extensions to HOST-RESOURCES-MIB::hrDeviceTable

This table has been extended to include status for the external facing disk drives that are owned by CServer. The columns of primary interest are hrDeviceDescr and hrDeviceStatus. The combination of these two columns identify a disk device and its current operating state.

The following is a snip of the table showing the extension:

[root@utah97 ~]# snmptable -v2c -cpublic -M/opt/Supermicro/SuperDoctor5/mibs:/usr/share/snmp/mibs -mall localhost HOST-RESOURCES-MIB::hrDeviceTableSNMP table: HOST-RESOURCES-MIB::hrDeviceTable

hrDeviceIndex hrDeviceType hrDeviceDescr hrDeviceID hrDeviceStatus hrDeviceErrors

... 7680 HOST-RESOURCES-TYPES::hrDeviceDiskStorage Cisco csd1 SAS HDD SNMPv2-SMI::zeroDotZero running 0 7681 HOST-RESOURCES-TYPES::hrDeviceDiskStorage Cisco csd2 SAS HDD SNMPv2-SMI::zeroDotZero running 0



... 7751 HOST-RESOURCES-TYPES::hrDeviceDiskStorage Cisco csd72 SAS HDD SNMPv2-SMI::zeroDotZero running 0

According to the HOST-RESOURCES-MIB definition, valid values of hrDeviceStatus are unknown, running, warning, testing, and down.

[root@utah97 ~]# snmptranslate HOST-RESOURCES-MIB::hrDeviceStatus -Tp +-- -R-- EnumVal hrDeviceStatus(5) Values: unknown(1), running(2), warning(3), testing(4), down(5)

The SNMP device status is mapped from output of the cddm utility. For more information on cddm, see the cddm man page on any COS node with man cddm.

The mapping of cddm output to SNMP status is as follows:

• cddm state contains DEV_READY and cddm smart_status is OK => SNMP running

• cddm state contains DEV_READY and cddm smart_status is ADVISORY => SNMP warning

• cddm state contains (DEV_SUSPENDING or DEV_SUSPENDED or DEV_TRACK_MAP or DEV_USER_PREP or DEV_ABANDONED) => SNMP warning

• cddm state contains (DEV_LOG_REMOVED|DEV_SICK|DEV_REMOVED) => SNMP down

All other cddm status are mapped to an SNMP unknown state.

Extensions to HOST-RESOURCES-MIB::hrDiskStorageTable

This table has been extended to show disk capacity for disk drives used by CServer for storing of object data.

The following is a snip of the table showing the extension:

[root@utah97 ~]# snmptable -v2c -cpublic -M/opt/Supermicro/SuperDoctor5/mibs:/usr/share/snmp/mibs -mall localhost HOST-RESOURCES-MIB::hrDiskStorageTableSNMP table: HOST-RESOURCES-MIB::hrDiskStorageTable

hrDiskStorageAccess hrDiskStorageMedia hrDiskStorageRemoveble hrDiskStorageCapacity readWrite unknown false 194335744 KBytes readWrite hardDisk true 3907018583 KBytes

... readWrite hardDisk true 3907018583 KBytes readWrite hardDisk true 3907018583 KBytes

Viewing the table in this manner does not easily correlate to a particular device, because the table does not define a device description. You must manually correlate the entries from the HOST-RESOURCES-MIB::hrDiskStorageTable and HOST-RESOURCES-MIB::hrDeviceTable tables.

Extensions to HOST-RESOURCES-MIB::hrStorageTable

This table has been extended to provide information about the CServer file system that includes the file system storage totals and used.

[root@utah97 ~]# snmptable -v2c -cpublic -M/opt/Supermicro/SuperDoctor5/mibs:/usr/share/snmp/mibs -mall localhost HOST-RESOURCES-MIB::hrStorageTableSNMP table: HOST-RESOURCES-MIB::hrStorageTable

hrStorageIndex hrStorageType hrStorageDescr hrStorageAllocationUnits hrStorageSize hrStorageUsed hrStorageAllocationFailures



1 HOST-RESOURCES-TYPES::hrStorageRam Physical memory 1024 Bytes 264524232 246219724 ? 3 HOST-RESOURCES-TYPES::hrStorageVirtualMemory Virtual memory 1024 Bytes 268620224 246219724 ?

... 7680 HOST-RESOURCES-TYPES::hrStorageRemovableDisk Cisco CServer Storage 2097152 Bytes 135448398 179575 ?

The file system byte size and bytes used can be obtained by multiplying the hrStorageAllocationUnits by hrStorageSize and hrStorageUsed respectively.

Extensions to UCD-SNMP-MIB::dskTable

As with HOST-RESOURCES-MIB::hrStorageTable, this table has been extended to include CServer file system information. Along with size (total) and used, this table includes an available count as well as a percentage used.

[root@utah97 ~]# snmptable -v2c -cpublic -M/opt/Supermicro/SuperDoctor5/mibs:/usr/share/snmp/mibs -mall localhost UCD-SNMP-MIB::dskTableSNMP table: UCD-SNMP-MIB::dskTable

dskIndex dskPath dskDevice dskMinimum dskMinPercent dskTotal dskAvail dskUsed dskPercent dskPercentNode dskTotalLow dskTotalHigh dskAvailLow dskAvailHigh dskUsedLow dskUsedHigh dskErrorFlag dskErrorMsg 1 / /dev/sda8 100000 -1 25728092 21299064 3122100 13 2 25728092 0 21299064 0 3122100 0 noError 2 /arroyo/db /dev/sda2 100000 -1 70555056 66755080 215976 0 0 70555056 0 66755080 0 215976 0 noError

... 7680 Cisco CServer Storage -1 -1 ? ? ? 0 ? 2520412160 64 2152642560 64 367769600 0 noError

This table shows sizes in kilobyte units. The SNMP values used to store the size are 32-bit values, and can only represent file systems less than 2 TB in size in a single value. To accommodate file systems of 2 TB or larger, this table has Low and High values that combine to hold a 64-bit value. In the example output above, the total byte size would be computed as:

((64 * 2^32) + 2520412160) * 1024 bytes = 284,055,878,762,496 bytes

Monitored ItemsThe default Net-SNMP configuration in /etc/snmp/snmpd.conf includes directives to monitor several items on the host COS node. This includes monitoring for a running cosd service, memory errors, Linux disk errors, and the Linux load average.

The default snmpd configuration has also been extended to include monitor directives to alert in the event of a CServer disk device status change, as well as to alert on overall CServer percentage of file system usage.

In the event that a disk status changes or file system usage percentage changes above or below certain thresholds, Net-SNMP generates a local trap that is received by the local snmptrapd process. The default behavior of snmptrapd is to log the event to /var/log/messages.



Disk device status change monitors are defined as follows:

monitor -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in unknown state" hrDeviceStatus == 1monitor -S -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in running state" hrDeviceStatus == 2monitor -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in warning state" hrDeviceStatus == 3monitor -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in testing state" hrDeviceStatus == 4monitor -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in down state" hrDeviceStatus == 5

Net-SNMP monitors for disk status changes every 60 seconds with this configuration. The following is an example trap generated by Net-SNMP for a disk device with warning status:

Sep 10 12:15:59 utah97 snmptrapd[23614]: 2014-09-10 12:15:59 localhost.localdomain [UDP: [127.0.0.1]:45956->[127.0.0.1]]:Sep 10 12:15:59 utah97 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (131) 0:00:01.31 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: Cisco device in warning state DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: HOST-RESOURCES-MIB::hrDeviceStatus.7698 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 3 HOST-RESOURCES-MIB::hrDeviceIndex.7698 = INTEGER: 7698 HOST-RESOURCES-MIB::hrDeviceDescr.7698 = STRING: Cisco csd19 SAS HDD HOST-RESOURCES-MIB::hrDeviceStatus.7698 = INTEGER: warning(3)

Further details for device status can be obtained from the cddm utility. In this particular case, the device is in warning status due to an AVISORY S.M.A.R.T. status:

[root@utah97 ~]# cddm -s all 19--- csd19 ---alleged_media_errors 0bytes_read 3363291648bytes_written 31124533760companion 20connection 0.19device_type SAS HDD dev_link_rate 6000dev_max_operating_temp 69dev_max_operational_starts 1048576dev_mfg_wk 01dev_mfg_yr 2012dev_temp 40direct_submits 66679dubious_LBAs 0errors 0errors_reported 0errors_reported_rate 0hard_resets 0location 1.2.3.1media_error_rate 0media_errors 0model WD4001FYYG-01SL3name csd19proc_flags 0x20reqs_in_cb_queue 0reqs_in_progress 0reqs_lost 0reqs_queued 0requests 126882reset_rate 0resets 0



retries 0rev VR02sector_size 512serial WMC1F0762907slot 10smart_age 00.26.11smart_exceptions 0x5d.0x32.0x28 - Impending - data error rate too high; smart_glist_count 11smart_nonmedium_errors 0smart_rd_corrected_errors_long 0smart_rd_corrected_errors_short 0smart_rd_correction_algorithm_use 0smart_rd_retries 0smart_rd_total_bytes_processed 0smart_rd_total_corrected_errors 0smart_rd_uncorrected_errors 0smart_startups 10smart_status ADVISORYsmart_wr_corrected_errors_long 0smart_wr_corrected_errors_short 0smart_wr_correction_algorithm_use 0smart_wr_retries 0smart_wr_total_bytes_processed 0smart_wr_total_corrected_errors 0smart_wr_uncorrected_errors 0state 0x800007; DEV_ALLOCATED DEV_ATTACHED DEV_READY timeout_rate 0timeouts 0total_sectors 7814037167vendor WD

CServer file system usage monitors are defined as follows:

monitor -I -r 60 -o UCD-SNMP-MIB::dskPath.7680 -o UCD-SNMP-MIB::dskPercent.7680 "Cisco CServer Storage High" UCD-SNMP-MIB::dskPercent.7680 >= 98monitor -S -I -r 60 -o UCD-SNMP-MIB::dskPath.7680 -o UCD-SNMP-MIB::dskPercent.7680 "Cisco CServer Storage Normal" UCD-SNMP-MIB::dskPercent.7680 < 98

Net-SNMP monitors for high-usage events if the percentage used is 98 percent or above. Net-SNMP monitors for normal usage events if the percentage used falls below 98 percent.

The following is an example trap logged to /var/log/messages for high and normal usage status:

Sep 10 13:28:27 utah97 snmptrapd[23614]: 2014-09-10 13:28:27 localhost.localdomain [UDP: [127.0.0.1]:46969->[127.0.0.1]]:Sep 10 13:28:27 utah97 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (6137) 0:01:01.37 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: Cisco CServer Storage High DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: UCD-SNMP-MIB::dskPercent.7680 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 98 UCD-SNMP-MIB::dskPath.7680 = STRING: UCD-SNMP-MIB::dskPercent.7680 = INTEGER: 98

Sep 10 13:30:27 utah97 snmptrapd[23614]: 2014-09-10 13:30:27 localhost.localdomain [UDP: [127.0.0.1]:46969->[127.0.0.1]]:Sep 10 13:30:27 utah97 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (18138) 0:03:01.38 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: Cisco CServer Storage Normal DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: UCD-SNMP-MIB::dskPercent.7680 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 97 UCD-SNMP-MIB::dskPath.7680 = STRING: UCD-SNMP-MIB::dskPercent.7680 = INTEGER: 97



Manual customizations can be made to the snmpd and snmptrapd services to monitor additional items, and to forward traps and notifications to customer-specific operations.


Cisco Cl

A

P P E N D I X A
Reference Information
This section contains additional reference material for further understanding the COS system, and information on performing commonly executed tasks and system maintenance.

COS Service ModelThe COS service model is shown in the Unified Modeling Language (UML) diagram below:

Figure A-1 COS Service Model

A COS operator can assume one of the following roles:

• System Operator

• Tenant

System Operator

Resources

Network Service Instance

EndPointAssetWorkflow

Template

Tenant

Policy

COS Cluster

IP PoolLogs

AuthProfile

SLA

ResourceSLA

CapacitySLA

Capture

ContentRedundancy

Use API

Storage

Status

Statistics

Events

Alarms

Diagnostics

DNS

NTP

COS Node

COS Cluster

COS NodeRegion

AuthorizationProfile

Zone

HighAvailability

*

*

*

*-Owns

-Owns

-creates

* -creates

*

* -creates

1

1

1

1

1

1

1

1

1*

1*

3830

81

1..*

A-1oud Object Storage Release 2.1.2 User Guide

Appendix A Reference Information Using the COS Service Manager GUI

The System Operator provisions the system resources (marked in yellow in the diagram above) and creates the Tenant.

The entities managed by the Tenant are marked in green in the diagram above.

The tenant creates service instances using pre-existing service templates. To provision a service instance, the tenant assigns system resources to the service instance and configures it. When the service instance is activated, all configurations related to the service are performed and the service becomes available.

When activated, the service instance also produces certain outputs that are marked in blue in the diagram above.

Using the COS Service Manager GUIThe COS Service Manager GUI enables you to quickly and easily access many COS deployment and monitoring functions. This section provides basic login and navigation instructions for the GUI.

Steps for using the GUI to perform specific deployment and monitoring tasks are provided in chapters devoted to these topics.

Note The COS Service Manager GUI is described here as it appears in a COS-only deployment. When COS is deployed as a service of MOS, its GUI is presented as a subset of the MOS Service Manager GUI, and some navigational details differ accordingly.

Before You BeginPlease note the following before logging in and using the COS Service Manager GUI:

• The COS Service Manager GUI runs on the following operating systems and browsers:

– Windows Internet Explorer 9 (IE9) or later for Windows 7

– Mozilla Firefox 20 or later for Windows 7

– Google Chrome 30.x for Windows 7

– Apple Safari 7.x for Widows 7 on MAC OS Version 10.9 or later

• The GUI requires a display resolution of 1600 x 900 or better. If the PAM IP address on which you access the GUI is NATed, response time can be noticeably slow.

• The GUI times out after one our (60 minutes) of activity. To avoid timeouts, make and save a configuration change, or click a tab or refresh icon, before the one-hour idle timeout expires.

• Configuration changes that you make in any section of the GUI are saved when you click the Save button. If you log out of the GUI or if the GUI times out before you save your changes, all changes made since your last save are lost.

• If you try to delete an object in the GUI that has been associated with another object (for example, a capture endpoint that has been associated with an asset workflow), the GUI prevents the deletion and displays an explanatory error message.

• If you redeploy or upgrade the COS OVA, you must clear your browser cache for the GUI changes to take effect.

A-2Cisco Cloud Object Storage Release 2.1.2 User Guide


Logging InTo access the COS Service Manager GUI, perform the following steps:

Step 1 Access the GUI using the PAM IP address. The URL format is:

https://pam_ip_address:8443

Step 2 Log in to the GUI with your Username and Password.

The default username is admin. The default password is default. We recommend that you change these after deployment is complete.

Figure A-2 COS Service Manager GUI Login Page

The home page of the GUI appears, and you can now begin using the GUI.

Step 3 To exit the GUI from any page, click the Username displayed at upper right in the GUI and choose Log out from the drop-down menu.

Using the Home PageThe Home page appears by default, and provides a high-level overview information for COS. The Home page displays the following information, which is refreshed every 5 minutes.



Figure A-3 COS Service Manager GUI Home Page

Nodes Allocated

The Nodes Allocated section of the Home page displays COS node usage broken down by service type (such as Live or VOD), available nodes, and nodes that are in maintenance mode. You can click on sections of the chart to navigate to pages containing more details.

Figure A-4 Home Page - Nodes Allocated Section

Icons at the bottom of this section let you choose between chart view and table view, choose different chart types, and export the associated data.



Alarm Summary

The Alarms Summary section of the Home page displays the number of alarms logged for each service type, further broken down by severity of the alarms (critical, major, or minor).

Figure A-5 Home Page - Alarm Summary Section

Icons at the bottom of this section let you choose between chart view and table view, choose different chart types, and export the associated data.

Service Summary

Located below these two panes is a Service Summary table that lists all of the service instances currently defined for the deployment.

Figure A-6 Home Page - Service Summary Section

The Service Summary section displays the following information for each COS node (you cannot edit these fields):

• Service Type — Type of service instance, such as Live or VOD.

• Service Instance Name — Name of the service instance.

• Service State — Indicates whether the service instance is currently Enabled or Disabled.

• Nodes Allocated — Number of nodes allocated for use by the service instance.

• Nodes Used — Number of nodes currently being used by the service instance.

• Node Status — Current status of the node associated with the service instance.

• Storage Status — Current status of the storage associated with the service instance.



• Total Alarms — Total number of alarms logged for the service instance.

• Critical — Number of Critical alarms logged for the service instance.

• Major — Number of Major alarms logged for the service instance.

• Warning — Number of Warning alarms logged for the service instance.

To display the endpoint summary for a service instance, click its Expand arrow.

Figure A-7 Home Page - Endpoint Summary Section

The Endpoint Summary table displays the following information for each endpoint (you cannot edit these fields):

• Endpoint Name — Name of the endpoint.

• Endpoint Type — Type of endpoint. Valid values are Capture and Playback.

• Nodes Allocated — Number of nodes allocated for use by the service instance.

• Nodes Used — Number of nodes currently being used by the service instance.

• Node Status — Current status of the node associated with the service instance.

• Storage Status — Current status of the storage associated with the service instance.

• Total Alarms — Total number of alarms logged for the service instance.

• Critical — Number of Critical alarms logged for the service instance.

• Major — Number of Major alarms logged for the service instance.

• Warning — Number of Warning alarms logged for the service instance.

To collapse the interface information, click the Collapse arrow.

Note Clicking a Service Instance Name takes you directly to a page with more detailed information.



Using the Navigation PanelClicking the Navigation button at upper left in the GUI opens the Navigation panel.

Figure A-8 Home Page - Navigation Panel

Selecting the View

Use the icons at the top of the panel let you toggle between three views:

• The Navigation menu, open by default

• The Index, which includes a Search function

• The Favorites list

Bookmarking a Page

You can bookmark a page of interest and add it to the Favorites list. To do this, navigate to the page and click the Pin icon located to the right of the Favorites button.

Home Button

Click the Home button to return to the COS Service Manager GUI Home page from any other page. This also closes the Navigation panel.



Infrastructure

Click the Infrastructure button to open (or if open, to close) a list of pages for viewing and configuring various infrastructure components.

• Choose Storage > COS Nodes to configure COS nodes.

• Choose Storage > COS Clusters to configure clusters.

• Choose IP Pools and Networks to configure IP pools.

• Choose Platform Services to set up, edit, or view server information.

• Choose Regions and Zones to display and configure regions and zones.

Note Step-by-step instructions related to all of the above tasks are provided in Chapter 2, “Deploying COS”.

Service Domain Objects

To configure COS service domain objects, select Service Domain Objects, and then select one of the following:

• Choose Profiles > Auth Profiles to configure authorization profiles.

• Choose Policies > Asset Redundancy Policies to configure asset redundancy policies.

Services

To configure COS service domain objects, select Services, and then choose Services > Cloud Object Store to configure the COS service.

Note After choosing Services > Cloud Object Store, an Overview tab appears that lists all Cache Interfaces. If an interface is in Active status, the associated port is ready for a connection. However, if there are any issues with the port, they are listed on the Fault Detail tab only.

Note Additional menu options such as Services > Unified Media Service indicate a COS + MOS deployment. See the Cisco Media Origination System Release 2.3 User Guide for instructions on using these options.

Monitoring

To access COS monitoring features, select Monitoring, and then select one of the following

• Choose Alarms > Active to access a list of currently active alarms.

• Choose Alarms > History to access a list of all alarms, both active and inactive.

• Choose Diagnostic Settings > System Diagnostics to access system diagnostics.

• Choose Diagnostic Settings > Service Diagnostics to access service diagnostics.

• Choose Events to access a list of significant system events.

• Choose Configuration Viewer to access an interactive and searchable view of network objects, relationships, and configurations. Choose between tree view and graph view, then click any object to view its configuration.



Note Step-by-step instructions related to all of the above tasks are provided in Chapter 3, “System Monitoring”.

Working with TablesMost of the pages that make up the COS Service Manager GUI present information in tables. A toolbar at the top of each table provides common features that let you quickly perform needed functions.

Figure A-9 GUI Table Tools

On most pages, you can:

• Add, select and edit, or select and delete table entries.

• Sort entries by table column

• Change the order of the columns.

• Use filters to control what is displayed.

This section describes each of these operations.

Adding an Entry

To add an entry to a table, click Add Row at the top of the table. Enter or select values for the new entry, and then click Save to add it to the table.



Editing an Entry

To edit an entry in a table, use one of the following methods:

• Select it and click Edit at the top of the table.

• Click in any of the fields for the entry.

• Enter or select new values for the entry, then click Save to save the new values in the table.

Deleting an Entry

To delete an entry form a table, select the entry and click delete at the top of the table. When prompted, click OK. The entry is removed from the table.

Refreshing the Table

To refresh the entries in a table, click Refresh at the top of the table.

Sorting Entries

To sort entries in a table based on the contents of a column, click the column header. Click once to order the table in alphanumeric ascending order. Click again for descending order.

Changing the Order of Columns

To change the order of columns in a table, drag-and-drop the column headings.

Filtering Contents

To filter the contents of a table, choose a filter from the drop-down list at top right of the table, or click the Filter icon to toggle between different filters.

• All — Display all of the table contents. This is the default display.

• Quick Filter — Display only those elements of one or more columns that contain specified text strings.

• Advanced Filter — Display only those elements of the tale that match a set of one or more rules that you specify.

For each rule, specify a column; a type of filter, such as Contains or Starts with; and a text string.

To add another rule to the filter, click the plus sign (+). You can specify more than one rule for a given column.

If you have specified more than one rule, the Match drop-down list appears.

To specify that an element of the table must match all of the specified rules in order to be included in the list, select Match > All.

– To specify that an element of the table need only match one or more of the specified rules in order to be included in the list, select Match > Any.

– To delete a rule from the filter, click the minus sign (-).

– To apply the filter to the elements of the table, click Go.



– To clear the filter and display all elements of the table, click Clear Filter.

• Manage Preset Filters — Edit or remove Advanced Filters that you have saved.

Note When you refresh the pages in your browser, all saved filters are removed.

Managing GUI UsersCOS stores the usernames and passwords for the GUI in the file etc/opt/cisco/cos/public/users.properties stored on the PAM VM.

Note We recommend that you change the default username and password following installation.

COS provides the following scripts to help you manage usernames and passwords:

• To list current users, enter the following command:

listusers-gui

• To add a new user, enter the following command:

adduser-gui username password

• To change an existing user, enter the following command:

chpass-gui username password

• To delete a user, enter the following command:

deluser-gui username

You can enter multiple commands when adding, changing, or deleting multiple users or passwords. For example, if you are adding two users, changing two users, and deleting one user, you can enter the following sequence of commands:

adduser-gui username1 password1adduser-gui username 2 password2chpass-gui username3 password3chpass-gui username4 password4deluser-gui username5

Usernames and passwords for the COS GUI:

• Can be strings of any length.

• Can include uppercase or lowercase letters, numbers, and any special characters. However, the characters <, >, |, !, and & must be preceded by a backslash.

• Are not case-sensitive.


Appendix A Reference Information COS Network Ports and Services

COS Network Ports and ServicesThe following table identifies open network ports for COS nodes and the services that own these ports.

* The sd5 service chooses a single available port in the range of 31000-32999 on startup.

COS MaintenanceIt may be necessary to reboot or shut down a COS node for conditions such as routine maintenance. Care should be taken in the process to avoid impact to the services provided by the COS cluster.

COS Node RebootTo reboot a COS node, execute the reboot command from a terminal console or remote shell. The system will begin a shutdown phase to reboot.

Table A-1 COS Network Ports and Services

Scope/Interface Port Purpose Owning Service

Management TCP 7000 Cassandra internode communication cassandra

Management TCP 7199 Cassandra JMX communication cassandra

Management TCP 9042 Cassandra CQL native transport port cassandra

Management TCP 9160 Cassandra Thrift client API cassandra

Management TCP 9090 Cosd request listener cosd

Data TCP 80 HTTP traffic for Swift and Swauth interfaces

cserver

Data UDP 3478 STUN traffic cserver

Data UDP 48879 Internal COS node communication cserver

Data UDP 57005 Internal COS node communication cserver

Management UDP 123 Network Time Protocol ntpd

Management TCP 25 Postfix mail system traffic postfix

Management TCP 5666 SuperDoctor5 Nagios Remote Plugin Extender (NRPE)

sd5

Management TCP 5999 SuperDoctor5 Nagios Remote Plugin Extender (NRPE)

sd5

Local TCP 31000-32999 * SuperDoctor5 internal communication sd5

Management UDP 7777 SuperDoctor5 internal communication sd5

Local TCP 199 Simple Network Management Protocol snmpd

Management UDP 161 Simple Network Management Protocol snmpd

Management UDP 162 SNMP Traps snmpdtrapd

Management TCP 22 Secure Shell (SSH) ssh


Appendix A Reference Information COS Maintenance

• Any active HTTP and TCP sessions with the COS node data network interfaces will be reset by the node. The client will have the responsibility of retrying operations with the remaining COS nodes in the cluster.

• The COS management system will automatically update the DNS registry to remove listings for the COS node.

• The COS services will automatically be restarted when the system is back online after reboot.

• If the COS cluster has been configured to replica data, copies of any object data residing on the COS node will be accessible from the remaining COS nodes.

Behavior of COS Services on COS Node BootThere are four primary system services that provide COS functionality on a COS node: cassandra, cosd, cos_aicc, and cserver. These services can be manipulated using standard Linux system service tools.

To prevent these services from starting automatically on a COS node boot, execute the following commands as the root user from a shell on that node:

[root@cos-node-1 ~]# chkconfig cassandra off[root@cos-node-1 ~]# chkconfig cosd off[root@cos-node-1 ~]# chkconfig cos_aicc off[root@cos-node-1 ~]# chkconfig cserver off

To enable automatic service loading on node boot, execute the following commands:

[root@cos-node-1 ~]# chkconfig cassandra on[root@cos-node-1 ~]# chkconfig cosd on[root@cos-node-1 ~]# chkconfig cos_aicc on[root@cos-node-1 ~]# chkconfig cserver on

To view the current state of service loading, execute the following command:

[root@cos-node-1 ~]# chkconfig --listauditd 0:off 1:off 2:on 3:on 4:on 5:on 6:offcos_aicc 0:off 1:off 2:on 3:on 4:on 5:on 6:offcosd 0:off 1:off 2:on 3:on 4:on 5:on 6:offcassandra 0:off 1:off 2:on 3:on 4:on 5:on 6:offcrond 0:off 1:off 2:on 3:on 4:on 5:on 6:offcserver 0:off 1:off 2:on 3:on 4:on 5:on 6:offip6tables 0:off 1:off 2:on 3:on 4:on 5:on 6:offiptables 0:off 1:off 2:on 3:on 4:on 5:on 6:offnetconsole 0:off 1:off 2:off 3:off 4:off 5:off 6:offnetfs 0:off 1:off 2:off 3:on 4:on 5:on 6:offnetwork 0:off 1:off 2:on 3:on 4:on 5:on 6:offntpd 0:off 1:off 2:off 3:off 4:off 5:off 6:offntpdate 0:off 1:off 2:off 3:off 4:off 5:off 6:offpostfix 0:off 1:off 2:on 3:on 4:on 5:on 6:offrdisc 0:off 1:off 2:off 3:off 4:off 5:off 6:offrestorecond 0:off 1:off 2:off 3:off 4:off 5:off 6:offrsyslog 0:off 1:off 2:off 3:off 4:off 5:off 6:offsaslauthd 0:off 1:off 2:off 3:off 4:off 5:off 6:offsshd 0:off 1:off 2:on 3:on 4:on 5:on 6:offsyslog-ng 0:off 1:off 2:on 3:on 4:on 5:on 6:offudev-post 0:off 1:on 2:on 3:on 4:on 5:on 6:off

To manually start the services, execute the following commands:

[root@cos-node-1 ~]# service cassandra start[root@cos-node-1 ~]# service cosd start[root@cos-node-1 ~]# service cos_aicc start[root@cos-node-1 ~]# service cserver start


Appendix A Reference Information COS Service Reliability

COS Service ReliabilityThis section describes the response of the COS AIC server to changes in the states of disks, interfaces, and services on COS nodes. The COS AIC client and the Service Monitor convey the changes in state by sending appropriate events to the COS AIC server.

If a change is critical and indicates that the node cannot service requests, the AIC server ensures that the node’s interfaces are not part of the DNS, so that service requests are not addressed to the node.

COS Node DisksWhen the COS AIC server receives an event indicating the failure of all the disks on a COS node, it checks the status of the node in its records.

• If the node status is down, the server removes the node’s interfaces from the DNS server.

• If the node status is up, no action is necessary.

When the COS AIC server receives an event indicating some or all of the disks on the COS node are functional, it checks the status of the node in its records.

• If the node status is down, the server adds the node’s interfaces back to the DNS server.


COS ServicesWhen the COS AIC server receives an event indicating that one or more of the COS services — Cisco Cache Server (CServer), Cisco Cloud Object Storage Daemon (cosd), and Cassandra Server — are not functional, it checks the status of the node in its records.



When the COS AIC server receives an event indicating all the COS services are operational, it checks the status of the node in its records.



COS Node Interfaces When the COS AIC server receives an event indicating that one or more interfaces are not functional, along with a list of the faulty interfaces, it checks the status of these interfaces in its records.

• If the fault interfaces are part of the DNS, they are removed from it.

• If an interface is not in the list of faulty interfaces, and is not part of the DNS, the AIC server adds the interface to the DNS server.


Appendix A Reference Information COS Node Hard Drive Replacement

Server ReachabilityThe Service Monitor periodically receives the status of a COS node from the COS AIC client on that node. If the Service Monitor does not receive status information for a COS node within the stipulated time, it determines that the node is not reachable. Consequently, it sends an event to the COS AIC server indicating that the node is down. In response, the server checks the status of the node in its records.



When the Service Monitor receives status information for a COS node, indicating that the node is reachable, it sends an event to the COS AIC server conveying that the node is up. In response, the server checks the status of the node in its records.



COS Node Hard Drive ReplacementCOS software can be deployed currently on the Cisco Content Delivery Engine (CDE) platforms CDE 460 and CDE 470. For information about replacing the hard drives of the CDE 460 and CDE 470 platforms, see the section Replacing External Storage Drives of the chapter Installing the Content Delivery Engines in the Cisco Content Delivery Engine 205/220/250/420/460/470 Hardware Installation Guide.

Replicating Objects During Swift Write OperationsWhile an object is being created or modified using Swift write operations, copies of the object data can be stored in real time on the local COS node as well as the node’s peers in the COS cluster. This functionality works only if the RAID feature on the node is disabled.

To disable the RAID feature on the node, open /arroyo/test/setupfile and set allow vault raid to 0.

To replicate object data on the node, open /arroyo/test/setupfile and set vault local copy count to a value greater than 1. This value specifies the how many copies of the object data are to be stored on the node.

To replicate object data on the peer nodes, open /arroyo/test/setupfile and set vault mirror copies to a value greater than 1. This value specifies how many remote copies of the object data are to be maintained.



Appendix A Reference Information Replicating Objects During Swift Write Operations


Cisco Cl

A

P P E N D I X B
Troubleshooting
Viewing the Statuses of Primary System ServicesThe status of the primary system services that provide COS services on a COS node can be viewed by executing the following commands:

[root@utah97 ~]# service cassandra statuscassandra is running[root@utah97 ~]# service cosd statuscosd (pid 9235) is running...[root@utah97 ~]# service cserver statuscserver is running

Service High-AvailabilityEach COS node leverages the monit framework to provide high-availability of the cos service. In the event that the cosd process crashes, the monit framework will restart the service.

Troubleshooting Swift and Swauth API ErrorsIf a Swift or Swauth API operation returns a 500 level HTTP error status, this can be an indication of an issue with one of the three primary system services — cassandra, cosd, or cserver. These error statuses are sometimes returned due to temporary resource exhaustion on the COS node. However, if the error statuses persist for a long period of time, verify that the primary system services are running. See Viewing the Statuses of Primary System Services.

Information that is helpful in tracing and analyzing Swift, Swauth, and Cassandra transactions can be found in the following log files collected on each individual COS node:

• /arroyo/log/http.log.<DATE>

– This log collects HTTP transaction information from Swift and Swauth operations.

• /arroyo/log/cosd.log.<DATE>.

– This log records Cassandra transactions that are executed when a client invokes a Swift or Swauth API operation.

Each COS node also includes a cos_stats utility that can be executed from a shell. The utility reports the current operational state of the COS node with respect to Swift and Swauth operations. This utility also provides information on resource and network utilization. The following is a sample output obtained by executing the cos_stats utility.

B-1oud Object Storage Release 2.1.2 User Guide

Appendix B Troubleshooting Unresponsive COS Service Manager GUI

Fri Jun 27 13:23:12 PDT 2014----------------------------------------------------------------------------| Operation | Active | Ops/Sec | Errors | Total |----------------------------------------------------------------------------| SWAUTH Account Create | 0 | 0.0 | 0 | 6 || SWAUTH Account Delete | 0 | 0.0 | 0 | 0 || SWAUTH Account Meta | 0 | 0.0 | 0 | 0 || SWAUTH User Create | 0 | 0.0 | 0 | 6 || SWAUTH User Delete | 0 | 0.0 | 0 | 0 || SWIFT Container Create | 0 | 0.0 | 0 | 0 || SWIFT Container Delete | 0 | 0.0 | 0 | 0 || SWIFT Container List | 0 | 0.0 | 0 | 0 || SWIFT Container Meta | 0 | 0.0 | 0 | 3298 || SWIFT Object Delete | 0 | 0.0 | 0 | 0 || SWIFT Object List | 0 | 0.0 | 0 | 0 || SWIFT Object Meta | 0 | 0.0 | 0 | 0 || SWIFT Object Read | 3999 | 1.9K | 0 | 13818 || SWIFT Object Write | 0 | 0.0 | 0 | 6694580 || SWIFT Token Get | 0 | 0.0 | 0 | 6 |----------------------------------------------------------------------------| Eth | Sessions | TCP RX bps | TCP RX pps | TCP TX bps | TCP TX pps |----------------------------------------------------------------------------| eth6 | 2004 | 196.320M | 407.518K | 8.511G | 716.957K || eth7 | 1024 | 170.025M | 352.604K | 8.239G | 688.469K || eth8 | 1095 | 159.576M | 328.401K | 8.038G | 673.944K || Total | 4123 | 525.920M | 1.089M | 24.788G | 2.079M |----------------------------------------------------------------------------| Poll: 0% : 823K/s | NetInt: 62% | RX: 37% | TCP TX: 74% | FQ: 0% |----------------------------------------------------------------------------

Unresponsive COS Service Manager GUIIf the COS Service Manager GUI is not responding, you should check COS AIC service status on the COS Service Manager CLI.

Step 1 Login to the COS Service Manager CLI.

Step 2 To view the status of all running services, execute this command: supervisorctl

Step 3 Verify the status of the cos-controller. This represents the status of the COS AIC.

[root@PM-35-C90 supervisor]# supervisorctl cli_insVmConfig_app RUNNING pid 3661, uptime 10 days, 14:51:03cli_mongoWatcher RUNNING pid 3656, uptime 10 days, 14:51:03cli_mosdns_app RUNNING pid 3655, uptime 10 days, 14:51:03cli_pamds_app RUNNING pid 3659, uptime 10 days, 14:51:03cli_platformservice_app RUNNING pid 3658, uptime 10 days, 14:51:03cli_restapi_app RUNNING pid 3654, uptime 10 days, 14:51:03cli_vmLoadBalancer_app RUNNING pid 3664, uptime 10 days, 14:51:03cli_zone_app RUNNING pid 3660, uptime 10 days, 14:51:03cos-controller RUNNING pid 30717, uptime 0:43:01docserver_app RUNNING pid 3644, uptime 10 days, 14:51:03drm-system_app RUNNING pid 3650, uptime 10 days, 14:51:03mos-controller_app RUNNING pid 3648, uptime 10 days, 14:51:03redis RUNNING pid 3640, uptime 10 days, 14:51:03roleserver_app RUNNING pid 3653, uptime 10 days, 14:51:03sm_app RUNNING pid 3646, uptime 10 days, 14:51:03statsd RUNNING pid 3643, uptime 10 days, 14:51:03unified_translogd RUNNING pid 3638, uptime 10 days, 14:51:03

B-2Cisco Cloud Object Storage Release 2.1.2 User Guide

Appendix B Troubleshooting Viewing COS AIC Logs

Viewing COS AIC LogsIn addition to the COS Status and COS Events reported in COS Service Manager GUI, detailed COS AIC logs can be found on the COS Service Manager CLI. There are three logs:

• /var/log/opt/cisco/mos/errorlog/COSController-errorlog.current

– This log records COS AIC execution information.

• /var/log/supervisor/cos-aic.log

– This log records COS AIC related stdout information. Typically, any node.js related error messages written to stdout are captured in this log.

• /var/log/supervisor/cos-aic.err.log

– This log records COS AIC related stderr information. Typically, any node.js related error messages written to stderr are captured in this log.


Appendix B Troubleshooting Viewing COS AIC Logs


Cisco Cl

A

P P E N D I X C
COS Tunables
Tunables are configuration parameters for CServer, which is a core component in COS. This appendix describes three categories of COS tunables:

• Debug Tunables

• Info Tunables

• Configuration Tunables

Debug TunablesDebug tunables are used in production environment to debug issues. Changes to these tunables affect the behavior of the system.

Table C-1 lists the Debug tunables.

Table C-1 Debug Tunables

Tunable name/LocationRead/Write/RW Description Units/Type

Default Value

shutdown_adapter_index RW Brings down the designated adapter. Echo X is the input command line used. For example, Echo 2 brings down the second adapter. This affects driver code as well. This is equivalent to if down.

Index ‐1

/proc/calypso/internal/enable_receive_capture

RW Enables logging of the received packets on the server.

Related tunable: /proc/calypso/internal/enable_transmit_capture

functionality (enable/disable)

0

/proc/calypso/internal/enable_transmit_capture

RW Enable logging of the transmitted packets from the server.

Related tunable: /proc/calypso/internal/enable_receive_capture


0

C-1oud Object Storage Release 2.1.2 User Guide

Appendix C COS TunablesInfo Tunables

Info TunablesThe Info tunables are used to get a snapshot of various parameters in the system.

Table C-2 lists the Info tunables.

/proc/calypso/internal/reset_adapter_index

RW Forces execution of the Adapter Reset Logic on a specified adapter index.


0

/proc/calypso/tunables/http_log_level

RW Enables verbose level logging on the http.log.

LogLevel LOG_I

NFO

Table C-1 Debug Tunables

Tunable name/LocationRead/Write/RW Description Units/Type

Default Value

Table C-2 Info Tunables

Tunable name/Location

Read/Write/RW Description Units/Type

Default Values

/proc/calypso/cm/cache2app_operational

Read Checks whether the cache2app services are operational or not.

Note This tunable is no longer used.

Status Value

0

/proc/calypso/cm/cachehits

Read Validates the number of objects fully present on the server.

Count 0

/proc/calypso/cm/cachemisses

Read Checks the number of objects not fully present on the server.

Count 0

/proc/calypso/cm/diskreadsectors

Read Counts the number of sectors read from the disk after starting a stream.

Count 0

/proc/calypso/cm/diskreadsegments

Read Counts the number of object segments read from the disk for streaming.

Count 0

/proc/calypso/cm/freedsegments

Read Counts the total number of free object segments.

Count 0

/proc/calypso/cm/memoryhits

Read Traces the memory hits of the object. Count 0

/proc/calypso/cm/romreadsectors

Read Reports the length of the object received from ROM in sectors.

Count 0

/proc/calypso/cmm/mapped/availableblocks

Read Reports the amount of memory that is available for use.

Count 0

/proc/calypso/cmm/mapped/totalblocks

Read Counts the total number of memory blocks on the system.

Count 0

C-2Cisco Cloud Object Storage Release 2.1.2 User Guide


/proc/calypso/status/bwm_settings

Read Queries the status of BandwidthManager and the thin pipes configured.

String N/A

/proc/calypso/status/diskbandwidthinfo

Read Returns verbose details on the disk bandwidth information.

String N/A

/proc/calypso/status/diskinfo

Read Returns verbose details on the configured disks on the server.

String N/A

/proc/calypso/status/hw_validation

Read Validates whether the hardware configured is according to the model of the device.

String N/A

/proc/calypso/status/networkbandwidth

Read Returns network bandwidth related information.

String N/A

/proc/calypso/status/pipe_settings

Read Reports the connected number of thin pipes.

String N/A

/proc/calypso/status/resiliencyinfo

Read Queries the configured ServiceAddress information (Setup or Control Service, HTTP Redirector Service, and Bandwidth Manager Service).

String N/A

/proc/calypso/status/resiliencystatus

Read Returns verbose details on the configured ServiceAddress in ACTIVE state.

String N/A

/proc/calypso/status/server_settings

Read Queries the server configurations and other details related to the server.

string N/A

/proc/calypso/status/streamer/activestreamslots

Read Holds a count of the streams currently streamed from this server.

Count 0

/proc/calypso/status/streamer/allocatedstreamslots

Read Counts the “Allocated streams,” that is, the streams that have been created. The sum of the allocated streams from this streamer group should be close to the value in the alloc field in the LRR line present in proto log.

Count 0

/proc/calypso/status/streamer/cutthruinfo

Read Displays the limitation on the disk write allowed for the object based on the PopularityBased Caching design.

This tunable gathers information about CutThruMark that is used in CServer to determine the disk write required.

Count 0

/proc/calypso/status/streamer/fillinfo

Read Logs the CCP Fill status. String N/A

Table C-2 Info Tunables (continued)



Default Values



/proc/calypso/status/streamer/resiliencyinfo

Read Queries the configured ServiceAddress information (Setup or Control Service, HTTP Redirector Service, and Bandwidth Manager Service).

String N/A

/proc/calypso/status/streamer/resiliencystatus

Read Returns verbose details on the configured ServiceAddress.

String N/A

/proc/calypso/status/streamer/streaminfo

Read Queries the stream related information. String N/A

/proc/calypso/status/syslog/last_open_error

Read Stores the return value if a file cannot be opened through syslog layer.

Integer 0

/proc/calypso/status/syslog/last_write_error

Read Stores the return value if a write to syslog fails.

Integer 0

/proc/calypso/status/syslog/multiple_writes

Read Reports the number of times a log message required multiple writes to the syslog socket. Generally, it should be possible to send one full message in one single write to the syslog socket.

Integer 0

/proc/calypso/status/syslog/open_errors

Read Reports the number of times the system encountered errors at the time of opening a log file using the syslog API.

Integer 0

/proc/calypso/status/syslog/write_errors

Read Reports the number of times log messages have failed to write to the syslog layer.

Integer 0

/proc/calypso/status/vault/fillinfo

Read Queries the CCP Fill status on the vault. String N/A

/proc/calypso/status/vault/httpinfo

Read Displays the configured HTTP Adapter information.

String N/A

/proc/calypso/status/vault/laninfo

Read Displays the details of Capacity, Connections, Inbound Request, and Outbound Request.

String N/A

/proc/calypso/status/vault/settings

Read Displays vault settings such as smoothing, mirroring, local copy, and remote mirror copy

String N/A

/proc/calypso/status/vault/streaminfo

Read Displays stream-related information for the vault.

String N/A

/proc/calypso/status/cache/fillinfo

Read Queries the CCP Fill status of the cache. String N/A

/proc/calypso/status/cache/httpinfo

Read Displays the configured HTTP Adapter information.

String N/A

/proc/calypso/status/cache/streaminfo

Read Displays stream-related information for the cache.

String N/A




Default Values


Appendix C COS TunablesConfiguration Tunables

Configuration TunablesThe Configuration tunables are used to configure the various parameters in the system. These tunables have a GUI counterpart.

Table C-3 lists the Configuration tunables.

/proc/calypso/tunables/runtimedebuggerversion

Read Provides information related to the version of runtimedebugger used in the system.

Version 1

/proc/calypso/internal/ingest_vault_max_display_count

Read Counts the maximum number of ingesting vaults configured to display statistics.

Count 10




Default Values

Table C-3 Configuration Tunables


Read/Write/RW type Description Units/Type

default Values

/proc/calypso/cm/active_trickspeed_0

Read Reads the configured trick speed 0, which is configured from CDSM.

Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



/proc/calypso/test/arp_readroutingtable

Write Reads and updates the entries from the RoutingTable.

To make a change to a routing table entry take effect, enter echo 1 > /proc/calypso/test/arp_readroutingtable.

Boolean 0

/proc/calypso/test/readarptable

Write Reads and updates the entries in the configuration file /arroyo/test/ArpTable.

To make a change to an ArpTable entry take effect, enter echo 1 > /proc/calypso/test/readarptable.

Boolean 0

/proc/calypso/test/readfillsourceconfig

Write Reads and updates the entries in the configuration file /arroyo/test/FillSourceConfig.

To make a change to a FillSourceConfig entry take effect, enter echo 1 >/proc/calypso/test/readfillsourceconfig.

Boolean 0

/proc/calypso/test/readrecordingchannelmap

Write Reads and updates the entries in the configuration file /arroyo/test/RecordingChannelMap.

To make a change to a RecordingChannelMap entry take effect, enter echo 1 >/proc/calypso/test/readrecordingchannelmap.

Boolean 0

/proc/calypso/test/readremoteservers

Write Reads and updates the entries in the configuration file /arroyo/test/RemoteServers.

To make a change to a RemoteServers entry take effect, enter echo 1 >/proc/calypso/test/readremoteservers.

Boolean 0

/proc/calypso/test/readstreamdestinationmap

Write Reads and updates the entries in the configuration file /arroyo/test/StreamDestinationMap.

To make a change to a StreamDestinationMap entry take effect, enter echo 1 >/proc/calypso/test/readstreamdestinationmap.

Boolean 0




default Values



/proc/calypso/test/readsubnettable

Write Reads and updates the entries in the configuration file /arroyo/test/SubnetTable.

To make a change to a SubnetTable entry take effect, enter echo 1 >/proc/calypso/test/readsubnettable.

Boolean 0

/proc/calypso/test/readtgidtable

Write Reads and updates the entries in the configuration file /arroyo/test/TransportGroupIdTable.

To make a change to a TransportGroupIdTable entry take effect, enter echo 1 >/proc/calypso/test/readsubnettable.

Note This tunable is no longer used.

Boolean 0

/proc/calypso/test/shutdown

Write Shuts down all processes running in the system.

Boolean 0

/proc/calypso/tunables/trickspeed_0

Write Sets both positive and negative trick speed 0 as a pair.

Because this tunable does not have directional information, there cannot be different positive and negative speeds.

Note This is an older version tunable.

Speed/Hex -1





Speed/Hex -1





Speed/Hex -1




default Values



/proc/calypso/tunables/trickspeedv2_0

Write Sets positive or negative trick speed 0. Speed/Hex -1























/proc/calypso/tunables/stream_null_packets

Write Enables or disables the stream null packet logic at the remaining length.

This tunable fills the remaining length with null packets, while adding the stream data in the stream slot.

Boolean 1

/proc/calypso/tunables/stream_using_qam

Write Configures the IP or QAM based setup.

If the flag is not set, the IP setup and Motorola PreEncryption use the original program number for the Encyrption data (Inband).

If the flag is set, the QAM based setup and Motorola PreEncryption reference the program number and data from the port (Outband).

Boolean 1




default Values



/proc/calypso/tunables/bwm_readConfFile

Write Reads and updates the entries in the configuration file /arroyo/test/BandwidthManager.

To make a change to a BandwidthManager entry take effect, enter echo 1 >/proc/calypso/tunables/bwm_readConfFile.

Boolean 0

/proc/calypso/tunables/cm_playnowatlivepoint

Write Enables or disables play at the live point. When enabled, this tunable allows for play at the live point.

Boolean 1

/proc/calypso/tunables/convert_npt_using_pts

Write Improves accuracy of Play computation by using actual frame PTS information rather than an NPT offset.

When enabled, the NPT computation uses actual frame PTS information to improve play accuracy.

Boolean 1

/proc/calypso/tunables/disable_fade_frame

Write Enables or disables the frame fading option at the file transition.

When enabled, there is no extra fade frame inserted at the file transition.

When disabled, 4 fade frames are inserted at the splice point.

Boolean 0

/proc/calypso/tunables/ftpoutmaxrate

Write Configures the ftpmax rate for total active FTP sessions.

Hex/bps 19000000

/proc/calypso/tunables/ftpoutmaxsessions

Write Configures the number of FTP sessions. No of session/Hex

0

/proc/calypso/tunables/popularity_half_life

Read Affects how often CServer plays the content. The popularity_half_life value determines the rate of increase of the bump-on-play value.

The bump-on-play value is managed by the class BumpOnUse, which resides in cm/PopularityValue and is set via CDSM.

Sec/Hex A880

/proc/calypso/tunables/read_etc_hosts

Write Reads the ETC hosts from /etc/hosts. Boolean 1




default Values



/proc/calypso/tunables/pipe_add

Write Adds the new pipe with any reboot. Pipe format Empty

Format:

pipe <pipeName> transmit <bw in Mbps> receive <bw in Mbps> [httpnetwork <ip>/<netmask>]

. . . .

[arrayid|serverid <id>] > /proc/calypso/tunables/pipe_add

/proc/calypso/tunables/pipe_del

Write Deletes the existing pipe. Pipe id Empty

Format:

<pipeName> > /proc/calypso/tunables/pipe_del




default Values



/proc/calypso/tunables/pipe_modify

Write Modifies the existing pipe configuration.

Pipe format Empty

Format:

pipe <pipeName> transmit <bw in Mbps> receive <bw in Mbps> [httpnetwork <ip>/<netmask>]

. . . .

[arrayid|serverid <id>] > /proc/calypso/tunables/pipe_modify

/proc/calypso/tunables/stream_from_local_array

Write Enables streaming content from a local server. This tunable is intended specifically to control the server.

Boolean 0

/proc/calypso/tunables/vaultdecommission

Write Decommissions the vault. When the vault is removed, this tunable migrates all information to another vault and logs the status under decommission information.

Boolean 0

/proc/calypso/tunables/vaultmirrorcopies

Read Reads the number of mirror copies that need to be present on the site from the setup file.

Number of copies

2




default Values



/proc/calypso/tunables/bwm_updateInterval

Write Defines the time interval at which Bandwidth management updates are exchanged by the servers.

At each update interval, the servers report their bandwidth usage to the primary server, and the primary assigns bandwidth to other servers as needed.

This tunable also reports BMW when the committed bandwidth reaches first 90% of total bandwidth, and again when it reaches 95% of total bandwidth.

Seconds 10

/proc/calypso/internal/offline

Write Takes the server offline when the designated server goes offline.

Boolean 0

/proc/calypso/tunables/bPSIErrorFailsIngest

RW If enabled, ensures that a trick code aborts the ingest if the bitstream does not contain PAT/PMT information.

Boolean TRUE

/proc/calypso/tunables/bRateErrorFailsIngest

RW If enabled, causes the ingest to fail if the ingest bit rate cannot be computed.

Boolean TRUE

/proc/calypso/tunables/nCCErrorsFailIngest

RW Defines the number of CC errors allowed after which the ingest is marked as failed.

Number None

/proc/calypso/tunables/nDiscErrorsFailIngest

RW Defines the number of discontinuity errors allowed after which the ingest is marked as failed.

Number None

/proc/calypso/tunables/nPicGapsFailIngest

RW Defines the number of PicGaps allowed after which the ingest is marked as failed.

Number 3

/proc/calypso/tunables/nPicGapsinHSFailIngest

RW Defines the duration of a PicGap in hundredths of a second that is counted toward an ingest failure.

Time in hundredth of seconds

5 (seconds)

/proc/calypso/tunables/nSyncLossesFailIngest

RW Defines the number of sync losses allowed after which the ingest is marked as failed.

Number 3

/proc/calypso/tunables/nSyncLossesinHSFailIngest

RW Defines the duration of a sync loss in hundredths of a second that is counted toward an ingest failure.

Time in hundredth of seconds

5 (seconds)

/proc/calypso/tunables/maintenancemirrorcopies

RW Defines the number of copies for mirroring during server maintenance.

Number 2

/proc/calypso/tunables/enable_rtp_rtx

RW Enables VOD error repair feature with RTP RTX.

Boolean default 0

Range: 0,1




default Values



/proc/calypso/tunables/nEveryNMinutes

RW Allows ingestion depending on the ingestknob settings, such as a check of a sample based on the time specified.

Minutes 30

/proc/calypso/tunables/tm_restrict_enable_ffw_resume

Write Enables or disables the streamer resuming fast forward after playing fast forward trick restricted content in 1x.

Boolean FALSE

/proc/calypso/tunables/tm_restrict_enable_forward_jump

Write Disables or enables forward jump restriction.

Boolean FALSE

/proc/calypso/tunables/tm_restrict_enable_forward_jump_resume

Write Enables or disables the streamer resuming forward jump after playing forward jump restricted content in 1x.

Boolean FALSE

/proc/calypso/tunables/tm_restrict_enable_rewind_skip

Write Enables or disables rewind skip. If enabled, rewind jumps to the beginning of the restricted content and continues rewinding from there.

Boolean FALSE

/proc/calypso/tunables/vaultarraymirror

Write Configures mirror partner, array id to which the server is mirroring.

To stop the array mirroring, enter echo <negative value of the partner array id>.

Related tunable:

/proc/calypso/tunables/allowvaultarraymirroring

Array id 1

/proc/calypso/cm/active_maxpacketsize

Read Specifies the Active Max packet size to be sent out on the streaming network.

Packet Size 7 MPEG Cells for Standard Size packets, 21 MPEG Cells for Jumbo size packets

/proc/calypso/tunables/vaultlocalcopycount

RW Configures the number of local mirror copy count on Mirroring Policy.

Count 1

/proc/calypso/tunables/maintenancearraymirroringallowed

RW Enables array mirroring during the maintenance interval.


TRUE




default Values




Cisco Cl

A

P P E N D I X D
COS Command Line Utilities
COS provides the following command line utilities (CLIs) for use on a separate Linux installation that leverage the APIs exposed by the COS node:

• cos-swift — provides command-line access to the Swift API

• cos-swauth — provides command-line access to the Swauth API

This appendix provides the information needed to install and use these CLI utilities.

Note For details on the Swift and Swauth APIs, see the Cisco Cloud Object Storage Release 2.1.1 API Guide.

Hardware PrerequisitesBefore installing the COS CLI utilities, you must have the following hardware available:

• A COS node, release 2.1.1 or later, staged in either standalone or cluster mode

• A CENTOS 6 Linux workstation on which to install the CLI utilities

Installing the CLI UtilitiesPerform the following steps to install the COS CLI utilities.

Note In the examples shown below, commands that you enter are shown in bold, while system responses are shown in normal type. System responses are typical examples and may vary by installation.

Step 1 Copy the COS 2.1.2 ISO image to the Linux workstation that will host the CLI utilities.

ll cos_repo-2.1.2.iso

-rw-r--r-- 1 root root 250613760 Dec 12 20:39 cos_repo-2.1.2.iso

Step 2 Mount the ISO image.

mount -oloop cos_repo-2.1.2.iso /mnt/cdrom

ls /mnt/cdromcos-2.1.2-0b8-local.repo local_repo_setup Packages repodata

D-1oud Object Storage Release 2.1.2 User Guide

Appendix D COS Command Line Utilities Installing the CLI Utilities

Step 3 Install the utilities on the local workstation.

./local_repo_setup

Loaded plugins: fastestmirror, securityLoading mirror speeds from cached hostfile * base: mirror.cogentco.com * extras: mirrors.advancedhosters.com * updates: centos.mirror.nac.netbase | 3.7 kB 00:00base/group_gz | 216 kB 00:00base/filelists_db | 6.1 MB 00:00base/other_db | 2.8 MB 00:00cos-2.1.2-0b8 | 3.6 kB 00:00 ...cos-2.1.2-0b8/group_gz | 536 B 00:00 ...cos-2.1.2-0b8/filelists_db | 66 kB 00:00 ...cos-2.1.2-0b8/primary_db | 43 kB 00:00 ...cos-2.1.2-0b8/other_db | 20 kB 00:00 ...extras | 3.4 kB 00:00extras/filelists_db | 31 kB 00:00extras/prestodelta | 605 B 00:00extras/other_db | 37 kB 00:00updates | 3.4 kB 00:00updates/filelists_db | 1.5 MB 00:00updates/prestodelta | 194 kB 00:00updates/other_db | 19 MB 00:02Metadata Cache CreatedLoaded plugins: fastestmirror, securityLoading mirror speeds from cached hostfile * base: mirror.cogentco.com * extras: mirrors.advancedhosters.com * updates: centos.mirror.nac.netrepo id repo name statusbase CentOS-6 - Base 6,518cos-2.1.2-0b8 Cisco Cloud Object Store 2.1.2 57extras CentOS-6 - Extras 37updates CentOS-6 - Updates 748repolist: 7,360

Step 4 Run the YUM installation for the COS CLI utilities.

yum install cos-cli

Loaded plugins: fastestmirror, securityLoading mirror speeds from cached hostfile * base: mirror.cogentco.com * extras: mirrors.advancedhosters.com * updates: centos.mirror.nac.netSetting up Install ProcessResolving Dependencies--> Running transaction check---> Package cos-cli.x86_64 0:2.1.2-cos0.1 will be installed--> Finished Dependency Resolution

Dependencies Resolved

==================================================================================================================================================================================== Package Arch Version Repository Size====================================================================================================================================================================================Installing: cos-cli x86_64 2.1.2-cos0.1 cos-2.1.2-0b8 142 k

D-2Cisco Cloud Object Storage Release 2.1.2 User Guide

Appendix D COS Command Line Utilities Using the CLI Utilities

Transaction Summary====================================================================================================================================================================================Install 1 Package(s)

Total download size: 142 kInstalled size: 635 kIs this ok [y/N]: yDownloading Packages:Running rpm_check_debugRunning Transaction TestTransaction Test SucceededRunning Transaction Installing : cos-cli-2.1.2-cos0.1.x86_64 1/1 Verifying : cos-cli-2.1.2-cos0.1.x86_64 1/1

Installed: cos-cli.x86_64 0:2.1.2-cos0.1

Complete!

The CLI Utilities are now ready for use. The following section provides usage information.

Using the CLI Utilities

cos swauthThe cos-swauth command line utility is used to manage authentication accounts and users in a COS cluster. The command line utility has the following form:

cos-swauth [–a <auth-ip>] [–u <admin-user>] [–k <admin-key>] [–h/– –help <subcommand>]

[–v/– –verbose] [subcommand <options>]

You can use the following parameters with the command line utility:

• –a <auth-ip>

– The IP address of the COS authentication service.

– The IP address can also be specified by using the COS_AUTH_IP environment variable.

• –u <admin-user>

– Name of the user authorizing the command.

– When unspecified, the user name defaults to .super_admin.

– The user name, except in the case of the super admin, must be specified in the form <account>:<user>.

– The user name can also be specified by using the COS_ADMIN_USER environment variable.

• –k <admin-key>

– The authorization key (password) of the user.

– The key can also be specified by using the COS_ADMIN_KEY environment variable.

• –h/– –help <subcommand>

– Display help information for the specified subcommand.



• –v/– –verbose

– Display detailed information as part of the output.

• subcommand <options>

– The subcommands include –l/list, –sa/set-account, –su/set-user, –g/get, –i/info, –d/delete.

– Each subcommand is associated with a set of relevant options. Table D-1 provides a listing of the subcommands and their options.

Table D-1 cos-swauth Subcommands and Their Options

Subcommand/Option Description

–l/list

No account specified List the accounts for the specified admin user.

<account> List the users belonging to the specified account.

–sa/set-account

<account> Create or update the specified account.

–s/– –suffix Specify the suffix to be used in creating an account ID.

If a suffix is not provided, a random UUID will be used.

Note A suffix may only be specified for new accounts.

–e/– –endpoint < name>:[<IP>] [–t/– –type <service type>]

Specify a service endpoint for the account.

If the IP address is omitted, the specified endpoint will be deleted.

For the reserved endpoint name default, the name of the default endpoint for the service type must be specified instead of the IP address.

If unspecified, the service type will default to storage.

–su/set-user

<user> Create or update the details of the specified user.

–K/– –key <user key> Specify the user’s key (password).

This option can be used by admin or reseller-admin users to assign a password to a new user. It can also be used by existing users to change their password.

–A/– –admin Assign the user admin privileges for the specified account.

Account admins can create, delete, or modify users within an account, and have admin privilege in the services associated with the account.

The user privileges default to –N/– –normal if –A/– –admin and –R/– –reseller_admin options are omitted.



cos-swiftThe cos-swift command line utility can be used to manage storage accounts, storage containers, and storage objects in a COS cluster. The command line utility has the following form:

cos-swift [–t <auth-token>] [–a <storage-url>] [–v/– –verbose][–h/– –help <subcommand>] [subcommand <options>]

You can use the following parameters with the command line utility:

• –t <auth-token>

– The authentication token returned by the cos-swauth get operation.

–R/– –reseller_admin Assign the user reseller-admin privileges for the COS cluster.

Reseller-admins can create, delete, or modify accounts, and have all the privileges of an account admin in the accounts defined in the COS authentication service.

The user privileges default to –N/– –normal if –A/– –admin and –R/– –reseller_admin options are omitted.

–N/– –normal Specify that the user has no special privileges for the specified account.

This option can also be used to withdraw the existing privileges of a user.

The user privileges default to –N/– –normal if a privilege option is not specified.

–g/get

<account>:<user> Get an authentication token and storage service URL for the specified user.

–K/– –key <user key> Specify the user’s key (password).

This option may be omitted if the user does not have a key.

–e/– –endpoint Retrieve the service endpoint catalog associated with the user account.

–i/info

<account> Retrieve account information, including the service endpoint catalog.

<account>:<user> Retrieve detailed user information.

–d/delete

<account> Delete the specified account.

Note Only an empty account can be deleted.

<account>:<user> Delete the specified user.

Table D-1 cos-swauth Subcommands and Their Options




– The authentication token can also be specified by using the COS_AUTH_TOKEN environment variable.

• –a <storage-url>

– The storage URL returned by the cos-swauth get operation.

– The storage URL can also be specified by using the COS_STORAGE_URL environment variable .

• –v/– –verbose

– Display detailed information as part of the output.

• –h/– –help <subcommand>

– Display help information for the specified subcommand.

• sub-command <options>

– The subcommands include –i/info, –l/list, –s/set, –g/get, –u/update, –d/delete.

– Each subcommand is associated with a set of relevant options. Table D-2 provides a listing of the subcommands and their options.

Table D-2 cos-swift Subcommands and Their Options


–i/info

No container or object specified. Display metadata for the storage account in the storage URL.

<container name> Display metadata for the specified storage container.

<container name>/<object name> Display metadata for the specified storage object.

–l/list

No container specified. List the containers of the storage account in the storage URL.

<container name> List the objects of the specified storage container.

<container name>/<object path> Form a prefix by appending a slash to the specified object path if the path does not end with a slash.

Then, list the objects of the specified storage container whose names have this prefix.

–L/– –limit <value> Restrict the number of entries listed to less than or equal to the specified value.

–m/– –marker <value> List the storage containers or objects with names greater than the specified marker value.

–e/– –end_marker <value> List the storage containers or objects with names less than the specified marker value.

–p/– –prefix <value> List the storage containers or objects whose names begin with the specified prefix.

–d/– –delimiter <character> List the storage containers or objects whose names do not have the specified delimiting character.

Note If the –p/– –prefix <value> option is used, the delimiting character may be present in the specified prefix.



–V/– –Verbose If no container name is specified, display the metadata of the containers belonging to the storage account in the storage URL. For each container, the container name, the number of subordinate objects, and the total content size of all subordinate objects are displayed.

If a container name is specified, display the metadata of the objects in that storage container. For each object, the object name, MD5 checksum, content type, content size, and modification timestamp are displayed.

–s/set

<container name> Create a container with the specified name.

<container name>/<object name> Create an object with the specified name within the mentioned container.

–m/– –meta <name>:<value> Create a custom metadata item with the specified name and value. This option may be repeated multiple times.

–r/– –read <ACL> Specify a read access control list (ACL) for a container. Specifying an empty string ("") as the ACL will cause the existing ACL to be deleted.

–w/– –write <ACL> Specify a write access control list (ACL) for a container.

–f/– –file <name> Upload the named file as object content. If an existing object is specified, this file will replace prevailing object content.

–T/– –type <content type> Specify the content type of an object. The specified content type must be a valid mime type.

–u/update

No container or object specified. Update the metadata of the storage account in the storage URL.

<container name> Update the metadata of the specified container.

<container name>/<object name> Update the metadata of the specified object in the container.

–m/– –meta <name>:<value> Create a custom metadata item with the specified name and value. This option may be repeated multiple times.

–r/– –read <ACL> Specify a read access control list (ACL) for a container. Specifying an empty string ("") as the ACL will cause the existing ACL to be deleted.

–w/– –write <ACL> Specify a write access control list (ACL) for a container.

–g/get

<container name>/<object name> Retrieve the content of the specified object.

–f/– –file <name> Store the retrieved object content in the specified file.

Note If this option is not used and file is not specified, the object content will be directed to stdout.

–d/delete





<container name> Delete the specified container.

Note Only an empty container can be deleted.

<container name>/<object name> Delete the specified object.




Cisco Cloud Object Storage Release 2.1.2 User Guide · Contents vi Cisco Cloud Object Storage Release 2.1.2 User Guide Replicating Objects During Swift Write Operations A-15 APPENDIX

Documents