Hitachi Storage Plug-in for Containers Quick Reference Guide This Quick Reference Guide provides an implementation overview and describes the usage requirements, installation, and configuration of Storage Plug-in for Containers. MK-92ADPTR142-01 May 2018
54
Embed
Hitachi Storage Plug-in for Containers Quick Reference Guideitdoc.hitachi.co.jp/manuals/st_plugcon/dkc/MK-92ADPTR142-01.pdf · Docker swarm Consists of a manager and worker nodes
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Hitachi Storage Plug-in for Containers
Quick Reference GuideThis Quick Reference Guide provides an implementation overview and describes the usagerequirements, installation, and configuration of Storage Plug-in for Containers.
No part of this publication may be reproduced or transmitted in any form or by anymeans, electronic or mechanical, including copying and recording, or stored in adatabase or retrieval system for commercial purposes without the express writtenpermission of Hitachi, Ltd., or Hitachi Vantara Corporation (collectively “Hitachi”).Licensee may make copies of the Materials provided that any such copy is: (i) created asan essential step in utilization of the Software as licensed and is used in no othermanner; or (ii) used for archival purposes. Licensee may not make any other copies ofthe Materials. “Materials” mean text, data, photographs, graphics, audio, video anddocuments.
Hitachi reserves the right to make changes to this Material at any time without noticeand assumes no responsibility for its use. The Materials contain the most currentinformation available at the time of publication.
Some of the features described in the Materials might not be currently available. Refer tothe most recent product announcement for information about feature and productavailability, or contact Hitachi Vantara Corporation at https://support.hitachivantara.com/en_us/contact-us.html.
Notice: Hitachi products and services can be ordered only under the terms andconditions of the applicable Hitachi agreements. The use of Hitachi products is governedby the terms of your agreements with Hitachi Vantara Corporation.
By using this software, you agree that you are responsible for:
1. Acquiring the relevant consents as may be required under local privacy laws orotherwise from authorized employees and other individuals; and
2. Verifying that your data continues to be held, retrieved, deleted, or otherwiseprocessed in accordance with relevant laws.
Notice on Export Controls. The technical data and technology inherent in thisDocument may be subject to U.S. export control laws, including the U.S. ExportAdministration Act and its associated regulations, and may be subject to export orimport regulations in other countries. Reader agrees to comply strictly with all suchregulations and acknowledges that Reader has the responsibility to obtain licenses toexport, re-export, or import the Document and any Compliant Products.
All other trademarks, service marks, and company names in this document or websiteare properties of their respective owners.
Hitachi Storage Plug-in for Containers Quick Reference Guide ii
Chapter 1: Overview............................................................................... 9About Hitachi Storage Plug-in for Containers...................................................... 9Docker Swarm or Kubernetes/OpenShift framework.........................................12About the environment setup tasks................................................................... 13Requirements.....................................................................................................14
Server requirements.....................................................................................14Storage requirements...................................................................................14
Best practice...................................................................................................... 18
Chapter 2: Docker framework..............................................................19About installing Storage Plug-in for Containers................................................. 19About configuring storage and host group settings........................................... 19Install Storage Plug-in for Containers for Docker framework.............................21Manage data......................................................................................................22Command references........................................................................................ 23Docker examples............................................................................................... 26
Contents
Hitachi Storage Plug-in for Containers Quick Reference Guide 3
Chapter 3: Kubernetes/OpenShift framework.................................... 31Install Storage Plug-in for Containers for Kubernetes/OpenShift framework.....31Manage data......................................................................................................33About configuring storage and host group settings .......................................... 34About configuring persistent volume claim settings...........................................35Command references........................................................................................ 36Kubernetes/OpenShift examples....................................................................... 38
Chapter 4: Uninstall Storage Plug-in for Containers.........................41Uninstall Storage Plug-in for Containers from the Docker framework............... 41Uninstall Storage Plug-in for Containers from the Kubernetes/OpenShiftframework.......................................................................................................... 42
Hitachi Storage Plug-in for Containers Quick Reference Guide 4
Preface
Hitachi Storage Plug-in for Containers lets you create containers and run statefulapplications inside those containers by using the VSP series volumes as dynamicallyprovisioned persistent volumes. This Quick Reference Guide provides an implementationoverview and describes the usage requirements, installation, and configuration ofStorage Plug-in for Containers.
Please read this document carefully to understand how to install and use the plug-in,and maintain a copy for your reference.
Intended audience
This document is intended for system administrators, Hitachi Vantara representatives,and authorized service providers who install, configure, and run Hitachi Storage Plug-infor Containers.
Readers of this document should be familiar with the following:■ Containerized environments and their basic functions■ Hitachi VSP series storage systems■ Hitachi Thin Image software■ Hitachi Dynamic Provisioning software
Product version
This document applies to Hitachi Storage Plug-in for Containers version 1.1.0.
Document conventions
This document uses the following typographic conventions:
Preface
Hitachi Storage Plug-in for Containers Quick Reference Guide 5
Convention Description
Bold ■ Indicates text in a window, including window titles, menus,menu options, buttons, fields, and labels. Example:
Click OK.■ Indicates emphasized words in list items.
Italic ■ Indicates a document title or emphasized words in text.■ Indicates a variable, which is a placeholder for actual text
provided by the user or for output by the system. Example:
pairdisplay -g group
(For exceptions to this convention for variables, see the entry forangle brackets.)
Monospace Indicates text that is displayed on screen or entered by the user.Example: pairdisplay -g oradb
< > anglebrackets
Indicates variables in the following scenarios:■ Variables are not clearly separated from the surrounding text or
from other variables. Example:
Status-<report-name><file-version>.csv
■ Variables in headings.
[ ] squarebrackets
Indicates optional values. Example: [ a | b ] indicates that you canchoose a, b, or nothing.
{ } braces Indicates required or expected values. Example: { a | b } indicatesthat you must choose either a or b.
| vertical bar Indicates that you have a choice between two or more options orarguments. Examples:
[ a | b ] indicates that you can choose a, b, or nothing.
{ a | b } indicates that you must choose either a or b.
This document uses the following icons to draw attention to information:
Icon Label Description
Note Calls attention to important or additional information.
Document conventions
Preface
Hitachi Storage Plug-in for Containers Quick Reference Guide 6
Icon Label Description
Tip Provides helpful information, guidelines, or suggestions forperforming tasks more effectively.
Caution Warns the user of adverse conditions and/or consequences(for example, disruptive operations, data loss, or a systemcrash).
WARNING Warns the user of a hazardous situation which, if notavoided, could result in death or serious injury.
Conventions for storage capacity values
Physical storage capacity values (for example, disk drive capacity) are calculated basedon the following values:
Physical capacity unit Value
1 kilobyte (KB) 1,000 (10 3) bytes
1 megabyte (MB) 1,000 KB or 1,0002 bytes
1 gigabyte (GB) 1,000 MB or 1,0003 bytes
1 terabyte (TB) 1,000 GB or 1,0004 bytes
1 petabyte (PB) 1,000 TB or 1,0005 bytes
1 exabyte (EB) 1,000 PB or 1,0006 bytes
Logical capacity values (for example, logical device capacity, cache memory capacity) arecalculated based on the following values:
Logical capacity unit Value
1 block 512 bytes
1 cylinder Mainframe: 870 KB
Open-systems:■ OPEN-V: 960 KB■ Others: 720 KB
1 KB 1,024 (210) bytes
Conventions for storage capacity values
Preface
Hitachi Storage Plug-in for Containers Quick Reference Guide 7
Logical capacity unit Value
1 MB 1,024 KB or 1,0242 bytes
1 GB 1,024 MB or 1,0243 bytes
1 TB 1,024 GB or 1,0244 bytes
1 PB 1,024 TB or 1,0245 bytes
1 EB 1,024 PB or 1,0246 bytes
Comments
Please send us your comments on this document [email protected]. Include the document title and number, includingthe revision level (for example, -07), and refer to specific sections and paragraphswhenever possible. All comments become the property of Hitachi Vantara Corporation.
Thank you!
Getting help
Hitachi Vantara Support Connect is the destination for technical support of products andsolutions sold by Hitachi Vantara. To contact technical support, log on to Hitachi VantaraSupport Connect for contact information: https://support.hitachivantara.com/en_us/contact-us.html.
Hitachi Vantara Community is a global online community for Hitachi Vantara customers,partners, independent software vendors, employees, and prospects. It is the destinationto get answers, discover insights, and make connections. Join the conversation today!Go to community.hitachivantara.com, register, and complete your profile.
Release notes
Read the release notes before installing and using this product. They may containrequirements or restrictions that are not fully described in this document or updates orcorrections to this document. Release notes are available on Hitachi Vantara SupportConnect: https://knowledge.hitachivantara.com/Documents.
Comments
Preface
Hitachi Storage Plug-in for Containers Quick Reference Guide 8
Storage Plug-in for Containers is a software component that contains libraries, settings,and commands that you can use to create a container in order to run your statefulapplications. It enables the stateful applications to persist and maintain data after thelife cycle of the container has ended. Storage Plug-in for Containers provides persistentvolumes from Hitachi VSP series storage.
About Hitachi Storage Plug-in for Containers
Storage Plug-in for Containers utilizes built-in high-availability to enable a Docker swarmmanager or a Kubernetes/OpenShift master node to orchestrate storage tasks betweenhosts in a cluster. However, Storage Plug-in for Containers can also be used in non-clustered environments.
Following are examples of containerized environments, as clustered implementations,using a Docker swarm manager and Kubernetes master node. In both examples, theHitachi Configuration Manager server is optional.
Figure 1 Clustered implementation using Docker Swarm Manager
The following table describes the components that comprise a containerizedenvironment using Docker.
Chapter 1: Overview
Hitachi Storage Plug-in for Containers Quick Reference Guide 9
Components of a containerized environment using Docker
Component Purpose
conf The configuration file for Storage Plug-in forContainers.
Container Created by Storage Plug-in for Containers, itcontains libraries and settings that arerequired to run stateful applications.
Docker swarm Consists of a manager and worker nodes (inthe example, Node 1 and Node 2) that runservices:■ A manager distributes tasks across the
cluster and orchestrates the workernodes (or, workers) that comprise theswarm.
■ Workers run Docker containers assignedto them by a manager.
etcd Stores LDEV information related to Hitachistorage, and must be installed on all hosts inthe cluster.
hctl Supports the command line interface forrunning snapshot create, restore, andremoval commands, and commands forchecking volumes.
Hitachi Thin Image HTI is used for taking snapshots of containerdata.
Hitachi VSP series storage Provides storage for the containers, andsupports snapshots of container data.
log Log file for Storage Plug-in for Containers.
Storage Plug-in for Containers For Docker, Storage Plug-in for Containersmust be installed on all hosts in a cluster.
SVP/Hitachi Configuration Manager Used for communication between StoragePlug-in for Containers and Hitachi VSP seriesstorage.
About Hitachi Storage Plug-in for Containers
Chapter 1: Overview
Hitachi Storage Plug-in for Containers Quick Reference Guide 10
Figure 2 Clustered implementation using Kubernetes Master node
The following table describes the components that comprise a containerizedenvironment using Kubernetes.
About Hitachi Storage Plug-in for Containers
Chapter 1: Overview
Hitachi Storage Plug-in for Containers Quick Reference Guide 11
Components of a containerized environment using Kubernetes
Component Purpose
Hitachi VSP series storage Provides storage for the containers, andsupports cloning of container data.
hspc.yaml The configuration file for the Storage Plug-infor Containers pod.
Kubernetes cluster A master node and a set of worker nodesthat typically run in a distributedenvironment on multiple nodes.
log Log file for Storage Plug-in for Containers.
PersistentVolumeClaim.yaml The configuration file for the PersistentVolume Claim.
StorageClass.yaml The configuration file for Storage Class.
Storage Plug-in for Containers The Master node deploys only one StoragePlug-in for Containers Pod in a cluster.
SVP/Hitachi Configuration Manager Used for communication between StoragePlug-in for Containers and Hitachi VSP seriesstorage.
Docker Swarm or Kubernetes/OpenShift framework
Storage Plug-in for Containers supports the managed plug-in system of Docker and theexternal provisioner of Kubernetes/OpenShift. Depending on your site's requirements,you can employ either orchestrator for your containerized environment. The frameworkoption you choose depends on the current (or future planned) scale of yourenvironment.
The following table summarizes the types of environment configurations that best fiteach option.
Docker and Kubernetes options
Option Details
Docker What it is:
With Docker Swarm manager, clients are entirely statelessand the entire state is managed on the Docker host. Anodd number of managers is recommended in case of anetwork split.
Docker Swarm or Kubernetes/OpenShift framework
Chapter 1: Overview
Hitachi Storage Plug-in for Containers Quick Reference Guide 12
Docker and Kubernetes options
Option Details
When to use it:
Best for sites that have small-scale container needs.
Kubernetes What it is:
Kubernetes is container agnostic, and is a platform thatallows your site to maintain deploying containers toproduction. It provides the means to deploy, scale, andmonitor your containers.
When to use it:
Best for sites that currently have--or plan to have--largescaling needs (for example, a Web application with a large/growing volume of users).
Red Hat OpenShiftContainer Platform
What it is:
Red Hat OpenShift Container Platform is an enterprisecontainer platform that provides container orchestrationand management with natively integrated Kubernetes.
When to use it:
Best for sites that currently have--or plan to have--largescaling needs in a production environment.
About the environment setup tasks
Storage Plug-in for Containers enables dynamic operation of storage systems whencontainers are used. In order to use Storage Plug-in for Containers with Docker orKubernetes, pre-installation tasks must be completed.
About the environment setup tasks
Chapter 1: Overview
Hitachi Storage Plug-in for Containers Quick Reference Guide 13
After the environment is set up, you can manage container data.
1. Requirements (on page 14)2. Docker framework (on page 19)3. Kubernetes/OpenShift framework (on page 31)
Requirements
Before you install Hitachi Storage Plug-in for Containers, check that your sever andstorage meet the minimum requirements that are outlined in the following tables.
Server requirements
Before you prepare your Docker or Kubernetes/OpenShift framework for the installationof Storage Plug-in for Containers, ensure that the server meets the followingrequirements.
Server requirements
Component Requirement
CPU x86_64
Memory 2 GB
RHEL v7.0 or later for Docker/Kubernetes
v7.4 for OpenShift
Docker v1.13.0 or later for Docker/Kubernetes
v1.12.0 for OpenShift
Multipathing software Device-mapper-multipath is required formultipath environments.
Applies to Docker Swarm only.
etcd v3 or later
Applies to Docker swarm only.
Kubernetes 1.6 or 1.9
User account Root user required
Storage requirements
Before you prepare your Docker or Kubernetes framework for the installation of StoragePlug-in for Containers, ensure that your storage meets the following requirements.
Requirements
Chapter 1: Overview
Hitachi Storage Plug-in for Containers Quick Reference Guide 14
Pool ■ Create an HDP pool■ Optional: Create a snapshot (HTI) pool
Swarm only
FC connection Create Host groups for each host.
Add WWNs to the Host groups of each host. For example, incase where you have 10 hosts in your Docker Swarm cluster,you may need at least 10 host groups.
ISCSI connection Create ISCSI Targets for each host.
Add IQNs to the Host group of each host that will join theSwarm cluster, to the iSCSI Target.
Kubernetes only
FC connection Create one Host group for the Kubernetes cluster.
Add all WWNs of hosts that will join the Kubernetes cluster, tothe host group. For example, if you have 10 hosts in yourKubernetes cluster, you only need 1 host group.
ISCSI connection Create one ISCSI Target for the Kubernetes cluster.
Add all IQNs of hosts that will join the Kubernetes cluster, to theiSCSI Target.
Best practice
The following is an important point to keep in mind when working in the Docker orKubernetes framework.
To prevent data corruption, only one node should access a volume for Read/Writeoperations.
Note: For Docker, Storage Plug-in for Containers programmatically prevents avolume from being accessed from multiple nodes. This is not the case forKubernetes (no such built-in handling exists).
Best practice
Chapter 1: Overview
Hitachi Storage Plug-in for Containers Quick Reference Guide 18
Chapter 2: Docker framework
The installation for the Docker framework involves installing Storage Plug-in forContainers and using the commands packaged with the plug-in to create a container,create a persistent volume, and attach the persistent volume to the container.
About installing Storage Plug-in for Containers
The file, hspc.tar.gz, contains all required components (binary files, scripts, etc.) that youwill use to create a container and manage container data.
You can run Storage Plug-in for Containers from the Docker swarm manager (forclustered environments) or on a designated node (for non-clustered environments). In aDocker swarm, Storage Plug-in for Containers must exist on both the swarm managerand each worker node.
During the installation, you will work with a configuration file, config.json, which containsstorage, host group, and other required settings that you will specify about yourenvironment (for example, the HDP pool ID to use). After you modify it, it will be used byStorage Plug-in for Containers.
About configuring storage and host group settings
The config.json file contains storage, host group, and other settings that are necessaryfor Storage Plug-in for Containers to work with your environment.
The following table provides information about the required parameters that areassociated with this file.
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 19
Parameters of the config.json file
Parameters and parameter descriptions
base Storage base configuration
serialNumber Storage serial number
url Storage URL*
user Storage user ID
password Storage user password
options Storage configuration
poolID HDP Pool ID
snapshotPoolID HTI Pool ID
This parameter is necessary forclone and snapshot. From SVOS7.2, you can use HDP Pool andSnapshot Pool as the same pool.
scsiTarget List of Host Group/iSCSI Targetsettings
portID
This is the Host Group port ID oriSCSI Target port ID.
scsiTargetNumber
This is the Host Group targetnumber or iSCSI Target number.This parameter is not the sameon each host.
databaseURL List of etcd endpoints
Use multiple endpoints for high-availability. If multiple endpointsare set here, requests to etcd can be redirected to one of theendpoints you specified. Otherwise, that can be a single-point-of-failure.
* IPv6 is not supported.
Example of config.json file with specified parameters
{ "base":{ "serialNumber": 54321,
About configuring storage and host group settings
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 20
Install Storage Plug-in for Containers for Docker framework
This procedure describes how to install Storage Plug-in for Containers so you can createa container.
During the installation, you will use the config.json file to specify storage, host group, andother required environment settings. For information about this file and the parametersyou will work with, see About configuring storage and host group settings (on page 19) .
Important: Upgrading Storage Plug-in for Containers from v1.0.0 to v1.1.0 orlater is not supported. You need to delete the volume which you created withv1.0.0, first. If you need to move to v1.1.0 or later from v1.0.0, complete thefollowing steps:
1. Uninstall the HSPC v1.0.0. See Uninstall Storage Plug-in for Containersfrom the Docker framework (on page 41) .
2. Because the host group setting is differrent between v1.0.0 and otherversions, reconfigure the host group setting. See Storage pre-installation(on page 17) .
3. Install HSPC v1.1.0 by completing the instructions in this section.
Procedure
1. Place the file, hspc.tar.gz, on the Docker swarm manager or designated node.2. Extract the file content: # tar -xvf hspc.tar.gz3. Install Storage Plug-in for Containers and create a container:
#./installer_hspcd.sh4. Check if Storage Plug-in for Containers is installed:
#docker plugin ls
Install Storage Plug-in for Containers for Docker framework
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 21
Note: Upgrading to the new plugin version requires the docker registry.Otherwise, you will have to uninstall Storage Plug-in for Containers, theninstall the new version. Use docker registry with the -r option:
#./installer_hspcd.sh -r <registry_ip>:<port>
After completing this step, If you run #docker plugin ls, ENABLED shouldindicate 'false'.
5. Complete one of the following options:
■ If you do not have your own registry, repeat steps 1-3 to install the plug-in on allhosts that will join the swarm cluster.
■ If you have your own registry, use the following commands:
#docker plugin push <registry_ip>:<port>/hspc
#docker plugin install <registry_ip>:<port>/hspc
Afterward, complete steps 6, 7, and 9. Skip step 8.
6. Copy configSample.json to config.json:#cd /opt/hitachi/hspc#cp configSample.json config.json
7. Specify your environment settings (storage, hosts, and so on) by modifying theconfig.json file to include required parameters:#vi config.json
8. Enable Storage Plug-in for Containers on all hosts:#docker plugin enable hspcThe log file, hspc-d.log, is created. If you run #docker plugin ls, ENABLEDshould indicate 'true'.
9. Confirm the existence of the log file:# tail -f /opt/hitachi/hspc/log/hspc-d.log
Manage data
After Storage Plug-in for Containers is installed for your Docker environment, you cancomplete tasks such as inspecting volumes and creating snapshots.
Manage data
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 22
Tip: For help with command usage, run the -h or --help option.
Note:■ The maximum number of concurrent volume creations/deletions is
approximately 20 (depending on the workload of the ConfigurationManager and storage device).
■ Regarding Docker commands, the -replica option for "docker servicecreate" is not supported for use with Storage Plug-in for Containers.
Command references
There are several Hitachi specific commands you will use with Storage Plug-in forContainers in a Docker environment, for snapshots: They are hctl prefaced commands.
There are also some Docker commands that include Hitachi specific options andinformation:■ docker volume create
The ext4 volume format is supported. Also, it is possible to create a volume with anexisting volume name (given options are ignored, however).
■ docker volume inspect
Note: For Docker specific commands, such as those used for detaching apersistent volume and stopping a container, refer to the Docker Web site:https://docs.docker.com/reference/
The following tables describe commands, command options, and usage rules for hctl,docker volume create, and docker volume inspect.
Command references
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 23
hctl snapshot create –sourceVolName Create a snapshot of a persistentvolume.
The maximum number of snapshotsthat can be created for storage is8,192.
hctl snapshot rm –sourceVolName -mu Delete a snapshot of a persistentvolume.
hctl snapshot restore –sourceVolName–mu
Restore a snapshot of a persistentvolume.
hctl volume ls List all volumes attached to a hostthat is running this command.
hctl volume rescan Rescan HBA.
hctl volume expand –volName –size Expand a volume.
When the volume is attached to acontainer, this command must berun on a host where the container isrunning.
hctl command options
Option Details
--sourceVolName <volume name> Specifies the source volume namefor a snapshot.
--mu <mu number> Indicates the mu number ofsnapshots.
--volName <volume name> Specifies the volume name forexpansion.
--size <size> Specifies size to be added forexpansion. For example, “--size100g”
Command references
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 24
Note:■ The maximum additional size for “volume expand” is less than 7T.■ The minimum additional size for “volume expand” is 1G.■ To expand a volume more than 7T, run the “volume expand” command.■ Expanding large size, for example 6T, takes approximately 10 minutes.■ Volumes which are not attached to any containers cannot be expanded.■ The maximum ext4 filesystem size certified by Red Hat Enterprise Linux 7
is 50TB as of April 2018.
Docker volume create command options
Option Details
--driver hspc Uses Storage Plug-in for Containersas a volume driver.
--name <volume name> Applies <volume name> to apersistent volume.
The following characters cannot beused: . / \
--opt size=<volume size with unit> Specifies the volume size. Unit mustbe included (M,G,T)
--opt mode=clone Creates a clone of an existingvolume.
Use with the sourceVolName option.
--opt sourceVolName=<volume name> Specifies the source volume namefor a clone.
"Status" is a vendor specific value.■ "Ldev" means LDEV ID. The ID is
decimal.■ "Size" means volume capacity.
Size includes units down tosecond decimal places.
■ "Snapshots" is snapshotinformation, including split time,mu (mirror unit) and pair status(for example, 'PSUS'). If asnapshot does not exist, the"Snapshots" parameter is empty.
■ "Status" means LDEV status. (forexample, NML (normal)/BLK(block)...)
■ "VolAttr" means volume attribute.(for example, HDP (Dynamicprovisioning volume)/HTI(Snapshot or Clone volume)....)
Caution: Before attaching a volume to a container using “docker run”,confirm that the volume is not being used by any other containers.Otherwise, if the volume is being used, it will be detached from the containerand attached to a new container.
Docker examples
Docker examples
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 26
Following are examples of tasks (creating a clone and restoring a snapshot for Ubuntu)using commands in practice.
Docker examples
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 27
Figure 3 Volume clone
Figure 4 Snapshot restore
Docker examples
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 28
Figure 5 Snapshot restore (continued)
Docker examples
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 29
Figure 6 Snapshot restore (continued)
Docker examples
Chapter 2: Docker framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 30
Chapter 3: Kubernetes/OpenShift framework
The installation for the Kubernetes/OpenShift framework involves installing StoragePlug-in for Containers and using the commands packaged with the plug-in to create aservice account, Pod, storage class, and persistent volume claim.
Install Storage Plug-in for Containers for Kubernetes/OpenShift framework
The Storage Plug-in for Containers package, hspc.tar.gz, contains all requiredcomponents (binary files, scripts, etc.) that you will use to create a service account, createa Storage Plug-in for Containers Pod, and then verify the outcome.
In a clustered environment, the installation must be completed on all nodes. During theinstallation, there are several key files that you will be working with. The following tablesummarizes each file.
Important: Upgrading Storage Plug-in for Containers from v1.0.0 to v1.1.0 orlater is not supported. You need to delete the volume which you created withv1.0.0, first. If you need to move to v1.1.0 or later from v1.0.0, complete thefollowing steps:
1. Uninstall Storage Plug-in for Containers v1.0.0. See Uninstall StoragePlug-in for Containers from the Kubernetes/OpenShift framework (onpage 42) .
2. Install Storage Plug-in for Containers v1.1.0 by completing theinstructions in this section.
Chapter 3: Kubernetes/OpenShift framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 31
Configuration files for the Kubernetes/OpenShift environment
File Details
hspc-sa.yaml Contains information that is used for the Kubernetesservice account.
hspc-pod.yaml Contains information that ensures the Storage Plug-in forContainers Pod works within the Kubernetes framework.
sc-sample.yaml Contains information about the Kubernetes storage classand provides support for merged storage and mergedpools.
Note: In the following procedure, replace kubectl with oc for the OpenShiftenvironment. Unless instructed otherwise, run kubectl and oc commandsfrom the master node.
Procedure
1. Obtain hspc.tar.gz and extract the contents to a temp directory on all nodes:# tar -xvf hspc.tar.gz
2. Install the Kubernetes plug-in container image on all nodes:#./installer_hspck.sh
Note: If you have your own registry, you can use push from one nodeand pull from other nodes in the cluster:
#docker push <registry_ip>:<port>/hspc
#docker pull <registry_ip>:<port>/hspc3. Complete the following steps on the master node:
a. Create a service account:# cd /opt/hitachi/hspc/yaml# kubectl create -f hspc-sa.yaml
b. Check if the service account was created:# kubectl get sa
c. Bind the service accout of cluster-admin to the service account you created atstep a:# kubectl create clusterrolebinding hspc-bind--clusterrole=cluster-admin--serviceaccount=default:hspc
d. Check if the bind was established:# kubectl get clusterrolebinding -o wide
e. Create a Pod for Storage Plug-in for Containers:# kubectl create -f hspc-pod.yaml
Install Storage Plug-in for Containers for Kubernetes/OpenShift framework
Chapter 3: Kubernetes/OpenShift framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 32
f. Check the Pod where the node is running:# kubectl get pod -o wide
g. Check the log file of Storage Plug-in for Containers on the node you confirmedat step f: # tail -f /opt/hitachi/hspc/log/hspc-k.log
Manage data
After installing and working with Storage Plug-in for Containers for the Kubernetes/OpenShift environment, you can complete tasks such as creating a storage class ordeleting a persistent volume claim.
Data management tasks for the Kubernetes/OpenShift environment
Task Details
Create a storage class Edit the file: sc-sample.yaml
The storage class contains storageinformation.
Create a persistent volume claim Edit the file: pvc-sample-yaml
One persistent volume claim for eachvolume is required.
Delete a persistent volume claim Delete a persistent volume claim andpersistent volume.
Usage restrictions for a persistent volume claim
Task Notes
Create a persistent volume claim If a failure occurs when creating apersistent volume claim, a persistentvolume claim object will be created butnot with the persisent volume.
In this case, delete the persistent volumeclaim object using the following command:
kubectl delete pvc <pvc name>
Delete a persistent volume claim If a failure occurs when deleting apersistent volume claim, a persistentvolume claim object will be deleted butthe persistent volume object will remainand any storage asset associated with thepersistent volume object may also remain.
Manage data
Chapter 3: Kubernetes/OpenShift framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 33
Task Notes
In this case, delete the persistent volumeusing the following command:
kubectl delete pv <pv name>Also, delete the storage asset (LDEV). Referto the storage manual for yourenvironment for details.
About configuring storage and host group settings
The StorageClass.yaml file contains storage, host group, and other settings that arenecessary for Storage Plug-in for Containers to work with your environment.
Caution: The username and password written in storage class can be seenfrom any namespace or project.
The following table provides information about the required parameters that areassociated with this file.
About configuring storage and host group settings
Chapter 3: Kubernetes/OpenShift framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 34
Parameter references for StorageClass.yaml
Parameters Description
Kind: StorageClass -
apiVersion: storage.k8s.io/v1 -
metadata: -
name: sc-sample StorageClass Name
provisioner: hitachi.io/hspc Provisioner Name
parameters: -
serialNumber: "54321" Storage serial number
url: 172.16.1.1:23450 Storage URL*
user: User01 Storage user ID
password: Password01 Storage password
poolID: "1" HDP Pool ID
snapshotpoolID: "2" Snapshot Pool ID
This parameter is necessary for clonemode. From SVOS 7.2 or later, you canuse HDP Pool and Snapshot Pool asthe same pool.
You can ignore this parameter for anFC configuration.
* IPv6 is not supported.
About configuring persistent volume claim settings
The Persistent Volume Claim file, PVC.yaml, contains volume information that is used byStorage Plug-in for Containers to create persistent volumes.
The following table provides information about the required parameters that areassociated with this file.
About configuring persistent volume claim settings
Chapter 3: Kubernetes/OpenShift framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 35
Parameter references for PVC.yaml
Parameter Description
kind: PersistentVolumeClaim -
apiVersion: v1 -
metadata: -
name: pvc-sample PVC Name
hitachi.io/mode: “clone” Provisioning mode
Specify “clone” to create a clonedvolume from “sourceVolID”.
For a new volume, “hitachi.io/mode”itself is not required.
hitachi.io/sourceVolID: “1” Source Volume ID for clone
Obtain this ID by using the command,kubectl describe pv <PV_NAME>,which is displayed in the parameter,hitachi.io/LDEVID.
specifications:
storageClassName: sc-sample StorageClass Name
accessModes: - ReadWriteOnce
resources: requests:
storage: 100M Volume Size
This is ignored in clone mode.
Command references
The following table lists the commands you will use with Storage Plug-in for Containersto create a storage class and a persistent volume claim.
Command references
Chapter 3: Kubernetes/OpenShift framework
Hitachi Storage Plug-in for Containers Quick Reference Guide 36
Commands for use with Kubernetes/OpenShift
Command Usage
# kubectl create -f sc-sample.yaml Create a storage class.
# kubectl create -f pvc-sample.yaml Create a persistent volumeclaim.
# kubectl delete <pvc name> Delete a persistent volumeclaim.
The persistent volumes for Kubernetes that are created by Storage Plug-in for Containersuse the parameters that are outlined in the following table. You can view applied valuesin your environment using the following command:
# kubectl describe pv <pv name>
Parameter references
Parameter Notes
Name: pvc-<UUID> --
Labels: <none>
hitachi.io/LDEVID=123 LDEV ID
hitachi.io/mode=normal Provision mode
hitachi.io/provisionerName=Hitachi Provisioner name
When needed, you can uninstall Storage Plug-in for Containers from the Docker or theKubernetes/OpenShift framework.
Uninstall Storage Plug-in for Containers from the Dockerframework
The following instructions describe how to uninstall Storage Plug-in for Containers fromthe Docker framework, which includes removing any containers, volumes, snapshots,and the plug-in.
Procedure
1. Stop all containers/services which are using the volumes created by Storage Plug-infor Containers.
2. Check the volume name: docker volume ls3. Check if the specified volume has a snapshot:
docker volume inspect <VOLUME_NAME>■ If it exists, proceed to step 4.
■ If it does not exist, proceed to step 5.
4. Delete the snapshot:hctl snapshot rm -sourceVolName <VOLUME_NAME> -mu <MU#>
5. Delete the volume:docker volume rm <VOLUME_NAME>
Note: If a clone copy is currently in process, wait until the processcompletes before deleting the volume.
6. Complete the following steps on all hosts:a. Check the driver name: docker plugin lsb. Disable the plug-in:
docker plugin -f disable <DRIVER_NAME>:<TAG>
Note: If you use your own registry, add the registry IP address andport to the driver name.
c. Remove the plug-in:
Chapter 4: Uninstall Storage Plug-in for Containers
Hitachi Storage Plug-in for Containers Quick Reference Guide 41
docker plugin -f rm <DRIVER_NAME>:<TAG>
Note: If you use your own registry, add the registry IP address andport to the driver name.
Uninstall Storage Plug-in for Containers from theKubernetes/OpenShift framework
The following instructions describe how to uninstall Storage Plug-in for Containers fromthe Kubernetes framework, which includes removing any Persistent Volume Claims,persistent volumes, storage classes, Storage Plug-in for Containers pods, BINDs andservice accounts.
Note: In the following procedure, replace kubectl with oc for the OpenShiftenvironment.
Procedure
1. Delete all pods which are using the volumes created by Storage Plug-in forContainers.
2. Check the Persistent Volume Claim (PVC) name: kubectl get pvc3. Delete the Persistent Volume Claim created by Storage Plug-in for Containers:
kubectl delete pvc <PVC_NAME>4. Check the storage class (SC) name: kubectl get sc5. Delete the storage class created by Storage Plug-in for Containers:
kubectl delete sc <SC_NAME>6. Delete the plug-in pod: kubectl delete pod hspc-pod7. Delete the plug-in bind: kubectl delete clusterrolebinding hspc-bind8. Delete the plug-in service account: kubectl delete sa hspc-sa9. Remove the plug-in container image on all nodes:
docker rmi hspc:latest hspc:1.0.0
Uninstall Storage Plug-in for Containers from the Kubernetes/OpenShift framework
Chapter 4: Uninstall Storage Plug-in for Containers
Hitachi Storage Plug-in for Containers Quick Reference Guide 42
This chapter describes the error messages that may be returned while runningcommands. It also provides guidelines for troubleshooting volume failover.
Error codes
The "Error code information" table lists the error codes that may be returned whilerunning commands. Each error code includes the error details and suggestedworkaround. Errors are recorded to the log file that is applicable to the frameworkoption that your site implemented.
Error logs and location
Log file Location
Docker Swarm as the container orchestrator: hspc-d.log /opt/hitachi/hspc/log/
Kubernetes as the container orchestrator: hspc-k.log
Storage Plug-in specific command: hctl.log
Chapter 5: Troubleshoot Storage Plug-in for Containers
Hitachi Storage Plug-in for Containers Quick Reference Guide 43
Error code information
Error Code Cause Solution
HSPC0x00001001 - -
HSPC0x00001002 the storage device waslocked by another user
wait until the storage deviceis unlocked, then try theoperation again
HSPC0x00001003 the LDEV is already pairedas a local pair
delete the pair first
HSPC0x00001004 the LDEV is not defined check if the LDEV is defined
HSPC0x00001005 an invalid size wasspecified
check if the size unit is anyone of the following: g,m,kor t
HSPC0x0000100f a time out occurred whilewaiting for the REST APIjob complete state
wait until the job completes
HSPC0x00001010 Unknown ASYNC APIfailure
try again
HSPC0x00001011 rollback operation failed -
HSPC0x00001013 Either there is no free LDEVin the storage, or thedatabase of HitachiConfiguration Manager isnot ready.
delete unnecessary LDEVsin the storage or refresh theHitachi ConfigurationManager database
HSPC0x00001016 a volume that is processinga clone or snapshot cannotbe deleted
if the clone process isrunning, wait for theprocess to complete. If asnapshot of the volume is inprocess, delete thesnapshot first using the"hctl snapshot rm"command, then try again
Error codes
Chapter 5: Troubleshoot Storage Plug-in for Containers
Hitachi Storage Plug-in for Containers Quick Reference Guide 44
Error code information
Error Code Cause Solution
HSPC0x00001017 a time out occurred whilewaiting for SMPL
wait until the pair statuschanges to SMPL
HSPC0x00001018 the specified volumecannot be removed
check if the specifiedvolume can be removed
HSPC0x00001019 the specified storage is notsupported
check if the specifiedstorage is supported
HSPC0x0000101a the REST server isunavailable
wait until the REST server isready
HSPC0x0000101b an invalid HTTP status wasreceived
check if the REST server isoperating normally
HSPC0x0000101c an invalid port is specified check if the specified porthas the TARGET attribute
HSPC0x0000101d the specified SCSI targetname already exists
specify a name that doesnot exist
HSPC0x0000101e the specified resourcedoes not exist in thestorage device.
check if the specifiedresource exists
HSPC0x0000101f Either the DP volume isbeing expanded or thestorage is busy
wait until the DP volumehas finished expanding,then try again
HSPC0x00001020 target volume and storageserial number mismatch
check the target storage inconfig.json
HSPC0x00001021 the specified pair is aninvalid pair status
check the status of thespecified pair
HSPC0x00001022 the specified volumerejected the operationbecause of anincompatible combinationof storage features
-
HSPC0x00002004 the config.json file doesnot exist
check if config.json exists
HSPC0x00002005 a storage serial number isnot specified in config.json
specify a storage serialnumber in config.json
HSPC0x00002006 a storage URL is notspecified in config.json
specify a storage URL inconfig.json
Error codes
Chapter 5: Troubleshoot Storage Plug-in for Containers
Hitachi Storage Plug-in for Containers Quick Reference Guide 45
Error code information
Error Code Cause Solution
HSPC0x00002007 a storage user is notspecified in config.json
specify a storage user inconfig.json
HSPC0x00002008 a storage password is notspecified in config.json
specify a storage passwordin config.json
HSPC0x00002009 a DP pool ID is notspecified in config.json
specify a DP pool ID inconfig.json
HSPC0x0000200a a snapshot pool ID is notspecified in config.json
specify a snapshot pool IDin config.json
HSPC0x0000200b a port ID is not specified inconfig.json
specify a port ID inconfig.json
HSPC0x0000200c a SCSI target number is notspecified in config.json
specify a SCSI targetnumber in config.json
HSPC0x0000200d a database (etcd) URL isnot specified in config.json
specify a database (etcd)URL in config.json
HSPC0x0000200e the config.json file containsan invalid key valuecombination
check config.json for the keyinformation and change it, ifneeded
HSPC0x00002011 the specified URL inconfig.json is invalid
YYYY is the year, MM is the month, DD is the date, hh is the hour, mm is the minute, andss is the second. For example, a log file titled log-09112017-105527.log was created onSeptember 11, 2017 at 10:55:27.
Note: Old logs are deleted if the total number of current and old log files ismore than 10.
Example log output
2017/11/09 16:32:15.012[12345][0052][ERROR]docker-driver/hrd.(*StorageDevice).createNormalVolume(215) [HSPC0x00001005] invalid size specified :check if the size unit is either g, m or k
Name of function in source code whereerror occured in case of [ERROR].
(215) Line number of source code where erroroccured in case of [ERROR].
[HSPC0x00001005] Error code
invalid size specified Root cause of the error.
check if the size unit is either g, m or k Possible solution for the error.<RootCause>:<PossibleSolution>
Volume failover
When a host joining Docker Swarm is down, containers failover to other hosts. Thepersistent volumes created by Storage Plug-in for Containers will also failover to thesame host. At that time, a snapshot of the volume is automatically created by StoragePlug-in for Containers to protect data before the failover.
To recover data before the failover, use the hctl snapshot resotre command.
Volume failover
Chapter 5: Troubleshoot Storage Plug-in for Containers
Hitachi Storage Plug-in for Containers Quick Reference Guide 53