Managing Cloud Hardware Alfie Lew, Inspur John Leung, Intel Sai Dasari, Facebook
Manag ing C loud Hardware
Alfie Lew, Inspur
John Leung, Intel
Sai Dasari, Facebook
2
Agenda
• Hardware Management Overview
• Leverage Existing Standards
• Open Source Projects
• OpenBMC firmware
• Open Source Firmware
• OpenRMC
• Summary
• Q&A
Open Software
Open Hardware
Open Firmware
Hardware Management (Legacy)
• In-band
⎻ Uses main CPU, primary
memory, etc.
• Off-platform
⎻ Manageability of the
system to a remote client
⎻ IPMI over RMCP+
Platform
Out-of-band
(IPMI)
BMC
OS
NIC
NC-SI
CPUBIOS
BMC FW
SW
Agent
In-band
device
device
IPMI
Remote
Client
Issues:
• BMC firmware is not open
• Security concerns with RMCP+
Out-of-band
▪ Uses controller (BMC)
▪ May operate at low power
states
On-platform
▪ Manageability of devices
within the platform
▪ IPMI over various
protocols and connections
On-platformother
DMTF Manageability Standards
• Off-platform (Redfish)
⎻ A RESTful interface for manageability (HTTP, JSON)
⎻ Includes resource models for systems, chassis, managers. Other SDO's have extended Redfish to manage other type of platforms and services
• On-platform (PLDM over MCTP)
⎻ MCTP - A route-able transport protocol for manageability messages (connection independent)
⎻ PLDM - A bit-based request/response command model
⎻ RDE - A PLDM model designed for devices to support Redfish semantics
Platform
Out-of-band
(Redfish)
BMC
OS
NIC
NC-SI
CPUBIOS
BMC FW
SW
Agent
In-band
device
device
PLDM
MCTP
Remote
Client
other
Redfish Overview
⎻ Uses cloud/web protocols, structures, security models
• HTTP, JSON (leverage existing tool chains)
⎻ Resource schemas are accessible, to enable interface programmatically
• json-schema, OpenAPI/swagger schema
rawData = urllib.urlopen(‘https://<ip_addr>/redfish/v1/Systems/CS_1’
jsonData = json.loads(rawData)
print( jsonData[‘SerialNumber’] )
1A87CA442KOutput
Python
code
HTTP GET https://<ip_addr>/redfish/v1/Systems/CS_1HTTP/S
6
Redfish Computer System Model
HTTP GET /redfish/v1/Systems/CS_1/Processors/2
Root /redfish/v1
Processors
Disks
NICs
Power
Thermal
LogService
NW protocol
Sessions
Accounts
Events
Registries
Collection Resource Single ResourceService Root
Schemas
Tasks
Collection of Chassis
"Physical view"
/redfish/v1/Chassis
Collection of Systems
"Logical view"
/redfish/v1/Systems
Collection of Managers
"Mgmt hierarchy"
/redfish/v1/Managers
BMC
./Managers/<id>
Computer
System
./Systems/<id>
./Chassis/<id>
Managed By
Computer Systems
Chassis
Platform
HW Mgmt
ComputeMemory
Managing Beyond Computer Systems
The Redfish Forum works with other SDOs to
create models for new management domains
⎻ Storage services and networked storage
⎻ Ethernet Switch - map YANG to Redfish
⎻ Industrial IoT
⎻ Telco - Customer Premise Equipment
⎻ DCIM - DC Infrastructure Mgmt (e.g. CRACs,
PDUs, chillers)
System Storage Network
Aug 2015 Aug 2016
YANG
Internet Draft 2017
DCIM (facilities)
Telco
Redfish Client
Redfish Off-platform interface
(Starfish)
IIoT
With the number of models increasing, how would OCP
specify the required manageability of OCP platforms?
OCP Platform Manageability Direction
• The OCP Hardware Management project
⎻ Will specify the manageability common across OCP
platforms
⎻ Has approved the "OCP Baseline Hardware
Management Profile"1
• Other OCP projects
⎻ Will create platform specific profiles
⎻ Profiles will extend the "Baseline Hardware
Management Profile"
⎻ "OCP Server Profile" is available
Hardware Mgmt Baseline Profile
Server
Profile(s
)
Storage
Profile(s
)
Network
Profile(s
)
Rack &
Power
Profile(s)
Telco
Profile(s
)
Interface
• OCP will use Redfish profiles to specify platform manageability
1opencompute.org/wiki/Hardware_Management/SpecsAndDesigns
OCP San Jose Compute Node
➢ Successfully ran the Redfish Interop Validator testing against the "OCP Baseline
Hardware Management Profile"
➢ Demonstrated at OCP Summit (March 2018)
OCP Server Profile
Enabling Interoperable Manageability
• OCP will prescribe platform manageability in machine readable JSON profile
• Implementers can use the profile to test conformance of an implementation
• Linux Foundation governs the open-source implementation of BMC firmware (OpenBMC)
10
OCP San Jose Compute Node
➢ Incorporated OpenBMC
➢ The implementation passed the OCP Baseline HW Manageability Profile
➢ Demo’ed at OCP Regional Summit (Oct 2018, Amsterdam)
➢ Changes will be upstreamed to the OpenBMC repository1
➢ Target Date: March 2019 available on GitHub
1The Linux Foundation project
12
A Brief History of OpenBMC Project
Two independent implementations
Facebook IBM
Linux Foundation Project(IBM, Google, Intel, Microsoft, Facebook)
13
OpenBMC Development Model
Traditional OpenBMC
BMC FWSource Code
OpenBMCProject
3rd Party BMC FW Vendor
(Aspeed/Nuvoton)
3rd PartyHW Vendor(AIC/PS/NVMe)
3rd Party BMC FW Vendor
3rd PartyHW Vendor(AIC/PS/NVMe)
3rd Party BMC HW Vendor(Aspeed/Nuvoton)
BMC FWSource Code
3rd Party BMC FW Vendor
ODMServer Vendor
14
OpenBMC Architecture
BMC SoC / SPI Flash / TPM
Boot Loader (e.g. U-Boot)
Linux Kernel/Drivers
Hardware
Bootloader
& Kernel
Applications
bridgebridge
D-BUS
CHASSISCONTROL
CONFIGURATIONMANAGER
FIRMWAREUPDATE
RMCP+ENDPOINT
FSC BT MBOXEWS
ENDPOINT
REDFISHENDPOINT
KCS IPMI
SYSTEMD
GPIO/ADC
IPMB
INVENTORYMANAGER
15
Additional Resources
• Source Code:
• https://github.com/openbmc
• Continuous Integration Testing
• https://openpower.xyz
• Code Reviews
• http://gerrit.openbmc-project.xyz
• Contact
• Mail: [email protected]
• IRC: #openbmc on freenode.net
• Riot: #openbmc:matrix.org
• Web Page:
• www.openbmc.org
Collaborative Community development model
Infrastructure,Tools &Eco-system support
Mission: Develop an open source philosophy based ‘system firmware’ (BIOS) modules, to support different Operating Systems and different CPU silicon vendors.
Major Companies Contributing to OSF development: Microsoft, Intel, Google, Facebook, Lenovo, IBM, Two Sigma, ITRenew, 9 Elements, Cavium, AMD and many more.
Major Work Streams: Open EDKII DXE core, Linux Boot, Core Boot, Silicon Interface Firmware Module, Intel FSP, AMD AGESA, ARM boot code, HW platform module, Build tools, Automated test support, HW requirements, etc.
GitHub Repositories Collateral link:https://github.com/opencomputeproject/OSF
Bi-weekly OSF discussions : Architectural reviews, workstream progress, design reviews, agenda setting and other collaborative discussions on OSF development. Contact the following to subscribe, participate and contribute:[email protected]@[email protected]
Proposed OpenRMC sub-project
• Rack
⎻ The rack is the fundamental building block for the Data Center
• Rack Manager Controller (RMC)
⎻ The RMC is fundamental to Data Center Management hierarchy
⎻ The RMC can be located standalone, within power shelf, or within the switch
• Opportunity
⎻ Common language/model between RMC and nodes (aggregation, etc.)
⎻ Provide an open source reference RMC
nodes
18
Open Source Firmware for OCP Ecosystem
OCP Design Collateral
Firmware(Build Meta Data)
3D/CAD FilesDesign
Packages
Binary Blobs
SpecificationsPresentations
Data SheetsSchematics
Product/Company Specific Source Repo
OCP Specific Source Repo(e.g. LED Config, Redfish
Profiles)
Open Source Community Repo(e.g. OpenBMC, OpenRMC, LinuxBoot,
Coreboot, OpenEDKII)