Sun Fire Midrange Server Maintenance

Sun Services

Sun Fire™ Midrange Server Maintenance

Sun Fire™ Midrange ServerMaintenance

SM-340

Copyright 2004 Sun Microsystems, Inc. 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.

This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or document maybe reproduced in any form by any means without prior written authorization of Sun and its licensors, if any.

Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.

Sun, Sun Microsystems, the Sun logo, Java, Netra, OpenBoot, Solaris, Sun Enterprise, Sun Fire, Sun HPC Cluster Tools, Sun Java, and Sun StorEdge are trademarks or registeredtrademarks of Sun Microsystems, Inc. in the U.S. and other countries.

All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Products bearing SPARCtrademarks are based upon an architecture developed by Sun Microsystems, Inc.

UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd.

U.S. Government approval might be required when exporting the product.

RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) and FAR 52.227-19(6/87), or DFAR 252.227-7015(b)(6/95) and DFAR 227.7202-3(a).

DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTYOF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS AREHELD TO BE LEGALLY INVALID.

THIS MANUAL IS DESIGNED TO SUPPORT AN INSTRUCTOR-LED TRAINING (ILT) COURSE AND IS INTENDED TO BE USED FOR REFERENCE PURPOSES IN CONJUNCTIONWITH THE ILT COURSE. THE MANUAL IS NOT A STANDALONE TRAINING TOOL. USE OF THE MANUAL FOR SELF-STUDY WITHOUT CLASS ATTENDANCE IS NOTRECOMMENDED.

edited 02/08 by LG

Copyright 2004 Sun Microsystems Inc. 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés.

Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution, et la décompilation. Aucune partie de ceproduit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sans l’autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y en a.

Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licencié par des fournisseurs de Sun.

Sun, Sun Microsystems, le logo Sun, Java, Netra, OpenBoot, Solaris, Sun Enterprise, Sun Fire, Sun HPC Cluster Tools, Sun Java, et Sun StorEdge sont des marques de fabrique ou desmarques déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays.

Toutes les marques SPARC sont utilisées sous licence sont des marques de fabrique ou des marques déposées de SPARC International, Inc. aux Etats-Unis et dans d’autres pays. Lesproduits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems, Inc.

UNIX est une marques déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.

L’accord du gouvernement américain est requis avant l’exportation du produit.

LA DOCUMENTATION EST FOURNIE “EN L’ETAT” ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENTEXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, AL’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON

Ce manuel de référence doit être utilisé dans le cadre d’un cours de formation dirigé par un instructeur (ILT). Il ne s’agit pas d’un outil de formation indépendant. Nous vousdéconseillons de l’utiliser dans le cadre d’une auto-formation.

Sun Fire™ Midrange Server Maintenance ivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Course Contents

About This Course ..................................................................................... Preface-xivCourse Goals ...............................................................................................................................Preface-xvCourse Map ...............................................................................................................................Preface-xviiTopics Not Covered ............................................................................................................... Preface-xviiiHow Prepared Are You? ..........................................................................................................Preface-xixIntroductions .............................................................................................................................. Preface-xxIcons ............................................................................................................................................Preface-xxiTypographical Conventions ...................................................................................................Preface-xxiiAdditional Conventions .........................................................................................................Preface-xxiv

Introducing the Sun Fire™ Midrange Servers ....................................................... 1-1Objectives ................................................................................................................................................. 1-2Relevance .................................................................................................................................................. 1-3Sun Fire Midrange Server Product Goals ............................................................................................ 1-4Finding Sun Fire Midrange Server Information ................................................................................. 1-8Sun Fire Midrange Server Models ........................................................................................................ 1-9Sun Fire V1280/E2900 Server .............................................................................................................. 1-10Sun Fire 3800 Server ............................................................................................................................. 1-11Sun Fire 4800/E4900 Server ................................................................................................................. 1-12Sun Fire 4810 Server ............................................................................................................................. 1-13Sun Fire 6800/E6900 Server ................................................................................................................. 1-14Sun Fire E7900 Server Cluster Interconnect ...................................................................................... 1-15Sun Fire Midrange Server I/O Support ............................................................................................. 1-16Sun Fire Midrange Server Features .................................................................................................... 1-17System Controllers ................................................................................................................................ 1-18Platform Segmentation and Dynamic System Domains ................................................................. 1-19

Sun Services

Sun Fire™ Midrange Server Maintenance vCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Multipathed I/O ................................................................................................................................... 1-20Dynamic Reconfiguration (DR) .......................................................................................................... 1-21Platform Startup and Shutdown ......................................................................................................... 1-22

Field-Replaceable Units ........................................................................................... 2-1Objectives ................................................................................................................................................. 2-2Relevance .................................................................................................................................................. 2-4Administrative and Service Overview ................................................................................................ 2-5Sun Fire System Field-Replaceable Units ............................................................................................ 2-7FRU Administration ............................................................................................................................. 2-11Sun Fire V1280/E2900 Server IB System Controller Board ............................................................ 2-14Sun Fire Midrange Server System Controller Board ....................................................................... 2-15System Controller Physical Locations ................................................................................................ 2-17Sun Fire V1280/E2900 Server IB_SSC SC Board Faceplate ............................................................. 2-22Sun Fire Midrange Server System Controller Board Faceplate ...................................................... 2-23Sun Fire Midrange Server SC Board LED Definitions ..................................................................... 2-24Sun Fire V1280/E2900 Server System Boards ................................................................................... 2-25Sun Fire 3800, 4800/E4900, 4810, and 6800/E6900 Server System Boards ................................... 2-27Memory Configuration Rules ............................................................................................................. 2-28CPU and Memory Locations ............................................................................................................... 2-29Processor Memory Group Bank Assignments .................................................................................. 2-30Sun Fire V1280/E2900 Server System Board Slot Assignments .................................................... 2-31Sun Fire 3800 System Board Slot Assignments ................................................................................. 2-32Sun Fire 4810 and 4800/E4900 Server System Board Assignments .............................................. 2-33Sun Fire 6800/E6900 Server System Board Slot Assignments ....................................................... 2-34System Board LEDs .............................................................................................................................. 2-35Sun Fire Midrange I/O Assemblies ................................................................................................... 2-36Sun Fire V1280/E2900 Server Six-Slot PCI IB_SSC Assembly ....................................................... 2-37Sun Fire 4800/E4900, 4810, and 6800/E6900 Server Four-Slot cPCI I/O Assembly ................... 2-38Sun Fire 3800 Server Six-Slot cPCI I/O Assembly ........................................................................... 2-39

Sun Services

Sun Fire™ Midrange Server Maintenance viCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Sun Fire 4800/E4900, 4810, and 6800/E6900 Server Eight-Slot PCI I/O Assembly ................... 2-40Sun Fire V1280/E2900 Server I/O Assembly Location ................................................................... 2-41Sun Fire 3800 Server I/O Assembly Locations ................................................................................. 2-42Sun Fire 4800/E4900 Server I/O Assembly Locations .................................................................... 2-43Sun Fire 4810 Server I/O Assembly Locations ................................................................................ 2-44Sun Fire 6800/E6900 Server I/O Assembly Locations ................................................................... 2-45Eight-Slot PCI I/O Assembly Slot Locations and LEDs .................................................................. 2-46Eight-Slot Assembly Electrical Characteristics ................................................................................ 2-47Six-Slot cPCI I/O Assembly Slot Locations and LEDs .................................................................... 2-48Six-Slot cPCI I/O Slot Electrical Characteristics ............................................................................... 2-49Four-Slot cPCI I/O Assembly Slot Locations and LEDs ................................................................. 2-50Four-Slot cPCI I/O Slot Electrical Characteristics ............................................................................ 2-51PCI and cPCI I/O Adapters ................................................................................................................ 2-52Sun Fire V1280/E2900 Server Sun Fireplane Switchboard ............................................................. 2-54Sun Fire 4800/E4900, 4810, and 6800/E6900 Server Sun Fireplane Switchboard ....................... 2-55Sun Fireplane Switchboard Physical Locations ................................................................................ 2-56Sun Fireplane Switchboard LEDs ....................................................................................................... 2-58Sun Fire V1280/E2900 Server Baseplane .......................................................................................... 2-59System Configuration Card Reader (SCCR) .................................................................................... 2-60Sun Fire 6800/E6900 Server Centerplane and ID Board (Front View) ......................................... 2-61Sun Fire 6800/E6900 Server Centerplane and ID Board (Rear View) ........................................... 2-62ID Board ................................................................................................................................................. 2-63Replacing a Centerplane or ID Board ................................................................................................ 2-65ID Board MAC Addresses ................................................................................................................... 2-66AC Power Distribution ......................................................................................................................... 2-67Sun Fire 4810 and 4800/E4900 Server AC Component Locations ................................................. 2-69Sun Fire 6800/E6900 Server AC Component Locations .................................................................. 2-70RTU and RTS ......................................................................................................................................... 2-71Redundant Transfer Unit Panel .......................................................................................................... 2-72Redundant Transfer Unit LED Functions ......................................................................................... 2-73AC Input Box ......................................................................................................................................... 2-74

Sun Services

Sun Fire™ Midrange Server Maintenance viiCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

DC Power Distribution ......................................................................................................................... 2-75Sun Fire V1280/E2900 Server DC Power Distribution .................................................................... 2-76Sun Fire 3800 Server DC Power Distribution ................................................................................... 2-77Sun Fire 4800/E4900 Server DC Power Distribution ....................................................................... 2-78Sun Fire 4810 Server DC Power Distribution ................................................................................... 2-79Sun Fire 6800/E6900 Server DC Power Distribution ....................................................................... 2-80Power Grid Slot Assignments ............................................................................................................. 2-81DC Power Supplies ............................................................................................................................... 2-82DC Power Supply Locations ............................................................................................................... 2-83Sun Fire Midrange Server Fan Trays and Blower Assemblies ....................................................... 2-88Fan Tray Locations ................................................................................................................................ 2-89Fan Tray Assembly LEDs .................................................................................................................... 2-94FrameManager Cable Diagram ........................................................................................................... 2-95FrameManager Cap Front Panel ......................................................................................................... 2-96Sun StorEdge™ D240 Media Tray ...................................................................................................... 2-97Sun StorEdge D240 Media Tray Rear Panel ...................................................................................... 2-98Full-Bus Configuration SCSI ID Assignments .................................................................................. 2-99Full SCSI Bus Configuration Options .............................................................................................. 2-100Split SCSI Bus ...................................................................................................................................... 2-101Typical Split SCSI Bus SCSI ID Assignments ................................................................................. 2-102Sun StorEdge D240 Media Tray Status LEDs ................................................................................. 2-103Media Tray Status LED Descriptions ............................................................................................... 2-104Media Tray Power Supply LEDs ...................................................................................................... 2-105Media Tray Power Supply LED States ............................................................................................ 2-106Installing the Administration Console ............................................................................................. 2-107System Controller Patch Panel ......................................................................................................... 2-108Accessing the Platform Shell ............................................................................................................. 2-109Sun Fire Midrange Server Installation ............................................................................................. 2-110Rackmounting an Additional Sun Fire 3800 and 4800/E4900 Server ......................................... 2-111

Sun Services

Sun Fire™ Midrange Server Maintenance viiiCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Platform Assessment and Management ................................................................ 3-1Objectives ................................................................................................................................................. 3-2Relevance .................................................................................................................................................. 3-6Sun Fire V1280/E2900 Server Platform Assessment and Management ......................................... 3-7Lights-Out Management (LOM) ........................................................................................................... 3-8LOM Shell ................................................................................................................................................ 3-9LOM Shell Commands ......................................................................................................................... 3-10The help Command ............................................................................................................................. 3-11Connecting to the LOM Shell .............................................................................................................. 3-12The shownetwork Command .............................................................................................................. 3-13The setupnetwork Command ............................................................................................................ 3-14The logout Command ......................................................................................................................... 3-15Navigating Between Shell Environments on the Sun Fire V1280/E2900 Server ......................... 3-16The showescape Command ................................................................................................................ 3-17The password Command .................................................................................................................... 3-18The showsc Command ......................................................................................................................... 3-19The setupsc command ........................................................................................................................ 3-20Managing the LOM Time-of-Day (TOD) ........................................................................................... 3-22The bootmode Command .................................................................................................................... 3-23LOM Platform Monitoring Functions ................................................................................................ 3-24The showboards Command ................................................................................................................ 3-25The showcomponent Command ......................................................................................................... 3-31The inventory Command .................................................................................................................. 3-34The showenvironment Command ..................................................................................................... 3-35The history Command ....................................................................................................................... 3-36The showlogs Command .................................................................................................................... 3-37The showlocator Command .............................................................................................................. 3-38Sun Fire V1280/E2900 Server Power Operations ............................................................................. 3-39LOM poweron Command .................................................................................................................... 3-40LOM shutdown Command .................................................................................................................. 3-41

Sun Services

Sun Fire™ Midrange Server Maintenance ixCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

LOM poweroff Command .................................................................................................................. 3-42Power-Cycling the Sun Fire V1280/E2900 Server Using the Power Rocker Switch ................... 3-43Power-Cycle Operations ...................................................................................................................... 3-45Sun Fire 3800, 4800/E4900, 4810, and 6800/E6900 Server Platform Assessment and

Management ....................................................................................................................................... 3-46Platform Shell Commands ................................................................................................................... 3-47The help Command ............................................................................................................................. 3-48Connecting to the System Controller Shells ...................................................................................... 3-49Initiate a Remote Connection With SSH ............................................................................................ 3-50Initiate a Remote Connection With Telnet ........................................................................................ 3-51Navigating Between Shells on the Sun Fire 3800, 4800/E4900, 4810, and 6800/E6900

Servers ................................................................................................................................................. 3-52Managing Shell Passwords .................................................................................................................. 3-53Console Command Considerations .................................................................................................... 3-54Platform and System Controller Configuration ............................................................................... 3-56Configuring the Platform TOD ........................................................................................................... 3-59Viewing System Controller Details .................................................................................................... 3-60Viewing the SC Message Logs ............................................................................................................ 3-61Viewing System Controller Connections .......................................................................................... 3-62Viewing System Controller Command History ............................................................................... 3-63System Controller Management ......................................................................................................... 3-64System Controller Configuration ....................................................................................................... 3-65Rebooting the System Controller ........................................................................................................ 3-66System Controller Failover .................................................................................................................. 3-69System Controller Failover Prerequisites .......................................................................................... 3-70Failover ................................................................................................................................................... 3-71Controlling System Controller Failover Behavior ............................................................................ 3-72Determining the System Controller Failover State .......................................................................... 3-76Platform Assessment ............................................................................................................................ 3-77Assessing the Platform Configuration ............................................................................................... 3-78The showplatform Command ............................................................................................................ 3-79

Sun Services

Sun Fire™ Midrange Server Maintenance xCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Viewing Platform Component Status ................................................................................................ 3-84The showboards Command ................................................................................................................ 3-85Viewing Platform Component Details ............................................................................................... 3-91The showcomponent Command ......................................................................................................... 3-92Powering On and Off System Components ...................................................................................... 3-95Updating the Platform Firmware ....................................................................................................... 3-96Using the Root or a User Account to Flash Update the System ..................................................... 3-97Introducing Segments and Domains .................................................................................................. 3-99Segments .............................................................................................................................................. 3-100Domains ................................................................................................................................................ 3-101Sun Fire 6800/E6900 Servers Configured With Four Domains ................................................... 3-102Server Configuration Domain IDs .................................................................................................... 3-103Segment and Domain Configurations ............................................................................................. 3-105Domain Access Control List (ACL) .................................................................................................. 3-107Configuring ACLs ............................................................................................................................... 3-108Viewing ACLs ..................................................................................................................................... 3-109Starting, Stopping, and Power-Cycling Domains .......................................................................... 3-110Introducing Device Configuration ................................................................................................... 3-111OpenBoot PROM Capabilities ........................................................................................................... 3-113Device Tree ........................................................................................................................................... 3-115Sun Fire V1280/E2900 Server Device Tree Components .............................................................. 3-116Sun Fire 3800, 4800/E4900, 4810, and 6800/E6900 Server Device Tree Components .............. 3-117Mapping Node Devices ...................................................................................................................... 3-118CPU and Memory AID Assignments ............................................................................................... 3-121Mapping I/O Devices ........................................................................................................................ 3-122Decoding IOC AID ............................................................................................................................. 3-124IOC AID Assignments ........................................................................................................................ 3-126IOC PCI Bus Offset ............................................................................................................................. 3-127Device Number ................................................................................................................................... 3-128Sun Fire V1280/E2900 Server Six-Slot PCI Chassis ....................................................................... 3-129Sun Fire 4800/E4900, 4810, and 6800/E6900 Server Eight-Slot PCI Chassis ............................. 3-130

Sun Services

Sun Fire™ Midrange Server Maintenance xiCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Sun Fire 3800 Server Six-Slot cPCI Chassis ..................................................................................... 3-131Sun Fire 4800/E4900, 4810, and 6800/E6900 Server Four-Slot cPCI Chassis ............................. 3-132

Troubleshooting the Sun Fire Midrange Servers .................................................. 4-1Objectives ................................................................................................................................................. 4-2Relevance .................................................................................................................................................. 4-6Additional Resources ............................................................................................................................. 4-7Operational Overview ............................................................................................................................ 4-9Sun Fireplane Interconnect Operational View ................................................................................ 4-10Functionality .......................................................................................................................................... 4-11Snoopy Bus Architecture ..................................................................................................................... 4-12Sun Fire Servers’ Logical Architecture .............................................................................................. 4-13Sun Fire 6800/E6900 Server Crossbar Configuration ...................................................................... 4-14UltraSPARC IV Processor Chip Block Diagram .............................................................................. 4-15System Board Logical Block Diagram ................................................................................................ 4-16IB_SSC PCI I/O Assembly Logical Block Diagram ......................................................................... 4-17Eight-Slot PCI I/O Assembly Logical Block Diagram ..................................................................... 4-18Compact PCI I/O Assemblies ............................................................................................................. 4-19Four-Slot cPCI I/O Assembly Logical Block Diagram ................................................................... 4-20Six-Slot cPCI I/O Assembly Logical Block Diagram ....................................................................... 4-21Sun Fireplane Interconnect .................................................................................................................. 4-22Address Interconnect Levels ............................................................................................................... 4-23Data Interconnect Levels ...................................................................................................................... 4-24System Controller Interconnect Logical Block Diagram ................................................................. 4-25Sun Fire V1280/E2900 Server System Controller Logical Block Diagram ................................... 4-26Sun Fire 3800, 4800/E4900, 4810, and 6800/E6900 Server System Controller Block

Diagram ............................................................................................................................................... 4-27Boot Bus Controller (SBBC) ASIC ....................................................................................................... 4-28System Data Controller (SDC) Console Bus Connections .............................................................. 4-29Basic Clock Distribution ....................................................................................................................... 4-30

Sun Services

Sun Fire™ Midrange Server Maintenance xiiCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Fault Analysis Method ......................................................................................................................... 4-31Eight Steps of Fault Analysis and Diagnosis .................................................................................... 4-32Sun Fire Midrange Server Fault Analysis Methodology ................................................................. 4-33Rule of Three Fault Isolation ............................................................................................................... 4-36Gathering Background Information ................................................................................................... 4-37Error Repositories and Commands .................................................................................................... 4-38Sun Explorer Software Data Collector ............................................................................................... 4-40Running Sun Explorer Software on the Sun Fire Midrange Server ............................................... 4-41Viewing a Sun Explorer Software Capture ....................................................................................... 4-42Interpreting Sun Fire Midrange Server LEDs ................................................................................... 4-43LED Status Code Summary ................................................................................................................. 4-44Testing the Platform ............................................................................................................................. 4-50OpenBoot PROM Commands ............................................................................................................. 4-51POST on the Sun Fire V1280/E2900 Server ...................................................................................... 4-55Controlling System Controller POST Behavior ................................................................................ 4-56Controlling OpenBoot PROM POST Behavior ................................................................................. 4-57POST on the Sun Fire 3800, 4800/E4900, 4810, and 6800/E6900 Servers ..................................... 4-59DIMM Memory Errors ......................................................................................................................... 4-63Controlling System Controller POST Behavior ................................................................................ 4-69System Controller testboard Command ......................................................................................... 4-70Sun Fire Interconnect Link Errors ...................................................................................................... 4-71Identifying Suspect FRUs From Error Messages .............................................................................. 4-72Data Parity Coverage From CPU to CPU Through Memory ......................................................... 4-75Parity Detection in the Address Network ......................................................................................... 4-76Parity Protection for Address Interconnects ..................................................................................... 4-77Error Correcting Code Errors .............................................................................................................. 4-78ECC Error Types ................................................................................................................................... 4-80ECC Error Persistence .......................................................................................................................... 4-81Console Port Errors ............................................................................................................................... 4-82Environmental Errors ........................................................................................................................... 4-83Enhanced Availability Features Implemented in Firmware Update 5.15.3 ................................. 4-84

Sun Services

Sun Fire™ Midrange Server Maintenance xiiiCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Diagnosis Engines ................................................................................................................................. 4-86Auto-Diagnosis (AD) Engine .............................................................................................................. 4-87Fault Event and Error Reporting ........................................................................................................ 4-89AD Engine Logs and Records ............................................................................................................. 4-90Decoding AD Engine Diagnosis Messages ....................................................................................... 4-91Viewing AD Engine Diagnosis Messages ......................................................................................... 4-92Automatic Restoration of Stopped Domains .................................................................................... 4-95Identifying Disabled Components ..................................................................................................... 4-97Sun Fire Midrange Server Blacklisting .............................................................................................. 4-98Blacklisting Components ................................................................................................................... 4-100Managing the Blacklist on Sun Fire V1280/E2900 Server ............................................................. 4-101The setls Command ......................................................................................................................... 4-102Domain Shell Operating Messages ................................................................................................... 4-103Recovering From a Hung Domain ................................................................................................... 4-105Verifying the Recovery ....................................................................................................................... 4-106Collecting Data .................................................................................................................................... 4-107Obtaining a Solaris OS Core File ....................................................................................................... 4-111Obtaining Registers ............................................................................................................................ 4-114

Sun Services


Preface

About This Course

dited 02/08 by LG

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xv of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Course Goals

Upon completion of this course, you should be able to:

• Locate online resources for the Sun Fire™ midrangeserver product line, which includes the followingservers:

• Sun Fire V1280 server and Sun Fire E2900 server

• Sun Fire 3800 server

• Sun Fire 4800 server and Sun Fire E4900 server

• Sun Fire 4810 server

• Sun Fire 6800 server and Sun Fire E6900 server

• Describe the server configuration and key features ofeach model in the Sun Fire midrange server line

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xvi of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Course Goals

• Configure the Sun Fire midrange server platforms

• Perform system maintenance-related activities on theSun Fire midrange servers

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xvii of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Course Map

Sun Fire™ Midrange Introduction

Sun Fire Midrange Server Configuration

Sun Fire Midrange Server Troubleshooting

Introducing theField-Replaceable Units

Troubleshooting the

Servers

Sun Fire™ MidrangeServers

and Management

Sun Fire Midrange

Platform Assessment

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xviii of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Topics Not Covered

• Solaris™ Operating System (Solaris OS)administration – Covered in SA-118: Fundamentals ofSolaris™ Operating Environment for SystemAdministrators

• Sun Fire workgroup servers – Covered in SM-258: SunFire™ Workgroup Server Maintenance and Administration

• Sun Fire high-end servers – Covered in ES-421: SunFire™ 15K/12K Server Administration

• Storage maintenance – Covered in ES-255: Sun StorageSystem Administration

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xix of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

How Prepared Are You?

To be sure you are prepared to take this course, can youanswer yes to the following questions?

• Can you create and edit text files using a text editor?

• Can you use a World Wide Web (WWW) browser?

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xx of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Introductions

• Name

• Company affiliation

• Title, function, and job responsibility

• Experience related to topics presented in this course

• Reasons for enrolling in this course

• Expectations for this course

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xxi of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Icons

Additional resources

Discussion

Note

Caution

?!

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xxii of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Typographical Conventions

• Courier is used for the names of commands, files,directories, programming code, programmingconstructs, and on-screen computer output.

• Courier bold is used for characters and numbersthat you type, and for each line of programming codethat is referenced in a textual description.

• Courier italic is used for variables andcommand-line placeholders that are replaced with areal name or value.

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xxiii of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Typographical Conventions

• Courier italic bold is used to represent variableswhose values are to be entered by the student as part ofan activity.

• Palatino italic is used for book titles, new words orterms, or words that are emphasized.

Sun Services

Sun Fire™ Midrange Server Maintenance Preface, slide xxiv of xxivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Additional Conventions

Java™ programming language examples use the followingadditional conventions:

• Courier is used for the class names, methods, andkeywords.

• Methods are not followed by parentheses unless aformal or actual parameter list is shown.

• Line breaks occur where there are separations,conjunctions, or white space in the code.

• If a command on the Solaris OS is different from theMicrosoft Windows platform, both commands areshown.

Sun Services


Module 1

Introducing the Sun Fire™ MidrangeServers

edited 02/08 by LG

Sun Services

Sun Fire™ Midrange Server Maintenance Module 1, slide 2 of 22Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C

Objectives

• List the functional goals of the Sun Fire™ midrangeserver product line

• Locate Sun Microsystems web sites containingimportant Sun Fire midrange server information

• List the server models that comprise the Sun Firemidrange server product line

• Identify the input and output (I/O) components thatthe Sun Fire midrange servers support

• Describe the key features of each Sun Fire midrangeserver model

• Power on and off each Sun Fire midrange server

Sun Services


Relevance

• Which Sun Fire midrange server models are available?

• How is each Sun Fire midrange server model used?

• What are the key features of each Sun Fire midrangeserver model?

Sun Services


Sun Fire Midrange Server Product Goals

• Sun Fire midrange server products are a family ofsymmetric-memory multiprocessor (SMP) systems.

• They are binary-compatible servers for mission-criticaland high-performance computing applications.

• They include many reliability, availability, andserviceability (RAS) features.

Sun Services



Reliability is defined as a measure of how dependable a systemis after you actually use it. More importantly, it is also definedas how intact your data is as it travels from component tocomponent. Reliability features are provided in the Sun Firemidrange server line:

• Parity and error checking code (ECC) technologies

• Power-on self-test (POST)

• Application-specific integrated-circuit built-in self-test(ASIC BIST)

• Environmental monitoring

• Automatic system reconfiguration (ASR)

Sun Services



Availability is defined as up-time, which is the amount of timea system is available for processing transactions. Availabilityfeatures are provided in the Sun Fire midrange server line:

• Redundancy (N + 1)

• Environmental monitoring

• Dynamic reconfiguration

Sun Services



Serviceability is defined as the ease in which a system might beserviced. Serviceability features are provided in the Sun Firemidrange server line:

• Hot-plug and hot-swap components

• Remote administration

• Minimal tool requirements

• Reduced field-replaceable unit (FRU) list

• Keyed components

• OK to Remove light-emitting diode (LED) indicators

Sun Services


Finding Sun Fire Midrange ServerInformation

Sun Microsystems, Inc. has established numerous web sites tohelp you maintain and administer the Sun Fire midrangeserves. These include:

• http://sunsolve.sun.com

• http://www.sun.com/servers/midrange

• http://docs.sun.com

• http://www.sun.com/solutions/blueprints

• http://www.sun.com/products-n-solutions

• http://www.sun.com/solutions/bigadmin

Sun Services


Sun Fire Midrange Server Models

Sun Fire 6800/E6900server

Sun Fire V1280/E2900server

Sun Fire 4810server


Sun Fire 3800server

Sun Services


Sun Fire V1280/E2900 Server

• The Sun Fire V1280/E2900 servers have features fromthe Netra™ server class and Sun Fire 4800/E4900 and4810 midrange servers. It also incorporates RASfeatures.

• The Sun Fire V1280/E2900 servers are highly available,four- to twelve-processor compact servers.

• Up to two Sun Fire V1280/E2900 servers can bemounted in a standard enterprise rack.

Sun Services


Sun Fire 3800 Server

• The Sun Fire 3800 server is a service provider platformthat is a data-center cabinet or rack-mounted, highlyavailable, compact server.

• The Sun Fire 3800 server supports two to eightprocessors.

• Up to three Sun Fire 3800 servers can be mounted in astandard enterprise rack.

Sun Services


Sun Fire 4800/E4900 Server

• The Sun Fire 4800/E4900 servers are highly available,flexible standalone or rack-mounted two- totwelve-processor servers.

• They deliver mainframe-class availability and haveresource management features.

• They are ideal for use as an application, computer, orsmall database server.

Sun Services


Sun Fire 4810 Server

• The Sun Fire 4810 server is a highly available,rack-optimized, two- to twelve-processor server.

• It delivers mainframe-class availability and hasresource management features.

• Packed in 30-inch deep racks, all components arecompletely front-accessible, making it ideal for Internetdata centers.

Sun Services


Sun Fire 6800/E6900 Server

• The Sun Fire 6800/E6900 servers are highly available,large-scale two- to twenty-four-processor servers.

• They offer mainframe-class availability and resourcemanagement features, including fault-isolateddynamic system domains and full hardwareredundancy.

• They are ideal for server consolidation, datawarehousing, data mining, online transactionprocessing (OLTP), and large database applications.

Sun Services


Sun Fire E7900 Server Cluster Interconnect

• The Sun Fire Link hardware and software product isSun’s highest performing cluster interconnect.

• It is available on the Sun Fire 6800/E6900 servers andSun Fire high-end servers.

• This is supported by Sun Java™ System Clustersoftware and Sun HPC ClusterTools™ software. TheSun Fire E7900 server is a bundle consisting of two SunFire E6900 servers, with each having two Sun Fire Linkboards (a total of four).

Sun Services


Sun Fire Midrange Server I/O Support

• Supported I/O interface cards:

• Peripheral component interface (PCI)

• Compact peripheral component interface (cPCI)

• SBus interface cards are not supported

• Support variety of I/O adapters

• Support variety of disks and disk arrays

• Support variety of backup storage devices

Sun Services


Sun Fire Midrange Server Features

The Sun Fire midrange servers provide a number of advancedfeatures usually found on mainframe-class servers. Thesefeatures include:

• Embedded service processors (system controllers)

• Platform segmentation and dynamic system domains

• Multipathed I/O

• Dynamic reconfiguration

• Cluster support

Sun Services


System Controllers

• Sets up the system and coordinates the boot process

• Generates system clocks

• Monitors the environmental sensors

• Analyzes errors and takes corrective action

• Sets up the system partitions and domains

• Provides the system console capabilities

Sun Services


Platform Segmentation and DynamicSystem Domains

L2 Switches

Domain A

Segment 0 Segment 1

Domain B Domain C Domain D

CPUMemory

CPUMemory

CPUMemory

I/O I/O

L2 Switches

CPUMemory

CPUMemory

CPUMemory

I/O I/O

Sun Services


Multipathed I/O

• Sun StorEdge™ Traffic Manager software – Provides ahigh level of disk availability and performance usingmultipath access to I/O devices. This was formerlyknown as Multiplexed IO or MPxIO.

• Internet Protocol Multipathing – Provides a high levelof network availability and performance usingautomatic failover and load balancing on existingInternet Protocol-based networking products.

Sun Services


Dynamic Reconfiguration (DR)

Use DR to do the following:

• Minimize the interruption of system applications whileinstalling or removing a board

• Disable a failing device by removing it from thedomain

• Display the operational status of boards in a domain

• Initiate system tests of a board while the systemcontinues to run

• Reconfigure a domain while Solaris OS continues torun in the domain

• Invoke hardware-specific functions of a board or arelated attachment

Sun Services


Platform Startup and Shutdown

• Understanding the correct procedures for powering upand powering down the Sun Fire midrange servers iskey to ensuring proper operation

• For the proper power up and down procedure for theSun Fire 1280 and E2900 server refer to the Sun Fire™V1280/Netra™ 1280 Service Manual

• For the proper power up and down procedure for theSun Fire 3800, 4800/E4900, 4810, and 6800/E6900servers, refer to the Sun Fire™ 6800/4810/4800/3800Systems Power On Off Guide

Sun Services


Module 2

Field-Replaceable Units

edited 02/08 by LG

Sun Services


Objectives

• Describe the various administrative and service layerson the Sun Fire midrange server products

• Describe the Sun Fire midrange server FRU strategy

• Locate and describe the function of the Sun Firemidrange server system controller boards

• Locate and describe the function of the Sun Firemidrange server system boards

• Locate and describe the function of the Sun Firemidrange server I/O boards

• Locate and describe the function of the Sun Firemidrange server Sun Fireplane switchboards

Sun Services


Objectives

• Locate and describe the function of the Sun Firemidrange server baseplane and centerplanes

• Locate and describe the function of the Sun Firemidrange server AC and DC power distribution FRUs

• Locate and describe the function of the Sun Firemidrange server fan tray assemblies

• Locate and describe the function of the Sun Firemidrange server FrameManager

• Locate and describe the function of the Sun StorEdge™D240 media tray

• Install and administer the console

• Install the server in a rack configuration

Sun Services


Relevance

• What URLs are needed to install and configure the SunFire midrange server models?

• What is the difference between hot-plug and hot-swap?

• Which components make up the Sun Fire midrangeserver models?

• Where are the components located?

• Which status indicators are associated with eachcomponent?

Sun Services


Administrative and Service Overview

Applications

SolarisOperatingSystem

OpenBoot PROM 5

LOM Shell

Sun Fire V1280/E2900Midrange Server

PlatformHardware

Sun Services


Administrative and Service Overview

Applications Applications Applications Applications





Open Boot PROM Open Boot PROM Open Boot PROM Open Boot PROM

Domain Shell A(SC)

Domain Shell B(SC)

Domain Shell C(SC)

Domain Shell D(SC)

Platform Shell (System Controller)

Platform Hardware

Sun Services


Sun Fire System Field-Replaceable Units

The Sun Fire midrange servers consist of FRUs. The followingitems are the main FRUs for the Sun Fire 3800, 4800/E4900,4810, and 6800/E6900 servers:

• Centerplane

• Fans

• ID board

• I/O boards (cPCI I/O, PCI I/O)

• cPCI I/O and PCI I/O cards

• Memory

Sun Services



• Power (alternating current [AC] and direct current[DC])

• Rack (patch panel, FrameManager, Sun Fire cabinet)

• System board (CPU/memory)

• System controller board

• Switchboard (L2 repeater board)

Sun Services



The following items are the main FRUs for the Sun FireV1280/E2900 server:

• Baseplane

• Fans

• System controller-I/O board (PCI I/O)

• Memory

• Power (AC and DC)

• System board (CPU/memory)

• IDE interface

Sun Services



• Internal drives (disk, tape, and digital versatile disk[DVD])

• Switchboard (L2 repeater board)

Sun Services


FRU Administration

Non-hot-pluggable FRUs:

• PCI cards

• Centerplane

• Sun Fireplane switchboard

Sun Services


FRU Administration

Hot-pluggable FRUs:

• System boards

• I/O boards

• System controller (only with failover enabled)

Sun Services


FRU Administration

Hot-swappable FRUs:

• DC power supplies

• Fan trays

• cPCI cards

Sun Services


Sun Fire V1280/E2900 Server IB SystemController Board

The IB_SSC systemcontroller board is asystem serviceprocessor thatcontrolsmaintenance andsystem clocks.

It is combined withthe I/O board onthe Sun Fire V1280/E2900 servers.

PCI chassis PCI riser board

IB_SSC riser board-I/O Controller (I/O)

-System Controller (SC)

Sun Services


Sun Fire Midrange Server SystemController Board

Sun Services


Sun Fire Midrange Server SystemController Board

The system controller board is a system service processor thatcontrols maintenance and system clocks. You can install oneadditional system controller board for redundancy in all ofthe Sun Fire systems.

Sun Services


System Controller Physical Locations

Sun Fire V1280/E2900 Servers’ IB_SSC Board Location

IB_SSC FRU

Sun Services



Sun Fire 3800 Server SC Board Slot Assignments

SC1

SC0

Sun Services



Sun Fire 4800/E4900 Server SC Board Slot Assignments

SC1

SC0

Sun Services



Sun Fire 4810 Server SC Board Slot Assignments

SC1

SC0

Sun Services



Sun Fire 6800/E6900 Server SC Board Slot Assignments

SC1

SC0

Sun Services


Sun Fire V1280/E2900 Server IB_SSC SCBoard Faceplate

TTYB(reserved)

LOM serialport

SSC1 10/100 Ethernet

SSC1 B

SSC1 A

Sun Services


Sun Fire Midrange Server SystemController Board Faceplate

Status LED

Serial portsA/B

Resetbutton

Ethernet RJ45port

Sun Services


Sun Fire Midrange Server SC Board LEDDefinitions

Name, Color, andSymbol

On Off

Activated (green) The board is activated. Donot remove the board whenthis LED is on.

The board is notactivated. You can removethe board when this LEDis off.

Fault (amber) An internal fault occurred. No internal faultoccurred.

Removal OK(amber)

You can safely remove thecomponent underhot-pluggable conditions.

Do not remove thecomponent underhot-pluggable conditions.

Sun Services


Sun Fire V1280/E2900 Server SystemBoards

The Sun Fire V1280/E2900 servers system board houses eitherfour UltraSPARC® III or four UltraSPARC IV microprocessors(Scalable Processor ARChitecture).

Sun Services


Sun Fire V1280/E2900 Server SystemBoards

.

Sun Services


Sun Fire 3800, 4800/E4900, 4810, and6800/E6900 Server System Boards

The Sun Fire midrange server system board houses either twoor four UltraSPARC III or UltraSPARC IV processors. Eachprocessor supports two physical banks of memory.

Sun Services


Memory Configuration Rules

• Each CPU on the system board can be configured asfollows:

• No memory installed

• One bank (bank 0) installed

• Two banks (bank 0 and bank 1) installed

• A populated dual-inline memory module (DIMM)bank must have a corresponding CPU processorinstalled.

• When memory is installed, all DIMMs in a physicalbank must be present.

Sun Services


CPU and Memory Locations

CPU 1(P1)

J14600J14601J14500J14501J14400J14401J14300J14301J13600J 13601J13500J 13501J13400J13401J13300J13301J16600J16601J16500J16501J16400J16401J16300J16301J15600J15601J15500J15501J15400J15401J15300J15301

CPU 0(P0)

CPU 3(P3)

CPU 2(P2)

Sun Services


Processor Memory Group BankAssignments

DIMM 3, bank 0DIMM 3, bank 1DIMM 2, bank 0DIMM 2, bank 1DIMM 1, bank 0DIMM 1, bank 1DIMM 0, bank 0DIMM 0, bank 1

Sun Services


Sun Fire V1280/E2900 Server SystemBoard Slot Assignments

SB0

SB2

SB4

Sun Services


Sun Fire 3800 System Board SlotAssignments

SB2

SB0

Sun Services


Sun Fire 4810 and 4800/E4900 ServerSystem Board Assignments

SB0SB2SB4

SB0SB2SB4

Sun Services


Sun Fire 6800/E6900 Server SystemBoard Slot Assignments

SB4

SB2

SB0

SB1

SB3

SB5

Sun Services


System Board LEDs

Removal OKLED (amber)

Fault LED(amber)

ActivatedLED (green)

Sun Services


Sun Fire Midrange I/O Assemblies

Four types of I/O assemblies are used in the Sun Firemidrange products:

• The six-slot IB_SSC assembly (used in the Sun FireV1280/E2900 servers only)

• The four-slot cPCI assembly (used in the Sun Fire4800/E4900, 4810, and 6800/E6900 servers only)

• The six-slot cPCI assembly (used in the Sun Fire 3800server only)

• The eight-slot PCI assembly (used in the Sun Fire4800/E4900, 4810, and 6800/E6900 servers only)

Sun Services


Sun Fire V1280/E2900 Server Six-Slot PCIIB_SSC Assembly

PCI chassis PCI riser board

IB_SSC riser board-I/O Controller (I/O)

-System Controller (SC)

Sun Services


SunFire4800/E4900,4810,and6800/E6900Server Four-Slot cPCI I/O Assembly

Sun Services


Sun Fire 3800 Server Six-Slot cPCI I/OAssembly

Sun Services


SunFire4800/E4900, 4810, and 6800/E6900Server Eight-Slot PCI I/O Assembly

Sun Services


Sun Fire V1280/E2900 Server I/OAssembly Location

33MHz PCI 0

33MHz PCI 1

33MHz PCI 2

33MHz PCI 3

33MHz PCI 4

66MHz PCI 5

32-Bit

32-Bit

32-Bit

32-Bit

32-Bit

32-Bit

64-Bit

64-Bit

64-Bit

64-Bit

64-Bit

64-Bit

Sun Services


Sun Fire 3800 Server I/O AssemblyLocations

IB6IB8

Sun Services


Sun Fire 4800/E4900 Server I/O AssemblyLocations

IB8

IB6

Sun Services


Sun Fire 4810 Server I/O AssemblyLocations

IB6

IB8

Sun Services


Sun Fire 6800/E6900 Server I/O AssemblyLocations

IB9

IB7

IB8

IB6

Sun Services


Eight-Slot PCI I/O Assembly Slot Locationsand LEDs

76 024

135 Slots Activated LED(Green)

Fault LED(Amber)

Removal OK LED(Amber)

Sun Services


Eight-Slot Assembly ElectricalCharacteristics

SlotNumber

Frequency Bit-Size Voltage

0 33 MHz 64-bit 5VDC



3 66 MHz/33 MHz 64-bit 3.3VDC





Sun Services


Six-Slot cPCI I/O Assembly Slot Locationsand LEDs

cPCIBoardLEDs

cPCISlotLEDs

Activated LED(Green)

Fault LED(Amber)

Removal OK LED(Amber)0

4

21

5

3

Slots

Sun Services


Six-Slot cPCI I/O Slot ElectricalCharacteristics

SlotNumber

Frequency Bit-Size Voltage







Sun Services


Four-Slot cPCI I/O Assembly Slot Locationsand LEDs

cPCIBoardLEDs

cPCISlotLEDs

Activated LED(Green)

Fault LED(Amber)

Removal OK LED(Amber)

0

21

3

Slots

Sun Services


Four-Slot cPCI I/O Slot ElectricalCharacteristics

SlotNumber

Frequency Bit-size Voltage





Sun Services


PCI and cPCI I/O Adapters

Sun 10/100 Ethernet Plus SCSI cPCI Adapter

Ejector handle

Hot swap indicator

Sun Services


PCI and cPCI I/O Adapters

Sun Dual Fast Ethernet and Dual SCSI PCI Adapter

Sun Services


Sun Fire V1280/E2900 Server SunFireplane Switchboard

AR

SDC

DX0

DX1

D107 DC-DC

D110 DC-DC I2C Aux

SEPROM

Echip

Air Vent

Lever

Lever

Sheetmetal FillerFireplane Switchboard

LEDsOnFaultOk-to-Remove

Air Vent

Sun Services


Sun Fire 4800/E4900, 4810, and 6800/E6900Server Sun Fireplane Switchboard

Sun Services


Sun Fireplane Switchboard PhysicalLocations

Sun Fire V1280/E2900 Server Switchboard Slot Assignments

RP0

RP2

Sun Services


Sun Fireplane Switchboard PhysicalLocations

Sun Fire 4810 and 4800/E4900 Server Switchboard SlotAssignments

RP0

RP2

RP0

RP2

Sun Fire 4800/E4900Sun Fire 4810 Server

Sun Services


Sun Fireplane Switchboard LEDs

Activated LED (green)Fault LED (amber)Removal OK LED (amber)

Sun Services


Sun Fire V1280/E2900 Server Baseplaneto 530-3079

Fan Tray PCB

530-3073

J9001

1B6_

SS

C1

1B6_

SS

C1

Sys

tem

Boa

rd 0

Sys

tem

Boa

rd 2

Sys

tem

Boa

rd 0

Sys

tem

Boa

rd 2

Sys

tem

Boa

rd 4

Sun

Fire

plan

e S

witc

hboa

rd 0

Sun

Fire

plan

e S

witc

hboa

rd 2

Sun

Fire

plan

e S

witc

hboa

rd 0

Sun

Fire

plan

e S

witc

hboa

rd 2

J9301to DC Distrubution Board

Sys

tem

Boa

rd 4

Sun Services


System Configuration Card Reader (SCCR)

System configurationcard reader (SCCR)

Sun S

ervices

Sun F

ire™ M

idrange Server M

aintenanceM

odule 2, slide 61 of 112C

opyright 2004 Sun M

icrosystems, Inc. A

ll Rights R

eserved. Sun S

ervices, Revision C

Sun

Fire

6800/E6900

ServerC

enterplaneand

IDB

oard(FrontV

iew)

System Controller 0

System Board 0

System Board 2

System Board 4

System Board 1

System Board 3

System Board 5

System Controller 1

System Board 0

System Board 2

System Board 4

System Board 1

System Board 3

System Board 5

SB

0S

B2

SB

4S

B1

SB

3S

B5

Sun Services


Sun Fire 6800/E6900 Server Centerplane andID Board (Rear View)

I/O B

oard

9

Sun

Fire

plan

e S

witc

hboa

rd 1

Sun

Fire

plan

e S

witc

hboa

rd 0

Sun

Fire

plan

e S

witc

hboa

rd 3

Sun

Fire

plan

e S

witc

hboa

rd 2

I/O B

oard

8

Sun

Fire

plan

e S

witc

hboa

rd 1

Sun

Fire

plan

e S

witc

hboa

rd 0

Sun

Fire

plan

e S

witc

hboa

rd 3

Sun

Fire

plan

e S

witc

hboa

rd 2

I/O B

oard

8

I/O B

oard

9

Bus Bar 340-4796to Power Centerplane

Bus Bar 340-4797to Power Centerplane

I2C Cable 530-2546to Power Centerplane

ID Board IBS PR2 RP3 PR0 RP1 IB8

Sun Services


ID Board

Sun Services


ID Board

The ID board contains a serial electrically erasableprogrammable read-only memory (SEEPROM)application-specific integrated circuit (ASIC) with thefollowing information:

• It has the server chassis ID.

• It has the server serial number/host ID.

• It incorporates six media access control (MAC)addresses for the Sun Fire 6800/E6900 server and fourMAC addresses for the Sun Fire 3800, 4800/E4900, and4810 servers. This includes one per possible domainand one each for the system controllers.

• It has the server and component power-on hours.

Sun Services


Replacing a Centerplane or ID Board

The system controller application (ScApp) shouldautomatically program the static portion (such as MACaddress, serial number, and so on) of the ID board, but if youcannot do this, the following prompts appear.

Please enter System Serial Number from label:Please enter Platform Model Number (3800, 4800, E4900, 4810,6800, E6900):Please enter System MAC Address from label (aa:bb:cc:dd:ee:ff):Please enter Host ID from label (aabbccdd):Is this COD (Capacity on Demand) system? (Y/N)

Sun Services


ID Board MAC Addresses

Sun Fire 3800, 4800/E4900,and 4810 Servers

Sun Fire 6800/E6900 Servers

Domain A = Base MAC address Domain A = Base MAC address

Domain B = Base MAC addressplus 1

Domain B = Base MAC addressplus 1

SC0 = Base MAC address plus 2 Domain C = Base MAC addressplus 2

SC1 = Base MAC address plus 3 Domain D = Base MAC addressplus 3

N/A SC0 = Base MAC address plus 4

N/A SC1 = Base MAC address plus 5

Sun Services


AC Power Distribution

All Sun Fire midrange servers installed in a data centercabinet are equipped with the following AC components:

• The redundant transfer unit (RTU)

• The redundant transfer switch (RTS)

• An AC input box

Sun Services


AC Power Distribution

RackFan

Trays(2)

Primary200–240VAC

220VAC

220VACSecondary200–240VAC

RTS

RTS

RTU

To AC input box for all serversexcept the Sun Fire 3800 servers.To power supplies for the Sun Fire 3800 servers only.

Sun Services


Sun Fire 4810 and 4800/E4900 ServerAC Component Locations

Sun Fire 4810 Server Sun Fire 4800/E4900

ACinput box

Sun Services


Sun Fire 6800/E6900 ServerAC Component Locations

ACinputboxes

RTU

RTSFront Rear

Sun Services


RTU and RTS

The RTU supplies AC power to cabinet peripherals and fans.

Input AC powerRTS

RTS

RTUEight switched receptacles are intended to supply power to the peripheral equipment.

The 6800 has two special AC Input Modules which deliver power from the RTUs to the AC input module.

Two unswitched outlets provide power to the cabinet equipment such as fans.

Sun Services


Redundant Transfer Unit Panel

Remote Transfer Switches Switched

J11 Unswitched

Switched

J12 Unswitched

J3 J5 J7 J9On-Off breakers

J4 J6 J8 J10On-Off breakers

J11J11

J3J3J5J5

J7J7J9J9

J13J13

J15J15J124AJ124A

J12J12

J4J4J6J6

J8J8J10J10

J14J14

J16J16J224AJ224A

J13 J15To FrameManager

LEDs

RTUR RTS0RTUF RTS2

RTUR RTS1RTUF RTS3

J14 J16Not Used

Sun Services


Redundant Transfer Unit LED Functions

LEDPosition

LEDColor

State Meaning

Left Green OnOff

Flashing

The source is present and is okay.The source is not present or is lower thanthe specification.The source is out of the specification.

Middle Green On

OffFlashing

The relay is energized and connected tothe outlets.The relay is de-energized and docked.The relay is de-energized and undocked.

Right Amber OnOff

The module has a fault.The module does not have a fault.

Sun Services


AC Input Box

• For the Sun Fire 6800/E6900 systems, the AC input boxreceives power from the RTU through two powercables, each with a corresponding power switch.

• For the Sun Fire 4810 and 4800/E4900 systems, the ACinput box receives power from the RTU through threepower cables, each with a corresponding power switch.

• Sun Fire 3800 systems do not use AC input boxes.

Sun Services


DC Power Distribution

Sun Fire midrange DC power distribution systems includecomprised of the following major components:

• System centerplane

• Power centerplane

• Fan centerplane

• DC power supplies

Sun Services


Sun Fire V1280/E2900 Server DC PowerDistribution

ACInputBox

Power Distribution Board (PDB)

AC-DC Converters (PS0-PS3)

Main48VDC

Standby12VDC

Main 48VDCAuxiliary48VDC

RackFan

Trays(2)

220VAC

220VAC

RTU

DC-DC (3) CPU/Memory Boards(2) L2 Repeater Boards

DC-DC(1) I/O Controller Board

(+12V, D12V, 5V, 3V3, 1V5)

DC-DC(1) System Controller

Board

Primary200–240VAC

Secondary200–240VAC

RTS

RTS

Feed A (2)

Feed B (2)

(1) Fan Tray(48VDC_IL D>3.5 A.)

48VDC_IL

Auxiliary 3V3

Auxiliary 3V3

Sun Services


Sun Fire 3800 Server DC Power Distribution

System Centerplane


Main56VDC

Auxiliary56VDC

Main56VDC

Auxiliary56VDC

3

RackFan

Trays(2)

220VAC

To fans (4)Main 56VDC

220VAC

RTU

DC-DC Board Converters(system, I/O, and Sun

Fireplane switch boards)

DC-DC Board Converters(system controller and

ID boards)

Primary200–240VAC


RTS

RTS

Sun Services


Sun Fire 4800/E4900 Server DC PowerDistribution

System Centerplane


Main56VDC

Auxiliary56VDC

Main56VDC

Auxiliary56VDC

220VAC

ACInputBox

3

RackFan

Trays(2)

220VAC

220VAC

To fans (3)Main 56VDC

220VAC

RTU




ID boards)

Primary200–240VAC


RTS

RTS

Sun Services


Sun Fire 4810 Server DC Power Distribution

Power Centerplane


Main56VDC

Auxiliary56VDC

System Centerplane

Main56VDC

Auxiliary56VDC

Main56VDC

Auxiliary56VDC

220VAC

ACInputBox

3

RackFan

Trays(2)

220VAC

220VAC

To fans (3)Main 56VDC220VAC

RTU




ID boards)

Primary200–240VAC


RTS

RTS

Sun Services


Sun Fire 6800/E6900 Server DC PowerDistribution

FAN CENTERPLANE

RTS

RTS

RTS

RTUR

RTS

RTUF AC0

AC1

ACInputBox

PowerCenterplane

PowerCenterplane

Primary

200-240VAC

220VAC

Auxiliary56V

Main56V

Main56V

AC-DCCONVPS0-2

AC-DCCONVPS3-5

DC-DCSB0, 2, 4

ib6, 8RP0, 1

DC-DCSB1, 3, 5

ib7, 9RP2, 3

SC0SC1

DC-DC

220VAC

Secondary

200-240VAC

Primary

200-240VAC

Secondary

200-240VAC

1

1

Auxiliary56V

Sun Services


Power Grid Slot Assignments

ServerPowerGrid

Slots

Sun Fire V1280 0 SB0, SB2, SB4, IB_SSC, RP0, and RP2

Sun Fire E2900 0 SB0, SB2, SB4, IB_SSC, RP0, and RP2

Sun Fire 3800 0 SB0, SB2, IB6, and IB8

Sun Fire 4800 0 SB0, SB2, SB4, IB6, IB8, RP0, and RP2

Sun Fire E4900 0 SB0, SB2, SB4, IB6, IB8, RP0, and RP2






Sun Services


DC Power Supplies

Power Supply Specifications

ServerSlotNumber

Main VoltageOutput

AuxiliaryVoltage Output

Sun Fire V1280/E2900 PS0, PS1, PS2,PS3

48VDC 48VDC

Sun Fire 3800 PS0, PS1, PS2 56VDC 56VDC

Sun Fire 4800/E4900 PS0, PS1, PS2 56VDC 56VDC

Sun Fire 4810 PS0, PS1, PS2 56VDC 56VDC

Sun Fire 6800/E6900 PS0–PS5 56VDC 56VDC

Sun Services


DC Power Supply Locations

Sun Fire V1280/E2900 Server DC Power Supply Locations

PS3

PS2

PS1

PS0

Sun Services



Sun Fire 3800 Server DC Power Supply Locations

PS2 PS1 PS0

Sun Services



Sun Fire 4800/E4900 Server DC Power Supply Locations

PS2

PS1

PS0

Sun Services



Sun Fire 4810 Server DC Power Supply Locations

PS1

PS0PS2

Sun Services



Sun Fire 6800/E6900 Server DC Power Supply Locations

PS3

PS0

PS1PS2

PS5

PS4Grid 1

Grid 0

}{

Sun Services


Sun Fire Midrange Server Fan Trays andBlower Assemblies

• The Sun Fire V1280/E2900 servers have no redundantfan trays. One fan tray supports eight individuallyreplaceable fans.

• The Sun Fire 3800 server has four blower assemblies.

• The Sun Fire 4800/E4900 and 4810 servers have threefan trays.

• The Sun Fire 6800/E6900 servers have four fan trays.

Sun Services


Fan Tray Locations

Sun Fire V1280/E2900 Server Fan Tray Location

V1280/2900 ServerFront View

Fan tray

Sun Services


Fan Tray Locations

Sun Fire 3800 Server Fan Tray Locations

FT0

FT2

FT1

FT3

Sun Services


Fan Tray Locations

Sun Fire 4800/E4900 Server Fan Tray Locations

FT0

FT1

FT2

Sun Services


Fan Tray Locations

Sun Fire 4810 Server Fan Tray Locations

FT1

FT0

FT2

Sun Services


Fan Tray Locations

Sun Fire 6800/E6900 Server Fan Tray Locations

FT1

FT3

FT0

FT2

Front Rear

Sun Services


Fan Tray Assembly LEDs

Active LED(green)

Fault LED(amber)

Removal OK LED(amber)

Sun Services


FrameManager Cable Diagram

SerialPort B

SC1

SC0Rear RTU

Front RTU

Sun FireMidrange

Server

FrameManager

Rack Fan Tray Rack Fan Tray

RTS RTS

Sun Services


FrameManager Cap Front Panel

Keyswitch LCD status manager

Fault LED (amber)

Power LED (green) Service LED (amber)

Sun Services


Sun StorEdge™ D240 Media Tray

The Sun StorEdge D240 media tray for the Sun Fire datacenter cabinet is a supported boot device for the Sun Firemidrange server.

Sun Services


Sun StorEdge D240 Media Tray Rear Panel

In a full Small Computer System Interface (SCSI) busconfiguration, all the media tray internal drives are connectedto one domain using a single SCSI bus.

Full busFull bus

Configuration switchConfiguration switch

UltraWide SCSI bus cables (full bus)To domain

Sun Services


Full-Bus Configuration SCSI IDAssignments

DVD ROM (ID6) DAT tape (ID4)Disk (ID1) Disk (ID0)

Media Tray Front View

Sun Services


Full SCSI Bus Configuration Options

Configuration OptionsHardDrives

TapeDrives

DVD-ROMDrives

Maximum hard drives and tapedrives

Two Two None

Maximum hard drives only Four None None

Mixed hard drives, tape drive, andDVD-ROM drive

Two One One

Two None Two

Sun Services


Split SCSI Bus

Sun StorEdge D240 Media Tray Rear Panel

Split busSplit bus

Configuration switchConfiguration switch

To domain To domain

Sun Services


Typical Split SCSI Bus SCSI IDAssignments

DVD ROM (ID6) DAT tape (ID6)Disk (ID0) Disk (ID0)

Media Tray Front View

Sun Services


Sun StorEdge D240 Media Tray StatusLEDs

System fault LED(amber)

Power LED(green)

Sun Services


Media Tray Status LED Descriptions

Indication Status

Power is green The power supply is inserted and cabled on,normal.

Both LEDs are off The power supply is absent, or the power cords arenot connected.

System fault isamber

The power supplies have failed, the fan has failed,or the system is running from a single powersupply.

Sun Services


Media Tray Power Supply LEDs

DC output

Fault

Remove OK

AC input

Sun Services


Media Tray Power Supply LED States

LED IndicationNormal(All PowerSupplies)

Fault(Good PowerSupply)

Fault(Bad PowerSupply)

DC output Green On On Off

Fault Amber Off Off On

Remove OK Blue On Off On

AC input Green On On Off

Sun Services


Installing the Administration Console

• ASCII terminal

• Network terminal server (NTS)

• Workstation

• Laptop

• Personal digital assistant (PDA)

Sun Services


System Controller Patch Panel

Sun Services


Accessing the Platform Shell

You can access the system controller in one of two ways:

• Serial port console connection:

• ASCII terminal or terminal emulation (vt100)


• Workstation

• Ethernet port shell connection

Sun Services


Sun Fire Midrange Server Installation

Tools:

• System site planning guides

• Installation manuals

• Configuration manual

• Sun Fire Midrange Server Configuration Best Practices

• Sun Fire Midrange Server Administration BestPractices

• Enterprise Installation Services (EIS) Installation andPatch CD (EIS-CD)

Sun Services


Rackmounting an Additional Sun Fire 3800and 4800/E4900 Server

Rackmounting Considerations

• Install systems in the cabinet from the bottom up.

• Install storage devices directly above server systems inthe cabinet, from the bottom up.

• Install Sun StorEdge D240 media trays from the top ofthe cabinet down.

• The Sun Fire cabinet does not support any side-to-sidecooled devices. Only front-to-back cooled products aresupported.

• Up to three Sun Fire 3800 systems per Sun Fire cabinetare supported.

Sun Services


Rackmounting an Additional Sun Fire 3800and 4800/E4900 Server

Rackmounting Considerations

• Use the Sun Fire 3800 system rack mounting kits in theSun Fire cabinet only.

• When considering third-party rackmounting, reviewtheir thermal and rackmounting guidelines.

• Mount the Sun Fire 280R server, Sun Enterprise™ 420Rhardware, Netra T1 AC200/DC200 servers, and SunFire V100/V120 servers above the Sun Fire systems.

• Install front plastic filler panels to fill in any openspaces within the Sun Fire cabinet.

Sun Services


Module 3

Platform Assessment and Management

edited 02/08 by LG

Sun Services


Objectives

• Describe an overview of the Sun Fire V1280/E2900server platform administrative functions

• Perform user maintenance and administrativefunctions using the LOM shell

• Display and change system controller parameters byusing LOM shell commands

• Monitor the Sun Fire V1280/E2900 server platform byusing LOM shell commands

• Power cycle the Sun Fire V1280/E2900 server withLOM shell commands and with the front panel powerrocker switch

Sun Services


Objectives

• Identify the administrative and service tasks that youcan perform with the platform shell

• Identify the platform shell commands you can use todisplay system information, set up system parameters,and test system hardware

• Describe three methods you can use to connect to thesystem controller shells

• Describe how to navigate between shells on the SunFire 3800, 4800/E4900, 4810, and 6900/E6900 servers

• Configure the platform and system controller by usingthe platform shell

Sun Services


Objectives

• Display and change system controller parameters byusing the platform shell

• Describe how to manage the system controller forreboot and failover operations

• Describe how to use system controller commands tomonitor platforms and domains

• Describe how to power on and off the systemcomponents

• Describe how to update the platform firmware

• Describe the capability and effects of splitting the SunFire 3800, 4800/E4900, 4810, and 6800/E6900 serversinto segments and domains

Sun Services


Objectives

• Describe how to start, stop, and power cycle domains

• Describe how to perform device configuration withOpenBoot™ programmable read-only memory(PROM) to add or remove devices from a Sun Firemidrange server

• Describe OpenBoot PROM capabilities

• Describe how to use OpenBoot PROM to representinterconnected busses and their devices in a device tree

• Describe how to map Sun Fire midrange serverphysical devices

Sun Services


Relevance

• Which functions does the system controller perform?

• How do the system controller maintenance busescommunicate with the platform?

• Which role does the platform shell play in configuringthe Sun Fire midrange servers?

• Which commands are available in the platform shell?

• How is each platform shell command used to configurethe Sun Fire midrange server platform?

Sun Services


Sun Fire V1280/E2900 Server PlatformAssessment and Management

Applications


OpenBoot PROM 5

LOM Shell


PlatformHardware

Sun Services


Lights-Out Management (LOM)

• LOM is a set of software tools on the Sun FireV1280/E2900 servers.

• LOM is used to perform user maintenance andadministrative activities.

• LOM is composed of a shell and UNIX® utility.

• The LOM shell runs under Vxworks, the systemcontroller (SC) operating environment.

• The LOM utility is a utility command (lom ) that runsunder the Solaris OS and can only be accessed by a rootlevel user.

Sun Services


LOM Shell

The LOM shell that runs on the system controller has beensignificantly modified from versions on earlier Netra serverplatforms. Commands that were originally developed for theSun Fire 3800, 4800, 4810, and 6800 server domain shell havebeen adopted.

Sun Services


LOM Shell Commands

The LOM shell has a number of commands to display systeminformation, to set up system parameters, and to test systemhardware.

Sun Services


The help Command

lom> help setlocatorsetlocator -- set the system locator ledUsage: setlocator on|offsetlocator -h-h -- display this help messagelom>

Sun Services


Connecting to the LOM Shell

You can access the system controller in one of two ways:

• Serial port console connection:

• ASCII terminal or terminal emulation (vt100)


• Workstation

• Ethernet port shell connection

Sun Services


The shownetwork Command

lom> shownetwork -vNetwork-------The system controller is configured to be on a network.Network settings: staticHostname: commando-scIP Address: 129.xxx.xxx.xxxGateway: 129.xxx.xxx.xxxDNS Domain: noone.somewhere.comPrimary DNS Server: 129.xxx.xxx.xxxSecondary DNS Server: 129.xxx.xxx.xxx

Sun Services


The setupnetwork Command

lom> setupnetworkNetwork Configuration---------------------Is the system controller on a network? [no]: yesUse DHCP or static network settings? [DHCP]: staticHostname []: somenameIP Address []: 129.xxx.xxx.xxxNetmask [255.255.255.0]: 255.255.255.0Gateway []: 129.xxx.xxx.xxxDNS Domain []: somewhere.nowhere.comPrimary DNS Server []: 129.xxx.xxx.xxxSecondary DNS Server []: 129.xxx.xxx.xxxRebooting the SC is required for changes in network settings totake effect.lom>

Sun Services


The logout Command

lom> logout

Sun Services


Navigating Between Shell Environments onthe Sun Fire V1280/E2900 Server

DefaultEscape

Sequence

DefaultEscapeSequence

Solaris Operating System (Solaris OS)

Open Boot PROM (OBP)

Lights Out Management (LOM) Shell

Solaris OS not booted

lom> console lom> break

ok #.

#.

Sun Services


The showescape Command

lom> showescape#.lom> setescape ~~~lom> showescape~~~lom>

Sun Services


The password Command

lom> passwordEnter new password: cangetinEnter new password again: cangetinlom>

Sun Services


The showsc Command

lom> showsc -vSC: SSC1Clock failover disabled.SC date: Tue Jan 13 17:19:17 MST 2004 MST GMT-7 Mountain Standard TimeSC uptime: 1 hour 31 minutes 28 secondsScApp version: 5.13.0015 LW8_build0.15Version build: 1.0Version String: LW8_build0.15RTOS version: 32SC POST diag level: minClock source is: 75MHzSolaris Host Status: Active - Solarislom>

Sun Services


The setupsc command

This command configures optional system controller features.

The options and parameters are:

• off – SCPOST is not run.

• min – Minimum level SCPOST is run (default).

• max – Maximum level SCPOST is run.

Sun Services


The setupsc command

lom> setupscSystem Controller Configuration-------------------------------SC POST diag Level [off]: min

Sun Services


Managing the LOM Time-of-Day (TOD)

lom> setdate 020413232004.14

lom> setdate -r hostnameMon Feb 05 09:30:58 EST 2004

lom> showdateMon Feb 06 12:31:40 EDT 2004

Sun Services


The bootmode Command

This command sets the highest level of POST tests to run onthe next reboot.

lom> bootmode diaglom> console# init 6

Sun Services


LOM Platform Monitoring Functions

• showboards

• showcomponent

• inventory

• showenvironment

• history

• showlogs

• showlocator

Sun Services


The showboards Command

schostname :SC> showboardsSlot Pwr Component Type State Status Domain---- --- -------------- ----- ------ ------SSC0 On System Controller Main Passed -SSC1 On System Controller Spare - -ID0 On Sun Fire 3800 Centerplane - OK -PS0 On A145 Power Supply - OK -PS1 On A145 Power Supply - OK -PS2 On A145 Power Supply - OK -FT0 On Fan Tray Low Speed OK -FT1 On Fan Tray Low Speed OK -FT2 On Fan Tray Low Speed OK -FT3 On Fan Tray Low Speed OK -RP0 On Repeater Board (F3800) - OK -RP2 On Repeater Board (F3800) - OK -/N0/SB0 On CPU Board Assigned Not tested A/N0/SB2 On CPU Board Assigned Not tested A/N0/IB6 On CPCI I/O board (F3800) Assigned Not tested A/N0/IB8 On CPCI I/O board (F3800) Assigned Not tested A

Sun Services



schostname :SC> showboards -p cpus

CPUs----/partition0/domain0/SB0/bbcGroup0/cpuAB/cpusafariagent0: part_no=0x919c version=2 manufacturer=0x07d devid=0x102234058a9 (lot=0x022340 bin=1 wafer=22 col=5row=9) speed=750 e$ (8MB): J4400: 4MB, part# 0, serial# 090AXB J4300: 4MB, part# 0, serial# 090AVU DRAM DIMM Group 0 (1024MB): J13300: 256MB, part# 0, serial# 005898 J13400: 256MB, part# 0, serial# 005870 J13500: 256MB, part# 0, serial# 005878 J13600: 256MB, part# 0, serial# 005881 DRAM DIMM Group 1 (Empty):

Sun Services



schostname :SC> showboards -p clock

Component SSC0 Signal SSC1 Signal Signal Used Failover--------- ----------- ----------- ----------- --------SSC0 OK OK SSC0 EnabledRP0 OK OK SSC0 EnabledRP2 OK OK SSC0 Enabled/N0/SB0 OK OK SSC0 Enabled/N0/SB2 OK OK SSC0 Enabled/N0/IB6 OK OK SSC0 Enabled/N0/IB8 OK OK SSC0 Enabled

Sun Services



schostname :SC> showboards -p cpu

Component Description--------- -----------/N0/SB0/P2 UltraSPARC-III, 0MHz, 8M ECache/N0/SB0/P3 UltraSPARC-III, 0MHz, 8M ECache/N0/SB2/P2 UltraSPARC-III, 0MHz, 8M ECache/N0/SB2/P3 UltraSPARC-III, 0MHz, 8M ECache

Sun Services



schostname :SC> showboards -p memory

Component Size Reason--------- ---- ------/N0/SB0 2048 MB/N0/SB2 2048 MB

schostname :SC> showboards -p power

Component Pwr Grid--------- --- ----/N0/SB0 On Grid 0/N0/SB2 On Grid 0/N0/IB6 On Grid 0/N0/IB8 On Grid 0

Sun Services



schostname :SC> showboards -p version

Component Compatible Version--------- ---------- -------SSC0 Reference 5.13.0 Build 12/N0/IB6 Yes 5.13.0 Build 12/N0/IB8 Yes 5.13.0 Build 12/N0/SB0 Yes 5.13.0 Build 12/N0/SB2 Yes 5.13.0 Build 12

Sun Services


The showcomponent Command

schostname :SC> showcomponent sb0Component Status Pending POST Description--------- ------ ------- ---- -----------/N0/SB0/P0 enabled - untest empty/N0/SB0/P1 enabled - untest empty/N0/SB0/P2 enabled - pass UltraSPARC-III,750MHz, 8M ECache/N0/SB0/P3 enabled - pass UltraSPARC-III,750MHz, 8M ECache/N0/SB0/P0/B0/L0 enabled - untest empty/N0/SB0/P0/B0/L2 enabled - untest empty/N0/SB0/P0/B1/L1 enabled - untest empty/N0/SB0/P0/B1/L3 enabled - untest empty/N0/SB0/P1/B0/L0 enabled - untest empty/N0/SB0/P1/B0/L2 enabled - untest empty/N0/SB0/P1/B1/L1 enabled - untest empty/N0/SB0/P1/B1/L3 enabled - untest empty/N0/SB0/P2/B0/L0 enabled - pass 512M DRAM/N0/SB0/P2/B0/L2 enabled - pass 512M DRAM

Sun Services



/N0/SB0/P2/B1/L1 enabled - untest empty/N0/SB0/P2/B1/L3 enabled - untest empty/N0/SB0/P3/B0/L0 enabled - pass 512M DRAM/N0/SB0/P3/B0/L2 enabled - pass 512M DRAM/N0/SB0/P3/B1/L1 enabled - untest empty/N0/SB0/P3/B1/L3 enabled - untest empty

Sun Services



schostname :SC> showcomponent ib6Component Status Pending POST Description----------- ------- ------- ---- -----------/N0/IB6/P0 enabled - pass IO controller 0/N0/IB6/P1 enabled - pass IO controller 1/N0/IB6/P0/B0 enabled - pass 66/33MHz PCI Bus/N0/IB6/P0/B1 enabled - pass 33MHz PCI Bus/N0/IB6/P0/B1/C0 enabled - pass 33MHz 5V Short PCI card/N0/IB6/P0/B1/C1 enabled - failed 33MHz 5V Short PCI card/N0/IB6/P0/B1/C2 enabled - pass 33MHz 5V LOng/Short PCI card/N0/IB6/P0/B1/C3 enabled - pass 33MHz 5V LOng/Short PCI card

Sun Services


The inventory Command

lom> inventory PS1/N0/PS1: PS: 300-1523-01-02 serial# E00254 "Power Supply(A166,V1280)"Made on Fri Nov 30 11:47:41 PST 2001 by 03ad at DELTAELECTRONICSCHUNGLITAIWANPowered on for 87 days 12 hours 1 minute

Sun Services


The showenvironment Commandlom> showenvironmentSlot Device Sensor Value Units Age Status---- ------- -------- ------- --------- ------- ------SSC1 SBBC 0 Temp. 0 37 Degrees C 1 sec OKSSC1 Board 0 Temp. 2 28 Degrees C 1 sec OKSSC1 Board 0 1.5 VDC 0 1.49 Volts DC 1 sec OKSSC1 Board 0 3.3 VDC 0 3.35 Volts DC 1 sec OKSSC1 Board 0 5 VDC 0 5.01 Volts DC 1 sec OK/N0/PS0 Input 0 Volt. 0 - - 6 sec OK/N0/PS0 48 VDC 0 Volt. 0 48.00 Volts DC 6 sec OK/N0/PS1 Input 0 Volt. 0 - - 5 sec OK/N0/PS1 48 VDC 0 Volt. 0 48.00 Volts DC 5 sec OK/N0/FT0 Fan 0 Cooling 0 Auto 5 sec OK/N0/FT0 Fan 7 Cooling 0 Auto 5 sec OK/N0/RP0 Board 0 1.5 VDC 0 1.51 Volts DC 5 sec OK/N0/RP0 Board 0 3.3 VDC 0 3.35 Volts DC 5 sec OK/N0/RP0 DX 1 Temp. 0 66 Degrees C 5 sec OK/N0/RP2 Board 0 1.5 VDC 0 1.49 Volts DC 4 sec OK/N0/RP2 Board 0 3.3 VDC 0 3.33 Volts DC 4 sec OK/N0/RP2 Board 0 Temp. 0 24 Degrees C 5 sec OK/N0/RP2 DX 0 Temp. 0 53 Degrees C 5 sec OK/N0/RP2 DX 1 Temp. 0 56 Degrees C 5 sec OK/N0/SB0 Board 0 1.5 VDC 0 1.50 Volts DC 4 sec OK/N0/SB0 Board 0 3.3 VDC 0 3.33 Volts DC 5 sec OK.

Sun Services


The history Command

lom> historyNov 26 14:34:37 : showalarm 1Nov 26 14:34:40 : showalarm 2Nov 26 14:34:45 : showalarm systemNov 26 14:40:01 : showeventreportingNov 26 15:06:00 : showfaultNov 26 15:53:05 : shownetworkNov 26 16:15:32 : help setlocatorNov 26 16:17:32 : history

Sun Services


The showlogs Command

lom> showlogsNov 26 16:48:29 interpol-sc lom: [ID 746300 local0.warning] Alarm1 ONNov 26 16:48:36 interpol-sc lom: [ID 222144 local0.notice] Alarm2 OFFNov 27 11:20:38 interpol-sc lom: [ID 218048 local0.notice] Alarm1 OFFNov 27 11:20:43 interpol-sc lom: [ID 877372 local0.warning] Alarm2 ONNov 27 11:21:17 interpol-sc lom: [ID 352276 local0.notice]Locator ON

Sun Services


The showlocator Command

lom> showlocatorlocator is off

Sun Services


Sun Fire V1280/E2900 Server PowerOperations

You can power-cycle the Sun Fire V1280/E2900 server byusing the following LOM commands:

• poweron

• shutdown

• poweroff

Sun Services


LOM poweron Command

The poweron command powers on the entire system orindividual components.

To power on system board 2 (sb2), type:

lom> poweron sb2

To power on the system and boot the Solaris OS, type:

lom> poweron

Sun Services


LOM shutdown Command

The LOM shutdown command shuts down the Solaris OS andpowers off the system to standby mode.

To power the system to standby mode, type:

lom> shutdown

Sun Services


LOM poweroff Command

The poweroff command powers off the entire system tostandby, a FRU, or a list of FRUs.

To power off system board 2 (sb2), type:

lom> poweroff sb2

To power off the entire system, type:

lom> poweroff

Sun Services


Power-Cycling the Sun Fire V1280/E2900Server Using the Power Rocker Switch

Sun Services


Power-Cycling The Sun Fire V1280/E2900Server Using the Power Rocker Switch

The switch is only operational if it has not been disabled usingthe setupsc LOM command.

To determine if the rocker switch is disabled or enabled usethe setupsc command.

lom> setupsc

System Controller Configuration-------------------------------SC POST diag Level [off]:Host Watchdog [enabled]:Rocker Switch [enabled]:Secure Mode [off]:

Sun Services


Power-Cycle Operations

The following power-cycle operations are available when youenable the system indicator board rocker switch.

• If the system is in standby mode, pressing the switchpowers on the system. This action is equivalent toexecuting the LOM poweron command.

• If the system running the Solaris OS, pressing theswitch for four seconds executes an orderly shutdown.This action is the equivalent of executing the LOMshutdown command.

• If the system is powered on, pressing this switch formore than four seconds executes a system power downto standby mode. This action is equivalent to the LOMpoweroff command.

Sun Services


Sun Fire 3800, 4800/E4900, 4810, and6800/E6900 Server Platform Assessment

and Management

Using the platform shell, you can:

• Configure the system controller network parameters

• Configure platform-wide parameters

• Configure segments and domains

• Monitor platform environments

• Display hardware configuration information

• Power on and power off the system and systemcomponents

Sun Services


Platform Shell Commands

The platform shell has a number of commands to displaysystem information, to set up system parameters, and to testsystem hardware.

Sun Services


The help Command

Without arguments, the help command lists all availableshell commands. When an argument is supplied, helpdisplays basic usage of the specified command.

schostname :SC> help addboardaddboard -- assign a board to a domainUsage: addboard -d <domain> <board> ... addboard -h

-d -- the domain to assign the board to -h -- display this help message

Sun Services


Connecting to the System Controller Shells

There are three methods to connect to the system controllerconsole:

• Serial connection – Use the serial port to connectdirectly to an ASCII terminal or to a network terminalserver (NTS).

• Secure Shell (SSH) connection – Use the Ethernet portto connect to the network using the SSH protocol.

• Telnet connection – Use the Ethernet port to connect tothe network using the Telnet protocol.

Sun Services


Initiate a Remote Connection With SSH

• The SC provides SSH server functionality.

• For security reasons, only connections requested byclient software running the SSHv2 protocol areaccepted by the SSH server on the SC.

Type the following at the system prompt:

% ssh schostname

Sun Services


Initiate a Remote Connection With Telnet

The SC provides the ability to connect using a networkconnection and telnet.

Type the following at the system prompt:

% telnet schostname

Sun Services


Navigating Between Shells on the Sun Fire3800, 4800/E4900, 4810, and 6800/E6900

Servers

Telnet ConnectionPress: CTRL ]

at the telnet> prompttype: send break

SSH Connection #.

Tip Connection ~.

Telnet ConnectionPress: CTRL ]

at the telnet> prompttype: send break

SSH Connection #.

Tip Connection ~.

Type: resume Type: break

Type: disconnect

Type: console-d domain IDPlatform Shell


OpenBoot PROM

Domain Shell

Sun Services


Managing Shell Passwords

• The password command sets the password for:

• The platform shell

• The domain shell

• An example of the password command example:schostname :SC> passwordEnter new password: xxyyzzEnter new password again: xxyyzz

If the password is already set and needs to be changed:

schostname :SC> passwordEnter current password: xxyyzzEnter new password: aabbccEnter new password again: aabbcc

Sun Services


Console Command Considerations

When you execute the console command from the platformshell, you get the domain shell prompt only when thekeyswitch is in the off position.

• If the domain is currently posting, posting messagesappear.

• If the domain is at run level 0, the OpenBoot PROM okprompt appears.

• If the domain is loading the Solaris OS, Solaris OSbooting messages appear.

Sun Services


Console Command Considerations

• If the domain is running the Solaris OS and no one haslogged in yet in this console, the Login: promptappears.

• If the domain is running Solaris OS and someone haspreviously logged in without logging out, you areplaced in this person’s shell.

Sun Services


Platform and System ControllerConfiguration

• Server platform setup is done from the platform shell.

• The setupplatform command sets up configurationparameters that pertain to the entire system.

Sun Services



• As the setupplatform command runs, promptsappear for the following parameters:

• Network configuration

• Loghost parameter identifier

• SNMP

• ACLs

• System controller POST level

• System controller failover

• Partition mode

Sun Services



Use the setupplatform command to manage the platform.

Example:

schostname :SC> setupplatformNetwork Configuration---------------------Is the system controller on a network? [yes]:Use DHCP or static network settings? [static]:Hostname [schostname]:IP Address [10.7.6.153]:Netmask [255.255.0.0]:Gateway [10.7.0.1]:DNS Domain [none]:Primary DNS Server [0.0.0.0]:Secondary DNS Server [0.0.0.0]:

Sun Services


Configuring the Platform TOD

• Use the setdate command to manage TOD.schostname :SC> setdate 020619152004.10

schostname :SC> setdate -r hostname Mon Apr 06 18:30:58 PDT2001

• Use the showdate command to display the currentTOD.schostname :SC> showdateMon Apr 06 18:31:40 PDT 2001

Sun Services


Viewing System Controller Details

schostname :SC> showsc -v

SC: SSC0Main System ControllerSC Failover: disabledClock failover enabled.

SC date: Wed Jan 14 08:23:01 PST 2004 PST GMT-8 Pacific Standard TimeSC uptime: 16 hours 1 minute 18 seconds

ScApp version: 5.16.0 Build_06Version build: 6.0Version String: 5.16.0RTOS version: 37

SC POST diag level: min

Clock source is: 75MHz

Sun Services


Viewing the SC Message Logs

The showlogs command displays the systemcontroller-logged events stored in the system controllermessage buffer. For example:

schostname :SC> showlogs -v

Sun Services


Viewing System Controller Connections

schostname :SC> connectionsID Hostname Idle Connected on Connected to1 localhost 01:02 Dec 31 16:00 Platform2 freedonia.Eng.Sun.COM 09:23 Dec 31 16:02 Domain B4 mos-eisley.Eng.Sun.COM 03:49 Dec 31 20:06 Platform8 awe177-5.AWE.Sun.COM 00:03 Jan 02 01:22 DomainA9 awe177-5.AWE.Sun.COM - Jan 02 01:26 Platform

Sun Services


Viewing System Controller CommandHistory

schostname :SC> history

Jan 14 08:20:29 : showboardsJan 14 08:20:54 : showb -hJan 14 08:21:05 : showb -p clockJan 14 08:21:14 : showb -p cpuJan 14 08:21:22 : showb -p memJan 14 08:21:28 : showb -p powerJan 14 08:21:35 : showb -p versionJan 14 08:22:04 : showboards -p versionJan 14 08:22:37 : showcom sb0Jan 14 08:22:50 : showcom -v ib6Jan 14 08:23:01 : showsc -vJan 14 08:23:31 : showenv -vJan 14 08:23:45 : history

Sun Services


System Controller Management

All of the server configuration information is stored on thesystem controller.

This consists of two types of information:

• Parameters specific to the system controller

• Parameters specific to the platform

Sun Services


System Controller Configuration

The system controller requires that network and POSTparameters be defined when initially configuring the Sun Firemidrange platforms.

• Configuring network parameters for the systemcontroller

• Configuring POST parameters

Sun Services


Rebooting the System Controller

• The reboot command reboots the system controller.

• It does not change any configuration settings.

Sun Services



schostname :SC> rebootAre you sure you want to reboot the system controller now? [no]yesWaiting for critical processes to finish. This may take a while.Critical processes have finished.

Rebooting. All SSH or telnet connections closed. Reestablish anyneeded connections.Dec 05 11:07:16 schostname Platform.SC: Stopping all services onthis SCDec 05 11:07:16 schostname Platform.SC: All services on this SChave been stopped.

Software Reset......

Sun Services



Type 0 for Platform Shell

Type 1 for domain A consoleType 2 for domain B consoleType 3 for domain C consoleType 4 for domain D consoleInput:

Sun Services


System Controller Failover

The system controller has failover capabilities forclock-related and processor function failures. If there is asystem controller failure, the remaining system controllercontinues to provide clocks and service to the platform.

Sun Services


System Controller Failover Prerequisites

The following support configuration is needed to configurethe system controller for failover:

• Both system controllers must have the same revision offirmware (a minimum of revision 5.13.x).

• A virtual IP address is assigned to the systemcontrollers using the setupplatform command. Thelogical host name or IP address identifies the workingmain system controller even after a failover occurs.

• You must synchronize the date and time between thetwo system controllers to ensure that the same timeservice is provided to the domains.

• You must set the system controller failover to enabled.

Sun Services


Failover

The system controller failover event is logged in the platformmessage log file, which is viewed on the console of the newmain system controller or through the showlogs commandon the system controller.

Platform Shell - Spare System Controllersp4-sc0:sc> Nov 12 01:15:42 sp4-sc0 Platform.SC: SC Failover: enabled andactive.Nov 12 01:16:42 sp4-sc0 Platform.SC: SC Failover: no heartbeat detectedfrom the Main SCNov 12 01:16:42 sp4-sc0 Platform.SC: SC Failover: becoming main SCNov 12 01:16:49 sp4-sc0 Platform.SC: Chassis is in single partition mode.Nov 12 01:17:04 sp4-sc0 Platform.SC: Main System ControllerNov 12 01:17:04 sp4-sc0 Platform.SC: SC Failover: disabledsp4-sc1:SC>

Sun Services


Controlling System Controller FailoverBehavior

You can enable system controller failover by using thesetupplatform command or the setfailover command.

Example of using the setupplatform command:

schostname :SC> setupplatform -p scSC POST diag Level [min]:Enable SC failover? [ yes ]:Logical Hostname or IP address [sp1-sc]:

Sun Services



The three options that can be used are: on, off , and force , ifsystem controller failover has not been enabled using thesetupplatform command. For example:

schostname :SC> setfailover onSC Failover: enabled but not active.

If configured in the setupplatform command, the followingoutput appears:

schostname :SC> setfailover onSC Failover: enabled and active.

Sun Services



You can turn off system controller failover by using thesetfailover off command:

schostname :SC> setfailover offSC Failover: disabled

You can force system controller failover by using thesetfailover force command:

schostname :SC> setfailover forceSC: SSC0Spare System ControllerSC Failover: enabled and active.Clock failover enabled.

Sun Services



This will abruptly interrupt operations on the other SystemController.This System Controller will become the main System Controller.Do you want to continue? [no] yesOct 26 00:10:33 schostname-sc0 Platform.SC: SC Failover: becomingmain SC ...Oct 26 00:10:37 schostname-sc0 Platform.SC: Chassis is in singlepartition mode.schostname-sc0:sc> Oct 26 00:10:42 schostname-sc0 Platform.SC:Main System ControllerOct 26 00:10:42 Oct 26 00:10:42 schostname-sc0 Platform.SC: Addedlogical IPaddressxxx.xxx.xxx.xxx-sc0Oct 26 00:10:43 schostname-sc0 Platform.SC: SC Failover: disabled

Sun Services


Determining the System Controller FailoverState

Use the showfailover command to determine the failoversystem controller state. For example:

schostname :SC> showfailover -v

SC: SSC0Main System ControllerSC Failover: disabledClock failover enabled

Sun Services


Platform Assessment

You can use system controller commands to monitor theplatform and domains. These commands include:

• showplatform

• showboards

• showcomponent

• showsc

• showenvironment

• history

• connections

• showlogs

• showfru

Sun Services


Assessing the Platform Configuration

The showplatform command displays all configurationvariables for this platform, including:

• Network attributes of the platform

• ACLs for each domain

• Domain status

Sun Services


The showplatform Command

schostname:SC> showplatform -vDomain Solaris Nodename Domain Status Keyswitch-------- ------------------ ----------------------- -------------A - Active - OpenBoot PROM onB - Powered Off offC - Powered Off offD - Powered Off off

Network-------The system controller is configured to be on a network.Network settings: staticHostname: marlins0IP Address: 10.6.11.10Netmask: 255.0.0.0Gateway:DNS Domain:Primary DNS Server:Secondary DNS Server:

Sun Services



MAC Address HostID ----------------- --------Domain A 08:00:20:e0:e6:f4 80e0e6f4Domain B 08:00:20:e0:e6:f5 80e0e6f5Domain C ff:ff:ff:ff:ff:ff ffffffffDomain D ff:ff:ff:ff:ff:ff ffffffffSSC0 08:00:20:e0:e6:f6 80e0e6f6SSC1 08:00:20:e0:e6:f7 80e0e6f7

System Serial Number: 105H25AA

Loghosts--------Loghost for Platform: 10.6.5.120Log Facility for Platform: local0

Sun Services



SNTP----SNTP server:

SNMP----SNMP Agent: enabledChassis Description: Sun Fire 3800Chassis Contact: Bob,Jackson,Dave , John , Sam ,Matt , TedChassis Location: back_of_roomTrap Hosts: 10.6.5.120Public Community String:Private Community String:SNMP packets received: 0SNMP packets sent: 263SNMP traps sent: 263

Sun Services



ACLs----ACL for Domain A: SB0 SB2 IB6 IB8ACL for Domain B:ACL for Domain C:ACL for Domain D:

SC--SC POST diag Level: minSC Failover: disabledLogical Hostname:

Security Options----------------Telnet servers: EnabledIdle connection timeout : No timeout

Sun Services



Partition Mode--------------Chassis is in single partition mode.

Frame information is not available.

Sun Services


Viewing Platform Component Status

The showboards command displays the assignmentinformation and status for all the components in the system;for example, system boards, I/O boards, fan trays, and so on.

For the domain shell, the display only shows componentswith the proper access permission for the current domain thatare listed in the ACL.

Sun Services



schostname :SC> showboardsSlot Pwr Component Type State Status Domain---- --- -------------- ----- ------ ------SSC0 On System Controller Main Passed -SSC1 On System Controller Spare - -ID0 On Sun Fire 3800 Centerplane - OK -PS0 On A145 Power Supply - OK -PS1 On A145 Power Supply - OK -PS2 On A145 Power Supply - OK -FT0 On Fan Tray Low Speed OK -FT1 On Fan Tray Low Speed OK -FT2 On Fan Tray Low Speed OK -FT3 On Fan Tray Low Speed OK -RP0 On Repeater Board (F3800) - OK -RP2 On Repeater Board (F3800) - OK -/N0/SB0 On CPU Board Assigned Not tested A/N0/SB2 On CPU Board Assigned Not tested A/N0/IB6 On CPCI I/O board (F3800) Assigned Not tested A/N0/IB8 On CPCI I/O board (F3800) Assigned Not tested A

Sun Services



schostname :SC> showboards -p cpus

CPUs----/partition0/domain0/SB0/bbcGroup0/cpuAB/cpusafariagent0: part_no=0x919c version=2 manufacturer=0x07d devid=0x102234058a9 (lot=0x022340 bin=1 wafer=22 col=5row=9) speed=750 e$ (8MB): J4400: 4MB, part# 0, serial# 090AXB J4300: 4MB, part# 0, serial# 090AVU DRAM DIMM Group 0 (1024MB): J13300: 256MB, part# 0, serial# 005898 J13400: 256MB, part# 0, serial# 005870 J13500: 256MB, part# 0, serial# 005878 J13600: 256MB, part# 0, serial# 005881 DRAM DIMM Group 1 (Empty):

Sun Services



schostname :SC> showboards -p clock

Component SSC0 Signal SSC1 Signal Signal Used Failover--------- ----------- ----------- ----------- --------SSC0 OK OK SSC0 EnabledRP0 OK OK SSC0 EnabledRP2 OK OK SSC0 Enabled/N0/SB0 OK OK SSC0 Enabled/N0/SB2 OK OK SSC0 Enabled/N0/IB6 OK OK SSC0 Enabled/N0/IB8 OK OK SSC0 Enabled

Sun Services



schostname :SC> showboards -p cpu

Component Description--------- -----------/N0/SB0/P2 UltraSPARC-III, 0MHz, 8M ECache/N0/SB0/P3 UltraSPARC-III, 0MHz, 8M ECache/N0/SB2/P2 UltraSPARC-III, 0MHz, 8M ECache/N0/SB2/P3 UltraSPARC-III, 0MHz, 8M ECache

Sun Services



schostname :SC> showboards -p memory

Component Size Reason--------- ---- ------/N0/SB0 2048 MB/N0/SB2 2048 MB

schostname :SC> showboards -p power

Component Pwr Grid--------- --- ----/N0/SB0 On Grid 0/N0/SB2 On Grid 0/N0/IB6 On Grid 0/N0/IB8 On Grid 0

Sun Services



schostname :SC> showboards -p version

Component Compatible Version--------- ---------- -------SSC0 Reference 5.13.0 Build 12/N0/IB6 Yes 5.13.0 Build 12/N0/IB8 Yes 5.13.0 Build 12/N0/SB0 Yes 5.13.0 Build 12/N0/SB2 Yes 5.13.0 Build 12

Sun Services


Viewing Platform Component Details

The showcomponent command displays a component or a listof components. When this command is typed in the platformshell, the output shows all components in the platform.

In the domain shell, it shows only components with theproper access permissions.

This command relies on the hierarchical nature of the devicetree traversing all child devices.

Sun Services



schostname :SC> showcomponent sb0Component Status Pending POST Description--------- ------ ------- ---- -----------/N0/SB0/P0 enabled - untest empty/N0/SB0/P1 enabled - untest empty/N0/SB0/P2 enabled - pass UltraSPARC-III,750MHz, 8M ECache/N0/SB0/P3 enabled - pass UltraSPARC-III,750MHz, 8M ECache/N0/SB0/P0/B0/L0 enabled - untest empty/N0/SB0/P0/B0/L2 enabled - untest empty/N0/SB0/P0/B1/L1 enabled - untest empty/N0/SB0/P0/B1/L3 enabled - untest empty/N0/SB0/P1/B0/L0 enabled - untest empty/N0/SB0/P1/B0/L2 enabled - untest empty/N0/SB0/P1/B1/L1 enabled - untest empty/N0/SB0/P1/B1/L3 enabled - untest empty/N0/SB0/P2/B0/L0 enabled - pass 512M DRAM/N0/SB0/P2/B0/L2 enabled - pass 512M DRAM

Sun Services



/N0/SB0/P2/B1/L1 enabled - untest empty/N0/SB0/P2/B1/L3 enabled - untest empty/N0/SB0/P3/B0/L0 enabled - pass 512M DRAM/N0/SB0/P3/B0/L2 enabled - pass 512M DRAM/N0/SB0/P3/B1/L1 enabled - untest empty/N0/SB0/P3/B1/L3 enabled - untest empty

Sun Services



schostname :SC> showcomponent ib6Component Status Pending POST Description----------- ------- ------- ---- -----------/N0/IB6/P0 enabled - pass IO controller 0/N0/IB6/P1 enabled - pass IO controller 1/N0/IB6/P0/B0 enabled - pass 66/33MHz PCI Bus/N0/IB6/P0/B1 enabled - pass 33MHz PCI Bus/N0/IB6/P0/B1/C0 enabled - pass 33MHz 5V Short PCI card/N0/IB6/P0/B1/C1 enabled - failed 33MHz 5V Short PCI card/N0/IB6/P0/B1/C2 enabled - pass 33MHz 5V LOng/Short PCI card/N0/IB6/P0/B1/C3 enabled - pass 33MHz 5V LOng/Short PCI card

Sun Services


Powering On and Off System Components

• Use the poweron command to power on components.

• Use the poweroff command to power off components.

• Powering on and powering off examples:schostname :SC> poweron grid0schostname :SC> poweroff grid0

Sun Services


Updating the Platform Firmware

You can update flash PROMs in the following systemcomponents:

• A system controller board (operating system orapplication)

• A system board

• An I/O board

• A domain

• All components in the platform

Sun Services


Using the Root or a User Account to FlashUpdate the System

Examples of the flashupdate command:

schostname :SC> flashupdate -y -f ftp://\root:[email protected]/firmware scapp rtos

As part of this update, the system controller will automaticallyreboot.RTOS will be upgraded automatically during the next boot.ScApp will be upgraded automatically during the next boot.Rebooting will interrupt any current operations.This includes keyswitch changes, Solaris rebootsand all current connections.Do you want to continue? yes (-y)Waiting for critical processes to finish. This may take a while.Critical processes have finished.No boards can be updated.

Sun Services


Using the Root or a User Account to FlashUpdate the System

Rebooting the SC to automatically complete the upgrade.

Rebooting. All telnet connections closed. Reestablish any neededconnections.

Sun Services


Introducing Segments and Domains

Applications Applications Applications Applications





Open Boot PROM Open Boot PROM Open Boot PROM Open Boot PROM

Domain Shell A(SC)

Domain Shell B(SC)

Domain Shell C(SC)

Domain Shell D(SC)

Platform Shell (System Controller)

Platform Hardware

Sun Services


Segments

• A segment refers to all or part of the Sun Fireplaneinterconnect.

• Dual-partition (segment) mode splits the Sun Fireplaneinterconnect into two independent snoopy coherentsystems.

• The Sun Fireplane switch boards are dividedbetween the two segments.

• All connections between segments are disabled.

• To enable dual-segment mode, run:schostname :SC> setupplatform -p partitionConfigure chassis for single or dual-partition mode? [single]:dual

Sun Services


Domains

• A domain is a logical division of a segment.

• Each domain has an independent instance of theSolaris OS.

• Each segment can have a maximum of two domains.

• Domains are useful for testing new applications oroperating system updates.

• Temporary resources can be borrowed from existingdomains.

• Upon completion, resources can be returned.

• System reboot is not required.

Sun Services


Sun Fire 6800/E6900 Servers ConfiguredWith Four Domains

RP0/RP1 RP2/RP3

SB0 SB2 IB6 SB4 IB8 SB1 SB3 IB7 SB5 IB9

Domain A Domain B Domain C Domain D

Segment 0 Segment 1

Sun Services


Server Configuration Domain IDs


Sun Fire 3800 server One segment, one domain A

One segment, two domains A, B

Two segments, two domains A, C


One segment, one domain A



Sun Fire 4810 server One segment, one domain A



Sun Services





One segment, one domain A


Two segments, two domains A, C or A, D orB, C or B, D

Two segments, three domains A, B, C or A, B, D orA, C, D or B, C, D

Two segments, four domains A, B, C, D

Sun Services


Segment and Domain Configurations

* You need to understand performance and availabilitytrade-offs before choosing this configuration.

Server SegmentsDomainsperSegment

DomainAddressSnoopingBandwidth

FireplaneDataBandwidth

AvailableConfiguration

SunFire3800,4800/E4900,or4810server

1 1 100% 100% Yes

1 2 50% 100% * See note

2 1 100% 50% Yes

2 2 N/A N/A N/A

Sun Services


Segment and Domain Configurations

* You need to understand performance and availabilitytrade-offs before choosing this configuration.

Server SegmentsDomainsperSegment

DomainAddressSnoopingBandwidth

FireplaneDataBandwidth

AvailableConfiguration

SunFire6800/E6900servers

1 1 100% 100% Yes

1 2 50% 100% * See note

2 1 100% 50% Yes

2 2 50% 50% * See note

Sun Services


Domain Access Control List (ACL)

• Each domain has an associated ACL.

• The ACL restricts:

• testboard , addboard , or deleteboard requests tospecific platform components.

• addboard or deleteboard requests to all platformcomponents.

Sun Services


Configuring ACLs

Configure ACLs by using the showplatform -p aclscommand. Type:

schostname :SC> setupplatform -p acls

ACL for domain A [SB0 SB1 SB2 SB3 SB4 SB5 IB6 IB7 IB8 IB9]: -rSB1 SB3 SB5 IB7 IB9

ACL for domain B [SB0 SB1 SB2 SB3 SB4 SB5 IB6 IB7 IB8 IB9]: -

ACL for domain C [SB0 SB1 SB2 SB3 SB4 SB5 IB6 IB7 IB8 IB9]: -rSB0 SB2 SB4 SB5 IB6 IB8 IB9

ACL for domain D [SB0 SB1 SB2 SB3 SB4 SB5 IB6 IB7 IB8 IB9]: -rSB0 SB1 SB2 SB4 IB6 IB7 IB8

Sun Services


Viewing ACLs

Display the current ACLs by using theshowplatform -p acls command. Type:

schostname :SC> showplatform -p aclsACL for Domain A: SB0 SB2 IB4 IB6 IB8ACL for Domain B:ACL for Domain C: SB1 SB3 IB7ACL for Domain D: SB3 SB5 IB9

Sun Services


Starting, Stopping, and Power-CyclingDomains

Domains are power-cycled and started by running thesetkeyswitch command from the domain shell.

Use the setkeyswitch command to start a domain.

schostname :A> setkeyswitch on

Use the setkeyswitch command to stop a domain.

To stop a domain:schostname :A> setkeyswitch off

Sun Services


Introducing Device Configuration

The system maintains information about all configureddevices at the hardware level under the control of OpenBootPROM, and at the software level under the control of theoperating system.

Whenever a device is physically added or removed structuresthat describe the device to the hardware and the operatingsystem must be created or removed.

OpenBoot PROM maintains information about availabledevices in a structure called the device tree. If a device is notpresent in the OpenBoot PROM device tree, it is not availablefor use by the operating system.

Sun Services


Introducing Device Configuration

Solaris OS maintains its knowledge of available devices usinga set of hierarchically organized device files. These files arelocated in the /devices directory of the root file system.

Additionally, the Solaris OS uses the path_to_inst file in the/etc directory to manage device instance names and links inthe /dev directory to enable logical device addressing.

Sun Services


OpenBoot PROM Capabilities

OpenBoot firmware enables you to do the following tasks:

• Boot the operating system

• Run diagnostics

• Modify system start-up parameters

• Load and execute programs

• Troubleshoot

Sun Services


OpenBoot PROM Capabilities

Applications


OpenBoot PROM 5

LOM Shell


PlatformHardware

Applications


OpenBoot PROM 5

Domain Shell

Platform Shell

Platform Hardware

Sun FireMidrange Server

Sun Services


Device Tree

Each device node can have the following components:

• Properties – Data structures describing the node and itsassociated device.

• Methods – The software procedures used to access thedevice.

• Data – The initial values of the private data used by themethods.

• Children – Other device nodes attached to a given nodeand that lie directly below it in the device tree.

• Parent – The node that lies directly above a given nodein the device tree.

Sun Services


Sun Fire V1280/E2900 ServerDevice Tree Components

ssm@0,0

Memory-controller SUNW,UltraSPARC-III orSUNW,UltraSPARC-IV12

12

IOC 0Pci@18

600000 700000

IOC 1Pci@19

600000 700000

PCI Slot 4

PCI Slot 3

PCI Slot 2

100 Mbit Ethernet 0

GigabitEthernet 1

GigabitEthernet 0

DVD Drive

PCI Slot 1

PCI Slot 0

PCI Slot 5

Tape DriveTarget 5

Hard DriveTarget 1

Hard Drive Target 0

Sun Services


SunFire3800,4800/E4900,4810,and6800/E6900Server Device Tree Components

ssm@0,0

Memory-controller SUNW, UltraSPARC III or SUNW, UltraSPARC IV

IOC

2424

8 pci@ 18,600000 pci@ 18,700000

PCI slot

pci@1

PCI slot

pci@1

I/O controller

SUNW,xyz

Device

I/O controller

SUNW,xyz

Device

3

Sun Services


Mapping Node Devices

Node Identifier

/ssm@0,0

Node ID 0

Sun Services



Device Tree Entries

CPU offsetCPU AID

/ssm@0,0/SUNW,UltraSPARC-III@b,0

Sun Services



Memory Controller Device Tree Entries

Memory offsetMemory AID

/ssm@0,0/SUNW,memory-controller@b,400000

Sun Services


CPU and Memory AID Assignments

BoardNumber

Location P0 AID P1 AID P2 AID P3 AID

0 SB0 0 (0x0) 1 (0x1) 2 0x(2) 3 (0x3)

1 SB1 4 (0x4) 5 (0x5) 6 (0x6) 7 (0x7)

2 SB2 8 (0x8) 9 (0x9) 10 (0xa) 11 (0xb)

3 SB3 12 (0xc) 13 (0xd) 14 (0xe) 15 (0xf)

4 SB4 16 (0x10) 17 (0x11) 18 (0x12) 19 (0x13)

5 SB5 20 (0x14) 21 (0x15) 22 (0x16) 23 (0x17)

Sun Services


Mapping I/O Devices

Nodes in a Sun Fire midrange server I/O path are:

• Node ID

• IOC AID

• IOC PCI bus offset

• Device number

• PCI controller

• Device instance

Sun Services


Mapping I/O Devices

Six Device Tree Nodes

/ssm@0,0/pci@19,700000/pci@3/SUNW,isptwo@4/sd@5,0

Device instancePCI controllerDevice #Bus offsetNode ID IOC AID

Sun Services


Decoding IOC AID

The following is an example of IOC AID:

/ssm@0,0/pci@19.....

You can calculate the IOC AID by performing the followingsteps:

1. Convert the IOC AID from hexadecimal to decimal.

For example:

19 (hexadecimal) = 25 (decimal)

Sun Services


Decoding IOC AID

2. Calculate the even or odd IOC by dividing the IOCAID by 2 and subtracting 6.

For example:

(25/2) - 6 = 6.5

A remainder of 6.x indicates that the I/O board is IB6. The factthat the result has a fraction (x.5), indicates that this is an oddAID (IOC 1).

Sun Services


IOC AID Assignments

BoardNumber

LocationEven AID(IOC 0)

Odd AID(IOC 1)

0 IB6 24 (0x18) 25 (0x19)

1 IB7 26 (0x1a) 27 (0x1b)

2 IB8 28 (0x1c) 29 (0x1d)

3 IB9 30 (0x1e) 31 (0x1f)

Sun Services


IOC PCI Bus Offset

• Bus A is referenced by offset 600000.

• Bus B is referenced by offset 700000.

Bus offset:

Bus offset

/ssm@0,0/pci@19,700000.....

Sun Services


Device Number

The PCI controller slots, located in the PCI (cPCI) chassis, arereferenced by the device number.

Device number:

/ssm@0,0/pci@19,700000/pci@3.......

Device #

Sun Services


Sun Fire V1280/E2900 ServerSix-Slot PCI Chassis

PCI Slot 4pci@3

PCI Slot 3pci@2

PCI Slot 2pci@1

ide@3

PCI Slot 1pci@2

PCI Slot 0pci@1

pci@18,600000

pci@18, 700000

pci@19,600000

pci@19,700000

IOC 0

BA

IOC 1

AB

PCI Slot 5pci@1

Hard DriveTarget 1

disk@1,0

Hard Drive Target 0

disk@0,0

Tape DriveTarget 5st@5,0

network@2

network@1

DVD-ROMsd0,0

scsi@2

Sun Services


SunFire4800/E4900,4810,and6800/E6900Server Eight-Slot PCI Chassis

pci@18,700000

pci@18,600000

pci@19,700000

pci@19,600000

IOC 0

AB

IOC 1

BA

pci@1 (Slot 0)

pci@2 (Slot 1)

pci@3 (Slot 2)

pci@1 (Slot 3)

pci@1 (Slot 4)

pci@2 (Slot 5)

pci@1 (Slot 7)

pci@3 (Slot 6)

Sun Services


Sun Fire 3800 Server Six-Slot cPCI Chassis

pci@18,600000

pci@18,700000

pci@19,600000

pci@19,700000

IOC 0

BA

IOC 1

AB

pci@1 (Slot 0)

pci@1 (Slot 1)

pci@1 (Slot 2)

pci@2 (Slot 3)

pci@1 (Slot 4)

pci@2 (Slot 5)

Sun Services


SunFire4800/E4900,4810,and6800/E6900Server Four-Slot cPCI Chassis

pci@18,600000

pci@18,700000

pci@19,600000

pci@19,700000

IOC 0

BA

IOC 1

AB

pci@1 (Slot 0)

pci@1 (Slot 1)

pci@1 (Slot 2)

pci@1 (Slot 3)

Sun Services


Module 4

Troubleshooting the Sun Fire MidrangeServers

edited 02/08 by LG

Sun Services


Objectives

• Describe the basic architecture of the Sun Fire serversystem

• Describe the two levels of Sun Fireplane interconnectswitches

• Describe how the system boards provide CPU andmemory resources to the operating system in Sun Firemidrange servers

• Describe how Sun Fire midrange servers use PCI andcPCI I/O assemblies

• Describe how the Sun Fireplane interconnect plane isthe main system bus of the Sun Fire family of servers

Sun Services


Objectives

• Describe the different integrated service processorssupported by the Sun Fire midrange server architecture

• Describe the troubleshooting methodology for faultanalysis and diagnosis of failed components

• Describe the system tools available for gatheringbackground information on Sun Fire midrange serverproblems

• Describe the testing tools available for isolating faultsin the Sun Fire midrange servers

• Describe how Sun Fire midrange servers use parity todetect system interconnect errors

Sun Services


Objectives

• Describe how Sun Fire midrange server subsystemscan use error correcting code (ECC) to recover fromerrors

• Describe how console port error messages are reportedto help isolate faulty components in the console bushub (CBH)

• Describe how Sun Fire midrange server environmentalfaults are reported

• Describe the enhanced availability featuresimplemented in the new firmware update 5.15.3

• Describe how blacklisting is used to reconfigure SunFire midrange server hardware to avoid parts witherrors

Sun Services


Objectives

• Describe how domain shell operating messagesprovide an aid to troubleshooting a system error in SunFire midrange servers.

• Recover from a hung domain

Sun Services


Relevance

• Which diagnostic tools are available to test the Sun Fireservers?

• How do you free a hung domain?

• How do you create an action plan to replace failedFRUs?

Sun Services


Additional Resources

• Sun Microsystems, Inc. Sun Fire™ 6800/4810/4800/3800Systems Platform Administration Manual, part number817-0999.

• Sun Microsystems, Inc. Sun Fire™ 6800/4810/4800/3800System Controller Command Reference Manual, partnumber 817-1000.

• Sun Microsystems, Inc. Sun Fire™ V1280/Netra™ 1280System Administration Guide, 817-0509.

• Sun Microsystems, Inc. Sun Fire™ V1280/Netra™ 1280Systems Service Manual, part number 817-0510.

• Sun Microsystems, Inc. Sun Fire™ V1280/Netra™ 1280System Controller Command Reference Manual, partnumber 817-0511.

Sun Services


Additional Resources

• Sun Microsystems, Inc. Sun Fire™ Midrange SystemsHardware Reference Manual, part number 805-7363.

• Sun Microsystems, Inc. Sun Fire™ 6800/4800/4810/3800Systems Site Planning Guide, part number 805-7365.

URL Resources – Public Web sites

• http://sunsolve.sun.com/handbook_pub/

• http://www.sun.com/software/solaris/sunmanagementcenter/hwds/

• http://docs.sun.com

• http://www.sun.com/blueprints/0803/817-3342.pdf

Sun Services


Operational Overview

Sun Fire midrange server architecture includes:

• New system bus architecture based on the SunFireplane interconnect

• High-performance Sun Fireplane interconnect switchtechnology

• An enhanced CPU architecture starting at speeds of750 MHz and greater

• An industry standard I/O incorporating PCI and cPCItechnology

Sun Services


Sun Fireplane InterconnectOperational View

Data Interconnect

CPU

EPCI PCI

Address

IOC

CPU DCDS

Memory Memory

CPU

EPCI PCI

IOC

CPU DCDS

Memory Memory

Sun Services


Functionality

• The Sun Fireplane interconnect provides a 288-bit datapath between the UltraSPARC III processors and thePCI I/O bridge (IOC) with a high clock frequency of150 MHz.

• The connection between the Sun Fireplane interconnectdevices (UltraSPARC III processors and PCI andenhanced PCI [EPCI] bridges) and the data path usespoint-to-point connections.

• The UltraSPARC III processors are interfaced to thedata path using the dual CPU data switch (DCDS).

Sun Services


Snoopy Bus Architecture

• The Sun Fireplane interconnect implements cachecoherency through a technique known as snooping.

• Each cache controller monitors the addresses of alltransactions on the system interconnect, watching fortransactions that update addresses it already possesses.

• Address snooping is implemented between all CPUs ina domain.

Sun Services


Sun Fire Servers’ Logical Architecture

2

DX

AR

DX

AR

DX

AR

CPUand

Memory

CPUand

Memory

DX

AR

4

PCI IOC

2

2

6 System boards

I/O boards

Centerplane/Baseplane

Sun Fireplanesecond-level(L2) switches

Sun Fireplane first-level

(L1) switches

Data

Address

Sun Services


Sun Fire 6800/E6900 ServerCrossbar Configuration

SB0

SB1

SB2

SB3

SB4

SB5

IB6

IB7

IB8

IB9

SB0

SB1

SB2

SB3

SB4

SB5

IB6

IB7

IB8

IB9Crossbar configuration:

SB0, SB2, SB4, and IB6SB1, SB3, SB5, and IB7

Sun Services


UltraSPARC IV Processor ChipBlock Diagram

E-Cache Data (SRAM)Max: 16MB (8MB each)

Data256+18ECC

250-300MHzAddress19

E-CacheTag

E-CacheTag

Core 0(US-III Cu)

Core 1(US-III Cu)

MCU

UltraSPARC IV

Transaction Request Signals

Safari Bus 150MHz

256+18ECC+14MTagData

Data

Data

512+36ECC+28MTag

128+9ECC+7MTag

150MHz

DCDS

Memory(SDRAM)

1575MHz

Address

Sun Services


System Board Logical Block Diagram

SDCDIMMs

DIMMs

DCDS

CPU 1

SBBC SRAM FPROM

AR

DX

0,1

2,3

4,5

SBBC0

SBBC1

0

1

2

3

6,7

8,9

SC0

SC1

8,9

6,7

5

4

0,1

2,3

5

4

DataAddressData RouteBootbusConsole BusControl Signals

CPU 0

DIMMs

DIMMs

DCDS

CPU 3

SBBC SRAM FPROM

CPU 2

Sun Services


IB_SSC PCI I/O Assembly LogicalBlock Diagram

SBBC SRAM FPROM

Internal SCCRConnection

LEDs

ExternalAlarms

Connection

AddressRepeater

IOC 1

IOC 0

DataController

DataSwitch

DataSwitch

MaintenanceBus Devices

ExternalGBE

Connection

GigabitEthernet

Transceiver

GigabitEthernetController

ExternalGBE

Connection

GigabitEthernet

Transceiver

GigabitEthernetController

PCI Riser

3 x 33MHz Slots

PCI#2

PCI#3

PCI#4

PCI#0

PCI#1

PCI#5

Dual ChannelSCSI

ControllerIDE

Controller

InternalIDE

Connection

To DVD

1 x 66MHz Slot

2 x 33MHz Slots

PCI Riser

InternalSCSI

Connection

ExternalSCSI

Connection

To 2 x SCSI disksand DDS-4 tape

Con

nect

ion

to F

irepl

ane

A

A

B

B

Sun Services


Eight-Slot PCI I/O AssemblyLogical Block Diagram

SDC

IOC 1BA

Enhanced PCIPCIPCIPCI

AR

DX

2,3

0,1

1

0

8,9

6,7

SC0

SC1

9

8

7

6

2

0

6,7

8,9

DataAddressData RouteBootbusConsole BusControl SignalsPCI Bus

IOC 0

SBBC SRAM FPROM

BA

Enhanced PCIPCIPCIPCI

Slot 7 Slot 6 Slot 5 Slot 4

Slot 3 Slot 2 Slot 1 Slot 0

Sun Services


Compact PCI I/O Assemblies

• The Sun Fire 4800/E4900, 4810, and 6800/E6900 servermodels support the four-slot cPCI I/O assembly.

• The Sun Fire 3800 server model only supports thesix-slot cPCI I/O assembly.

Sun Services


Four-Slot cPCI I/O AssemblyLogical Block Diagram

SDC

IOC 1BA

Enhanced PCIPCI

AR

DX

2,3

0,1

1

0

8,9

6,7

SC0

SC1

9

8

7

6

2

0

6,7

8,9


IOC 0

SBBC SRAM FPROM

BA

Enhanced PCIPCI

Slot 1 Slot 3

Slot 0 Slot 2

Sun Services


Six-Slot cPCI I/O AssemblyLogical Block Diagram

SDC

IOC 1BA

Enhanced PCIPCIPCI

AR

DX

2,3

0,1

1

0

8,9

6,7

SC0

SC1

9

8

7

6

2

0

6,7

8,9


IOC 0

SBBC SRAM FPROM

BA

Enhanced PCIPCIPCI

Slot 1Slot 4Slot 5

Slot 0Slot 2Slot 3

Sun Services


Sun Fireplane Interconnect

• The Sun Fireplane interconnect is the main system busof the Sun Fire family of servers.

• The Sun Fireplane interconnect is the coherentshared-memory architecture.

• The Sun Fireplane interconnect operates similar to anetwork.

Sun Services


Address Interconnect Levels

Address Repeater Address Repeater

Sun Fireplane Switch

Level 2:Fireplane

Switch

Level 1:Board

System Board4 CPUs/4 Memory Units

I/O Board2 PCI Controllers

Proc Proc ProcProc

Memory Memory MemoryMemory

PCIController

PCIController

PCI Cards PCI Cards

Sun Services


Data Interconnect Levels

Data Switch

GB/s*4.8 GB/s2.4

GB/s2.4GB/s2.4GB/s2.4GB/s2.4

GB/s2.4GB/s2.4GB/s2.4GB/s2.4

GB/s4.8 GB/s4.8 GB/s1.2

0.4 GB/s 0.4 GB/s

0.2 GB/s 0.2 GB/s

GB/s1.2

Data Switch

Level 2 Fireplane Data Switch

Level 2:Fireplane

Switch

Level 1:Board

Level 0:Processor/

Memory

System Board4 CPUs/4 Memory Units

I/O Board2 PCI Controllers

Dual CPUData Switch

Dual CPUData Switch

Proc Proc ProcProc

Memory Memory MemoryMemory

PCIController

PCIController

PCICard

PCICard

PCICard

PCICard

* “GB/s” numbers are peak bandwidths at each part of the interconnect.

Sun Services


System Controller Interconnect LogicalBlock Diagram

BootBus SBBC

SBBC

SBBC

SBBCService

procCBH

Console Arbitration

6 CPU/Mem + 4 I/O + 4 L2

Fireplane Switch

Fireplane Switch

AR

AR

AR

SDC

SDC

AR

SDC

SDC

PCIBootBus

CPU 0 CPU 1

CPU 0 CPU 1

CPU/Mem

I/O

SC

SBBC Serviceproc

CBHPCI

PCI

SC

IOCIOC

Sun Services


Sun Fire V1280/E2900 Server SystemController Logical Block Diagram

Baseplane Error Lines (14) Console Buses* (32) Global I2C Buses* Clocks SC Serial Link

Echip

TODNVRAM

NVRAMFPROM

ScApp FPROM

DRAM

bootFPROM

Clocks

16552Serial

Controller

16552Serial

Controller

SRAM

SEEPROM

SensorsMisc.

Registers

SBBC

RIO

MicroSPARC IIep

ConsoleBusHUB

Console Bus

(8) I2C Buses I2CMUXs

Loca

l I2 C

Bus

Rear Panel 10/100BASE-T Ethernet TTYA, TTYB (reserved)

PC

I (33

MH

z)

* Not all Console and I2C Buses are used.

Sun Services


Sun Fire 3800, 4800/E4900, 4810, and6800/E6900 Server System Controller

Block DiagramCenterplane

ResetLines

ErrorLines

14x ConsoleBuses

Serial toOther SC Global I C BusesClocks

TODNVRAM

NVCIFPROM

ScappsFPROM

16552SC serial

16552TTY

clockscontrol

vxworksboot

FPROMDRAM

SRAM

temp SEEPROMvolt

Echip Misc.Registers

SBBC

RIO

CBHConsole Bus

PROMBus

I C Buses

I CMUXs

Local I C

Ebus

Panel 10/100BASE-T Ethernet TTYA, TTYB

PCI (33 MHz)

SC (MicroSPARC IIep)

2

2

2

2

Sun Services


Boot Bus Controller (SBBC) ASIC

SRAM FPROM

Sensors LEDs

Console Bus

JTAG

PROM Bus

To PCI Controller

To Processer

To DatapathController

To Processer

PCI Bus

Boot Bus

I C BusesBoot Bus Controller 2

Sun Services


System Data Controller (SDC)Console Bus Connections

SC0

AR

3

2

SC1

4

SBBC0

0

SBBC1(SB only)

1

SDC

Console Bus

Multiplexing Section

Sun Services


Basic Clock Distribution

SelectPLL

150 MHz

75 MHz

LocalClockLoads

FireplaneSwitchBoard

This is the failback loop.It determinesthe currentactive clock.

I/O Board

Clock SC 0

75 MHz Outputs

Clock SC 1

75 MHz Outputs

Sun Services


Fault Analysis Method

• Fault analysis – Identify the problem and organize factgathering and comparisons

• Diagnosis – Organize the actual discovery, testing,repair, and reporting of the problem

Sun Services


Eight Steps of Fault Analysis and Diagnosis

1. State the problem.

2. Describe the problem.

3. Identify differences.

4. List relevant changes.

5. Generate likely causes.

6. Test likely causes.

7. Verify the most likely cause.

8. Take action to correct the fault.

Sun Services


Sun Fire Midrange Server Fault AnalysisMethodology

• Identify the FRUs referenced in the error logs on thesystem controller or loghost

• Determine the current state of those FRUs using thevarious show commands

• Use the Rule of 3 method to isolate the faulty FRUs

• Replace identified FRUs one at a time per the actionplan

Sun Services



Parity errors can be:

• Data bus errors

• Single-bit errors – Domain is paused, erroridentified and corrected using ECC, error logged

• Multi-bit errors – Domain is paused, multi-bit errorcondition identified, domain fails, core dump of OS,error logged

• Address bus errors – Domain is paused, address buserror condition identified, domain fails, core dump ofOS, error logged

Sun Services



Data bus errors are recorded for the following ASICS:

• Level 1 and Level 2 data switch (DX)

• System data controller (SDC)

• Dual CPU data switch (DCDS)

Address bus errors are recorded for the Level 1 and Level 2address repeater (AR)

Sun Services


Rule of Three Fault Isolation

• If a single system board or I/O board are identified inthe error message, replace it.

• If multiple FRUs are identified:

• Replace the system board (or system boards) first

• Replace the I/O board (or boards) next

• Replace the switchboard (or switchboards) next

• Replace the centerplane (only if one of the identifiedboards has been recently removed and pins havebeen bent)

Sun Services


Gathering Background Information

• The system controller has the primary responsibilityfor the collection, interpretation, and response.

• Each system controller gets error messages from eachof the boards in a domain.

• The system boot bus controller (SBBC) determines theaction to take on the errors.

• The following are typical actions:

• Setting the appropriate error status bits

• Asserting error pause to stop further addresspackets

• Interrupting the system controller

Sun Services


Error Repositories and Commands

Platform/Prompt Domain/Shell Loghost

showlogs (:SC>) showlogs (domain shell ) The platform or domain shellmessages can be diverted toan external loghost byentering the loghost IPaddress or host name whenusing the setupplatform orsetupdomain commands,respectively.

show-post-results(OpenBoot PROM)

Sun Services


Error Repositories and Commands

Platform/Prompt Domain/Shell Loghost

prtdiag(Solaris OS)

/var/adm/messages(Solaris OS)Set up syslog.conf toredirect error messages tothis file.

Sun Services


Sun Explorer Software Data Collector

• Gathers extensive domain information

• Generates tar.Z and email messages to the Sundatabase

• Uses a lightweight shell script for less executionoverhead

Sun Services


Running Sun Explorer Software on the SunFire Midrange Server

The Sun Explorer software has a script specifically for the SunFire midrange servers. Run the scextended script from thedomain.

# explorer -w default,scextended,fru

To download a free copy the Sun Explorer software, click theDownload Explorer link.

Sun Services


Viewing a Sun Explorer Software Capture

# ls -al /opt/SUNWexplo/explorer.82a84a2f.domain-b-\2001.08.22.06.43-rw-r--r-- 1 dave crsdev 1538 Aug 21 23:46 README-rw-r--r-- 1 dave crsdev 1596 Aug 21 23:46 defaultsdrwxr-xr-x 6 dave crsdev 4096 Aug 21 23:46 disksdrwxr-xr-x 6 dave crsdev 4096 Aug 21 23:43 etcdrwxr-xr-x 8 dave crsdev 4096 Aug 21 23:43 initdrwxr-xr-x 2 dave crsdev 96 Aug 21 23:43 licensedrwxr-xr-x 2 dave crsdev 4096 Aug 21 23:46 lpdrwxr-xr-x 2 dave crsdev 4096 Aug 21 23:46 messagesdrwxr-xr-x 3 dave crsdev 4096 Aug 21 23:46 netinfodrwxr-xr-x 2 dave crsdev 4096 Aug 21 23:46 patch+pkg-rw-r--r-- 1 dave crsdev 11 Aug 21 23:46 revdrwxr-xr-x 3 dave crsdev 96 Aug 21 23:45 sf15000drwxr-xr-x 4 dave crsdev 4096 Aug 21 23:46 sysconfigdrwxr-xr-x 3 dave crsdev 96 Aug 21 23:46 systemdrwxr-xr-x 3 dave crsdev 4096 Aug 21 23:46 var

Sun Services


Interpreting Sun Fire Midrange ServerLEDs

• System boards (system controller, system board, SunFireplane switchboards, and I/O assemblies)

• Fan trays

• Power supplies

• Redundant transfer units (RTUs)

• FrameManager

• cPCI cards

• Disk drives

Sun Services


LED Status Code Summary

FRU Green(Normal)

Amber(Fault)

Amber(Hot-plug)

State

Frame-Manager(used onthe Sun Fire3800, 4810,and 6800serverchassis)

On – – The redundant transfer switch isenabled.

– On – A rack or chassis failed.

– – On Might apply to the rack orchassis.Certain FRUs can behot-plugged.

All other combinations areillegal.

Sun Services



FRUGreen(Normal)

Amber(Fault)

Amber(Hot-plug)

State

System,SunFireplaneswitch, andI/O boards

– – – The board power is off.

On – – The board power is on, nofailure, and the board is active.

On On – The component failed (a faultcondition is present), and theboard is still active.

– On On The board failed, and it can behot-plugged.

– On – Illegal.

On – On Illegal.

Sun Services



FRUGreen(Normal)

Amber(Fault)

Amber(Hot-plug)

State

SC(SC0 andSC1)

On – – Auxiliary power is on for bothprimary and secondary systemcontrollers.

SC0(with SC1present)

– On On The system controller is faulty(with SC failover).A failover (SC1 present) mightresult in turning on the faultyLED, depending on the natureof the failure that caused thefailover.

SC1 On The system controller is faulty(check).

Sun Services



FRUGreen(Normal)

Amber(Fault)

Amber(Hot-plug)

State

Redundanttransferswitch andpowersupplies

– – On The auxiliary power is on, thebreakers are on, and the mainpower is off.

On – – The main power supplies areon; that is, the unit is active.

– On On The fault is on the main powersupply.

– On – The fault affects both the mainand auxiliary power supplies.

On On – The fault is on the auxiliarypower supply.

Sun Services



FRUGreen(Normal)

Amber(Fault)

Amber(Hot-plug)

State

Fans On – – The fans are on.

– On On The power is on; the fan isfaulty. You can remove the fan.

cPCI slots – – On The card is plugged in.

On – – The LED state after running thecfgadm command.

On On – On fault, the hardware card isset.

Sun Services



FRUGreen(Normal)

Amber(Fault)

Amber(Hot-plug)

State

cPCI card(hot-pluglight)

N/A(does notexist)

N/A(doesnotexist)

N/A(doesnotexist)

The blue light on the card—thislight is on when the card is firstplugged in or when it can bepulled.

Sun Services


Testing the Platform

• OpenBoot PROM commands

• System controller testboard command

• POST

Sun Services


OpenBoot PROM Commands

ok show-post-resultsBoard 0, Type: CPU/Memory port 2 - Status = Pass, Type: CPU/Memory E-cache: 8 MB R-freq: 750 MHz A-freq: 750 MHz port 3 - Status = Pass, Type: CPU/Memory E-cache: 8 MB R-freq: 750 MHz A-freq: 750 MHzBoard 2, Type: CPU/Memory port 10 - Status = Pass, Type: CPU/Memory E-cache: 8 MB R-freq: 750 MHz A-freq: 750 MHz port 11 - Status = Pass, Type: CPU/Memory E-cache: 8 MB R-freq: 750 MHz A-freq: 750 MHzBoard 6, Type: PCI IO port 24 - Status = Pass, Type: PCI IO port 25 - Status = Pass, Type: PCI IOBoard 8, Type: PCI IO port 28 - Status = Pass, Type: PCI IO port 29 - Status = Pass, Type: PCI IOok

Sun Services



ok test-allTesting /ssm@0,0/pci@19,700000/pci@3/SUNW,[email protected] /ssm@0,0/pci@19,700000/pci@3/SUNW,hme@0,1...Hme register test --- succeeded.Internal loopback test -- succeeded.Transceiver check -- Using Onboard Transceiver - Link Up.passedHme register test --- succeeded.Doing more loopback tests -- passedok

Sun Services



ok probe-scsi-all/ssm@0,0/pci@18,700000/pci@1/SUNW,isptwo@4Target 1Unit 0 Disk SEAGATE ST39103LCSUN9.OG34aATarget 2Unit 0 Disk SEAGATE ST39103LCSUN9.OG34aATarget 3Unit 0 Disk SEAGATE ST39103LCSUN9.OG34aATarget 4Unit 0 Disk SEAGATE ST39103LCSUN9.OG34aAok

Sun Services



ok watch-net-all/ssm@0,0/pci@18,700000/SUNW,hme@2,1Hme register test--succeeded.Internal loopback test--succeeded.Transceiver check--Using Onboard Transceiver- Link Up.passedDoing more loopback tests--passedUsing Onboard Transceiver-Link Up.Looking for AEthernet Packets..is a Good Packet. X is a Bad Packet.Type any key to stop..............................................................................................[CR]

Sun Services


POST on the Sun Fire V1280/E2900 Server

• POST executes to check operational capability ofhardware

• Two types:

• SCPOST – System Controller POST

• SPOST – System POST

• LPOST – Local POST

• IOPOST – I/O POST

Sun Services


Controlling System Controller POSTBehavior

lom> setupscSystem Controller Configuration-------------------------------SC POST diag Level [off]: max

lom> setupscSystem Controller Configuration-------------------------------SC POST diag Level [max]:Host Watchdog [enabled]: enabledRocker Switch [disabled]: disabledSecure Mode [off]: on

Sun Services


Controlling OpenBoot PROM POSTBehavior

You can change the diag-level to max with the followingcommand:

{ok} setenv diag-level max

Sun Services


Controlling OpenBoot PROM POSTBehavior

The following OpenBoot PROM parameters should be set tothe following default values:

• error-reset-recovery=sync

• auto-boot?=true

• reboot-on-error=true

• diag-switch?=true

• use-nvramrc?=false

Sun Services


POST on the Sun Fire 3800, 4800/E4900,4810, and 6800/E6900 Servers

• POST executes to check operational capability ofhardware

• Two types:

• SCPOST – System controller POST

• SPOST – System POST

• LPOST – Local POST

• IOPOST – I/O POST

Sun Services



Example of an error output during LPOST:

r24-13a:A> setkeyswitch on..Oct 05 05:22:38 r24-13a Chassis-Port.SC: Chassis.pass2ICT: Slot 2Dx 3 Stuck attesting failed

Sun Services



r24-13a:A> showlogs.error] Interconnect test: Board 5 address repeaterconnection to RP0 failedOct 17 14:37:47 r24-13a Domain-A.SC: [ID 788592 local0.error] Bitin error: L2_ADDR[29]

Oct 17 14:37:47 r24-13a Domain-A.SC: [ID 668033 local0.error] Bitin error: L2_ADDR[28]




Sun Services


POST on the Sun Fire 3800/4800/E4900/4810/6800/E6900 Servers



Oct 17 14:37:47 r24-13a Domain-A.SC: [ID 498274 local0.error]Interconnect test: Board 5 address repeater connection toRP1 failed

Sun Services


DIMM Memory Errors

Example of a DIMM memory error occurring during LPOST:

/N0/SB0/P0/ Subtest: Memory Addressing/N0/SB0/P2/ Subtest: Memory Addressing/N0/SB0/P3/ Subtest: Memory Addressing/N0/SB0/P1/ Subtest: Memory AddressingSB0/p2/ ERROR: TEST=MemoryTests,SUBTEST=Memory Addressing ID=61.1/N0/SB0/P2/ Component under test: Board 0 Memory/N0/SB0/P0/ Data Access Error from address 00000004.00020040.AFSR = 00100004.000000c6/N0/SB0/P2/ tl tt tstate tpc tnpc/N0/SB0/P2/ 01 32 00000044.80001607 000007ff.f0042da4000007ff.f0042da8/N0/SB0/P3/ Subtest: Memory MATS+/N0/SB0/P0/ Subtest: Memory MATS+/N0/SB0/P1/ Subtest: Memory MATS+/N0/SB0/P2/ (PRIV) Privileged code access error(s)/N0/SB0/P2/ (UE) Uncorrectable system data ECC error/N0/SB0/P2/ N0/SB0/P2/B0 on J15300 J15400 J15500 J15600

Sun Services


DIMM Memory Errors

/N0/SB0/P2/ Failed Memory Addressing/N0/SB0/P0/ Subtest: Memory MATS+/N0/SB0/P3/ Subtest: Memory MARCH C-/N0/SB0/P1/ Subtest: Memory MARCH C-/N0/SB0/P0/ Subtest: Memory MARCH C-/N0/SB0/P2/ ERROR: TEST=Memory Tests,SUBTEST=Memory MATS+ ID=61.2/N0/SB0/P2/ Component under test: Board 0 Memory/N0/SB0/P2/ Data Access Error from address 00000005.00002000.AFSR = 00100004.000001f3/N0/SB0/P2/ tl tt tstate tpc tnpc/N0/SB0/P2/ 01 32 00000099.80001607 0000073f.8f0004e80000073f.8f0004ec/N0/SB0/P2/ (PRIV) Privileged code access error(s)/N0/SB0/P2/ (UE) Uncorrectable system data ECC error/N0/SB0/P2/ N0/SB0/P2/B0/D0 DIMM bit DQ139 on J15300 DRAM 3/N0/SB0/P2/ Failed Memory MATS+

Sun Services


DIMM Memory Errors

/N0/SB0/P0/ Passed/N0/SB0/P1/ Passed/N0/SB0/P2/ FailedMemory DIMM J15300 failedMemory DIMM J15400 failedMemory DIMM J15500 failedMemory DIMM J15600 failed

Sun Services


DIMM Memory Errors

This example shows show-post-results after the error wasreceived.

ok show-post-resultsBoard 0, Type: CPU/Memoryport 0 - Status = Pass, Type: CPU/MemoryE-cache: 4 MB R-freq: 750 MHz A-freq: 750 MHzport 1 - Status = Pass, Type: CPU/MemoryE-cache: 4 MB R-freq: 750 MHz A-freq: 750 MHzport 2 - Status = Fail, Type: CPU/Memoryagent status = Passspd_dcds[2] = Unknownspd_dcds[3] = Unknownspd_dcds[4] = Unknownspd_dcds[5] = Unknownspd_dcds[6] = Unknownspd_dcds[7] = Unknownport 3 - Status = Pass, Type: CPU/Memory

Sun Services


DIMM Memory Errors

This example shows showcomponent results after the errorwas received.

r24-13a:SC> showcomponent -v SB0Slot Device Status Pending POST Description---- ------ ------ ------- ---- -----------------------/NO/SB0/P0 enabled - pass UltraSparc III, 750MHz, 4M ECache/NO/SB0/P1 enabled - pass UltraSparc III, 750MHz, 4M ECache/NO/SB0/P2 enabled - fail UltraSparc III, 750MHz, 4M ECache/NO/SB0/P3 enabled - pass UltraSparc III, 750MHz, 4M ECache/NO/SB0/P0/B0/L0 enabled - pass 256M DRAM/NO/SB0/P0/B0/L1 enabled - pass 256M DRAM/NO/SB0/P0/B1/L0 enabled - untest empty/NO/SB0/P0/B1/L1 enabled - untest empty/NO/SB0/P1/B0/L0 enabled - pass 256M DRAM/NO/SB0/P1/B0/L1 enabled - pass 256M DRAM/NO/SB0/P1/B1/L0 enabled - untest empty/NO/SB0/P1/B1/L1 enabled - untest empty

Sun Services


DIMM Memory Errors

/NO/SB0/P2/B0/L0 enabled - fail empty/NO/SB0/P2/B0/L1 enabled - fail empty/NO/SB0/P2/B1/L0 enabled - untest empty/NO/SB0/P2/B1/L1 enabled - untest empty/NO/SB0/P3/B0/L0 enabled - pass 256M DRAM/NO/SB0/P3/B0/L1 enabled - pass 256M DRAM/NO/SB0/P3/B1/L0 enabled - untest empty/NO/SB0/P3/B1/L1 enabled - untest emptyr24-13a:SC>

Sun Services


Controlling System Controller POSTBehavior

You control the level of SCPOST by changing the SCPOSTdiag level setting.

schostname :SC> setupplatform -p scSC POST diag Level [min]: max

Sun Services


System Controller testboard Command

To test CPU/memory board, sb0 , in the Domain A shell,enter:

schostname :A> testboard sb0

Sun Services


Sun Fire Interconnect Link ErrorsSun Fireplane switch boards

System boards

Centerplane

I/O boards

Possible pin damage

L2 ASICreporting errors



System boards

Centerplane

I/O boards

Possible pin damage



Sun Services


Identifying Suspect FRUs From ErrorMessages

Board System Port # Connection

SB Sun Fire 48x0 and 3800 servers 0–3 Internal to board

4 RP0

5 RP2

IB 0–5 Internal to board

6/7 RP0

8/9 RP2

SB Sun Fire 6800 server 0–3 Internal to board

4 RP0 (DX0/DX1)

RP1 (DX2/DX3)

5 RP2 (DX0/DX1)

RP3 (DX2/DX3)

IB 0–5 Internal to board

6/7 RP0

8/9 RP2

Sun Services




RP Sun Fire 48x0 and 3800 servers (Sun Fire 3800repeaters are integrated as part of thecenterplane)

0/1 SB0

2/3 SB2

4/5 SB4

6/7 IB6

8/9 IB8

Sun Services




RP Sun Fire 6800 server 0 SB0

1 SB1

2 SB2

3 SB3

4 SB4

5 SB5

6 SB6

7 SB7

8 SB8

9 SB9

Sun Services


Data Parity Coverage From CPU to CPUThrough Memory

Sun Fireplane switch boards

System board x

Centerplane

System board y

ParityData

L2DX(Parity detected)



System board z


DCDS(Parity detected)



CPU(Parity

generated)Memory CPU

Sun Services


Parity Detection in the Address Network

The following example shows an address repeater (AR) SunFireplane port parity address error:

Oct 30 10:42:33 r24-10a Domain-A.SC:/partition0/domain0/SB5/ar0:>>> SafariPortError6[0x260]: 0x00018001AdrPErr [00:00] : 0x1 Address parity errorFE [15:15] : 0x1

Sun Services


Parity Protection for Address Interconnects


System board x

Centerplane ParityData

L2AR(Parity detected)


System board z


CPU(Parity

generated)

CPU(Parity

corrected)

Sun Services


Error Correcting Code Errors

Data Path ECC Points

Sun Fireplane Switch Boards

System Boards

Centerplane

I/O Boards

L2DX

L1DX

DCDS PCI IOCCPU

DRAMEcache

L1DXES ES

EDED

ED ES

ED

DataParityECC checked by SCECC checked by domain

Sun Services


Error Correcting Code Errors

ECC Generation and Correction Path


System board x

Centerplane

System board y

L2DX

L1DX(ECC detected)

L1DX

System board z

L1DX(ECC detected)

DCDS DCDS DCDS

CPU(ECC generated)

Memory(ECC stored

with data)

CPU(ECC corrected)

Ecache Ecache

Sun Services


ECC Error Types

The Sun Fire midrange servers can detect two types of ECCerrors:

• Correctable errors

• Uncorrectable errors

Sun Services


ECC Error Persistence

There are three types of ECC error persistence:

• Intermittent

• Persistent

• Sticky

Sun Services


Console Port Errors

Console port error messages can be reported by the followingASICS:

• CBH – Console bus hub

• SBBC – System boot bus controller

• AR – Address repeater

• SDC – System data controller

Sun Services


Environmental Errors

Usually caused by faulty (or blocked) fan trays or powersupplies.

Sun-Fire-sc0:SC> showlogsDec 12 08:31:00 Sun-Fire-sc0 Chassis-Port.SC: DomainA has a SYSTEM ERRORDec 12 08:31:07 Sun-Fire-sc0 Chassis-Port.SC: Thisdomain is still running because error pause is not enabled forthis domainDec 12 08:31:18 Sun-Fire-sc0 Chassis-Port.SC: Devicetemperature problem: /N0/SB5 auto power off may occur due todevice: Cheetah 3 Temp. 0 Value: 127 Degrees CDec 12 08:31:19 Sun-Fire-sc0 Chassis-Port.SC: Devicetemperature problem: Shutting down /N0/SB5 due to temperature ofdevice: Cheetah 3 Temp. 0 Value: 127 Degrees CDec 12 08:31:19 Sun-Fire-sc0 Chassis-Port.SC: /N0/SB5,sensor status, over limit (7,1,0x201050603030000)Dec 12 08:32:08 Sun-Fire-sc0 Chassis-Port.SC: ...boardsuccessfully powered off.

Sun Services


Enhanced Availability FeaturesImplemented in Firmware Update 5.15.3

• ScApp firmware update 5.15.3 is also known as the SunFire Enterprise Server Product (ESP) Availability Phase1.x.

• It focuses primarily on enhancing availability andserviceability.

• Enhancements are accomplished by the addition ofnew features.

Sun Services


Enhanced Availability FeaturesImplemented in Firmware Update 5.15.3

Functionality updates affect:

• How the system detects, handles, and records faultconditions

• How resources are reconfigured and mapped when anerror is detected

• How the Solaris OS interfaces with the firmware inhandling correctable and uncorrectable errors relatingto the CPU, memory, and L2 SRAM

• How a domain hang is handled

Sun Services


Diagnosis Engines

The following automatic diagnosis engines (DEs) identify anddiagnose hardware errors that affect the availability of thesystem and its domains:

• SMS DE

• Solaris OS DE

• POST DE

Sun Services


Auto-Diagnosis (AD) Engine

The Auto-Diagnosis (AD) engine analyzes event informationand updates the Component Health Status (CHS):

• Records CHS information into the appropriate segmentof field-replaceable unit (FRU) SEEPROM

• Presents the error messages in a Solaris OSFMA-compliant format

• Provides tunable parameters for the AD engine andCHS recording

Sun Services


Auto-Diagnosis (AD) Engine

Automatic Diagnosis Process for Non-Fatal Hardware Errors

Domain is running.

Domain is running.

Hardware error detection

Automatic diagnosis and resourcedeconfiguration by the

Solaris operating environment

Error and fault event reporting

Component health status updates

Deconfiguration of appropriateresources (next domain reboot)

Sun Services


Fault Event and Error Reporting

The DEs report diagnosis information through the followingchannels:

• Event messages displayed in the domain and platformlog files

• Email notification of fault and error events

• Fault event notification through Sun™ ManagementCenter software or SunSM Remote Services NetConnect service

Sun Services


AD Engine Logs and Records

The results of each AD engine session update CHS and aresent to the appropriate users, as follows:

• Platform administrator:

• Platform logfile

• Loghost

• Error buffers

• Domain administrator:

• Console logfile

• Domain logfile

Sun Services


Decoding AD Engine Diagnosis Messages

The following is an example of an AD engine log message:

[ AD] Event: SF3800.ASIC.SDC.PAR_SGL_ERR. 60121010CSN: 124H58EE DomainID: A ADInfo : 1.SCAPP.15.0Time: Thu Jan 23 20:47:11 PST 2003FRU-List-Count : 1; FRU-PN: 5014362; FRU-SN: 01160; FRU-LOC:/N0/SB0Recommended-Action : Service-action required

Sun Services


Viewing AD Engine Diagnosis Messages

The three possible levels of FRU diagnosis are as follows:

1. Single FRU – Identified as faulty

[AD] Event: SF3800.ASIC.SDC.PAR_SGL_ERR.60121010CSN: 124H58EE; DOMAIN-ID: AADInfo: 1.SCAPP.15.0TIME: Thu Jan 23 20:47:11 PST 2003FRU-List-Count: 1; FRU-PN: 5014362 ; FRU-SN: 01160 ; FRU-LOC:

/N0/SB0Recommended-Action: Service action required

Sun Services



2. Multiple FRUs – List of possible or suspect FRUs

[AD] Event: SF3800.ASIC.SDC.PAR_SGL_ERR.60113022CSN: 124H58EE; DOMAIN-ID: AADInfo: 1.SCAPP.15.0TIME: Thu Jan 23 21:07:51 PST 2003FRU-List-Count: 2; FRU-PN: 5015876 ; FRU-SN: 00429 ; FRU-LOC:

RP0FRU-PN: 5014362 ; FRU-SN: 011570 ; FRU-LOC: /N0/SB2

Recommended-Action: Service action required

Sun Services



3. Unresolved or undiagnosable – Indicating noautomated diagnosis is available at this time

[AD] Event: SF3800CSN: 124H58EE; DOMAIN-ID: AADInfo: 1.SCAPP.15.0TIME: Thu Jan 23 21:47:28 PST 2003FRU-List-Count: 0; FRU-PN: ; FRU-SN: ; FRU-LOC: UNRESOLVED

Sun Services


Automatic Restoration of Stopped Domains

POST reviews CHS information to determine whichcomponents to de-configure from the system.

The next time POST runs it reads the CHS database on the SC:

• Treats any faulty or suspect component (or components)as a blacklisted FRU (or FRUs).

• POST is not executed on the FRU (or FRUs).

Sun Services


Automatic Restoration of Stopped Domains

This is an example of the POST message when identifying acomponent that has its CHS set to faulty:

SEEPROM probe took 0 seconds.

Reading Component Health Status (CHS) information ...

CHS reports Port SB1/P0 status NOT_GOOD. Treating asblacklisted.

CPU_Brds: Proc Mem P/B: 3/1 3/0 2/1 2/0 1/1 1/0 0/1 0/0

Slot Gen 3210 /L: 10 10 10 10 10 10 10 10 CDC

SB01: ? ??cb ?? ?? ?? ?? cc cc cc cc ?

Sun Services


Identifying Disabled Components

schostname : SC> showcomponentComponent Status Pending POST Description--------- ------ ------- ---- -----------/N0/SB0/P0 disabled - chs UltraSPARC-III+, 900MHz .../N0/SB0/P1 disabled - chs UltraSPARC-III+, 900MHz .../N0/SB0/P2 disabled - chs UltraSPARC-III+, 900MHz .../N0/SB0/P3 disabled - chs UltraSPARC-III+, 900MHz .../N0/SB0/P0/B0/L0 disabled - chs empty/N0/SB0/P0/B0/L2 disabled - chs empty/N0/SB0/P0/B1/L1 disabled - chs 512M DRAM/N0/SB0/P0/B1/L3 disabled - chs 512M DRAM.../N0/SB4/P0 enabled - pass UltraSPARC-III+, 900MHz .../N0/SB4/P1 enabled - pass UltraSPARC-III+, 900MHz .../N0/SB4/P2 enabled - pass UltraSPARC-III+, 900MHz .../N0/SB4/P3 enabled - pass UltraSPARC-III+, 900MHz ...

Sun Services


Sun Fire Midrange Server Blacklisting

• Blacklisting gives administrative and service personnelthe ability to disable components.

• Blacklisting can be performed by using systemcontroller software or Sun Management Center 3.0software.

• Blacklisted parts are not presented to the domain.

• Blacklisted data is stored in nonvolatile configurationinformation (NVCI) and the failed part data is stored inserial electrically erasable programmable read-onlymemory (SEEPROM) on the system controller.

Sun Services


Sun Fire Midrange Server Blacklisting

• System controller software

• Sun Management Center 3.x software

Sun Services


Blacklisting Components

Component Component Name

System boards SB0, SB1, SB2, SB3, SB4, and SB5

Ports on the system board or I/Oassembly

P0, P1, P2, and P3

Memory banks on system boards B0 and B1

I/O assemblies IB6, IB7, IB8, and IB9

Ports on the I/O assembly P0 (C0, C1, C2, and C3)P1 (C4, C5, C6, and C7)

I/O cards in the I/O assemblies C0, C1, C2, C3, C4, C5, C6, and C7

Sun Services


Managing the Blacklist on Sun FireV1280/E2900 Server

There are three system controller commands for blacklisting:

• disablecomponent

• enablecomponent

• showcomponent

Sun Services


The setls Command

To enable a board that has been disabled by the setlscommand:

schostname :A> setls -s enable -l sb0

Sun Services


Domain Shell Operating Messages

When an active domain encounters a system error, thefollowing occurs:

• The domain is paused

• The domain is rebooted

• Depends on the setting of the reboot-on-error flag,which is set by the setupdomain command

• True

• False

Sun Services


Domain Shell Operating Messages

Following is an example domain system error message:

Thu Oct 05 05:22:49 PDT 2000:Domain A has a SYSTEM ERROROct 05 05:22:50 r24-13a Chassis-Port.SC:/partition0/RP3/dx1:>>> Safari Port Error Status 2[0x21] : 0x2000a000AccSafPar [29:29] : 0x1FirstError [15:15] : 0x1SafPar [13:13] : 0x1 Safari Input bus parity error This domain iscurrently paused and must be turned off via "setkeyswitch off" torecover

Sun Services


Recovering From a Hung Domain

• Run the system controller reset command from thedomain shell.

• The reset command terminates the Solaris OS in thedomain.

• It sends an externally initiated reset (xir ) to thedomain and collects data for debugging thehardware.

• The domain returns to the OpenBoot PROM okprompt.

Sun Services


Verifying the Recovery

• Force the domain to OpenBoot PROM by using thesystem controller break command from the domainshell.

• If the break command succeeds, run the OpenBootPROM sync command.

The sync command terminates the Solaris OS.

Sun Services


Collecting Data

Apart from a Solaris OS core file, various other bits ofinformation might provide insight into the failure. Followthese steps to collect this data:

1. Ensure you that have a record of what you didduring the previous verification step.

2. If you do not have a hardware error, assess currentconditions on the domain:

• Is there any output being printed on the domainconsole?

• Does the domain console echo characters you type?

• Does the domain respond to the ping command?

• Does the domain respond to the rup command?

Sun Services


Collecting Data

3. Record the results of all the preceding tests.

4. Run the following commands from the systemcontroller domain shell, and collect the output in afile:

• showlogs

• showenvironment

• showdomain

Sun Services


Collecting Data

5. Run the following commands from the systemcontroller platform shell, and collect the output in afile:

• showsc

• showlogs

• showplatform

• history

Sun Services


Collecting Data

6. Run the Sun Explorer software utility to collect thesystem configuration information.

If the domain is paused because of an error, nothing else canbe done after collecting all the previous information. Rebootthe domain with the following command in the domain shell:

setkey off ; setkey on

Sun Services


Obtaining a Solaris OS Core File

# Mar 29 14:47:06 sp4-sc0 Domain-A.SC: Domain is not respondingto interrupts.

telnet> send brksp4-sc0:A> reset

This will abruptly terminate Solaris in domain A.Do you want to continue? [no] yNOTICE: XIR on CPU 3

panic[cpu3]/thread=30002368580: zero

fail: trap type = 0x3pid=320, pc=0x10314080, sp=0x2a100336f81, tstate=0x4400001407,context=0x887g1-g7: 14, 104f0228, 30000827800, 6c0, 0, 0, 30002368580

000000001040c110 unix:sync_handler+160 (1040c290, 7fcb4090, 1, 0,0, 6c005f6900)

Sun Services



%l0-3: 00000000ff000000 0000000000ff0000 000000000000ff00000000007f6b6e62%l4-7: 000000006c6f6300 0000000000000000 000000007efefeff0000000081010100000000001040c1e0 unix:prom_rtt+0 (2, 0, 0, 4, 705aac, 100000530)%l0-3: 0000000000000000 0000000000001400 00000044000014070000000010027d14%l4-7: ffffffff7f100134 0000000100000531 000000000000000f000000001040c290000000001040c330 faildev:faildev_ioctl+78 (be00000000, 2, 0,202003, 300012e5aa8, 2a100337aec)%l0-3: 0000000000000002 ffffffff7f71e3c8 ffffffff7f721480ffffffff7ffffad8%l4-7: 0000000000705aac ffffffff7f200128 ffffffff7f720838ffffffff7f30af80000002a100337830 genunix:cdev_ioctl+40 (7073722e736f2e31, 4,310003000211ed28, 0, 2a10033784c, 2a10033785c)%l0-3: 0000000000000001 ffffffff7f354000 00000300023511480000000000000003

Sun Services



%l4-7: 0000000000000000 0000000000000000 000002a1003370f1000000001002d200

syncing file systems... 1 1 donedumping to /dev/dsk/c0t0d0s1, offset 62259200100% done: 17422 pages dumped, compression ratio 8.03, dumpsucceededrebooting...Resetting ...

Sun Services


Obtaining Registers

If you cannot get a core file out of the Solaris OS, there shouldstill be CPU register information to collect.

The reset command causes all the CPUs in the target domainto save their registers in a save area in CPU static randomaccess memory (SRAM).

showresetstate -v

Sun Fire Midrange Server Maintenance

Documents