Skip to main content

Raymii.org Logo (IEC resistor symbol)logo

Quis custodiet ipsos custodes?
Home | About | All pages | RSS Feed | Gopher

HP-UX 11.31 System information and find out part number of a failed disk with sasmgr

Published: 18-05-2018 | Author: Remy van Elst | Text only version of this article


Table of Contents


On one of my regular scheduled datacenter visits one of the older HP-UX Itaniummachines had an orange light on the front. These systems are not (yet)monitored, but still in use so the disk had to be replaced. Not knowing anythingabout this system or which parts were used, I managed to find the exact partnumber and device type so we could order a spare. This small guide uses sasmgrto get the data on HP-UX 11.31.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get $100 credit for 60 days). (referral link)

Hardware Information on HP-UX

Since I normally don't manage these systems and have little experience with HP-UX, I first had to find out some information on the system. Using a few regularUnix tools like netstat, uname and a few HP-UX specific tools like model,machinfo I was able to gather basic hardware information of this machine.

$ TERM=vt100 ssh hpux09 -l rootLast successful login:       Fri May 18 08:18:05 METDST 2018     Last authentication failure: Fri May 18 08:10:59 METDST 2018     Last login: Thu Jun 24 09:46:55 2010 from   (c)Copyright 1983-2006 Hewlett-Packard Development Company, L.P.(c)Copyright 1979, 1980, 1983, 1985-1993 The Regents of the Univ. of California(c)Copyright 1980, 1984, 1986 Novell, Inc.(c)Copyright 1986-2000 Sun Microsystems, Inc.(c)Copyright 1985, 1986, 1988 Massachusetts Institute of Technology(c)Copyright 1989-1993  The Open Software Foundation, Inc.(c)Copyright 1990 Motorola, Inc.(c)Copyright 1990, 1991, 1992 Cornell University(c)Copyright 1989-1991 The University of Maryland(c)Copyright 1988 Carnegie Mellon University(c)Copyright 1991-2006 Mentat Inc.(c)Copyright 1996 Morning Star Technologies, Inc.(c)Copyright 1996 Progressive Systems, Inc.Confidential computer software. Valid license from HP required forpossession, use or copying.  Consistent with FAR 12.211 and 12.212,Commercial Computer Software, Computer Software Documentation, andTechnical Data for Commercial Items are licensed to the U.S. Governmentunder vendor's standard commercial license.You have mail.Value of TERM has been set to "vt100". WARNING:  YOU ARE SUPERUSER !!hpux09:/root# swlist | grep -i oe  HPUX11i-BOE                           B.11.31.0809   HP-UX Base Operating Environment hpux09:/root# modelia64 hp server rx3600hpux09:/root# uname -aHP-UX hpux09 B.11.31 U ia64 [...] unlimited-user licensehpux09:/root# machinfoCPU info:  2 Intel(R) Itanium 2 9100 series processors (1.67 GHz, 18 MB)          666 MT/s bus, CPU version A1          4 logical processors (2 per socket)Firmware info:   Firmware revision:  04.03   FP SWA driver revision: 1.18   IPMI is supported on this system.   BMC firmware revision: 5.25Platform info:   Model:                  "ia64 hp server rx3600"   Machine ID number:      2e6e7[...]f36   Machine serial number:  D[...]DOS info:   Nodename:  hpux09   Release:   HP-UX B.11.31   Version:   U (unlimited-user license)   Machine:   ia64   ID Number: [...]   vmunix _release_version:               @(#) $Revision: vmunix:    B.11.31_LR FLAVOR=perf 

This system was installed a long while ago:

hpux09:/root# swlist -l bundle -a date | grep HPUX  HPUX-FTPServer        Wed Dec 10 16:58:16 MET 2008   HPUX-MailServer       Wed Dec 10 16:58:16 MET 2008 

print_manifest is a usefull command for information on the system.

hpux09:/root# print_manifest | grep -i created    The system was created December 10, 2008, 18:02:06 MET.    It was created with Ignite-UX revision C.7.7.93.

This will go into the hardware documentation, since, sadly, that is veryincomplete.

Disk information

Now to the good part, for which we logged in to this machine. Find the serialnumber and order a new (probably refurbished) disk. The system uses LVM withRAID 1 underneath via a SAS card so when the disk is ordered we can just replaceit. If it was just LVM we had to do some configuration

This PDF When Good Disks Go Bad.pdf is very helpfull in preparing andresearching this issue. When the new disk is here, this page will behelpfull for the replacement.

The first command shows which disk is the failed one plus the raidconfiguration:

sasmgr get_info -D /dev/sasd0 -q raid

Output:

Wed May 16 14:54:21 2018---------- LOGICAL DRIVE 2 ----------Raid Level                                         : RAID 1              Volume sas address                                 : 0x3c5c0600358833e   Device Special File                                : /dev/rdsk/c0t3d0    Raid State                                         : DEGRADED            Raid Status Flag                                   : ENABLED             Raid Size                                          : 139136              Rebuild Rate                                       : 20.00 %             Rebuild Progress                                   : 100.00 %            Participating Physical Drive(s) :SAS Address         Enc   Bay   Size(MB)      Type            State           0x500000e01d33a9c2   1     8    140014        PRIMARY         ONLINE         0x5000c50003aeb459   1     2    140014        SECONDARY       FAILED         ---------- LOGICAL DRIVE 7 ----------Raid Level                                         : RAID 1              Volume sas address                                 : 0xacb8d6daa52eb4d   Device Special File                                : /dev/rdsk/c0t2d0    Raid State                                         : OPTIMAL             Raid Status Flag                                   : ENABLED             Raid Size                                          : 69878               Rebuild Rate                                       : 0.00 %              Rebuild Progress                                   : 100.00 %            Participating Physical Drive(s) :SAS Address         Enc   Bay   Size(MB)      Type            State           0x5000039668124686   1     1    286102        SECONDARY       ONLINE         0x5000c5000b9f5965   1     7    70007         PRIMARY         ONLINE         

The second command shows the location (controller-wise) and the part number ofthe disk:

sasmgr get_info -D /dev/sasd0 -q lun=all -q lun_locate 

Output:

RAID VOL ID is 7 : /dev/rdsk/c0t2d0          0/4/1/0.0.0.2.0          Physical disks in volume are :        1     1     OFF           HP            EG0300FCSPH        HPD0        1     7     OFF           HP            DG072BB975         HPDCRAID VOL ID is 2 : /dev/rdsk/c0t3d0          0/4/1/0.0.0.3.0          Physical disks in volume are :        1     8     OFF           HP            DG146BABCF         HPD5        1     2     OFF           HP            DG146ABAB4         HPD5

In my case disk 1,2 failed:

    0x5000c50003aeb459   1     2    140014        SECONDARY       FAILED             [...]        1     2     OFF           HP            DG146ABAB4         HPD5

With this part number we can order a new disk.

dmesg was also spitting out warning about this disk issue:

SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400LVM: VG 64 0x010000: Lost quorum.This may block configuration changes and I/Os. In order to reestablish quorum at least 1 of the following PVs (represented by current link) must become available:<3 0x000006> LVM: VG 64 0x010000: PVLink 3 0x000006 Failed! The PV is not accessible.SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x0acb8d6daa52eb4d (dev_t 0x03000004) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x03c5c0600358833e (dev_t 0x03000006) failed with CDB status 0x00000400class : lunpath, instance 3lun path (class = lunpath, instance = 3) belonging to LUN (default minor = 0x6) has gone offline.  The lunpath hwpath is 0/4/1/0.0x3c5c0600358833e.0x0SAS driver at 0/4/1/0 (/dev/sasd0) : RAID volume at SAS address 0x03c5c0600358833e is in a degraded state.SAS driver at 0/4/1/0 (/dev/sasd0) : RAID volume at SAS address 0x03c5c0600358833e is in a degraded state.SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x0acb8d6daa52eb4d (dev_t 0x03000004) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x0acb8d6daa52eb4d (dev_t 0x03000004) failed with CDB status 0x00000400SAS driver at 0/4/1/0 (/dev/sasd0) : TM (0x00000002) to SAS address 0x0acb8d6daa52eb4d (dev_t 0x03000004) failed with CDB status 0x00000400LVM: VG 64 0x010000: Reestablished quorum.LVM: VG 64 0x010000: PVLink 3 0x000006 Recovered.LVM: NOTICE: VG 64 0x010000: LV 1: All I/O requests to this LV that were        waiting indefinitely for an unavailable PV have now completed.
Tags: hp, hp-ux, itanium, lvm, raid, sas, sasmgr, tutorials, unix