Sun(TM) Integrated Lights Out Manager Version 2.0.2.16 Copyright 2007 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. -> [steve.scherr@oracle.com] start /SP/console Are you sure you want to start /SP/console (y/n)? y Serial console started. To stop, type ESC ( mccfs4-bui:> status show Storage:  mccsp: Used 991G bytes Avail 2.66T bytes State online Compression: 1x Dedup: 1x Memory:  Cache 7.36G bytes Unused 1.81G bytes Mgmt 286M bytes Other 295M bytes Kernel 6.26G bytes Services:  ad disabled cifs disabled dns online dynrouting online ftp disabled http online identity online idmap online ipmp online iscsi disabled ldap disabled ndmp disabled nfs online nis disabled ntp online replication disabled scrk disabled sftp disabled shadow disabled smtp online snmp disabled srp disabled ssh online syslog disabled tags disabled tftp disabled vscan disabled [4Hardware:  CPU online Cards online Disks online Fans online Memory online PSU online Activity:  CPU 4 %util Sunny FTP 0 bytes/sec Sunny Disk 634 ops/sec Partly Cloudy iSCSI 0 ops/sec Sunny NFSv3 312 ops/sec Sunny NFSv4 0 ops/sec Sunny Network 9.05M bytes/sec Sunny SMB 0 ops/sec Sunny Recent Alerts:  2013-01-29 16:51: The appliance failed to upload support bundle '/cores/3... 2013-01-29 17:34: A cluster interconnect link has been restored. 2013-01-29 17:34: A cluster interconnect link has been restored. 2013-01-29 17:34: A cluster interconnect link has been restored. 2013-01-29 17:35: The appliance has rejoined the cluster. mccfs4-bui:> [confirm shell Executing raw shell; "exit" to return to appliance shell ... +-----------------------------------------------------------------------------+ | You are entering the operating system shell. By confirming this action in | | the appliance shell you have agreed that THIS ACTION MAY VOID ANY SUPPORT | | AGREEMENT. If you do not agree to this -- or do not otherwise understand | | what you are doing -- you should type "exit" at the shell prompt. EVERY | | COMMAND THAT YOU EXECUTE HERE IS AUDITED, and support personnel may use | | this audit trail to substantiate invalidating your support contract. The | | operating system shell is NOT a supported mechanism for managing this | | appliance, and COMMANDS EXECUTED HERE MAY DO IRREPARABLE HARM. | | | | NOTHING SHOULD BE ATTEMPTED HERE BY UNTRAINED SUPPORT PERSONNEL UNDER ANY | | CIRCUMSTANCES. This appliance is a non-traditional operating system | | environment, and expertise in a traditional operating system environment | | in NO WAY constitutes training for supporting this appliance. THOSE WITH | | EXPERTISE IN OTHER SYSTEMS -- HOWEVER SUPERFICIALLY SIMILAR -- ARE MORE | | LIKELY TO MISTAKENLY EXECUTE OPERATIONS HERE THAT WILL DO IRREPARABLE | | HARM. Unless you have been explicitly trained on supporting this | | appliance via the operating system shell, you should immediately return | | to the appliance shell. | | | | Type "exit" now to return to the appliance shell. | +-----------------------------------------------------------------------------+ mccfs4-bui# zpool status pool: mccsp state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: resilvered 3.98G in 0h2m with 0 errors on Thu Jan 24 19:13:10 2013 config: NAME STATE READ WRITE CKSUM mccsp ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t5000C5001A108945d0 ONLINE 0 0 0 c3t5000C50015A3038Dd0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c3t5000C50015BAD93Dd0 ONLINE 0 0 0 c3t5000C50019D05DD1d0 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 c3t5000C50019D08DADd0 ONLINE 0 0 0 c3t5000C50019D09E3Ad0 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 c3t5000C50019D09F37d0 ONLINE 0 0 0 c3t5000C50019D061B8d0 ONLINE 0 0 0 logs c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 ONLINE 0 0 0 cache c0t0d0 ONLINE 0 0 0 c0t1d0 ONLINE 0 0 0 spares c3t5000C50019D0727Fd0 AVAIL c3t5000C500150DC4EFd0 AVAIL errors: No known data errors pool: system state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: none requested config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t1d0s0 ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 errors: No known data errors mccfs4-bui# exit logout mccfs4-bui:> maintenance hardware mccfs4-bui:maintenance hardware> list NAME STATE MANUFACTURER MODEL SERIAL RPM chassis-000 mccfs4-bui ok Sun Microsystems, Inc. Sun Storage 7310 1035QAP001 -- chassis-001 1032QAK00F ok Sun Microsystems, Inc. J4400 1032QAK00F -- mccfs4-bui:maintenance hardware> show Chassis:  NAME STATE MANUFACTURER MODEL SERIAL RPM chassis-000 mccfs4-bui ok Sun Microsystems, Inc. Sun Storage 7310 1035QAP001 -- cpu-000 CPU 0 ok AMD Six-Core AMD Opteron(tm) Processor 2427 unknown cpu-001 CPU 1 absent - - - disk-000 HDD 0 ok STEC MACH8 STM0000D0D28 -- disk-001 HDD 1 ok STEC MACH8 STM0000DB12C -- disk-002 HDD 2 absent - - - -- disk-003 HDD 3 absent - - - -- disk-004 HDD 4 absent - - - -- disk-005 HDD 5 absent - - - -- disk-006 HDD 6 ok HITACHI HTE5450SASUN500G BK0F20VEGE1GJB -- disk-007 HDD 7 ok HITACHI HTE5450SASUN500G BK0F20VEGEH4YB -- fan-000 FB 0 ok unknown ASY,FAN,BOARD,H1M1G1 0226LHF-0920A402CX fan-001 FB 0 FM 0 ok Sun Microsystems, Inc. 541-2940 unknown fan-002 FB 0 FM 1 ok Sun Microsystems, Inc. 541-2940 unknown fan-003 FB 0 FM 2 ok Sun Microsystems, Inc. 541-2940 unknown fan-004 FB 0 FM 3 ok Sun Microsystems, Inc. 541-2940 unknown fan-005 FB 1 ok unknown ASY,FAN,BOARD,H1M1G1 0226LHF-0920A402D4 fan-006 FB 1 FM 0 ok Sun Microsystems, Inc. 541-2940 unknown fan-007 FB 1 FM 1 ok Sun Microsystems, Inc. 541-2940 unknown fan-008 FB 1 FM 2 ok Sun Microsystems, Inc. 541-2940 unknown memory-000 DIMM 0/0 absent - - - memory-001 DIMM 0/1 absent - - - memory-002 DIMM 0/2 absent - - - memory-003 DIMM 0/3 absent - - - memory-004 DIMM 0/4 ok HYNIX 4096MB DDR-II 666 (PC2 5300) ADDRESS/COMMAND PARITY/ECC 5234328A memory-005 DIMM 0/5 ok HYNIX 4096MB DDR-II 666 (PC2 5300) ADDRESS/COMMAND PARITY/ECC 52542361 memory-006 DIMM 0/6 ok HYNIX 4096MB DDR-II 666 (PC2 5300) ADDRESS/COMMAND PARITY/ECC 5244328E memory-007 DIMM 0/7 ok HYNIX 4096MB DDR-II 666 (PC2 5300) ADDRESS/COMMAND PARITY/ECC 5264236A memory-008 DIMM 1/0 absent - - - memory-009 DIMM 1/1 absent - - - memory-010 DIMM 1/2 absent - - - memory-011 DIMM 1/3 absent - - - memory-012 DIMM 1/4 absent - - - memory-013 DIMM 1/5 absent - - - memory-014 DIMM 1/6 absent - - - memory-015 DIMM 1/7 absent - - - psu-000 PSU 0 ok Delta Energy Systems A221 1841DET-1012A57163 psu-001 PSU 1 ok Delta Energy Systems A221 1841DET-1014A59733 slot-000 PCIe 1 ok Sun Microsystems, Inc. Dual 4x3Gb External SAS HBA (Rev C0) P371074909 slot-001 PCIe 0 ok Sun Microsystems, Inc. Fishworks CLUSTRON 100 unknown slot-002 PCIe 2 absent - - - chassis-001 1032QAK00F ok Sun Microsystems, Inc. J4400 1032QAK00F -- disk-000 HDD 0 ok SEAGATE ST31000NSSUN1.0T 9QJ5TMK7 7200 disk-001 HDD 1 ok SEAGATE ST31000NSSUN1.0T 9QJ47BQ5 7200 disk-002 HDD 2 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVJM 7200 disk-003 HDD 3 ok SEAGATE ST31000NSSUN1.0T 9QJ4TLW1 7200 disk-004 HDD 4 ok STEC ZeusIOPS STM0000D5B58 20000 disk-005 HDD 5 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVBC 7200 disk-006 HDD 6 ok SEAGATE ST31000NSSUN1.0T 9QJ5RWAM 7200 disk-007 HDD 7 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVB4 7200 disk-008 HDD 8 ok STEC ZeusIOPS STM000107DD0 20000 disk-009 HDD 9 ok SEAGATE ST31000NSSUN1.0T 9QJ67EAW 7200 disk-010 HDD 10 ok SEAGATE ST31000NSSUN1.0T 9QJ4YWQM 7200 disk-011 HDD 11 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVZP 7200 disk-012 HDD 12 absent - - - -- disk-013 HDD 13 absent - - - -- disk-014 HDD 14 absent - - - -- disk-015 HDD 15 absent - - - -- disk-016 HDD 16 unavail unknown unknown unknown -- disk-017 HDD 17 absent - - - -- disk-018 HDD 18 absent - - - -- disk-019 HDD 19 absent - - - -- disk-020 HDD 20 absent - - - -- disk-021 HDD 21 absent - - - -- disk-022 HDD 22 absent - - - -- disk-023 HDD 23 absent - - - -- fan-000 FAN 0/0 ok Sun Microsystems, Inc. - unknown fan-001 FAN 0/1 ok Sun Microsystems, Inc. - unknown fan-002 FAN 1/0 ok Sun Microsystems, Inc. - unknown fan-003 FAN 1/1 ok Sun Microsystems, Inc. - unknown psu-000 PSU 0 ok Sun Microsystems, Inc. 300-2169-01 2029QTF-1002QC5031 psu-001 PSU 1 ok Sun Microsystems, Inc. 300-2169-01 2029QTF-1002QC5032 slot-000 SIM 0 ok Sun Microsystems, Inc. 375-3584-02 2029QTF-0951QC1205 slot-001 SIM 1 ok Sun Microsystems, Inc. 375-3584-02 2029QTF-1021QC101D mccfs4-bui:maintenance hardware> [michael.a.pruitt@oracle.com] cd / mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  [1]+ Stopped aksh You have new mail in /var/mail/root mccfs4-bui# fg aksh mccfs4-bui:> [confggigiiguration cluster show Properties:  state = AKCS_CLUSTERED description = Active peer_asn = 183e6d0e-9646-c970-a098-e043fb0d609f peer_hostname = mccfs3-bui peer_state = AKCS_CLUSTERED peer_description = Active Children:  resources => Configure resources mccfs4-bui:> configuration storage show Properties:  pool = mccsp status = online owner = mccfs4-bui profile = mirror log_profile = log_stripe cache_profile = cache_stripe scrub = resilver completed after 0h2m with 0 errors mccfs4-bui:>  [1]+ Stopped aksh mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# pwd /var/ak/stash/com/sun/ak/xmlrpc/disk mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-16 1032QAK00F-23 1035QAP001-0 1035QAP002-2 1032QAK00F-1 1032QAK00F-17 1032QAK00F-3 1035QAP001-1 1035QAP002-3 1032QAK00F-10 1032QAK00F-18 1032QAK00F-4 1035QAP001-2 1035QAP002-4 1032QAK00F-11 1032QAK00F-19 1032QAK00F-5 1035QAP001-3 1035QAP002-5 1032QAK00F-12 1032QAK00F-2 1032QAK00F-6 1035QAP001-4 1032QAK00F-13 1032QAK00F-20 1032QAK00F-7 1035QAP001-5 1032QAK00F-14 1032QAK00F-21 1032QAK00F-8 1035QAP002-0 1032QAK00F-15 1032QAK00F-22 1032QAK00F-9 1035QAP002-1 mccfs4-bui# aknv ./1032QAK00F-16/obj Stash Header for ./1032QAK00F-16/obj: aksh_id[AK_SHDR_MAGn] = \x7faks aksh_id[AK_SHDR_VERSION] = 1 aksh_id[AK_SHDR_FORMAT] = 1 aksh_id[AK_SHDR_PAD0] = 0 aksh_id[AK_SHDR_PAD1] = 0 Stash Object: nvlist version: 0 _class = com/sun/ak/xmlrpc/disk _version = 0x1 _atom = obj fru = hc://:product-id=SUN-Storage-J4400:server-id=:chassis-id=1032QAK00F/ses-enclosure=0/bay=16/disk=0 rid = 1032QAK00F-16 mccfs4-bui# zpool list& [2] 7785 mccfs4-bui# NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT mccsp 3.62T 991G 2.66T 26% 1.00x ONLINE - system 464G 130G 334G 27% 1.00x ONLINE - z  [2]- Done zpool list mccfs4-bui# mccfs4-bui# zpool status & [2] 7786 pool: mccsp state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. mccfs4-bui# scan: resilvered 3.98G in 0h2m with 0 errors on Thu Jan 24 19:13:10 2013 config: NAME STATE READ WRITE CKSUM mccsp ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t5000C5001A108945d0 ONLINE 0 0 0 c3t5000C50015A3038Dd0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c3t5000C50015BAD93Dd0 ONLINE 0 0 0 c3t5000C50019D05DD1d0 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 c3t5000C50019D08DADd0 ONLINE 0 0 0 c3t5000C50019D09E3Ad0 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 c3t5000C50019D09F37d0 ONLINE 0 0 0 c3t5000C50019D061B8d0 ONLINE 0 0 0 logs c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 ONLINE 0 0 0 cache c0t0d0 ONLINE 0 0 0 c0t1d0 ONLINE 0 0 0 spares c3t5000C50019D0727Fd0 AVAIL c3t5000C500150DC4EFd0 AVAIL errors: No known data errors pool: system state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: none requested config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t1d0s0 ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 errors: No known data errors [2]- Done zpool status mccfs4-bui# mdb -k Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs sata sd emlxs qlc ip hook neti arp usba stmf stmf_sbd sockfs lofs random md mpt crypto idm smbsrv nfs cpc ufs logindmux ptm nsmb mpt_sas pmcs ] > ::spa -v ADDR STATE NAME  fffff6000daff080 ACTIVE mccsp ADDR STATE AUX DESCRIPTION  fffff600397466c0 HEALTHY - root fffff600397459c0 HEALTHY - mirror fffff60039746080 HEALTHY - /dev/dsk/c3t5000C5001A108945d0s0 fffff60039747980 HEALTHY - /dev/dsk/c3t5000C50015A3038Dd0s0 fffff60039746d00 HEALTHY - mirror fffff60039747340 HEALTHY - /dev/dsk/c3t5000C50015BAD93Dd0s0 fffff6001cd41cc0 HEALTHY - /dev/dsk/c3t5000C50019D05DD1d0s0 fffff6001cd41040 HEALTHY - mirror fffff6001cd41680 HEALTHY - /dev/dsk/c3t5000C50019D08DADd0s0 fffff60039742c80 HEALTHY - /dev/dsk/c3t5000C50019D09E3Ad0s0 fffff60039742640 HEALTHY - mirror fffff60039742000 HEALTHY - /dev/dsk/c3t5000C50019D09F37d0s0 fffff60039739940 HEALTHY - /dev/dsk/c3t5000C50019D061B8d0s0 fffff60039739300 HEALTHY - hole fffff60039738cc0 HEALTHY - /dev/dsk/c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0s0 - - - cache fffff60039737980 HEALTHY - /dev/dsk/c0t0d0s0 fffff60039737340 HEALTHY - /dev/dsk/c0t1d0s0 >> More [, , q, n, c, a] ?   - - - spares fffff6003a0530c0 HEALTHY - /dev/dsk/c3t5000C50019D0727Fd0s0 fffff60039738680 HEALTHY - /dev/dsk/c3t5000C500150DC4EFd0s0 fffff60005581580 ACTIVE system fffff600000eb680 HEALTHY - root fffff600000eb040 HEALTHY - mirror fffff6000789e980 HEALTHY - /dev/dsk/c1t1d0s0 fffff6000789e340 HEALTHY - /dev/dsk/c1t0d0s0 > > > >  [2]+ Stopped mdb -k mccfs4-bui# fg mdb -k > fffff60039739300::print vdev_t vdev_fru vdev_fru = 0 > A > fffff60039739300::print vdev_t vdev_fru   guid| - =E 0 >  [2]+ Stopped mdb -k mccfs4-bui# fg 1 aksh mccfs4-bui:> [maintenance hardware select ch chassis-000 chassis-001 mccfs4-bui:> maintenance hardware select chassis-000 11 slee select set [1mccfs4-bui:>[0 maintenance hardware select chassis-001 select disk show Disks:  LABEL STATE MANUFACTURER MODEL SERIAL RPM disk-000 HDD 0 ok SEAGATE ST31000NSSUN1.0T 9QJ5TMK7 7200 disk-001 HDD 1 ok SEAGATE ST31000NSSUN1.0T 9QJ47BQ5 7200 disk-002 HDD 2 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVJM 7200 disk-003 HDD 3 ok SEAGATE ST31000NSSUN1.0T 9QJ4TLW1 7200 disk-004 HDD 4 ok STEC ZeusIOPS STM0000D5B58 20000 disk-005 HDD 5 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVBC 7200 disk-006 HDD 6 ok SEAGATE ST31000NSSUN1.0T 9QJ5RWAM 7200 disk-007 HDD 7 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVB4 7200 disk-008 HDD 8 ok STEC ZeusIOPS STM000107DD0 20000 disk-009 HDD 9 ok SEAGATE ST31000NSSUN1.0T 9QJ67EAW 7200 disk-010 HDD 10 ok SEAGATE ST31000NSSUN1.0T 9QJ4YWQM 7200 disk-011 HDD 11 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVZP 7200 disk-012 HDD 12 absent - - - -- disk-013 HDD 13 absent - - - -- disk-014 HDD 14 absent - - - -- disk-015 HDD 15 absent - - - -- disk-016 HDD 16 unavail unknown unknown unknown -- disk-017 HDD 17 absent - - - -- disk-018 HDD 18 absent - - - -- disk-019 HDD 19 absent - - - -- disk-020 HDD 20 absent - - - -- disk-021 HDD 21 absent - - - -- disk-022 HDD 22 absent - - - -- disk-023 HDD 23 absent - - - -- mccfs4-bui:>  [1]+ Stopped aksh mccfs4-bui# zpool sttaus      atus -v & [3] 7789 pool: mccsp state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. mccfs4-bui# action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: resilvered 3.98G in 0h2m with 0 errors on Thu Jan 24 19:13:10 2013 config: NAME STATE READ WRITE CKSUM mccsp ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t5000C5001A108945d0 ONLINE 0 0 0 c3t5000C50015A3038Dd0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c3t5000C50015BAD93Dd0 ONLINE 0 0 0 c3t5000C50019D05DD1d0 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 c3t5000C50019D08DADd0 ONLINE 0 0 0 c3t5000C50019D09E3Ad0 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 c3t5000C50019D09F37d0 ONLINE 0 0 0 c3t5000C50019D061B8d0 ONLINE 0 0 0 logs c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 ONLINE 0 0 0 cache c0t0d0 ONLINE 0 0 0 c0t1d0 ONLINE 0 0 0 spares c3t5000C50019D0727Fd0 AVAIL c3t5000C500150DC4EFd0 AVAIL errors: No known data errors pool: system state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: none requested config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t1d0s0 ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 errors: No known data errors [3] Done zpool status -v mccfs4-bui# mccfs4-bui# mccfs4-bui# svcs -l   pv akd STATE NSTATE STIME CTID FMRI online - Jan_29 51 svc:/appliance/kit/akd:default Jan_29 939 akd Jan_29 1111 akd mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# svcadm disable -t akd mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# svcs p -pv akd STATE NSTATE STIME CTID FMRI online disabled 17:09:19 51 svc:/appliance/kit/akd:default Jan_29 939 akd mccfs4-bui# mccfs4-bui# mccfs4-bui# svcs -pv akd STATE NSTATE STIME CTID FMRI online disabled 17:09:19 51 svc:/appliance/kit/akd:default Jan_29 939 akd mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# wd -bash: wd: command not found mccfs4-bui# pwd /var/ak/stash/com/sun/ak/xmlrpc/disk mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-16 1032QAK00F-23 1035QAP001-0 1035QAP002-2 1032QAK00F-1 1032QAK00F-17 1032QAK00F-3 1035QAP001-1 1035QAP002-3 1032QAK00F-10 1032QAK00F-18 1032QAK00F-4 1035QAP001-2 1035QAP002-4 1032QAK00F-11 1032QAK00F-19 1032QAK00F-5 1035QAP001-3 1035QAP002-5 1032QAK00F-12 1032QAK00F-2 1032QAK00F-6 1035QAP001-4 1032QAK00F-13 1032QAK00F-20 1032QAK00F-7 1035QAP001-5 1032QAK00F-14 1032QAK00F-21 1032QAK00F-8 1035QAP002-0 1032QAK00F-15 1032QAK00F-22 1032QAK00F-9 1035QAP002-1 mccfs4-bui# lspwdwd[10@svcs -pv akd STATE NSTATE STIME CTID FMRI online disabled 17:09:19 51 svc:/appliance/kit/akd:default Jan_29 939 akd mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# s aklog rm | egep    rep -v 'lib/|LOCK|^$' | tail mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# akklog akd        k log akd | tail Wed Feb 13 17:09:19 2013: CIO disabled: EAK_MUSTEXIT Wed Feb 13 17:09:19 2013: ALUA proxy communication with cluster peer interrupted by error: detected dropped or reset peer mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# !svcs svcs -pv akd STATE NSTATE STIME CTID FMRI disabled - 17:09:57 - svc:/appliance/kit/akd:default mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-16 1032QAK00F-23 1035QAP001-0 1035QAP002-2 1032QAK00F-1 1032QAK00F-17 1032QAK00F-3 1035QAP001-1 1035QAP002-3 1032QAK00F-10 1032QAK00F-18 1032QAK00F-4 1035QAP001-2 1035QAP002-4 1032QAK00F-11 1032QAK00F-19 1032QAK00F-5 1035QAP001-3 1035QAP002-5 1032QAK00F-12 1032QAK00F-2 1032QAK00F-6 1035QAP001-4 1032QAK00F-13 1032QAK00F-20 1032QAK00F-7 1035QAP001-5 1032QAK00F-14 1032QAK00F-21 1032QAK00F-8 1035QAP002-0 1032QAK00F-15 1032QAK00F-22 1032QAK00F-9 1035QAP002-1 mccfs4-bui# mv 1032QAK00F-16 /var/tmp mccfs4-bui# mccfs4-bui# mccfs4-bui# svcadm enable -rs     akd mccfs4-bui# svcs -pv akd STATE NSTTE STIME CTID FMRI offline online 17:11:12 262 svc:/appliance/kit/akd:default 17:11:12 7821 akd 17:11:12 7822 akd 17:11:13 7825 akd mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# ! svcs -pv akd[5@adm enable akd mccfs4-bui# mv 1032QAK00F-16 /var/tmp mccfs4-bui# lssvcs -pv akdaklog akd | tail[25@rm | egrep -v 'lib/|LOCK|^$' | tail mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# svcs sysconfigd svcs: Pattern 'sysconfigd' doesn't match any instances STATE STIME FMRI mccfs4-bui# svcs sysconfigd  STATE STIME FMRI online Jan_29 svc:/milestone/sysconfig:default mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# pwd /var/ak/stash/com/sun/ak/xmlrpc/disk mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-16 1032QAK00F-23 1035QAP001-0 1035QAP002-2 1032QAK00F-1 1032QAK00F-17 1032QAK00F-3 1035QAP001-1 1035QAP002-3 1032QAK00F-10 1032QAK00F-18 1032QAK00F-4 1035QAP001-2 1035QAP002-4 1032QAK00F-11 1032QAK00F-19 1032QAK00F-5 1035QAP001-3 1035QAP002-5 1032QAK00F-12 1032QAK00F-2 1032QAK00F-6 1035QAP001-4 1032QAK00F-13 1032QAK00F-20 1032QAK00F-7 1035QAP001-5 1032QAK00F-14 1032QAK00F-21 1032QAK00F-8 1035QAP002-0 1032QAK00F-15 1032QAK00F-22 1032QAK00F-9 1035QAP002-1 mccfs4-bui# ls -lhtr total 108 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-20 drwx------ 2 root root 3 Jan 29 17:34 1035QAP002-3 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-18 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-12 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-15 drwx------ 2 root root 3 Jan 29 17:34 1035QAP002-4 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-23 drwx------ 2 root root 3 Jan 29 17:34 1035QAP001-3 drwx------ 2 root root 3 Jan 29 17:34 1035QAP002-0 drwx------ 2 root root 3 Jan 29 17:34 1035QAP001-4 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-14 drwx------ 2 root root 3 Jan 29 17:34 1035QAP002-5 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-21 drwx------ 2 root root 3 Jan 29 17:34 1035QAP002-2 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-13 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-19 drwx------ 2 root root 3 Jan 29 17:34 1035QAP001-5 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-17 drwx------ 2 root root 3 Jan 29 17:34 1032QAK00F-22 drwx------ 2 root root 3 Jan 29 17:34 1035QAP001-2 drwx------ 2 root root 3 Jan 29 17:34 1035QAP002-1 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-8 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-2 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-10 drwx------ 2 root root 3 Jan 29 17:35 1035QAP001-1 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-5 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-1 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-6 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-4 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-11 drwx------ 2 root root 3 Jan 29 17:35 1035QAP001-0 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-3 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-9 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-7 drwx------ 2 root root 3 Jan 29 17:35 1032QAK00F-0 drwx------ 2 root root 3 Feb 13 17:11 1032QAK00F-16 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# cd mccfs4-bui# cd /var/ak/logs mccfs4-bui# tail debug.sys Jan 29 17:47:59 mccfs4-bui statd[2375]: [ID 859811 daemon.warning] statd: host ipv4.172.27.8.33 is not responding Jan 29 17:48:29 mccfs4-bui statd[2375]: [ID 859811 daemon.warning] statd: host ipv4.172.27.8.82 is not responding Jan 29 17:48:59 mccfs4-bui statd[2375]: [ID 859811 daemon.warning] statd: host ipv4.172.27.8.23 is not responding Feb 6 18:07:59 mccfs4-bui login: [ID 644210 auth.notice] ROOT LOGIN /dev/console Feb 13 17:11:41 mccfs4-bui ntpd[2492]: [ID 702911 daemon.notice] ntpd exiting on signal 15 Feb 13 17:12:02 mccfs4-bui ntpd[7964]: [ID 702911 daemon.notice] ntpd 4.2.5p200@1.1948-o Tue Oct 12 02:08:19 PDT 2010 (1) Feb 13 17:12:02 mccfs4-bui ntpd[7966]: [ID 702911 daemon.notice] proto: precision = 0.300 usec Feb 13 17:12:11 mccfs4-bui sendmail[8004]: [ID 702911 mail.notice] alias database /var/ak/etc/mail/aliases rebuilt by root Feb 13 17:12:12 mccfs4-bui sendmail[8053]: [ID 702911 mail.notice] alias database /var/ak/etc/mail/aliases rebuilt by root Feb 13 17:12:14 mccfs4-bui sendmail[8115]: [ID 702911 mail.notice] alias database /var/ak/etc/mail/aliases rebuilt by root mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# svcs -pv akd STATE NSTATE STIME CTID FMRI online - 17:12:18 262 svc:/appliance/kit/akd:default 17:11:12 7822 akd 17:11:13 7825 akd mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# fg  jobs [1]+ Stopped aksh (wd: /var/ak/stash/com/sun/ak/xmlrpc/disk) [2]- Stopped mdb -k (wd: /var/ak/stash/com/sun/ak/xmlrpc/disk) mccfs4-bui# fg 2 mdb -k (wd: /var/ak/stash/com/sun/ak/xmlrpc/disk) > > > :; :spa -v ADDR STATE NAME  fffff6000daff080 ACTIVE mccsp ADDR STATE AUX DESCRIPTION  fffff600397466c0 HEALTHY - root fffff600397459c0 HEALTHY - mirror fffff60039746080 HEALTHY - /dev/dsk/c3t5000C5001A108945d0s0 fffff60039747980 HEALTHY - /dev/dsk/c3t5000C50015A3038Dd0s0 fffff60039746d00 HEALTHY - mirror fffff60039747340 HEALTHY - /dev/dsk/c3t5000C50015BAD93Dd0s0 fffff6001cd41cc0 HEALTHY - /dev/dsk/c3t5000C50019D05DD1d0s0 fffff6001cd41040 HEALTHY - mirror fffff6001cd41680 HEALTHY - /dev/dsk/c3t5000C50019D08DADd0s0 fffff60039742c80 HEALTHY - /dev/dsk/c3t5000C50019D09E3Ad0s0 fffff60039742640 HEALTHY - mirror fffff60039742000 HEALTHY - /dev/dsk/c3t5000C50019D09F37d0s0 fffff60039739940 HEALTHY - /dev/dsk/c3t5000C50019D061B8d0s0 fffff60039739300 HEALTHY - hole fffff60039738cc0 HEALTHY - /dev/dsk/c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0s0 - - - cache fffff60039737980 HEALTHY - /dev/dsk/c0t0d0s0 fffff60039737340 HEALTHY - /dev/dsk/c0t1d0s0 >> More [, , q, n, c, a] ?   - - - spares fffff6003a0530c0 HEALTHY - /dev/dsk/c3t5000C50019D0727Fd0s0 fffff60039738680 HEALTHY - /dev/dsk/c3t5000C500150DC4EFd0s0 fffff60005581580 ACTIVE system fffff600000eb680 HEALTHY - root fffff600000eb040 HEALTHY - mirror fffff6000789e980 HEALTHY - /dev/dsk/c1t1d0s0 fffff6000789e340 HEALTHY - /dev/dsk/c1t0d0s0 > > > > z ::mdb   pt   mpt    mpt mpt_t inst mpx ntargs ioc cmds/depth wait power devs online hi / tq ================================================================================ fffff60008740000 0 1 1024 2 3/829 0 ON=D0 26+2 25+2 0/0 > > > > >  [2]+ Stopped mdb -k (wd: /var/ak/stash/com/sun/ak/xmlrpc/disk) (wd now: /var/ak/logs) mccfs4-bui# fg mdb -k (wd: /var/ak/stash/com/sun/ak/xmlrpc/disk) > $q mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# fg aksh (wd: /var/ak/stash/com/sun/ak/xmlrpc/disk) mccfs4-bui:> maintenance hardware mccfs4-bui:maintenance hardware>[0 select ch chassis-000 chassis-001 mccfs4-bui:maintenance hardware> select chassis-001 select disk list LABEL STATE MANUFACTURER MODEL SERIAL RPM disk-000 HDD 0 ok SEAGATE ST31000NSSUN1.0T 9QJ5TMK7 7200 disk-001 HDD 1 ok SEAGATE ST31000NSSUN1.0T 9QJ47BQ5 7200 disk-002 HDD 2 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVJM 7200 disk-003 HDD 3 ok SEAGATE ST31000NSSUN1.0T 9QJ4TLW1 7200 disk-004 HDD 4 ok STEC ZeusIOPS STM0000D5B58 20000 disk-005 HDD 5 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVBC 7200 disk-006 HDD 6 ok SEAGATE ST31000NSSUN1.0T 9QJ5RWAM 7200 disk-007 HDD 7 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVB4 7200 disk-008 HDD 8 ok STEC ZeusIOPS STM000107DD0 20000 disk-009 HDD 9 ok SEAGATE ST31000NSSUN1.0T 9QJ67EAW 7200 disk-010 HDD 10 ok SEAGATE ST31000NSSUN1.0T 9QJ4YWQM 7200 disk-011 HDD 11 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVZP 7200 disk-012 HDD 12 absent - - - -- disk-013 HDD 13 absent - - - -- disk-014 HDD 14 absent - - - -- disk-015 HDD 15 absent - - - -- disk-016 HDD 16 unavail unknown unknown unknown -- disk-017 HDD 17 absent - - - -- disk-018 HDD 18 absent - - - -- disk-019 HDD 19 absent - - - -- disk-020 HDD 20 absent - - - -- disk-021 HDD 21 absent - - - -- disk-022 HDD 22 absent - - - -- disk-023 HDD 23 absent - - - -- mccfs4-bui:maintenance hardware>  [1]+ Stopped aksh (wd: /var/ak/stash/com/sun/ak/xmlrpc/disk) (wd now: /var/ak/logs) mccfs4-bui# zpool status -v pool: mccsp state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: resilvered 3.98G in 0h2m with 0 errors on Thu Jan 24 19:13:10 2013 config: NAME STATE READ WRITE CKSUM mccsp ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t5000C5001A108945d0 ONLINE 0 0 0 c3t5000C50015A3038Dd0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c3t5000C50015BAD93Dd0 ONLINE 0 0 0 c3t5000C50019D05DD1d0 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 c3t5000C50019D08DADd0 ONLINE 0 0 0 c3t5000C50019D09E3Ad0 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 c3t5000C50019D09F37d0 ONLINE 0 0 0 c3t5000C50019D061B8d0 ONLINE 0 0 0 logs c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 ONLINE 0 0 0 cache c0t0d0 ONLINE 0 0 0 c0t1d0 ONLINE 0 0 0 spares c3t5000C50019D0727Fd0 AVAIL c3t5000C500150DC4EFd0 AVAIL errors: No known data errors pool: system state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: none requested config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t1d0s0 ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 errors: No known data errors mccfs4-bui# fg  cd /var/ak/stash/com/sun/ak/xmlrpc/disk mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-16 1032QAK00F-23 1035QAP001-0 1035QAP002-2 1032QAK00F-1 1032QAK00F-17 1032QAK00F-3 1035QAP001-1 1035QAP002-3 1032QAK00F-10 1032QAK00F-18 1032QAK00F-4 1035QAP001-2 1035QAP002-4 1032QAK00F-11 1032QAK00F-19 1032QAK00F-5 1035QAP001-3 1035QAP002-5 1032QAK00F-12 1032QAK00F-2 1032QAK00F-6 1035QAP001-4 1032QAK00F-13 1032QAK00F-20 1032QAK00F-7 1035QAP001-5 1032QAK00F-14 1032QAK00F-21 1032QAK00F-8 1035QAP002-0 1032QAK00F-15 1032QAK00F-22 1032QAK00F-9 1035QAP002-1 mccfs4-bui# aknv ./       fg aksh mccfs4-bui:maintenance hardware>[0 list NAME STATE MANUFACTURER MODEL SERIAL RPM chassis-000 mccfs4-bui ok Sun Microsystems, Inc. Sun Storage 7310 1035QAP001 -- chassis-001 1032QAK00F ok Sun Microsystems, Inc. J4400 1032QAK00F -- mccfs4-bui:maintenance hardware>[0  [1]+ Stopped aksh mccfs4-bui# fg aknv ./1032QAK00F-4/obj Stash Header for ./1032QAK00F-4/obj: aksh_id[AK_SHDR_MAGn] = \x7faks aksh_id[AK_SHDR_VERSION] = 1 aksh_id[AK_SHDR_FORMAT] = 1 aksh_id[AK_SHDR_PAD0] = 0 aksh_id[AK_SHDR_PAD1] = 0 Stash Object: nvlist version: 0 _class = com/sun/ak/xmlrpc/disk _version = 0x1 _atom = obj fru = hc://:product-id=SUN-Storage-J4400:server-id=:chassis-id=1032QAK00F/ses-enclosure=0/bay=4/disk=0 rid = 1032QAK00F-4 refuri = ak:/zfs/mccsp mccfs4-bui# aknv ./1032QAK00F-4/obj       8/obj Stash Header for ./1032QAK00F-8/obj: aksh_id[AK_SHDR_MAGn] = \x7faks aksh_id[AK_SHDR_VERSION] = 1 aksh_id[AK_SHDR_FORMAT] = 1 aksh_id[AK_SHDR_PAD0] = 0 aksh_id[AK_SHDR_PAD1] = 0 Stash Object: nvlist version: 0 _class = com/sun/ak/xmlrpc/disk _version = 0x1 _atom = obj fru = hc://:product-id=SUN-Storage-J4400:server-id=:chassis-id=1032QAK00F/ses-enclosure=0/bay=8/disk=0 rid = 1032QAK00F-8 refuri = ak:/zfs/mccsp mccfs4-bui# echo | format& [2] 8518 mccfs4-bui# Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t0d0 /pci@0,0/pci10de,cb84@5/disk@0,0 1. c0t1d0 /pci@0,0/pci10de,cb84@5/disk@1,0 2. c1t0d0 /pci@1,0/pci10de,cb84@5,1/disk@0,0 3. c1t1d0 /pci@1,0/pci10de,cb84@5,1/disk@1,0 4. c3t5000C5001A108945d0 /scsi_vhci/disk@g5000c5001a108945 5. c3t5000C50015A3038Dd0 /scsi_vhci/disk@g5000c50015a3038d 6. c3t5000C50015BAD93Dd0 /scsi_vhci/disk@g5000c50015bad93d 7. c3t5000C50019D05DD1d0 /scsi_vhci/disk@g5000c50019d05dd1 8. c3t5000C50019D08DADd0 /scsi_vhci/disk@g5000c50019d08dad 9. c3t5000C50019D09E3Ad0 /scsi_vhci/disk@g5000c50019d09e3a 10. c3t5000C50019D09F37d0 /scsi_vhci/disk@g5000c50019d09f37 11. c3t5000C50019D061B8d0 /scsi_vhci/disk@g5000c50019d061b8 12. c3t5000C50019D0727Fd0 /scsi_vhci/disk@g5000c50019d0727f 13. c3t5000C500150DC4EFd0 /scsi_vhci/disk@g5000c500150dc4ef 14. c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 /scsi_vhci/disk@gATASTECZeusIOPS018GBytesSTM0000D5B58 15. c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 /scsi_vhci/disk@gATASTECZeusIOPS018GBytesSTM000107DD0 Specify disk (enter its number): Specify disk (enter its number): [2]- Exit 1 echo | format mccfs4-bui# mccfs4-bui# zpool att  dd     list & [2] 8519 mccfs4-bui# NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT mccsp 3.62T 991G 2.66T 26% 1.00x ONLINE - system 464G 130G 334G 27% 1.00x ONLINE - [2]- Done zpool list mccfs4-bui# mccfs4-bui# mccfs4-bui# fg aksh mccfs4-bui:maintenance hardware>[0 select ch chassis-000 chassis-001 mccfs4-bui:maintenance hardware> select chassis-0001 sleelect disk mccfs4-bui:maintenance chassis-001 disk>[0 list LABEL STATE MANUFACTURER MODEL SERIAL RPM disk-000 HDD 0 ok SEAGATE ST31000NSSUN1.0T 9QJ5TMK7 7200 disk-001 HDD 1 ok SEAGATE ST31000NSSUN1.0T 9QJ47BQ5 7200 disk-002 HDD 2 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVJM 7200 disk-003 HDD 3 ok SEAGATE ST31000NSSUN1.0T 9QJ4TLW1 7200 disk-004 HDD 4 ok STEC ZeusIOPS STM0000D5B58 20000 disk-005 HDD 5 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVBC 7200 disk-006 HDD 6 ok SEAGATE ST31000NSSUN1.0T 9QJ5RWAM 7200 disk-007 HDD 7 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVB4 7200 disk-008 HDD 8 ok STEC ZeusIOPS STM000107DD0 20000 disk-009 HDD 9 ok SEAGATE ST31000NSSUN1.0T 9QJ67EAW 7200 disk-010 HDD 10 ok SEAGATE ST31000NSSUN1.0T 9QJ4YWQM 7200 disk-011 HDD 11 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVZP 7200 disk-012 HDD 12 absent - - - -- disk-013 HDD 13 absent - - - -- disk-014 HDD 14 absent - - - -- disk-015 HDD 15 absent - - - -- disk-016 HDD 16 unavail unknown unknown unknown -- disk-017 HDD 17 absent - - - -- disk-018 HDD 18 absent - - - -- disk-019 HDD 19 absent - - - -- disk-020 HDD 20 absent - - - -- disk-021 HDD 21 absent - - - -- disk-022 HDD 22 absent - - - -- disk-023 HDD 23 absent - - - -- mccfs4-bui:maintenance chassis-001 disk> select hddi disk-000 disk-003 disk-006 disk-009 disk-012 disk-015 disk-018 disk-021 disk-001 disk-004 disk-007 disk-010 disk-013 disk-016 disk-019 disk-022 disk-002 disk-005 disk-008 disk-011 disk-014 disk-017 disk-020 disk-023 mccfs4-bui:maintenance chassis-001 disk> select disk-008 show Properties:  label = HDD 8 present = true faulted = false manufacturer = STEC model = ZeusIOPS serial = STM000107DD0 revision = 0430 size = 17G type = log use = log rpm = 20000 device = c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 pathcount = 2 interface = SATA locate = false offline = false mccfs4-bui:maintenance chassis-001 disk>   [1mccfs4-bui:maintenance chassis-001 disk> [select disk-008 show4 sho error: invalid command "sho" mccfs4-bui:maintenance chassis-001 disk>   [1mccfs4-bui:maintenance chassis-001 disk> [select disk-004 show Properties:  label = HDD 4 present = true faulted = false manufacturer = STEC model = ZeusIOPS serial = STM0000D5B58 revision = 0430 size = 17G type = log rpm = 20000 device = c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 pathcount = 1 interface = SATA locate = false mccfs4-bui:maintenance chassis-001 disk> cd .. mccfs4-bui:maintenance chassis-001>  [1]+ Stopped aksh mccfs4-bui# pwd /var/ak/stash/com/sun/ak/xmlrpc/disk mccfs4-bui# pwdfgzpool list &[2@echo | format& mccfs4-bui# aknv ./1032QAK00F-8/obj Stash Header for ./1032QAK00F-8/obj: aksh_id[AK_SHDR_MAGn] = \x7faks aksh_id[AK_SHDR_VERSION] = 1 aksh_id[AK_SHDR_FORMAT] = 1 aksh_id[AK_SHDR_PAD0] = 0 aksh_id[AK_SHDR_PAD1] = 0 Stash Object: nvlist version: 0 _class = com/sun/ak/xmlrpc/disk _version = 0x1 _atom = obj fru = hc://:product-id=SUN-Storage-J4400:server-id=:chassis-id=1032QAK00F/ses-enclosure=0/bay=8/disk=0 rid = 1032QAK00F-8 refuri = ak:/zfs/mccsp mccfs4-bui# zpool status & | grep c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0  & [2] 8523 mccfs4-bui# [2]- Exit 1 zpool status | grep c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58 mccfs4-bui# mccfs4-bui# z svcadm disk able -t akd mccfs4-bui# pwd /var/ak/stash/com/sun/ak/xmlrpc/disk mccfs4-bui# svcs -pv akd STATE NSTATE STIME CTID FMRI disabled - 17:21:18 - svc:/appliance/kit/akd:default mccfs4-bui# mccfs4-bui# mccfs4-bui# an  ls 1032QAK00F-0 1032QAK00F-16 1032QAK00F-23 1035QAP001-0 1035QAP002-2 1032QAK00F-1 1032QAK00F-17 1032QAK00F-3 1035QAP001-1 1035QAP002-3 1032QAK00F-10 1032QAK00F-18 1032QAK00F-4 1035QAP001-2 1035QAP002-4 1032QAK00F-11 1032QAK00F-19 1032QAK00F-5 1035QAP001-3 1035QAP002-5 1032QAK00F-12 1032QAK00F-2 1032QAK00F-6 1035QAP001-4 1032QAK00F-13 1032QAK00F-20 1032QAK00F-7 1035QAP001-5 1032QAK00F-14 1032QAK00F-21 1032QAK00F-8 1035QAP002-0 1032QAK00F-15 1032QAK00F-22 1032QAK00F-9 1035QAP002-1 mccfs4-bui# mv 1032QAK00F-4 /var/tmp mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-15 1032QAK00F-21 1032QAK00F-8 1035QAP001-5 1032QAK00F-1 1032QAK00F-16 1032QAK00F-22 1032QAK00F-9 1035QAP002-0 1032QAK00F-10 1032QAK00F-17 1032QAK00F-23 1035QAP001-0 1035QAP002-1 1032QAK00F-11 1032QAK00F-18 1032QAK00F-3 1035QAP001-1 1035QAP002-2 1032QAK00F-12 1032QAK00F-19 1032QAK00F-5 1035QAP001-2 1035QAP002-3 1032QAK00F-13 1032QAK00F-2 1032QAK00F-6 1035QAP001-3 1035QAP002-4 1032QAK00F-14 1032QAK00F-20 1032QAK00F-7 1035QAP001-4 1035QAP002-5 mccfs4-bui# s svcadm enable akd mccfs4-bui# mccfs4-bui# mccfs4-bui# svcs -pv akd STATE NSTATE STIME CTID FMRI offline online 17:22:19 323 svc:/appliance/kit/akd:default 17:22:19 8545 akd 17:22:19 8546 akd 17:22:21 8549 akd mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-15 1032QAK00F-21 1032QAK00F-8 1035QAP001-5 1032QAK00F-1 1032QAK00F-16 1032QAK00F-22 1032QAK00F-9 1035QAP002-0 1032QAK00F-10 1032QAK00F-17 1032QAK00F-23 1035QAP001-0 1035QAP002-1 1032QAK00F-11 1032QAK00F-18 1032QAK00F-3 1035QAP001-1 1035QAP002-2 1032QAK00F-12 1032QAK00F-19 1032QAK00F-5 1035QAP001-2 1035QAP002-3 1032QAK00F-13 1032QAK00F-2 1032QAK00F-6 1035QAP001-3 1035QAP002-4 1032QAK00F-14 1032QAK00F-20 1032QAK00F-7 1035QAP001-4 1035QAP002-5 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-15 1032QAK00F-21 1032QAK00F-8 1035QAP001-5 1032QAK00F-1 1032QAK00F-16 1032QAK00F-22 1032QAK00F-9 1035QAP002-0 1032QAK00F-10 1032QAK00F-17 1032QAK00F-23 1035QAP001-0 1035QAP002-1 1032QAK00F-11 1032QAK00F-18 1032QAK00F-3 1035QAP001-1 1035QAP002-2 1032QAK00F-12 1032QAK00F-19 1032QAK00F-5 1035QAP001-2 1035QAP002-3 1032QAK00F-13 1032QAK00F-2 1032QAK00F-6 1035QAP001-3 1035QAP002-4 1032QAK00F-14 1032QAK00F-20 1032QAK00F-7 1035QAP001-4 1035QAP002-5 mccfs4-bui# mccfs4-bui# svcs -pv akd STATE NSTATE STIME CTID FMRI offline online 17:22:19 323 svc:/appliance/kit/akd:default 17:22:19 8545 akd 17:22:19 8546 akd 17:22:21 8549 akd mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklkog        !aklog aklog rm | egrep -v 'lib/|LOCK|^$' | tail Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-10 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-5 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-1 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-6 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-11 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-3 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-9 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-7 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-0 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/replication/mccsp to 2 (2) succeeded mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog rm | egrep -v 'lib/|LOCK|^$' | tail Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-5 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-1 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-6 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-11 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-3 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-9 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-7 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/disk/1032QAK00F-0 to 8 (SU13) succeeded Wed Feb 13 17:13:22 2013: update ak:/replication/mccsp to 2 (2) succeeded Wed Feb 13 17:22:59 2013: ak_rm_post_init: loaded gen 1001 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog rm | egrep - 'lib/|LOCK|^$' | tail Wed Feb 13 17:23:06 2013: export of ak:/disk/1035QAP002-1 succeeded in 0.000s Wed Feb 13 17:23:06 2013: insert_svc ak:/smb/nge0 Wed Feb 13 17:23:06 2013: insert_svc ak:/smb/nge1 Wed Feb 13 17:23:06 2013: update_svc ak:/role/basic Wed Feb 13 17:23:06 2013: update_svc ak:/user/bd1c7e89-5b83-4e01-9dae-fa6fded03cbe Wed Feb 13 17:23:06 2013: update_svc ak:/object/2f558fc7-e689-cfab-fe33-e7fed1186248 Wed Feb 13 17:23:06 2013: update_svc ak:/object/939a5f56-9859-6505-dfe8-cb0dc97e54c3 Wed Feb 13 17:23:07 2013: update_svc ak:/object/8cd412a8-4ca6-ce34-ee62-dbb33a6f3d25 Wed Feb 13 17:23:07 2013: update_svc ak:/object/3136b9ce-6324-6048-a27e-e990460c306c Wed Feb 13 17:23:07 2013: update_svc ak:/object/bcc07800-8711-66f8-8034-e897fce8f974 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog rm | egrep -v 'lib/|LOCK|^$' | tail Wed Feb 13 17:23:12 2013: update_svc ak:/object/60f9a9fc-1266-cb68-b6ce-a948043a6a0c Wed Feb 13 17:23:12 2013: update_svc ak:/object/b5ddbf9f-ccfc-62e6-c09d-e191431bb5ff Wed Feb 13 17:23:12 2013: update_svc ak:/object/e8600ddd-cbb9-4bf3-c5c0-e11db0fd2c4b Wed Feb 13 17:23:12 2013: update_svc ak:/object/635b393e-c8d7-4922-bb75-ff124e0f29f8 Wed Feb 13 17:23:12 2013: update_svc ak:/object/ccfc8d17-8957-c802-db72-d9b40bf35c7e Wed Feb 13 17:23:13 2013: update_svc ak:/object/22aa582c-05b6-6e89-c30e-ac4b29b14ad8 Wed Feb 13 17:23:13 2013: update_svc ak:/object/e0471260-4aa4-43d5-d588-965cda7019b9 Wed Feb 13 17:23:13 2013: update_svc ak:/object/c3b8d882-8b65-6a18-83d4-c31f1dc61f32 Wed Feb 13 17:23:13 2013: update_svc ak:/object/4028da59-4b38-c909-990a-90acb3b15bdb Wed Feb 13 17:23:13 2013: update_svc ak:/object/c63c1483-85b6-c468-d29a-e317d8ddcf57 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# svcs -pv akd STATE NSTATE STIME CTID FMRI online - 17:23:01 323 svc:/appliance/kit/akd:default 17:22:19 8546 akd 17:22:21 8549 akd mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# svcs -pv akdaklog rm | egrep -v 'lib/|LOCK|^$' | tail Wed Feb 13 17:23:21 2013: update_svc ak:/svc/ipmp Wed Feb 13 17:23:21 2013: update_svc ak:/svc/nis Wed Feb 13 17:23:21 2013: update_svc ak:/svc/scrk Wed Feb 13 17:23:22 2013: update_svc ak:/svc/ssh Wed Feb 13 17:23:22 2013: update_svc ak:/svc/syslog Wed Feb 13 17:23:22 2013: update_svc ak:/svc/idmap Wed Feb 13 17:23:22 2013: update_svc ak:/svc/ad Wed Feb 13 17:23:22 2013: update_svc ak:/svc/dns Wed Feb 13 17:23:23 2013: update_svc ak:/svc/http Wed Feb 13 17:23:23 2013: update_svc ak:/svc/dynrouting mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog rm | egrep -v 'lib/|LOCK|^$' | tail Wed Feb 13 17:23:26 2013: import of ak:/alert/8e22c9bd-c44e-c79f-d613-b766165fa93a succeeded in 0.000s Wed Feb 13 17:23:26 2013: update_svc ak:/replication/mccsp Wed Feb 13 17:23:26 2013: update_svc ak:/shadow/mccsp Wed Feb 13 17:23:26 2013: update_svc ak:/fct/mccsp Wed Feb 13 17:23:26 2013: update_svc ak:/smb/nge2 Wed Feb 13 17:23:26 2013: resource change 0 pending at lock exit: ak:/object/2f558fc7-e689-cfab-fe33-e7fed1186248 Wed Feb 13 17:23:26 2013: resource change 0 pending at lock exit: ak:/object/939a5f56-9859-6505-dfe8-cb0dc97e54c3 Wed Feb 13 17:23:26 2013: resource change 0 pending at lock exit: ak:/object/8cd412a8-4ca6-ce34-ee62-dbb33a6f3d25 Wed Feb 13 17:23:26 2013: resource change 0 pending at lock exit: ak:/object/3136b9ce-6324-6048-a27e-e990460c306c Wed Feb 13 17:23:26 2013: resource change 0 pending at lock exit: ak:/object/bcc07800-8711-66f8-8034-e897fce8f974 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog rm | egrep -v 'lib/|LOCK|^$' | tail Wed Feb 13 17:23:28 2013: import of ak:/zfs/system succeeded in 0.001s Wed Feb 13 17:23:29 2013: update ak:/zfs/system to 1 (1) succeeded Wed Feb 13 17:23:29 2013: import of ak:/svc/identity succeeded in 0.000s Wed Feb 13 17:23:29 2013: import of ak:/net/nge0 succeeded in 0.000s Wed Feb 13 17:23:29 2013: import of ak:/smb/nge0 succeeded in 0.276s Wed Feb 13 17:23:30 2013: import of ak:/disk/1035QAP001-2 failed in 0.000s: not enumerated Wed Feb 13 17:23:30 2013: import of ak:/disk/1035QAP001-3 failed in 0.000s: not enumerated Wed Feb 13 17:23:30 2013: import of ak:/disk/1035QAP001-4 failed in 0.000s: not enumerated Wed Feb 13 17:23:30 2013: import of ak:/disk/1035QAP001-5 failed in 0.000s: not enumerated Wed Feb 13 17:23:30 2013: import of ak:/disk/1032QAK00F-12 failed in 0.000s: not enumerated mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog rm | egrep -v 'lib/|LOCK|^$' | tail Wed Feb 13 17:23:30 2013: import of ak:/disk/1032QAK00F-11 succeeded in 0.000s Wed Feb 13 17:23:31 2013: import of ak:/disk/1035QAP001-0 succeeded in 0.000s Wed Feb 13 17:23:31 2013: import of ak:/disk/1032QAK00F-3 succeeded in 0.000s Wed Feb 13 17:23:31 2013: import of ak:/disk/1032QAK00F-9 succeeded in 0.000s Wed Feb 13 17:23:31 2013: import of ak:/disk/1032QAK00F-7 succeeded in 0.000s Wed Feb 13 17:23:31 2013: import of ak:/disk/1032QAK00F-0 succeeded in 0.000s Wed Feb 13 17:23:31 2013: import of ak:/zfs/mccsp succeeded in 0.002s Wed Feb 13 17:23:31 2013: update ak:/zfs/mccsp to 11 (11) succeeded Wed Feb 13 17:23:31 2013: [nas import] discovery completed in 0.025s Wed Feb 13 17:23:31 2013: [nas import] mounted 14 datasets in 0.016s mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog rm | egrep -v 'lib/|LOCK|^$' | tail Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-14 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-15 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-16 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-17 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-18 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-19 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-20 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-21 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-22 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-23 failed in 0.000s: not enumerated mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog rm | egrep - 'lib/|LOCK|^$' | tail Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-14 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-15 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-16 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-17 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-18 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-19 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-20 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-21 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-22 failed in 0.000s: not enumerated Wed Feb 13 17:23:37 2013: import of ak:/disk/1032QAK00F-23 failed in 0.000s: not enumerated mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# echo ::spa| mdb -k ADDR STATE NAME fffff6000daff080 ACTIVE mccsp fffff60005581580 ACTIVE system mccfs4-bui# echo ::spa -v | md b-k    b -k ADDR STATE NAME fffff6000daff080 ACTIVE mccsp ADDR STATE AUX DESCRIPTION fffff600397466c0 HEALTHY - root fffff600397459c0 HEALTHY - mirror fffff60039746080 HEALTHY - /dev/dsk/c3t5000C5001A108945d0s0 fffff60039747980 HEALTHY - /dev/dsk/c3t5000C50015A3038Dd0s0 fffff60039746d00 HEALTHY - mirror fffff60039747340 HEALTHY - /dev/dsk/c3t5000C50015BAD93Dd0s0 fffff6001cd41cc0 HEALTHY - /dev/dsk/c3t5000C50019D05DD1d0s0 fffff6001cd41040 HEALTHY - mirror fffff6001cd41680 HEALTHY - /dev/dsk/c3t5000C50019D08DADd0s0 fffff60039742c80 HEALTHY - /dev/dsk/c3t5000C50019D09E3Ad0s0 fffff60039742640 HEALTHY - mirror fffff60039742000 HEALTHY - /dev/dsk/c3t5000C50019D09F37d0s0 fffff60039739940 HEALTHY - /dev/dsk/c3t5000C50019D061B8d0s0 fffff60039739300 HEALTHY - hole fffff60039738cc0 HEALTHY - /dev/dsk/c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0s0 - - - cache fffff60039737980 HEALTHY - /dev/dsk/c0t0d0s0 fffff60039737340 HEALTHY - /dev/dsk/c0t1d0s0 - - - spares fffff6003a0530c0 HEALTHY - /dev/dsk/c3t5000C50019D0727Fd0s0 fffff60039738680 HEALTHY - /dev/dsk/c3t5000C500150DC4EFd0s0 fffff60005581580 ACTIVE system fffff600000eb680 HEALTHY - root fffff600000eb040 HEALTHY - mirror fffff6000789e980 HEALTHY - /dev/dsk/c1t1d0s0 fffff6000789e340 HEALTHY - /dev/dsk/c1t0d0s0 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# pwd /var/ak/stash/com/sun/ak/xmlrpc/disk mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-16 1032QAK00F-23 1035QAP001-0 1035QAP002-2 1032QAK00F-1 1032QAK00F-17 1032QAK00F-3 1035QAP001-1 1035QAP002-3 1032QAK00F-10 1032QAK00F-18 1032QAK00F-4 1035QAP001-2 1035QAP002-4 1032QAK00F-11 1032QAK00F-19 1032QAK00F-5 1035QAP001-3 1035QAP002-5 1032QAK00F-12 1032QAK00F-2 1032QAK00F-6 1035QAP001-4 1032QAK00F-13 1032QAK00F-20 1032QAK00F-7 1035QAP001-5 1032QAK00F-14 1032QAK00F-21 1032QAK00F-8 1035QAP002-0 1032QAK00F-15 1032QAK00F-22 1032QAK00F-9 1035QAP002-1 mccfs4-bui# aknv ./102 32QAK00F-4 aknv: failed to read input file: Is a directory mccfs4-bui# aknv ./1032QAK00F-4/obj Stash Header for ./1032QAK00F-4/obj: aksh_id[AK_SHDR_MAGn] = \x7faks aksh_id[AK_SHDR_VERSION] = 1 aksh_id[AK_SHDR_FORMAT] = 1 aksh_id[AK_SHDR_PAD0] = 0 aksh_id[AK_SHDR_PAD1] = 0 Stash Object: nvlist version: 0 _class = com/sun/ak/xmlrpc/disk _version = 0x1 _atom = obj fru = hc://:product-id=SUN-Storage-J4400:server-id=:chassis-id=1032QAK00F/ses-enclosure=0/bay=4/disk=0 rid = 1032QAK00F-4 refuri = ak:/zfs/mccsp mccfs4-bui# aknv ./1032QAK00F-4/obj       8/obj Stash Header for ./1032QAK00F-8/obj: aksh_id[AK_SHDR_MAGn] = \x7faks aksh_id[AK_SHDR_VERSION] = 1 aksh_id[AK_SHDR_FORMAT] = 1 aksh_id[AK_SHDR_PAD0] = 0 aksh_id[AK_SHDR_PAD1] = 0 Stash Object: nvlist version: 0 _class = com/sun/ak/xmlrpc/disk _version = 0x1 _atom = obj fru = hc://:product-id=SUN-Storage-J4400:server-id=:chassis-id=1032QAK00F/ses-enclosure=0/bay=8/disk=0 rid = 1032QAK00F-8 refuri = ak:/zfs/mccsp mccfs4-bui# aknv ./1032QAK00F-8/obj       16/obj Stash Header for ./1032QAK00F-16/obj: aksh_id[AK_SHDR_MAGn] = \x7faks aksh_id[AK_SHDR_VERSION] = 1 aksh_id[AK_SHDR_FORMAT] = 1 aksh_id[AK_SHDR_PAD0] = 0 aksh_id[AK_SHDR_PAD1] = 0 Stash Object: nvlist version: 0 _class = com/sun/ak/xmlrpc/disk _version = 0x1 _atom = obj fru = hc://:product-id=SUN-Storage-J4400:server-id=:chassis-id=1032QAK00F/ses-enclosure=0/bay=16/disk=0 rid = 1032QAK00F-16 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# fg aksh mccfs4-bui:maintenance chassis-001> select disk ls Disks:  LABEL STATE MANUFACTURER MODEL SERIAL RPM disk-000 HDD 0 ok SEAGATE ST31000NSSUN1.0T 9QJ5TMK7 7200 disk-001 HDD 1 ok SEAGATE ST31000NSSUN1.0T 9QJ47BQ5 7200 disk-002 HDD 2 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVJM 7200 disk-003 HDD 3 ok SEAGATE ST31000NSSUN1.0T 9QJ4TLW1 7200 disk-004 HDD 4 ok STEC ZeusIOPS STM0000D5B58 20000 disk-005 HDD 5 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVBC 7200 disk-006 HDD 6 ok SEAGATE ST31000NSSUN1.0T 9QJ5RWAM 7200 disk-007 HDD 7 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVB4 7200 disk-008 HDD 8 ok STEC ZeusIOPS STM000107DD0 20000 disk-009 HDD 9 ok SEAGATE ST31000NSSUN1.0T 9QJ67EAW 7200 disk-010 HDD 10 ok SEAGATE ST31000NSSUN1.0T 9QJ4YWQM 7200 disk-011 HDD 11 ok SEAGATE ST31000NSSUN1.0T 9QJ5RVZP 7200 disk-012 HDD 12 absent - - - -- disk-013 HDD 13 absent - - - -- disk-014 HDD 14 absent - - - -- disk-015 HDD 15 absent - - - -- disk-016 HDD 16 unavail unknown unknown unknown -- disk-017 HDD 17 absent - - - -- disk-018 HDD 18 absent - - - -- disk-019 HDD 19 absent - - - -- disk-020 HDD 20 absent - - - -- disk-021 HDD 21 absent - - - -- disk-022 HDD 22 absent - - - -- disk-023 HDD 23 absent - - - -- mccfs4-bui:mantenance chassis-001>   [1mccfs4-bui:maintenance chassis-001> select disk lsselect di disk-000 disk-003 disk-006 disk-009 disk-012 disk-015 disk-018 disk-021 disk-001 disk-004 disk-007 disk-010 disk-013 disk-016 disk-019 disk-022 disk-002 disk-005 disk-008 disk-011 disk-014 disk-017 disk-020 disk-023 mccfs4-bui:maintenance chassis-001> select disk select disk-004 show Properties:  label = HDD 4 present = true faulted = false manufacturer = STEC model = ZeusIOPS serial = STM0000D5B58 revision = 0430 size = 17G type = log rpm = 20000 device = c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 pathcount = 1 interface = SATA locate = false mccfs4-bui:maintenance chassis-001>   [1mccfs4-bui:maintenance chassis-001> select disk select disk-004 show16 show Properties:  label = HDD 16 present = true faulted = false rpm = -- pathcount = 0 locate = false mccfs4-bui:maintenance chassis-001>  [1]+ Stopped aksh mccfs4-bui# zpool staus    tus -v & [2] 9221 mccfs4-bui# pool: mccsp state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: resilvered 3.98G in 0h2m with 0 errors on Thu Jan 24 19:13:10 2013 config: NAME STATE READ WRITE CKSUM mccsp ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t5000C5001A108945d0 ONLINE 0 0 0 c3t5000C50015A3038Dd0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c3t5000C50015BAD93Dd0 ONLINE 0 0 0 c3t5000C50019D05DD1d0 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 c3t5000C50019D08DADd0 ONLINE 0 0 0 c3t5000C50019D09E3Ad0 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 c3t5000C50019D09F37d0 ONLINE 0 0 0 c3t5000C50019D061B8d0 ONLINE 0 0 0 logs c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 ONLINE 0 0 0 cache c0t0d0 ONLINE 0 0 0 c0t1d0 ONLINE 0 0 0 spares c3t5000C50019D0727Fd0 AVAIL c3t5000C500150DC4EFd0 AVAIL errors: No known data errors pool: system state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: none requested config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t1d0s0 ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 errors: No known data errors [2]- Done zpool status -v mccfs4-bui# zpool add mccsp log c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0& [2] 9222 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# [2]- Done zpool add mccsp log c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 mccfs4-bui# mccfs4-bui# zpool sat  tatus -v & [2] 9240 pool: mccsp state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. mccfs4-bui# action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: resilvered 3.98G in 0h2m with 0 errors on Thu Jan 24 19:13:10 2013 config: NAME STATE READ WRITE CKSUM mccsp ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t5000C5001A108945d0 ONLINE 0 0 0 c3t5000C50015A3038Dd0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c3t5000C50015BAD93Dd0 ONLINE 0 0 0 c3t5000C50019D05DD1d0 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 c3t5000C50019D08DADd0 ONLINE 0 0 0 c3t5000C50019D09E3Ad0 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 c3t5000C50019D09F37d0 ONLINE 0 0 0 c3t5000C50019D061B8d0 ONLINE 0 0 0 logs c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 ONLINE 0 0 0 c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 ONLINE 0 0 0 cache c0t0d0 ONLINE 0 0 0 c0t1d0 ONLINE 0 0 0 spares c3t5000C50019D0727Fd0 AVAIL c3t5000C500150DC4EFd0 AVAIL errors: No known data errors pool: system state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: none requested config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t1d0s0 ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 errors: No known data errors [2]- Done zpool status -v mccfs4-bui# mccfs4-bui# mccfs4-bui# fg aksh mccfs4-bui:mantenance chassis-001> sleep eelect disk mccfs4-bui:mantenance chassis-001>  [1]+ Stopped aksh mccfs4-bui# josg  bs [1]+ Stopped aksh mccfs4-bui# mdb -k Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs sata sd emlxs qlc ip hook neti arp usba stmf stmf_sbd sockfs lofs random md mpt crypto idm smbsrv nfs cpc ufs logindmux ptm nsmb mpt_sas pmcs ] > ::sap  pa -v ADDR STATE NAME  fffff6000daff080 ACTIVE mccsp ADDR STATE AUX DESCRIPTION  fffff600397466c0 HEALTHY - root fffff600397459c0 HEALTHY - mirror fffff60039746080 HEALTHY - /dev/dsk/c3t5000C5001A108945d0s0 fffff60039747980 HEALTHY - /dev/dsk/c3t5000C50015A3038Dd0s0 fffff60039746d00 HEALTHY - mirror fffff60039747340 HEALTHY - /dev/dsk/c3t5000C50015BAD93Dd0s0 fffff6001cd41cc0 HEALTHY - /dev/dsk/c3t5000C50019D05DD1d0s0 fffff6001cd41040 HEALTHY - mirror fffff6001cd41680 HEALTHY - /dev/dsk/c3t5000C50019D08DADd0s0 fffff60039742c80 HEALTHY - /dev/dsk/c3t5000C50019D09E3Ad0s0 fffff60039742640 HEALTHY - mirror fffff60039742000 HEALTHY - /dev/dsk/c3t5000C50019D09F37d0s0 fffff60039739940 HEALTHY - /dev/dsk/c3t5000C50019D061B8d0s0 fffff603a6371900 HEALTHY - /dev/dsk/c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0s0 fffff60039738cc0 HEALTHY - /dev/dsk/c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0s0 - - - cache fffff60039737980 HEALTHY - /dev/dsk/c0t0d0s0 >> More [, , q, n, c, a] ?   fffff60039737340 HEALTHY - /dev/dsk/c0t1d0s0 - - - spares fffff6003a0530c0 HEALTHY - /dev/dsk/c3t5000C50019D0727Fd0s0 fffff60039738680 HEALTHY - /dev/dsk/c3t5000C500150DC4EFd0s0 fffff60005581580 ACTIVE system fffff600000eb680 HEALTHY - root fffff600000eb040 HEALTHY - mirror fffff6000789e980 HEALTHY - /dev/dsk/c1t1d0s0 fffff6000789e340 HEALTHY - /dev/dsk/c1t0d0s0 > > > > ::mpt mpt_t nst mpx ntargs ioc cmds/depth wait power devs online hi / tq ================================================================================ fffff60008740000 0 1 1024 2 12/829 0 ON=D0 26+2 25+2 0/0 > > > > > ::mpt -yw ! cut -c17- inst mpx ntargs ioc cmds/depth wait power devs online hi / tq ================================================================ 0 1 1024 2 6/829 0 ON=D0 26+2 25+2 0/0 Entry: Minor Target SAS_WWN Port Expander_WWN:Phy Ena/Dis DeviceName Status ----------------------------------------------------------------------------------------------------------- 0: 3 12 5001636000281444 0 500163600028147f:16 1/1 saspath@0,500163600028147f:16 Present 1: 4 20 500163600028144c 0 500163600028147f:8 1/0 saspath@0,500163600028147f:8 Enabled Present 2: 5 21 500163600028144d 0 500163600028147f:9 1/0 saspath@0,500163600028147f:9 Enabled Present 3: 6 22 500163600028144e 0 500163600028147f:10 1/0 saspath@0,500163600028147f:10 Enabled Present 4: 7 23 500163600028144f 0 500163600028147f:11 1/0 saspath@0,500163600028147f:11 Enabled Present 5: 8 24 5001636000281450 0 500163600028147f:4 1/0 saspath@0,500163600028147f:4 Enabled Present 6: 9 25 5001636000281451 0 500163600028147f:5 1/0 saspath@0,500163600028147f:5 Enabled Present 7: 10 26 5001636000281452 0 500163600028147f:6 1/0 saspath@0,500163600028147f:6 Enabled Present 8: 11 27 5001636000281453 0 500163600028147f:7 1/0 saspath@0,500163600028147f:7 Enabled Present 9: 12 28 5001636000281454 0 500163600028147f:0 1/0 saspath@0,500163600028147f:0 Enabled Present 10: 13 29 5001636000281455 0 500163600028147f:1 1/0 saspath@0,500163600028147f:1 Enabled Present 11: 14 30 5001636000281456 0 500163600028147f:2 1/0 saspath@0,500163600028147f:2 Enabled Present 12: 15 31 5001636000281457 0 500163600028147f:3 1/0 saspath@0,500163600028147f:3 Enabled Present 13: 16 70 50016360002414cc 4 50016360002414ff:8 1/0 saspath@4,50016360002414ff:8 Enabled Present 14: 17 71 50016360002414cd 4 50016360002414ff:9 1/0 saspath@4,50016360002414ff:9 Enabled Present 15: 18 72 50016360002414ce 4 50016360002414ff:10 1/0 saspath@4,50016360002414ff:10 Enabled Present 16: 19 73 50016360002414cf 4 50016360002414ff:11 1/0 saspath@4,50016360002414ff:11 Enabled Present 17: 20 75 50016360002414d1 4 50016360002414ff:5 1/0 saspath@4,50016360002414ff:5 Enabled Present 18: 21 76 50016360002414d2 4 50016360002414ff:6 1/0 saspath@4,50016360002414ff:6 Enabled Present 19: 22 77 50016360002414d3 4 50016360002414ff:7 1/0 saspath@4,50016360002414ff:7 Enabled Present 20: 23 78 50016360002414d4 4 50016360002414ff:0 1/0 saspath@4,50016360002414ff:0 Enabled Present 21: 24 79 50016360002414d5 4 50016360002414ff:1 1/0 saspath@4,50016360002414ff:1 Enabled Present 22: 25 80 50016360002414d6 4 50016360002414ff:2 1/0 saspath@4,50016360002414ff:2 Enabled Present 23: 26 81 50016360002414d7 4 50016360002414ff:3 1/0 saspath@4,50016360002414ff:3 Enabled Present > > > ::  $q mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# fg aksh mccfs4-bui:maintenance chassis-001> select disk mccfs4-bui:mantenance chassis-001 disk> list ^Z [1]+ Stopped aksh mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# iostat -xczn 2 4 cpu us sy wt id 0 6 0 94 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 7.7 7.0 101.0 653.4 0.1 0.0 5.5 0.8 1 1 c0t0d0 7.9 7.0 102.7 654.2 0.1 0.0 5.4 0.8 1 1 c0t1d0 0.7 25.0 35.5 484.0 0.2 0.0 7.1 1.2 3 3 c1t0d0 0.4 25.0 25.8 484.0 0.2 0.0 6.8 1.1 3 3 c1t1d0 0.0 0.0 0.0 0.1 0.0 0.0 17.6 138.7 0 0 c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 0.0 307.5 0.0 8897.2 0.0 0.3 0.0 1.0 0 6 c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.8 0 0 c3t5000C50019D0727Fd0 7.1 31.7 119.3 2087.3 0.0 0.4 0.0 10.0 0 7 c3t5000C50019D09E3Ad0 7.1 31.7 119.5 2087.3 0.0 0.4 0.0 9.9 0 7 c3t5000C50019D08DADd0 7.1 31.8 119.0 2087.9 0.0 0.4 0.0 9.9 0 7 c3t5000C50015A3038Dd0 7.1 31.8 119.7 2087.3 0.0 0.4 0.0 10.0 0 7 c3t5000C50019D05DD1d0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.8 0 0 c3t5000C500150DC4EFd0 10.5 31.7 122.8 2087.9 0.0 0.4 0.0 9.4 0 7 c3t5000C5001A108945d0 7.1 31.5 119.3 2087.4 0.0 0.4 0.0 10.1 0 6 c3t5000C50019D061B8d0 7.1 31.8 119.5 2087.3 0.0 0.4 0.0 10.0 0 7 c3t5000C50015BAD93Dd0 10.4 31.5 122.9 2087.4 0.0 0.4 0.0 9.4 0 7 c3t5000C50019D09F37d0 cpu us sy wt id 3 18 0 79 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 17.9 0.0 272.7 0.0 0.0 0.0 0.0 0.4 0 1 c0t0d0 18.9 0.0 302.6 0.0 0.0 0.0 0.0 0.4 0 1 c0t1d0 0.0 41.8 0.0 863.6 0.7 0.0 16.2 0.3 10 1 c1t0d0 0.0 47.3 0.0 888.2 0.6 0.0 12.6 0.3 10 1 c1t1d0 0.0 8.0 0.0 746.4 0.7 3.1 89.6 388.0 23 77 c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 0.0 40.8 0.0 4846.8 0.0 0.3 0.0 6.3 0 3 c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 0.5 0.0 0.5 0.0 0.0 0.0 0.0 8.4 0 0 c3t5000C50015A3038Dd0 0.5 0.0 0.5 0.0 0.0 0.0 0.0 46.9 0 2 c3t5000C50015BAD93Dd0 cpu us sy wt id 3 23 0 74 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 79.5 1.5 3693.3 16.0 0.4 0.1 4.6 0.6 5 5 c0t0d0 123.5 2.5 6035.9 56.0 0.5 0.1 4.1 0.7 8 8 c0t1d0 0.0 55.0 0.0 340.3 0.5 0.0 9.5 0.3 13 2 c1t0d0 0.0 49.5 0.0 315.5 0.2 0.0 3.8 0.3 7 1 c1t1d0 0.0 6.5 0.0 694.1 0.4 5.2 55.0 803.6 11 89 c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 0.0 49.5 0.0 6086.9 0.0 0.3 0.0 6.0 0 3 c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 6.0 0.0 180.0 0.0 0.0 0.0 0.0 7.8 0 2 c3t5000C50019D09E3Ad0 10.0 0.0 352.8 0.0 0.0 0.1 0.0 8.6 0 4 c3t5000C50019D08DADd0 1.5 0.0 3.8 0.0 0.0 0.1 0.0 35.4 0 3 c3t5000C50015A3038Dd0 9.5 0.0 504.3 0.0 0.0 0.1 0.0 10.9 0 4 c3t5000C5001A108945d0 9.5 0.0 224.0 0.0 0.0 0.1 0.0 11.4 0 4 c3t5000C50019D061B8d0 4.0 0.0 59.8 0.0 0.0 0.1 0.0 26.8 0 4 c3t5000C50015BAD93Dd0 7.5 0.0 125.8 0.0 0.0 0.1 0.0 11.3 0 3 c3t5000C50019D09F37d0 cpu us sy wt id 5 17 0 78 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 47.5 49.0 770.7 5772.0 0.6 0.1 5.9 0.9 8 9 c0t0d0 110.0 42.5 1961.7 4903.7 0.6 0.1 3.9 0.7 9 10 c0t1d0 0.0 34.5 0.0 154.0 0.0 0.0 0.0 0.2 0 1 c1t0d0 0.0 34.5 0.0 154.0 0.0 0.0 0.0 0.2 0 1 c1t1d0 0.0 5.0 0.0 622.0 1.0 8.9 208.9 1771.2 11 89 c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 12.5 0.0 175.0 0.0 0.0 0.1 0.0 11.3 0 8 c3t5000C50019D09E3Ad0 12.0 0.0 224.2 0.0 0.0 0.1 0.0 11.1 0 10 c3t5000C50019D08DADd0 18.5 0.0 186.5 0.0 0.0 0.2 0.0 10.9 0 13 c3t5000C50015A3038Dd0 13.5 0.0 476.7 0.0 0.0 0.3 0.0 20.5 0 10 c3t5000C50019D05DD1d0 20.0 0.0 262.7 0.0 0.0 0.4 0.0 17.6 0 14 c3t5000C5001A108945d0 13.0 0.0 210.5 0.0 0.0 0.2 0.0 13.2 0 9 c3t5000C50019D061B8d0 7.5 0.0 62.2 0.0 0.0 0.1 0.0 10.9 0 6 c3t5000C50015BAD93Dd0 17.0 0.0 135.0 0.0 0.0 0.2 0.0 14.3 0 13 c3t5000C50019D09F37d0 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# fg aksh ^Cerror: interrupted by user mccfs4-bui:maintenance chassis-001 disk>[0  mccfs4-bui:maintenance chassis-001 disk>  mccfs4-bui:maintenance chassis-001 disk>  mccfs4-bui:maintenance chassis-001 disk>  [1]+ Stopped aksh mccfs4-bui# mccfs4-bui# mccfs4-bui# fgiostat -xczn 2 4 cpu us sy wt id 0 6 0 94 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 7.7 7.0 101.0 653.4 0.1 0.0 5.5 0.8 1 1 c0t0d0 7.9 7.0 102.7 654.2 0.1 0.0 5.4 0.8 1 1 c0t1d0 0.7 25.0 35.5 484.0 0.2 0.0 7.1 1.2 3 3 c1t0d0 0.4 25.0 25.8 484.0 0.2 0.0 6.8 1.1 3 3 c1t1d0 0.0 0.0 0.0 0.2 0.0 0.0 20.7 164.2 0 0 c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 0.0 307.5 0.0 8897.0 0.0 0.3 0.0 1.0 0 6 c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.8 0 0 c3t5000C50019D0727Fd0 7.1 31.7 119.3 2087.3 0.0 0.4 0.0 10.0 0 7 c3t5000C50019D09E3Ad0 7.1 31.7 119.5 2087.3 0.0 0.4 0.0 9.9 0 7 c3t5000C50019D08DADd0 7.1 31.8 119.0 2087.9 0.0 0.4 0.0 9.9 0 7 c3t5000C50015A3038Dd0 7.1 31.8 119.6 2087.2 0.0 0.4 0.0 10.0 0 7 c3t5000C50019D05DD1d0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.8 0 0 c3t5000C500150DC4EFd0 10.5 31.7 122.8 2087.9 0.0 0.4 0.0 9.4 0 7 c3t5000C5001A108945d0 7.1 31.5 119.3 2087.3 0.0 0.4 0.0 10.1 0 6 c3t5000C50019D061B8d0 7.1 31.8 119.5 2087.2 0.0 0.4 0.0 10.0 0 7 c3t5000C50015BAD93Dd0 10.4 31.5 122.9 2087.3 0.0 0.4 0.0 9.4 0 7 c3t5000C50019D09F37d0 cpu us sy wt id 6 20 0 74 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.0 77.6 0.0 1335.6 0.9 0.0 11.3 0.3 22 3 c1t0d0 0.0 79.6 0.0 1335.6 0.8 0.0 9.5 0.3 22 3 c1t1d0 0.0 5.0 0.0 557.3 1.0 8.9 208.0 1781.8 11 89 c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 0.0 0.5 0.0 61.7 0.0 0.0 0.0 0.6 0 0 c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 0.5 0.0 1.0 0.0 0.0 0.0 0.0 8.6 0 0 c3t5000C50019D061B8d0 0.5 0.0 0.5 0.0 0.0 0.0 0.0 11.3 0 1 c3t5000C50015BAD93Dd0 cpu us sy wt id 5 14 0 80 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 1.0 0.0 16.0 0.0 0.0 0.0 0.0 0.4 0 0 c0t1d0 0.0 51.0 0.0 236.0 0.0 0.0 0.0 0.2 0 1 c1t0d0 0.0 51.0 0.0 236.0 0.0 0.0 0.0 0.2 0 1 c1t1d0 0.0 8.0 0.0 992.1 2.1 7.7 259.9 965.2 23 77 c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 cpu us sy wt id 5 16 0 80 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.0 61.0 0.0 1150.0 0.1 0.0 2.1 0.7 4 4 c1t0d0 0.0 61.0 0.0 1150.0 0.1 0.0 2.1 0.7 4 4 c1t1d0 0.0 1.0 0.0 124.0 1.0 8.9 1039.3 8862.0 11 89 c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 0.5 0.0 0.5 0.0 0.0 0.0 0.0 7.1 0 0 c3t5000C50019D09E3Ad0 0.5 0.0 0.5 0.0 0.0 0.0 0.0 0.5 0 0 c3t5000C50019D05DD1d0 15.0 0.0 15.0 0.0 0.0 0.0 0.0 0.4 0 1 c3t5000C5001A108945d0 9.5 0.0 9.5 0.0 0.0 0.0 0.0 1.1 0 1 c3t5000C50019D09F37d0 mccfs4-bui# zpool status & [2] 9361 pool: mccsp state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. mccfs4-bui# scan: resilvered 3.98G in 0h2m with 0 errors on Thu Jan 24 19:13:10 2013 config: NAME STATE READ WRITE CKSUM mccsp ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t5000C5001A108945d0 ONLINE 0 0 0 c3t5000C50015A3038Dd0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c3t5000C50015BAD93Dd0 ONLINE 0 0 0 c3t5000C50019D05DD1d0 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 c3t5000C50019D08DADd0 ONLINE 0 0 0 c3t5000C50019D09E3Ad0 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 c3t5000C50019D09F37d0 ONLINE 0 0 0 c3t5000C50019D061B8d0 ONLINE 0 0 0 logs c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 ONLINE 0 0 0 c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 ONLINE 0 0 0 cache c0t0d0 ONLINE 0 0 0 c0t1d0 ONLINE 0 0 0 spares c3t5000C50019D0727Fd0 AVAIL c3t5000C500150DC4EFd0 AVAIL errors: No known data errors pool: system state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scan: none requested config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t1d0s0 ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 errors: No known data errors [2]- Done zpool status mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# fg aksh mccfs4-bui:mantenance chassis-001 disk> cd / mccfs4-bui:> maintenance problems mccfs4-bui:maintenance problems>[0 lsit error: invalid command "lsit" mccfs4-bui:mantenance problems> list COMPONENT DIAGNOSED TYPE DESCRIPTION  mccfs4-bui:mantenance problems>  mccfs4-bui:maintenance problems>  mccfs4-bui:maintenance problems>  [1]+ Stopped aksh mccfs4-bui# fg aksh mccfs4-bui:mantenance problems> cd / mccfs4-bui:> status activity mccfs4-bui:status activity> show Activity:  CPU 12 %util Sunny FTP 0 bytes/sec Sunny Disk 482 ops/sec Partly Cloudy iSCSI 0 ops/sec Sunny NFSv3 279 ops/sec Sunny NFSv4 0 ops/sec Sunny Network 6.33M bytes/sec Sunny SMB 0 ops/sec Sunny mccfs4-bui:sttus activity>  mccfs4-bui:status activity>  mccfs4-bui:status activity>  mccfs4-bui:status activity> show Activity:  CPU 12 %util Sunny FTP 0 bytes/sec Sunny Disk 478 ops/sec Sunny iSCSI 0 ops/sec Sunny NFSv3 278 ops/sec Sunny NFSv4 0 ops/sec Sunny Network 6.25M bytes/sec Sunny SMB 0 ops/sec Sunny mccfs4-bui:sttus activity>  mccfs4-bui:status activity>  mccfs4-bui:status activity>  mccfs4-bui:status activity> cd / mccfs4-bui:>  [1]+ Stopped aksh mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# pwd /var/ak/stash/com/sun/ak/xmlrpc/disk mccfs4-bui# ls 1032QAK00F-0 1032QAK00F-16 1032QAK00F-23 1035QAP001-0 1035QAP002-2 1032QAK00F-1 1032QAK00F-17 1032QAK00F-3 1035QAP001-1 1035QAP002-3 1032QAK00F-10 1032QAK00F-18 1032QAK00F-4 1035QAP001-2 1035QAP002-4 1032QAK00F-11 1032QAK00F-19 1032QAK00F-5 1035QAP001-3 1035QAP002-5 1032QAK00F-12 1032QAK00F-2 1032QAK00F-6 1035QAP001-4 1032QAK00F-13 1032QAK00F-20 1032QAK00F-7 1035QAP001-5 1032QAK00F-14 1032QAK00F-21 1032QAK00F-8 1035QAP002-0 1032QAK00F-15 1032QAK00F-22 1032QAK00F-9 1035QAP002-1 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog       !aklog aklog rm | egrep -v 'lib/|LOCK|^$' | tail Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-14 failed in 0.000s: not enumerated Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-15 failed in 0.000s: not enumerated Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-16 failed in 0.000s: not enumerated Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-17 failed in 0.000s: not enumerated Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-18 failed in 0.000s: not enumerated Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-19 failed in 0.000s: not enumerated Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-20 failed in 0.000s: not enumerated Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-21 failed in 0.000s: not enumerated Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-22 failed in 0.000s: not enumerated Wed Feb 13 17:33:25 2013: import of ak:/disk/1032QAK00F-23 failed in 0.000s: not enumerated mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# aklog akd | tail Wed Feb 13 17:23:20 2013: rx_join: waiting on rejoin Wed Feb 13 17:23:21 2013: rx_join: waiting on rejoin Wed Feb 13 17:23:22 2013: rx_join: waiting on rejoin Wed Feb 13 17:23:23 2013: rx_join: waiting on rejoin Wed Feb 13 17:23:25 2013: rx_join: waiting on rejoin Wed Feb 13 17:23:26 2013: rx_join: waiting on rejoin Wed Feb 13 17:23:27 2013: rx_join: waiting on rejoin Wed Feb 13 17:23:27 2013: initial cluster state AKCS_CLUSTERED Wed Feb 13 17:23:28 2013: rx_join: done, state is AKCS_CLUSTERED Wed Feb 13 17:23:36 2013: mccsp: successfully issued LIP to 0 target ports mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# svcs -xv mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# fg aksh mccfs4-bui:>0m [0 mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:> zpcofngnfiguration storage mccfs4-bui:cofiguration storage>[0 show Properties: ^Z [1]+ Stopped aksh mccfs4-bui# jobs [1]+ Stopped aksh mccfs4-bui# mdb -k Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs sata sd emlxs qlc ip hook neti arp usba stmf stmf_sbd sockfs lofs random md mpt crypto idm smbsrv nfs cpc ufs logindmux ptm nsmb mpt_sas pmcs ] > ::spa ADDR STATE NAME  fffff6000daff080 ACTIVE mccsp fffff60005581580 ACTIVE system > fffff6000daff080::ps  spa -v ADDR STATE NAME  fffff6000daff080 ACTIVE mccsp ADDR STATE AUX DESCRIPTION  fffff600397466c0 HEALTHY - root fffff600397459c0 HEALTHY - mirror fffff60039746080 HEALTHY - /dev/dsk/c3t5000C5001A108945d0s0 fffff60039747980 HEALTHY - /dev/dsk/c3t5000C50015A3038Dd0s0 fffff60039746d00 HEALTHY - mirror fffff60039747340 HEALTHY - /dev/dsk/c3t5000C50015BAD93Dd0s0 fffff6001cd41cc0 HEALTHY - /dev/dsk/c3t5000C50019D05DD1d0s0 fffff6001cd41040 HEALTHY - mirror fffff6001cd41680 HEALTHY - /dev/dsk/c3t5000C50019D08DADd0s0 fffff60039742c80 HEALTHY - /dev/dsk/c3t5000C50019D09E3Ad0s0 fffff60039742640 HEALTHY - mirror fffff60039742000 HEALTHY - /dev/dsk/c3t5000C50019D09F37d0s0 fffff60039739940 HEALTHY - /dev/dsk/c3t5000C50019D061B8d0s0 fffff603a6371900 HEALTHY - /dev/dsk/c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0s0 fffff60039738cc0 HEALTHY - /dev/dsk/c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0s0 - - - cache fffff60039737980 HEALTHY - /dev/dsk/c0t0d0s0 >> More [, , q, n, c, a] ?   fffff60039737340 HEALTHY - /dev/dsk/c0t1d0s0 - - - spares fffff6003a0530c0 HEALTHY - /dev/dsk/c3t5000C50019D0727Fd0s0 fffff60039738680 HEALTHY - /dev/dsk/c3t5000C500150DC4EFd0s0 >  [2]+ Stopped mdb -k mccfs4-bui# fg 1 aksh pool = mccsp status = online owner = mccfs4-bui profile = mirror log_profile = log_stripe cache_profile = cache_stripe scrub = resilver completed after 0h2m with 0 errors mccfs4-bui:configuration storage>  [1]+ Stopped aksh mccfs4-bui# zpool iostat -v mcsp  csp 4 ^Z [3]+ Stopped zpool iostat -v mccsp 4 mccfs4-bui# mccfs4-bui# mccfs4-bui# bg 3 [3]+ zpool iostat -v mccsp 4 & capacity operations bandwidth mccfs4-bui# pool alloc free read write read write -------------------------------------- ----- ----- ----- ----- ----- ----- mccsp 991G 2.66T 146 1.39K 939K 16.8M mirror 248G 680G 38 280 236K 2.04M c3t5000C5001A108945d0 - - 10 31 123K 2.04M c3t5000C50015A3038Dd0 - - 7 31 119K 2.04M mirror 248G 680G 34 279 233K 2.04M c3t5000C50015BAD93Dd0 - - 7 31 119K 2.04M c3t5000C50019D05DD1d0 - - 7 31 120K 2.04M mirror 248G 680G 34 280 233K 2.04M c3t5000C50019D08DADd0 - - 7 31 119K 2.04M c3t5000C50019D09E3Ad0 - - 7 31 119K 2.04M mirror 248G 680G 38 280 236K 2.04M c3t5000C50019D09F37d0 - - 10 31 123K 2.04M c3t5000C50019D061B8d0 - - 7 31 119K 2.04M c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 53.1M 16.8G 0 10 282 893K c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 53.1M 16.8G 0 307 0 8.69M cache - - - - - - c0t0d0 93.1G 8M 7 6 101K 654K c0t1d0 93.1G 8.00M 7 6 103K 654K -------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write -------------------------------------- ----- ----- ----- ----- ----- ----- mccsp 991G 2.66T 8 143 10.6K 2.13M mirror 248G 680G 3 32 3.96K 221K c3t5000C5001A108945d0 - - 3 3 3.65K 221K c3t5000C50015A3038Dd0 - - 0 3 326 221K mirror 248G 680G 0 32 752 217K c3t5000C50015BAD93Dd0 - - 0 3 565 217K c3t5000C50019D05DD1d0 - - 0 3 186 217K mirror 248G 680G 0 32 1.35K 221K c3t5000C50019D08DADd0 - - 0 4 626 221K c3t5000C50019D09E3Ad0 - - 0 4 752 221K mirror 248G 680G 3 32 4.55K 220K c3t5000C50019D09F37d0 - - 3 3 4.25K 220K c3t5000C50019D061B8d0 - - 0 3 311 220K c3tATASTECZEUSIOS018GBYTESSTM0000D5B58d0 49.6M 16.8G 0 6 0 643K c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 49.7M 16.8G 0 6 0 657K cache - - - - - - c0t0d0 93.1G 8M 3 1 20.2K 140K c0t1d0 93.1G 7.09M 1 1 8.09K 131K -------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write -------------------------------------- ----- ----- ----- ----- ----- ----- mccsp 991G 2.66T 7 590 11.3K 4.95M mirror 248G 680G 3 151 4.17K 1.02M c3t5000C5001A108945d0 - - 3 18 4.17K 1.02M c3t5000C50015A3038Dd0 - - 0 17 0 1.02M mirror 248G 680G 0 143 88 1.01M c3t5000C50015BAD93Dd0 - - 0 14 0 1.01M c3t5000C50019D05DD1d0 - - 0 14 88 1.01M mirror 248G 680G 0 141 4.05K 1.03M c3t5000C50019D08DADd0 - - 0 17 2.45K 1.03M c3t5000C50019D09E3Ad0 - - 0 17 1.59K 1.03M mirror 248G 680G 2 147 3.03K 1.02M c3t5000C50019D09F37d0 - - 2 17 2.95K 1.02M c3t5000C50019D061B8d0 - - 0 16 88 1.02M mccfs4-bui# mccfs4-bui# jobs [1]+ Stopped aksh [2] Stopped mdb -k [3]- Running zpool iostat -v mccsp 4 & mccfs4-bui# kill % c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 43.1M 16.8G 0 5 0 609K c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 43.1M 16.8G 0 2 0 276K cache - - - - - - c0t0d0 93.1G 7.99M 4 1 25.1K 65.0K c0t1d0 93.1G 8M 1 1 7.49K 156K -------------------------------------- ----- ----- ----- ----- ----- ----- 4 -bash: kill: %4: no such job mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# jobs [1]+ Stopped aksh [2] Stopped mdb -k [3]- Running zpool iostat -v mccsp 4 & mccfs4-bui# kil %3 -bash: kil: commad not found mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# capacity operations bandwidth pool alloc free read write read write -------------------------------------- ----- ----- ----- ----- ----- ----- mccsp 991G 2.66T 10 223 28.8K 3.10M mirror 248G 680G 3 49 7.69K 356K c3t5000C5001A108945d0 - - 3 7 4.47K 356K c3t5000C50015A3038Dd0 - - 0 7 3.22K 356K mirror 248G 680G 0 49 3.03K 352K c3t5000C50015BAD93Dd0 - - 0 5 1.21K 352K c3t5000C50019D05DD1d0 - - 0 5 1.82K 352K mirror 248G 680G 0 52 4.42K 363K c3t5000C50019D08DADd0 - - 0 6 3.03K 363K c3t5000C50019D09E3Ad0 - - 0 6 1.59K 363K mirror 248G 680G 4 51 13.6K 363K c3t5000C50019D09F37d0 - - 3 7 8.58K 363K c3t5000C50019D061B8d0 - - 0 7 6.44K 363K c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 27.8M 16.8G 0 9 0 864K c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 27.8M 16.8G 0 9 0 874K cache - - - - - - c0t0d0 93.1G 8M 3 0 21.3K 15.2K c0t1d0 93.1G 8M 2 0 13.2K 25.7K -------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write -------------------------------------- ----- ----- ----- ----- ----- ----- mccsp 991G 2.66T 9 277 15.5K 3.59M mirror 248G 680G 3 65 5.19K 436K c3t5000C5001A108945d0 - - 3 5 4.91K 436K c3t5000C50015A3038Dd0 - - 0 5 280 436K mirror 248G 680G 0 65 3.11K 441K c3t5000C50015BAD93Dd0 - - 0 9 2.26K 441K c3t5000C50019D05DD1d0 - - 0 8 870 441K mirror 248G 680G 0 63 2.18K 431K c3t5000C50019D08DADd0 - - 0 8 1.60K 431K c3t5000C50019D09E3Ad0 - - 0 9 590 431K mirror 248G 680G 4 67 4.99K 440K c3t5000C50019D09F37d0 - - 3 8 4.41K 440K c3t5000C50019D061B8d0 - - 0 8 590 440K c3tATASTECZEUSIOPS018GBYTESSTM0000D5B58d0 36.4M 16.8G 0 6 0 771K c3tATASTECZEUSIOPS018GBYTESSTM000107DD0d0 36.6M 16.8G 0 9 0 1.14M cache - - - - - - c0t0d0 93.1G 8M 1 0 9.20K 3.13K c0t1d0 93.1G 8M 0 0 5.02K 1.74K -------------------------------------- ----- ----- ----- ----- ----- ----- mccfs4-bui# kill %3 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# [3]- Terminated zpool iostat -v mccsp 4 mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# mccfs4-bui# jobs [1]+ Stopped aksh [2] Stopped mdb -k mccfs4-bui# mccfs4-bui# mccfs4-bui# fg 2 mdb -k > $q  ::mpt -l v   v lv!tail 31825: 2013 Feb 13 17:41:44 0 024 0x3afbfe65 mpt_sas_probe_lun from mpt_config_luns ret=0 lun=0 31826: 2013 Feb 13 17:41:44 0 024 0x3afbfe65 mpt_config_luns rval=0 31827: 2013 Feb 13 17:41:44 0 024 0x3afbfe65 mpt_check_scsi_io_error scsi_status=0x2, ioc_status=0x0, ioc_loginfo=0x0, scsi_state=0x1, command=0x2a 'write(10)' 31828: 2013 Feb 13 17:41:44 0 024 0x3afbfe65 check_condition: sense_key=0x6 (Unit_Attention) asc/ascq=0x29/0x0 (power on, reset, or bus reset occurred) command=0x2a (write(10)) 31829: 2013 Feb 13 17:41:44 0 --- 0x3afbfe65 !mpt_handle_event_sync: IOCStatus=0x8000, IOCLogInfo=0x31120403, event=0xf 31830: 2013 Feb 13 17:41:44 0 024 0x3afbfe65 MPI_EVENT_SAS_DEVICE_STATUS_CHANGE wwn is 5001636000281450 rc=0x8 devinfo=0x81 31831: 2013 Feb 13 17:41:44 0 024 0x3afbfe65 EVENT 0xf: rc=0x8 (INTERNAL DEVICE RESET) wwn=5001636000281450 31832: 2013 Feb 13 17:41:44 0 --- 0x3afbfe65 send mpt_handle_event_sync success 31833: 2013 Feb 13 17:41:44 0 --- 0x3afbfe65 !mpt_handle_event: IOCStatus=0x8000, IOCLogInfo=0x31120403, event=0xf 31834: 2013 Feb 13 17:41:44 0 --- 0x3afbfe65 sas device change ack. > > > > > > !dat -l: dat: command nt found > !date Wed Feb 13 17:42:02 UTC 2013 > > > > ::mpt -l!tail 2013 Feb 13 17:42:0 0 --- sas discovery ack. 2013 Feb 13 17:42:10 0 024 report_luns(2)=0 retry=1 2013 Feb 13 17:42:10 0 024 mpt_scsi_hba_probe=0, 418 times, lun=0 2013 Feb 13 17:42:10 0 024 mpt_sas_create_lun: entry lun=0 2013 Feb 13 17:42:10 0 024 guid='ATASTECZeusIOPS018GBytesSTM0000D5B58' 2013 Feb 13 17:42:10 0 024 mpt_create_virt_lun=0 dtype=0 2013 Feb 13 17:42:10 0 024 mpt_sas_probe_lun from mpt_config_luns ret=0 lun=0 2013 Feb 13 17:42:10 0 024 mpt_config_luns rval=0 2013 Feb 13 17:42:10 0 024 mpt_check_scsi_io_error scsi_status=0x2, ioc_status=0x0, ioc_loginfo=0x0, scsi_state=0x1, command=0x2a 'write(10)' 2013 Feb 13 17:42:10 0 024 check_condition: sense_key=0x6 (Unit_Attention) asc/ascq=0x29/0x0 (power on, reset, or bus reset occurred) command=0x2a (write(10)) > > > > > $q mccfs4-bui# exit logout There are stopped jobs. mccfs4-bui# fg aksh mccfs4-bui:configuration storage>[ cd / mccfs4-bui:> exit mccfs4-bui# exit logout mccfs4-bui:> [ mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:> status activity show Activity:  CPU 18 %util Sunny FTP 0 bytes/sec Sunny Disk 215 ops/sec Partly Cloudy iSCSI 0 ops/sec Sunny NFSv3 221 ops/sec Sunny NFSv4 0 ops/sec Sunny Network 2.30M bytes/sec Sunny SMB 0 ops/sec Sunny mccfs4-bui:>0m  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>  mccfs4-bui:>