ndoo.sg

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Inspur Hotswap Cages ======

Taobao has many 2U hot-swap cages for sale with SAS expanders and daisy-chain capabilities. These are disassembled from 2U Inspur servers disposed by hyperscalers' reverse supply chains.

A cottage industry of accessories and enclosures for these hot-swap cages has also sprung up around them.

===== Accessories =====

FIXME TODO

===== SA5212M4 =====

  * Bays: 12 × 3.5″ SAS3/SATA3
  * Ports: 3 × SAS3 - Mini-SAS HD (SFF-8643) connectors
  * Tray P/N: RM13919-01D (OEM: Chenbro)
  * Expander chip: PMC Sierra PM8388 SXP 24x3G (24 ports - 12 connected to rear ports, 12 connected to drives)

==== Usage ====

  * Daisy chain mode
    * Use any 1 × Mini-SAS HD connector as 4 × SAS wide link to HBA
    * Use any 1 × Mini-SAS HD connector as 4 × SAS wide link to downstream cages' expanders
  * Passthrough mode
    * Connect all 3 × Mini-SAS HD connectors to HBA(s), all disks are passed through 1:1
    * If only (any) 2 × Mini-SAS HD connectors are connected to HBA(s), 4 × drive trays will be non-functional i.e. there is no support for 8:12 HBA:drive SAS links

==== PHY Mapping ====

Front view of hotswap cage:

|  phy12  |  phy13  |  phy14  |  phy15  |
|  phy16  |  phy17  |  phy18  |  phy19  |
|  phy20  |  phy21  |  phy22  |  phy23  |

phy0-phy11 are mapped to the mini-SAS HD connectors.
==== LED Control ====

LEDs can be controlled by ''ledctl'', e.g.:

  * ''ledctl normal=/dev/disk/by-path/pci-0000\:01\:00.0-sas-exp0xdeadbeefdeadbeef-phy12-lun-0''
  * ''ledctl normal=/dev/sda''

Supported patterns:

  * ''normal''/''off'': Status LED off
  * ''locate'': Blue status LED
  * ''rebuild'': Purple status LED
  * ''failure'': Red status LED
  * All other patterns cause LED color(s) to get stuck until valid patterns are used to set and clear stuck LED color(s)

===== Useful Utilities =====

  * sasutils
    * ''sas_discover -vv'' - SAS devices, topology (tree) view
    * ''sas_devices [-v...]'' - SAS devices, list view
  * ''ledctl''
    * OpenZFS' ''zed'' [[https://github.com/openzfs/zfs/blob/master/cmd/zed/zed.d/statechange-led.sh|automatically indicates vdev failures on supported enclosures' drive failure LED(s)]], if failed drives are replaced into other enclosures/bays, the failure LED may get stuck; ''ledctl'' can be used to clear the failure indicator
  * udisks2
    * ''%%find /dev/disk/by-path -name "*sas-exp0x56c92bf*lun-0" -exec sudo udisksctl power-off -b {} ';'%%'' - spindown all disks in all expander backplanes
ndoo.sg

User Tools

Site Tools

Page Tools
