STI connections to I/O cages, STI-3 extender cards, and ICB-4 links are driven from the Memory Bus Adapters (MBAs) fanout cards that are located on the front of the book. Figure 2-6 shows the location of the Self-Timed Interconnect (STI) connectors and the MBA fanout cards.

Each MBA fanout card connects to up to two STI cables. There are up to 8 MBA fanout cards (numbered D1 to D8) per book, each driving two STIs, resulting in 16 STIs per book. An MBA fanout card can be repaired concurrently with the use of Redundant I/O Interconnect. See “Redundant I/O Interconnect” on page 36.

All 16 STIs in a book have a data rate of 2.7 GB per second. Depending on the channel types installed, a maximum of 1024 channels per server is supported.

Two STIs are related to an MBA fanout card. When configuring for availability, channels, links, and OSAs across books, MBAs, and STIs should be balanced. In a system configured for maximum availability, alternate paths will maintain access to critical I/O devices, such as disks, networks, etc.

Enhanced Book Availability allows a single book in a multi book server to be concurrently removed and reinstalled for an upgrade or a repair. Removing a book would also mean that the connectivity to the I/O connected to that book is lost. To prevent connectivity loss the z9 EC Redundant I/O Interconnect feature allows to maintain full connection to critical devices (except ICBs) when a book is removed.

Redundant I/O Interconnect is accomplished by the facilities of the Self-Timed Interconnect Multiplexer (STI-MP) card. Each STI-MP card is connected to an STI jack located in the MBA fanout card of a book. STI-MP cards are half-high cards and are interconnected with cards called STI-A8 and STI-A4, allowing Redundant I/O Interconnect in case the STI connection coming from a book ceases to function, as is the case when, for example, a book is removed.  A conceptual view of how Redundant I/O Interconnect is accomplished is shown in

Figure 2-7.

Normally book 0 MBA/STI connects to the STI-MP (A) card and services domain 0 I/O connections (slots 01, 03, 06, and 08). In the same fashion book 1 MBA/STI connects to the STI-MP (B) card and services domain 1 (slots 02, 04, 07, and 09). If book 1 is removed, or the connections from book 1 to the cage are removed, connectivity to domain 1 is maintained by guiding the I/O to domain 1 through the interconnect between STI-MP (A) and STI-MP (B).  In configuration reports, books are numbered 0, 1, 2, and 3; MBAs are numbered from D1 to D8; the STIs are identified as jacks numbered J00 and J01 for each MBA.

With Enhanced Book Availability the impact of book replacement is minimized. In a multi book system a single book can be concurrently removed and reinstalled for an upgrade or repair.  To be able to do so requires sufficient resources in the remaining books to remove a book without impacting the workload. CPs and memory from the book must be relocated before the book can be removed. Not only additional PUs need to be available on the remaining books to replace the deactivated book, but also sufficient redundant memory must be available if it is

required that no degradation of applications is allowed. The “Flexible Memory option” on page 33, should be considered to ensure that the server configuration supports removal of a book with minimal impact to the workload. Any book can be replaced, including book 0, that initially contains the HSA.  Removal of a book also removes the book connectivity to the I/O cages. The impact of the removal of the book on the system is limited by the use of Redundant I/O Interconnect as described in “Redundant I/O Interconnect” on page 36; however, all ICBs on the removed book have to be configured offline.

When Enhanced Book Availability and the Flexible Memory option are not used and when a book must be replaced, for example due to an upgrade or a repair action, the memory in the failing book is removed as well. Until the removed book is replaced, a Power-on Reset of the server with the remaining books is supported.

All MBA fanout cards used for I/O are concurrently rebalanced as part of the book addition.  However, to have MBA fanouts used for ICBs rebalanced, the STI rebalance feature (FC 2400) needs to be ordered. The STI rebalance feature is disruptive but is highly recommended when upgrading from a S08 to a larger model. When going from a multi-book to a larger model, individual evaluation is also recommended. See “STI Rebalance feature” on page 93 for more information.