Hardware •
Quadrics QsNetI - HPC interconnect based on the
elan3/elite3 ASICs (350MB/s @ 5us MPI latency) •
Quadrics QsNetII - HPC interconnect based on the
elan4/elite4 ASICs (912MB/s on SR1400 EM64T and 1.26us MPI latency on HP DL145G2) • QsTenG - 10 Gigabit Ethernet switches, from 24-port (1U) to very large switches. • QsNet III - HPC interconnect based on the
elan5/elite5 ASICs (approximately 2GB/s each direction and 1.3us MPI latency). This is the first product from Quadrics that is compatible with a standard - in this case 10Gbit Ethernet.
QsNet QsNet was a high speed interconnect designed by Quadrics used in
high-performance computing computer clusters, particularly Linux
Beowulf clusters. Although it can be used with TCP/IP; like
SCI,
Myrinet and
InfiniBand it is usually used with a communication
API such as
Message Passing Interface (MPI) or
SHMEM called from a
parallel program. The interconnect consists of a PCI card in each compute node and one or more dedicated switch chassis. These are connected with a copper cables. Within the switch chassis are a number of line cards that carry
Elite switch
ASICs. These are internally linked to form a
fat tree topology. Like other interconnects such as
Myrinet very large systems can be built by using multiple switch chassis arranged as spine (top-level) and leaf (node-level) switches. Such systems were called "federated networks". It was announced in 1998 and used PCI 66-64 cards that had 'elan3' Custom ASIC on them. These gave an MPI bandwidth of around 350MB/s unidirectional with 5us latency.
QsNet II QsNet II was the fourth and penultimate generation of Quadrics
interconnect family products, and was launched in 2003. QsNetII interfaced to the host computer through the standard IO
PCI-X bus. Later versions of the card had PCIe physical interfaces although this was bridged on the card to PCI-X with a performance penalty. A native PCIe version was never developed. Instead resource was focused on QsNetII's successor QsNetIII which although completed was never released commercially. The architecture of the network interface has been developed to offload the entire task of interprocessor communication from the main processor, and to avoid the overhead of system calls for user process to user process messaging. QsNetII is designed for use within
SMP systems — multiple, concurrent processes can utilise the network interface without any task switching overhead. A I/O processor offloads protocol handling from the main CPU. Local memory on the PCI card provides storage for buffers, translation tables and I/O adapter code. All the PCI bandwidth is available to data communication. QsNetII's core design is based on two ASICs: Elan4 and Elite4. Elan4 is a communication processor that forms the interface between a high-performance multistage network and a processing node with one or more CPUs. Elite4 is a switching component that can switch eight bidirectional communications links, each of which carrying data in both directions simultaneously at 1.3GB/s. Quadrics QsNetII interconnect like its predecessor
QsNet uses a 'fat tree' topology, QsNetII scales up to 4096 nodes, each node might have multiple CPUs so that systems of >10,000 CPUs can be constructed. Multiple, parallel QsNet networks can be employed in a system to maintain the compute to communications ratio where high CPU count SMP nodes are employed. The fat tree topology is resilient with large amounts of redundancy in the higher levels of the switch. Performance depends on platform used and configuration of the system, QsNetII MPI latency on standard AMD
Opteron starts at 1.22μs; Bandwidth on Intel Xeon
Intel 64 is 912MB/s. In 2004, Quadrics started releasing small to medium switch stand-alone switch configurations called QsNetII E-Series, these configurations range from the 8 to the 128-way systems.
QsTenG In November 2005, Quadrics announced a new product based on
10 Gigabit Ethernet (10 GigE), called QsTenG. The first QsTenG switch was an 8U chassis with 12 slots for 10 GigE line cards, making 96 ports in total. Each line card had eight 10 GigE ports that connect using
10GBASE-CX4 connectors. Each line card also had four internal ports that connected the line cards together into a fat tree configuration. Since then, Quadrics brought out a second generation of 10 GigE switches, starting with a compact 1U switch with 24 ports, which comes in two variants, TG201-CA, 24 ports CX4, and TG201-XA, 24 ports in total, 12 XSP and 12 CX4. They were expected to bring out a range of larger switches in 2009, the chassis was planned to be the same as the
QsNetIII, the switch to have been called TG215. Late in 2007, the Quadrics management decided to cancel the QsTenG Ethernet developments and concentrate efforts on the
QsNet product line. This caused a group employees to leave and help found
Gnodal, to develop large scalable Ethernet systems.
Software Software included a cluster resource manager software package called QuadricsRms, and Quadrics
Linux Software, core components of the QsNet software release for Linux under the GNU LGPL License ==See also==