HPC2N
Sarek - HPC2N Opteron Cluster

During the spring of 2004 HPC2N bought a new large cluster based on the 64-bit AMD Opteron CPU from HP. The acquisition was made possible through a generous donation from the Knut and Alice Wallenberg Foundation.

The cluster consists of 384 CPUs in 192 dual nodes connected with a Myrinet-2000 High-performance interconnect. The cluster achived 1.33 Tflops/s on HP Linpack benchmark, the highest recorded in Sweden to date. In addition to offer the highest perfomance, the cluster also boasts a total of over 1.5 Terabyte of memory. All in all this makes it the most powerful computer in Sweden.

The AMD Opteron is a 64-bit CPU with excellent performance, especially in dual and multiprocessor configurations. The CPUs in the new HPC2N cluster runs at 2.2 GHz giving a peak performance of 4.4 Gflops/s for each CPU, giving a total of 1.69 Tflops/s peak performance.

Of the 192 nodes in the cluster 190 are identically configured compute nodes based on the HP Proliant DL145 server. Each compute node includes 2 AMD 248 Opteron CPUs running at 2.2 GHz, 8 Gigabyte of RAM and 80 GB hardisk. Each node also connects to the Gigabit ethernet communication network and to the Myrinet high-performance network.

The remaining two nodes in the cluster serve as access-server and management server respectively. Their configuration is similar to the compute nodes, but based instead on the HP proliant DL585 server.

The high-performance network is a Myrinet-2000 network connecting all 192 nodes in the cluster. The network is switched which gives similar performance between each pair of nodes. The performance offered is a bandwidth of about 250 MB/s and a latency of a few microseconds. Each of the nodes are connected with a PCI-X adapter card, taking full advantage of the speed of the PCI-X bus.

In order to take advantage of the superior performance offered by the 64-bit CPUs, the software needs to fully support the 64-bit architecture, though old 32-bit software will probably still work. The system runs Ubuntu 6.06 LTS. In addition to gcc, development tools from the Portland Group with full 64-bit support ensures that user can take full advantage of the 64-bit architecture.

Further information

The software installation was done using FAI, Fully Automatic Installation.

If you have any questions about the new cluster please send a mail to either info@hpc2n.umu.se (non technical questions) or support@hpc2n.umu.se (technical questions)

Facts

Hardware

  • 384 CPUs
  • 192 Nodes
  • 1.54 Terabyte RAM
  • 150 Terabyte disk
  • 1.69 Tflops/s peak performance
  • 1.33 Tflops/s HP Linpack
380 CPU's in 190 Dual nodes
  • HP DL 145
  • 2 AMD Opteron 248 Processors
  • 8 GB Memory
  • 80 GB Hard disk
  • Gigabit Ethernet NIC
  • Myrinet M3F-PCIXD-2

  • 11.2 GB/s memory bandwidth
  • 8.8 Gflops/s peak performance
4 CPU's in 2 Dual nodes
  • HP DL 585
  • 2 AMD Opteron 848 Processors
  • 8 GB Memory
  • 146 GB Hard disk
  • Gigabit Ethernet NIC
  • Myrinet M3F-PCIXD-2

  • 11.2 GB/s memory bandwidth
  • 8.8 Gflops/s peak performance
AMD Opteron 248

Software

  • Ubuntu 6.06 LTS
  • OpenAFS AFS client
  • MX

  • MPICH-MX
  • Goto Blas
  • ScaLAPACK
  • BLACS
  • FFTW
  • PGI Compiler suite
  • PathScale Compiler suite
Links

HPC2N Resources Accounts Support Sysinfo Search News

2009-06-12 - wmaster