German Climate Data Centre

Story Details

Understanding global climate change is critical to making responsible decisions for the planet. To support climate studies, the German Climate Research Data Centre (DKRZ) in Hamburg provides advanced supercomputing and data and other services for German research. Today, Sun servers help power that research.

DKRZ works with the Max Planck Institute for Meteorology (MPI-M), where scientists use numeric models that use high-performance computing (HPC) to study interactions between the atmosphere, ocean, territory, and vegetation. Until recently, computing operations for DKRZ climate simulations were performed mostly on vector systems with a relatively small number of collateral processors. But IT specialists, Dr. Joachim Biercamp, the manager of computing science at DKRZ, and Rainer Weigle, the manager of Central IT Services at MPI-M, wanted a new, highly scalable solution to efficiently allocate compute-intensive tasks in many collateral processes for the climate models that they were running. Sun presented an advantage by offering a commodity-based massively parallel computing environment as an alternative to the current vector systems. In addition, a problem that DKRZ experienced was that its file systems distributed data from a central server to each machine node, which took a lot of time. Of course, a new solution would also need to support huge storage and I/O requirements. After careful consideration, Biercamp and Weigle decided a Linux cluster would address those needs.

The project of developing and implementing the Linux cluster solution was publicly put out to tender, and the IT specialists selected Sun. “The overall concept, the good cost/performance ratio of the solution, and Sun's comprehensive experience in high-performance computing were very important in selecting Sun for this project,” says Weigle.

DKRZ deployed the cluster using a mix of Sun servers to support the enormous computing power required for climate modeling. The system core consists of 256 Sun Fire XZ200 M2 server nodes, each with two AMD Opteron dual core processors — for a total of 1,024 processor cores. Each machine node includes a primary memory of 16 GB. The solution uses five high-speed Sun Fire X4600 M2 server as front-end nodes, each with eight AMD Opteron double-core processors. The hardware is connected using a DDR Infiniband solution that results in a rate of 20 Gbit per second for each Infiniband link.

An additional eight Sun Fire X4500 servers, each of which offers a gross capacity of 34 TB that significantly accelerates data transfers. To ensure interoperability for the Sun StorageTek QFS that MPI-M uses, two Sun Fire X4200 M2 servers perform as gateway nodes using Lustre and QFS to support resource sharing between the two storage systems. Data will be produced on the cluster into the Lustre parallel filesystem, then the date will be copied and moved into the QFS filesystem via the gateway nodes. After this processing is complete, the data will be archived from a Unitree based HSM system into SAM for long-term retention and archiving.

Sun Grid Engine provides the Linux cluster resource management system. This system software controls the management of all jobs by optimally linking them to each machine node. “The Sun Grid Engine works well and is easy to use,” says Weigle. Finally, the Sun Customer Ready Program allowed DKRZ to receive a completely integrated solution in a matter of weeks, including the non-Sun componets.

As a result of its deployment with the Sun hardware and software, the German climate scientists now have an HPC system that effectively meets all of its requirements — it's fast and reliable; it has high availability; and it's easy to use.. The system performance reaches 5.6 teraflops, or trillions of floating-point operations per second, delivering the power that users need to improve and adjust given simulation models in far less time than before. In addition, the collateral Lustre file system dramatically accelerates data transfers, and the system efficiently supports the storage and I/0 requirements that the IT specialists desired.

The IT specialists are impressed with the Sun solution. “We did the right thing. Our cluster was quickly implemented, and the production phase has proved the high reliability of the system.” says Biercamp. And because Sun server and file systems are so easily scalable, DKRZ and MPI-M can also be assured that their needs can be met in the future, as scientists continue their work to find solutions to the climate crisis.