Durham University, Institute for Computational Cosmology

Story Details

When researching the formation of the universe, scientists need high-performance computing power to run simulations of the interactions of the earliest forms of matter. The Institute for Computational Cosmology (ICC) at Durham University is one of the largest computing centers in the world for studying these phenomena and depends on powerful yet economical computers to handle its compute-intensive research. “Our simulations take weeks or even months to run and if the computing environment crashed all the time, we would never get through them,” says Professor Carlos Frenk, director of the ICC and Ogden Professor for Fundamental Physics.

So in 2001, the center established the Cosmology Machine (COSMA) computing environment as its stable, flexible platform for running these simulations. COSMA consists of three separate Sun clusters running the Solaris 10 Operating System, and Sun Grid Engine and Sun HPC ClusterTools software. But to keep pace with an increasing number of more intricate simulations, COSMA is periodically refreshed with the latest technology. In 2006, the ICC decided to triple COSMA’s processing power in a cost-effective way, while maximizing valuable datacenter space and power resources in the process.

After evaluating several bids, the ICC chose to add Sun Fire X2100, Sun Fire X4100 and Sun Fire V40z servers because of their ability to cost-effectively handle floating point calculations with AMD Opteron processors and meet datacenter requirements. ICC completed installation of its compute cluster environment in about two days with the Sun Customer Ready Program. The first rack was powered up and working within three hours of delivery. In addition, Sun provided a Sun Fire V890 server with UltraSPARC IV+ processors and 64 gigabytes of RAM to handle memory-intensive processing. 256 existing Sun Fire V210 servers have remained in the COSMA grid because they still perform well, processing about 500 jobs at a time. The result is 4.2 teraflops of processing power with more than 580 servers and 1300 CPUs.

As for software, this center relies on the Solaris 10 Operating System because of its stability and flexibility to run on Sun and other systems. Dr. Lydia Heck, senior computer manager at the center, recently migrated an existing Linux cluster to the Solaris 10 OS to ease administration and ensure stability. The migration from Linux to Solaris eliminated server rebooting, which is an invaluable benefit, added Dr. Heck. She also uses the Solaris OS management tools and the Solaris OS Network File System (NFS) technology for communication with the 90-terabyte storage environment. This provides all the benefits of NFS4 and a resilient interaction between the compute cluster and the storage nodes where more than 500 concurrent accesses to one data space can happen at any one time.

Together, Sun grid software and high-performance clustering tools provide the stability and performance needed to run long simulations without interruption. The grid software’s provisioning and management tools automate tasks and make the environment easy to manage. Sun high-performance clustering software uses the Open MPI interface and this enables simulations to be developed and tuned in Sun Studio 12, which enhances performance on the grid.

SunSpectrum support and excellent open source software are an unbeatable combination. “I think the overall benefit of the Sun solution is that it helps us stay on the cutting edge. We cannot produce the best research without its capabilities. And without excellent research, we will not be awarded new research grants. So, Sun's technology and service is essential to meeting our goals,” concludes Dr. Heck.