Background
CLUMEQ is a supercomputing consortium of universities in the province of Quebec, Canada. It includes McGill University, Université Laval, and all nine components of the Université du Québec network. CLUMEQ supports scientific research in disciplines such as climate and ecosystems modeling, high energy particle physics, cosmology, nanomaterials, supramolecular modeling, bioinformatics, biophotonics, fluid dynamics, data mining and intelligent systems.
CLUMEQ Challenges
- Increase computing power and performance at the Quebec City datacenter
- Improve datacenter efficiency
- Implement flexible technology
- Expand research opportunities
CLUMEQ's Results
- 10x increase in computing resources
- Ability to run substantially more complex simulations and to return results more quickly
- Substantial room to add additional capacity for future expansion
The Sun Solution
CLUMEQ, working closely with Sun consultants, implemented a Sun Constellation System, which combines high-performance computing, networking, storage, and software into a fully integrated system. The system includes Sun Blade 6048 Modular Systems with Sun Blade X6275 Server Modules, a Sun Lustre Storage System, Sun Datacenter InfiniBand Switches, and Sun HPC software. Sun Services worked with the CLUMEQ team to provide project management, and to optimize the placement of cluster components to maximize the cooling efficiency of the facility. Sun also provided a comprehensive, customized training and a maintenance package for the CLUMEQ system administration team.
Sun Products and Services at CLUMEQ
About the Project
In 2006, CLUMEQ started to design a high-performance computing (HPC) system at its Université Laval datacenter, in an effort to give researchers the most advanced HPC technology possible. This new supercomputer was to reside in the silo of a former Van de Graaff particle accelerator on the campus of Université Laval, which was once used for experiments related to subatomic particles. The structure was transformed into an innovative, energy-efficient cooling enclosure leveraging a 3-level cylindrical floor plan with up to 1.5 megawatts of cooling capacity and space to host up to 56 standard-sized racks. Not only was the refurbishment able to preserve a campus landmark, but the Laval researchers were able to exploit the natural efficiencies of a cylindrical design, and incorporated additional efficiencies in their air cooling system.
As an exceptionally efficient way to maintain the perfect temperature inside the silo, free cooling is used for most of the year. Free cooling conditions the air inside the silo by using cooler outside ambient air, along with a system of water filled pipes and coils, drizzle humidifiers and very high-efficiency fans. In summer, the datacenter operates in a closed environment. In winter, heat energy produced by the supercomputer is redistributed underground to nearby buildings and is used to pre-heat the central heating air. Sun consultants worked with the CLUMEQ team to optimize the placement of cluster components to maximize the cooling efficiency of the facility.
Get on the Path to Petascale Computing Learn why Sun's systemic design approach to large scale environments with Sun Constellation System is the best design for high-performance computing. Get the White Paper Now
Solve the HPC I/O Bottleneck With Sun Lustre The Sun Lustre file system is simple to deploy and designed specifically for the needs of high-performance computing environments. Get the White Paper Now
|
Technology History
In early 2009, Sun Microsystems, along with IBM, HP, and two other companies, responded to the CLUMEQ RFP. "We asked for bids for solutions that would specifically fit the silo design," remarks Parizeau. Over the course of two years, an account team from Sun presented technical designs to CLUMEQ at the Sun Executive Briefing Center (EBC) in Menlo Park, California, and the HPC Solution Center in Hillsboro, Oregon. CLUMEQ ultimately decided to go with Sun because of the company's combination of computing power, file system architecture, operating system support, and node management. "Sun won the bid because they offered the best hardware and software at the best price, and its solution fit our criteria," says Parizeau.
“We chose Sun because they offered the highest performance using a very well-balanced, robust, and affordable architecture. HPC is all about performance and balance -- fast CPUs, fast memory access, fast interconnects, and fast storage access. The complete, end-to-end, open solution engineered by Sun makes no compromises in performance while maintaining this all-important balance within the constraints of budget. ”
— Marc Parizeau, Deputy Director of CLUMEQ and Professor of Computer Engineering, Université Laval
New to HPC? Get the Book Get your free eBook special edition of HPC for Dummies from Sun and AMD! Download Your Book Now
In March 2009, Sun began to assist CLUMEQ with the implementation of an end-to-end solution based on the Sun Constellation System. The Sun Constellation System, the world's first open HPC architecture, provides an open petascale computing environment that combines ultra-dense high-performance computing, networking, storage, and software into one system. The Sun Constellation System is designed to be flexible, so that it can fit into a variety of physical datacenters, and was a perfect match for the cylindrical layout of the silo.
The system includes 10 fully loaded Sun Blade 6048 Modular Systems with Sun Blade X6275 Server Modules, a Sun Lustre Storage System, a full-bisection InfiniBand switch interconnect, and a complete HPC software stack.
Customer Ready HPC Configurations The reference configurations described in this white paper are starting points for building your HPC cluster. Get Started Now
Sun's total solution package also included Sun Professional Services, and consultants from its Data Center Services practice successfully integrated the solution into the existing CLUMEQ project ecosystem by complementing the consortium's architectural and construction efforts. The Sun team used three-dimensional Building Information Modeling (BIM) to plan and visualize system installation to facilitate the precise placement of new equipment. Sun also worked closely with its own partner community, which provided expertise in areas such as architecture design and construction management. Sun Services also provided a comprehensive, customized training and a maintenance package for the CLUMEQ system administration team.
The new system, which will be up and running by the end of 2009, features 7680 processor cores with a power envelop below 374kW. As a result, the computing resources of CLUMEQ have increase ten-fold and will allow more complex simulations that were previously out of reach to their researchers. "Our new supercomputer will enable CLUMEQ and other Compute Canada researchers to implement significantly more complex models in search of answers to questions in a wide range of fields, including climate and ecosystems modeling, high-energy-particle physics, cosmology, data mining, and intelligent systems," says Parizeau.
|
