News

Starting with empty racks and a set of CDs, a team of cluster computing experts from the San Diego Supercomputer Center (SDSC) assembled a supercomputer and ran real scientific applications in one hour and 52 minutes at SC2003, the annual conference for high performance networking and computing.

The demonstration illustrated how the latest generation of cluster administration software enables users to easily set up and manage powerful yet relatively inexpensive cluster computers.

The SDSC/NPACI (National Partnership for Advanced Computational Infrastructure) Rocks cluster team began assembling RockStar - a 128-node Sun Fire V60x supercomputer on the opening night of the exhibition. The live demonstration featured the use of Rocks software, an open-source solution for managing Linux-based clusters from SDSC and NPACI.

In less than two hours four server racks were populated with Sun dual-CPU servers, loaded with Rocks and Sun Grid Engine Enterprise Edition software, synchronized with a master server, and brought on line. Other popular Grid and HPC software packages, including Globus, Condor, Ganglia monitor, Ninf-G, MPICH, High-Performance Linpack, and ATLAS BLAS were automatically installed and configured during the system build.

"A few years ago it would have taken a month to do this," said Philip Papadopoulos, director of SDSC's Grid and Cluster Computing group. "The demo showed that a small group of people can quickly and easily asssemble the hardware and software of a high-performance cluster or Grid node and do real science. Our technologies enable researchers to put together powerful cluster systems without having to earn another degree in computer system administration - it's actually faster to provision software on a cluster than to configure the latest version of Windows on a desktop machine. Scientists can spend most of their time on a lab or departmental supercomputer accomplishing their research objectives instead of fighting with the system."

During SC2003, one rack - 64 processors - of the RockStar cluster was dedicated to a long-running GAMESS simulation. GAMESS performed accurate quantum mechanical calculations on 43-atom molecules, which are enormous by previous standards, and the results will be submitted for journal publication.

"This is not a stunt," said project scientist Jerry Greenberg. "We're taking advantage of this opportunity to show people how science is actually done. GAMESS is performing accurate quantum mechanical calculations on 43-atom molecules, which are enormous by previous standards, and our results here will be submitted for journal publication."

During the later stages of the build, an anomaly in one of the processor nodes was detected by the system diagnostics. The node was swapped out within 15 minutes without affecting the function of the rest of the RockStar cluster, which was processing a biomedical application on a portion of its nodes at the same time the rest of the machine was being configured.

The completed RockStar machine ranks #201 on the current Top 500 list of the world's most powerful supercomputers, and will be used on campus at the University of California, San Diego (UCSD) for scientific research after the SC2003 conference adjourns. "This is a very powerful system, and we built it with standard, low-cost, off-the-shelf components," Papadopoulos said.

With more than 140 systems voluntarily registered as having been created using the NPACI Rocks toolkit (and many more not registered), the total processing potential of computers administered by the software suite now exceeds 30 teraflops. At least six cluster systems - from Sun, Mellanox, Dell, Intel, ProMicro, and SDSC itself - on the SC2003 conference exhibit floor are administered using Rocks, with aggregate computing power of more than 3.5 teraflop.

Doing Real Science

The RockStar demo showed the versatility and robustness of the software suite by running applications during the opening night's initial demonstration, while the cluster was still being assembled on stage.

"The Rocks development effort began three and a half years ago, in May of 2000," Papadopoulos said, "and our constant focus has been to come up with a way to make clusters as easy and painless as possible for researchers to install and run. The scientific community is telling us that we've succeeded."

Papadopoulos is principal investigator on a research project that includes joint activities with the California Institute for Telecommunications and Information Technology (Cal-[IT]2), the Biomedical Informatics Research Network (BIRN), and the Geosciences Network (GEON). Several leading scientists from these and other programs gave presentations during the RockStar build, in which they described how Rocks enables them to do networked, high-performance computing to advance their research.

Papadopoulos Leads the Making of Rocks

NPACI Rocks software is a turn-key open-source clustering system that is based on the latest Red Hat Linux platform. It allows users to download CD disk images, burn them on their local PCs, follow a simple setup procedure, and then install a complete working cluster very quickly. It includes an expanding collection of community-standard cluster-aware components including MPI, Ganglia Monitoring, Sun ONE Grid Engine, MPI, Ganglia, National Science Foundation's Middleware Initiative including Globus, Condor, ATLAS BLAS, High-Performance Linpack (HPL), IOZone, and Streams benchmarks. Standard Grid software packages such as Globus and Condor are also included and are based on the National Science Foundation's National Middleware Initiative (NMI) packaging

"This all started when we began working with the UC Berkeley NOW effort, which had developed the Ganglia monitoring tool for their Networks of Workstations project," Papadopoulos explained. "This was one building block in the structure that Rocks assembled. We had to tackle a lot of problems in Grid and cluster administration. We used the best open source packages for the job - when they existed. When they didn't, we created them ourselves."

NPACI Rocks uses a description mechanism to define system configurations without resorting to disk images or requiring a resident cluster expert. NPACI Rocks is compatible with all hardware that Red Hat supports on both IA-32 and IA-64 platforms and will soon include support for AMD Opteron processors.

About SDSC

The San Diego Supercomputer Center's (SDSC) mission is to innovate, develop and deploy technology to advance science. SDSC is involved in an extensive set of collaborations and activities at the intersection of technology and science whose purpose is to enable and facilitate the next generation of scientific advances. Founded in 1985 and primarily funded by the National Science Foundation (NSF), SDSC is an organized research unit of the University of California, San Diego. With a staff of more than 400 scientists, software developers and support personnel, SDSC is an international leader in data management, biosciences, geosciences, grid computing and visualization.

Contact:
Mike Gannis
858-534-5143
mgannis@sdsc.edu

Greg Lund
San Diego Supercomputer Center
858-534-8314
greg@sdsc.edu