by Maciej Cytowski, Bartosz Borucki and Maciej Remiszewski
The Nautilus supercomputer is a development system for hybrid computing. It is the result of a deep technology collaboration between IBM and the Interdisciplinary Centre for Mathematical and Computational Modelling (ICM) at the University of Warsaw, and is looking at the future of adaptive computing and high-performance computing systems. Nautilus is a testbed for new programming frameworks that will help harness the power and efficiency of next-generation processors and accelerators for a wide variety of scientific applications. Approaching the first anniversary of its initial installation, the system has already made several significant achievements.
Though the two-rack cluster was conceived as a small prototype system, the sheer compute power of the PowerXCell8i CPUs meant it was listed in the November 2008 edition of the Top500 list of the world’s fastest supercomputers. In fact, at one point it ranked as high as #220, with a LINPACK result of over 18 TFlops. It now stands at #421. Even more impressive is the power consumption of the system relative to its performance. With 536 MFlops/Watt, it has set a new world record in supercomputer efficiency and was officially ranked #1 on the Green500 list. This record still holds, with none of the new systems in the June 2009 edition of the top500 list able to improve on it.
Nautilus was developed, installed and tuned under close collaboration with IBM Development. The project involved specialists from Poland, Germany and the US working both on-site and remotely on the machine installed at an IBM data centre in Warsaw. The project schedule for meeting the Top500 submission deadlines was very tight and at that point, several parts of the solution were still not officially supported while others were attempted for the first time. One example was the cluster network. The system used a fully non-blocking double data rate InfiniBand fabric (DDR IB) directly interconnecting all QS22 blades. Throughout the installation and tuning process, new drivers and patches had to be developed to achieve the best performance and stability from the system. These software components have eventually made it to the official software stack, which was released at a later stage together with official support for DDR IB adapters for the QS22 blades.
Figure 1: The Nautilus system.
Another example, which demonstrates the unique nature of this system, is the CPU clock speed. After successfully completing the LINPACK runs at the standard frequency of 3.2 GHz, the system achieved 530 MFlops/Watt, breaking the previous record of 488 MFlops/Watt as well as the magical 500 MFlops/Watt barrier for the first time in history. Thanks to the strategic relationship between ICM and IBM, the companies have decided to use this prototype system to demonstrate the capability of the PowerXCell8i CPUs to operate at 4 GHz. This involved changing the blade firmware and increasing the frequency of several components, including not only the CPUs but also memory, which has been clocked at 1 GHz. This could only be done under the close supervision of IBM Development engineers who were closely monitoring the system hardware. The procedure turned out to be a great success: the system delivered stable LINPACK results and was able to further improve its power efficiency to 536 MFlops/Watt, setting a new world record a few days before the final submission deadline.
The Nautilus system is dedicated to the scientific projects of ICM's high-performance computing centre users. The first project began over two years ago. A periodicity searching application used by scientists from the Astronomical Observatory of University of Warsaw in the OGLE project (Optical Gravitational Lensing Experiment) was ported to the Cell architecture with great success. It achieved a greater than 9x speedup compared to the initial implementation. This enabled the whole computation to be finished in less than two weeks rather than a few months. The lessons we learnt from porting and optimizing the scientific application for the OGLE project enabled us to select other classes of scientific computing code that could benefit from the Nautilus design. These classes range from bioinformatics, molecular dynamics and epidemiology to scientific visualization, number theory and optimal control problems, and we hable to achieve similar performance results in those disciplines. The Nautilus system is also available for computer science students. Their work is additionally supported on the basis of the IBM Faculty Award that ICM received last year.
Figure 1: The Nautilus system.
One of the very promising applications of the Nautilus supercomputer is the visualization system developed in deep collaboration with IBM. Medical images are first analysed and processed by ICM’s visualization system called VisNow, which produces high-resolution 3D models. Those models are later rendered on part of the Nautilus system using an interactive ray-tracing tool called iRT, designed and developed by IBM Research Laboratory in Austin, USA. The resulting framework enables researchers and medical doctors to visualize and handle medical data at unprecedented levels of detail and image quality.
Short descriptions of all these projects are available online (see link below).
The collaboration between IBM and ICM goes back several years. Initial discussions around hybrid computing started in late 2006 with the introduction of the Cell Broadband Engine processor. As both companies were interested in forming a strategic partnership, the discussions quickly evolved into a continuous collaboration involving several specialists and spanning multiple departments on both sides. In 2008, a formal collaboration agreement was signed establishing the Joint Cell Competence Centre as the first teaming initiative. Several other collaborative activities based on IBM Deep Computing technology are emerging, as the partnership between the two companies continues to deliver value to the scientific community.
Links:
ICM: http://www.icm.edu.pl
ICM & IBM Joint Cell Competence Centre: http://cell.icm.edu.pl
Nautilus projects: http://cell.icm.edu.pl/index.php/Projects
VisNow visualization system: http://visnow.icm.edu.pl
Please contact:
Maciej Cytowski, Bartosz Borucki
Interdisciplinary Centre for Mathematical and Computational Modelling (ICM),
University of Warsaw, Poland
E-mail:
Maciej Remiszewski
Deep Computing Leader, Central and Eastern Europe (CEE)
IBM Poland
E-mail:
