Vendor Spotlight:

CRAY ELABORATES ON ORNL SUPERCOMPUTING COLLABORATION
by Tim Curns, Editor

HPCwire's Tim Curns talked with Cray Chairman and CEO Jim Rottsolk and Peter Ungaro, Cray's vice president of worldwide sales and marketing, about the DOE's recently announced plan for Cray and Oak Ridge National Laboratory to deploy the world's fastest supercomputer.

HPCwire: For our readers, can you summarize the Oak Ridge plan and Cray's role in it?

Jim: Sure. The plan is to provide the world's most powerful supercomputer, one with 250 peak teraflops of performance that delivers 50 actual, sustained teraflops. This will be done in stages. The starting point is Oak Ridge's current Cray X1 supercomputer. The Cray X1 technology will grow to 20 teraflops this year, and next year we'll also deliver a 20-teraflop Red Storm- based system, so in 2005 Cray will have 40 teraflops installed at Oak Ridge. The Cray system will increase to 100 peak teraflops in 2006, and the next stage is to expand that to the 250 peak teraflops system with 50 sustained teraflops in 2007. Of course, not all the funding is yet in place.

HPCwire: Please explain your relationship with ORNL and the decisions involved to supply them with the new leadership-class supercomputer.

Jim: I can talk about it from Cray's perspective. You'd need to go to the DOE Office of Science for deeper insight into the process they used to select Oak Ridge. For us, things began in February 2003, when the DOE Office of Science funded a Cray X1 system at Oak Ridge to test its effectiveness for leadership- class capability. ORNL, Cray and many other institutions in the research community worked together to put that system to the test. This involved an incredible amount of work by many people. The ORNL CCS (Center for Computational Sciences) staff was in constant motion upgrading the hardware and software capabilities of the system. It was a real team effort. In November, Oak Ridge reported that the X1 was running challenging applications up to 25 times faster than on other HPC systems. It ran a standard climate code 50 percent faster per processor than the Earth Simulator. It was very effective. When Secretary Abraham announced the outcome he said proposals from four DOE labs were subjected to rigorous peer reviews, and then Dr. Raymond Orbach, Director of the DOE Office of Science, made a decision with those recommendations in mind.

HPCwire: Briefly discuss the Cray system already in use at ORNL and what it is used for.

Pete: Today, the Cray X1 system at Oak Ridge has 256 multi-streaming processors and is used for solving important problems across a wide variety of scientific disciplines. On this three-teraflop system, many applications are achieving over a teraflop of sustained performance. Some of these science- driven applications are attaining performance levels that have already allowed them to push scientific discovery forward. It is exciting to think about what will happen as the system begins to grow further.

HPCwire: How will the system be used specifically and what benefits do you feel the implementation of this supercomputer will have on the HPC community? Will it have an impact on supercomputing architectures of the future?

Pete: It will be used for DOE mission-related work, especially to enable breakthrough discoveries in life sciences, fusion energy, global climate prediction, astrophysics, nanoscience and many other scientific disciplines. But it will truly be a national resource. It will be open to scientists and engineers throughout the country, so the DOE and Oak Ridge expect to see some major innovations benefiting not just basic science, but also other industries that rely on high-performance computing for strategic advantage, such as automotive, aerospace, pharmaceutical, chemical and process industries, to name a few. Oak Ridge and the DOE Office of Science will be collaborating with other national labs, research institutions, supercomputing centers, universities and private industry to make this all happen. Oak Ridge is definitely going to be a hub of important scientific breakthroughs -- not evolutionary breakthroughs but true revolutionary breakthroughs.

Jim: When some of the world's most creative scientific and engineering minds gain access to an HPC tool of unprecedented power like the planned Oak Ridge system, the resulting innovation will have a major impact on science, engineering and the HPC industry itself. The Cray X1 is already yielding important results. From Cray's standpoint, it's very encouraging to see leading users like Oak Ridge, Sandia, the Army HPC Research Center, Boeing, Arctic Region Supercomputing Center, Pittsburgh Supercomputing Center, Warsaw University-ICM, Spain's National Institute of Meteorology, Korea Meteorological Administration and others opting for our high-bandwidth, purpose-built HPC systems. There was a hope that business servers and PCs could handle any HPC application efficiently and cost-effectively. That has turned out not to be true. It's good for customers to have choices.

HPCwire: How will this development change or affect Cray Inc.? Describe the manpower resources Cray will dedicate to this engagement. Where will it place ORNL relative to other research institutes?

Jim: Being selected to work with Oak Ridge to provide the world's most powerful supercomputer is another major milestone toward Cray's renewed HPC leadership. When Cray Research and Tera came together as a new company in April 2000, we were starting almost from scratch. We had a deep pool of talent that we've added to since then, we had important development programs under way, and we had a reservoir of good will from customers who were unhappy with the limited product choices in the marketplace. We've come a long way in a short time, thanks to our customers and our employees.

Pete: One of the most important things is that many, many researchers in the U.S. and around the world are getting access to extremely powerful Cray systems again. High-volume user experience and feedback are invaluable for tailoring products to meet HPC customer requirements. Unlike most vendors, Cray is 100 percent dedicated to the HPC market, so this kind of close partnering with customers is essential for us. Cray already has the major resources we need to handle our part of the Oak Ridge collaboration- development, manufacturing, applications and so on. As part of the Cray proposal to ORNL and the Office of Science, we have committed to form and support a center of competency at ORNL. This center will include people skilled in development, systems support and applications, with the goal of working with ORNL and the scientific community to integrate the various Cray architectures and work with the users to best take advantage of the Leadership System. It is very exciting for all of us here at Cray to know that we will become an important part of the scientific fabric of the United States at Oak Ridge and other sites.

Jim: I think Oak Ridge can best answer what this means for them. Certainly, there will be increased recognition for the quality of research done at the lab.

HPCwire: Do you think a 100-teraflop system slated for 2006-2007 will still be considered the fastest in the world by then? Where are others' efforts in comparison to yours?

Pete: The Oak Ridge system may be at 250 teraflops or even a bit larger by 2007. The DOE expects this to be the world's fastest non-classified supercomputer at that time, and we do, too. Some others have talked about installing systems with higher peak performance ratings, but none are likely to produce the type of sustained performance on demanding applications that will be possible with this system. This is because Oak Ridge is using high- bandwidth systems designed to achieve a high percentage of peak performance in practice. The DOE is focusing on achieving sustained performance, where real work is accomplished. We applaud them for that.

HPCwire: Why did Oak Ridge choose Cray over its competition?

Pete: Oak Ridge chose to go primarily with Cray after thoroughly evaluating the Cray X1 and finding that it was up to 25 times faster than other high-end systems on their applications versus competitors' systems. We're grateful to DOE, Oak Ridge, and their partners such as Argonne National Laboratory and Pacific Northwest National Laboratory for the opportunity to prove what our purpose-built systems could do. It has been a great partnership with them from day one.

HPCwire: What kinds of hurdles will Cray face while implementing the ORNL system? How does Cray plan to get past these obstacles?

Jim: We'll face the usual challenges vendors face in scaling up advanced technologies, but we've done this a number of times before. I don't think there's any group of employees in the HPC industry with greater experience than Cray employees in implementing innovative technologies and successfully scaling them up. Think of the Cray T3E, which by the way is a conceptual predecessor of both the Cray X1 and the Red Storm architectures. We will learn a lot from this effort, which will also help us in our Cascade program that focuses on building systems to sustain petaflops of performance. Of course, Oak Ridge and their partners will play a role in that program as well.

HPCwire: Will Cray interface with other supercomputer hardware vendors at ORNL, or will you operate independently?

Pete: We're committed to doing everything we can to help Oak Ridge and the scientific community achieve their goals. While we aren't quite sure how all of this will happen yet, we are committed to working with the entire scientific and vendor community to make this a success.

HPCwire: What else is on Cray's horizon? Any more big surprises lingering for the future?

Pete: We have a lot of things coming up soon. The annual Cray User Group (CUG) meeting took place May 17-20 in Knoxville. Cray and CUG will be co- hosting an advanced technical workshop at CINECA in Bologna, Italy on June 16- 18. We'll also have our strongest presence ever as a company at the ISC2004 conference in Heidelberg in the third week of June. Bill Camp from Sandia National Laboratories will be delivering a keynote on Red Storm and we'll be talking about our upcoming Cray X1E, Red Storm and Cray XD1 products-that's the extremely exciting product we acquired through our OctigaBay merger. I know that many people will also be very interested to hear how the Leadership system at Oak Ridge will evolve over time. There are definitely some exciting times ahead.