Features:

INTERVIEW WITH EYAL WALDMAN, CEO, MELLANOX
By Alan Beck, Editor-in-Chief, HPCwire

HPCwire: Eyal, what has Mellanox and InfiniBand been up to since we last talked this spring?

EYAL WALDMAN: Overall, it is Mellanox's charter to improve InfiniBand each and every day and we have been doing this. With sites located in the U.S. and Israel, I can say that the team has been working hard since we started the company and in the past six months we have achieved a tremendous results. I'd like to discuss five topics in particular. They are:

• InfiniBand's role creating the #3 Supercomputer in the world: The Virginia Tech cluster
• Our recently announced 3rd Generation switch device called InfiniScale III
• InfiniBand in the Enterprise Data Center as realized through both Oracle and IBM's DB2
• The large number of InfiniBand Cluster Deployments
• And, TeraFlops Off-The-Shelf or TOTS for short, and how it clearly demonstrates that InfiniBand provides computing at a fraction of the cost.

HPC: Okay, let's start with the Virginia Tech results; they have been very impressive.

EW: Yes, we have been very impressed with Va. Tech and their ability to build a major cluster. It is really quite amazing to create the world's third fastest computer in such a short time. Apple delivered on the G5 platforms and the installation of the first systems began on Sept 9. Virginia Tech then posted the 10.28 TeraFlop result on Nov. 2. It is truly amazing. The whole project went from award to deployment in less than four months. During that time Mellanox built and delivered the 24 96-Port switches used to create the InfiniBand fabric. We ported our software over to Apple's Darwin OS and got great MPI support from the Network Based Computing Lab at Ohio State University. We were there for the installation and bring up of the fabric and helped in the performance tuning process. The total cost of the cluster is just $5.2 million. People from a number of companies worked very hard on this project but obviously Va. Tech had the right vision, and the results are extremely impressive.

HPC: I'm surprised that InfiniBand has already produced 3rd Generation silicon.

EW: As I mentioned the team is working very hard and we get results. We shipped our first device in January of 2001 and we followed that up with the second generation InfiniScale device in October of 2002 and this month we have our new InfiniScale III device.

InfiniScale III supports 24-Ports of 10 Gb/sec InfiniBand and it is the first device to support 30 Gb/sec communications. At 30 Gb/sec the device can support up to eight ports. It is a very remarkable device that delivers one-half Terabit/sec of bandwidth. With a fully integrated physical layer, only one active device is required to create a full 24-Port 10 Gb/sec switch. And we have created this switch called the MTS2400 in a compact 1U chassis. We are demonstrating two versions of the switch at multiple locations at SC2003. The first configuration has 24-Ports at 10 Gb/sec and the other has four ports at 30 Gb/sec plus 12-Ports at 10 Gb/sec.

We are supplying this switch to our OEM partners with general availability in Q1 2004 and we expect the end user price to be only around $400 per 10 Gb/sec port. This price is the best in the market. At $400 a port it is cheaper than 2 Gb/sec Fibre Channel and about one-twentieth the price of 10 GB/sec Ethernet ports (currently running over $10K per port).

HPC: Okay, what about Oracle and DB2?

EW: InfiniBand promises to have the same affect on corporate database computing as it is now having on supercomputers. You have seen in recent months that InfiniBand native support is now being included in both the Oracle and DB2 industry standard clustered database products. We expect deployments to begin to ramp in 2004.

HPC: You mentioned that other InfiniBand clusters are deploying.

EW: Yes, in addition to Va. Tech, there have been more than a dozen deployment announcements, some of them very significant. Los Alamos Labs, Sandia Labs, NCSA, Linux Networx, Intel, Mississippi State and RIKEN (in Japan) are all making major InfiniBand deployments. In fact, it's now so easy to deploy InfiniBand that a TeraFlop or more of performance can be deployed anytime, anywhere, with no specialized infrastructure required. And we have proved it this week at SC2003.

Also, we have worked hard on the software to ensure these clusters are very robust. We have open source MPI support from Ohio State, NCSA, Argonne Labs and commercial support from MPI Softtech. We have solutions available today that support a complete HPC cluster and our OEM partners have solutions available today for all aspects of the enterprise data center.

In addition, the IBAL or InfiniBand Access Layer open source project has come a long way. We are working on this with Intel and other vendors to provide a comprehensive set of open InfiniBand software statck that support complete enterprise data center needs. The software stack includes support for: block storage, communications, databases, HPC and more. Early next year the complete solution will be ready for distribution.

HPC: You mentioned you proved InfiniBand deployment is easy. What is Mellanox showing at SC2003?

EW: We call it TOTS for TeraFlops Off-The-Shelf. We have worked with Intel and a number of other vendors to create an Intel architecture 192-node InfiniBand cluster that is delivering well over 1 TeraFlop of performance, completely from industry standard components. All of the nodes landed at Intel just last week for initial staging and we have completed the cluster on the show floor and are demonstrating it live in the Mellanox booth. The total bill of materials for this cluster is under $1 million. It really shows that TeraFlop performance can be deployed anywhere, anytime, in a short time period, and at record low price points.

It's truly amazing. Only two years ago there were only 12 TeraFlop systems in the world with an average cost of over $10 million per TeraFlop. Today this level of performance can be deployed at SC2003 with a cost of well under $1 million. This is the effect that industry standard servers and InfiniBand is having on the market.

HPC: Would you like to add anything else?

EW: I'd like to thank all of our industry partners including the InfiniBand OEMs, reseller, software partners and customers that have supported InfiniBand over the past year and have made this quick ramp to huge deployments possible. Without all of them Mellanox wouldn't have been able to prove the robustness and value of InfiniBand that is being seen with the major deployments and our TOTS demo. Because of these partners and customers Mellanox can really say that InfiniBand is computing at a fraction of the cost.