Features:

WEATHER FORECASTING ON A 'REMOTE' SUPERCOMPUTER?
by Tim Curns, Editor

For years, researchers around the world, in government and academic labs have been using IBM supercomputers to explore and answer some of the most challenging questions about the effects of climate change on the earth.

Through its relationship with IBM, the National Weather Service has deployed several projects including an IBM supercomputer that by its completion in 2009 will reach a peak speed of 100 teraflops. At that speed, it would take a single person more than 80 million years to tabulate the number of calculations that the supercomputer could do in a single second. Upon its completion, the supercomputer will be one of the largest systems in the world.

HPCwire had a chance to speak with David Blaskovich, IBM Deep Computing, about IBM's recent endeavors in the forecasting field.

HPCwire: What kinds of developments/contributions can we expect to see IBM making in the forecasting field?

David Blaskovich: For starters, the supercomputer deployed by the National Weather Service is notable for pioneering a new way of supplying deep computing power. The system is located at IBM's e-business Hosting Center in Gaithersburg, Maryland, with processing power and storage capability delivered to the government via an ultra-fast network.

Supercomputers -- large machines that sometimes require more floor space than a basketball court -- have traditionally been located at customer sites. IBM's flexible hosting service enables customers to reap the benefits of powerful supercomputers via a high-speed connection, without having to provide the physical space to house the computer. In addition to making strides in the delivery of supercomputer power for forecasting agencies, IBM is also set to deliver the fastest supercomputer yet, by 2009.

HPCwire: How are other businesses leveraging weather system information to benefit their endeavors and strategies? How is this information affecting society at large?

DB: The weather forecasts produced by the National Weather Service form the basis of television and newspaper forecasts across the country. The forecasts are also used in aviation, agriculture, disaster response and a host of other important areas.

Further statistics include:

The National Oceanic and Atmospheric Administration (NOAA) reports that 10 percent of industries that contribute to the nation's Gross Domestic Product (GDP) are "weather and climate sensitive," including outdoor sports and recreation, construction, energy distribution and agriculture. The percentage nearly triples when industries that are indirectly affected by climate and weather are considered. NOAA estimates that almost 25 percent of the GDP totaling $2.7 trillion, is impacted by weather and climate. (http://www.nado.org/pubs/aug023.html)

The National Weather Service estimates that weather forecasts save America's airlines around $500 million a year: avoiding a cancellation saves $40,000, and avoiding a diversion costs $150,000.

HPCwire: Please describe the relationship between IBM and NWS and how this partnership can foster the most effective developments.

DB: Earlier this year, the National Weather Service deployed a supercomputer - based on a cluster of 44 IBM p690 systems supported by 42 terabytes of IBM TotalStorage FAStT500 Storage Server disk storage - that doubles the current computing power for the National Oceanic and Atmospheric Administration's (NOAA) National Weather Service.

In December the National Weather Service (NWS) also replaced existing computer systems with powerful IBM IntelliStation computers running Linux. The IBM system increased the speed of weather forecasting capabilities by four times while reducing costs by nearly 40 percent.

Currently NWS meteorologists use IBM Intellistations at 137 locations nationwide, to interpret weather data streaming from variety of sources. One of these includes an immensely powerful IBM supercomputer that models atmospheric changes and data from a network of Doppler radars, National Oceanic and Atmospheric Administration satellites, and other terrestrial and airborne sensors. The NWS also uses the 160 IBM eServer xSeries technology to archive the data. This IBM project has resulted in a 40-percent hardware maintenance cost savings.

HPCwire: Could you please describe each of your weather predicting systems and how they are being used to advance our estimates of our future world?

DB: IBM has provided large weather/climate research supercomputers to the world's most prestigious meteorological organizations, including NCEP, the European Centre for Medium-Range Weather Forecasts, Naval Oceanographic Office, National Center for Atmospheric Research, German Meteorological Office, and Hungarian National Meteorological Service. Currently, IBM supercomputers are installed at more than 50% of those national weather services that use supercomputers.

HPCwire: Though this work is obviously important to scientists, particularly in the geosciences, how important is this work to the HPC community? Why?

DB: First of all, atmospheric, oceanographic and climatic predictions are of enormous social and economic value to society at large, in terms of protection of property, prevention of loss of life, and in better managing the natural environment.

Secondly, the numerical applications that are used to simulate the atmosphere and oceans remain limited by computational speeds, although today these same applications require the use of the most powerful computers available. Therefore, atmospheric and oceanographic simulations are commonly used as the first to test and validate new computer architectures as soon as they become available - leading the way for many other sciences including the geosciences.

HPCwire: What challenges in general stand in the way of future advances? For the NWS? What are some performance issues for IBM?

DB: As noted above, for the atmospheric, oceanographic and climatic sciences, there is always room for improvement to make the models more accurate by making them more realistic and improving the quantity and quality of initializing data. For the foreseeable future, the applications are expected to be able to consume the largest machines to be designed.

For instance, today a one-hundred year climate model might run for hundreds of hours on a very large supercomputer just to make one climate simulation. Climate scientists need to make hundreds of such runs in order to test and improve the realism of a climate model and then be able to provide to society more reasonable estimates of climate change for instance. In order to achieve such an improvement in performance, the capability of supercomputers needs to be improved many times over what is available today.

HPCwire: How can performance be increased with the current machines? How will performance be increased for future systems?

DB: Currently the supercomputer has extended the government's ability to forecast hurricanes from three days to five days in advance. This doubles current computing power for the National Oceanic and Atmospheric Administration's (NOAA) National Weather Service. When fully deployed by 2009, the NCEP system will be about four times faster than the most powerful supercomputer in the world today.

IBM will expand the system to reach a peak speed well in excess of 100 teraflops (100 trillion calculations per second) - equivalent to the number of calculations it would take one person with a calculator more than 80 million years to perform.

Additionally the powerful new IBM Intellistation systems will provide the NWS with tools to help meet and exceed its 2004 mission of providing the public with warnings on tornados in under 12 minutes and flash flood in 52 minutes or less.

HPCwire: What kind of competition do you have? How will IBM work with the NWS to stay on top of weather prediction supercomputing?

DB: IBM technology has been the key to dramatic improvements in weather forecasting in the past several years. The IBM supercomputer that the NWS has deployed, today is capable of a peak speed of 7.3 teraflops, and will be expanded to reach a peak speed well in excess of 100 teraflops by 2009. That will be the fastest supercomputer in the world. It would take one person with a calculator more than 80 million years to tabulate the number of calculations a 100-teraflop supercomputer can handle in a single second.

The recent server marketshare numbers from IDC and Gartner demonstrate that a two-tiered server market is fast emerging with money-making IBM on one side and all other competitors struggling on the other.

HPCwire: A recent movie, The Day After Tomorrow, is a fictionalized portrayal of disastrous weather events. Based on your research, could any of the events in the movie become a reality?

DB: Not likely. However, there are many climate scientists whose studies seem to indicate that there are anthropogenic (human-induced) changes taking place in the global climate. The exact nature and magnitude of this changes is yet to be determined, thus the need for more capability to more accurately understand and to be able to make such long term predictions.

HPCwire: The movie also depicted the NWS as having limited ability to process mass data and seemed to rely heavily on other centers, which seemed to slow the prediction process significantly. Is the NWS equipped to handle an emergency of this magnitude? Do you feel IBM products, scalability and service are up for the task?

DB: The U.S. NWS does an excellent job of making weather predictions out to ten days, and is now also doing a good job of making what are called seasonal to interannual predictions, i.e. indications of changes months in advance. However, the NWS does not make operational long-term climate predictions as this is not yet possible. As we discussed above, in order to so on a daily, operational basis would indeed require the use of supercomputers many times more powerful than what is available today. In addition, improvements in the science of the models themselves would need to be improved significantly.

HPCwire: I also found it interesting that the NWS could not tap into other resources directly. Is the NWS working with IBM or any other vendor to GRID its resources?

DB: The U.S. NWS, as well as several other national weather centers, are beginning to study ways in which they can work together, possibly as a global grid, to provide additional improved weather severe weather forecasts on very short notice. The techniques being examined offer promise to deliver additional forecast value, but at this point in time, have yet to be implemented.

HPCwire: Please provide any other comments you'd like our readers to see.

DB: The scientists and technicians involved in the applications of atmospheric, oceanographic, climatic, hydrologic, and whole-earth simulations are among the best in the world in delivering to society enormous value in the form of timely and accurate predictions. Great strides have been made in forecast accuracy over the past two decades. In general, three-day forecasts in 2004 are as accurate as one-day forecasts were in 1984. Consider the value of that capability in terms of supporting personal or commercial decision- making. These same people are working on significantly improving forecast accuracy as far into the future as possible, limited by the quantity and quality of initial data, limited by the "realism" of numerical algorithms in the models, and limited by computing power. IBM is thrilled to be in a position to assist these sciences by designing and delivering to these customers the most powerful and capable computers in the world. This is a great business with a great set of customers. How often do those opportunities arise?

HPCwire: Not often! Thanks for your insights. We look forward to witnessing supercomputing developments in the weather forecasting arena.