Deep Thunder

Having detailed, high-caliber forecasts as a service is a critical requirement for enabling the optimization of weather-sensitive operations. We have constructed the system behind the Deep Thunder service to have several key components: receiving and processing of data; modeling; and post-processing analysis, visualization, and dissemination. Although we are working in each of these areas, our focus has been on business applications using high-performance computing, visualization, and automation; we are also designing, evaluating, and optimizing an integrated system. The research that has led to the simulation codes used for weather modeling has been taking place for decades. We are not inventing new weather models as part of this project. Rather, we are adapting, refining, and applying existing models.

As part of the project, we are developing new methods of data visualization, analysis, and dissemination, as well as techniques for improving computational performance and system automation. Part of the rationale for this focus is practicality. In weather-sensitive business decisions, the weather prediction has no value if it can not be completed quickly enough. Therefore, such predictive simulations must be completed at least an order of magnitude faster than real-time (for example, a hour or so for a 24-hour forecast.) Rapidly-generated, fixed visualizations and highly-interactive, flexible visualizations focused on the applications enable the weather model data to be used quickly, especially for near-real-time decision making in business operations.

NWS focuses on uniform services for the whole U.S. by providing detailed observations (spacecraft, radar, stations, etc.) and global- to continental-scale simulations on a large IBM System p™ Cluster 1600 (12 km for all of North America and the surrounding oceans), which are considerable tasks. It is not their mission to provide customized, detailed services for specific industries or geographies. Deep Thunder provides local, cloud-scale, high-resolution (100 x 100 km at 1 km) simulations (on a small IBM System p Cluster 1600) aimed at business applications using detailed physics and customized operations, products, and integration.

As the technology improved and become more practical, we started to consider other potential applications. When the Deep Computing began at IBM Research, it made sense to make a formal project of Deep Thunder. Therefore, a weather modeling project was started within the Mathematical Sciences Department in 2000. Mr. Christidis and Mr. Treinish moved to that department. Later in the year, Tony Praino joined us. Because one focus of Deep Thunder is on high-performance computing, Deep Thunder moved to the Deep Computing Systems Department in late 2004.

• Relatively low cost of sufficiently-powerful computing: A small supercomputer such as a modestly-sized IBM System p Cluster 1600 can be used and enables cost-effective implementation of a range of forecasting services.
• Availability of relevant input data: The National Weather Service makes its modeled and observed data available to everyone via broadcast satellite technology (called NOAAport). We have one of the receivers here at the Watson labs in Yorktown that we purchased from Planetary Data Systems, Inc.
• Maturity of the modeling codes: Although scientific research continues leading to new or enhanced codes, several popular mesoscale numerical weather prediction systems have become powerful and flexible enough to be used for both research and operational projects. We are using a couple of these software packages, which we have customized.
• Emergence of low-cost platforms for visualization: These platforms include not only high computational capability, but specialized hardware for interactive, three-dimensional rendering. Ubiquitous, inexpensive graphic cards, originally used in computer games, can be used for visualization of data generated by weather models. These platforms now outperform expensive workstations of only a few years ago.

When we put all these factors together with a growing understanding of the weather sensitivity of many business decisions, we are able to implement highly-targeted forecasts focused on specific operational problems. This is not the same as taking continental-scale forecasts from the U.S. government or private weather companies and tailoring the results to a particular region or city. Each forecasting environment is customized not only by the application but by the local geographic conditions and weather concerns, which are not captured by the broad-scale weather models. In addition, we can incorporate the more detailed physics while avoiding the considerable computational cost associated with trying to scale weather models for the entire United States.

Similarly, there are a number of research projects in methods of visualizing complex data for a variety of purposes. We contribute in this area using real-time weather modeling and observation as a testbed for new ideas. We need to make use of the investment in these observations using the capability of modeling codes in an operational fashion.

But the biggest challenge for the viability of this project has been on the business side. Deep Thunder's capability creates a new market. Because this market is still emerging, there is uncertainty about its size, about the willingness and ability of potential customers to invest, etc. But given the magnitude of the relevant weather sensitivity, the potential is quite large. Moreover, the complexity and size can, in itself, intimidate potential investors. However, that potential has been sufficient reason for us to work on this project.

We have worked with several potential customers, such as local government agencies, airlines, surface transportation companies, and energy businesses, to explore interest and opportunity. We see that the Deep Thunder concept represents a new market with enormous potential that can take advantage of IBM's strengths in services, high-performance computing, systems integration, etc.

We used this facility to build a prototype system for creating high-resolution forecasts for the New York area. We generate "nested" 24-hour forecasts at 16, 4, and 1 km resolution (areas of 976 x 976, 244 x 244, and 61 x 61 km in size, respectively) centered over New York City and tied to multi-resolution visualizations. Fixed sets of qualitative products (three-dimensional images and animations) are posted on an internal IBM Web site and updated at least twice a day. In our lab and the IBM Industry Solutions Laboratory in Hawthorne, N.Y., we also have several interactive visualization applications that support more in-depth analysis of the model data and have the ability to examine real-time observations made by the National Weather Service. We have built an operational, end-to-end infrastructure and automation (data ingest, pre-processing, simulation, post-processing, visualization, dissemination). This infrastructure has been augmented with new products geared specifically to users at certain geographic sites as well as to particular applications in order to enable their use as a service. Our customers and collaborators access these products via password-protected Web sites outside the IBM firewall.

We have been able to illustrate how the forecasts can be tailored to the geographic region of interest and specific applications by building an operational infrastructure and experience that enables a viable and practical service with both business and meteorological value. It also has enabled us to give potential customers some real capabilities. Although we are generating regular forecasts, we have sufficient capacity in the near-term for more runs per day (on-demand, in response to customer needs or severe weather) or to expand our current forecasts by length or geographic area, as well as to provide custom forecasts for partners and continued development for Deep Thunder and other weather modeling projects. For example, we have extended these ideas to other metropolitan areas in response to customer interest. Similar nested forecasts to 2 km resolution are now being produced operationally for the Chicago, Kansas City, Atlanta, Baltimore, and Washington metropolitan areas, and 1.5 km resolution for the Miami-Fort Lauderdale area. Experimental forecasts to 1 km resolution are being done for the San Diego area.

The image below places all but one of these forecasts in a geographic context, which shows a map of the eastern two-thirds of the continental United States. On the map are three regions associated with six of the seven aforementioned metropolitan areas. They correspond to the triply-nested, multiple-resolution forecasting domains used to produce each high-resolution weather forecast. The outer nests are in gray, the intermediate nests are in magenta, and the inner nests are in white. The areas within the gray borders are covered at 32 km resolution for Kansas City, Chicago, Atlanta, and Baltimore/Washington, 24 km for Miami-Fort Lauderdale, and 16 km resolution for New York. The areas within the magenta borders are covered at 8 km resolution for Kansas City, Chicago, Atlanta and Baltimore/Washington, 6 km for Miami-Fort Lauderdale, and 4 km resolution for New York. The areas within the white borders are covered at 2 km resolution for Kansas City, Chicago, Atlanta, and Baltimore/Washington, 1.5 km for Miami-Fort Lauderdale, and 1 km resolution for New York.

In order to provide a sense of the computational capabilities, all of the computing for any one of our current, multi-resolution, 24-hour forecasts are completed in 30 to 60 minutes using twenty 1.7-GHz Power4 processors for computing and a single Power4 processor for I/O, which includes the post-processing visualization for populating the Web site. After a run is initiated, the entire process from data ingestion to updating the Web site is fully automated.

Consider the local and short-term impact of weather events. For example, it has been estimated that the annual cost of under- or over-predicting electricity demand due to poor weather forecasts is several hundred million dollars in the U.S. alone. The value for weather forecast services for U.S. households in 2001 was estimated at $11.4 billion. According to the Air Transport Association, air traffic delays caused by weather cost about $4.2 billion in 2000, of which $1.3 billion was estimated to be avoidable. According to the United States Department of Transportation, about 7000 people are killed and 800,000 are injured each year in weather-related accidents on U.S. highways. The economic impact of these and other weather-related problems on the roads are estimated to lead to 544 million vehicle hours of delay and an economic impact of about $42 billion annually. In addition, an industry is emerging for weather derivatives (as hedges against weather-related financial risk), which has grown from nothing in 1997 to tens of billions of dollars today. Initially, this market was for energy-related commodities, but it has expanded to other markets such as agriculture and retail. Although it focuses primarily on the seasonal scale, it might evolve to include the dynamics of the short-term market, as the local impact of energy commodities grows. A summary of these and other statistics is available from the U.S. Government. Granted, these types of weather sensitivities span a wide range of geographies and temporal scales. But a significant fraction is within the window of one to two days for local weather phenomena. Thus, we feel that there is significant potential in this emerging marketplace.

A handful of small companies are also doing somewhat related work, most of which began after our initial work at the 1996 Olympics and subsequent technology demonstrations. In fact, the existence of these relatively new projects is further evidence that there is a market for such capabilities. A few other companies doing similar work are established weather service providers who have added custom modeling on a broad geographic scale to complement their traditional capabilities based upon data products generated by the National Weather Service. For the most part, these other projects have concentrated either on the modeling or the data assimilation portion of the forecasting system, often with a focus on the meteorology alone. This focus is in contrast to our work, whose emphasis is on business-oriented services.

The first image (above) shows a terrain map, colored by a forecast of total precipitation, where darker shades of blue indicate heavier accumulations. The map is marked with the location of major cities or airports as well as river, coastline, and county boundaries. In addition, colored lines indicate predicted winds, with the lighter color indicating faster winds. The lines flow in the direction of the predicted winds as indicated by little arrows. Above the terrain is a forecast of clouds, for which the typical anvil-shaped structure of a thunderstorm cell is visible. Within the clouds are cyan surfaces that correspond to rain shafts, where the precipitation is forming.

The second image (above) illustrates additional details about the properties of the forecast above and at the ground. The coloring corresponds to the hourly accumulation of precipitation on the ground. In the back of the image, the thunderstorm cell shown in the first image is "sliced open;" there we see an irregular colored surface that shows the predicted reflectivity inside the cell, and thus, the internal stucture of the rain shaft. The amount of rain in the cloud is also shown via the brown surface.

The third image (above) illustrates additional details about the properties of the forecast at the ground. The coloring corresponds to surface temperature. The surface itself shows "lifted index," a variable that corresponds to relative stability in the atmosphere. The areas where the surface is more deformed (peaks and valleys) illustrate storm activity. As in the first image, there are lines showing wind flow and local maps. In addition, there are contour lines for predicted reflectivity, corresponding to what a weather radar system would observe. The contours are concentrated in the area of high lifted index and cooler temperatures, where the thunderstorm cell has formed according to the model.

An additional image below illustrates the notion of the multi-resolution nature of the forecasts by showing a forecast for a thunderstorm at 16, 4, and 1 km resolution using techniques similar to those in the first image above.