Features:

BABY COSMOS GROWS UP
by Katie Greene, Pittsburgh Supercomputing Center

The universe may have begun with a bang, but the images that reach us from 379,000 years after that singular instant 13 billion years ago present a fairly mundane picture. The most notable characteristic is uniformity. Over immense distances, the temperature of the unimaginably hot matter spread evenly through the early universe fluctuated by mere thousandths of a degree. Yet those tiny fluctuations generated the diverse splendor of the galaxies, nebulas, stars and planets we see today.

Two years ago, a satellite -- the Wilkinson Microwave Anisotropy Probe (WMAP) -- captured the first light that escaped from that hot, uniform early time, providing astronomers with a baby picture of the universe. With each passing month, sky surveys and x-ray observatories add more details to fill in the gaps between then and now. And these observations are only the beginning. In the coming decade, a new wave of missions promises deeper, sharper views into early periods of structure formation.

It's an exciting time for astrophysicists, with one question uppermost in their minds. How well will the new information match up with theories about formation and evolution of the universe? If gravity is the primary force sculpting the heavens, as theories predict, then what structural features should astronomers expect to find, if they look in the right places, within the huge forest of emerging data?

Large-scale computational simulations play an indispensable role bridging the gap between theory and reality in our burgeoning knowledge of the cosmos. To help narrow that gap, astrophysicists Paul Bode and Jeremiah Ostriker of Princeton University used LeMieux to carry out the largest simulation of the universe to date. Starting with the baby picture from WMAP, and depicting the universe with unprecedented detail, they harnessed LeMieux's parallel- processing power to evolve the baby cosmos forward to the present.

Unlike most simulations of cosmic structure, which start with a section of the universe and look only at the end result, Bode and Ostriker assembled a photo album with which to view the universe as it grows up. Designed to facilitate comparison with observations, their album presents the universe as an Earth observer sees it. "We end up producing a virtual night sky," says Ostriker, "which anyone can then study in a computer." With analysis still underway, they've already turned up hints of some as-yet unconfirmed characteristics of the early universe.

Computing in the Dark

With big help from LeMieux, Bode and Ostriker populated their universe with two billion virtual particles -- each the size of several galaxies -- twice as much granular detail as the most ambitious similar simulations. As a concession to computational economy, however, their simulation takes place in the dark. The virtual universe contains no flowing gases and igniting stars.

All two billion particles represent dark matter -- a mysterious type of mass we cannot see. These particles, which attract each other, are also interacting with a still more inscrutable, gravity-less component that makes up about 73 percent of the energy and mass balance of the universe, so-called dark energy, which scientists theorize tries to push space and everything in it apart.

Tracking the interactions of two billion particles over 13 billion years to build a virtual model of the universe presents a large computational challenge. "It's just at the edge of what you can do," says Ostriker, "that's why you need the biggest supercomputers."

LeMieux's combination of number-crunching power and storage capacity provided the combination needed to compute the position of the particles and store their arrangement through time. "It's the whole package, really," says Bode, "lots of processors and lots of memory, lots of disc storage as well."

Even with these computing resources, however, modeling the gravitational landscape shaped by two billion dark-matter particles depends on software ingenuity. Gravity acts over long distances, and every particle shapes the gravity that acts on every other particle. To take advantage of parallel processing, particles are parceled out to different processors, and the need to calculate the force exerted by the particles at one processor on particles elsewhere can create an intra-processor traffic jam of messages.

"You have to figure out a way to avoid spending all your time passing messages around," says Bode. The solution, first developed by former Princeton graduate student Guohong Xu, and continually modified and refined by Bode, splits the force affecting each particle into two parts, a long-range part that comprises the effect of all particles and a short-range part that accounts for the gyrations of the particle's neighbors.

The software implementing this algorithm, called Tree-Particle-Mesh, made efficient use (90 percent scaling) of 420 LeMieux processors, and with five days of computing built the virtual dark-matter universe.

Cold Dark Water in the Valleys

Much more than cold interstellar dust, black holes, and dark, dead stars, the exact nature of most dark matter is unknown. Scientists suspect, however, that dark matter makes up about 24 percent of the universe's mass and energy and exerts gravitational force. Luminous matter contributes only 3 percent, meaning that the gravitational landscape of the universe is defined largely by dark matter. In the Cold Dark Matter theory, which Bode and Ostriker implemented on LeMieux, this means that dozens, hundreds, sometimes even thousands of galaxies cluster in clumps of dark matter, called halos.

"If we can track all of the dark matter," says Ostriker, "then we have a good picture of the structure within which the galaxies find themselves. We take what we think is the right model of cosmology, we put in the initial ingredients -- which are basically the fluctuations that have been seen by the WMAP satellite -- then we turn the crank on the computer and allow gravity to act with these little ripples. We find dark matter accumulating into halos and more massive halos. And they have substructure and merge and do all sorts of wonderful things."

Many of the photos from this virtual album will provide key points for comparison with observations. Because galaxies are packed inside dark matter and carried along by the speed of the dark-matter halo surrounding them, for instance, it's possible to compare with observational data on galaxy velocity. Ostriker and one of his students are cataloging the speed of dark-matter clusters from the simulation to see how this velocity distribution compares with the speed of galaxies astronomers are cataloging from observations.

With a working assumption that galaxy structures are influenced by the dark matter that envelops them, Bode has tracked the evolving shape of the largest clusters of dark matter in the simulation. In early periods of structure formation, Bode found that clusters were more aligned and elongated than expected, supporting the idea that matter pooled into strung-out filaments, much as water migrates to and flows down the center of a river valley. This effect is more striking than expected, says Bode, and as observations of large galaxy clusters at earlier times come in, it will be interesting to see how well the simulation matches up.

Bode is also looking forward to comparing the number of giant clusters of dark matter in the simulation with the number of galaxy swarms in the real universe. If the simulation's mass density -- a key theoretical parameter that describes how closely mass was packed in the beginning -- is larger than in the real universe, the simulation clusters would come together faster and form larger clusters than in the universe. If the simulation is off in the other direction, it will have fewer giant clusters than the real universe.

Bode and Ostriker are also using the gravitational potential of the dark matter distribution to calculate the temperature of gas within dark-matter clusters. "It's an imperfect connection," says Ostriker, "but right now it's the best tool we have. Until now we didn't even have that option because we couldn't make simulations of anything on a big enough scale to compare to the real universe. We could only do little pieces." With dark matter particles and LeMieux, it was possible to do a much larger section of the universe, one of the largest volumes of space ever simulated. "These simulations enable you to look all the way back through space to the beginning."

What if the simulation doesn't match up with observations? That's the beauty of computational simulations, says Ostriker. They make it possible to systematically test and adjust theory. "We can then do another simulation, with a different cosmology. We'll increase dark-matter content, or we'll change the dark-energy content, because in fact we don't know these quantities very well." As observations become more detailed and simulations more accurate in representing theory, science will move step-by-step, says Ostriker, toward knowing what initial features went into creating the universe.

More information, including graphics: http://www.psc.edu/science/bode.html.