MASSIVELY PARALLEL PROCESSING
By Laura Hunt
Massively parallel processing (MPP) is used to solve large computational problems. Its architecture may comprise up to thousands of processors in one system. Each processor includes its own bus, memory, disks, copy of the operating system and applications. In MPP operation, the problem or query is broken up into separate pieces, which can be processed simultaneously.
If your company is looking for a high-speed, high-performance system that can execute complex analysis on huge amounts of data, then massively parallel processing (MPP) may be just the technology you need.
MPP involves placing up to thousands of processors in a single box. Each processor has its own stored memory. When a query is sent, software breaks it up so that each processor completes a different part of the problem, making the response time very fast.
The concept is simple. "Think of people trying to dig a 1,000-foot trench. If one guy can do 10 feet an hour, 10 guys could do 100, and 100 could do the whole thing in one hour," says Richard Winter, an analyst at Winter Corp. in Waltham, Mass.
Other multiprocessing systems include symmetrical multiprocessing (SMP), in which processors share the database and memory. SMP is good for large databases that are updated continually. Because the database is shared, it's easier to update, says Rich Partridge, an analyst at Port Chester, N.Y.-based D.H. Brown Associates Inc.
Another form of multiprocessing is clustering technology, in which many servers are connected together.
The choice between MPP, SMP and clustering technology depends on the nature of the problem being solved. If the problem can be easily partitioned or split, MPP is a good solution, Partridge says.
Many large companies, such as Fingerhut Cos., Sears, Roebuck and Co., Wal-Mart Stores Inc., Citigroup Inc., MCI WorldCom Inc., Sprint Corp. and Dayton Hudson Corp., have discovered the benefits of MPP and use it to store customer data, analyze customer behavior and segment customer categories for optimum marketing and sales activities.
Winter advises that users look at scalability requirements before implementing MPP, including how much detailed data is stored, how much of the data is used to find potential customers, how many transactions or queries are performed and how many concurrent users will be accessing the system.
"Once a database gets to a certain size, your user population suddenly gets much bigger or the level of complexity of your problem increases, only MPP can help," Winter says.
One thing to keep in mind: These powerful systems don't come without a powerful price. According to Winter, a system with just a few processors or nodes can be implemented for $400,000 or $500,000. That would typically support a warehouse with 100G bytes to 300G bytes of data. A multinode system that can support a 1T- byte warehouse would cost a few million dollars, he says.