NEAR REAL TIME DATA MANAGEMENT QUERY ENGINE ON NEC SX-5
by Christopher Lazou

An interesting business idea for broadening the user base of supercomputers was recently presented, in Paris, by Bernard Nivelet. He described the Data Management query engine, SUPER SELECT, developed and patented by Bull, France, and running on an NEC SX-5. In 1998 Bull has become distributor of NEC SX systems in France. As part of this agreement Bull was also charged to find new applications for the SX-5 outside the traditional scientific technical market. At present they are focusing on two offerings, simulators for financial trading rooms and a Query engine for Data Management.

Analysing the Data Management functions of customer relationships, they can see that businesses would like to offer the right products to the right customers, develop customer loyalty, identify loyal customers and unsatisfied ones. For mail order business one needs to identify customer needs, define sale promotion and prepare mailing targeted to customers more likely to buy. With the data available, the queries make heavy demands and using current RDBMS tools it takes too long, days to prepare and days to execute.

The SUPER SELECT, has come up with three basic ideas to speed up query applications. First, it uses the NEC SX-5s server's large memory to obtain high performance when using heavy queries. Current RDBMS spent large amounts of time doing I/O. Because the SX-5s memory is large, by keeping data in memory, this time can be saved. Secondly, it uses the property of data regularity to speed up processing.

Current RDBMS spent large amount of time in useless computations; managing rows as lists of predicates of any size using pointer computations and accessing data in memory through I/O caches. However, databases are made of tables which are very regular structures; the trick is to transform data tables into numerical arrays and work on columns, (millions of components) rather than rows with tenths of components.

Thirdly, it identified the NEC HPC server as the system answering the market requirements. It is a powerful vector processor, with large memory, running UNIX and an entry level of about half a million dollars. In order to overcome reluctance of the commercial market to use vector computers, Bull uses it as a back-end co-processor managed by a server, such as the Bull EPC440, which is well accepted by the commercial market.

To give you an idea, the performance of SUPER SELECT on a one CPU NEC SX- 5s server, compared to Oracle 8.05, running on a 12 CPU Bull EPC 2400, is 62.5 times faster when computing, on a data base of 1.5 million rows, the accumulated volumes for the last 12 months, grouped on 8 regions, 4 models, 2 providers, 2 power ranges and 1 customer type.

In addition, Dyade is setting up the Parana project, headed by Amedeo Napoli, to evaluate SUPER SELECT as a data mining engine. (Dyade is a joint venture between Bull and INRIA).

In summary, SUPER SELECT consists of 4 modules, three modules running on the multiprocessor Bull ESCALA EPC 440 (AIX) server, for data import and preparation, as an interface agent, and as a supervisor. The fourth module runs on the NEC HPC server as a query engine. SUPER SELECT offers a near real time query service, extracting data from MVS or Bull DPS7000 or UNIX or NT plus SGBD-R operating systems, using ESCALA plus RAID disks and an NEC HPC SX- 5s server as co-processor to perform the query. SUPER SELECT is aimed to deliver query performance for desktop tools in exploration system analysis, data mining and statistics.

Copyright: Christopher Lazou, HiPerCom Consultants Ltd, UK, and Author of "Supercomputers and their Use". He can be contacted by E-mail: Chris@lazou.demon.co.uk