Results are in from Winter Corporation's 2003 TopTen Program, a survey of the world's largest and most heavily used databases. Winter Corporation, a consulting and research company, focuses on databases near, at or beyond the scalability frontier. The findings are striking and remind us just how far we've come and what lies ahead.
Winter Corporation has tracked the world's leading databases through its survey programs since 1995. Participation is open-ended and impartial. The eligibility requirements for the 2003 program were simple: databases must be in production and contain a minimum of 500GB of data for Windows systems and 1TB for all other operating environments. Winter Corporation defines database size as the sum of user data, summaries, aggregates and indexes; free space and redundancy are not included. Thus, database size represents the amount of actual data, not the complement of disk space.
Winter Corporation launched the 2003 program last May and collected surveys for five months. The Web-based surveys took approximately 15 minutes to complete. Winter Corp received 313 completed surveys from 23 different countries. To finalize submissions, participants were required to validate metrics by running scripts developed by their database vendor and Winter Corporation or provide system-generated documentation that supported their survey figures. Of the 313 completed surveys, 141 were qualified through the demanding validation process.
Surveys were submitted by organizations in a wide range of industries and represent all leading database management systems, server platforms and storage solutions. HP, Microsoft, Oracle, Sybase and Teradata sponsored the TopTen Program. Sponsorship conveyed promotional benefits but did not influence the program findings in any way.
Databases were judged in the following categories: Database Size, Decision Support Systems; Database Size, Transaction Processing Systems; Database Size, Decision Support, Hybrid Systems; Normalized Data Volume, Decision Support Systems; Rows/Records, Decision Support Systems; Rows/Records, Transaction Processing Systems; Peak Workload, Decision Support Systems; and Peak Workload, Transaction Processing Systems. Within these categories, databases were further sorted by operating environment as follows: All Environments, UNIX Only and Windows Only. Finally, the databases were assessed using four metrics: database size, normalized data volume, rows/records and highest workload. Each category named ten award winners, including a Grand Prize (first-place) award winner.
The Grand Prize award winners in the 2003 TopTen Program follow.
Database Size, Decision Support Systems
All Environments & UNIX Only: Top honors went to France Telecom for the largest database in the All Environments and UNIX Only categories. At 29.2TB, the database was three times as big as that of the 2001 winner. France Telecom runs the Oracle Database on HP Superdome servers and HP RAID storage systems.
Windows Only: The Windows Grand Prize was awarded to comScore Networks, Inc. The 8.9TB implementation was six times the size of the database of the previous winner none other than comScore itself! comScore is a Sybase IQ DBMS site, with Dell PowerEdge servers and EMC Symmetrix 5 storage arrays.
Database Size, Transaction Processing Systems
All Environments: The largest entry came from the Land Registry. At 18.3TB, the database was nearly two times the size of the 2001 winner in this category. The Land Registry environment includes DB2 for z/OS, IBM eServer zSeries servers and Hitachi storage management systems.
UNIX Only: Among UNIX Only systems, the leader was the United States Patent & Trademark Office (PTO) at 5.4TB. The U.S. PTO runs the Oracle Database, an IBM eServer pSeries server and EMC Symmetrix DMX storage platforms.
Windows Only: Verizon Communications won the Grand Prize for the largest Windows implementation in this category. Containing 5.3TB of data, the Verizon database utilizes Microsoft SQL Server DBMS on an HP ProLiant server and EMC Symmetrix DMX storage devices.
Database Size, Decision Support, Hybrid Systems
All Environments: The 2003 program was the first to track hybrid databases, which store data on both tape and disk. In general, these are data archives in which the majority of the data that can be queried is on tape. However, because their sizes dwarf the size of other databases, they deserve notice. Approaching a petabyte of data, the Stanford Linear Accelerator Center (SLAC) database, at 828.8TB, earned the Grand Prize. The SLAC database is managed by Objectivity DBMS on Sun Fire servers and Sun StorEdge storage arrays.
Normalized Data Volume*, Decision Support Systems
*amount of data actually being managed by the database
All Environments & UNIX Only: Two Grand Prizes, All Environments and UNIX Only, went to AT&T for a managing 94.3TB of data. AT&T runs the Daytona DBMS and resides on Sun Enterprise 10000 servers and Sun storage arrays.
Rows/Records, Decision Support Systems
All Environments & UNIX Only: AT&T received two additional Grand Prizes for most rows, All Environments and UNIX Only categories. The 496 billion-row system represents a doubling of the highest figure for this category in 2001.
Windows Only: Among Windows entries, comScore again took top honors. Its 61.4 billion-row system represents a 400-percent increase since the last survey program.
Rows/Records, Transaction Processing Systems
All Environments: Within the All Environments category, UPS supports a 42 billion-row database. Supporting the database are DB2 for z/OS, an IBM eServer zSeries and EMC Symmetrix DMX storage solutions.
UNIX Only: In the UNIX Only category, top honors went to GevityHR, Inc., for its 12.7 billion-row system. The Gevity environment comprised the Oracle Database, an HP 9000 server and HP StorageWorks storage platforms.
Windows Only: Verizon Communications captured a second Windows Only Grand Prize for its 33.4 billion-row database.
Peak Workload*, Decision Support Systems
*maximum number of concurrent, in-flight queries
All Environments & UNIX Only: In the All Environments and UNIX Only categories, the leader was Experian Marketing Services, with 887 concurrent queries. Experian utilizes the Oracle Database, Sun Fire servers and EMC Symmetrix DMX storage arrays.
Windows Only: Among Windows entries, Stratapult Enterprises took the topmost honor, as it did in the 2001 program. This achievement was enabled by a 67-percent increase in Stratapult's workload to 167 concurrent queries. Microsoft SQL Server manages Stratapult's data on an IBM eServer xSeries and Hitachi Freedom storage solutions.
Peak Workload*, Transaction Processing Systems
*peak transactions per second, tps
All Environments: The U.S. Bureau of Customs and Border Protection is a repeat Grand Prize award winner in All Environments, almost doubling its workload from the 2001 program to 51, 450 tps. The Bureau is a CA- Datacom site, supported by IBM eServer zSeries systems and Hitachi storage devices.
UNIX Only: Within the UNIX world, ICICI Bank captured the leading spot by processing 450 tps. Oracle Database handles ICICI's data on Sun Fire servers and HP StorageWorks disk arrays.
Windows Only: The leading Windows entry was the Internet Auction Co., which achieved 3,600 tps. Microsoft SQL Server manages the data on Unisys ES7000 Servers and EMC CLARiiON FC storage systems.
Significant Future Growth Predicted
According to Richard Winter, president of Winter Corporation, "The 2003 program confirms that the frontiers of database scalability have been redefined and are pushing ever outward. Windows established itself as a viable platform choice for large database projects. Our respondents reported extraordinary growth in database size in all environments. In fact, with the larger size requirements of the 2003 program, 45 percent of the databases in the last program would not have even qualified for this campaign."
As far as where we go from here, it's no surprise that those surveyed expect their databases to grow significantly in size and workload over the next two years. Participants believe decision support systems will double in size and triple in workload. They project modest yet steady growth for transaction processing systems. Winter Corporation plans to launch the next TopTen Program later this year. Complete the survey at www.wintercorp.com to determine how your database compares to the leaders.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access