I spent the better part of ten years following the High Performance Computing (otherwise known as "HPC") industry. This industry became one of the most hyped sectors of the computer biz in the mid-eighties. How many of you remember minisupercomputers, MPPs and visualization? Not too many, I'll bet. HPC fell off the press' radar screen in the early nineties. Most of the companies purporting to be in the biz died without fanfare, some amid talk of securities scams; the few survivors were mainly either swallowed by a goliath (e.g., HP's takeover of Convex) or struggled to maintain a dark-colored bottom line. But let me tell you folks, like Zero Mostel and Phil Silvers, funny things happened (to HPC) on its way to the mid- nineties. HPC discovered commercial computing (as opposed to technical computing where HPC was established), and the segment of commercial computing that could most benefit from HPC technologies was discovered to be no less than data warehousing/decision support! All the stuff the HPC guys were really good at--scaling, big data management, analysis, graphics, etc., are just what the doctor orders for decision support, especially where lots of data is involved.

I recall that nearly 10 years ago Fermi Labs was doing data mining on a 200 terabyte database! I haven't checked lately, but it's bound to be a lot bigger today. And how about the EOS (Earth Orbiting Satellite)? NASA's original plan called for accumulating and analyzing data being gathered at a rate of more than 10GB of new data PER DAY! A five terabyte database, which is normally considered "huge" in the commercial space, makes these guys yawn.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access