For the January 2002 issue of DM Review, I wrote a column titled "We Can" (available in the resource library at or the archives of The column spoke to the need for a massive data integration project to prevent future terrorist attacks: "More than any war that has gone before, this is an information war. Information is so critically valuable because our enemies are hiding in open and available information. By integrating and exposing this information to secure access, analysis, data mining and real-time alerts, we can expose and eliminate our terrorist enemies."

The column spoke to the tracks that our enemies leave in the available information: "At this very moment, it is more than likely that additional cells of terrorists are doing the same thing ­ buying, traveling and living in our society ­ generating transactions in open and available data. In addition, each of these people generates transactions and data points in the various systems of government agencies. From visas to ports of entry, to watch lists, to voice and data surveillance systems, they all leave tracks."

The column spoke to the specific challenge we face: "The challenge is to integrate the key data points across the heterogeneous systems of the public and private sector to allow the detection and identification of our enemies."

The column spoke about the only group of people in the world who are qualified to accomplish this mission ­ you, the business intelligence (BI) professionals: "We are the only group of people with extensive experience in integrating multiple data sources into accessible and distributable tactical and strategic decision support resources. We've built systems for every type of government agency, public and secret, and every type of commercial business, nonprofit and educational institution. We've faced the challenges, figured out the solutions, implemented the technologies and developed the best practices. In short, we are the keys to success in this information war. Our country literally cannot win this battle without our skills and experience."

Finally, the column asked for support in forming an oversight board for these initiatives: "It is my goal to establish a vendor, architecture and technology-neutral oversight board of BI practitioners who can review and provide oversight for the billions of dollars that will soon be spent on these projects."

More than one year later, what is the current status?

On January 5, 2003, the Chicago Tribune carried a story titled "Experts fear inability to share data will hinder homeland security office." The story reported, "None of the computer systems of the merging bureaucracies, not to mention the CIA or FBI, can speak to each other, and there is no mandate, or funding, in the nearly 500-page homeland security law to change that."

The story relates that Senator Dick Durbin, who has worked on the issue for a year, is pessimistic about achieving success due to cultural and political issues. "I'm afraid that bureaucratic inertia and turf rivalry are more powerful forces than the threat of terrorism," Durbin said. Durbin and others believe those turf wars will never be resolved as long as government officials maintain the ability to hold onto their own data. "Each of these agencies is inclined to protect its own marbles and not let anyone else touch them. It's about protecting their turf and their jobs. That runs 180 degrees counter to what this nation needs at this moment," the senator said.

Does that sound familiar to you? It reads like a list of the political and cultural challenges that we overcome every day to ensure the success of our BI projects.

In other developments, the Defense Advanced Research Projects Agency (DARPA) funded – to the tune of $500 million to $1 billion – a project called Total Information Awareness (TIA), led by John Poindexter. One of their goals is to develop groundbreaking technologies that will allow the integration of detailed data on a society-wide basis, basically federating the world's data resources.

Unfortunately, the TIA team ignored political and cultural rule #1 of federated BI architectures: Identify, acknowledge and honor the agendas of all political stakeholders. The subsequent backlash included more than 30 civil liberties groups, the New York Times, the Washington Post and other leading newspapers calling for Congress to kill the TIA ASAP.

What can we learn from this wasted year?

Our biggest need remains a vendor, architecture and technology neutral oversight board of BI practitioners who can review and provide oversight for these projects. This oversight board would ensure that BI best practices are implemented. Until then, all we will see are more examples of these BI amateurs repeating the elementary mistakes we all learned how to avoid over the last decade of designing, building and implementing thousands of successful BI systems.

Clearly, while we can, they can't.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access