(Bloomberg) -- The Central Intelligence Agency is putting its clandestine history out in the open with a declassified trove of 12 million pages that’s available online.

Documents covering the agency’s work from the 1940s through 1990s were previously accessible -- but only by visiting the National Archives in College Park, Maryland. The CIA is required to declassify most records that are 25 years or older.

“This is one of the things that we think improves transparency for us, and it’s a simple thing” to make information “more widely available,” Joseph Lambert, the CIA’s director of information management, said in a phone interview.

Known as the CIA Records Search Tool, or CREST, the collection includes reports on policy and intelligence operations, with topics from the Cold War and Vietnam to terrorism and global economics. The agency continues to review documents for declassification, but the number of records to be assessed is growing rapidly, beyond the realm of humans “being able to scale to that kind of volume,” Lambert said.

Lambert said the agency is looking at emerging technology, including artificial intelligence and machine-learning tools, to help process the staggering amount of documents officials must pore through. That requires judging whether releasing the content would harm national security and redacting certain sensitive names and details.

“Human beings can only do so many pages,” Lambert said. “It’s a difficult endeavor to make sure that you can put together the right technologies to assist a human being going forward to scale to hundreds of millions of pages.”

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access