First comprehensive index of the Dark Web planned for 2020 release

Register now

A six-year effort to create the first comprehensive index of the dark web is on track for completion by 2020, according to Resecurity, the cyber security company behind the indexing effort.

The project began in stealth mode in 2014 and dives deeper into the hidden recesses of the Internet than any previous undertaking, the company said. It’s expected to yield multiple petabytes of data that law enforcement agencies and cyber security customers can use to stop cyber crime and investigate threats.

The dark web, the portions of the Internet that traditional search engines don’t index and where individuals mask their identities through anonymization tools, is home to hidden marketplaces, communities, groups, and forums used to traffic in illegal goods and services as well as tools for malware and ransomware distribution, Resecurity said.

The consequences of cyber crime originating from the dark web affect nearly every industry and cost the global economy as much as $600 billion, the company said. Security industry experts project that organizations worldwide could incur costs and lost revenue amounting to more than $5 trillion over the next five years due to cyber attacks.

To analyze and store the enormous volumes of dynamic data involved in the indexing, Resecurity relies on advanced big data, data science, and artificial intelligence (AI) technologies. It has trained machine learning models and AI engines to recognize relevant content by category, and to mine meaningful information about threat actors and their operations in a near real time.

For reprint and licensing requests for this article, click here.