• Corporate
  • Jul 28, 2021
  • By QOMPLX

QOMPLX Releases the Arkscrape Community Edition: open source internet archiving for investigators and researchers

QOMPLX Releases the Arkscrape Community Edition: open source internet archiving for investigators and researchers

QOMPLX joins the fight to end human trafficking and enable cyber researchers across many domains. The internet is ever-changing. From one minute to the next, there is no guarantee that a web page will be the same. Posts, profiles, and whole sites can be taken down with no notice. This proves a problem for researchers and investigators, and can be an especially critical problem in time-sensitive use cases such as human trafficking, cyber fraud, and digital brand protection investigations.

A frequent method of saving webpages is to take a photo of or screenshot the computer screen. This is a poor way of capturing and archiving information, which can easily get lost or confused when assembling it for later use. There is no web trail to accompany screenshots, and everything becomes rapidly cluttered.

Luckily, there are tools to capture and archive web pages, even those that are deleted at a later point in time. By archiving content, these tools save copies of webpages as they were when they were first accessed (with separate files for subsequent access), allowing for storage of pages as they were when originally seen by the researcher.

Browser extension-based archiving options include Singlefile, Webrecorder, and Archiveror.

Online archives include Wayback Machine, Archive.today, and Perma.cc.

These tools, while effective for archiving, are not necessarily designed for sensitive use cases. That is where Arkscrape comes in.

Arkscrape is an open source tool designed by the QOMPLX team to archive, identify, and enumerate different web applications and devices. It is a Python tool that uses archivebox, Shodan, and gobuster as components.

Unlike some other tools, Arkscrape is local and does not upload archived content to the internet, allowing for privacy and security of data. If there are simultaneous projects in progress, Arkscrape prevents crosstalk, keeping the projects separate and enabling easy organization for later reference.

As a free, open-source tool, Arkscrape enables archiving for worldwide law enforcement organizations, researchers, enterprises, academic institutions, nonprofit organizations and journalists of all budget and funding levels.

Arkscrape was designed to meet the needs of organizations such as the National Child Protection Task Force to aid law enforcement in investigations into human trafficking, child exploitation, and missing persons cases.

Archiving smooths the investigative process, making it easier for investigators to compile, store, and present evidence. Tools such as QOMPLX’s Arkscrape make this possible.


To learn more about or to download Arkscrape, visit our github page.

You might also be interested in

Empowering enterprises to stay ahead of evolving threats

Empowering enterprises to stay ahead of evolving threats

QOMPLX recently joined the IBM Security App Exchange. Here’s why the integration will take your security to the next level.

Read more
Identify and Fight the Phish #CyberMonth

Identify and Fight the Phish #CyberMonth

Phishing attacks are an easy way for a bad actor to gain access to a network. Once inside, they can cause devastating losses.

Read more
How much automation?

How much automation?

Automation of underwriting decisions has a very tangible benefit - cost savings. When rules are automated and decisions are made based on reliable supporting data, underwriters can focus on the outliers and make the most of their precious time.

Read more
Request a Demo

Interested in learning more?

Subscribe today to stay informed and get regular updates from QOMPLX.