warc

Star

Here are 11 public repositories matching this topic...

crissyfield / troll-a

Star

Drill into WARC web archives

security internet-archive command-line-tool warc security-tools common-crawl

Updated Oct 16, 2024
Go

internetarchive / gowarc

Star

Read and write WARC files in Go

go archiving warc

Updated Sep 14, 2026
Go

NationalLibraryOfNorway / warchaeology

Star

Command line tool for digging into WARC files

cli command-line warc

Updated Aug 5, 2026
Go

datatogether / warc

Star

Golang WARC (Web ARChive) Library

golang package archiving warc iipc

Updated Aug 6, 2019
Go

tamnd / ccrawl-cli

Star

A fast, friendly command line for Common Crawl: URL index search, WARC fetch, Parquet columnar queries, and dataset building.

cli golang scraper open-data dataset web-scraping warc common-crawl web-archive duckdb

Updated Aug 24, 2026
Go

AlexGustafsson / larch

Star

A self-hosted service and toolset for managing, archiving, viewing and sharing bookmarks

golang links self-hosted archiver warc bookmark-manager

Updated Jan 6, 2026
Go

grey-land / warc-browser

Star

a cli toolkit for working with web archives

go golang devtools warc rod web-archive chromedp

Updated Jan 16, 2024
Go

datatogether / archive

Star

golang package for creating/working with warc & cdxj archives

golang package warc cdxj

Updated Apr 6, 2018
Go

wabarc / warcraft

Star

A toolkit to help download webpage as warc file

golang downloader webpage wget warc

Updated Jul 29, 2026
Go

hadrianw / abracabra-legacy

Star

A search engine, but currently a filtering pipeline for WARC files. Legacy repo, look for abracabra repo.

golang search-engine warc adblocking

Updated Aug 6, 2019
Go

LOKESH-loky / Concurrent-Web-Crawler

Star

The Concurrent Web Crawler is a Go-based application designed to crawl web pages efficiently using Go's powerful concurrency features.

java scraper spider multithreading async-python aiohttp asyncio html-parsing warc webcrawler graph-generation web-spider concurrent-web-crawler pluggable-modules

Updated Feb 21, 2026
Go

Add this topic to your repo

To associate your repository with the warc topic, visit your repo's landing page and select "manage topics."

Learn more