#

content-extraction

Here are 6 public repositories matching this topic...

Nibir1 / Aether

Aether: JSON + TOON-powered, robots.txt-compliant open-web retrieval for AI pipelines. A high-performance Go library with SmartQuery routing, article extraction, metadata normalization, RSS/Atom parsing, OpenAPI connectors, caching layers, plugin systems, & JSONL/TOON streaming - a fully free alternative to paid web-search APIs for AI applications.

golang search-engine json web-crawler open-data robots-txt structured-data data-normalization content-extraction toon jsonl rag ai-engineering llm agentic-ai

Updated Dec 11, 2025
Go

scrapfly / scrapfly-mcp

🚀 Official Scrapfly MCP Server - brings enterprise-grade web scraping, extraction, and anti-bot–aware data access to Cursor, Claude, and any MCP-compatible LLM client.

mcp web-crawler web-scraping data-collection content-extraction claude scrapfly model-context-protocol mcp-server

Updated Dec 17, 2025
Go

flaviodelgrosso / marky

Marky helps you convert things into Markdown 📝

markdown cli parser mcp golang-library data-processing content-extraction document-loader

Updated Dec 15, 2025
Go

Solrikk / DataDigger

DataDigger is a powerful and intuitive web application designed to extract and analyze data from web pages.

go data-mining web-application web-scraping html-parser business-intelligence data-extraction data-analysis data-collection web-crawling content-extraction metadata-extraction marketing-tools seo-tools web-tools golang-api research-tools

Updated Mar 2, 2025
Go

minarc / godensity

This repository is implematation of 📄 DOM based content extraction via text density. Tested for Korean web pages.

content-extraction web-content-extractor

Updated Apr 16, 2025
Go

LandWhale2 / TD-Spider

Via Text Density Simple Web Crawler With Go

golang data-mining opensource dom web-crawler scraping content-extraction keyword-search text-density

Updated Mar 19, 2023
Go

Improve this page

Add a description, image, and links to the content-extraction topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the content-extraction topic, visit your repo's landing page and select "manage topics."