A CLI tool that converts webpages to clean markdown. It uses a headless Chromium browser to render pages (including JavaScript-heavy ones), extracts the readable content, and outputs markdown.

Use it to:

• Convert any webpage to context for an LLM
• Save an article or blog post for offline reading
• Archive documentation as plain text
• Extract content from JS-heavy SPAs that simple scrapers can't handle

It runs in stealth mode to bypass basic bot detection, though it won't get past more advanced protections like Cloudflare Turnstile or aggressive CAPTCHAs.

Grab the latest binary for your platform from the releases page.

sol requires Rust (2024 edition). Build and install with:

cargo install --path .

# download chromium (one-time)
sol setup

# print markdown to stdout
sol fetch https://example.com

# save to a file
sol fetch https://example.com --output article.md

# wait longer for JS-heavy pages (default: 5000ms)
sol fetch https://example.com --wait-for 10000

# set a custom user-agent
sol fetch https://example.com --user-agent "MyBot/1.0"

1. Launches headless Chromium and navigates to the URL
2. Waits for the page to render (handles JavaScript)
3. Extracts readable content using Mozilla's Readability algorithm (strips nav, ads, sidebars)
4. Converts the cleaned HTML to markdown

macOS and Linux. Windows is not supported.