OpenAI crawlers IP ranges. Here are the complete and updated lists of OpenAI IP addresses

A plugin for wechat-gptbot that shows you short videos with pretty girls 🎉

Machine-readable AI permissions for websites. A consolidated spec at /.well-known/ai-policy.json for declaring how AI agents may train on, search, or use your content.

Continuously-updated public IP ranges for major cloud providers, SaaS services, and bots — one txt file per provider, refreshed every 4 hours via GitHub Actions. Drop-in for firewalls, allowlists, and geo blocks.

Self-hosted server log analytics for AI bot detection, AI agent monitoring, and AI visibility. Classifies GPTBot, ClaudeBot, ChatGPT-User, Perplexity & more from edge JSONL — no JS tag, no SaaS. Open-source alternative to Cloudflare AI Crawl Control.

A plugin for wechat-gptbot that offers you daily weather report for free🎉

🛡️ An GEO/SEO-friendly rate limiter for Go. Protects your server from abuse/DDoS while safely whitelisting verified Search Engines & AI Bots (Google, Bing, GPTBot, etc.) via IP/RDNS verification.

Claude Code skill that generates and audits ai.txt, robots.txt (AI-crawler rules), llms.txt, and TDM opt-out files for websites.

🤖 Generate optimized robots.txt files for AI search engine crawlers (GPTBot, PerplexityBot, ClaudeBot, and more)

Validates whether GPTBot, ClaudeBot, PerplexityBot and other LLM crawlers are allowed by a robots.txt file.

Complete database of AI search engine crawler user-agents (GPTBot, PerplexityBot, ClaudeBot) with robots.txt configuration examples

Agent-native multi-bot web crawler simulator. See your site through the eyes of Googlebot, GPTBot, ClaudeBot, and PerplexityBot.

Interactive crawler IP intelligence dashboard for search, AI, and user-triggered fetchers.

The best AEO/GEO skill for Claude Code. Audit, fix, and monitor your website's visibility across ChatGPT, Claude, Perplexity, Gemini, and Google AI Overviews. 100+ optimization techniques, 33 evidence collectors, 4-vector GEO Score. Built on Princeton KDD 2024 paper. Free, MIT.

Free, open-source robots.txt tester — Google's retired tester rebuilt with RFC 9309 matching. Test any URL against Googlebot, GPTBot, ClaudeBot, PerplexityBot and more, with a one-click all-bots matrix.

Visual robots.txt manager for the AI era. Block or allow GPTBot, ClaudeBot, PerplexityBot, and 20+ AI crawlers with one click.

Generate robots.txt with smart defaults. Allow or block AI crawlers (GPTBot, ClaudeBot) based on your strategy. Framework presets for Next.js and Gatsby. Staging lockdown. Zero dependencies.

User-agent simulator for major LLM crawlers: measures response code, content length, and JS-dependency

Official IP ranges for AI bot crawlers (OpenAI, Anthropic, Google, Microsoft, Perplexity). Weekly auto-updates.

Streaming Apache/Nginx log analyzer for SEO and crawl-budget analysis on multi-GB logs. 70+ bot families incl. modern AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Bytespider), reverse-DNS verification, 24 reports, React dashboard. Tested at 16 GB / 50.9M lines.