fix: OpenAI SDK User-Agent triggers Cloudflare 502 on custom endpoints; remove DOC_MAXIMUM_SIZE check blocking page-range sub-tasks #17947

Summary

Two fixes:

1. OpenAI SDK User-Agent triggers Cloudflare bot detection when using custom OpenAI-compatible endpoints (like beeapi.ai)
2. DOC_MAXIMUM_SIZE check incorrectly blocks page-range sub-tasks for large PDFs

Fix 1: Override OpenAI SDK User-Agent on custom endpoints

Problem

When configuring an OpenAI-compatible API provider with a custom base_url, the OpenAI Python SDK sends User-Agent: OpenAI/Python X.X.X. If the target API is behind Cloudflare (e.g., beeapi.ai), Cloudflare's bot detection may block this request with a 502 error:

Error code: 502 - {'type': '...cloudflare-5xx-errors/error-502/', ...}

The request to the API hangs until timeout because the OpenAI SDK receives an HTML challenge page instead of a JSON API response.

Fix

In Base.__init__, when base_url is not api.openai.com, override User-Agent to RAGFlow via default_headers.

File

rag/llm/chat_model.py

Reproduction

# Works
curl https://<custom-api>/v1/chat/completions -H "Authorization: Bearer $KEY" ...

# Fails (502 from Cloudflare)
curl https://<custom-api>/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -H "User-Agent: OpenAI/Python 2.41.0" ...

Fix 2: Remove DOC_MAXIMUM_SIZE check blocking page-range sub-tasks

Problem

task["size"] is the FULL document size, not the page-range size. For large PDFs, queue_tasks() already splits the document into per-page-range sub-tasks (e.g., 12 pages each). But the DOC_MAXIMUM_SIZE check (128MB default) uses the full file size and blocks every sub-task for files >128MB, even though the work is properly chunked.

Fix

Remove the DOC_MAXIMUM_SIZE check from build_chunks in both the old and refactored code paths. A hard size limit should be enforced at the file-upload step or in queue_tasks() before producing page-range sub-tasks.

Files

• rag/svr/task_executor.py
• rag/svr/task_executor_refactor/chunk_service.py