Skip to content

GoogleChat (Vertex AI) fails with 401 invalid_api_key on async chat paths since 0.25.6 #15992

@glu000

Description

@glu000

GoogleChat (Vertex AI) fails with 401 invalid_api_key on async chat paths since 0.25.6

Description

GoogleChat (rag/llm/chat_model.py, _FACTORY_NAME = "Google Cloud") fails with an OpenAI 401 invalid_api_key error whenever the async chat code paths are used (_async_chat, _async_chat_streamly, inherited from Base).

Root Cause

Base.__init__ always sets:

self.async_client = AsyncOpenAI(api_key=key, base_url=base_url, timeout=timeout)

For GoogleChat, key is the raw JSON string containing the Vertex AI service account credentials ({"google_service_account_key": "...", "google_project_id": "...", "google_region": "..."}), not a valid OpenAI API key.

GoogleChat.__init__ correctly overrides self.client with a genai.Client(vertexai=True, ...) instance (sync Vertex AI SDK), and chat_streamly correctly uses self.client.models.generate_content_stream(...).

However, GoogleChat does not override self.async_client, and does not implement its own _async_chat / _async_chat_streamly. These are inherited from Base, which call:

response = await self.async_client.chat.completions.create(...)

This sends the raw Vertex AI service-account JSON as an OpenAI api_key, resulting in:

openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: {"google...'..., 'code': 'invalid_api_key'}}

Why this surfaced in 0.25.6

Some chat/agent code paths that previously used the sync chat_streamly (which works correctly) now go through the async async_chat / async_chat_streamly methods (inherited _async_chat / _async_chat_streamly in Base). For GoogleChat, these async paths were never implemented and silently fall through to the broken AsyncOpenAI client.

Suggested Fix

GoogleChat should implement its own _async_chat and _async_chat_streamly using self.client.aio.models.generate_content / generate_content_stream (the async namespace of the same genai.Client), mirroring the existing sync _chat / chat_streamly implementations (Content/GenerateContentConfig/ThinkingConfig construction).

Environment

  • RAGFlow version: v0.25.6
  • LLM factory: Google Cloud (Vertex AI), model: gemini-2.5-flash
  • Deployment: Docker, infiniflow/ragflow:v0.25.6 image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions