fix: add tool support to Anthropic LLM for graph memory compatibility by nickrahman · Pull Request #1 · nickrahman/mem0

nickrahman · 2026-03-15T03:09:23Z

Description

Graph memory (Neo4j) doesn't work with Anthropic as the LLM provider. The AnthropicLLM classes in both Python and TypeScript always return plain text strings from generate_response, but graph memory operations call the LLM with tools and expect a dict/object containing tool_calls. This mismatch causes graph memory to silently produce empty results when using Anthropic.

This PR adds proper tool support to both Anthropic LLM implementations: converting OpenAI-format tool definitions to Anthropic's format, and parsing tool_use response blocks into the structured format that graph memory expects. It also removes a redundant structuredLlm field in the TS MemoryGraph class that was an identical copy of llm.

Fixes mem0ai#3711

Type of change

Bug fix (non-breaking change which fixes an issue)

Review Guide

The Problem

Both Python and TS Anthropic providers passed OpenAI-format tools straight through to the Anthropic API (wrong format), then returned response.content[0].text regardless of whether tools were used:

# Python — before
params["tools"] = tools              # ❌ OpenAI format, not Anthropic format
params["tool_choice"] = tool_choice  # ❌ Bare string, Anthropic needs {"type": "auto"}
response = self.client.messages.create(**params)
return response.content[0].text      # ❌ Always string, ignores tool_use blocks

The Fix

Tool definitions are converted from OpenAI format to Anthropic format, and responses are parsed to extract tool_use blocks:

# Python — after
params["tools"] = self._convert_tools(tools)   # ✅ Converts to Anthropic format
params["tool_choice"] = {"type": tool_choice}   # ✅ Dict wrapper

response = self.client.messages.create(**params)
return self._parse_response(response, tools)     # ✅ Extracts tool_calls when present

Review Path

mem0/llms/anthropic.py — _convert_tools and _parse_response follow the same pattern as OpenAILLM._parse_response in mem0/llms/openai.py
mem0-ts/src/oss/src/llms/anthropic.ts — TS mirror of the same changes, with JSON.stringify(block.input) for arguments (matching TS convention in OpenAIStructuredLLM)
mem0-ts/src/oss/src/memory/graph_memory.ts — structuredLlm → llm (was created with identical config)
Tests — both test files cover: no-tools path, tools with tool_use, tools without tool_use, and format conversion verification

How Has This Been Tested?

Unit Test

pytest tests/llms/test_anthropic.py -v                    # 4 passed
npx jest --testPathPattern="anthropic-llm" --no-coverage  # 4 passed
npx jest --testPathPattern="graph-memory-parsing" --no-coverage  # 27 passed (regression)

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

Maintainer Checklist

closes Bug: MemoryGraph.structuredLlm hardcoded to openai_structured prevents use of non-OpenAI providers mem0ai/mem0#3711
Made sure Checks passed

The Anthropic LLM providers (Python and TypeScript) returned plain text strings even when called with tools, causing graph memory operations to silently fail. This adds tool format conversion (OpenAI → Anthropic) and response parsing that extracts tool_use blocks into the dict/object format that graph memory expects. Also removes the redundant `structuredLlm` field from the TS `MemoryGraph` class — it was initialized identically to `llm`. Fixes mem0ai#3711 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

nickrahman

Good fix for the core issue — tool calling and response parsing are correct. A few gaps I noticed:

`tool_choice` mapping is incomplete

params["tool_choice"] = {"type": tool_choice}

This works for "auto" but Anthropic doesn't accept {"type": "required"} — that needs to map to {"type": "any"}. If graph_memory.py passes tool_choice="required" (which it does for entity extraction and contradiction detection steps), this will 400.

The reference implementation (mem0-mcp-selfhosted) handles the full mapping:

if tool_choice == "required":
    params["tool_choice"] = {"type": "any"}
elif tool_choice == "auto":
    params["tool_choice"] = {"type": "auto"}
elif tool_choice == "none":
    pass  # Don't set tool_choice
else:
    params["tool_choice"] = {"type": "tool", "name": tool_choice}

`top_p` conflict not addressed

_get_common_params() sends both temperature and top_p to the Anthropic API, which rejects the combination. This is a pre-existing bug but still affects anyone using this fix — graph memory calls will fail at the API level before tool parsing even matters.

No structured output support

response_format parameter is still ignored. Not blocking for graph memory (which uses tools), but the vector memory pipeline could benefit from output_config on Claude 4.x models for more reliable JSON extraction.

Overall this is a solid ~80% fix for mem0ai#3711. The tool format conversion and response parsing are correct. The tool_choice mapping is the main thing that needs attention before this can work end-to-end with graph memory.

…mat handling Address PR review feedback: - Map tool_choice correctly: "required" → {"type": "any"}, specific tool names → {"type": "tool", "name": ...}, "none" → omit param - Strip top_p from Anthropic API calls (rejects temperature + top_p combo) - Append JSON instruction to user message when response_format is json_object and no tools are provided (same pattern as Ollama provider) Applies to both Python and TypeScript implementations with full test coverage. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…x docstring - Stop passing raw messages to _get_supported_params; filtered_messages (sans system) is set correctly via params.update below. - Raise ValueError on empty response.content instead of IndexError. - Correct response_format docstring — the param IS used, not reserved. - Only drop top_p when temperature is also present (not unconditionally). - Add tests: empty-content guard, _convert_tools validation paths, response_format suppression when tools are present, top_p behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Wrap JSON.parse in _establishNodesRelationsFromData and _getDeleteEntitiesFromSearchOutput with try/catch for malformed args. - Add _sanitizeRelationshipType to reject relationship strings with characters outside [a-z0-9_] before Cypher interpolation. - Remove duplicate LLMResponse/ToolCall interfaces, simplify llmProvider init. - Extract systemContent variable in anthropic.ts. - Add TS tests: empty content guard, _convertTools validation paths. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Revert top_p to unconditional drop (config always provides temperature default, so the conditional was never reachable) and remove broken test. - Add runtime typeof guard in generateChat instead of unsafe `as string` cast. - Move empty-response guard before the tools branch in _parse_response. - Remove noise comments, extract shared test helpers, deduplicate test setup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…e guard - _sanitizeRelationshipType now lowercases and replaces spaces before validating, matching the Python behavior. Previously uppercase LLM output (e.g., "LIKES") was rejected as unsafe. - Add type guard in Python _parse_response: check content[0].type is "text" before accessing .text to avoid AttributeError on unexpected block types. - Replace cross-provider comment with self-contained explanation. - Refactor Python tests: extract llm fixture, _make_text_response helper, and SIMPLE_TOOL constant to reduce duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

nickrahman

Inline comments explaining the non-obvious changes in this PR — why interfaces were removed, what the tool format conversion does, security measures added, etc.

nickrahman · 2026-03-15T04:49:54Z

  similarity: number;
 }

-interface ToolCall {


These local ToolCall and LLMResponse interfaces were duplicates of the canonical definitions already exported from ../llms/base.ts. Removed to avoid divergence — if base.ts adds a field (e.g., role), this local copy would silently fall out of sync.

nickrahman · 2026-03-15T04:50:34Z

      this.config.embedder.config,
    );


Simplified from a three-step if/if override pattern to nullish coalescing. Same precedence: graphStore.llm.provider > llm.provider > "openai" default.

nickrahman · 2026-03-15T04:51:14Z

    if (typeof extractedEntities !== "string" && extractedEntities.toolCalls) {
      const toolCall = extractedEntities.toolCalls[0];
      if (toolCall && toolCall.arguments) {
-        const args = JSON.parse(toolCall.arguments);
-        entities = args.entities || [];
+        try {
+          const args = JSON.parse(toolCall.arguments);
+          entities = args.entities || [];
+        } catch (e) {
+          logger.error(`Failed to parse relation tool arguments: ${e}`);
+        }


Added try/catch around JSON.parse of LLM tool call arguments. Previously, malformed JSON from the LLM would throw an uncaught SyntaxError that crashed the entire add() operation. Now it logs and falls back to an empty entity list — consistent with the existing catch block in _retrieveNodesFromData above.

nickrahman · 2026-03-15T04:51:17Z


    try {


Cypher injection prevention. relationship comes from LLM output and is interpolated directly into Cypher queries (Neo4j doesn't support parameterized relationship types). _sanitizeRelationshipType normalizes to lowercase, replaces spaces with underscores, then validates against [a-z0-9_]+ — rejecting any value with special characters that could alter the query.

nickrahman · 2026-03-15T04:52:21Z

+    tools?: any[],
+    toolChoice: string = "auto",


New tools and toolChoice parameters enable graph memory to use Anthropic for entity extraction via function calling. Return type widens from Promise<string> to Promise<string | LLMResponse> — callers that don't pass tools still get a plain string.

nickrahman · 2026-03-15T04:52:51Z

+        # Anthropic has no native JSON mode; inject instruction into the last user message
+        if response_format and response_format.get("type") == "json_object" and not tools:
+            if filtered_messages and filtered_messages[-1]["role"] == "user":
+                filtered_messages[-1] = {
+                    **filtered_messages[-1],
+                    "content": filtered_messages[-1]["content"] + "\n\nYou must respond with valid JSON only.",
+                }


Anthropic has no native response_format: json_object mode (unlike OpenAI). Instead we inject a prompt instruction into the last user message. Skipped when tools are provided since tool responses are already structured JSON.

nickrahman · 2026-03-15T04:52:53Z

+        params = self._get_supported_params(**kwargs)
+        # Anthropic rejects requests containing both temperature and top_p;
+        # drop top_p since the config always provides a temperature default.
+        if "top_p" in params:
+            params.pop("top_p")


Previously messages was passed into _get_supported_params(), where _get_common_params() stored it in params — then immediately overwritten by the params.update({"messages": filtered_messages}) below. Removed to avoid confusion: filtered_messages (with system message extracted) is the only message list that should reach the API.

top_p is unconditionally dropped because AnthropicConfig always provides a temperature default (0.1), and Anthropic rejects requests containing both.

nickrahman · 2026-03-15T04:52:56Z

+        if tools:
+            params["tools"] = self._convert_tools(tools)
+            mapped = self._map_tool_choice(tool_choice)
+            if mapped is not None:
+                params["tool_choice"] = mapped

        response = self.client.messages.create(**params)


Previously tools were passed through as-is (OpenAI format) with a TODO comment. Now _convert_tools transforms them to Anthropic's format (parameters → input_schema, strips the function wrapper), and _map_tool_choice translates the OpenAI-style choice values. _map_tool_choice returns None for "none" so the param is omitted entirely.

nickrahman · 2026-03-15T04:53:18Z

@@ -52,7 +43,7 @@ export class MemoryGraph {
  private graph: Driver;
  private embeddingModel: Embedder;


structuredLlm (removed) was a second LLM instance created with the identical config as this.llm. Now that generateResponse accepts tools directly, there's no need for a separate instance. All three call sites (_retrieveNodesFromData, _establishNodesRelationsFromData, _getDeleteEntitiesFromSearchOutput) now use this.llm instead.

nickrahman · 2026-03-15T04:53:21Z

+            raise ValueError("Empty response from Anthropic API")
+
+        if tools:
+            content = ""
+            tool_calls = []
+            for block in response.content:
+                if block.type == "text":
+                    content = block.text
+                elif block.type == "tool_use":
+                    tool_calls.append(
+                        {
+                            "name": block.name,
+                            "arguments": block.input,
+                        }
+                    )
+            return {"content": content, "tool_calls": tool_calls}
+        first_block = response.content[0]
+        if first_block.type != "text":
+            raise ValueError(f"Unexpected response type from Anthropic API: {first_block.type}")
+        return first_block.text


New _parse_response method handles both paths:

With tools: iterates content blocks, extracts text and tool_use blocks into a {content, tool_calls} dict. Note arguments is the raw dict (block.input) — the TS implementation JSON-stringifies it instead, since TS graph_memory expects to JSON.parse it.

Without tools: validates the first block is type "text" before accessing .text, preventing a confusing AttributeError on unexpected block types.

The empty-content guard at the top protects both paths.

Uses pytest.importorskip to gracefully skip the test module in CI environments where the anthropic optional dependency isn't available. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The anthropic package was missing from the llms extras, causing test collection to fail in CI where dev envs include the llms feature. Reverts the importorskip workaround in favor of the proper fix. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

nickrahman commented Mar 15, 2026

View reviewed changes

nickrahman and others added 5 commits March 14, 2026 20:51

nickrahman commented Mar 15, 2026

View reviewed changes

nickrahman and others added 3 commits March 14, 2026 21:57

fix: skip anthropic tests when anthropic package not installed

5e9cc8a

Uses pytest.importorskip to gracefully skip the test module in CI environments where the anthropic optional dependency isn't available. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: add noqa E402 for imports after importorskip

b6f5ca7

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

nickrahman marked this pull request as ready for review March 15, 2026 14:26

nickrahman merged commit 437eb77 into main Mar 15, 2026
5 checks passed

Navigation Menu

fix: add tool support to Anthropic LLM for graph memory compatibility - #1

fix: add tool support to Anthropic LLM for graph memory compatibility#1
nickrahman merged 9 commits into
mainfrom
fix/anthropic-tool-support

nickrahman commented Mar 15, 2026

Uh oh!

nickrahman left a comment

Uh oh!

nickrahman left a comment

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

nickrahman Mar 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		@@ -52,7 +43,7 @@ export class MemoryGraph {
		private graph: Driver;
		private embeddingModel: Embedder;

Conversation

nickrahman commented Mar 15, 2026

Description

Type of change

Review Guide

The Problem

The Fix

Review Path

How Has This Been Tested?

Checklist:

Maintainer Checklist

Uh oh!

nickrahman left a comment

Choose a reason for hiding this comment

tool_choice mapping is incomplete

top_p conflict not addressed

No structured output support

Uh oh!

nickrahman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

`tool_choice` mapping is incomplete

`top_p` conflict not addressed