Skip to content

refactor(loop): remove interim text retry, use system prompt constraint instead - #928

Merged
Re-bin merged 1 commit into
mainfrom
refactor/remove-interim-retry
Feb 21, 2026
Merged

Re-bin merged 1 commit into
mainfrom
refactor/remove-interim-retry

Conversation

@Re-bin

@Re-bin Re-bin commented Feb 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Remove the interim text retry mechanism (introduced in #825, patched in #887) and replace it with a system prompt constraint.

Problem

PR #825 added a retry mechanism: if the model returns text without tool calls on the first response, the framework retries once, hoping the model will call tools on the second attempt.

This approach had several issues:

  1. Wasted tokens on every pure-text conversation — saying "hi" triggers two LLM calls instead of one, because the framework always retries before accepting a text-only response
  2. Identical input on retry — the retry sends the exact same message list, so the model is likely to produce the same result, making the retry ineffective for the target scenario (weak models like MiniMax m2.5 sending "Let me investigate" before tool calls)
  3. Introduced a regression — retrying and discarding the first response could lose valid answers, requiring PR fix: preserve interim content as fallback when retry produces empty response #887 to add fallback logic (interim_content)
  4. Increased complexity — two extra state variables (text_only_retried, interim_content), conditional branching, and a continue in the agent loop, all for a marginal edge case

Solution

From first principles: the framework should not guess whether the model intended to call tools. If the model returns text, that's its answer.

For models that tend to send preliminary messages ("Let me check") without calling tools, the correct fix is at the prompt level, not the framework level:

If you need to use tools, call them directly — never send a preliminary message like "Let me check" without actually calling a tool.

This single sentence in the system prompt addresses the root cause without any runtime cost.

Changes

  • loop.py: Removed text_only_retried, interim_content, retry continue, and fallback logic. The else branch is now just: get content → break. (-15 lines)
  • context.py: Added one-line tool-calling constraint to system prompt. (+1 line)

Net effect

@Re-bin
Re-bin merged commit 6076f98 into main Feb 21, 2026
@Re-bin
Re-bin deleted the refactor/remove-interim-retry branch February 21, 2026 07:37
sorker pushed a commit to sorker/nanobot that referenced this pull request Mar 24, 2026
… prompt constraint instead

refactor(loop): remove interim text retry, use system prompt constraint instead
WTHDonghai pushed a commit to WTHDonghai/nanobot that referenced this pull request Apr 2, 2026
… Jina (HKUDS#928)

Add jina-code-embeddings-1.5b and jina-code-embeddings-0.5b to
JINA_MODEL_DIMENSIONS so dimension validation works correctly.

Add _raise_task_error() helper that catches 422 validation errors
mentioning 'task' and produces a user-friendly error message guiding
users to set query_param and document_param in their embedding config.

Fixes HKUDS#912.
WTHDonghai pushed a commit to WTHDonghai/nanobot that referenced this pull request Apr 2, 2026
… Jina (HKUDS#928)

Add jina-code-embeddings-1.5b and jina-code-embeddings-0.5b to
JINA_MODEL_DIMENSIONS so dimension validation works correctly.

Add _raise_task_error() helper that catches 422 validation errors
mentioning 'task' and produces a user-friendly error message guiding
users to set query_param and document_param in their embedding config.

Fixes HKUDS#912.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant