Classify Anthropic's "input length and max_tokens exceed context limit" as ContextLengthExceededError - #907
Conversation
…t" as ContextLengthExceededError Anthropic checks input + max_tokens against the context window before it checks the prompt alone, and the chat payload always carries max_tokens, so on a long-context model this is the first error a growing conversation hits. None of the context-length patterns matched it and it surfaced as BadRequestError. Closes crmne#906
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #907 +/- ##
=======================================
Coverage 94.44% 94.44%
=======================================
Files 238 238
Lines 11940 11940
Branches 2030 2030
=======================================
Hits 11277 11277
Misses 285 285
Partials 378 378 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
The Likely cause, in case it helps: the MySQL branch of |
What this does
Adds
/context limit/itoErrorMiddleware::CONTEXT_LENGTH_PATTERNSso Anthropic's second context-window error raisesRubyLLM::ContextLengthExceededErrorinstead ofBadRequestError:{"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 900000 + 128000 > 1000000"}}Closes #906.
Why
Anthropic checks
input + max_tokens <= context_windowbefore it checks the prompt alone, and the chat payload always sendsmax_tokens(model.max_tokens || 4096, 128k forclaude-opus-4-8). So on a long-context model this is the first error a growing conversation hits, at roughlycontext_window - max_output_tokensinput tokens. The"prompt is too long"pattern from #755 only fires later, once the prompt alone no longer fits. Applications rescuingContextLengthExceededErrorfor Anthropic therefore missed the case that actually occurs.Type of change
Scope check
Quality check
bundle exec rspec spec/ruby_llm/error_middleware_spec.rb spec/ruby_llm/error_spec.rb(48 examples, 0 failures) andbundle exec rubocopon the touched files"prompt is too long"oneovercommit --run— not run locally (mysql2 does not build on this machine); RuboCop and the affected specs were run by handAI-generated code
API changes
Related issues
#906 (this), #755 (the earlier
"prompt is too long"half). The streaming-status half of #906, where 1.16.0's Anthropic adapter labels every failure a 500, is already fixed onmainbyfailed_http_status; this PR only adds the missing pattern.