refactor: use trimmer runnable instead of manually trimming #1275
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Which also counts in the system message, including tool schemas
Contributor
There was a problem hiding this comment.
Pull Request Overview
Refactors the tool picker and chatbot initiation to use the trim_messages runnable and centralized token management instead of manually slicing message lists.
- Introduces
trim_messagesrunnable increate_tool_pickerand chatbot pipeline. - Adds
token_counterandcontext_lengthparameters withresolve_token_management_params. - Removes manual trimming of
windowed_messagesin__init__.py.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| api/chatbot/agent/toolpicker.py | Added trim_messages runnable, token management parameters, and extracted system instruction. |
| api/chatbot/agent/init.py | Chained trim_messages into the prompt runnable pipeline and removed manual windowed trimming. |
Comments suppressed due to low confidence (3)
api/chatbot/agent/toolpicker.py:23
- Add a docstring explaining the new
token_counterandcontext_lengthparameters and their expected behavior to aid maintainability and user understanding.
def create_tool_picker(
api/chatbot/agent/toolpicker.py:64
- No test cases cover the new trimming behavior; please add tests to verify that
trim_messagescorrectly limits tokens and retains system/human messages as intended.
trimmer = trim_messages(
api/chatbot/agent/toolpicker.py:17
- [nitpick] The constant name
SYS_INSTis terse and may be unclear; consider renaming to something more descriptive likeSYSTEM_INSTRUCTIONorTOOL_PICKER_INSTRUCTION.
SYS_INST = """You are a helpful assistant with access to a set of tools. Use them only when truly necessary.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which also counts in the system message, including tool schemas
Pull Request Checklist
Describe what you have changed in this PR.