Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When I need to use tools, llm use two tools simultaneously, and then an error occurs. #8400

Closed
5 tasks done
YYyp99 opened this issue Sep 14, 2024 · 7 comments
Closed
5 tasks done

Comments

@YYyp99
Copy link

YYyp99 commented Sep 14, 2024

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.8.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When I use the search tool and the web scraping tool, it uses them together, calling both tools simultaneously. Afterwards, the LLM throws an error. [bedrock] Error: ValidationException: A conversation must alternate between user and assistant roles. Make sure the conversation alternates between user and assistant roles and try again.
image
image

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Copy link

dosubot bot commented Sep 14, 2024

The error you're encountering, ValidationException: A conversation must alternate between user and assistant roles, suggests that the conversation flow is not adhering to the required sequence of alternating between user and assistant roles. This issue can arise when multiple tools are invoked simultaneously, causing the conversation to break this sequence.

To address this, you can manage the sequence of tool invocations to ensure they do not execute simultaneously. Dify provides a mechanism to handle this through the use of a variable pool, which stores variables, files, etc., generated during the tool's operation. This allows tools to cooperate with each other and avoid conflicts.

For example, you can save the output of one tool to the variable pool and then use it as input for another tool, ensuring a sequential execution:

# Example: Saving the output of the first tool to the variable pool
result.append(self.create_blob_message(blob=b64decode(image.b64_json), 
                                       meta={ 'mime_type': 'image/png' },
                                       save_as=self.VARIABLE_KEY.IMAGE.value))

# Example: Retrieving the output from the variable pool for the second tool
image_binary = self.get_variable_file(self.VARIABLE_KEY.IMAGE)
if not image_binary:
    return self.create_text_message('Image not found, please request user to generate image firstly.')

This approach ensures that tools are invoked in a sequence, preventing simultaneous execution errors and maintaining the required conversation flow [1].

Additionally, ensure that your conversation logic explicitly alternates between user and assistant roles to comply with the validation requirements.

To continue talking to Dosu, mention @dosu.

@crazywoola
Copy link
Member

Claude does not support system role. Try to switch to another one.

@YYyp99
Copy link
Author

YYyp99 commented Sep 14, 2024

Claude does not support system role. Try to switch to another one.

I'm using Claude from Bedrock, and this issue is caused by calling two different tools simultaneously, right? When I used it before, writing it this way didn't cause any problems.

@crazywoola
Copy link
Member

We have a PR 2 days ago. #8317 maybe you can check with @ybalbert001

@YYyp99
Copy link
Author

YYyp99 commented Sep 14, 2024

We have a PR 2 days ago. #8317 maybe you can check with @ybalbert001

I'm not using a Bedrock cross-region inference model.
image
I just tried it on the cloud version as well, and it also experienced this issue where calling two tools simultaneously results in an error.

@ybalbert001
Copy link
Contributor

ybalbert001 commented Sep 14, 2024

@YYyp99 how to reproduce you problem? share DSL?
This issue seems have no relationship with Bedrock Cross Region Inference

@YYyp99
Copy link
Author

YYyp99 commented Sep 14, 2024

@YYyp99 how to reproduce you problem? share DSL? This issue seems have no relationship with Bedrock Cross Region Inference

image
GoogleSearchPoint is my custom Google search tool, while the other two are official tools. When I ask a question, both tools are used together, causing an error.
image

@dosubot dosubot bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Oct 15, 2024
@YYyp99 YYyp99 closed this as completed Oct 24, 2024
@dosubot dosubot bot removed the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants