Skip to content

Conversation

@farzadab
Copy link
Contributor

@farzadab farzadab commented Jun 14, 2024

This PR adds:

  • Training version of BoolQ (extended with GPT-based "explanations")
  • True/False (exact match) evaluations for BoolQ
  • Text-only evaluations for BoolQ and AnyInstruct to show the text-audio gap

@farzadab farzadab marked this pull request as ready for review June 14, 2024 23:53
@farzadab farzadab requested a review from juberti June 14, 2024 23:53
@farzadab farzadab marked this pull request as draft June 17, 2024 16:10
@farzadab farzadab marked this pull request as ready for review June 19, 2024 23:44
@farzadab farzadab merged commit 4202b56 into main Jun 21, 2024
@farzadab farzadab deleted the farzad-boolq-evals branch June 21, 2024 16:00
akshat0311 pushed a commit to jiviai/audio-llm that referenced this pull request Jan 30, 2025
* set default to include_context=True

* boolq extended dataset for training

* improved evals + boolq T/F eval + text-only
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants