hi, qwen-vl team members, great thanks for the impressive model. Recently i encountered great difficulty in reproducing the results on muirbench with qwen3-vl-8b-instruct with lmms-eval, there is nothing wrong with the prompts, parsing logic, or image preprocessing logic. i constantly received evaluation score as 49.40%, significantly lower than 64.40% reported in qwen3-vl report.
sincerely looking forward to your instructions or suggestions.
many thanks, again.
hi, qwen-vl team members, great thanks for the impressive model. Recently i encountered great difficulty in reproducing the results on muirbench with qwen3-vl-8b-instruct with lmms-eval, there is nothing wrong with the prompts, parsing logic, or image preprocessing logic. i constantly received evaluation score as 49.40%, significantly lower than 64.40% reported in qwen3-vl report.
sincerely looking forward to your instructions or suggestions.
many thanks, again.