Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on Jun 11, 2026

• fix: pass through list outputs for generate_until_multi_round (#1364)

  Show description for 49de0a5
  kcz358

  authored
  49de0a5

  Copy full SHA for 49de0a5

• feat(visfactor): add VisFactor benchmark task (#1362)

  Show description for 1856e97
  anxiangsir

  authored
  1856e97

  Copy full SHA for 1856e97

• Fix PointBench image/question misalignment and add binary metric (#1360)

  Show description for 58c949f
  

  njb-nvidia

  and

  claude

  authored
  58c949f

  Copy full SHA for 58c949f

Commits on Jun 2, 2026

• feat(ovobench, chat): run OVO-Bench on chat models via multi-round (#1359)

  Show description for d675c79
  

  kcz358

  and

  github-actions[bot]

  authored
  d675c79

  Copy full SHA for d675c79

• feat: add HoliSafe task (#1358)

  Show description for 8712507
  youngwanLEE

  authored
  8712507

  Copy full SHA for 8712507

Commits on May 30, 2026

• feat: add MVP-Mini (minimal_video_pairs mini split) (#1356)

  Show description for 824d292
  njb-nvidia

  authored
  824d292

  Copy full SHA for 824d292

Commits on May 28, 2026

• feat: add OmniSpatial task (#1357)

  Show description for d1c8ab7
  njb-nvidia

  authored
  d1c8ab7

  Copy full SHA for d1c8ab7

• feat: add CRPE-Relation task (#1354)

  Show description for b71d25a
  njb-nvidia

  authored
  b71d25a

  Copy full SHA for b71d25a

• feat: add Physical AI Understanding task (#1353)

  Show description for 6377299
  njb-nvidia

  authored
  6377299

  Copy full SHA for 6377299

Commits on May 25, 2026

• feat(llava_onevision2): add codec sub-mode (use_codec, codec_*) (#1352)

  Show description for 6906172
  yiyexy

  authored
  6906172

  Copy full SHA for 6906172

• fix: add acc metric and fix data path for Video-MME-v2 (#1351)

  EliYuan30

  authored
  394a8e4

  Copy full SHA for 394a8e4

• feat: add RoboSpatial task (#1347)

  Show description for bf5e8b8
  njb-nvidia

  authored
  bf5e8b8

  Copy full SHA for bf5e8b8

• feat: add CrossPoint-Bench task (#1349)

  Show description for aa2ecb6
  njb-nvidia

  authored
  aa2ecb6

  Copy full SHA for aa2ecb6

Commits on May 22, 2026

• feat: add SAT task (#1348)

  Show description for 1979f32
  njb-nvidia

  authored
  1979f32

  Copy full SHA for 1979f32

• feat: add Open-X VQA task (#1346)

  Show description for 761342f
  njb-nvidia

  authored
  761342f

  Copy full SHA for 761342f

Commits on May 20, 2026

• fix: update llava_onevision2 checkpoint repo path (#1345)

  Show description for 247bebd
  

  3 peopleauthored
  247bebd

  Copy full SHA for 247bebd

• feat: add MetaVQA task (#1340)

  Show description for cb84a30
  

  njb-nvidia

  and

  nilsjohanbjorck

  authored
  cb84a30

  Copy full SHA for cb84a30

• feat: add EgoPlan-Bench2 task (#1339)

  Show description for 051ee0d
  

  njb-nvidia

  and

  nilsjohanbjorck

  authored
  051ee0d

  Copy full SHA for 051ee0d

• egotaskqa: add task port to upstream lmms-eval subtree (#1338)

  Show description for 531dfa9
  njb-nvidia

  authored
  531dfa9

  Copy full SHA for 531dfa9

• fix(mmmu_pro): use post_prompt for vision-only doc_to_text (#1336)

  ts-kim

  authored
  22b5163

  Copy full SHA for 22b5163

• fix(llava_onevision2): forward static images to image_processor (#1344)

  Show description for 7091718
  yiyexy

  authored
  7091718

  Copy full SHA for 7091718

Commits on May 19, 2026

• feat: add LLaVA-OneVision2 chat model wrapper (#1337)

  Show description for 7108c2c
  yiyexy

  authored
  7108c2c

  Copy full SHA for 7108c2c

Commits on May 15, 2026

• fix: preserve per-request vllm sampling params (#1326)

  Travor278

  authored
  9c7a55a

  Copy full SHA for 9c7a55a

• feat: Add Spatial-DISE benchmark task (#1327)

  Show description for bd71e82
  shinmohuang

  authored
  bd71e82

  Copy full SHA for bd71e82

• fix: handle text-only tasks and generate failure in llava_hf and instructblip (#1328)

  Show description for 9c78edc
  Abhishek8108

  authored
  9c78edc

  Copy full SHA for 9c78edc

• fix(jumpscore): align message format and video lookup (#1330)

  Show description for a1ba778
  

  mathCrazyy

  and

  github-actions[bot]

  authored
  a1ba778

  Copy full SHA for a1ba778

Commits on May 11, 2026

• feat: add JumpScore evaluation task (#1329)

  Show description for 4510f3e
  mathCrazyy

  authored
  4510f3e

  Copy full SHA for 4510f3e

Commits on May 8, 2026

• style: format HD-EPIC files (#1324)

  Show description for 13fd645
  aliazani

  authored
  13fd645

  Copy full SHA for 13fd645

Commits on May 7, 2026

• feat: add HD-EPIC VQA benchmark (CVPR 2025) (#1316)

  Show description for 5505089
  aliazani

  authored
  5505089

  Copy full SHA for 5505089

Commits on May 6, 2026

• feat: add timelens benchmark (#1323)

  kcz358

  authored
  26f1d8e

  Copy full SHA for 26f1d8e

• feat: add ReVSI evaluation (#1307)

  Show description for 78d6c72
  

  eamonn-zh

  and

  Copilot

  authored
  78d6c72

  Copy full SHA for 78d6c72

• feat: add VideoNet benchmark (#1308)

  Show description for ed09b9a
  yadavta

  authored
  ed09b9a

  Copy full SHA for ed09b9a

• fix: preserve OpenAI max_new_tokens (#1318)

  Genmin

  authored
  55c7eba

  Copy full SHA for 55c7eba

• fix(api/task): handle None generation responses in process_results (#1311)

  dankit

  authored
  a31a7de

  Copy full SHA for a31a7de

• Release accelerator model refs during cleanup (#1321)

  xk-huang

  authored
  3592b72

  Copy full SHA for 3592b72