Commits

Breadcrumbs

History for

onmain

User selector

All users

All time

Commit history

Commits on Sep 30, 2024

• 20240930 steven exception handling usage tokens (#1560)

  Show description for a32c982
  

  sjadler2004

  authored
  a32c982View commit details

  Copy full SHA for a32c982

  View code at this pointBrowse repository at this point

Commits on Jul 13, 2024

• [eval] Add IMO problems with exact answers (#1528)

  

  Justin W. Lin

  authored
  234bcdeView commit details

  Copy full SHA for 234bcde

  View code at this pointBrowse repository at this point

Commits on Apr 5, 2024

• Remove citation prediction eval (#1512)

  Show description for c124f98
  ojaffe

  authored
  c124f98View commit details

  Copy full SHA for c124f98

  View code at this pointBrowse repository at this point

• Allow for evals with no args (#1517)

  Show description for 4ed2f6f
  thesofakillers

  authored
  4ed2f6fView commit details

  Copy full SHA for 4ed2f6f

  View code at this pointBrowse repository at this point

Commits on Apr 2, 2024

• Switch from pyzstd to zstandard (#1514)

  Show description for b5853eb
  josnyder-2

  authored
  b5853ebView commit details

  Copy full SHA for b5853eb

  View code at this pointBrowse repository at this point

Commits on Mar 28, 2024

• Updates on existing solvers and bugged tool eval (#1506)

  Show description for 2420c62
  

  4 peopleauthored
  2420c62View commit details

  Copy full SHA for 2420c62

  View code at this pointBrowse repository at this point

Commits on Mar 26, 2024

• Add Gemini Solver (#1503)

  Show description for 5a92ac3
  

  ojaffe

  and

  JunShern

  authored
  5a92ac3View commit details

  Copy full SHA for 5a92ac3

  View code at this pointBrowse repository at this point

• Unified create_retrying for all solvers (#1501)

  Show description for 150dcb9
  ojaffe

  authored
  150dcb9View commit details

  Copy full SHA for 150dcb9

  View code at this pointBrowse repository at this point

Commits on Mar 25, 2024

• Log model and usage stats in `record.sampling` (#1449)

  Show description for 9b2e1b1
  JunShern

  authored
  9b2e1b1View commit details

  Copy full SHA for 9b2e1b1

  View code at this pointBrowse repository at this point

• Address sporadic hanging of evals on certain samples (#1482)

  Show description for bfe3925
  thesofakillers

  authored
  bfe3925View commit details

  Copy full SHA for bfe3925

  View code at this pointBrowse repository at this point

Commits on Mar 22, 2024

• TogetherSolver (#1502)

  Show description for 5805c20
  

  3 peopleauthored
  5805c20View commit details

  Copy full SHA for 5805c20

  View code at this pointBrowse repository at this point

Commits on Mar 21, 2024

• AnthropicSolver (#1498)

  Show description for e30e141
  thesofakillers

  authored
  e30e141View commit details

  Copy full SHA for e30e141

  View code at this pointBrowse repository at this point

• Add Human-Relative MLAgentBench (#1496)

  Show description for 4f97ce6
  danesherbs

  authored
  4f97ce6View commit details

  Copy full SHA for 4f97ce6

  View code at this pointBrowse repository at this point

• Add Multi-Step Web Tasks (#1500)

  Show description for 5b84993
  

  danesherbs

  and

  ianmckenzie-oai

  authored
  5b84993View commit details

  Copy full SHA for 5b84993

  View code at this pointBrowse repository at this point

Commits on Mar 19, 2024

• Add In-Context RL eval (#1491)

  Show description for ff994b5
  james-aung-aisi

  authored
  ff994b5View commit details

  Copy full SHA for ff994b5

  View code at this pointBrowse repository at this point

• Add Function Deduction eval (#1492)

  Show description for dfeaac4
  

  james-aung-aisi

  and

  johny-b

  authored
  dfeaac4View commit details

  Copy full SHA for dfeaac4

  View code at this pointBrowse repository at this point

• Identifying Variables Eval (#1488)

  Show description for c207dba
  thesofakillers

  authored
  c207dbaView commit details

  Copy full SHA for c207dba

  View code at this pointBrowse repository at this point

• Track the Stat Eval (#1489)

  Show description for 99bfada
  thesofakillers

  authored
  99bfadaView commit details

  Copy full SHA for 99bfada

  View code at this pointBrowse repository at this point

• Already Said That Eval (#1490)

  Show description for baa12d0
  thesofakillers

  authored
  baa12d0View commit details

  Copy full SHA for baa12d0

  View code at this pointBrowse repository at this point

• Add 20 questions eval (#1499)

  Show description for bd1736e
  inwaves

  authored
  bd1736eView commit details

  Copy full SHA for bd1736e

  View code at this pointBrowse repository at this point

• Add skill acquisition eval (#1497)

  Show description for 76a9f4e
  inwaves

  authored
  76a9f4eView commit details

  Copy full SHA for 76a9f4e

  View code at this pointBrowse repository at this point

• Error Recovery Eval (#1485)

  Show description for 80ac60d
  

  ojaffe

  and

  ianmckenzie-oai

  authored
  80ac60dView commit details

  Copy full SHA for 80ac60d

  View code at this pointBrowse repository at this point

• Can't Do That Anymore Eval (#1487)

  Show description for f72afb9
  ojaffe

  authored
  f72afb9View commit details

  Copy full SHA for f72afb9

  View code at this pointBrowse repository at this point

• Bugged Tools Eval (#1486)

  Show description for ad377e4
  ojaffe

  authored
  ad377e4View commit details

  Copy full SHA for ad377e4

  View code at this pointBrowse repository at this point

Commits on Mar 13, 2024

• Updates on existing evals; readmes; solvers (#1483)

  Show description for 11c30b2
  ojaffe

  authored
  11c30b2View commit details

  Copy full SHA for 11c30b2

  View code at this pointBrowse repository at this point

Commits on Mar 12, 2024

• Drop two datasets from steganography (#1481)

  Show description for 7e958fe
  thesofakillers

  authored
  7e958feView commit details

  Copy full SHA for 7e958fe

  View code at this pointBrowse repository at this point

Commits on Feb 23, 2024

• Suppress 'HTTP/1.1 200 OK' logs from openai library (#1468)

  Show description for 82ec660
  JunShern

  authored
  82ec660View commit details

  Copy full SHA for 82ec660

  View code at this pointBrowse repository at this point

Commits on Jan 29, 2024

• Updates for Solvers (#1461)

  Show description for 4a105ae
  

  4 peopleauthored
  4a105aeView commit details

  Copy full SHA for 4a105ae

  View code at this pointBrowse repository at this point

Commits on Jan 26, 2024

• Logged spec now includes overridden args (#1460)

  Show description for 3040d6f
  ojaffe

  authored
  3040d6fView commit details

  Copy full SHA for 3040d6f

  View code at this pointBrowse repository at this point

• Add run_id to final_report from LocalRecorder (#1452)

  Show description for cf002f2
  ianmckenzie-oai

  authored
  cf002f2View commit details

  Copy full SHA for cf002f2

  View code at this pointBrowse repository at this point

Commits on Jan 10, 2024

• Fix formatting/typing so pre-commit hooks pass (#1451)

  Show description for c66b5c1
  ianmckenzie-oai

  authored
  c66b5c1View commit details

  Copy full SHA for c66b5c1

  View code at this pointBrowse repository at this point

• icelandic gec eval (#1400)

  Show description for 105c2b9
  svanhvitlilja

  authored
  105c2b9View commit details

  Copy full SHA for 105c2b9

  View code at this pointBrowse repository at this point

Commits on Jan 9, 2024

• Add eval yaml for Theory of Mind eval (#1453)

  Show description for 877d555
  ojaffe

  authored
  877d555View commit details

  Copy full SHA for 877d555

  View code at this pointBrowse repository at this point

Commits on Jan 3, 2024

• Improve MMMU performance with prompt engineering (#1450)

  Show description for 2981e65
  etr2460

  authored
  2981e65View commit details

  Copy full SHA for 2981e65

  View code at this pointBrowse repository at this point

• Add eval japanese prime minister (#1422)

  Show description for f1bb7cb
  return-nil

  authored
  f1bb7cbView commit details

  Copy full SHA for f1bb7cb

  View code at this pointBrowse repository at this point