Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Jun 25, 2024

• Merge pull request #10 from alinaryan/mmlu

  Show description for 100c512
  alimaredia

  authored
  100c512

  Copy full SHA for 100c512

• Add fstring

  Show description for 899aaf9
  alinaryan

  committed
  899aaf9

  Copy full SHA for 899aaf9

• suppress import err

  Show description for 88b2e65
  alinaryan

  committed
  88b2e65

  Copy full SHA for 88b2e65

• Fix lint errors

  Show description for 270bf53
  alinaryan

  committed
  270bf53

  Copy full SHA for 270bf53

• Add missing model_path param and description

  Show description for 2e607f5
  alinaryan

  committed
  2e607f5

  Copy full SHA for 2e607f5

• Remove testing code and change the MMLU class names and descriptions

  Show description for 05233b4
  alinaryan

  committed
  05233b4

  Copy full SHA for 05233b4

• working MMLU_Evaluator.run()

  Show description for 8e024b8
  

  alimaredia

  authored and

  alinaryan

  committed
  8e024b8

  Copy full SHA for 8e024b8

• Introduce lm-evaluation-harness dependency

  Show description for 6db41b1
  alinaryan

  committed
  6db41b1

  Copy full SHA for 6db41b1

• Merge pull request #9 from danmcp/main

  Show description for ffe1aa1
  nathan-weinberg

  authored
  ffe1aa1

  Copy full SHA for ffe1aa1

• Working toward a functional state

  Show description for 68185e4
  danmcp

  committed
  68185e4

  Copy full SHA for 68185e4

Commits on Jun 17, 2024

• start to incorporate gen_judgement.py into library

  Show description for ad020ef
  

  alimaredia

  authored and

  danmcp

  committed
  ad020ef

  Copy full SHA for ad020ef

• ran through show_result step

  Show description for cf2ea02
  

  alimaredia

  authored and

  danmcp

  committed
  cf2ea02

  Copy full SHA for cf2ea02

• show what filesystem looks like after gen_judgement.py is run

  Show description for 810dcfc
  

  alimaredia

  authored and

  danmcp

  committed
  810dcfc

  Copy full SHA for 810dcfc

• merge gen_api_answer.py into codebase

  Show description for 3e5cac4
  

  alimaredia

  authored and

  danmcp

  committed
  3e5cac4

  Copy full SHA for 3e5cac4

• Initial commit trying to merge Fastchat mtbench into eval repo

  Show description for fefdb92
  

  alimaredia

  authored and

  danmcp

  committed
  fefdb92

  Copy full SHA for fefdb92

• Merge pull request #6 from nathan-weinberg/skeleton

  Show description for 6632002
  nathan-weinberg

  authored
  6632002

  Copy full SHA for 6632002

• Add badges to README.md

  Show description for 20d2fc4
  nathan-weinberg

  committed
  20d2fc4

  Copy full SHA for 20d2fc4

• Update attributes of ModelNotFoundError class

  Show description for b422048
  nathan-weinberg

  committed
  b422048

  Copy full SHA for b422048

• Merge pull request #8 from instructlab/dependabot/github_actions/pypa/gh-action-pypi-publish-1.9.0

  Show description for a770a86
  nathan-weinberg

  authored
  a770a86

  Copy full SHA for a770a86

• Add docstrings to `run` methods

  Show description for 3eadb4d
  nathan-weinberg

  committed
  3eadb4d

  Copy full SHA for 3eadb4d

• Bump pypa/gh-action-pypi-publish from 1.8.14 to 1.9.0

  Show description for fab7385
  dependabot[bot]

  authored
  fab7385

  Copy full SHA for fab7385

• Change ret value from single dict to multiple ret values

  Show description for 11ad758
  nathan-weinberg

  committed
  11ad758

  Copy full SHA for 11ad758

Commits on Jun 14, 2024

• Change 'fewshots' and 'batchsize' to snake_case

  Show description for 4e72d07
  nathan-weinberg

  committed
  4e72d07

  Copy full SHA for 4e72d07

• Change 'model' to 'model_path' for clarity

  Show description for 0db0be4
  nathan-weinberg

  committed
  0db0be4

  Copy full SHA for 0db0be4

• Seperated out individual `run` commands for each child class

  Show description for 0023da3
  nathan-weinberg

  committed
  0023da3

  Copy full SHA for 0023da3

• Change 'server' to 'server_url' for clarity

  Show description for f62821a
  nathan-weinberg

  committed
  f62821a

  Copy full SHA for f62821a

• Add attribute descriptors to class docstrings

  Show description for fb2c51b
  nathan-weinberg

  committed
  fb2c51b

  Copy full SHA for fb2c51b

• Merge pull request #7 from instructlab/dependabot/github_actions/rojopolis/spellcheck-github-actions-0.38.0

  Show description for 75cb9ce
  alimaredia

  authored
  75cb9ce

  Copy full SHA for 75cb9ce

• Add additional child classes for PR-Bench and PR-MMLU

  Show description for 3558524
  nathan-weinberg

  committed
  3558524

  Copy full SHA for 3558524

• Bump rojopolis/spellcheck-github-actions from 0.37.0 to 0.38.0

  Show description for a47e1ea
  dependabot[bot]

  authored
  a47e1ea

  Copy full SHA for a47e1ea

Commits on Jun 13, 2024

• Initial skeleton for Evaluator classes and exceptions

  Show description for 50adbe7
  nathan-weinberg

  committed
  50adbe7

  Copy full SHA for 50adbe7

• Merge pull request #5 from instructlab/dependabot/github_actions/step-security/harden-runner-2.8.1

  Show description for f3a3677
  nathan-weinberg

  authored
  f3a3677

  Copy full SHA for f3a3677

• Bump step-security/harden-runner from 2.8.0 to 2.8.1

  Show description for d164b03
  dependabot[bot]

  authored
  d164b03

  Copy full SHA for d164b03

Commits on Jun 12, 2024

• Merge pull request #4 from bjhargrave/patch-1

  Show description for dc9a992
  nathan-weinberg

  authored
  dc9a992

  Copy full SHA for dc9a992

• Fixes pyproject.toml

  Show description for 0040edb
  bjhargrave

  authored
  0040edb

  Copy full SHA for 0040edb

Pagination

Previous

Next