Commits

Commit history

1. Commits on Jun 20, 2024

   • Add BertaQA dataset tasks (#1964)

     Show description for 6f7b4a0
     

     juletx

     and

     haileyschoelkopf

     authored
     6f7b4a0View commit details

     Copy full SHA for 6f7b4a0

     Browse repository at this point
2. Commits on Jun 19, 2024

   • Fix Datasets `--trust_remote_code` (#1998)

     haileyschoelkopf

     authored
     d14b36eView commit details

     Copy full SHA for d14b36e

     Browse repository at this point

   • Added ArabicMMLU (#1987)

     Show description for a08bc3c
     Yazeed7

     authored
     a08bc3cView commit details

     Copy full SHA for a08bc3c

     Browse repository at this point

   • Log `fewshot_as_multiturn` in results files (#1995)

     Show description for 78a54e1
     

     haileyschoelkopf

     and

     lintangsutawika

     authored
     78a54e1View commit details

     Copy full SHA for 78a54e1

     Browse repository at this point

   • Fix Paloma Template yaml (#1993)

     Show description for ead2964
     

     3 peopleauthored
     ead2964View commit details

     Copy full SHA for ead2964

     Browse repository at this point

   • [New Task] Add Paloma benchmark (#1928)

     Show description for f257d38
     

     zafstojano

     and

     lintangsutawika

     authored
     f257d38View commit details

     Copy full SHA for f257d38

     Browse repository at this point
3. Commits on Jun 18, 2024

   • Fix self assignment in neuron_optimum.py (#1990)

     LSinev

     authored
     bdb78d2View commit details

     Copy full SHA for bdb78d2

     Browse repository at this point

   • add trust_remote_code for piqa (#1983)

     Show description for 72bb624
     changwangss

     authored
     72bb624View commit details

     Copy full SHA for 72bb624

     Browse repository at this point
4. Commits on Jun 13, 2024

   • fix: add directory filter to os.walk to ignore 'ipynb_checkpoints' (#1956)

     Show description for 568af94
     

     johnwee1

     and

     lintangsutawika

     authored
     568af94View commit details

     Copy full SHA for 568af94

     Browse repository at this point

   • make write_out.py explicitly error if no splits match (#1796)

     Show description for ed72238
     

     haileyschoelkopf

     and

     lintangsutawika

     authored
     ed72238View commit details

     Copy full SHA for ed72238

     Browse repository at this point

   • Fix `--gen_kwargs` and VLLM (`temperature` not respected) (#1800)

     Show description for 5c7cba2
     

     haileyschoelkopf

     and

     lintangsutawika

     authored
     5c7cba2View commit details

     Copy full SHA for 5c7cba2

     Browse repository at this point

   • `samples` is newline delimited (#1930)

     Show description for 3850e21
     

     baberabb

     and

     lintangsutawika

     authored
     3850e21View commit details

     Copy full SHA for 3850e21

     Browse repository at this point
5. Commits on Jun 12, 2024

   • Fix self.max_tokens in anthropic_llms.py (#1848)

     Show description for 793469e
     nikitml

     authored
     793469eView commit details

     Copy full SHA for 793469e

     Browse repository at this point

   • Update interface.md (#1955)

     sadra-barikbin

     authored
     6f43493View commit details

     Copy full SHA for 6f43493

     Browse repository at this point
6. Commits on Jun 11, 2024

   • add hacky add_bos_token forcing for Gemma to VLLM too (#1857)

     haileyschoelkopf

     authored
     b3e4c49View commit details

     Copy full SHA for b3e4c49

     Browse repository at this point

   • add include_defaults kwarg to taskmanager, add tests for include_path (#1856)

     haileyschoelkopf

     authored
     4bb77e8View commit details

     Copy full SHA for 4bb77e8

     Browse repository at this point

   • Remove AMMLU Due to Translation (#1948)

     Show description for d0f6e01
     haileyschoelkopf

     authored
     d0f6e01View commit details

     Copy full SHA for d0f6e01

     Browse repository at this point

   • Results filenames handling fix (#1926)

     Show description for 6995258
     KonradSzafer

     authored
     6995258View commit details

     Copy full SHA for 6995258

     Browse repository at this point
7. Commits on Jun 10, 2024

   • Add the Arabic version with refactor to Arabic pica to be in alghafa folder (#1940)

     khalil-Hennara

     authored
     305fb63View commit details

     Copy full SHA for 305fb63

     Browse repository at this point
8. Commits on Jun 9, 2024

   • Update __main__.py (#1939)

     sadra-barikbin

     authored
     bea1a85View commit details

     Copy full SHA for bea1a85

     Browse repository at this point
9. Commits on Jun 7, 2024

   • Test output table layout consistency (#1916)

     Show description for 40f5458
     zafstojano

     authored
     40f5458View commit details

     Copy full SHA for 40f5458

     Browse repository at this point

   • Update basque-glue (#1913)

     Show description for 59418aa
     zhabuye

     authored
     59418aaView commit details

     Copy full SHA for 59418aa

     Browse repository at this point

   • Update siqa.yaml (#1909)

     haileyschoelkopf

     authored
     3f0ef80View commit details

     Copy full SHA for 3f0ef80

     Browse repository at this point

   • Add The Arabic version of the PICA benchmark (#1917)

     khalil-Hennara

     authored
     923852bView commit details

     Copy full SHA for 923852b

     Browse repository at this point
10. Commits on Jun 6, 2024

    • Implement NoticIA (#1912)

      Show description for f2843b2
      ikergarcia1996

      authored
      f2843b2View commit details

      Copy full SHA for f2843b2

      Browse repository at this point

    • Add new Lambada translations (#1897)

      Show description for b9d96b5
      

      4 peopleauthored
      b9d96b5View commit details

      Copy full SHA for b9d96b5

      Browse repository at this point

    • [add] fld logical formula task (#1931)

      MorishT

      authored
      33eef48View commit details

      Copy full SHA for 33eef48

      Browse repository at this point
11. Commits on Jun 5, 2024

    • Modify pre-commit hook to check merge conflicts accidentally committed not at current merge commit (#1927)

      LSinev

      authored
      e39df01View commit details

      Copy full SHA for e39df01

      Browse repository at this point

    • Multiple Choice Questions and Large Languages Models: A Case Study with Fictional Medical Data (#1867)

      Show description for 7257aa2
      maximegmd

      authored
      7257aa2View commit details

      Copy full SHA for 7257aa2

      Browse repository at this point
12. Commits on Jun 3, 2024

    • Add chat template (#1873)

      Show description for 070d31d
      

      3 peopleauthored
      070d31dView commit details

      Copy full SHA for 070d31d

      Browse repository at this point

    • Complete task list from pr 1727 (#1901)

      Show description for 3e500e9
      

      3 peopleauthored
      3e500e9View commit details

      Copy full SHA for 3e500e9

      Browse repository at this point

    • Fix fewshot seed only set when overriding num_fewshot (#1914)

      Show description for 42d5c4b
      LSinev

      authored
      42d5c4bView commit details

      Copy full SHA for 42d5c4b

      Browse repository at this point
13. Commits on May 31, 2024

    • Try to make existing tests run little bit faster (#1905)

      LSinev

      authored
      1060b68View commit details

      Copy full SHA for 1060b68

      Browse repository at this point

    • Making hardcoded few shots compatible with the chat template mechanism (#1895)

      Show description for 4902aaa
      

      clefourrier

      and

      haileyschoelkopf

      authored
      4902aaaView commit details

      Copy full SHA for 4902aaa

      Browse repository at this point

    • Add dataset card when pushing to HF hub (#1898)

      Show description for f4f5925
      

      3 peopleauthored
      f4f5925View commit details

      Copy full SHA for f4f5925

      Browse repository at this point