Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on Jan 25, 2026

• Merge pull request #14 from ajroetker/fix/unigram-vocab-parsing

  Show description for b0f56ca
  janpfeifer

  authored
  b0f56ca

  Copy full SHA for b0f56ca

• Upgrade go-sentencepiece to v0.7.0 and optimize Encode

  Show description for 551b072
  ajroetker

  committed
  551b072

  Copy full SHA for 551b072

• Support BPE merges in array-of-arrays format

  Show description for 3cac291
  ajroetker

  committed
  3cac291

  Copy full SHA for 3cac291

• Fix Unigram vocab parsing to use array index as token ID

  Show description for d3987c0
  ajroetker

  committed
  d3987c0

  Copy full SHA for d3987c0

Commits on Jan 24, 2026

• Merge pull request #13 from ajroetker/feature/tokenizer-encode-with-offsets

  Show description for be83641
  janpfeifer

  authored
  be83641

  Copy full SHA for be83641

Commits on Jan 23, 2026

• Rename TokenOffset to TokenSpan and clarify byte positions

  Show description for 319caa4
  ajroetker

  committed
  319caa4

  Copy full SHA for 319caa4

Commits on Jan 22, 2026

• Add EncodeWithOffsets for token classification tasks

  Show description for abfce8b
  ajroetker

  committed
  abfce8b

  Copy full SHA for abfce8b

• Merge pull request #12 from ajroetker/feature/hftokenizer

  Show description for 50988b6
  janpfeifer

  authored
  50988b6

  Copy full SHA for 50988b6

• Improve hftokenizer: add Unicode normalization and BPE tests

  Show description for 9a9c308
  ajroetker

  committed
  9a9c308

  Copy full SHA for 9a9c308

• Add hftokenizer package for HuggingFace tokenizer.json format

  Show description for 6e1cb34
  ajroetker

  committed
  6e1cb34

  Copy full SHA for 6e1cb34

Commits on Jan 17, 2026

• Merge pull request #11 from gomlx/safetensors-fixes

  Show description for b09d810
  janpfeifer

  authored
  b09d810

  Copy full SHA for b09d810

• Typo.

  janpfeifer

  committed
  2aed4ac

  Copy full SHA for 2aed4ac

• Added IterTensorsFromRepo.

  janpfeifer

  committed
  b06dd67

  Copy full SHA for b06dd67

• Renamed `safetensor` -> `safetensors`

  janpfeifer

  committed
  01edac6

  Copy full SHA for 01edac6

• - Renaming to more Go idiomatic names;

  Show description for 822d899
  janpfeifer

  committed
  822d899

  Copy full SHA for 822d899

• Merge pull request #10 from siherrmann/main

  Show description for 6684cb9
  janpfeifer

  authored
  6684cb9

  Copy full SHA for 6684cb9

Commits on Jan 16, 2026

• updated Load function signature

  siherrmann

  committed
  0daacf2

  Copy full SHA for 0daacf2

• shorten tensor creation

  siherrmann

  committed
  3a2964e

  Copy full SHA for 3a2964e

• update safetensor method names, unify tensor reading, clean up, add tests for reader

  siherrmann

  committed
  0881523

  Copy full SHA for 0881523

Commits on Jan 14, 2026

• fix model check

  siherrmann

  committed
  65ce551

  Copy full SHA for 65ce551

• clean up safetensors, refactor into new folder structure

  siherrmann

  committed
  e71dfb4

  Copy full SHA for e71dfb4

Commits on Jan 13, 2026

• add load safetensors from files functionality

  siherrmann

  committed
  a8de60a

  Copy full SHA for a8de60a

Commits on Nov 11, 2025

• Updated CHANGELOG; bumping to v0.3.1.

  janpfeifer

  committed
  930a356

  Copy full SHA for 930a356

• go mod tidy.

  janpfeifer

  committed
  80c8a1c

  Copy full SHA for 80c8a1c

• Added missing go.sum dependency.

  janpfeifer

  committed
  33398c6

  Copy full SHA for 33398c6

• Merge pull request #9 from gomlx/v03

  Show description for aae1dc7
  janpfeifer

  authored
  aae1dc7

  Copy full SHA for aae1dc7

• Creating v0.3.0

  janpfeifer

  committed
  52dcf7f

  Copy full SHA for 52dcf7f

• Fixed notebook imports.

  janpfeifer

  committed
  db79919

  Copy full SHA for db79919

Commits on Oct 21, 2025

• Fixed comments.

  janpfeifer

  committed
  02f49c4

  Copy full SHA for 02f49c4

Commits on Jun 27, 2025

• Bump version number.

  janpfeifer

  committed
  b1ef8b6

  Copy full SHA for b1ef8b6

Commits on Jun 26, 2025

• Update CHANGELOG.md

  janpfeifer

  authored
  e515b14

  Copy full SHA for e515b14

• Merge pull request #8 from RJKeevil/download-fix

  Show description for 7f3a938
  janpfeifer

  authored
  7f3a938

  Copy full SHA for 7f3a938

• fix file truncation

  RJKeevil

  committed
  760fa2f

  Copy full SHA for 760fa2f

Commits on Jun 3, 2025

• Merge pull request #7 from gomlx/refresh

  Show description for ee5a011
  janpfeifer

  authored
  ee5a011

  Copy full SHA for ee5a011

• Formatting normalization.

  janpfeifer

  committed
  668a864

  Copy full SHA for 668a864

Pagination

Previous

Next