Skip to content

Tags: Lightning-AI/litData

Tags

v0.2.76

Toggle v0.2.76's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
chore: bump version to 0.2.76 (#935)

Co-authored-by: Codex <noreply@openai.com>

v0.2.75

Toggle v0.2.75's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
chore: bump version to 0.2.75 (#900)

Co-authored-by: Cursor <cursoragent@cursor.com>

v0.2.74

Toggle v0.2.74's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
chore: bump version to 0.2.74 after Hub streaming merge (#898)

Co-authored-by: Cursor <cursoragent@cursor.com>

v0.2.73

Toggle v0.2.73's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
chore: bump version 0.2.73 (#895)

Release 0.2.73 with the optimize pickle memory fix (#890), multi-node append index merge (#866), and DataLoader worker-loop restore after profiling (#893).

Co-authored-by: Cursor <cursoragent@cursor.com>

v0.2.72

Toggle v0.2.72's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
chore: bump version 0.2.72 (#892)

Release 0.2.72 with the S3 credential refresh reliability work (#891)
and the numpy deserialize copy fix (#887).

Also corrects the changelog: the multi-node FullShuffle fix and the
max_cache_size change were left under [unreleased] by #886, but #886 is
the commit v0.2.71 was tagged at, so they shipped in 0.2.71. Moved them
into that section and added the missing entry for #887.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

v0.2.71

Toggle v0.2.71's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
feat(streaming): pin node shards when they fit, else reshuffle global…

…ly (#886)

* feat(streaming): pin node shards when they fit, else reshuffle globally

Keep unique chunks on a node across epochs when they fit in cache, and default max_cache_size to a conservative slice of free disk so checkpoints still have room.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs: keep shuffle and cache-size notes under unreleased

Co-authored-by: Cursor <cursoragent@cursor.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tests: drop unused shuffle expected list

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(streaming): cache shuffle chunk count and drop duplicate resume line

Co-authored-by: Cursor <cursoragent@cursor.com>

* tests: pin emulate-S3 cache size so CI does not evict under mmap

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(streaming): accept max_cache_size as size or free-disk fraction

None still uses 75% of free disk with a 50GB reserve. "100G" pins bytes;
0.90 uses that share of currently free space.

Co-authored-by: Cursor <cursoragent@cursor.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* style: drop escaped quotes in max_cache_size docstring for ruff D301

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

v0.2.70

Toggle v0.2.70's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
bump version 0.2.70 (#883)

Co-authored-by: Cursor <cursoragent@cursor.com>

v0.2.69

Toggle v0.2.69's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
bump version 0.2.69 (#881)

Co-authored-by: Cursor <cursoragent@cursor.com>

v0.2.68

Toggle v0.2.68's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
bump version 0.2.68 (#877)

* bump version 0.2.68

Align skills/README with POSIX-fast: WindowShuffle only on parallel FS,
worker/WILLNEED RAM caps, and TokensLoader copies off mmap.

Co-authored-by: Cursor <cursoragent@cursor.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

v0.2.67

Toggle v0.2.67's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
bump version 0.2.67 (#864)