Skip to content

Bound pack-index caching and verify replacement detection - #71

Merged
aman-a-shah merged 2 commits into
mainfrom
55-week-8---m1-measured-storage-optimizations-packfile-regression-checks
Nov 18, 2026
Merged

aman-a-shah merged 2 commits into
mainfrom
55-week-8---m1-measured-storage-optimizations-packfile-regression-checks

Conversation

@mahis1067

@mahis1067 mahis1067 commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Pack metadata now uses a 64-entry least-recently-used cache. The signature includes each pack/index file's size, modification time, and inode, so atomic replacement invalidates prior validation even when size and modification time are preserved. Selected records remain decoded and hash-checked on every read.

Adds eviction and replacement regressions, preserves source imports in CLI subprocess tests, and handles platform-specific executable-bit, filename, and symlink constraints in the existing tests. Packfile documentation states the cache's immutability assumptions and per-index memory limits.

The performance report includes commands, exact production revision, environment, fixture shape, and cached/uncached measurements. Earlier Windows observations are distinguished from the reproduced run. The 512-blob fixture does not complete the shared project/network benchmark.

Related to #55. Shared fixture and final integration acceptance remain tracked in #47 and #58; this PR does not close those prerequisites.

Validation

  • scripts/quality-check.sh: lint, formatting, and 445 tests pass on Python 3.13.5/macOS.
  • scripts/benchmark_packfiles.py --profile and --no-cache --profile: sequential runs against production code at d00685dc1d853c22eb3bb744db23929b3d6fcecf, with commands/results in docs/performance.md.
  • Cache regressions cover eviction, discovery, corrupt data, and pack/index replacement preserving size and modification time.
  • Windows-specific skips were inspected; Windows execution was not repeated in this validation.

Checklist

  • The change is focused and easy to review.
  • Tests were added or updated for behavior changes.
  • Documentation matches the delivered behavior and measured scope.
  • No secrets, generated artifacts, or local-only files are included.

@mahis1067 mahis1067 linked an issue Nov 15, 2026 that may be closed by this pull request
5 tasks
@aman-a-shah aman-a-shah changed the title intial commit Bound pack-index caching and verify replacement detection Nov 18, 2026

@aman-a-shah aman-a-shah left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the bounded cache, metadata invalidation, selected-record validation, and publication guarantees. Added regressions for replaced pack/index files retaining the original size and modification time. Lint, formatting, and all 445 tests pass locally, including the network integration tests. Reproduced the cached/uncached storage fixture sequentially and corrected the scope and revision claims in the performance documentation. Shared project/network measurements remain tracked in #47/#58.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Week 8 - M1: Measured storage optimizations + packfile regression checks

2 participants