Skip to content

//zml/io/vfs: promote to //vfs#680

Open
gwenzek wants to merge 7 commits into
masterfrom
gw/vfs
Open

//zml/io/vfs: promote to //vfs#680
gwenzek wants to merge 7 commits into
masterfrom
gw/vfs

Conversation

@gwenzek

@gwenzek gwenzek commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Proposals:

  1. simplify vfs architecture: merge //zml/io and //zml/io/vfs. They have the same deps, and we only use //zml/io. New name: //zml/vfs

  2. move vfs to top level (it becomes //vfs)

  3. expose vfs and stdx to any zig user through zig fetch by adding build.zig and build.zig.zon

All of the above are no-op for current consumer of zml, they can still access vfs through zml.io.VFS

The PR also include two bug fix:

  • S3: when no content length we were trying to read exactly 1M, while we should read up to 1M
  • GCP: restore the commented code for parsing bucket info

Test plan:

bazel run --config=debug //examples/io:playground -- ls gs://gcp-public-data-landsat/
bazel run --config=debug //examples/io:playground -- tree s3://noaa-goes19/ABI-Flood-Day-Shapefiles/2025/08

zig build run_vfs_example -- tree s3://noaa-goes19/ABI-Flood-Day-Shapefiles/2025/08
zig build run_vfs_example -- ls gs://gcp-public-data-landsat/

@gwenzek gwenzek changed the title Gw/vfs zml/io/vfs: promote to vfs Jul 22, 2026
@gwenzek gwenzek changed the title zml/io/vfs: promote to vfs //zml/io/vfs: promote to //vfs Jul 22, 2026
@gwenzek
gwenzek marked this pull request as ready for review July 22, 2026 18:50
@gwenzek
gwenzek requested review from Corendos and hugomano July 22, 2026 18:50
Comment thread vfs/example.zig

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a fork of examples/io/main.zig that removes all the tensor related stuff, and therefore has no deps beyond vfs.
I don't know if it's worth maintaining the two of them.

Comment thread examples/io/main.zig
try stdout_writer.interface.print("Wrote {B:.2} to stdout from {s}\n", .{ read, path });
},
.ls => {
.ls, .tree => {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the default ls would take forever if you give it a huge bucket because of the recursive nature of printTree. I've make ls more similar to the Posix version by being non-recursive, and tree keeps ls previous behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant