Tags: foxglove/mcap
Tags
list schemas: print out protobuf nested types and enum definitions if… … present (#1507) … ### Changelog <!-- Write a one-sentence summary of the user-impacting change (API, UI/UX, performance, etc) that could appear in a changelog. Write "None" if there is no user-facing change --> ### Docs <!-- Link to a Docs PR, tracking ticket in Linear, OR write "None" if no documentation changes are needed. --> ### Description `mcap list schemas` now more faithfully renders proto definitions from their file descriptor sets. This includes enum definitions, nested message (and enum) definitions, imports and file names. We no longer attempt to provide all definitions in a single logical "file", instead we put each file's definition in it's own file (with its own syntax, package and filename) separated by a `===` delimiter. <!-- Describe the problem, what has changed, and motivation behind those changes. Pretend you are advocating for this change and the reader is skeptical. --> <!-- In addition to unit tests, describe any manual testing you did to validate this change. --> ### Before ``` syntax = "proto3"; message cardboard.Cost { optional dollars int32 = 1; optional cents int32 = 2; } message cardboard.Box { optional aesthetics .cardboard.Box.Aesthetics = 1; optional cost .cardboard.Cost = 2; } message shipping.Item { optional box .cardboard.Box = 1; optional grams int32 = 2; } ``` ### After ``` // cardboard.proto syntax = "proto3"; package cardboard; message Cost { optional int32 dollars = 1; optional int32 cents = 2; } message Box { message Aesthetics { enum Color { RED = 0; BLUE = 1; GREEN = 2; } enum Shape { SQUARE = 0; RECTANGULAR = 1; HEXAGONAL = 2; } optional .cardboard.Box.Aesthetics.Color color = 1; optional .cardboard.Box.Aesthetics.Shape shape = 2; } optional .cardboard.Box.Aesthetics aesthetics = 1; optional .cardboard.Cost cost = 2; } ================================================================================ // shipping.proto syntax = "proto3"; package shipping; import "cardboard.proto"; message Item { optional .cardboard.Box box = 1; optional int32 grams = 2; } ``` <table><tr><th>Before</th><th>After</th></tr><tr><td> <!--before content goes here--> </td><td> <!--after content goes here--> </td></tr></table> <!-- If necessary, link relevant Linear or Github issues. Use `Fixes: foxglove/repo#1234` to auto-close the Github issue or Fixes: FG-### for Linear isses. --> Closes: #1503 <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Improve `mcap list schemas` to faithfully render protobuf files, including imports, enums, and nested messages, separated per file. > > - **CLI (schemas printing)**: > - Render protobuf descriptors per file with headers (`// filename`, `syntax`, `package`, `import`), separated by an `====` divider. > - Include enum definitions and recursively print nested messages/enums via `printDescriptorEnum` and `printDescriptorMessage`. > - **Tests**: > - Add unit tests validating single-file and dependency cases using `.pb.bin` descriptors. > - Add proto fixtures in `go/cli/mcap/testdata/` for `cardboard.proto` and `shipping.proto`. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 4e260c2. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
cli: fix attachment log and creation time CLI args (#1498) ### Changelog - Fixed a bug where `mcap add attachment --log-time` would fail with a parse error. - Fixed a bug where `mcap add attachment --creation-time` would set the wrong creation time. ### Docs None. ### Description Fixes two bugs in the parsing logic for `mcap add attachment` creation time and log time. <!-- Describe the problem, what has changed, and motivation behind those changes. Pretend you are advocating for this change and the reader is skeptical. --> <!-- In addition to unit tests, describe any manual testing you did to validate this change. --> <table><tr><th>Before</th><th>After</th></tr><tr><td> <!--before content goes here--> </td><td> <!--after content goes here--> </td></tr></table> <!-- If necessary, link relevant Linear or Github issues. Use `Fixes: foxglove/repo#1234` to auto-close the Github issue or Fixes: FG-### for Linear isses. --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Fixes `mcap add attachment` to parse `--log-time` correctly and bind `--creation-time` to the proper variable. > > - **CLI** > - **Add Attachment** (`go/cli/mcap/cmd/attachment.go`): > - Parse `--log-time` using `addAttachmentLogTime` (was using creation-time var). > - Bind `--creation-time` flag to `addAttachmentCreationTime` (was bound to log-time var). > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit cdf15f7. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
rust: fix notifying EOF after end magic has been parsed (#1491) ### Changelog - rust: fix error when notifying `LinearReader` of EOF after end magic has been parsed. - rust: add `LinearReader` option to check for trailing garbage bytes after end magic. ### Docs None ### Description When `skip_end_magic` is false, `LinearReader::next_event()` currently returns an error when EOF is reached even if the end magic has been seen. This PR makes it return `None` instead. An option has also been added to check that there are no trailing bytes after the end magic. Users who care about the end magic being present probably also care that there is nothing after it, though it is not on by default because that would be a breaking change. The documentation for `skip_end_magic` has been updated to clarify that it is about expecting the end magic after the footer record, not at the end of the stream. Unit tests have been added to cover this behavior, and I have verified that they fail with the old code. As part of this change, `RwBuf`'s fields have been made private to protect its invariants from accidental violations. This bug was uncovered because some code I was testing found an underflow panic due to the `end - start` calculation in `tail_with_size()` (which should never happen with `LinearReader`), so as part of trying to localize that I made `RwBuf` more strictly encapsulated. I haven't yet been able to reproduce that panic in tests, but I've left in the associated changes since I think they're probably an improvement. There are still some unfixed problems with `LinearReader`'s EOF handling, but those require deeper restructuring. Currently it eagerly fails with `UnexpectedEof` if anything has been buffered and EOF has been notified, even if it could still finish parsing the file with what has been supplied. A naive attempt by deleting the top-level EOF check and putting EOF checks into `load!()` and `consume!()` causes tests to fail, which I'm not yet sure whether it indicates a bug.
Add S3 support and refactor related code for improved modularity (#1472) ### Changelog Add modular reader system with support for S3 and GCS, replacing the old GCS-only implementation with a unified `readers` registry. ### Docs None ### Description This change refactors how remote and local files are accessed in the MCAP CLI. Previously, only Google Cloud Storage (`gs://`) was supported via a custom `GCSReadSeekCloser`. Now, a **modular reader architecture** provides unified support for: - Local files - Google Cloud Storage (GCS) - Amazon S3 Key updates: - Added `utils/readers` package containing: - `registry.go` — manages reader registration and lookup - `local.go` — handles local file reading - `gcs.go` — refactored GCS reader (replaces old `gcs_read_seek_closer.go`) - `s3.go` — new AWS S3 reader using AWS SDK v2 - Updated `utils.go` to delegate all reading logic to the new registry - Removed the legacy GCS-only implementation - Added AWS SDK v2 dependencies --------- Co-authored-by: Artem Sokharev <asokhare@ford.com> Co-authored-by: Bennett Hardwick <hello@bennett.dev>
go: fix statistics writing when channel for channel count doesn't exi… …st (#1477) ### Changelog <!-- Write a one-sentence summary of the user-impacting change (API, UI/UX, performance, etc) that could appear in a changelog. Write "None" if there is no user-facing change --> None ### Docs <!-- Link to a Docs PR, tracking ticket in Linear, OR write "None" if no documentation changes are needed. --> None ### Description Fixes a bug with statistics writing. It was expecting there to be more channel counts than were actually written. This was causing issues with mcap recover, where the statistics record may be populated directly instead of using write message. Fixes FIRE-186. Fixes #1474.
go: fix statistics writing when channel for channel count doesn't exi… …st (#1477) ### Changelog <!-- Write a one-sentence summary of the user-impacting change (API, UI/UX, performance, etc) that could appear in a changelog. Write "None" if there is no user-facing change --> None ### Docs <!-- Link to a Docs PR, tracking ticket in Linear, OR write "None" if no documentation changes are needed. --> None ### Description Fixes a bug with statistics writing. It was expecting there to be more channel counts than were actually written. This was causing issues with mcap recover, where the statistics record may be populated directly instead of using write message. Fixes FIRE-186. Fixes #1474.
doctor: don't error on schema id 0 missing from summary section (#1453) ### Changelog Fixed a bug in `mcap doctor` which would raise an error for schemaless channels. ### Docs None ### Description Previously a schemaless channel would result in errors like: ``` Indexed chunk at offset 42 contains messages referencing schema (0) not duplicated in summary section ```
PreviousNext