Tags: openwd/mcap
Tags
rust: write attachments in parts (foxglove#1285) ### Changelog <!-- Write a one-sentence summary of the user-impacting change (API, UI/UX, performance, etc) that could appear in a changelog. Write "None" if there is no user-facing change --> rust: support writing attachments in multiple chunks ### Docs <!-- Link to a Docs PR, tracking ticket in Linear, OR write "None" if no documentation changes are needed. --> None ### Description This change supports writing large attachments to MCAP files without the need to load them into memory. Currently, in order to write an attachment, you need to load the entire file into memory, create an `Attachment` record, then use that with the `attach` method on the writer. This means that it is not possible to attach files when the file exceeds the available memory on the computer. This change adds The methods `start_attachment`, `put_attachment_bytes` and `finish_attachment` for interactively writing an attachment without needing to buffer the file in memory. It looks like this: ```rust let writer = Writer::new(..); let attachment_length = ..; let header = AttachmentHeader { .. }; writer.start_attachment(attachment_length, header)?; writer.put_attachment_bytes(..)?; writer.put_attachment_bytes(..)?; writer.put_attachment_bytes(..)?; writer.finish_attachment()?; ``` I've updated the current `attach` method to use this approach and the roundtrip tests pass. I've also added some tests for using this API directly. <!-- Describe the problem, what has changed, and motivation behind those changes. Pretend you are advocating for this change and the reader is skeptical. --> <!-- In addition to unit tests, describe any manual testing you did to validate this change. --> <table><tr><th>Before</th><th>After</th></tr><tr><td> attachments must be loaded into memory to be written <!--before content goes here--> </td><td> attachments can be written in parts <!--after content goes here--> </td></tr></table> <!-- If necessary, link relevant Linear or Github issues. Use `Fixes: foxglove/repo#1234` to auto-close the Github issue or Fixes: FG-### for Linear isses. --> --------- Co-authored-by: james-rms <james@foxglove.dev>
rust: release version 0.12.0 (foxglove#1281) ### Changelog <!-- Write a one-sentence summary of the user-impacting change (API, UI/UX, performance, etc) that could appear in a changelog. Write "None" if there is no user-facing change --> None ### Docs <!-- Link to a Docs PR, tracking ticket in Linear, OR write "None" if no documentation changes are needed. --> None ### Description Bump the Rust code to 0.12.0 to release the linear writer feature. <!-- Describe the problem, what has changed, and motivation behind those changes. Pretend you are advocating for this change and the reader is skeptical. --> <!-- In addition to unit tests, describe any manual testing you did to validate this change. --> <table><tr><th>Before</th><th>After</th></tr><tr><td> <!--before content goes here--> </td><td> <!--after content goes here--> </td></tr></table> <!-- If necessary, link relevant Linear or Github issues. Use `Fixes: foxglove/repo#1234` to auto-close the Github issue or Fixes: FG-### for Linear isses. -->
filter: fix end second parsing (foxglove#1269) ### Changelog - Fixed: the `-e` argument to `mcap filter` will be interpreted properly again. ### Docs None. ### Description The last change to `filter` mistakenly swapped the `nanos` and `seconds` arguments to `parseTimestampArgs` for the filter end time. ### Testing - Tested `mcap filter -s 10 -e 1000` manually, previously failed, now works. Fixes: foxglove#1268
CLI: accept RFC3339-formatted dates in relevant arguments (foxglove#1263 ) ### Changelog #### Added CLI: `mcap add attachment` accepts RFC3339-formatted dates to specify the log and creation times for the new attachment. CLI: `mcap filter` accepts RFC3339-formatted dates for start and end times. ### Docs None. ### Description Adds the ability for users to specify a date when filtering MCAP files, or adding attachments. I did a quick pass to see if there are any other arguments where we accept a date or date-like value, I can't see any more.
Add missing tslib dependencies (foxglove#1264) ### Changelog TypeScript: Added missing dependencies on `tslib` ### Docs None ### Description Fixes foxglove#1262 When `importHelpers` is true in tsconfig.json (as it is by default with `@foxglove/tsconfig`), `tslib` must be a dependency.
Add missing tslib dependencies (foxglove#1264) ### Changelog TypeScript: Added missing dependencies on `tslib` ### Docs None ### Description Fixes foxglove#1262 When `importHelpers` is true in tsconfig.json (as it is by default with `@foxglove/tsconfig`), `tslib` must be a dependency.
Add missing tslib dependencies (foxglove#1264) ### Changelog TypeScript: Added missing dependencies on `tslib` ### Docs None ### Description Fixes foxglove#1262 When `importHelpers` is true in tsconfig.json (as it is by default with `@foxglove/tsconfig`), `tslib` must be a dependency.
rust: add sans-io linear reader (foxglove#1244) ### Changelog - Rust: Added a `sans_io` module to allow users to read MCAP data while remaining agnostic to I/O. Replace the internal implementation details of both the memory-mapped I/O and tokio-async read modules with this sans_io reader. ### Docs See generated rust docs. ### Description As we need more functionality from the async MCAP reader, I feel less comfortable adding those and not also ensuring they're available for the memory-mapped reader. This PR strips out all of that code and moves it into a [sans-io](https://sans-io.readthedocs.io/) reader module. The sans-io reader is a state machine with no dependency on any particular I/O framework. <!-- In addition to unit tests, describe any manual testing you did to validate this change. --> <table><tr><th>Before</th><th>After</th></tr><tr><td> <!--before content goes here--> </td><td> <!--after content goes here--> </td></tr></table> <!-- If necessary, link relevant Linear or Github issues. Use `Fixes: foxglove/repo#1234` to auto-close the Github issue or Fixes: FG-### for Linear isses. --> --------- Co-authored-by: Jacob Bandes-Storch <jacob@foxglove.dev>
go: annotate unexpected token errors (foxglove#1247) ### Changelog <!-- Write a one-sentence summary of the user-impacting change (API, UI/UX, performance, etc) that could appear in a changelog. Write "None" if there is no user-facing change --> Adds in a `UnexpectedTokenError` error class ### Docs <!-- Link to a Docs PR, tracking ticket in Linear, OR write "None" if no documentation changes are needed. --> None ### Description Adds in an `UnexpectedTokenError` error class for when the reader expects a certain record but something else was retrieved. It could be useful to know when this error happens to differentiate it from io-related errors. <!-- Describe the problem, what has changed, and motivation behind those changes. Pretend you are advocating for this change and the reader is skeptical. --> <!-- In addition to unit tests, describe any manual testing you did to validate this change. --> <table><tr><th>Before</th><th>After</th></tr><tr><td> invalid header / metadata come through as generic errors <!--before content goes here--> </td><td> <!--after content goes here--> invalid header / metadata errors can be differentiated </td></tr></table> <!-- If necessary, link relevant Linear or Github issues. Use `Fixes: foxglove/repo#1234` to auto-close the Github issue or Fixes: FG-### for Linear isses. -->
python: specify packages to install for each module (foxglove#1246) ### Changelog - Fixed: python packages `mcap`, `mcap-protobuf-support`, `mcap-ros1-support`, `mcap-ros2-support` now only install their main packages rather than all packages including tests. - Fixed: Add type annotations to `__init__` methods with no arguments implementing this guide: https://mypy.readthedocs.io/en/stable/class_basics.html#annotating-init-methods ### Docs <!-- Link to a Docs PR, tracking ticket in Linear, OR write "None" if no documentation changes are needed. --> ### Description Rely on setuptools' auto-discovery mechanism to find the python packages we're trying to expose while auto-excluding `tests`, which were previously being included into users' install directories. ### Testing - Inspected wheel contents using `python -m zipfile --list mcap_protobuf_support-0.5.2-py3-none-any.whl` to ensure that the whole library content is present and no test directories are present.
PreviousNext