feat: add header and compressed string validation functions#48
Open
mhx wants to merge 1 commit into
Open
Conversation
The interface for `fsst_import` relies on being passed a pointer to a well-formed, valid header that was generated by `fsst_export`. Passing it a corrupted header can easily lead to out-of-bounds accesses. The same is true for `fsst_decompress`, albeit to a lesser extent. The function already takes care of not writing past the end of `output`, but it can read past the end of `strIn` if the last byte of the compressed string is (erroneously) an escape byte. In practice, this will only be the case if the compressed data is corrupted. I'm using FSST in a file system implementation and I'm currently in the process of hardening the implementation against out-of-bounds accesses wherever possible. For "trusted" file system images, internal checksums take care of corruption / bit-rot. But for "untrusted", potentially malicious, images with forged checksums, I'd like to make sure to catch all errors that could lead to OOB accesses during a full file system check. The `fsst_validate_header()` call is cheap enough to be always-on. In order to not impact API and/or performance, the validation checks are implemented as separate functions that can be called on demand.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The interface for
fsst_importrelies on being passed a pointer to a well-formed, valid header that was generated byfsst_export. Passing it a corrupted header can easily lead to out-of-bounds accesses.The same is true for
fsst_decompress, albeit to a lesser extent. The function already takes care of not writing past the end ofoutput, but it can read past the end ofstrInif the last byte of the compressed string is (erroneously) an escape byte. In practice, this will only be the case if the compressed data is corrupted.I'm using FSST in a file system implementation and I'm currently in the process of hardening the implementation against out-of-bounds accesses wherever possible. For "trusted" file system images, internal checksums take care of corruption / bit-rot. But for "untrusted", potentially malicious, images with forged checksums, I'd like to make sure to catch all errors that could lead to OOB accesses during a full file system check. The
fsst_validate_header()call is cheap enough to be always-on.In order to not impact API and/or performance, the validation checks are implemented as separate functions that can be called on demand.