Conversation
Nikon's HE NEF files store the raw data as a JPEG XS codestream, and reading one means pulling single bits, nibbles and whole bytes out of the same stream. This reader does all three, and fails on a bad read instead of handing back junk that looks fine.
Nikon's headers differ from the standard in three spots, but only one of them matters: every component carries a third byte saying how it was decomposed. Plain JPEG XS assumes only the last component can skip the wavelet, and Nikon's is component 2.
Bands are ordered by component, packets by subband, and the weights table follows the packet order. Get that mapping wrong and nothing complains, because the code is unary: wrong values still take up the right number of bits, so the file parses cleanly and only the pixels come out wrong. A test pins the mapping down.
Nikon always uses the long packet header, even where the standard's rule of thumb would pick the short one, and it writes sign bits for coefficients that run past the end of a short last group. Those have to be skipped, or everything after them comes out with the wrong sign. The tests compare against the C reference decoder; point RAWLER_JPEGXS_REFERENCE at its output directory to run them.
A precinct holds two plane lines, but finishing the second one needs data from the precinct after it, so the output runs one precinct behind the input. Each component says for itself whether it skips the wavelet, so the code reads that rather than assuming it is the last one. Nikon's is component 2.
The four components are a star-tetrix decorrelation of the Bayer planes rather than the planes themselves, and the header never says so: Cpih reads 0 and the markers that would carry the parameters aren't there. Nikon also scales the chroma planes and square-root compands the samples. Those constants are fitted from a single scene, so they're approximate, and there's no telling whether they hold for other cameras.
These files used to be rejected outright; now they decode, and the result correlates 0.99 with a lossless shot of the same scene. Only tested on a Z6III.
|
I'm not a maintainer here, but I would really love to see your notes on the format, so that I could learn from your work and potentially help implement it elsewhere. You could send them to gimlets_threads_41@icloud.com |
This branch is waiting to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I kept forgetting to switch my Z6III back off High Efficiency and then couldn't open the files, so I had a go at working out the format, with a lot of help from AI. Turns out it seems to be just JPEG XS (ISO/IEC 21122) with a few vendor modifications...
Disclaimer: I'm not a Rust programmer, more at home in embedded C and Python, so the implementation is largely AI-written too. Please review it properly and let me know if this is an issue
I've tested the code on NEFs from my Z6III, where the output seems to match with a lossless NEF of the same picture. I'd guess the Z8 and Z9 work too, but I haven't tried them (and can't as I don't own any of these bodies). So if someone with files from these bodies can test it, it would be much appreciated!
Happy to share my notes on the format if anyone's interested, separate from the code.
Should fix #591 (and hopefully bring NEF HE support to rapidraw) 😄