Add Nikon Z9 High-Efficiency (HE) RAW format support - #826

Great, thanks!

We will need additional time for testing, as we are currently busy testing the Sony Compressed/CompressedHQ decoder: #824

Yeah, it's quite a chonker, so I imagine it'll take a bit to chew through. But does seem to be working with the NEFs I tried from my camera, and figured might be generally useful.

Libraw already has the decoder for Nikon HE/HE* (Tico Raw) for quite some time (since 2024)- it’s just not publicly released yet. It is used in FRV and RawDigger

Wow, nice to see a working decoder for these formats.
Is Star-Tetrix color transformation used in any of the HE/HE* formats? I don't see a reference in the code but IIRC this transformation was used somewhere.

Also wonder: Does anyone know a source for some sample files with that compression? And might this be the one also used for the NEV video files? (I once looked at it with a hex editor and it looked roughly like JPEG XS with maybe a different/custom header.) Given that the company that provided the NEV codec was participating in JPEG XS standardization I'd guess it isn't unlikely to be close.

Star-Tetrix color transformation used in any of the HE/HE* formats? I don't see a reference in the code but IIRC this transformation was used somewhere

Nikon as it is widely documented uses Tico RAW (TICO own Jpeg XS implementation with a lot of quirks) but only a small subset of the Jpeg XS applicable to RAW files. Star-tetrix is a way to recombine components of the RGB image so it is implicitly there.

And might this be the one also used for the NEV video files

No - this is different part of the converter (Jpeg XS) and requires different processing from how RAW images are compressed. Underlying wavelet algorithms are essentially the same across CR3, Nikon HE and potentially new Sony compression, all being wavelet and generically adhereing to Jpeg XS. The container, stream organisation, entropy encoding and image reconstruction are all quite different

If there's already a working decoder available, then makes sense to go with that. Definitely feel free to take whatever is actually useful from the PR and gut the rest.

If there's already a working decoder available, then makes sense to go with that. Definitely feel free to take whatever is actually useful from the PR and gut the rest.

Well, as Alexey pointed out, it's not available (public). I will port your implementation to Rust into dnglab/rawler as soon as I find some time. Thanks again for the work.

Well, as Alexey pointed out, it's not available (public).

Not yet, I'll leave it to LibRaw guys to comment on when it will be released. The one we have though does decode all Nikon lossy Jpeg XS compressions so far AFAIK.

We will most likely include HE/HE* decoder in the next public snapshot this Fall

We will most likely include HE/HE* decoder in the next public snapshot this Fall

In case it helps, I don't see Nikon ZR files in raw.pixls, so hope these help: https://user.fm/files/v2-90f7b233340855ea50a052c0ac194c67/Nikon%20ZR.zip

Great work @yogthos. It works really well. I may implement some performance improvements in due course.
Depends if/when the LibRaw version arrives first.

@OneOddPhoton thanks! and glad it's coming in handy. I've been using it myself, and it's been nice for making my own tooling for processing photos from my camera. The whole thing started cause I wanted to see if I could just make my own version of Topaz using open source libraries and models for doing noise clean up, and I ran into the whole issue of there not being an open implementation of the codec. So, that's how I ended up in this rabbit hole in the first place. :)

1. Incorrect reconstruction in the last two rows

@yogthos @y-g-jiang

I've been putting together a format reference for Nikon HE/HE*, comparing Nicolai Buchwitz's dnglab work with this decoder. Testing PR head 499bfd4 against Adobe DNG Converter turned up three implementation issues and an exact replacement for the approximate transfer curve. I'll cover them in four comments, starting with the reconstruction at the bottom of the image.

In the nine NEFs I checked, every difference larger than one 14-bit raw-value step was confined to the final two rows. The cause is in step1_merge_4_to_2, which recalculates the next predicted value from clamped source rows. At the bottom of the image, that uses the final delta row twice and drops the preceding row's contribution.

The boundary needs to extend the already calculated intermediate instead: T[h,c] = T[h-1,c].

At the last component row, reuse the current prediction in each of the three calculations:

// First column:
if (is_last_tile && r == w_rows - 1)
    lh_next_predicted = lh_predicted;

// Carry initialization:
if (is_last_tile && r == w_rows - 1)
    carry_next_pred = carry_pred;

// Remaining columns:
if (is_last_tile && r == w_rows - 1)
    next_predict = cur_predict;

I checked nine NEFs from Z9, Z8 and Z6III bodies, including DX and 16:9 crops, against uncompressed raw mosaics from Adobe DNG Converter 17.5.1 (2319). Before this change, the maximum difference in each file ranged from 18 to 443 raw-value steps. This change reduces the maximum to one in every file, using the existing lookup curve. Interior rows are unchanged. Both comparison builds include the lookup bounds correction I'll describe in comment 2.

With the exact transfer in comment 3, both this C++ path and an independent whole-plane reconstruction match Adobe at every sample in those nine files and two additional Zf/Z5II files.

2. Out-of-bounds read when building the lookup table

There's an out-of-bounds read in iqx_iqp_lut(). When i reaches 81791, the loop advances k to 255, then reads kIqxIqpBreakpoints[256] for xb and yb. The array has 256 entries, indexed 0 through 255.

Keeping k at most 254 fixes it:

while (k < 254 && kIqxIqpBreakpoints[k + 1][0] <= i) ++k;

The last entry then interpolates to the explicit endpoint, 65534. All preceding entries stay the same. I reproduced the original overread with AddressSanitizer and UndefinedBehaviorSanitizer; it happens while constructing the table, so an image doesn't need to contain that index to trigger it.

The comparisons in comment 1 using the existing curve include this endpoint fix. It makes construction of that table safe; the remaining one-step output differences are addressed in comment 3.

3. Replace the approximate lookup to remove the remaining one-step errors

After the bottom-row correction in [comment 1](#826 (comment)), the current decoder still differs from Adobe DNG Converter 17.5.1 (2319) by exactly one 14-bit raw-value step at 37,848,830 of the 287,720,704 samples in the original nine-file comparison. The other samples match exactly. These comparisons also include the lookup endpoint fix in [comment 2](#826 (comment)).

Those remaining differences come from the nonlinear lookup in [nikon_he_iqx_iqp_lut_data.h](https://github.com/yogthos/LibRaw/blob/499bfd4cbf802c2fb5eac4c9a6f53bbeb776cad1/src/decoders/nikon_he/nikon_he_iqx_iqp_lut_data.h#L112). It interpolates between 256 stored breakpoints to build an 81,792-entry table. The decoder applies that approximate curve to the reconstructed Bayer signal, then scales the result to 14-bit raw values. The source comments already acknowledge the possible one-step error.

The proposed change here is to generate the lookup from the integer function below, using three parameters from the vendor extension to the JPEG XS picture header (PIH). This reproduces the final raw values, including the rounding, and removes the remaining differences on the tested files.

Here v is the signed Bayer value after the corrected spatial reconstruction, before the 32768 bias:

i = clamp(v + 32768, 0, 65535)
d = i - C
if d < 0:
    q = -d*d
else:
    t = (d*A + 32768) >> 16
    q = t*t
raw14 = clamp(B + ((q + 163840) >> 18), 0, 16383)

Use signed 64-bit intermediates. Right shifts here mean floor division, including for negative values. A is Q16 fixed point, and its multiplication rounds half up before squaring. The final bias of 163840 is intentional: it combines (q + 32768) >> 16 with the later (value + 2) >> 2.

The parameters are in the 13-byte vendor PIH extension, starting at PIH payload byte 24. With tail[0] meaning the first extension byte:

B = ((tail[3] & 15) << 8) | tail[4]
A = ((tail[7] & 3) << 16) | (tail[8] << 8) | tail[9]
C = (tail[10] << 8) | tail[11]

Edit (11 September 2026): Corrected A's mask from tail[7] & 15 to tail[7] & 3. Further header tests show that A is an 18-bit field; the two extra bits belong to another gain field. Both were zero in all 11 natural files, so the parameter values, transfer formula, rounding and comparison results reported here are unchanged.

All 11 natural files have B = 1008, A = 84429 and C = 16255. Changing only those fields in local test copies changes Adobe's raw output exactly as the rule predicts. I checked B at 0 and 1024, A at 65536 and 131072, and C at 16000, 16512 and 32768. The upper nibble of tail[3] stayed 0x8; its meaning is still unknown, so I wouldn't read the whole word as a signed black value.

A synthetic raw-GCLI ramp covers every index from 0 through 81791. It matches the rule at every sample. A second ramp, with A = 65905, distinguishes the rounding: at i = 49023 the gain product is 32952.5. Half-up gives raw 5151, while ties-to-even gives 5150. Adobe returns 5151 at all 260 occurrences.

That second ramp also confirms the clamp at 65535. Extending the function to the end of the current lookup allocation gives the wrong result; with the clamp, the complete ramp matches. Default parameters hide the distinction because the output has already saturated.

For the photographic check, the corrected C++ path and an independent reconstruction both match Adobe at all 336,717,824 samples across 11 NEFs: Z9, Z8, Z6III, Zf and Z5II. The Zf and Z5II files were held out until after the rule was recovered. This uses public decoder source and same-input Adobe raw conversions, with no Nikon SDK material.

To use this in the current lookup interface, store 4*raw14, since the decoder applies a final two-bit shift. The photographic comparisons include the bottom-row correction from [comment 1](#826 (comment)). The synthetic ramp also needs the raw-GCLI packet-decoding correction described in comment 4.

I would describe this as a bit-exact behavioral rule for the tested profile and Adobe version. The other vendor bits and untested profiles are still open.

3. Replace the approximate lookup to remove the remaining one-step errors

After the bottom-row correction in comment 1, the current decoder still differs from Adobe DNG Converter 17.5.1 (2319) by exactly one 14-bit raw-value step at 37,848,830 of the 287,720,704 samples in the original nine-file comparison. The other samples match exactly. These comparisons also include the lookup endpoint fix in comment 2.

Whilst all of this is fascinating, the HE raw format decoder that we already have in LibRaw has no such problems. So all those comments are technically irrelevant for LibRaw HE decoder implementation (that you cannot see yet since it is not public).

4. Raw-GCLI packets are accepted but decoded incorrectly

The synthetic ramp used to check the transfer function in comment 3 exposed a separate issue earlier in decoding: raw-GCLI packets are accepted, but the decoder processes them as predicted GCLI packets.

parse_precinct_header reads the leading packet bit into lb_f20_sign, but always allocates out.f20 significance bytes. decode_precinct then always calls the significance/unary GCLI decoder.

That bit selects raw GCLI. When it is set, there are no significance bytes, and each GCLI is an absolute four-bit value for this profile. It needs to bypass both significance decoding and GCLI prediction.

In parse_precinct_header, set the significance length according to the flag so the following offsets are correct:

out.lb_sig_bytes[lb] = out.lb_f20_sign[lb] ? 0 : static_cast<uint32_t>(out.f20);

After allocating gcli_out in decode_precinct, select the reader using the packet flag:

if (sizes.lb_f20_sign[lb]) {
    for (int group = 0; group < ng; ++group)
        gcli_out[group] = static_cast<uint8_t>(gcli_reader.read_bits(4));
} else {
    decode_gcli_values(sig_reader, gcli_reader, dpb_mode,
                       ng, gtli, predict_lut,
                       prev_gcli, gcli_out);
}

I checked a local correction at those two points. With the same exact transfer and bottom-edge fix in both builds, the original packet handling returned success but differed from Adobe at 24,497,260 of the ramp's 24,498,560 samples, with a maximum difference of 13649. The raw-GCLI correction makes every sample match. The dnglab core also decodes that ramp to the same raw values as Adobe.

This case is synthetic; I haven't established which camera-generated files use the raw-packet mode. The decoder currently accepts it and returns incorrect data, so it should either handle it or reject it explicitly. This correction covers raw GCLI in the tested four-bit profile; it doesn't claim to complete all other packet-mode combinations.

3. Replace the approximate lookup to remove the remaining one-step errors

After the bottom-row correction in comment 1, the current decoder still differs from Adobe DNG Converter 17.5.1 (2319) by exactly one 14-bit raw-value step at 37,848,830 of the 287,720,704 samples in the original nine-file comparison. The other samples match exactly. These comparisons also include the lookup endpoint fix in comment 2.

Whilst all of this is fascinating, the HE raw format decoder that we already have in LibRaw has no such problems. So all those comments are technically irrelevant for LibRaw HE decoder implementation (that you cannot see yet since it is not public).

@Alexey-Danilchenko Yes, I'm referring specifically to the code in this PR at 499bfd4. I should have been clearer that "the current decoder" meant this implementation. I haven't tested your unpublished decoder, other than using it within FastRawViewer and RawDigger.

It was not clear which decoder would be used for the public release, so the aim is to document what others can verify and use in their own implementations. That includes the header fields and exact rounding, as well as the fixes to this code. I posted the findings here so they stay with the public implementation they apply to.

@Alexey-Danilchenko Yes, I'm referring specifically to the code in this PR at 499bfd4. I should have been clearer that "the current decoder" meant this implementation. I haven't tested your unpublished decoder, other than using it within FastRawViewer and RawDigger.

It was not clear which decoder would be used for the public release

Considering that "my" decoder was done by old fashioned painstaking reversing of libraries Nikon decoder is using (2 years ago) and no AI nonsense, and it stood out the test of time in FRV and RawDigger it has no such problems and I am pretty sure what I would choose to release.

IMHO all this would be better served by taking it elsewhere - to the originating sources of this and not the current LibRaw. I may be wrong of course so I will leave it for LibRaw owners to comment/take it forward/close this request.

Yes, we've decided to release our/your version along with the next public snapshot this fall.

However, we don't see anything wrong with the existing AI-based version discussed in this PR being finalized: it's very interesting to see the mutual discussions among AI-powered developers.

In any case, this specific PR is not planned for use in the library and will be closed after promised public snapshot release

However, we don't see anything wrong with the existing AI-based version discussed in this PR being finalized: it's very interesting to see the mutual discussions among AI-powered developers.

Reversing with AI... I'll believe it when I see it done properly. So far all the development (not reversing even straight on dev with detailed enough instructions) I have done with help of Codex was ok-ish (and a bit disappointing after all the hype). Whilst it speeds things up - it is not there yet imho.

The reason I went down this rabbit hole was because I wanted to be able to read the files from my camera, and there was no public open decoder available. So, reversing with AI solved my problem since I can now read and edit my photos. However, even in terms of saving the time of figuring out the format it's clearly helpful. It's a lot easier to figure out what to do with the format from the source here than to do reverse engineering by hand from scratch.

The reason I went down this rabbit hole was because I wanted to be able to read the files from my camera, and there was no public open decoder available. So, reversing with AI solved my problem since I can now read and edit my photos. However, even in terms of saving the time of figuring out the format it's clearly helpful. It's a lot easier to figure out what to do with the format from the source here than to do reverse engineering by hand from scratch.

I disagree - AI tends to work ok with well defined goals. Reverse engineering the code is not well defined, you also are at mercy of disassembler/decompiler and compiled libraries optimisations that remove/obscure a lot of initial code. My reversing process is delving into those code clinging to whatever symbols are left in the code and deduce the logic from there and it takes a lot of time since it is iterative with tools like IDA/Ghidra. I have doubts AI will help with the uncertainties that reverse engineer has to deals with there. And tying the reversing to the successful decoding of the existing data (as successfully executed test cases) is not the way to go, it will not end up with correct decoding algorithm at all only some version of it.

Figuring out format should actually be simple - in Nikon's case it is actually ISO standard for JPEG XS (embedded within Nikon binary with Tico RAW implementation of course and their extensions but still pretty much standard)

You're free to disagree of course, but LLM clearly produced a working decoder that I'm currently using. Also, the process you describe isn't how I went about this at all. I had the LLM instrument the process when live decoding was happening, and reverse engineer the steps based on the data in memory. I also had it use Ghidra extensively along side observing a live decoder process. Again, if you feel like you can do this faster on your own, I'm not telling you to work differently. All I'm saying is that I had a specific need, and this is how I solved the problem, and now I have something that works for me. Since the LibRaw features was not published at the time, I didn't really have any other options available to me.