Gray elements render as RGB(0,0,8) on direct-color terminals #2078

On a direct-color terminal (TERM=xterm-direct and friends), every element htop draws in gray comes out as RGB(0, 0, 8), which is indistinguishable from black. On the default black background those elements are invisible.

Cause

CRT.c:1359 picks the gray foreground by colour count:

short int grayBlackFg = COLORS > 8 ? 8 : 0;
short int grayBlackBg = (colorScheme != COLORSCHEME_BLACKNIGHT) ? -1 : 0;
init_pair(ColorIndexGrayBlack, grayBlackFg, grayBlackBg);

COLORS > 8 is a reasonable test for "this terminal has the aixterm bright colours 8 to 15", and it holds on 256-colour terminals. On a direct-color terminal it is true as well (COLORS is 16777216), but there indices at or above 8 are no longer palette entries. ncurses' xterm-direct entry defines:

setaf=\E[%?%p1%{8}%<%t3%p1%d%e38:2::%p1%{65536}%/%d:%p1%{256}%/%{255}%&%d:%p1%{255}%&%d%;m

so an index below 8 takes the legacy 3<n> branch, and anything from 8 up is decomposed into an RGB triple. Index 8 becomes 38:2::0:0:8, that is red 0, green 0, blue 8.

Reproduction

Same binary at 6f33ddd, same htoprc (color_scheme=0, show_cached_memory=0, Memory meter in text mode so the cache classes use METER_SHADOW), only TERM differs:

TERM=xterm-256color   COLORS=256        158x  ESC[90m              <- real gray
TERM=xterm-direct     COLORS=16777216   175x  ESC[38:2::0:0:8m     <- RGB(0,0,8)

Scope

Counting elements bound to ColorPairGrayBlack per scheme:

In the default scheme that is METER_SHADOW, PROCESS_SHADOW, BAR_SHADOW, SWAP_FRONTSWAP, MEMORY_3, HELP_SHADOW, CPU_IOWAIT, DYNAMIC_GRAY and DYNAMIC_DARKGRAY.

There is a workaround, and I got this wrong earlier

In #2072 I said a user could not dodge this by picking another scheme. That was wrong, and measuring the schemes individually is what showed it. On TERM=xterm-direct:

scheme 0 Default       139x RGB(0,0,8)
scheme 7 Nord          163x RGB(0,0,8)
scheme 6 Broken Gray     0x
scheme 1 Monochrome      0x

Broken Gray is immune for a reason that is almost on the nose: CRT.c:1197-1198 builds it by walking the default scheme and replacing exactly A_BOLD | ColorPairGrayBlack with ColorPair(White, Black). It already exists as the escape hatch for terminals whose gray does not work, so it happens to cover this case too. Users hitting this today can select it.

On fixing it

Worth noting that the obvious fix needs the extended colour API rather than init_pair: a 24-bit value such as 0x808080 does not fit the short that init_pair takes, so a real gray on these terminals wants init_extended_color / init_extended_pair. That is the same machinery #2072 introduces, though the two are otherwise independent, and this one affects the schemes that already exist rather than adding a new one.

Filed separately at @rahawii's suggestion in that thread. Measured on Linux, ncurses 6.4, with a locally compiled xterm-direct terminfo entry carrying RGB and colors#0x1000000, since this box has no ncurses-term package.