gdImageConvolution(): UBSAN when elements of matrix are set with -INF. #965
Conversation
src/gd_filter.c
Outdated
| new_g = (new_g > 255.0f)? 255.0f : ((new_g < 0.0f)? 0.0f:new_g); | ||
| new_b = (new_b > 255.0f)? 255.0f : ((new_b < 0.0f)? 0.0f:new_b); | ||
|
|
||
| if (isnan(new_r)) { |
There was a problem hiding this comment.
I think isnan() returns true for INFINITE; wouldn't isfinite() be more suitable?
Also, I'm not sure whether that needs feature detection/fallback.
Anyway, thanks for the PR! (Consider to add a test.)
There was a problem hiding this comment.
but INF also provokes a crash. Sure I ll add a test
There was a problem hiding this comment.
ah nice you re member tough :)
There was a problem hiding this comment.
Oops, I meant that isnan(INFINITY) is 0, if I'm not mistaken.
I don't think that the test is exercised. It needs to be added to https://github.com/libgd/libgd/blob/master/tests/gdimageconvolution/CMakeLists.txt and also to the Makemodule.am files (and apparently, bug00369.c is missing from CMakeLists.txt).
There was a problem hiding this comment.
oh I did added to CMakeLists only locally ... sure.
7ca1415 to
4ec9f0a
Compare
4ec9f0a to
37b3e83
Compare
ref: php/php-src#19666