Repository navigation
BBC: MaxMind-DB-Writer-0.300003 fails since perl 5.31.2 #17245
Description
Activity
Bisect points to
commit 7d3bb7a
Author: David Mitchell davem@iabyn.com
Date: Wed May 29 15:57:06 2019 +0100make Perl_op_linklist() non-recursiveI suspect this is a duplicate of #17114 (MaxMind::DB::Writer depends on Sereal::Decoder) which is an op parent issue, there's a PR against Sereal at Sereal/Sereal#210
Applying the Sereal::Decoder fix didn't fix the above. All of the tests assert in a -DDEBUGGING build:
perl5.31.6: op.c:1686: Perl_op_linklist: Assertion `!o->op_next' failed.Backtrace:
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007ffff6cf642a in __GI_abort () at abort.c:89 #2 0x00007ffff6cede67 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x555555930be0 "!o->op_next", file=file@entry=0x55555592e411 "op.c", line=line@entry=1686, function=function@entry=0x555555937230 <__PRETTY_FUNCTION__.15991> "Perl_op_linklist") at assert.c:92 #3 0x00007ffff6cedf12 in __GI___assert_fail ( assertion=0x555555930be0 "!o->op_next", file=0x55555592e411 "op.c", line=1686, function=0x555555937230 <__PRETTY_FUNCTION__.15991> "Perl_op_linklist") at assert.c:101 #4 0x00005555555a6699 in Perl_op_linklist (o=0x5555564f45e0) at op.c:1686 #5 0x00005555555be07f in Perl_newCONDOP (flags=0, first=0x5555564a74e8, trueop=0x5555564f4450, falseop=0x5555564f42f0) at op.c:8860 #6 0x000055555566f32f in Perl_yyparse (gramtype=258) at perly.y:1116 #7 0x00005555558436b4 in S_doeval_compile (gimme=2 '\002', outside=0x0, seq=6648, hh=0x0) at pp_ctl.c:3543 #8 0x000055555584a1ed in S_require_file (sv=0x555556271c88) at pp_ctl.c:4363 #9 0x000055555584a31b in Perl_pp_require () at pp_ctl.c:4387 #10 0x0000555555713aa4 in Perl_runops_debug () at dump.c:2571 #11 0x00005555555ee589 in Perl_call_sv (sv=0x5555563bb278, flags=13) at perl.c:3041 #12 0x00005555555f825f in Perl_call_list (oldscope=10, paramList=0x555556271898) at perl.c:5082 #13 0x00005555555cc6f7 in S_process_special_blocks (floor=269, fullname=0x555555e0dea0 "BEGIN", gv=0x555556271a78, cv=0x5555563bb278) at op.c:10815 #14 0x00005555555cbe45 in Perl_newATTRSUB_x (floor=269, o=0x5555562ce7d8, proto=0x0, attrs=0x0, block=0x5555562ce7a0, o_is_gv=false) at op.c:10740 #15 0x00005555555ba9ef in Perl_utilize (aver=1, floor=269, version=0x0, idop=0x55555636efd8, arg=0x555556276590) at op.c:7907 #16 0x000055555566c844 in Perl_yyparse (gramtype=258) at perly.y:346 #17 0x00005555558436b4 in S_doeval_compile (gimme=2 '\002', outside=0x0, seq=5286, hh=0x0) at pp_ctl.c:3543 #18 0x000055555584a1ed in S_require_file (sv=0x555555e11380) at pp_ctl.c:4363 #19 0x000055555584a31b in Perl_pp_require () at pp_ctl.c:4387 #20 0x0000555555713aa4 in Perl_runops_debug () at dump.c:2571 #21 0x00005555555ee589 in Perl_call_sv (sv=0x555555e108d0, flags=13) at perl.c:3041 #22 0x00005555555f825f in Perl_call_list (oldscope=6, paramList=0x555555e10fa8) at perl.c:5082 #23 0x00005555555cc6f7 in S_process_special_blocks (floor=157, fullname=0x555555e0dea0 "BEGIN", gv=0x555555e10bd0, cv=0x555555e108d0) at op.c:10815 #24 0x00005555555cbe45 in Perl_newATTRSUB_x (floor=157, o=0x55555627cb70, proto=0x0, attrs=0x0, block=0x55555627cb38, o_is_gv=false) at op.c:10740 #25 0x00005555555ba9ef in Perl_utilize (aver=1, floor=157, version=0x0, idop=0x555555e17920, arg=0x0) at op.c:7907 #26 0x000055555566c844 in Perl_yyparse (gramtype=258) at perly.y:346 #27 0x00005555558436b4 in S_doeval_compile (gimme=2 '\002', outside=0x0, seq=4113, hh=0x0) at pp_ctl.c:3543 #28 0x000055555584a1ed in S_require_file (sv=0x555555e10918) at pp_ctl.c:4363 #29 0x000055555584a31b in Perl_pp_require () at pp_ctl.c:4387 #30 0x0000555555713aa4 in Perl_runops_debug () at dump.c:2571 #31 0x00005555555ee589 in Perl_call_sv (sv=0x555555e11590, flags=13) at perl.c:3041 #32 0x00005555555f825f in Perl_call_list (oldscope=2, paramList=0x555555e037d8) at perl.c:5082 #33 0x00005555555cc6f7 in S_process_special_blocks (floor=45, fullname=0x555555e0dea0 "BEGIN", gv=0x555555e037f0, cv=0x555555e11590) at op.c:10815 #34 0x00005555555cbe45 in Perl_newATTRSUB_x (floor=45, o=0x555555e0dcc8, proto=0x0, attrs=0x0, block=0x555555e0dc90, o_is_gv=false) at op.c:10740 #35 0x00005555555ba9ef in Perl_utilize (aver=1, floor=45, version=0x0, idop=0x555555e0d6f0, arg=0x555555e60f50) at op.c:7907 #36 0x000055555566c844 in Perl_yyparse (gramtype=258) at perly.y:346 #37 0x00005555555ebf77 in S_parse_body (env=0x0, xsinit=0x5555555a0377 <xs_init>) at perl.c:2526 #38 0x00005555555ea232 in perl_parse (my_perl=0x555555dd8010, xsinit=0x5555555a0377 <xs_init>, argc=3, argv=0x7fffffffb6e8, env=0x0) at perl.c:1817 #39 0x00005555555a02ba in main (argc=3, argv=0x7fffffffb6e8, env=0x7fffffffb708) at perlmain.c:126Maybe Sereal::Encoder?
(gdb) up #5 0x00005555555be07f in Perl_newCONDOP (flags=0, first=0x5555564a74e8, trueop=0x5555564f4450, falseop=0x5555564f42f0) at op.c:8860 8860 logop = alloc_LOGOP(OP_COND_EXPR, first, LINKLIST(trueop)); (gdb) l 8855 /* Mark the op as being unbindable with =~ */ 8856 live->op_flags |= OPf_SPECIAL; 8857 live->op_folded = 1; 8858 return live; 8859 } 8860 logop = alloc_LOGOP(OP_COND_EXPR, first, LINKLIST(trueop)); 8861 logop->op_flags |= (U8)flags; 8862 logop->op_private = (U8)(1 | (flags >> 8)); 8863 logop->op_next = LINKLIST(falseop); 8864 (gdb) call Perl_op_dump(trueop) 8 sereal_encode_with_object UNOP(0x5555564f4450) ===> [0x0] PARENT ===> [0x0] FLAGS = (UNKNOWN,KIDS,SLABBED) | 9 +--padsv OP(0x5555564a7478) ===> [SELF] | TARG = 3 | FLAGS = (SCALAR,MOD,SLABBED,MORESIB) | 10 +--aelem BINOP(0x5555564f44f8) ===> 11 [gv 0x5555564f45a8] *** WILD PARENT 0x5555564f45e0 FLAGS = (SCALAR,KIDS,MOD,SLABBED) PRIVATE = (LVDEFER,0x2) | 12 +--rv2av UNOP(0x5555564f4570) ===> 13 [const 0x5555564f4538] | FLAGS = (SCALAR,KIDS,REF,SLABBED,MORESIB) | PRIVATE = (STRICT,0x1) | | 11 | +--gv SVOP(0x5555564f45a8) ===> 12 [rv2av 0x5555564f4570] | FLAGS = (SCALAR,SLABBED) | GV = main::_ (0x555555dda460) | 13 +--const SVOP(0x5555564f4538) ===> 10 [aelem 0x5555564f44f8] FLAGS = (SCALAR,SLABBED) SV = IV(0)A similar monkey-fix to Sereal::Encoder didn't prevent the assertion failure in MaxMind::DB::Writer
Data-Serializer-Sereal-1.05 also fails since perl 5.31.2: http://fast-matrix.cpantesters.org/?dist=Data-Serializer-Sereal%201.05
The backtrace also stops at Perl_op_linklist, so it might be the same issue:Program terminated with signal SIGSEGV, Segmentation fault. #0 0x000055da7a4b9e97 in Perl_op_linklist () (gdb) bt #0 0x000055da7a4b9e97 in Perl_op_linklist () #1 0x000055da7a4cf6a8 in Perl_newATTRSUB_x () #2 0x000055da7a509e6b in Perl_yyparse () #3 0x000055da7a59eac1 in S_doeval_compile () #4 0x000055da7a59fe94 in Perl_pp_require () #5 0x000055da7a560893 in Perl_runops_standard () #6 0x000055da7a4d6107 in Perl_call_sv () #7 0x000055da7a4d87de in Perl_call_list () #8 0x000055da7a4b7cbc in S_process_special_blocks.isra.13 () #9 0x000055da7a4cf437 in Perl_newATTRSUB_x () #10 0x000055da7a4d2d61 in Perl_utilize () #11 0x000055da7a509815 in Perl_yyparse () #12 0x000055da7a59eac1 in S_doeval_compile () #13 0x000055da7a5a9921 in Perl_pp_entereval () #14 0x000055da7a560893 in Perl_runops_standard () #15 0x000055da7a4d6107 in Perl_call_sv () #16 0x000055da7a4d87de in Perl_call_list () #17 0x000055da7a4b7cbc in S_process_special_blocks.isra.13 () #18 0x000055da7a4cf437 in Perl_newATTRSUB_x () #19 0x000055da7a509e6b in Perl_yyparse () #20 0x000055da7a4dc7e9 in perl_parse () #21 0x000055da7a4b61f6 in main ()- This should be fixed by the more recent releases of Sereal, I will release 4.010 today based on 4.009_003 Yves…On Tue, 4 Feb 2020, 04:48 Slaven Rezić, ***@***.***> wrote: Data-Serializer-Sereal-1.05 also fails since perl 5.31.2: http://fast-matrix.cpantesters.org/?dist=Data-Serializer-Sereal%201.05 The backtrace also stops at Perl_op_linklist, so it might be the same issue: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x000055da7a4b9e97 in Perl_op_linklist () (gdb) bt #0 0x000055da7a4b9e97 in Perl_op_linklist () #1 0x000055da7a4cf6a8 in Perl_newATTRSUB_x () #2 0x000055da7a509e6b in Perl_yyparse () #3 0x000055da7a59eac1 in S_doeval_compile () #4 0x000055da7a59fe94 in Perl_pp_require () #5 0x000055da7a560893 in Perl_runops_standard () #6 0x000055da7a4d6107 in Perl_call_sv () #7 0x000055da7a4d87de in Perl_call_list () #8 0x000055da7a4b7cbc in S_process_special_blocks.isra.13 () #9 0x000055da7a4cf437 in Perl_newATTRSUB_x () #10 0x000055da7a4d2d61 in Perl_utilize () #11 0x000055da7a509815 in Perl_yyparse () #12 0x000055da7a59eac1 in S_doeval_compile () #13 0x000055da7a5a9921 in Perl_pp_entereval () #14 0x000055da7a560893 in Perl_runops_standard () #15 0x000055da7a4d6107 in Perl_call_sv () #16 0x000055da7a4d87de in Perl_call_list () #17 0x000055da7a4b7cbc in S_process_special_blocks.isra.13 () #18 0x000055da7a4cf437 in Perl_newATTRSUB_x () #19 0x000055da7a509e6b in Perl_yyparse () #20 0x000055da7a4dc7e9 in perl_parse () #21 0x000055da7a4b61f6 in main () — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#17245?email_source=notifications&email_token=AAAZ5R543ONSJMGE6T576YTRBB7ITA5CNFSM4JIL54CKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKVK4EA#issuecomment-581611024>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZ5R7FDDU5CG6SBVRRZ5DRBB7ITANCNFSM4JIL54CA> .
- added a commit that references this issue
on Feb 4, 2020 - Dangit. I was wrong. Dave M posted a patch to Sereal::Decoder that fixed this issue in the decoder. I then had to modify it to work on older perls lacking op_sibling_splice(). I then ported it to the Encoder. Unfortunately during porting I made a mistake and put a crucial line on the wrong side of an #if/#endif. To make things worse the tests for Sereal::Encoder and Sereal::Decoder for the custom opcode stuff was either missing or broken so I didnt notice. I fixed the problem and released Sereal 4.011 just now. With that version installed Data::Serealizer::Sereal installs fine. I couldnt verify if the new release actually fixes MaxMind-DB-Writer-0.300003 as it fails on other dependencies, specifically Sub::Quote fails test under 5.31.2 but currently I don't think that is related to Sereal. FWIW, here is the test fail for Sub::Quote: Building and testing Sub-Quote-2.006006 cp lib/Sub/Defer.pm blib/lib/Sub/Defer.pm cp lib/Sub/Quote.pm blib/lib/Sub/Quote.pm Manifying 2 pod documents PERL_DL_NONLAZY=1 "/home/yorton/perl5/perlbrew/perls/perl-blead/bin/perl5.31.9" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/croak-locations.t ........... ok t/hints.t ..................... ok t/inline.t .................... ok t/leaks.t ..................... ok t/quotify-5.6.t ............... ok # Failed test '"\e" utf8: string value maintained' # at ./t/quotify.t line 245. # got: '\e' # expected: ' ' # Failed test '"\e" utf8: string value maintained under utf8' # at ./t/quotify.t line 245. # got: '\e' # expected: ' ' # Looks like you failed 2 tests of 3041. t/quotify-no-hex.t ............ Dubious, test returned 2 (wstat 512, 0x200) Failed 2/3041 subtests # Failed test '"\e" utf8: string value maintained' # at t/quotify.t line 245. # got: '\e' # expected: ' ' # Failed test '"\e" utf8: string value maintained under utf8' # at t/quotify.t line 245. # got: '\e' # expected: ' ' # Looks like you failed 2 tests of 3041. t/quotify.t ................... Dubious, test returned 2 (wstat 512, 0x200) Failed 2/3041 subtests t/sub-defer-no-subname.t ...... ok t/sub-defer-threads.t ......... skipped: your perl does not support ithreads t/sub-defer.t ................. ok t/sub-quote-threads.t ......... skipped: your perl does not support ithreads t/sub-quote.t ................. ok t/subname-none.t .............. ok t/subname-sub-name-preload.t .. ok t/subname-sub-name.t .......... ok t/subname-sub-util.t .......... ok Test Summary Report ------------------- t/quotify-no-hex.t (Wstat: 512 Tests: 3041 Failed: 2) Failed tests: 2350-2351 Non-zero exit status: 2 t/quotify.t (Wstat: 512 Tests: 3041 Failed: 2) Failed tests: 2350-2351 Non-zero exit status: 2 Files=16, Tests=7726, 2 wallclock secs ( 0.62 usr 0.03 sys + 2.00 cusr 0.10 csys = 2.75 CPU) Result: FAIL…On Tue, 4 Feb 2020 at 02:23, demerphq ***@***.***> wrote: This should be fixed by the more recent releases of Sereal, I will release 4.010 today based on 4.009_003 Yves On Tue, 4 Feb 2020, 04:48 Slaven Rezić, ***@***.***> wrote: > Data-Serializer-Sereal-1.05 also fails since perl 5.31.2: > http://fast-matrix.cpantesters.org/?dist=Data-Serializer-Sereal%201.05 > The backtrace also stops at Perl_op_linklist, so it might be the same > issue: > > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x000055da7a4b9e97 in Perl_op_linklist () > (gdb) bt > #0 0x000055da7a4b9e97 in Perl_op_linklist () > #1 0x000055da7a4cf6a8 in Perl_newATTRSUB_x () > #2 0x000055da7a509e6b in Perl_yyparse () > #3 0x000055da7a59eac1 in S_doeval_compile () > #4 0x000055da7a59fe94 in Perl_pp_require () > #5 0x000055da7a560893 in Perl_runops_standard () > #6 0x000055da7a4d6107 in Perl_call_sv () > #7 0x000055da7a4d87de in Perl_call_list () > #8 0x000055da7a4b7cbc in S_process_special_blocks.isra.13 () > #9 0x000055da7a4cf437 in Perl_newATTRSUB_x () > #10 0x000055da7a4d2d61 in Perl_utilize () > #11 0x000055da7a509815 in Perl_yyparse () > #12 0x000055da7a59eac1 in S_doeval_compile () > #13 0x000055da7a5a9921 in Perl_pp_entereval () > #14 0x000055da7a560893 in Perl_runops_standard () > #15 0x000055da7a4d6107 in Perl_call_sv () > #16 0x000055da7a4d87de in Perl_call_list () > #17 0x000055da7a4b7cbc in S_process_special_blocks.isra.13 () > #18 0x000055da7a4cf437 in Perl_newATTRSUB_x () > #19 0x000055da7a509e6b in Perl_yyparse () > #20 0x000055da7a4dc7e9 in perl_parse () > #21 0x000055da7a4b61f6 in main () > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#17245?email_source=notifications&email_token=AAAZ5R543ONSJMGE6T576YTRBB7ITA5CNFSM4JIL54CKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKVK4EA#issuecomment-581611024>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAZ5R7FDDU5CG6SBVRRZ5DRBB7ITANCNFSM4JIL54CA> > . >-- perl -Mre=debug -e "/just|another|perl|hacker/"
- On Tue, 4 Feb 2020 at 06:25, demerphq ***@***.***> wrote: Dangit. I was wrong. Dave M posted a patch to Sereal::Decoder that fixed this issue in the decoder. I then had to modify it to work on older perls lacking op_sibling_splice(). I then ported it to the Encoder. Unfortunately during porting I made a mistake and put a crucial line on the wrong side of an #if/#endif. To make things worse the tests for Sereal::Encoder and Sereal::Decoder for the custom opcode stuff was either missing or broken so I didnt notice. I fixed the problem and released Sereal 4.011 just now. With that version installed Data::Serealizer::Sereal installs fine. I couldnt verify if the new release actually fixes MaxMind-DB-Writer-0.300003 as it fails on other dependencies, specifically Sub::Quote fails test under 5.31.2 but currently I don't think that is related to Sereal. FWIW, here is the test fail for Sub::Quote: Building and testing Sub-Quote-2.006006 cp lib/Sub/Defer.pm blib/lib/Sub/Defer.pm cp lib/Sub/Quote.pm blib/lib/Sub/Quote.pm Manifying 2 pod documents PERL_DL_NONLAZY=1 "/home/yorton/perl5/perlbrew/perls/perl-blead/bin/perl5.31.9" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/croak-locations.t ........... ok t/hints.t ..................... ok t/inline.t .................... ok t/leaks.t ..................... ok t/quotify-5.6.t ............... ok # Failed test '"\e" utf8: string value maintained' # at ./t/quotify.t line 245. # got: '\e' # expected: ' ' # Failed test '"\e" utf8: string value maintained under utf8' # at ./t/quotify.t line 245. # got: '\e' # expected: ' ' # Looks like you failed 2 tests of 3041. t/quotify-no-hex.t ............ Dubious, test returned 2 (wstat 512, 0x200) Failed 2/3041 subtests # Failed test '"\e" utf8: string value maintained' # at t/quotify.t line 245. # got: '\e' # expected: ' ' # Failed test '"\e" utf8: string value maintained under utf8' # at t/quotify.t line 245. # got: '\e' # expected: ' ' # Looks like you failed 2 tests of 3041.I made an attempt to debug this, instrumented quotify() to see what codepath it took and I got the following: ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perlbrew use 5.18.4; perl Makefile.PL; make Generating a Unix-style Makefile Writing Makefile for Sub::Quote Writing MYMETA.yml and MYMETA.json Skip blib/lib/Sub/Quote.pm (unchanged) Skip blib/lib/Sub/Defer.pm (unchanged) Manifying 2 pod documents ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && utf8::upgrade($s); print quotify($s); Dump($s)' 0 5.018004 using perlstring at /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm line 126. "\033" SV = PV(0xd71fd0) at 0xd924b8 REFCNT = 1 FLAGS = (PADMY,POK,pPOK) PV = 0xd93880 "\33"\0 CUR = 1 LEN = 16 ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && utf8::upgrade($s); print quotify($s); Dump($s)' 1 5.018004 using perlstring at /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm line 126. "\x{1b}" SV = PV(0x1a01fd0) at 0x1a224b8 REFCNT = 1 FLAGS = (PADMY,POK,pPOK,UTF8) PV = 0x1a23880 "\33"\0 [UTF8 "\x{1b}"] CUR = 1 LEN = 16 ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perlbrew use blead; perl Makefile.PL; make Generating a Unix-style Makefile Writing Makefile for Sub::Quote Writing MYMETA.yml and MYMETA.json Skip blib/lib/Sub/Quote.pm (unchanged) Skip blib/lib/Sub/Defer.pm (unchanged) Manifying 2 pod documents ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && utf8::upgrade($s); print quotify($s); Dump($s)' 0 5.031009 using perlstring at /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm line 126. "\033" SV = PV(0x5582f6c85fd0) at 0x5582f6cc4b08 REFCNT = 1 FLAGS = (POK,IsCOW,pPOK) PV = 0x5582f6ed2a40 "\33"\0 CUR = 1 LEN = 10 COW_REFCNT = 1 ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && utf8::upgrade($s); print quotify($s); Dump($s)' 1 5.031009 using perlstring at /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm line 126. "\\e" SV = PV(0x5597a9496fd0) at 0x5597a94fa4f8 REFCNT = 1 FLAGS = (POK,pPOK,UTF8) PV = 0x5597a94c5090 "\33"\0 [UTF8 "\e"] CUR = 1 LEN = 10 Which says to me that B::perlstring has been changed, or regressed from what it used to do with a utf8-on "\e" character. It used to output "\x{1b}" and now it outputs "\\e". Yves…-- perl -Mre=debug -e "/just|another|perl|hacker/"
- On Tue, 4 Feb 2020 at 07:50, demerphq ***@***.***> wrote: On Tue, 4 Feb 2020 at 06:25, demerphq ***@***.***> wrote: > Dangit. I was wrong. Dave M posted a patch to Sereal::Decoder that fixed > this issue in the decoder. I then had to modify it to work on older perls > lacking op_sibling_splice(). I then ported it to the Encoder. Unfortunately > during porting I made a mistake and put a crucial line on the wrong side > of an #if/#endif. > > To make things worse the tests for Sereal::Encoder and Sereal::Decoder > for the custom opcode stuff was either missing or broken so I didnt notice. > > I fixed the problem and released Sereal 4.011 just now. With that version > installed Data::Serealizer::Sereal installs fine. I couldnt verify if the > new release actually fixes MaxMind-DB-Writer-0.300003 as it fails on other > dependencies, specifically Sub::Quote fails test under 5.31.2 but currently > I don't think that is related to Sereal. > > FWIW, here is the test fail for Sub::Quote: > > Building and testing Sub-Quote-2.006006 > cp lib/Sub/Defer.pm blib/lib/Sub/Defer.pm > cp lib/Sub/Quote.pm blib/lib/Sub/Quote.pm > Manifying 2 pod documents > PERL_DL_NONLAZY=1 > "/home/yorton/perl5/perlbrew/perls/perl-blead/bin/perl5.31.9" > "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef > *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t > t/croak-locations.t ........... ok > t/hints.t ..................... ok > t/inline.t .................... ok > t/leaks.t ..................... ok > t/quotify-5.6.t ............... ok > > # Failed test '"\e" utf8: string value maintained' > # at ./t/quotify.t line 245. > # got: '\e' > # expected: ' ' > > # Failed test '"\e" utf8: string value maintained under utf8' > # at ./t/quotify.t line 245. > # got: '\e' > # expected: ' ' > # Looks like you failed 2 tests of 3041. > t/quotify-no-hex.t ............ > Dubious, test returned 2 (wstat 512, 0x200) > Failed 2/3041 subtests > > # Failed test '"\e" utf8: string value maintained' > # at t/quotify.t line 245. > # got: '\e' > # expected: ' ' > > # Failed test '"\e" utf8: string value maintained under utf8' > # at t/quotify.t line 245. > # got: '\e' > # expected: ' ' > # Looks like you failed 2 tests of 3041. > I made an attempt to debug this, instrumented quotify() to see what codepath it took and I got the following: ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perlbrew use 5.18.4; perl Makefile.PL; make Generating a Unix-style Makefile Writing Makefile for Sub::Quote Writing MYMETA.yml and MYMETA.json Skip blib/lib/Sub/Quote.pm (unchanged) Skip blib/lib/Sub/Defer.pm (unchanged) Manifying 2 pod documents ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && utf8::upgrade($s); print quotify($s); Dump($s)' 0 5.018004 using perlstring at /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm line 126. "\033" SV = PV(0xd71fd0) at 0xd924b8 REFCNT = 1 FLAGS = (PADMY,POK,pPOK) PV = 0xd93880 "\33"\0 CUR = 1 LEN = 16 ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && utf8::upgrade($s); print quotify($s); Dump($s)' 1 5.018004 using perlstring at /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm line 126. "\x{1b}" SV = PV(0x1a01fd0) at 0x1a224b8 REFCNT = 1 FLAGS = (PADMY,POK,pPOK,UTF8) PV = 0x1a23880 "\33"\0 [UTF8 "\x{1b}"] CUR = 1 LEN = 16 ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perlbrew use blead; perl Makefile.PL; make Generating a Unix-style Makefile Writing Makefile for Sub::Quote Writing MYMETA.yml and MYMETA.json Skip blib/lib/Sub/Quote.pm (unchanged) Skip blib/lib/Sub/Defer.pm (unchanged) Manifying 2 pod documents ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && utf8::upgrade($s); print quotify($s); Dump($s)' 0 5.031009 using perlstring at /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm line 126. "\033" SV = PV(0x5582f6c85fd0) at 0x5582f6cc4b08 REFCNT = 1 FLAGS = (POK,IsCOW,pPOK) PV = 0x5582f6ed2a40 "\33"\0 CUR = 1 LEN = 10 COW_REFCNT = 1 ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && utf8::upgrade($s); print quotify($s); Dump($s)' 1 5.031009 using perlstring at /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm line 126. "\\e" SV = PV(0x5597a9496fd0) at 0x5597a94fa4f8 REFCNT = 1 FLAGS = (POK,pPOK,UTF8) PV = 0x5597a94c5090 "\33"\0 [UTF8 "\e"] CUR = 1 LEN = 10 Which says to me that B::perlstring has been changed, or regressed from what it used to do with a utf8-on "\e" character. It used to output "\x{1b}" and now it outputs "\\e".This is almost certainly due to: daf6caf I think Sub::Quote needs a test patch. Yves…-- perl -Mre=debug -e "/just|another|perl|hacker/"
- On Tue, 4 Feb 2020 at 07:54, demerphq ***@***.***> wrote: On Tue, 4 Feb 2020 at 07:50, demerphq ***@***.***> wrote: > On Tue, 4 Feb 2020 at 06:25, demerphq ***@***.***> wrote: > >> Dangit. I was wrong. Dave M posted a patch to Sereal::Decoder that >> fixed this issue in the decoder. I then had to modify it to work on older >> perls lacking op_sibling_splice(). I then ported it to the Encoder. >> Unfortunately during porting I made a mistake and put a crucial line on >> the wrong side of an #if/#endif. >> >> To make things worse the tests for Sereal::Encoder and Sereal::Decoder >> for the custom opcode stuff was either missing or broken so I didnt notice. >> >> I fixed the problem and released Sereal 4.011 just now. With that >> version installed Data::Serealizer::Sereal installs fine. I couldnt verify >> if the new release actually fixes MaxMind-DB-Writer-0.300003 as it fails on >> other dependencies, specifically Sub::Quote fails test under 5.31.2 but >> currently I don't think that is related to Sereal. >> >> FWIW, here is the test fail for Sub::Quote: >> >> Building and testing Sub-Quote-2.006006 >> cp lib/Sub/Defer.pm blib/lib/Sub/Defer.pm >> cp lib/Sub/Quote.pm blib/lib/Sub/Quote.pm >> Manifying 2 pod documents >> PERL_DL_NONLAZY=1 >> "/home/yorton/perl5/perlbrew/perls/perl-blead/bin/perl5.31.9" >> "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef >> *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t >> t/croak-locations.t ........... ok >> t/hints.t ..................... ok >> t/inline.t .................... ok >> t/leaks.t ..................... ok >> t/quotify-5.6.t ............... ok >> >> # Failed test '"\e" utf8: string value maintained' >> # at ./t/quotify.t line 245. >> # got: '\e' >> # expected: ' ' >> >> # Failed test '"\e" utf8: string value maintained under utf8' >> # at ./t/quotify.t line 245. >> # got: '\e' >> # expected: ' ' >> # Looks like you failed 2 tests of 3041. >> t/quotify-no-hex.t ............ >> Dubious, test returned 2 (wstat 512, 0x200) >> Failed 2/3041 subtests >> >> # Failed test '"\e" utf8: string value maintained' >> # at t/quotify.t line 245. >> # got: '\e' >> # expected: ' ' >> >> # Failed test '"\e" utf8: string value maintained under utf8' >> # at t/quotify.t line 245. >> # got: '\e' >> # expected: ' ' >> # Looks like you failed 2 tests of 3041. >> > > I made an attempt to debug this, instrumented quotify() to see what > codepath it took and I got the following: > > ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perlbrew use 5.18.4; > perl Makefile.PL; make > Generating a Unix-style Makefile > Writing Makefile for Sub::Quote > Writing MYMETA.yml and MYMETA.json > Skip blib/lib/Sub/Quote.pm (unchanged) > Skip blib/lib/Sub/Defer.pm (unchanged) > Manifying 2 pod documents > ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib > -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && > utf8::upgrade($s); print quotify($s); Dump($s)' 0 > 5.018004 > using perlstring at > /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm > line 126. > "\033" > SV = PV(0xd71fd0) at 0xd924b8 > REFCNT = 1 > FLAGS = (PADMY,POK,pPOK) > PV = 0xd93880 "\33"\0 > CUR = 1 > LEN = 16 > ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib > -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && > utf8::upgrade($s); print quotify($s); Dump($s)' 1 > 5.018004 > using perlstring at > /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm > line 126. > "\x{1b}" > SV = PV(0x1a01fd0) at 0x1a224b8 > REFCNT = 1 > FLAGS = (PADMY,POK,pPOK,UTF8) > PV = 0x1a23880 "\33"\0 [UTF8 "\x{1b}"] > CUR = 1 > LEN = 16 > ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perlbrew use blead; > perl Makefile.PL; make > Generating a Unix-style Makefile > Writing Makefile for Sub::Quote > Writing MYMETA.yml and MYMETA.json > Skip blib/lib/Sub/Quote.pm (unchanged) > Skip blib/lib/Sub/Defer.pm (unchanged) > Manifying 2 pod documents > ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib > -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && > utf8::upgrade($s); print quotify($s); Dump($s)' 0 > 5.031009 > using perlstring at > /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm > line 126. > "\033" > SV = PV(0x5582f6c85fd0) at 0x5582f6cc4b08 > REFCNT = 1 > FLAGS = (POK,IsCOW,pPOK) > PV = 0x5582f6ed2a40 "\33"\0 > CUR = 1 > LEN = 10 > COW_REFCNT = 1 > ~/.cpanm/work/1580795739.2461/Sub-Quote-2.006006$ perl -Mblib > -MSub::Quote=quotify -MDevel::Peek -le'print $]; my $s= "\e"; $ARGV[0] && > utf8::upgrade($s); print quotify($s); Dump($s)' 1 > 5.031009 > using perlstring at > /home/yorton/.cpanm/work/1580795739.2461/Sub-Quote-2.006006/blib/lib/Sub/Quote.pm > line 126. > "\\e" > SV = PV(0x5597a9496fd0) at 0x5597a94fa4f8 > REFCNT = 1 > FLAGS = (POK,pPOK,UTF8) > PV = 0x5597a94c5090 "\33"\0 [UTF8 "\e"] > CUR = 1 > LEN = 10 > > Which says to me that B::perlstring has been changed, or regressed from > what it used to do with a utf8-on "\e" character. It used to output > "\x{1b}" and now it outputs "\\e". > This is almost certainly due to: daf6caf I think Sub::Quote needs a test patch.Nope, this is actually a bug in Perl. I have created a new ticket to get it fixed. #17526 The key part is that the backslash is escaped itself, which means it wont round trip back to "\e". Yves…-- perl -Mre=debug -e "/just|another|perl|hacker/"
Description
Since perl 5.31.2 all tests in the MaxMind-DB-Writer-0.300003 test suite segfault (@maxmind: FYI).
Overview of test reports: http://fast-matrix.cpantesters.org/?dist=MaxMind-DB-Writer%200.300003
A gdb backtrace for
perl5.31.5 -Mblib t/MaxMind/DB/Writer/Serializer-deduplication.t(sorry, perl was not built with -g):