Skip to content

cpan/Test-Simple/t/Legacy/More.t: frequent failures in non-threaded builds on FreeBSD-13 #16907

Description

@p5pRT

Migrated from rt.perl.org#133958 (status was 'resolved')

Searchable as RT133958$

Activity

  1. p5pRT commented on Mar 26, 2019

    @p5pRT
    Author

    From @jkeenan

    We are experiencing massive test failures in our smoke-testing of the
    Perl 5 core distribution on FreeBSD-13-CURRENT. I am filing two
    separate bug tickets for these failures, since one failure appeared long
    before the others. This is the first of the two tickets.

    cpan/Test-Simple/t/Legacy/More.t is repeatedly failing during
    smoke-testing of non-threaded builds on FreeBSD-13. See, e.g.​:

    http​://perl5.test-smoke.org/report/82470
    http​://perl5.test-smoke.org/report/82946

    Test failures​:
    ~~ ../cpan/Test-Simple/t/Legacy/More.t ......................... FAILED
    54 Non-zero exit status​: 1
      [stdio] -Dcc=clang++ -Duse64bitall
      [stdio] -Dcc=clang++
      [stdio] -Dcc=clang++ DEBUGGING

    We only have one regular smoke-testing rig set up for FreeBSD-13, that
    of course being one of Carlos Guevara's rigs. These failures​:

    i. Do not seem to occur on threaded builds.

    ii. Seem to occur on most, but not all, non-threaded builds.

    iii. Appear to have first reported during a smoke-test in
    http​://perl5.test-smoke.org/report/79958, which suggests that the
    failures were introduced in, or shortly before commit
    https://perl5.git.perl.org/perl.git/commitdiff/8021814b48.  That commit
    is in a deleted branch​: smoke-me/khw-vlb.

    iv. Both Carlos and I have attempted to reproduce this failure manually
    -- but it only seems to crop up during smoke-testing.

    v. Here's the end of the file which is failing​:

    #####
    $ tail cpan/Test-Simple/t/Legacy/More.t
      );

    # rt.cpan.org 53469 is_deeply with regexes
    is_deeply( qr/a/, qr/a/, "same regex" );

    # These two tests must remain at the end.
    is( $@​, $Err, '$@​ untouched' );
    cmp_ok( $!, '==', $Errno, '$! untouched' );
    #####

    Thank you very much.
    Jim Keenan

    perl perl perl

  2. p5pRT commented on Mar 28, 2019

    @p5pRT
    Author

    From @khwilliamson

    On 3/25/19 6​:45 PM, James E Keenan (via RT) wrote​:

    # New Ticket Created by James E Keenan
    # Please include the string​: [perl #133958]
    # in the subject line of all future correspondence about this issue.
    # <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133958 >

    We are experiencing massive test failures in our smoke-testing of the
    Perl 5 core distribution on FreeBSD-13-CURRENT. I am filing two
    separate bug tickets for these failures, since one failure appeared long
    before the others. This is the first of the two tickets.

    cpan/Test-Simple/t/Legacy/More.t is repeatedly failing during
    smoke-testing of non-threaded builds on FreeBSD-13. See, e.g.​:

    http​://perl5.test-smoke.org/report/82470
    http​://perl5.test-smoke.org/report/82946

    Test failures​:
    ~~ ../cpan/Test-Simple/t/Legacy/More.t ......................... FAILED
    54 Non-zero exit status​: 1
    [stdio] -Dcc=clang++ -Duse64bitall
    [stdio] -Dcc=clang++
    [stdio] -Dcc=clang++ DEBUGGING

    We only have one regular smoke-testing rig set up for FreeBSD-13, that
    of course being one of Carlos Guevara's rigs. These failures​:

    i. Do not seem to occur on threaded builds.

    ii. Seem to occur on most, but not all, non-threaded builds.

    iii. Appear to have first reported during a smoke-test in
    http​://perl5.test-smoke.org/report/79958, which suggests that the
    failures were introduced in, or shortly before commit
    https://perl5.git.perl.org/perl.git/commitdiff/8021814b48.  That commit
    is in a deleted branch​: smoke-me/khw-vlb.

    iv. Both Carlos and I have attempted to reproduce this failure manually
    -- but it only seems to crop up during smoke-testing.

    v. Here's the end of the file which is failing​:

    #####
    $ tail cpan/Test-Simple/t/Legacy/More.t
    );

    # rt.cpan.org 53469 is_deeply with regexes
    is_deeply( qr/a/, qr/a/, "same regex" );

    # These two tests must remain at the end.
    is( $@​, $Err, '$@​ untouched' );
    cmp_ok( $!, '==', $Errno, '$! untouched' );
    #####

    Thank you very much.
    Jim Keenan

    perl perl perl

    Have you tried manually running this with address sanitizer?

  3. p5pRT commented on Mar 28, 2019

    @p5pRT
    Author

    The RT System itself - Status changed from 'new' to 'open'

  4. p5pRT commented on Mar 28, 2019

    @p5pRT
    Author

    From @jkeenan

    On 3/27/19 11​:15 PM, karl williamson via RT wrote​:

    On 3/25/19 6​:45 PM, James E Keenan (via RT) wrote​:

    # New Ticket Created by James E Keenan
    # Please include the string​: [perl #133958]
    # in the subject line of all future correspondence about this issue.
    # <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133958 >

    We are experiencing massive test failures in our smoke-testing of the
    Perl 5 core distribution on FreeBSD-13-CURRENT. I am filing two
    separate bug tickets for these failures, since one failure appeared long
    before the others. This is the first of the two tickets.

    cpan/Test-Simple/t/Legacy/More.t is repeatedly failing during
    smoke-testing of non-threaded builds on FreeBSD-13. See, e.g.​:

    http​://perl5.test-smoke.org/report/82470
    http​://perl5.test-smoke.org/report/82946

    Test failures​:
    ~~ ../cpan/Test-Simple/t/Legacy/More.t ......................... FAILED
    54 Non-zero exit status​: 1
    [stdio] -Dcc=clang++ -Duse64bitall
    [stdio] -Dcc=clang++
    [stdio] -Dcc=clang++ DEBUGGING

    We only have one regular smoke-testing rig set up for FreeBSD-13, that
    of course being one of Carlos Guevara's rigs. These failures​:

    i. Do not seem to occur on threaded builds.

    ii. Seem to occur on most, but not all, non-threaded builds.

    iii. Appear to have first reported during a smoke-test in
    http​://perl5.test-smoke.org/report/79958, which suggests that the
    failures were introduced in, or shortly before commit
    https://perl5.git.perl.org/perl.git/commitdiff/8021814b48.  That commit
    is in a deleted branch​: smoke-me/khw-vlb.

    iv. Both Carlos and I have attempted to reproduce this failure manually
    -- but it only seems to crop up during smoke-testing.

    v. Here's the end of the file which is failing​:

    #####
    $ tail cpan/Test-Simple/t/Legacy/More.t
    );

    # rt.cpan.org 53469 is_deeply with regexes
    is_deeply( qr/a/, qr/a/, "same regex" );

    # These two tests must remain at the end.
    is( $@​, $Err, '$@​ untouched' );
    cmp_ok( $!, '==', $Errno, '$! untouched' );
    #####

    Thank you very much.
    Jim Keenan

    perl perl perl

    Have you tried manually running this with address sanitizer?

    1. I myself have never run address sanitizer. How would I do that?

    2. Even if I did that, how would that address the problem that we've
    only been able to observe this problem during smoke-testing?

    Thank you very much.
    Jim Keenan

  5. p5pRT commented on Mar 28, 2019

    @p5pRT
    Author

    From @khwilliamson

    On 3/28/19 6​:21 AM, James E Keenan wrote​:

    On 3/27/19 11​:15 PM, karl williamson via RT wrote​:

    On 3/25/19 6​:45 PM, James E Keenan (via RT) wrote​:

    # New Ticket Created by  James E Keenan
    # Please include the string​:  [perl #133958]
    # in the subject line of all future correspondence about this issue.
    # <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133958 >

    We are experiencing massive test failures in our smoke-testing of the
    Perl 5 core distribution on FreeBSD-13-CURRENT.  I am filing two
    separate bug tickets for these failures, since one failure appeared long
    before the others.  This is the first of the two tickets.

    cpan/Test-Simple/t/Legacy/More.t is repeatedly failing during
    smoke-testing of non-threaded builds on FreeBSD-13.  See, e.g.​:

    http​://perl5.test-smoke.org/report/82470
    http​://perl5.test-smoke.org/report/82946

    Test failures​:
    ~~ ../cpan/Test-Simple/t/Legacy/More.t ......................... FAILED
    54 Non-zero exit status​: 1
         [stdio] -Dcc=clang++ -Duse64bitall
         [stdio] -Dcc=clang++
         [stdio] -Dcc=clang++ DEBUGGING

    We only have one regular smoke-testing rig set up for FreeBSD-13, that
    of course being one of Carlos Guevara's rigs.  These failures​:

    i. Do not seem to occur on threaded builds.

    ii. Seem to occur on most, but not all, non-threaded builds.

    iii. Appear to have first reported during a smoke-test in
    http​://perl5.test-smoke.org/report/79958, which suggests that the
    failures were introduced in, or shortly before commit
    https://perl5.git.perl.org/perl.git/commitdiff/8021814b48.  That commit
    is in a deleted branch​: smoke-me/khw-vlb.

    iv. Both Carlos and I have attempted to reproduce this failure manually
    -- but it only seems to crop up during smoke-testing.

    v. Here's the end of the file which is failing​:

    #####
    $ tail cpan/Test-Simple/t/Legacy/More.t
                );

    # rt.cpan.org 53469  is_deeply with regexes
    is_deeply( qr/a/, qr/a/, "same regex" );

    # These two tests must remain at the end.
    is( $@​, $Err,               '$@​ untouched' );
    cmp_ok( $!, '==', $Errno,   '$! untouched' );
    #####

    Thank you very much.
    Jim Keenan

    perl perl perl

    Have you tried manually running this with address sanitizer?

    1. I myself have never run address sanitizer.  How would I do that?

    Instructions are in perlhacktips. It's just some Configure options to
    clang, which I believe you're already using. Then you run the test suite.

    2. Even if I did that, how would that address the problem that we've
    only been able to observe this problem during smoke-testing?

    There are two likely causes for something showing up only in smoke
    testing that I can think of off-hand. One is timing, with lots of
    parallel jobs. And the other is that there is a read of uninitialized
    memory. Address sanitizer will catch this latter cause. This is
    similar to valgrind on Linux. I have run valgrind on the test suite
    recently with no errors beyond the usual two in CPAN (and yes there is a
    ticket for these open)

    Thank you very much.
    Jim Keenan

  6. p5pRT commented on Mar 28, 2019

    @p5pRT
    Author

    From @jkeenan

    On 3/28/19 10​:16 AM, Karl Williamson wrote​:

    On 3/28/19 6​:21 AM, James E Keenan wrote​:

    On 3/27/19 11​:15 PM, karl williamson via RT wrote​:

    On 3/25/19 6​:45 PM, James E Keenan (via RT) wrote​:

    # New Ticket Created by  James E Keenan
    # Please include the string​:  [perl #133958]
    # in the subject line of all future correspondence about this issue.
    # <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133958 >

    We are experiencing massive test failures in our smoke-testing of the
    Perl 5 core distribution on FreeBSD-13-CURRENT.  I am filing two
    separate bug tickets for these failures, since one failure appeared
    long
    before the others.  This is the first of the two tickets.

    cpan/Test-Simple/t/Legacy/More.t is repeatedly failing during
    smoke-testing of non-threaded builds on FreeBSD-13.  See, e.g.​:

    http​://perl5.test-smoke.org/report/82470
    http​://perl5.test-smoke.org/report/82946

    Test failures​:
    ~~ ../cpan/Test-Simple/t/Legacy/More.t ......................... FAILED
    54 Non-zero exit status​: 1
         [stdio] -Dcc=clang++ -Duse64bitall
         [stdio] -Dcc=clang++
         [stdio] -Dcc=clang++ DEBUGGING

    We only have one regular smoke-testing rig set up for FreeBSD-13, that
    of course being one of Carlos Guevara's rigs.  These failures​:

    i. Do not seem to occur on threaded builds.

    ii. Seem to occur on most, but not all, non-threaded builds.

    iii. Appear to have first reported during a smoke-test in
    http​://perl5.test-smoke.org/report/79958, which suggests that the
    failures were introduced in, or shortly before commit
    https://perl5.git.perl.org/perl.git/commitdiff/8021814b48.  That commit
    is in a deleted branch​: smoke-me/khw-vlb.

    iv. Both Carlos and I have attempted to reproduce this failure manually
    -- but it only seems to crop up during smoke-testing.

    v. Here's the end of the file which is failing​:

    #####
    $ tail cpan/Test-Simple/t/Legacy/More.t
                );

    # rt.cpan.org 53469  is_deeply with regexes
    is_deeply( qr/a/, qr/a/, "same regex" );

    # These two tests must remain at the end.
    is( $@​, $Err,               '$@​ untouched' );
    cmp_ok( $!, '==', $Errno,   '$! untouched' );
    #####

    Thank you very much.
    Jim Keenan

    perl perl perl

    Have you tried manually running this with address sanitizer?

    1. I myself have never run address sanitizer.  How would I do that?

    Instructions are in perlhacktips.  It's just some Configure options to
    clang, which I believe you're already using.  Then you run the test suite.

    2. Even if I did that, how would that address the problem that we've
    only been able to observe this problem during smoke-testing?

    There are two likely causes for something showing up only in smoke
    testing that I can think of off-hand.  One is timing, with lots of
    parallel jobs.

    I should note that in this environment $TEST_JOBS is set to 2. So
    (assuming I understand the test code correctly), only handy00.t and
    handy01.t really matter. I tried running those two in parallel by​:

    $> cd t; ./perl harness ../ext/XS-APItest/t/handy00.t
    ../ext/XS-APItest/t/handy01.t; cd -

    I could not reproduce the problem. But it seems to occur regularly
    inside Test-Smoke's harness.

    And the other is that there is a read of uninitialized
    memory.  Address sanitizer will catch this latter cause.  This is
    similar to valgrind on Linux.

    I will try to test with valgrind today or tomorrow.

    I have run valgrind on the test suite
    recently with no errors beyond the usual two in CPAN (and yes there is a
    ticket for these open)

    Thank you very much.
    Jim Keenan

  7. p5pRT commented on Apr 2, 2019

    @p5pRT
    Author

    From @jkeenan

    On Thu, 28 Mar 2019 15​:15​:50 GMT, jkeenan@​pobox.com wrote​:

    On 3/28/19 10​:16 AM, Karl Williamson wrote​:

    On 3/28/19 6​:21 AM, James E Keenan wrote​:

    On 3/27/19 11​:15 PM, karl williamson via RT wrote​:

    On 3/25/19 6​:45 PM, James E Keenan (via RT) wrote​:

    # New Ticket Created by  James E Keenan
    # Please include the string​:  [perl #133958]
    # in the subject line of all future correspondence about this issue.
    # <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133958 >

    We are experiencing massive test failures in our smoke-testing of the
    Perl 5 core distribution on FreeBSD-13-CURRENT.  I am filing two
    separate bug tickets for these failures, since one failure appeared
    long
    before the others.  This is the first of the two tickets.

    cpan/Test-Simple/t/Legacy/More.t is repeatedly failing during
    smoke-testing of non-threaded builds on FreeBSD-13.  See, e.g.​:

    http​://perl5.test-smoke.org/report/82470
    http​://perl5.test-smoke.org/report/82946

    Test failures​:
    ~~ ../cpan/Test-Simple/t/Legacy/More.t ......................... FAILED
    54 Non-zero exit status​: 1
         [stdio] -Dcc=clang++ -Duse64bitall
         [stdio] -Dcc=clang++
         [stdio] -Dcc=clang++ DEBUGGING

    We only have one regular smoke-testing rig set up for FreeBSD-13, that
    of course being one of Carlos Guevara's rigs.  These failures​:

    i. Do not seem to occur on threaded builds.

    ii. Seem to occur on most, but not all, non-threaded builds.

    iii. Appear to have first reported during a smoke-test in
    http​://perl5.test-smoke.org/report/79958, which suggests that the
    failures were introduced in, or shortly before commit
    https://perl5.git.perl.org/perl.git/commitdiff/8021814b48.  That commit
    is in a deleted branch​: smoke-me/khw-vlb.

    iv. Both Carlos and I have attempted to reproduce this failure manually
    -- but it only seems to crop up during smoke-testing.

    v. Here's the end of the file which is failing​:

    #####
    $ tail cpan/Test-Simple/t/Legacy/More.t
                );

    # rt.cpan.org 53469  is_deeply with regexes
    is_deeply( qr/a/, qr/a/, "same regex" );

    # These two tests must remain at the end.
    is( $@​, $Err,               '$@​ untouched' );
    cmp_ok( $!, '==', $Errno,   '$! untouched' );
    #####

    Thank you very much.
    Jim Keenan

    perl perl perl

    Have you tried manually running this with address sanitizer?

    1. I myself have never run address sanitizer.  How would I do that?

    Instructions are in perlhacktips.  It's just some Configure options to
    clang, which I believe you're already using.  Then you run the test suite.

    2. Even if I did that, how would that address the problem that we've
    only been able to observe this problem during smoke-testing?

    There are two likely causes for something showing up only in smoke
    testing that I can think of off-hand.  One is timing, with lots of
    parallel jobs.

    I should note that in this environment $TEST_JOBS is set to 2. So
    (assuming I understand the test code correctly), only handy00.t and
    handy01.t really matter. I tried running those two in parallel by​:

    $> cd t; ./perl harness ../ext/XS-APItest/t/handy00.t
    ../ext/XS-APItest/t/handy01.t; cd -

    I could not reproduce the problem. But it seems to occur regularly
    inside Test-Smoke's harness.

    And the other is that there is a read of uninitialized
    memory.  Address sanitizer will catch this latter cause.  This is
    similar to valgrind on Linux.

    I will try to test with valgrind today or tomorrow.

    I mistyped. I meant to say that I would try to build a perl with AddressSanitizer. See below.

    I have run valgrind on the test suite
    recently with no errors beyond the usual two in CPAN (and yes there is a
    ticket for these open)

    My efforts to build and test a perl at blead with AddressSanitizer in this FreeBSD-13 were largely unsuccessful.

    After struggling with the syntax of the ./Configure invocation, I was eventually able to get 'make' to complete successfully. See attachment with name starting with 'perl_V'.

    However, I could not run 'make test_harness' successfully. Normally, in these VMs I have $TEST_JOBS set to 2. However, 'make test_harness' was 'Killed' in the t/re/*.t tests. See attachment starting with 'test_harness'.

    I then set TEST_JOBS=1 and ran 'make test' -- so no tests were running in parallel. I got only a little farther on. See attachment starting with 'make_test'.

    Now, I concede​: (a) I've never previously attempted to build with AddressSanitizer; (b) This VM may be too small to run a perl built with AddressSanitizer. But Carlos's FreeBSD-13 smoker, while not large, is larger than mine -- and we're still getting the failures on non-threaded builds in t/Legacy/More.t which are the subject of this RT. See, e.g., http​://perl5.test-smoke.org/report/83974.

    Thank you very much.

    --
    James E Keenan (jkeenan@​cpan.org)

  8. p5pRT commented on Apr 2, 2019

    @p5pRT
    Author

    From @jkeenan

    t/re/no_utf8_pm ................................................ ok
    t/re/overload .................................................. ok
    t/re/pat ....................................................... FAILED--expected 860 tests, saw 108
    t/re/pat_advanced .............................................. ok
    t/re/pat_advanced_thr .......................................... ok
    t/re/pat_psycho ................................................ # Test process timed out - terminating
    FAILED--expected 15 tests, saw 0
    t/re/pat_psycho_thr ............................................ Killed
    *** Error code 137

    Stop.
    make​: stopped in /home/jkeenan/gitwork/perl

  9. p5pRT commented on Apr 2, 2019

    @p5pRT
    Author

    From @jkeenan

    Summary of my perl5 (revision 5 version 29 subversion 10) configuration​:
      Commit id​: 8e48a48
      Platform​:
      osname=freebsd
      osvers=13.0-current
      archname=amd64-freebsd-thread-multi
      uname='freebsd perl-reporter-05 13.0-current freebsd 13.0-current r340361 generic amd64 '
      config_args='-des -Dusedevel -Duseithreads -Dcc=clang -Accflags=-fsanitize=address -Aldflags=-fsanitize=address -Alddlflags=-shared -fsanitize=address'
      hint=recommended
      useposix=true
      d_sigaction=define
      useithreads=define
      usemultiplicity=define
      use64bitint=define
      use64bitall=define
      uselongdouble=undef
      usemymalloc=n
      default_inc_excludes_dot=define
      bincompat5005=undef
      Compiler​:
      cc='clang'
      ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
      optimize='-O2'
      cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
      ccversion=''
      gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1 (tags/RELEASE_601/final 335540)'
      gccosandvers=''
      intsize=4
      longsize=8
      ptrsize=8
      doublesize=8
      byteorder=12345678
      doublekind=3
      d_longlong=define
      longlongsize=8
      d_longdbl=define
      longdblsize=16
      longdblkind=3
      ivtype='long'
      ivsize=8
      nvtype='double'
      nvsize=8
      Off_t='off_t'
      lseeksize=8
      alignbytes=8
      prototype=define
      Linker and Libraries​:
      ld='clang'
      ldflags ='-pthread -Wl,-E -fsanitize=address -fstack-protector-strong -L/usr/local/lib'
      libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
      libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil
      perllibs=-lpthread -ldl -lm -lcrypt -lutil
      libc=
      so=so
      useshrplib=false
      libperl=libperl.a
      gnulibc_version=''
      Dynamic Linking​:
      dlsrc=dl_dlopen.xs
      dlext=so
      d_dlsymun=undef
      ccdlflags=' '
      cccdlflags='-DPIC -fPIC'
      lddlflags='-shared -shared -fsanitize=address -L/usr/local/lib -fstack-protector-strong'

    Characteristics of this binary (from libperl)​:
      Compile-time options​:
      HAS_TIMES
      MULTIPLICITY
      PERLIO_LAYERS
      PERL_COPY_ON_WRITE
      PERL_DONT_CREATE_GVSV
      PERL_IMPLICIT_CONTEXT
      PERL_MALLOC_WRAP
      PERL_OP_PARENT
      PERL_PRESERVE_IVUV
      PERL_USE_DEVEL
      USE_64_BIT_ALL
      USE_64_BIT_INT
      USE_ITHREADS
      USE_LARGE_FILES
      USE_LOCALE
      USE_LOCALE_COLLATE
      USE_LOCALE_CTYPE
      USE_LOCALE_NUMERIC
      USE_LOCALE_TIME
      USE_PERLIO
      USE_PERL_ATOF
      USE_REENTRANT_API
      USE_THREAD_SAFE_LOCALE
      Built under freebsd
      Compiled at Mar 29 2019 14​:44​:36
      %ENV​:
      PERL2DIR="/home/jkeenan/gitwork/perl2"
      PERL_WORKDIR="/home/jkeenan/gitwork/perl"
      @​INC​:
      lib
      /usr/local/lib/perl5/site_perl/5.29.10/amd64-freebsd-thread-multi
      /usr/local/lib/perl5/site_perl/5.29.10
      /usr/local/lib/perl5/5.29.10/amd64-freebsd-thread-multi
      /usr/local/lib/perl5/5.29.10

  10. p5pRT commented on Apr 2, 2019

    @p5pRT
    Author

    From @jkeenan

    ./miniperl -Ilib make_ext.pl cpan/Archive-Tar/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl dist/Attribute-Handlers/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Attribute-Handlers directly
    ./miniperl -Ilib make_ext.pl cpan/AutoLoader/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/AutoLoader directly
    ./miniperl -Ilib make_ext.pl cpan/CPAN/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/CPAN-Meta/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/CPAN-Meta-Requirements/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/CPAN-Meta-Requirements directly
    ./miniperl -Ilib make_ext.pl cpan/CPAN-Meta-YAML/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/CPAN-Meta-YAML directly
    ./miniperl -Ilib make_ext.pl dist/Carp/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Carp directly
    ./miniperl -Ilib make_ext.pl cpan/Config-Perl-V/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Config-Perl-V directly
    ./miniperl -Ilib make_ext.pl dist/Devel-SelfStubber/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Devel-SelfStubber directly
    ./miniperl -Ilib make_ext.pl cpan/Digest/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Digest directly
    ./miniperl -Ilib make_ext.pl dist/Dumpvalue/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Dumpvalue directly
    ./miniperl -Ilib make_ext.pl dist/Env/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Env directly
    ./miniperl -Ilib make_ext.pl ext/Errno/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl dist/Exporter/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Exporter directly
    ./miniperl -Ilib make_ext.pl dist/ExtUtils-CBuilder/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/ExtUtils-CBuilder directly
    ./miniperl -Ilib make_ext.pl cpan/ExtUtils-Constant/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/ExtUtils-Constant directly
    ./miniperl -Ilib make_ext.pl cpan/ExtUtils-Install/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/ExtUtils-Install directly
    ./miniperl -Ilib make_ext.pl cpan/ExtUtils-MakeMaker/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/ExtUtils-Manifest/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl ext/ExtUtils-Miniperl/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for ext/ExtUtils-Miniperl directly
    ./miniperl -Ilib make_ext.pl dist/ExtUtils-ParseXS/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/File-Fetch/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/File-Fetch directly
    ./miniperl -Ilib make_ext.pl ext/File-Find/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for ext/File-Find directly
    ./miniperl -Ilib make_ext.pl cpan/File-Path/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/File-Path directly
    ./miniperl -Ilib make_ext.pl cpan/File-Temp/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/File-Temp directly
    ./miniperl -Ilib make_ext.pl ext/FileCache/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for ext/FileCache directly
    ./miniperl -Ilib make_ext.pl dist/Filter-Simple/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Filter-Simple directly
    ./miniperl -Ilib make_ext.pl cpan/Getopt-Long/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Getopt-Long directly
    ./miniperl -Ilib make_ext.pl cpan/HTTP-Tiny/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl dist/I18N-Collate/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/I18N-Collate directly
    ./miniperl -Ilib make_ext.pl dist/I18N-LangTags/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/I18N-LangTags directly
    ./miniperl -Ilib make_ext.pl dist/lib/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/IO-Compress/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/IO-Socket-IP/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/IO-Socket-IP directly
    ./miniperl -Ilib make_ext.pl cpan/IO-Zlib/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/IO-Zlib directly
    ./miniperl -Ilib make_ext.pl cpan/IPC-Cmd/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/IPC-Cmd directly
    ./miniperl -Ilib make_ext.pl ext/IPC-Open3/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for ext/IPC-Open3 directly
    ./miniperl -Ilib make_ext.pl cpan/JSON-PP/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl dist/Locale-Maketext/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Locale-Maketext directly
    ./miniperl -Ilib make_ext.pl cpan/Locale-Maketext-Simple/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Locale-Maketext-Simple directly
    ./miniperl -Ilib make_ext.pl cpan/Math-BigInt/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Math-BigInt directly
    ./miniperl -Ilib make_ext.pl cpan/Math-BigRat/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Math-BigRat directly
    ./miniperl -Ilib make_ext.pl cpan/Math-Complex/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Math-Complex directly
    ./miniperl -Ilib make_ext.pl cpan/Memoize/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Memoize directly
    ./miniperl -Ilib make_ext.pl dist/Module-CoreList/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/Module-Load/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Module-Load directly
    ./miniperl -Ilib make_ext.pl cpan/Module-Load-Conditional/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Module-Load-Conditional directly
    ./miniperl -Ilib make_ext.pl cpan/Module-Loaded/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Module-Loaded directly
    ./miniperl -Ilib make_ext.pl cpan/Module-Metadata/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/NEXT/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/NEXT directly
    ./miniperl -Ilib make_ext.pl dist/Net-Ping/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Net-Ping directly
    ./miniperl -Ilib make_ext.pl cpan/Params-Check/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Params-Check directly
    ./miniperl -Ilib make_ext.pl cpan/Perl-OSType/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Perl-OSType directly
    ./miniperl -Ilib make_ext.pl cpan/PerlIO-via-QuotedPrint/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/PerlIO-via-QuotedPrint directly
    ./miniperl -Ilib make_ext.pl cpan/Pod-Checker/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/Pod-Escapes/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Pod-Escapes directly
    ./miniperl -Ilib make_ext.pl cpan/Pod-Simple/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Pod-Simple directly
    ./miniperl -Ilib make_ext.pl ext/Pod-Functions/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl ext/Pod-Html/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/Pod-Parser/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/Pod-Perldoc/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/Pod-Usage/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl dist/Safe/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Safe directly
    ./miniperl -Ilib make_ext.pl dist/Search-Dict/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Search-Dict directly
    ./miniperl -Ilib make_ext.pl dist/SelfLoader/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/SelfLoader directly
    ./miniperl -Ilib make_ext.pl cpan/Term-ANSIColor/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Term-ANSIColor directly
    ./miniperl -Ilib make_ext.pl cpan/Term-Cap/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl dist/Term-Complete/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Term-Complete directly
    ./miniperl -Ilib make_ext.pl dist/Term-ReadLine/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Term-ReadLine directly
    ./miniperl -Ilib make_ext.pl dist/Test/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Test directly
    ./miniperl -Ilib make_ext.pl cpan/Test-Harness/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/Test-Simple/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Test-Simple directly
    ./miniperl -Ilib make_ext.pl dist/Text-Abbrev/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Text-Abbrev directly
    ./miniperl -Ilib make_ext.pl cpan/Text-Balanced/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Text-Balanced directly
    ./miniperl -Ilib make_ext.pl cpan/Text-ParseWords/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Text-ParseWords directly
    ./miniperl -Ilib make_ext.pl cpan/Text-Tabs/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Text-Tabs directly
    ./miniperl -Ilib make_ext.pl dist/Thread-Queue/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Thread-Queue directly
    ./miniperl -Ilib make_ext.pl dist/Thread-Semaphore/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Thread-Semaphore directly
    ./miniperl -Ilib make_ext.pl dist/Tie-File/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/Tie-File directly
    ./miniperl -Ilib make_ext.pl ext/Tie-Memoize/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for ext/Tie-Memoize directly
    ./miniperl -Ilib make_ext.pl cpan/Tie-RefHash/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Tie-RefHash directly
    ./miniperl -Ilib make_ext.pl cpan/Time-Local/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/Time-Local directly
    ./miniperl -Ilib make_ext.pl dist/XSLoader/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/autodie/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/autodie directly
    ./miniperl -Ilib make_ext.pl dist/autouse/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/autouse directly
    ./miniperl -Ilib make_ext.pl dist/base/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/bignum/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/bignum directly
    ./miniperl -Ilib make_ext.pl dist/constant/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/constant directly
    ./miniperl -Ilib make_ext.pl dist/encoding-warnings/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for dist/encoding-warnings directly
    ./miniperl -Ilib make_ext.pl cpan/experimental/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/experimental directly
    ./miniperl -Ilib make_ext.pl dist/if/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/libnet/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    ./miniperl -Ilib make_ext.pl cpan/parent/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/parent directly
    ./miniperl -Ilib make_ext.pl cpan/perlfaq/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/perlfaq directly
    ./miniperl -Ilib make_ext.pl cpan/podlators/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Manifying 2 pod documents
    ./miniperl -Ilib make_ext.pl cpan/version/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
    Running pm_to_blib for cpan/version directly
    ./miniperl -Ilib mkppport
    ppport.h in cpan/DB_File is up-to-date
    ppport.h in cpan/IPC-SysV is up-to-date
    ppport.h in cpan/Win32API-File is up-to-date
    ppport.h in dist/IO is up-to-date
    ppport.h in dist/Storable is up-to-date
    ./miniperl -Ilib make_ext.pl DynaLoader.o MAKE="make" LIBPERL_A=libperl.a LINKTYPE=static CCCDLFLAGS=
    ./miniperl -Ilib make_ext.pl lib/auto/Cwd/Cwd.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/dist/PathTools/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Cwd.bs ../../lib/auto/Cwd/Cwd.bs 644

      Making utilities
    ./miniperl -Ilib lib/unicore/mktables -C lib/unicore -P pod -maketest -makelist -p
    lib/unicore/mktables​: Files seem to be ok, not bothering to rebuild. Add '-w' option to force build
    ./miniperl -Ilib make_ext.pl lib/auto/B/B.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/B/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- B.bs ../../lib/auto/B/B.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Compress/Raw/Bzip2/Bzip2.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Compress-Raw-Bzip2/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Bzip2.bs ../../lib/auto/Compress/Raw/Bzip2/Bzip2.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Compress/Raw/Zlib/Zlib.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Compress-Raw-Zlib/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Zlib.bs ../../lib/auto/Compress/Raw/Zlib/Zlib.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/DB_File/DB_File.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/DB_File/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- DB_File.bs ../../lib/auto/DB_File/DB_File.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Data/Dumper/Dumper.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/dist/Data-Dumper/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Dumper.bs ../../lib/auto/Data/Dumper/Dumper.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Devel/PPPort/PPPort.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/dist/Devel-PPPort/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- PPPort.bs ../../lib/auto/Devel/PPPort/PPPort.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Devel/Peek/Peek.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/Devel-Peek/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Peek.bs ../../lib/auto/Devel/Peek/Peek.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Digest/MD5/MD5.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Digest-MD5/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- MD5.bs ../../lib/auto/Digest/MD5/MD5.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Digest/SHA/SHA.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Digest-SHA/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- SHA.bs ../../lib/auto/Digest/SHA/SHA.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Encode/Encode.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Encode/Byte/../../../miniperl "-I../../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Byte.bs ../../../lib/auto/Encode/Byte/Byte.bs 644
    /home/jkeenan/gitwork/perl/cpan/Encode/CN/../../../miniperl "-I../../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- CN.bs ../../../lib/auto/Encode/CN/CN.bs 644
    /home/jkeenan/gitwork/perl/cpan/Encode/EBCDIC/../../../miniperl "-I../../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- EBCDIC.bs ../../../lib/auto/Encode/EBCDIC/EBCDIC.bs 644
    /home/jkeenan/gitwork/perl/cpan/Encode/JP/../../../miniperl "-I../../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- JP.bs ../../../lib/auto/Encode/JP/JP.bs 644
    /home/jkeenan/gitwork/perl/cpan/Encode/KR/../../../miniperl "-I../../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- KR.bs ../../../lib/auto/Encode/KR/KR.bs 644
    /home/jkeenan/gitwork/perl/cpan/Encode/Symbol/../../../miniperl "-I../../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Symbol.bs ../../../lib/auto/Encode/Symbol/Symbol.bs 644
    /home/jkeenan/gitwork/perl/cpan/Encode/TW/../../../miniperl "-I../../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- TW.bs ../../../lib/auto/Encode/TW/TW.bs 644
    /home/jkeenan/gitwork/perl/cpan/Encode/Unicode/../../../miniperl "-I../../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Unicode.bs ../../../lib/auto/Encode/Unicode/Unicode.bs 644
    /home/jkeenan/gitwork/perl/cpan/Encode/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Encode.bs ../../lib/auto/Encode/Encode.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Fcntl/Fcntl.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/Fcntl/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Fcntl.bs ../../lib/auto/Fcntl/Fcntl.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/File/DosGlob/DosGlob.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/File-DosGlob/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- DosGlob.bs ../../lib/auto/File/DosGlob/DosGlob.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/File/Glob/Glob.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/File-Glob/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Glob.bs ../../lib/auto/File/Glob/Glob.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Filter/Util/Call/Call.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Filter-Util-Call/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Call.bs ../../lib/auto/Filter/Util/Call/Call.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/GDBM_File/GDBM_File.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/GDBM_File/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- GDBM_File.bs ../../lib/auto/GDBM_File/GDBM_File.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Hash/Util/Util.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/Hash-Util/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Util.bs ../../lib/auto/Hash/Util/Util.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Hash/Util/FieldHash/FieldHash.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/Hash-Util-FieldHash/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- FieldHash.bs ../../lib/auto/Hash/Util/FieldHash/FieldHash.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/I18N/Langinfo/Langinfo.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/I18N-Langinfo/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Langinfo.bs ../../lib/auto/I18N/Langinfo/Langinfo.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/IO/IO.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/dist/IO/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- IO.bs ../../lib/auto/IO/IO.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/IPC/SysV/SysV.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/IPC-SysV/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- SysV.bs ../../lib/auto/IPC/SysV/SysV.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/List/Util/Util.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Scalar-List-Utils/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Util.bs ../../lib/auto/List/Util/Util.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/MIME/Base64/Base64.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/MIME-Base64/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Base64.bs ../../lib/auto/MIME/Base64/Base64.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Math/BigInt/FastCalc/FastCalc.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Math-BigInt-FastCalc/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- FastCalc.bs ../../lib/auto/Math/BigInt/FastCalc/FastCalc.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/NDBM_File/NDBM_File.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/NDBM_File/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- NDBM_File.bs ../../lib/auto/NDBM_File/NDBM_File.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Opcode/Opcode.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/Opcode/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Opcode.bs ../../lib/auto/Opcode/Opcode.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/POSIX/POSIX.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/POSIX/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- POSIX.bs ../../lib/auto/POSIX/POSIX.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/PerlIO/encoding/encoding.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/PerlIO-encoding/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- encoding.bs ../../lib/auto/PerlIO/encoding/encoding.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/PerlIO/mmap/mmap.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/PerlIO-mmap/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- mmap.bs ../../lib/auto/PerlIO/mmap/mmap.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/PerlIO/scalar/scalar.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/PerlIO-scalar/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- scalar.bs ../../lib/auto/PerlIO/scalar/scalar.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/PerlIO/via/via.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/PerlIO-via/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- via.bs ../../lib/auto/PerlIO/via/via.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/SDBM_File/SDBM_File.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/SDBM_File/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- SDBM_File.bs ../../lib/auto/SDBM_File/SDBM_File.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Socket/Socket.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Socket/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Socket.bs ../../lib/auto/Socket/Socket.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Storable/Storable.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/dist/Storable/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Storable.bs ../../lib/auto/Storable/Storable.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Sys/Hostname/Hostname.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/Sys-Hostname/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Hostname.bs ../../lib/auto/Sys/Hostname/Hostname.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Sys/Syslog/Syslog.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Sys-Syslog/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Syslog.bs ../../lib/auto/Sys/Syslog/Syslog.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Tie/Hash/NamedCapture/NamedCapture.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/Tie-Hash-NamedCapture/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- NamedCapture.bs ../../lib/auto/Tie/Hash/NamedCapture/NamedCapture.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Time/HiRes/HiRes.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/dist/Time-HiRes/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- HiRes.bs ../../lib/auto/Time/HiRes/HiRes.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Time/Piece/Piece.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Time-Piece/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Piece.bs ../../lib/auto/Time/Piece/Piece.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Unicode/Collate/Collate.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/cpan/Unicode-Collate/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Collate.bs ../../lib/auto/Unicode/Collate/Collate.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/Unicode/Normalize/Normalize.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/dist/Unicode-Normalize/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Normalize.bs ../../lib/auto/Unicode/Normalize/Normalize.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/XS/APItest/APItest.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    ./miniperl -Ilib make_ext.pl lib/auto/XS/Typemap/Typemap.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/XS-Typemap/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Typemap.bs ../../lib/auto/XS/Typemap/Typemap.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/attributes/attributes.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/attributes/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- attributes.bs ../../lib/auto/attributes/attributes.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/mro/mro.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/mro/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- mro.bs ../../lib/auto/mro/mro.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/re/re.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/ext/re/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- re.bs ../../lib/auto/re/re.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/threads/threads.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/dist/threads/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- threads.bs ../../lib/auto/threads/threads.bs 644
    ./miniperl -Ilib make_ext.pl lib/auto/threads/shared/shared.so MAKE="make" LIBPERL_A=libperl.a LINKTYPE=dynamic
    /home/jkeenan/gitwork/perl/dist/threads-shared/../../miniperl "-I../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- shared.bs ../../lib/auto/threads/shared/shared.bs 644
    ./perl -Ilib -I. -f pod/buildtoc -q
    cd t && (rm -f perl; /bin/ln -s ../perl perl)
    TESTFILE=harness ./runtests choose
    No saved state, selection will be empty
    base/cond.t ........................................................ ok
    base/if.t .......................................................... ok
    base/lex.t ......................................................... ok
    base/num.t ......................................................... ok
    base/pat.t ......................................................... ok
    base/rs.t .......................................................... ok
    base/term.t ........................................................ ok
    base/translate.t ................................................... ok
    base/while.t ....................................................... ok
    comp/bproto.t ...................................................... ok
    comp/cmdopt.t ...................................................... ok
    comp/colon.t ....................................................... ok
    comp/decl.t ........................................................ ok
    comp/filter_exception.t ............................................ ok
    comp/final_line_num.t .............................................. ok
    comp/form_scope.t .................................................. ok
    comp/fold.t ........................................................ ok
    comp/line_debug.t .................................................. ok
    comp/hints.t ....................................................... ok
    comp/multiline.t ................................................... ok
    comp/opsubs.t ...................................................... ok
    comp/our.t ......................................................... ok
    comp/package.t ..................................................... ok
    comp/package_block.t ............................................... ok
    comp/parser.t ...................................................... ok
    comp/parser_run.t .................................................. ok
    comp/redef.t ....................................................... ok
    comp/proto.t ....................................................... ok
    comp/retainedlines.t ............................................... ok
    comp/require.t ..................................................... ok
    comp/term.t ........................................................ ok
    comp/uproto.t ...................................................... ok
    comp/use.t ......................................................... ok
    comp/utf.t ......................................................... ok
    run/cloexec.t ...................................................... ok
    run/dtrace.t ....................................................... skipped​: no dtrace
    run/exit.t ......................................................... ok
    run/fresh_perl.t ................................................... ok
    run/noswitch.t ..................................................... ok
    run/runenv.t ....................................................... ok
    run/locale.t ....................................................... ok
    run/switch0.t ...................................................... ok
    run/script.t ....................................................... ok
    run/switcha.t ...................................................... ok
    run/switchC.t ...................................................... ok
    run/switchd-78586.t ................................................ ok
    run/switchd.t ...................................................... ok
    run/switchDx.t ..................................................... skipped​: DEBUGGING build required
    run/switchF.t ...................................................... ok
    run/switches.t ..................................................... ok
    run/switchF1.t ..................................................... ok
    run/switchF2.t ..................................................... ok
    run/switchI.t ...................................................... ok
    run/switchM.t ...................................................... ok
    run/switchn.t ...................................................... ok
    run/switchp.t ...................................................... ok
    run/switcht.t ...................................................... ok
    run/switchx.t ...................................................... ok
    cmd/elsif.t ........................................................ ok
    cmd/for.t .......................................................... ok
    cmd/mod.t .......................................................... ok
    cmd/subval.t ....................................................... ok
    cmd/switch.t ....................................................... ok
    io/argv.t .......................................................... ok
    io/binmode.t ....................................................... ok
    io/bom.t ........................................................... ok
    io/closepid.t ...................................................... ok
    io/crlf.t .......................................................... ok
    io/data.t .......................................................... ok
    io/defout.t ........................................................ ok
    io/crlf_through.t .................................................. ok
    io/eintr.t ......................................................... skipped​: various portability issues
    io/dup.t ........................................................... ok
    io/eintr_print.t ................................................... skipped​: various portability issues
    io/errno.t ......................................................... ok
    io/fflush.t ........................................................ ok
    io/fs.t ............................................................ ok
    io/errnosig.t ...................................................... ok
    io/inplace.t ....................................................... ok
    io/iofile.t ........................................................ ok
    io/layers.t ........................................................ ok
    io/iprefix.t ....................................................... ok
    io/nargv.t ......................................................... ok
    io/openpid.t ....................................................... ok
    io/open.t .......................................................... ok
    io/paragraph_mode.t ................................................ ok
    io/perlio.t ........................................................ ok
    io/perlio_fail.t ................................................... ok
    io/perlio_leaks.t .................................................. ok
    io/perlio_open.t ................................................... ok
    io/pipe.t .......................................................... ok
    io/print.t ......................................................... ok
    io/pvbm.t .......................................................... ok
    io/read.t .......................................................... ok
    io/say.t ........................................................... ok
    io/sem.t ........................................................... ok
    io/semctl.t ........................................................ ok
    io/shm.t ........................................................... ok
    io/socket.t ........................................................ ok
    io/tell.t .......................................................... ok
    io/socketpair.t .................................................... ok
    io/utf8.t .......................................................... ok
    re/alpha_assertions.t .............................................. ok
    io/through.t ....................................................... ok
    re/charset.t ....................................................... ok
    # Test process timed out - terminating
    re/fold_grind_8.t ..................................................
    All 9 subtests passed
    # Test process timed out - terminating
    re/fold_grind_a.t ..................................................
    All 9 subtests passed
    # Test process timed out - terminating
    re/fold_grind_aa.t .................................................
    All 9 subtests passed
    # Test process timed out - terminating
    re/fold_grind_d.t ..................................................
    All 9 subtests passed
    # Test process timed out - terminating
    re/fold_grind_l.t ..................................................
    All 12 subtests passed
    re/fold_grind_T.t .................................................. skipped​: Couldn't find a UTF-8 turkic locale
    # Test process timed out - terminating
    re/fold_grind_u.t ..................................................
    All 9 subtests passed
    re/keep_tabs.t ..................................................... ok
    re/no_utf8_pm.t .................................................... ok
    re/overload.t ...................................................... ok
    Killed
    *** Error code 137

    Stop.
    make​: stopped in /home/jkeenan/gitwork/perl

  11. p5pRT commented on Apr 3, 2019

    @p5pRT
    Author

    From @iabyn

    On Tue, Apr 02, 2019 at 02​:31​:16PM -0700, James E Keenan via RT wrote​:

    Now, I concede​: (a) I've never previously attempted to build with
    AddressSanitizer; (b) This VM may be too small to run a perl built with
    AddressSanitizer.

    Address Sanitizer needs lots of memory. My 16Gb system struggles to
    run 10 tests in parallel when compiled with ASan.

    Just t/re/pat_psycho.t consumes about 1.7Gb on a threaded debugging build.

    --
    31 December 1661​: "I have newly taken a solemne oath about abstaining from
    plays". 1 January 1662​: "And after ... we went by coach to the play".
      -- The Diary of Samuel Pepys

  12. p5pRT commented on Apr 12, 2019

    @p5pRT
    Author

    From @jkeenan

    On Tue, 02 Apr 2019 21​:31​:16 GMT, jkeenan wrote​:

    On Thu, 28 Mar 2019 15​:15​:50 GMT, jkeenan@​pobox.com wrote​:

    On 3/28/19 10​:16 AM, Karl Williamson wrote​:

    On 3/28/19 6​:21 AM, James E Keenan wrote​:

    On 3/27/19 11​:15 PM, karl williamson via RT wrote​:

    On 3/25/19 6​:45 PM, James E Keenan (via RT) wrote​:

    # New Ticket Created by  James E Keenan
    # Please include the string​:  [perl #133958]
    # in the subject line of all future correspondence about this
    issue.
    # <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133958 >

    We are experiencing massive test failures in our smoke-testing
    of the
    Perl 5 core distribution on FreeBSD-13-CURRENT.  I am filing two
    separate bug tickets for these failures, since one failure
    appeared
    long
    before the others.  This is the first of the two tickets.

    cpan/Test-Simple/t/Legacy/More.t is repeatedly failing during
    smoke-testing of non-threaded builds on FreeBSD-13.  See, e.g.​:

    http​://perl5.test-smoke.org/report/82470
    http​://perl5.test-smoke.org/report/82946

    Test failures​:
    ~~ ../cpan/Test-Simple/t/Legacy/More.t .........................
    FAILED
    54 Non-zero exit status​: 1
         [stdio] -Dcc=clang++ -Duse64bitall
         [stdio] -Dcc=clang++
         [stdio] -Dcc=clang++ DEBUGGING

    We only have one regular smoke-testing rig set up for FreeBSD-
    13, that
    of course being one of Carlos Guevara's rigs.  These failures​:

    i. Do not seem to occur on threaded builds.

    ii. Seem to occur on most, but not all, non-threaded builds.

    iii. Appear to have first reported during a smoke-test in
    http​://perl5.test-smoke.org/report/79958, which suggests that
    the
    failures were introduced in, or shortly before commit
    https://perl5.git.perl.org/perl.git/commitdiff/8021814b48.  That
    commit
    is in a deleted branch​: smoke-me/khw-vlb.

    iv. Both Carlos and I have attempted to reproduce this failure
    manually
    -- but it only seems to crop up during smoke-testing.

    v. Here's the end of the file which is failing​:

    #####
    $ tail cpan/Test-Simple/t/Legacy/More.t
                );

    # rt.cpan.org 53469  is_deeply with regexes
    is_deeply( qr/a/, qr/a/, "same regex" );

    # These two tests must remain at the end.
    is( $@​, $Err,               '$@​ untouched' );
    cmp_ok( $!, '==', $Errno,   '$! untouched' );
    #####

    I have since built perl5 blead on FreeBSD-13 (i) unthreaded; (ii) without AddressSanitizer; (iii) $TEST_JOBS=1. In this manner​:

    1. In 'make test_harness' the last test in ../cpan/Test-Simple/t/Legacy/More.t FAILs.

    #####
    ../cpan/Test-Simple/t/Legacy/More.t (Wstat​: 256 Tests​: 54 Failed​: 1)
      Failed test​: 54
      Non-zero exit status​: 1
    #####

    2. However, when, in the same build, I run that file manually, that file PASSes.

    #####
    $ cd t;./perl harness ../cpan/Test-Simple/t/Legacy/More.t; cd -
    ../cpan/Test-Simple/t/Legacy/More.t .. ok
    All tests successful.
    Files=1, Tests=54, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.09 cusr 0.04 csys = 0.15 CPU)
    Result​: PASS
    #####

    The above was built/tested at​:

    #####
    commit a719c52 (HEAD -> blead, origin/blead, origin/HEAD)
    Author​: Tomasz Konojacki <me@​xenu.pl>
    AuthorDate​: Tue Apr 9 23​:15​:41 2019 +0200
    Commit​: Steve Hay <steve.m.hay@​googlemail.com>
    CommitDate​: Wed Apr 10 22​:52​:21 2019 +0100
    #####

    But when I build *threaded*, the test passes during 'make test_harness'.

    --
    James E Keenan (jkeenan@​cpan.org)

  13. p5pRT commented on Apr 12, 2019

    @p5pRT
    Author

    From @khwilliamson

    On 4/11/19 7​:57 PM, James E Keenan via RT wrote​:

    On Tue, 02 Apr 2019 21​:31​:16 GMT, jkeenan wrote​:

    On Thu, 28 Mar 2019 15​:15​:50 GMT, jkeenan@​pobox.com wrote​:

    On 3/28/19 10​:16 AM, Karl Williamson wrote​:

    On 3/28/19 6​:21 AM, James E Keenan wrote​:

    On 3/27/19 11​:15 PM, karl williamson via RT wrote​:

    On 3/25/19 6​:45 PM, James E Keenan (via RT) wrote​:

    # New Ticket Created by  James E Keenan
    # Please include the string​:  [perl #133958]
    # in the subject line of all future correspondence about this
    issue.
    # <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133958 >

    We are experiencing massive test failures in our smoke-testing
    of the
    Perl 5 core distribution on FreeBSD-13-CURRENT.  I am filing two
    separate bug tickets for these failures, since one failure
    appeared
    long
    before the others.  This is the first of the two tickets.

    cpan/Test-Simple/t/Legacy/More.t is repeatedly failing during
    smoke-testing of non-threaded builds on FreeBSD-13.  See, e.g.​:

    http​://perl5.test-smoke.org/report/82470
    http​://perl5.test-smoke.org/report/82946

    Test failures​:
    ~~ ../cpan/Test-Simple/t/Legacy/More.t .........................
    FAILED
    54 Non-zero exit status​: 1
         [stdio] -Dcc=clang++ -Duse64bitall
         [stdio] -Dcc=clang++
         [stdio] -Dcc=clang++ DEBUGGING

    We only have one regular smoke-testing rig set up for FreeBSD-
    13, that
    of course being one of Carlos Guevara's rigs.  These failures​:

    i. Do not seem to occur on threaded builds.

    ii. Seem to occur on most, but not all, non-threaded builds.

    iii. Appear to have first reported during a smoke-test in
    http​://perl5.test-smoke.org/report/79958, which suggests that
    the
    failures were introduced in, or shortly before commit
    https://perl5.git.perl.org/perl.git/commitdiff/8021814b48.  That
    commit
    is in a deleted branch​: smoke-me/khw-vlb.

    iv. Both Carlos and I have attempted to reproduce this failure
    manually
    -- but it only seems to crop up during smoke-testing.

    v. Here's the end of the file which is failing​:

    #####
    $ tail cpan/Test-Simple/t/Legacy/More.t
                );

    # rt.cpan.org 53469  is_deeply with regexes
    is_deeply( qr/a/, qr/a/, "same regex" );

    # These two tests must remain at the end.
    is( $@​, $Err,               '$@​ untouched' );
    cmp_ok( $!, '==', $Errno,   '$! untouched' );
    #####

    I have since built perl5 blead on FreeBSD-13 (i) unthreaded; (ii) without AddressSanitizer; (iii) $TEST_JOBS=1. In this manner​:

    1. In 'make test_harness' the last test in ../cpan/Test-Simple/t/Legacy/More.t FAILs.

    #####
    ../cpan/Test-Simple/t/Legacy/More.t (Wstat​: 256 Tests​: 54 Failed​: 1)
    Failed test​: 54
    Non-zero exit status​: 1
    #####

    2. However, when, in the same build, I run that file manually, that file PASSes.

    #####
    $ cd t;./perl harness ../cpan/Test-Simple/t/Legacy/More.t; cd -
    ../cpan/Test-Simple/t/Legacy/More.t .. ok
    All tests successful.
    Files=1, Tests=54, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.09 cusr 0.04 csys = 0.15 CPU)
    Result​: PASS
    #####

    The above was built/tested at​:

    #####
    commit a719c52 (HEAD -> blead, origin/blead, origin/HEAD)
    Author​: Tomasz Konojacki <me@​xenu.pl>
    AuthorDate​: Tue Apr 9 23​:15​:41 2019 +0200
    Commit​: Steve Hay <steve.m.hay@​googlemail.com>
    CommitDate​: Wed Apr 10 22​:52​:21 2019 +0100
    #####

    But when I build *threaded*, the test passes during 'make test_harness'.

    One possibility is to bisect this. You don't have that big a range of
    commits that it could be IIUC. It should be possible to construct a
    bisect that runs a whole 'make test'

  14. p5pRT commented on Apr 14, 2019

    @p5pRT
    Author

    From @jkeenan

    On Fri, 12 Apr 2019 15​:52​:53 GMT, public@​khwilliamson.com wrote​:

    One possibility is to bisect this. You don't have that big a range of
    commits that it could be IIUC. It should be possible to construct a
    bisect that runs a whole 'make test'

    We don't have a value for the '--target' switch to
    Porting/bisect.pl that would evaluate the overall result of
    'make test' or 'make test_harness'. I suppose that we could
    create such a target, but that's going to require many hours
    of work and more brain power than I currently have.

    I did, however, try to bisect manually -- but before I get
    to that, let's recall the original problem. We have​:

    * A new test failure in a very old, cpan-upstream, test.
    * The error only occurs on the development version of one
      operating system.
    * The error only occurs on unthreaded builds.
    * The error never seems to appear when the test file is run
      by itself.
    * The error appears frequently, but nonetheless
      intermittently, when run in test harnesses, such as during
      smoke-testing.

    In addition, it appears that when the error does appear, it
    looks like this​:

    #####
    cpan/Test-Simple/t/Legacy/More ...# Failed test '$! untouched'
    # at t/Legacy/More.t line 184.
    # got​: 12
    # expected​: 42
    # Looks like you failed 1 test of 54.
    FAILED at test 54
    #####

    Since the error is intermittent, it's quite likely that a
    single bisection run will be unable to pinpoint the first
    bad commit in a reproducible manner. Hence, it's quite
    likely that the best we can do is to identify a short range
    of commits in which the error first becomes apparent.

    So, over the past few days, I have built and run the test
    suite on an unthreaded perl on FreeBSD-13 several dozen
    times. I identified all the commits between v5.29.8 and
    v5.29.9. To make a long story short, I believe that the
    FAIL first appeared in one of these two adjacent commits​:

    #####
    commit 3b89859
    Author​: Karl Williamson <khw@​cpan.org>
    AuthorDate​: Sun Mar 17 21​:59​:52 2019 -0600
    Commit​: Karl Williamson <khw@​cpan.org>
    CommitDate​: Sun Mar 17 22​:17​:28 2019 -0600

    regcomp.c​: Use mnemonic for flag parameter

    ###
    commit 6ef7fe5
    Author​: Karl Williamson <khw@​cpan.org>
    AuthorDate​: Sun Mar 17 22​:11​:04 2019 -0600
    Commit​: Karl Williamson <khw@​cpan.org>
    CommitDate​: Sun Mar 17 22​:17​:29 2019 -0600

    PATCH​: [perl #131551] Too deep regex compilation recursion

    This patch, started by Yves Orton, and refined in
    consultation with Tony Cook, imposes a maximum depth of
    unclosed left parentheses, at which point it croaks. This
    is to prevent the segfault in the ticket.

    The patch adds a variable that can be set to increase or
    decrease this limit at run time (actually regex compilation
    time) should this be desired, and hence our pre-determined
    limit of 1000 can be changed if necessary.
    #####

    Why cannot I not say which of these was the first bad
    commit? My bisection process first identified
    6ef7fe5 as the first bad
    commit. Then I said to $self, let's verify this by running
    'make test' at the previous commit and this one. I started
    to get the failure at the first of the two commits. (I
    should note that the two commits entered the repository only
    a second apart, so they were probably part of the same 'git
    push'.)

    I don't understand the internals that were modified in those
    two commits, so I'm still at a loss as to explain these
    intermittent failures -- especially the fact that they only
    occur on non-threaded builds.

    The problem persists. See, e.g.,
    http​://perl5.test-smoke.org/report/85075.

    Thank you very much.

    --
    James E Keenan (jkeenan@​cpan.org)

  15. p5pRT commented on Apr 14, 2019

    @p5pRT
    Author

    From @khwilliamson

    On 4/14/19 1​:59 PM, James E Keenan via RT wrote​:

    On Fri, 12 Apr 2019 15​:52​:53 GMT, public@​khwilliamson.com wrote​:

    One possibility is to bisect this. You don't have that big a range of
    commits that it could be IIUC. It should be possible to construct a
    bisect that runs a whole 'make test'

    We don't have a value for the '--target' switch to
    Porting/bisect.pl that would evaluate the overall result of
    'make test' or 'make test_harness'. I suppose that we could
    create such a target, but that's going to require many hours
    of work and more brain power than I currently have.

    We could make a shell script that called make test and then returned
    appropriately. I could help out devising that should it become necessary.

    I did, however, try to bisect manually -- but before I get
    to that, let's recall the original problem. We have​:

    * A new test failure in a very old, cpan-upstream, test.
    * The error only occurs on the development version of one
    operating system.
    * The error only occurs on unthreaded builds.
    * The error never seems to appear when the test file is run
    by itself.
    * The error appears frequently, but nonetheless
    intermittently, when run in test harnesses, such as during
    smoke-testing.

    In addition, it appears that when the error does appear, it
    looks like this​:

    #####
    cpan/Test-Simple/t/Legacy/More ...# Failed test '$! untouched'
    # at t/Legacy/More.t line 184.
    # got​: 12
    # expected​: 42
    # Looks like you failed 1 test of 54.
    FAILED at test 54
    #####

    Since the error is intermittent, it's quite likely that a
    single bisection run will be unable to pinpoint the first
    bad commit in a reproducible manner. Hence, it's quite
    likely that the best we can do is to identify a short range
    of commits in which the error first becomes apparent.

    So, over the past few days, I have built and run the test
    suite on an unthreaded perl on FreeBSD-13 several dozen
    times. I identified all the commits between v5.29.8 and
    v5.29.9. To make a long story short, I believe that the
    FAIL first appeared in one of these two adjacent commits​:

    #####
    commit 3b89859
    Author​: Karl Williamson <khw@​cpan.org>
    AuthorDate​: Sun Mar 17 21​:59​:52 2019 -0600
    Commit​: Karl Williamson <khw@​cpan.org>
    CommitDate​: Sun Mar 17 22​:17​:28 2019 -0600

    regcomp.c​: Use mnemonic for flag parameter

    ###
    commit 6ef7fe5
    Author​: Karl Williamson <khw@​cpan.org>
    AuthorDate​: Sun Mar 17 22​:11​:04 2019 -0600
    Commit​: Karl Williamson <khw@​cpan.org>
    CommitDate​: Sun Mar 17 22​:17​:29 2019 -0600

    PATCH​: [perl #131551] Too deep regex compilation recursion

    This patch, started by Yves Orton, and refined in
    consultation with Tony Cook, imposes a maximum depth of
    unclosed left parentheses, at which point it croaks. This
    is to prevent the segfault in the ticket.

    The patch adds a variable that can be set to increase or
    decrease this limit at run time (actually regex compilation
    time) should this be desired, and hence our pre-determined
    limit of 1000 can be changed if necessary.
    #####

    Why cannot I not say which of these was the first bad
    commit? My bisection process first identified
    6ef7fe5 as the first bad
    commit. Then I said to $self, let's verify this by running
    'make test' at the previous commit and this one. I started
    to get the failure at the first of the two commits. (I
    should note that the two commits entered the repository only
    a second apart, so they were probably part of the same 'git
    push'.)

    I don't understand the internals that were modified in those
    two commits, so I'm still at a loss as to explain these
    intermittent failures -- especially the fact that they only
    occur on non-threaded builds.

    The problem persists. See, e.g.,
    http​://perl5.test-smoke.org/report/85075.

    It's much more likely to be 6ef7fe5.
    It seems to me that we should revert it, and see if the problem goes
    away. If it doesn't revert cleanly, let me know, and I'll set up a
    branch for you with it reverted. And if the bug persists revert the
    other one. That should tell us if it's really one of these two

    Thank you very much.

  16. 20 remaining items

  17. p5pRT commented on May 3, 2019

    @p5pRT
    Author

    From @khwilliamson

    On 5/3/19 9​:24 AM, James E Keenan via RT wrote​:

    On Fri, 03 May 2019 15​:08​:59 GMT, public@​khwilliamson.com wrote​:

    On 5/3/19 5​:40 AM, James E Keenan via RT wrote​:

    On Fri, 03 May 2019 10​:02​:35 GMT, tonyc wrote​:

    On Fri, May 03, 2019 at 02​:57​:20AM -0700, Hugo van der Sanden via RT
    wrote​:

    On Thu, 02 May 2019 22​:11​:19 -0700, tonyc wrote​:

    + RESTORE_ERRNO;
    #ifdef USE_MDH
    struct perl_memory_debug_header *const header
    = (struct perl_memory_debug_header *)ptr;

    I think you'll need to protect the 'header' declaration from the
    non-
    declaration above it, and similarly in the second RESTORE case.

    Oops, yep.

    Tony

    Tony, when you revise your patch to reflect Hugo's comments above,
    can you push it to a smoke-me branch?

    That will be easier for me to test in my FreeBSD-13 VM.

    Thank you very much.

    I made the changes suggested by Hugo, and pushed the result to
    smoke-me/khw-freebsd

    I can confirm that this getst rid of the spurious messages in #134076

    Now I'm confused. I've already started smoking the smoke-me/tonyc/133958-realloc-errno-success branch. How does that differ from the smoke-me/khw-freebsd branch?

    Thank you very much.

    I didn't know Tony had pushed a new branch. There wasn't a reply on the
    list, so I thought your request to him had come too late in his day.
    But my guess is that his branch and mine identical in functionality, so
    continue smoking his.

  18. p5pRT commented on May 4, 2019

    @p5pRT
    Author

    From @tonycoz

    On Fri, 03 May 2019 04​:40​:11 -0700, jkeenan wrote​:

    On Fri, 03 May 2019 10​:02​:35 GMT, tonyc wrote​:

    On Fri, May 03, 2019 at 02​:57​:20AM -0700, Hugo van der Sanden via RT
    wrote​:

    On Thu, 02 May 2019 22​:11​:19 -0700, tonyc wrote​:

    + RESTORE_ERRNO;
    #ifdef USE_MDH
    struct perl_memory_debug_header *const header
    = (struct perl_memory_debug_header *)ptr;

    I think you'll need to protect the 'header' declaration from the
    non-
    declaration above it, and similarly in the second RESTORE case.

    Oops, yep.

    Tony

    Tony, when you revise your patch to reflect Hugo's comments above, can
    you push it to a smoke-me branch?

    That will be easier for me to test in my FreeBSD-13 VM.

    Thank you very much.

    As you saw, I pushed it last night and this morning here's the patch for the record.

    Tony

  19. p5pRT commented on May 4, 2019

    @p5pRT
    Author

    From @tonycoz

    0001-perl-133958-preserve-errno-on-successful-malloc-real.patch
    From 373d744e311ee3721ab7a4ee2bd8a33b88a7f387 Mon Sep 17 00:00:00 2001
    From: Tony Cook <tony@develop-help.com>
    Date: Fri, 3 May 2019 14:49:50 +1000
    Subject: (perl #133958) preserve errno on successful malloc/realloc
    
    In general perl doesn't try to preserve errno (aka $!) since we're
    aiming at the same behaviour as for C code - errno is only meaningful
    if a function returned an error.
    
    The exception to that is when perl is working without an explicit
    request from the perl programmer.
    
    When code is performing assignments, concatenating strings, pushing on
    arrays etc, perl is exercising the memory allocation machinery,
    calling malloc() and realloc().
    
    It turns out that at least on one platform, realloc() can modify errno
    on success.
    
    It appears to be happening when jemalloc (the malloc() implementation
    used on FreeBSD) tries to extend a memory arena and fails, leaving the
    error number from that failure in errno, from truss:
    
    mmap(0x80142f000,32768,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON|MAP_EXCL,-1,0x0) ERR#12 'Cannot allocate memory'
    
    This magic call appears to be a FreeBSD specific mechanism to resize
    the anonymous mapping.  On Linux the equivalent seems to be calling
    mremap().
    
    In each case for the test code mmap() is successfully called
    immediately afterwards:
    
    mmap(0x0,69632,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) = 34390323200 (0x801d2b000)
    
    and realloc() succeeds.
    
    glibc() realloc seems to be simpler, AFAICT from reading the code it
    only uses mremap() when the memory block is the entire mapping,
    ie. for large blocks rather than for memory arenas, and it doesn't
    request the same address, so it doesn't fail.
    
    For blocks that are part of arenas, glibc tries to expand in-place
    within the current arena (with no extending the arena itself) or falls
    back to malloc, so there's no chance for errno to be changed on a
    successful realloc().
    ---
     util.c | 14 ++++++++++++++
     1 file changed, 14 insertions(+)
    
    diff --git a/util.c b/util.c
    index 7276fd901d..e5320c8b8c 100644
    --- a/util.c
    +++ b/util.c
    @@ -132,6 +132,7 @@ Perl_safesysmalloc(MEM_SIZE size)
         dTHX;
     #endif
         Malloc_t ptr;
    +    dSAVEDERRNO;
     
     #ifdef USE_MDH
         if (size + PERL_MEMORY_DEBUG_HEADER_SIZE < size)
    @@ -143,6 +144,7 @@ Perl_safesysmalloc(MEM_SIZE size)
     	Perl_croak_nocontext("panic: malloc, size=%" UVuf, (UV) size);
     #endif
         if (!size) size = 1;	/* malloc(0) is NASTY on our system */
    +    SAVE_ERRNO;
     #ifdef PERL_DEBUG_READONLY_COW
         if ((ptr = mmap(0, size, PROT_READ|PROT_WRITE,
     		    MAP_ANON|MAP_PRIVATE, -1, 0)) == MAP_FAILED) {
    @@ -182,6 +184,11 @@ Perl_safesysmalloc(MEM_SIZE size)
     	ptr = (Malloc_t)((char*)ptr+PERL_MEMORY_DEBUG_HEADER_SIZE);
     	DEBUG_m(PerlIO_printf(Perl_debug_log, "0x%" UVxf ": (%05ld) malloc %ld bytes\n",PTR2UV(ptr),(long)PL_an++,(long)size));
     
    +        /* malloc() can modify errno() even on success, but since someone
    +	   writing perl code doesn't have any control over when perl calls
    +	   malloc() we need to hide that.
    +	*/
    +        RESTORE_ERRNO;
         }
         else {
     #ifdef USE_MDH
    @@ -223,6 +230,7 @@ Perl_safesysrealloc(Malloc_t where,MEM_SIZE size)
     	ptr = safesysmalloc(size);
         }
         else {
    +        dSAVE_ERRNO;
     #ifdef USE_MDH
     	where = (Malloc_t)((char*)where-PERL_MEMORY_DEBUG_HEADER_SIZE);
             if (size + PERL_MEMORY_DEBUG_HEADER_SIZE < size)
    @@ -296,6 +304,12 @@ Perl_safesysrealloc(Malloc_t where,MEM_SIZE size)
     	    maybe_protect_ro(header->prev);
     #endif
     	    ptr = (Malloc_t)((char*)ptr+PERL_MEMORY_DEBUG_HEADER_SIZE);
    +
    +	    /* realloc() can modify errno() even on success, but since someone
    +	       writing perl code doesn't have any control over when perl calls
    +	       realloc() we need to hide that.
    +	    */
    +	    RESTORE_ERRNO;
     	}
     
         /* In particular, must do that fixup above before logging anything via
    -- 
    2.21.0
    
    
  20. p5pRT commented on May 4, 2019

    @p5pRT
    Author

    From @tonycoz

    On Fri, 03 May 2019 08​:20​:06 -0700, randir wrote​:

    On Thu, 02 May 2019 22​:11​:19 -0700, tonyc wrote​:

    Does the attached fix this for you?

    Shouldn't there be a new configure probe for a broken malloc, so
    systems without such behaviour do not suffer performance penalty?

    Such behaviour is permitted by the C standard.

    From the specification for errno (C11)​:

    The value of errno may be set to nonzero by a library function call
    whether or not there is an error, provided the use of errno is not documented in the
    description of the function in this International Standard.

    and the standard doesn't document any behaviour for errno for realloc(), malloc(), so the behaviour is permitted.

    Similarly POSIX says for errno​:

    The setting of errno after a successful call to a function is unspecified unless the description of that function specifies that errno shall not be modified.

    and there's nothing in the documentation of realloc() guaranteeing errno is unchanged on success.

    Tony

  21. p5pRT commented on May 9, 2019

    @p5pRT
    Author

    From @tonycoz

    On Fri, 03 May 2019 17​:57​:22 -0700, tonyc wrote​:

    As you saw, I pushed it last night and this morning here's the patch
    for the record.

    Applied as 9f30064.

    Tony

  22. p5pRT commented on May 9, 2019

    @p5pRT
    Author

    @tonycoz - Status changed from 'open' to 'pending release'

  23. p5pRT commented on May 22, 2019

    @p5pRT
    Author

    From @khwilliamson

    Thank you for filing this report. You have helped make Perl better.

    With the release today of Perl 5.30.0, this and 160 other issues have been
    resolved.

    Perl 5.30.0 may be downloaded via​:
    https://metacpan.org/release/XSAWYERX/perl-5.30.0

    If you find that the problem persists, feel free to reopen this ticket.

  24. p5pRT commented on May 22, 2019

    @p5pRT
    Author

    @khwilliamson - Status changed from 'pending release' to 'resolved'

  25. added this to the 5.30.0 milestone on Oct 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions