Repository navigation
"Scalar found where operator expected" on printing to indirect filehandle within s///e #16847
Description
Activity
From @Corion
Created by corion@corion.net
This is a bug report for perl from corion@corion.net,
generated with the help of perlbug 1.40 running under perl 5.26.0.-----------------------------------------------------------------
Hi all,last week I (together with E. Choroba) stumbled on a weird syntax
error (Windows shell quoting):perl -wle "s/(.*)/print $fh $1/e"
This surprisingly gives a syntax error, yet still seems to run the code:
Scalar found where operator expected at -e line 1, near "$fh $1"
(Missing operator before $1?)
Name "main::fh" used only once: possible typo at -e line 1.
Use of uninitialized value $_ in substitution (s///) at -e line 1.
Use of uninitialized value $_ in substitution (s///) at -e line 1.
Can't use an undefined value as a symbol reference at -e line 1.The code even outputs 'foo' in the below version:
perl -wle "my $fh = \*STDOUT; $_ = ''; s//print $fh 'foo'/e"
String found where operator expected at -e line 1, near "$fh 'foo'"
(Missing operator before 'foo'?)
fooThe "fix" for this (highly unconventional) code is to use curly brackets
around the filehandle, forcing the "complicated expression" version of
print:perl -wle "my $fh = \*STDOUT; $_ = ''; s//print {$fh} 'foo'/e"
fooI still think that the original code should parse without error, even
though it
is really weird and I'm no fan of side-effects during regexp evaluation.Perl Info
Flags: category=core severity=low Site configuration information for perl 5.26.0: Configured by strawberry-perl at Wed May 31 04:46:12 2017. Summary of my perl5 (revision 5 version 26 subversion 0) configuration: Platform: osname=MSWin32 osvers=6.3 archname=MSWin32-x64-multi-thread uname='Win32 strawberry-perl 5.26.0.1 #1 Wed May 31 04:42:31 2017 x64' config_args='undef' hint=recommended useposix=true d_sigaction=undef useithreads=define usemultiplicity=define use64bitint=define use64bitall=undef uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='gcc' ccflags =' -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE -D__USE_MINGW_ANSI_STDIO -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields' optimize='-s -O2' cppflags='-DWIN32' ccversion='' gccversion='7.1.0' gccosandvers='' intsize=4 longsize=4 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long long' ivsize=8 nvtype='double' nvsize=8 Off_t='long long' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='g++' ldflags ='-s -L"c:\strawberry-perl-5.26.0.1-64bit\perl\lib\CORE" -L"c:\strawberry-perl-5.26.0.1-64bit\c\lib"' libpth=c:\strawberry-perl-5.26.0.1-64bit\c\lib c:\strawberry-perl-5.26.0.1-64bit\c\x86_64-w64-mingw32\lib c:\strawberry-perl-5.26.0.1-64bit\c\lib\gcc\x86_64-w64-mingw32\7.1.0 libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32 perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32 libc= so=dll useshrplib=true libperl=libperl526.a gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs dlext=xs.dll d_dlsymun=undef ccdlflags=' ' cccdlflags=' ' lddlflags='-mdll -s -L"c:\strawberry-perl-5.26.0.1-64bit\perl\lib\CORE" -L"c:\strawberry-perl-5.26.0.1-64bit\c\lib"' @INC for perl 5.26.0: C:/strawberry-perl-5.26.0.1-64bit/perl/site/lib/MSWin32-x64-multi-thread C:/strawberry-perl-5.26.0.1-64bit/perl/site/lib C:/strawberry-perl-5.26.0.1-64bit/perl/vendor/lib C:/strawberry-perl-5.26.0.1-64bit/perl/lib Environment for perl 5.26.0: HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=C:\Program Files\Microsoft IntelliType Pro\;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Git\cmd;C:\Program Files\PuTTY;C:\Program Files\MiKTeX 2.9\miktex\bin\x64\;C:\strawberry-perl-5.20.1.1-x64\c\bin;C:\strawberry-perl-5.20.1.1-x64\perl\site\bin;C:\strawberry-perl-5.20.1.1-x64\perl\bin;C:\Program Files (x86)\Skype\Phone\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\GtkSharp\2.12\bin;C:\Program Files (x86)\Gpg4win\..\GnuPG\bin;C:\Program Files\nodejs\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Users\Corion\AppData\Local\Programs\Python\Python36\Scripts\;C:\Users\Corion\AppData\Local\Programs\Python\Python36\;C:\Program Files\Docker Toolbox;C:\ProgramData\chocolatey\bin;C:\strawberry-perl-5.26.0.1-64bit\perl\bin;c:\strawberry-perl-5.26.0.1-64bit\c\bin;C:\strawberry-perl-5.26.0.1-64bit\c\x86_64-w64-mingw32\bin;C:\Users\Corion\AppData\Roaming\npm PERL_BADLANG (unset) SHELL (unset)From @tonycoz
On Fri, 15 Feb 2019 11:37:16 -0800, corion@corion.net wrote:
last week I (together with E. Choroba) stumbled on a weird syntax
error (Windows shell quoting):perl -wle "s/(.*)/print $fh $1/e"
This surprisingly gives a syntax error, yet still seems to run the
code:Scalar found where operator expected at -e line 1, near "$fh $1"
(Missing operator before $1?)This isn't a syntax error, but a warning (but still buggy).
The code inside the replacement part of s///e is handled with a sublex, and the code that decides whether an operator or term is expected after C<print $fh> is mostly skipped for a sublex.
The attached fixes that, doing the same checks for C<print $fh> in a sublex within brackets as for a normal parse.
This fixes the same problem for C<< "... ${code} ..." >>.
Tony
From @tonycoz
0001-perl-133850-fix-parsing-hints-for-print-fh-foo-in-s-.patch
From 9e0f09ef2afa82ef1a96cba981d4fca094bba484 Mon Sep 17 00:00:00 2001 From: Tony Cook <tony@develop-help.com> Date: Tue, 19 Feb 2019 15:38:39 +1100 Subject: (perl #133850) fix parsing hints for print $fh "foo" in s///e The replacement code in s///e is parsed using a sublex, similarly to double-quotish replacement. The code that handles C< $id > tries to detect what the next token should be, but only special cased code similar to C< print $fh ... > in a non-sublex, so s/../print $fh $1/e set PL_expect to XOPERATOR which caused the parser to complain when the $1 wasn't an operator. The fix here handles two cases: s/.../code here/e In this case the code is inserted into a do {} block like: do { code here } so PL_lex_brackets ends up non-zero. The second case is code like: "xxx${code here}" which had the same problem, which this also fixes. --- t/lib/warnings/toke | 14 ++++++++++++++ toke.c | 2 +- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/t/lib/warnings/toke b/t/lib/warnings/toke index 1c85d7bc44..a500856283 100644 --- a/t/lib/warnings/toke +++ b/t/lib/warnings/toke @@ -1675,3 +1675,17 @@ Execution of - aborted due to compilation errors. use utf8; qw���foo ��� ��� bar��� EXPECT +######## +# NAME [perl #133850] print $fh $1 in s///e expression +use warnings; +my $fh = \*STDOUT; +$_ = "abc"; +s/(x)/ print $fh $1 /e; +EXPECT +######## +# NAME [perl #133850] side case +use warnings; +my $fh = \*STDOUT; +my $y = ""; +my $x = "${print $fh $y; \'x'}"; +EXPECT diff --git a/toke.c b/toke.c index 5a3fe78472..0eb63fcb55 100644 --- a/toke.c +++ b/toke.c @@ -6772,7 +6772,7 @@ Perl_yylex(pTHX) } PL_expect = XOPERATOR; - if (PL_lex_state == LEX_NORMAL && isSPACE((char)tmp)) { + if ((PL_lex_state == LEX_NORMAL || PL_lex_brackets) && isSPACE((char)tmp)) { const bool islop = (PL_last_lop == PL_oldoldbufptr); if (!islop || PL_last_lop_op == OP_GREPSTART) PL_expect = XOPERATOR; -- 2.11.0The RT System itself - Status changed from 'new' to 'open'
From @tonycoz
On Mon, 18 Feb 2019 20:44:26 -0800, tonyc wrote:
This fixes the same problem for C<< "... ${code} ..." >>.
There's some other similar cases - i.e. where PL_lex_state is checked for LEX_NORMAL but no other checks are made. The simplest I was able to reproduce was:
$ ./perl -we '%x = qw(a b c d); $_ = ""; $_ = @x {a}'
Scalar value @x{"a"} better written as $x{"a"} at -e line 1.
$ ./perl -we '%x = qw(a b c d); $_ = ""; s/^/ @x {a} /e'
(no warning)Kind of unlikely:
tony@mars:.../git/perl$ ./perl -Ilib -we '{ my $x : shared = 1; }'
tony@mars:.../git/perl$ ./perl -Ilib -we '$_ = ""; s/^/ { my $x : shared = 1; } /e'
Unquoted string "shared" may clash with future reserved word at -e line 1.
syntax error at -e line 1, near "$x :"Another:
$ ./perl -we '$a = ${time}'
Ambiguous use of ${time} resolved to $time at -e line 1.
Name "main::time" used only once: possible typo at -e line 1.
$ ./perl -we '$_ = ""; s/^/ ${time} /e'
Name "main::time" used only once: possible typo at -e line 1.
Use of uninitialized value $time in substitution (s///) at -e line 1.
(no ambiguous warning)Tony
From @tonycoz
On Mon, 18 Feb 2019 20:44:26 -0800, tonyc wrote:
On Fri, 15 Feb 2019 11:37:16 -0800, corion@corion.net wrote:
last week I (together with E. Choroba) stumbled on a weird syntax
error (Windows shell quoting):perl -wle "s/(.*)/print $fh $1/e"
This surprisingly gives a syntax error, yet still seems to run the
code:Scalar found where operator expected at -e line 1, near "$fh $1"
(Missing operator before $1?)This isn't a syntax error, but a warning (but still buggy).
The code inside the replacement part of s///e is handled with a
sublex, and the code that decides whether an operator or term is
expected after C<print $fh> is mostly skipped for a sublex.The attached fixes that, doing the same checks for C<print $fh> in a
sublex within brackets as for a normal parse.This fixes the same problem for C<< "... ${code} ..." >>.
Rebased on blead, and fixes for the related issues.
I'll apply this in a few days.
Tony
From @tonycoz
subparse-tokens.patch
From cb43f46efc62573c3470f5c2de01ee0e5b6ffd4a Mon Sep 17 00:00:00 2001 From: Tony Cook <tony@develop-help.com> Date: Tue, 19 Feb 2019 15:38:39 +1100 Subject: (perl #133850) fix parsing hints for print $fh "foo" in s///e The replacement code in s///e is parsed using a sublex, similarly to double-quotish replacement. The code that handles C< $id > tries to detect what the next token should be, but only special cased code similar to C< print $fh ... > in a non-sublex, so s/../print $fh $1/e set PL_expect to XOPERATOR which caused the parser to complain when the $1 wasn't an operator. The fix here handles two cases: s/.../code here/e In this case the code is inserted into a do {} block like: do { code here } so PL_lex_brackets ends up non-zero. The second case is code like: "xxx${code here}" which had the same problem, which this also fixes. --- t/lib/warnings/toke | 14 ++++++++++++++ toke.c | 2 +- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/t/lib/warnings/toke b/t/lib/warnings/toke index 83641e5106..944e5d92f4 100644 --- a/t/lib/warnings/toke +++ b/t/lib/warnings/toke @@ -1691,3 +1691,17 @@ EXPECT OPTION regex Malformed UTF-8 character: .*non-continuation.* The eval did not crash the program +######## +# NAME [perl #133850] print $fh $1 in s///e expression +use warnings; +my $fh = \*STDOUT; +$_ = "abc"; +s/(x)/ print $fh $1 /e; +EXPECT +######## +# NAME [perl #133850] side case +use warnings; +my $fh = \*STDOUT; +my $y = ""; +my $x = "${print $fh $y; \'x'}"; +EXPECT diff --git a/toke.c b/toke.c index b0fa2c35ba..dc376f6beb 100644 --- a/toke.c +++ b/toke.c @@ -6838,7 +6838,7 @@ Perl_yylex(pTHX) } PL_expect = XOPERATOR; - if (PL_lex_state == LEX_NORMAL && isSPACE((char)tmp)) { + if ((PL_lex_state == LEX_NORMAL || PL_lex_brackets) && isSPACE((char)tmp)) { const bool islop = (PL_last_lop == PL_oldoldbufptr); if (!islop || PL_last_lop_op == OP_GREPSTART) PL_expect = XOPERATOR; -- 2.11.0 From 2b67caa9b5055eb7441998a9538af722639982af Mon Sep 17 00:00:00 2001 From: Tony Cook <tony@develop-help.com> Date: Wed, 29 May 2019 11:46:01 +1000 Subject: (perl #133850) warn on ${time} even in sub-parse Code like: $a = ${time} warns with: Ambiguous use of ${time} resolved to $time at ... but: s/^/ ${time} /e didn't, since the parser is in a special state in sub-parses. --- t/lib/warnings/toke | 8 ++++++++ toke.c | 2 +- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/t/lib/warnings/toke b/t/lib/warnings/toke index 944e5d92f4..eab22fbbc9 100644 --- a/t/lib/warnings/toke +++ b/t/lib/warnings/toke @@ -1705,3 +1705,11 @@ my $fh = \*STDOUT; my $y = ""; my $x = "${print $fh $y; \'x'}"; EXPECT +######## +# NAME [perl #133850] another case +use warnings; +my $time = 1; +$_ = ""; +s/^/ ${time} /e +EXPECT +Ambiguous use of ${time} resolved to $time at - line 4. diff --git a/toke.c b/toke.c index dc376f6beb..c42d4fd802 100644 --- a/toke.c +++ b/toke.c @@ -9660,7 +9660,7 @@ S_scan_ident(pTHX_ char *s, char *dest, STRLEN destlen, I32 ck_uni) PL_lex_state = LEX_INTERPEND; PL_expect = XREF; } - if (PL_lex_state == LEX_NORMAL) { + if (PL_lex_state == LEX_NORMAL || PL_lex_brackets) { if (ckWARN(WARN_AMBIGUOUS) && (keyword(dest, d - dest, 0) || get_cvn_flags(dest, d - dest, is_utf8 -- 2.11.0 From f623089c7b47c9de58667b70063be135948356bd Mon Sep 17 00:00:00 2001 From: Tony Cook <tony@develop-help.com> Date: Wed, 29 May 2019 13:47:36 +1000 Subject: (perl #133850) allow @x{...} with space in sub-parse --- t/lib/warnings/toke | 8 ++++++++ toke.c | 2 +- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/t/lib/warnings/toke b/t/lib/warnings/toke index eab22fbbc9..5dbef1ab62 100644 --- a/t/lib/warnings/toke +++ b/t/lib/warnings/toke @@ -1713,3 +1713,11 @@ $_ = ""; s/^/ ${time} /e EXPECT Ambiguous use of ${time} resolved to $time at - line 4. +######## +# NAME [perl #133850] another case +use warnings; +%x = qw(a b c d); +$_ = ""; +s/^/ @x {a} /e +EXPECT +Scalar value @x{"a"} better written as $x{"a"} at - line 4. diff --git a/toke.c b/toke.c index c42d4fd802..e1b163334f 100644 --- a/toke.c +++ b/toke.c @@ -6911,7 +6911,7 @@ Perl_yylex(pTHX) if (!PL_tokenbuf[1]) { PREREF('@'); } - if (PL_lex_state == LEX_NORMAL) + if (PL_lex_state == LEX_NORMAL || PL_lex_brackets) s = skipspace(s); if ( (PL_expect != XREF || PL_oldoldbufptr == PL_last_lop) && intuit_more(s, PL_bufend)) -- 2.11.0 From d50d49c97634fa03d472f06bf4e70fe7dfc17fd2 Mon Sep 17 00:00:00 2001 From: Tony Cook <tony@develop-help.com> Date: Wed, 29 May 2019 14:37:25 +1000 Subject: (perl #133850) allow attributes in sub-parses --- t/op/attrs.t | 3 +++ toke.c | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/t/op/attrs.t b/t/op/attrs.t index 649525ce22..dd230b3fbd 100644 --- a/t/op/attrs.t +++ b/t/op/attrs.t @@ -507,4 +507,7 @@ BEGIN failed--compilation aborted at - line 1. EOS "attribute on our hash with sub of same name"); +fresh_perl_is('$_ = ""; s/^/ { my $x : shared = 1; } /e;', "", {}, + "attributes in sub-parse"); + done_testing(); diff --git a/toke.c b/toke.c index e1b163334f..0b3b76fee3 100644 --- a/toke.c +++ b/toke.c @@ -5991,7 +5991,7 @@ Perl_yylex(pTHX) switch (PL_expect) { case XOPERATOR: - if (!PL_in_my || PL_lex_state != LEX_NORMAL) + if (!PL_in_my || (PL_lex_state != LEX_NORMAL && !PL_lex_brackets)) break; PL_bufptr = s; /* update in case we back off */ if (*s == '=') { -- 2.11.0From @tonycoz
On Tue, 28 May 2019 22:08:50 -0700, tonyc wrote:
On Mon, 18 Feb 2019 20:44:26 -0800, tonyc wrote:
On Fri, 15 Feb 2019 11:37:16 -0800, corion@corion.net wrote:
last week I (together with E. Choroba) stumbled on a weird syntax
error (Windows shell quoting):perl -wle "s/(.*)/print $fh $1/e"
This surprisingly gives a syntax error, yet still seems to run the
code:Scalar found where operator expected at -e line 1, near "$fh $1"
(Missing operator before $1?)This isn't a syntax error, but a warning (but still buggy).
The code inside the replacement part of s///e is handled with a
sublex, and the code that decides whether an operator or term is
expected after C<print $fh> is mostly skipped for a sublex.The attached fixes that, doing the same checks for C<print $fh> in a
sublex within brackets as for a normal parse.This fixes the same problem for C<< "... ${code} ..." >>.
Rebased on blead, and fixes for the related issues.
I'll apply this in a few days.
Applied as eec8822 through 82de236.
Tony
Migrated from rt.perl.org#133850 (status was 'pending release')
Searchable as RT133850$