SPUG: Buggy shebang-line parsing

Tim Maher tim at consultix-inc.com
Sat Jan 15 10:54:08 PST 2005


On Sat, Jan 15, 2005 at 09:10:29AM -0800, David Dyck wrote:
> On Fri, 14 Jan 2005 at 23:45 -0800, Tim Maher <tim at consultix-inc.com> wrote:
> 
> > I've found some serious bugs in recent Perl versions in the
> > parsing of non-trivial shebang lines, involving option clusters after
> > the first being disregarded if too many spaces precede the next ones,
> > and the -s option being ignored altogether, if it doesn't come first
> > in some cases.  For example:
> >
> > 	#! /usr/bin/perl -wlnaF'\t' -s 	# BAD!
> >
> > 	#! /usr/bin/perl -s -wlnaF'\t'	# GOOD!
>          123456789012345678901234567890
> 
> Tim
> 
> Have you eliminated the operating system interaction with the command line?
>  (I see that you are close to 32 characters and perlrun has some warnings
>   about it - see below)

I figure the above example is only 27 characters, and in any
case, reordering the arguments fixes the problem, without
changing the string length, which smells like a bug to me!

> >From perldoc perlrun here are some concerns:
>
>    Because historically some operating systems silently chopped
>    off kernel interpretation of the #! line after 32
>    characters, some switches may be passed in on the command
>    line, and some may not

I'm aware of that warning, but the (somewhat vague)
wording could mean that only some ancient systems would exhibit the
problem being described, and I'm using Solaris 9.0 and Linux. 8-}
In any case, I'm not hitting the the 32 character boundary, as
mentioned above.

> Do you need the space between the #! and the interpreter name?

I remember it being required on some old versions of UNIX I
used, and so I'm reluctant to prescribe leaving it out in my
book, for fear somebody will try to do that on a system that
won't tolerate it.

> Does moving the flags past the 32-character have an effect?

Not sure, but I know how to find out. 8-}

> I see you are trying to build a test program, but do you have a
> few short ones that show a good and bad example?
>
>   David

You can see my earlier (different) bug report in attachment #1,
and a demonstration of the bug described above in attachment #2
(By the way, the bug disappears if the -s option is not kept
separate from the others, and only occurs if -F is used; my
guess is that anything coming after -F is ignored -- so there
are some clues!)

Anyway David, thanks for your interest! As you know, many of the
examples in my upcoming book rely heavily on shebang lines being
parsed correctly, so I'm pretty keen on getting these issues
resolved (and it's been 6 months since I submitted the first bug
report (#30660), with no action, AFAIK).

-Tim
*--------------------------------------------------------------------------*
| Tim Maher, PhD     (206) 781-UNIX      (866) DOC-PERL     (866) DOC-UNIX |
| tim(AT)Consultix-Inc.Com  http://TeachMePerl.Com  http://TeachMeUnix.Com |
*+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-*
|      Watch for my upcoming book: "Minimal Perl for UNIX/Linux People"    |
| Classes:  2/14: Minimal Perl  2/15: Hashes & Arrays  2/16-18: Int. Perl  |
*--------------------------------------------------------------------------*
-------------- next part --------------
>From tim at consultix-inc.com  Fri Jul  9 09:37:30 2004

This is a bug report for perl from tim at consultix-inc.com,
generated with the help of perlbug 1.34 running under perl v5.8.2.

-----------------------------------------------------------------
[Please enter your report here]

I've either found a bug in Perl, or there's an undocumented
"feature" that I've suddenly run into for the first time.
Either way, /I don't like it/! 8-{ 

In short, a parsing rule (or bug) seems to be operating that says
after two (or more) spaces are seen on the shebang line, no
additional options are recognized! So when one tries to use -s
for switch processing, and that switch is preceded by two spaces,
the result is that the -switch=whatever argument
is /retained/ on the command line, rather than used to set a
variable! When the -w option is in the ignored position, that
fails to enable warnings.

AFAIK, this is not documented behavior, and IMHO, it is not
/desirable/ behavior.

I've included sample programs and output below, and confirmed that
the same results are obtained with Perl versions 5.8.4 and 5.8.2.

-Tim Maher
tim at teachmeperl.com

PROGRAMS:

The _sw ending on a test program means the -s option is first,
and the -w second, and vice versa. The program invocation in each
case is "./scriptname -switch", which should set the $switch
variable in the script to 1. Note that there's nothing
switch-specific about this "two-space" bug -- that's just the
context in which I originally found it, and it demonstrates the
faulty behavior well, by (erroneously) leaving the -switch
argument in @ARGV, and triggering a "possible typo" warning
regarding $switch in that case.

The two_space* versions of the programs are expected to work like
the one_space ones, but they don't.

2004-07-05 12:31                   one_space_sw                   Page 1

#! /usr/bin/perl -s -w
$switch == 1;
warn "Arguments are: @ARGV";


2004-07-05 12:31                   one_space_ws                   Page 1

#! /usr/bin/perl -w -s
$switch == 1;
warn "Arguments are: @ARGV";


2004-07-05 12:32                  two_spaces_sw                   Page 1

#! /usr/bin/perl -s  -w
$switch == 1;
warn "Arguments are: @ARGV";


2004-07-05 12:32                  two_spaces_ws                   Page 1

#! /usr/bin/perl -w  -s
$switch == 1;
warn "Arguments are: @ARGV";


OUTPUT:

RUNNING 'one_space_sw':
Useless use of numeric eq (==) in void context at ./one_space_sw line 3.
Arguments are:  at ./one_space_sw line 5.

(That's the correct result; warnings enabled, and switch-argument processed)

RUNNING 'one_space_ws':
Useless use of numeric eq (==) in void context at ./one_space_ws line 3.
Arguments are:  at ./one_space_ws line 5.

(That's the correct result; warnings enabled, and switch-argument processed)

RUNNING 'two_spaces_sw':
Arguments are:  at ./two_spaces_sw line 5.

(That's the WRONG result; warnings DISabled;
	only switch-option recognized)

RUNNING 'two_spaces_ws':
Useless use of numeric eq (==) in void context at ./two_spaces_ws line 3.
Name "main::switch" used only once: possible typo at ./two_spaces_ws line 3.
Use of uninitialized value in numeric eq (==) at ./two_spaces_ws line 3.
Arguments are: -switch at ./two_spaces_ws line 5.

(That's the WRONG result; switch not processed;
	only warning option recognized)
 
==============================================================
| Tim Maher, Ph.D.                    tim(AT)TeachMePerl.com | 
| Seattle Perl Users Group        http://www.SeattlePerl.com |
| SPUG Wiki Site               http://Spugwiki.Perlocity.org |
==============================================================


[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl v5.8.2:

Configured by root at Sat Nov 15 08:54:26 PST 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 2) configuration:
  Platform:
    osname=linux, osvers=2.4.19-4gb, archname=i686-linux-thread-multi
    uname='linux jumpy 2.4.19-4gb #1 mon aug 4 23:38:42 utc 2003 i686 unknown '
    config_args='-Dusethreads'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O3',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing'
    ccversion='', gccversion='3.2', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/ucblib
    libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    

---
@INC for perl v5.8.2:
    /local/perl5/site_perl/5.8.2
    /usr/lib/perl5/site_perl/5.8.2
    /local/perl5/site_perl/5.8.0
    /usr/lib/perl5/site_perl/5.8.0
    /local/perl5/site_perl/5.6.0
    /local/perl5/site_perl/5.005
    /local/perl5/site_perl/i386-linux
    /usr/lib/perl5/site_perl/5.8.0/i586-linux-thread-multi
    /usr/local/lib/perl5/5.8.2/i686-linux-thread-multi
    /usr/local/lib/perl5/5.8.2
    /usr/local/lib/perl5/site_perl/5.8.2/i686-linux-thread-multi
    /usr/local/lib/perl5/site_perl/5.8.2
    /usr/local/lib/perl5/site_perl
    /local/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl/5.8.0 /local/perl5/site_perl/5.6.0 /local/perl5/site_perl/i386-linux /local/perl5/site_perl/5.005
    .

---
Environment for perl v5.8.2:
    HOME=/home/tim
    LANG=en_US
    LANGUAGE (unset)
    LC_COLLATE=POSIX
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/tim/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome2/bin:/opt/gnome/bin:/opt/kde3/bin:/opt/kde2/bin:/usr/lib/java/bin:.:/usr/X11R6/bin:/opt/kde/bin:/local/timbin:/local/dtp:/home/tim/bin:/usr/X11R6/bin:/opt/kde/bin:/local/timbin:/local/dtp:/home/tim/bin
    PERL5LIB=:/local/perl5/site_perl/5.8.2:/usr/lib/perl5/site_perl/5.8.2:/local/perl5/site_perl/5.8.0:/usr/lib/perl5/site_perl/5.8.0:/local/perl5/site_perl/5.6.0:/local/perl5/site_perl/5.005:/local/perl5/site_perl/i386-linux:/usr/lib/perl5/site_perl/5.8.0/i586-linux-thread-multi
    PERL_BADLANG (unset)
    SHELL=/bin/bash

-------------- next part --------------
contix at jumpy:/tmp> date
Sat Jan 15 10:26:02 PST 2005

contix at jumpy:/tmp> nl -ba good
     1  #! /usr/bin/perl -s -wlnaF:
     2  print "Switch variable is: $switch";
     3  print "F1: $F[0], F2: $F[1]";

contix at jumpy:/tmp> date | ./good -switch=3
Switch variable is: 3
F1: Sat Jan 15 10, F2: 26

contix at jumpy:/tmp> nl -ba bad
     1  #! /usr/bin/perl -wlnaF: -s
     2  print "Switch variable is: $switch";
     3  print "F1: $F[0], F2: $F[1]";

contix at jumpy:/tmp> date | ./bad -switch=3
Name "main::switch" used only once: possible typo at ./bad line 2.
Can't open -switch=3: No such file or directory.

[The switch argument was incorrectly interpreted as a filename!]

contix at jumpy:/tmp> diff bad good
1c1
< #! /usr/bin/perl -wlnaF: -s
---
> #! /usr/bin/perl -s -wlnaF:
contix at jumpy:/tmp> 


More information about the spug-list mailing list