I/O status Re: SPUG: proper checking of file status after read?

[Fred Morris]

At 9:47 PM 9/18/03, Ben Reser wrote:
>On Thu, Sep 18, 2003 at 08:12:48PM -0700, Fred Morris wrote:
>> At 9:08 AM 9/18/03, Ben Reser wrote:
>> >if (open(FH, $filename)) {
>> >    # Here $! is meaningless.
>> >    ...
>> >} else {
>> >    # ONLY here is $! meaningful.
>> >    ...
>> >    # Already here $! might be meaningless.
>> >}
>> ># Since here we might have either success or failure,
>> ># here $! is meaningless.
>>
>> In the real world, this has never been the case, but thanks for pointing it
>> out. This sounds like bad documentation more than anything else: there are
>> side effects which call system libraries which mere mortals are unaware or
>> uninformed of. (Nobody's encountered crufty and perhaps slightly arrogant
>> documentation before?) Or are we to believe that $! is used merely for
>> scratch when the whim strikes?
>
>What you seem to be missing is the key sentence in the following
>paragraph:
>
>> >A successful system or library call does not
>> >set the variable to zero.
>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>Which means, if errno got set for whatever reason previously it will
>still be set even after a successful library call.  This is by *DESIGN*.
>$! is not used to tell you that an error occured, the return value of
>the operation tells you if it was successful or not.  It simply provides
>you the error number of the *LAST* error.

There are times I consider documentation to be the final arbiter, and times
I consider behavior paramount.

I observe that perlvar seems to ignore status checking on read in its examples.

>Theoretically you could work around this by setting $! to 0 prior to
>starting your while loop.  But this is also possible to break.  Say perl
>ends up calling some function to request some information be put in a
>buffer and the buffer was too small so the call sets the errno.  Even if
>perl handles this buffer issue for you, then the errno will be set.
>
>At any rate I wouldn't consider the documentation wrong, crufty or
>arrogant.
>
>> Herein is the allusion to buffering gone bad... (do that, doesn't fail)
>>
>> >@lines = <INFILE> or die "Error reading /path/spec: $!";
>>
>> How is this different from a while loop on individual lines until
>> (presumably) an undef occurs?
>
>It's not different, except that it has error handling in a much simpler
>way.  To try and detect and error while reading with while (<FH>) {...}
>would be difficult.

I'm sorry.

Hold that thought....

>> In the newer versions of Perl, (no I stand
>> corrected, any version of Perl 5) this would fail if the file was empty.
>
>I can't imagine why you wouldn't consider reading an empty
>file an error condition...  If there's nothing to read it's not a
>successful read.

I consider it a success because the failure to read data was not due to a
failure of the storage management layer. Is there another language which
considers end-of-file on initial read to be a storage layer failure?
(Incidentally, I would say that Perl does not consider it an error
condition either.) Attempting to read beyond EOF is considered an error in
many languages; languages which do so on initial read typically provide an
EOF test, used as until EOF() read(). Try for mainstream here; provide an
example or be discarded. Even with SQL, reading 0 records returns an empty
set.. not an error. man errno(3) lists ENOTTY as Inappropriate I/O control
operation, but there is no error for end of file or empty file. BTW,
attempting to read beyond EOF in Perl 5.8 reads 0 lines and does not set
$!. That's really a side issue, I'm not going to go back and test in other
versions.

I can tell this might be the start of a religious difference of opinion, so
I'm simply going to state mine and move on: reading 0 bytes from a file is
not an error (at least initially) if the file is empty and there is no
provision for testing for EOF, and the reason for that is so that reading
bytes (or the lack thereof) is distinguishable from a storage management
layer error... unless the storage management layer chooses to return an end
of file condition as an error, of course.

>> However, if there was no error, are you saying that $! is "meaningless"?
>> This is the snag, you see.
>
>If there is no error the die call never happens so $! is never used.

That's predicated on the religious opinion that reading 0 bytes is an error.

>> And it's not just "any value", it's an ioctl.. every time.
>>
>> As a matter of fact, I just tried your suggestion on an empty file with
>> perl v5.8.0 i586-linux-thread-multi (plus SuSE patches) and by golly I got
>> Inappropriate ioctl for device.
>>
>> On 5.005_03 $! is .. well... false.
>>
>> Anything else I missed here? Yes. The system with 5.5.3 is on an ext2 fs,
>> and the system with 5.8.0 is reiserfs. FWIW.
>
>reiserfs is considering the attempt to read an empty file an error and
>setting errno.  ext2 apparently doesn't consider it as such.

This has just been proven to be an unfounded and in fact incorrect assumption.

>Not all
>filesystems are created equally.

Apparently they are in this case. Perhaps it's Linux? No, just tried it on
a Solaris box with 5.5.3, and $! was false. Anybody else out there care to
try and flesh out the failure matrix?

>However, perl does consider it an error because there is nothing to put
>in the array i.e. the assignment failed.

OK, I understand: your argument is that Perl has been "fixed" so that it
conforms to the documentation. That conforms to the matrix thus far, at any
rate. (Or taking it at face value, this three line program is generating a
valid Inappropriate ioctl error unrelated to reading an empty file? No,
just tested that and in fact $! is being set to Inappropriate ioctl by the
open, it's false prior to that; this is in spite of the fact that open
returns successfully.) (So Perl has been "fixed" to conform to the
documentation by having open set $! to an error number while returning
successfully? Occam, sharpen your razor!)

>> But again: I am sure that the file(s) in question in the real app are *not*
>> empty. How does one check for a "no problem" condition when at end of
>> file.. because we can ascertain, by both documentation and practice, that
>> "end of file" is reached when the end of file is reached as well as when
>> there is an I/O error.
>>
>> Am I a flipper baby for wanting to be able to check for I/O read status?
>> Does nobody do this? Wow.
>
>This is generally not a concern when reading files in perl because perl
>handles the I/O issues for you.  Sounds to me like you're trying to do
>work that's already handled for you...

"Generally" is not the issue here. I have a specific problem, and this
peccadillo of Perl is interfering with my ability to trace it.
Specifically, I appear to be having a problem with mod_perl on Apache 2
where file I/O (input, specifically) is failing in an inconsistent manner.
Therefore there exists a reasonable basis for concluding that Perl is *not*
handling the I/O issues for me. Reading $! is proving not to be a reliable
mechanism for determining I/O status.

I need a reliable mechanism for determining I/O status; I am appealing for
suggestions to this end. I am profoundly uninterested in continuing to
bloviate on the Perlishness of Perl, except insofar as it may provide a
means to refine this appeal.

The section in perlvar on _Error Indicators_ states that $! corresponds to
errors detected in the C library. perlvar says about $!: 'You can assign a
number to $! to set errno if, for instance, you want "$!" to return the
string for error n, or you want to set the exit value for the die()
operator." Should I set it to undef or 0 prior to the loop? It does
eliminate the Inappropriate ioctl message when used on the test program.

Is this best practice? As good as it gets? Can anybody else confirm this?

Have I been enlisted in yet another snark hunt because "it's not a problem
with Perl" that $! is not being altered by the read, and that "yes it is a
problem with Perl" that open is returning success (and successfully) yet
leaving $! set to Inappropriate ioctl (which it does even when the file is
*not* empty, for anybody who came in late or has a short attention span..
and I retested this just now with the test program to be sure)? Is that
cause for concern? (Occam? Occam?)

>"Conscience is the inner voice which warns us somebody may be looking."
>- H.L. Mencken

"Why do people go to the zoo?" -- H.L. Mencken, when asked why he didn't
leave America since he didn't have anything good to say about it.

--

Fred Morris
m3047 at inwa.net