APM: Some progress on efficient filehandle reading, but threads too big

Mike South msouth at shodor.org
Wed Feb 25 10:23:20 CST 2004


>From austin-bounces at mail.pm.org  Wed Feb 25 10:55:26 2004
>From: "Brian Michalk" <michalk at awpi.com>
>To: <austin at pm.org>
>Date: Wed, 25 Feb 2004 09:53:57 -0600
>
...
>Looking at CPAN, Elizabeth Mattijsen has done a lot with threads.  I see
>from her examples with Benchmark::Thread::Size the following output:
>   #   (ref)        bare        full        vars         our      unique
>   0    2172          +0          +0          +0          +0          +0
>   1    2624 ± 4      +4 ± 4      +4 ± 4     +27          +4 ± 4     +27
>   2    3004 ± 4      +2 ± 6      +2 ± 6     +33 ± 4      +8         +36 ± 6
>   5    4126 ± 6      -2 ± 6      -3 ± 8     +29 ± 4     +10 ± 2     +27 ± 4
>  10    5984 ± 8      -1 ± 8      +0 ± 4      +0 ± 6     +17 ± 4     +43 ± 6
>  20    9694 ± 4     +15 ± 4     +15 ± 2     +13 ± 6     +32 ± 6     +58 ± 6
>  50   20832 ± 4     +51 ±10     +50 ± 8     +50 ± 8     +68 ±12     +96 ± 6
> 100   39392 ± 8    +106 ±10    +156 ±12    +108 ±10    +131 ±10    +155 ±12
>
> ==== bare ========================================================
> $VERSION = '0.01';
>
> ==== full ========================================================
> $main::VERSION = '0.01';
>
> ==== vars ========================================================
> use vars qw($VERSION);
> $VERSION = '0.01';
>
> ==== our =========================================================
> our $VERSION = '0.01';
>
> ==== unique ======================================================
> our $VERSION : unique = '0.01';
>
> ==================================================================
>
>I'm not sure how VERSION plays into memory management, so this confuses me,

I suspect that the choice of VERSION has nothing to do with it, but that
the way they are stored does.  When you reference something as being
in package main, or declare it with use vars, our() it or give it
the unique attribute, all of those things affect where it gets put
and who can see it, so to speak (pure speculation here, though,
so hopefully if I'm wrong someone will speak up).

>but it shows that 20 threads are consuming less than 10MB total.  This would
>be great prformance for me.  From my 'top', I'm getting the following:
>
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 3815 root      16   0 28172  27m 1844 S  5.8 47.3   0:17.07 filter_dmi.pl
> 3817 root       8   0 28172  27m 1844 S  0.0 47.3   0:00.00 filter_dmi.pl
> 3818 root       9   0 28172  27m 1844 S  0.0 47.3   0:00.01 filter_dmi.pl
> 3819 root       9   0 28172  27m 1844 S  0.0 47.3   0:00.07 filter_dmi.pl
> 3821 root       9   0 28172  27m 1844 S  0.0 47.3   0:00.00 filter_dmi.pl
> 3822 root       9   0 28172  27m 1844 S  0.0 47.3   0:00.07 filter_dmi.pl
> 3823 root       9   0 28172  27m 1844 S  0.0 47.3   0:00.00 filter_dmi.pl
> 3824 root       9   0 28172  27m 1844 S  0.0 47.3   0:00.00 filter_dmi.pl
> 3825 root       9   0 28172  27m 1844 S  0.0 47.3   0:00.08 filter_dmi.pl
>
>In case you are wondering about the pids, perl -v has this little snippet:
>ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
>
>Why do my ten threads eat 27 megs, when Elizabeth doesn't get there until
>after 50 threads?

Just to be clear, (a) are Elizabeth's numbers up there for code running
exactly the lines above?  In other words, are they for one line scripts that
do only one declaration?  (b) Have you tried running whatever scripts she 
was running just to see if you get comparable results?  I'm just wondering
what the exact apples-to-apples comparison looks like, if you've done it.

mike



More information about the Austin mailing list