[Raleigh-talk] Parallel::ForkManager and progress
bradoaks at gmail.com
Fri Dec 19 11:21:51 PST 2008
On Fri, Dec 19, 2008 at 2:15 PM, Michael Peters <mpeters at plusthree.com> wrote:
> James Olin Oden wrote:
>> You can always just write your stats to a file. Have to add flock()
>> to the equation
>> but it isn't that bad
> I guess I should have said that this process will be doing hundreds of
> billions (with a B) comparisons, so I'd like to keep everything in memory or
> at least in sockets. Having all of those file reads/writes in there gives me
> the willies.
> But I guess that means that using the DB is out too since going over the
> network is even worse than going to the spinning platter.
You could update your counter only once every 10_000 records are
processed. I've done that in the past when I didn't want to overrun
my screen with status messages. Same would work for this if you don't
mind the total being off by less than a percent (times the number of
processes) when you check it.
More information about the Raleigh-talk