[VPM] sqlite and perl for performance

will mock at obscurity.org
Sat Mar 12 20:04:36 PST 2011


I've done this with the dbd::sqlite module before, but rather than using
threads, I used POE and POE::Component::EasyDBI.  If I recall correctly, you
needed to be careful about sqlite locking the database file for write (I
seem to recall implementing some retry logic to deal with that) and thus
preventing other processes from writing at the same time.  But that
shouldn't be an issue for you if you're reading.

mock

On Sat, Mar 12, 2011 at 6:46 PM, Brendan <brendan at ming.org> wrote:

>  On 11-03-12 05:27 PM, Jer A wrote:
>
> I want to get data from multiple databases using sqlite.
> instead of processing command at a time, or a chain sequence,
> I would like to execute them in multiple threads, and have them all join.
> this is going to be used in a cgi script, and possibly a cron job.
> are there any issues using sqlite and perl in this manner?
>
>
> it's probably safe, so long as you declare the dbi object inside each
> thread (if you declare it prior to starting up the worker threads, you could
> end up with problems). dbd::sqlite is probably thread safe, but if you are
> worried DBIx::Threaded purports to make any DBD module thread safe (though
> i'm sure you'll pay some price in memory usage).
>
> Threaded::Queue or Threaded::Queue::Duplex can help you get the data back
> to your master thread (instead of using a perl shared variable - the two
> queue modules use threadshared objects, but they are without any of the
> problems you might accidently cause yourself using :shared)
>
> _______________________________________________
> Victoria-pm mailing list
> Victoria-pm at pm.org
> http://mail.pm.org/mailman/listinfo/victoria-pm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/victoria-pm/attachments/20110312/5c8b1fc6/attachment.html>


More information about the Victoria-pm mailing list