<div dir="ltr">I've had very good experiences using Linux Copy on Write semantics in Perl programs, both in the context of Apache 2.4 + mod_perl2 and Perl scripts using Parallel::ForkManager.<div><br></div><div>The key is to set things up in the parent process - load modules and config files, pre-warm in-process caches or whatever. Then do not touch those variables again after you fork. Watch out for writes that look like reads, eg. hash key autovivification! Make good use of exists()</div><div><br></div><div>In Apache you can use the PerlPostConfigHandler config option to run things pre-fork in the parent.[0]</div><div><br></div><div>As an aside you can use PerlChildInitHandler for running things post-fork in the child - I recommend running 'srand' at the very least.[1]</div><div><br></div><div>One of the most useful things you can do is to learn how different ops tools treat CoW. Unfortunately the answer is basically that they all[2] ignore CoW and will show all your child processes as using lots of memory. The most reliable source of truth is /proc/<pid>/smaps which has {Shared/Private}/{Clean/Dirty} values.[4]</div><div><br></div><div>One final point - I specified at the top of the email that I was talking about "Linux Copy on Write". That's because Perl has its own concept of Copy on Write[5] which is used to optimise assignments which are semantically copies but might not need to be.</div><div><br></div><div>sub foo {</div><div> my $arg = $_[0];</div><div> return $arg + 4;</div><div>}</div><div><br></div><div>@_ contains aliases, and $arg = $_[0] makes a new scalar containing a "copy" of the contents. But actually Perl doesn't copy immediately, it sets up a CoW variable which avoids actually duplicating the memory until it absolutely has to.</div><div><br></div><div>Hope that helps :) I also hope it's all accurate... if anybody notices any errors please correct me :)</div><div><br></div><div>- Alex</div><div><br></div><div><br></div><div>[1] <a href="https://perl.apache.org/docs/2.0/user/handlers/server.html">https://perl.apache.org/docs/2.0/user/handlers/server.html</a></div><div>[2] <a href="http://blogs.perl.org/users/brian_phillips/2010/06/when-rand-isnt-random.html">http://blogs.perl.org/users/brian_phillips/2010/06/when-rand-isnt-random.html</a></div><div>[3] top, free, etc.</div><div>[4] <a href="https://www.brightbox.com/blog/2012/11/28/measuring-shared-ram-usage/">https://www.brightbox.com/blog/2012/11/28/measuring-shared-ram-usage/</a></div><div>[5] <a href="http://perldoc.perl.org/perlguts.html#Copy-on-Write">http://perldoc.perl.org/perlguts.html#Copy-on-Write</a></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 19 January 2017 at 06:03, Dean Hamstead <span dir="ltr"><<a href="mailto:dean@fragfest.com.au" target="_blank">dean@fragfest.com.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>perhaps TonyC on #australia (<a href="http://irc.perl.org" target="_blank">irc.perl.org</a>) can provide some
feedback<br>
</p>
he is seemingly perpetually working on tpf grants fixing core bugs<br>
<br>
<br>
D<div><div class="h5"><br>
<br>
<div class="m_2670084075193268093moz-cite-prefix">On 19/01/17 12:24, Mathew Robertson
wrote:<br>
</div>
</div></div><blockquote type="cite"><div><div class="h5">
<div dir="ltr">Here is an interesting read on how Python's
garbage-collector, causes Linux's copy-on-write, to become less
effective than would otherwise:
<div><br>
</div>
<div><a href="https://engineering.instagram.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172#.25rzyh6im" target="_blank">https://engineering.instagram.<wbr>com/dismissing-python-garbage-<wbr>collection-at-instagram-<wbr>4dca40b29172#.25rzyh6im</a></div>
<div><br>
<div>Essentially it amounts to the GC walking over Python's
read-only variables, but still adjusting the
reference-counters... thus causing Linux to see that the
memory that is backing the Python instance, to be written
to.</div>
<div><br>
</div>
<div>Python's cycle-detection could also cause memory-writes
due to the mark+sweep job.</div>
<div><br>
</div>
<div>And it also has a generational collection, for short-term
vs long-term objects. Adjusting lifetime lengths, would also
cause a memory-write.<br>
</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>Perl also uses reference counting. So I can also see that
Perl's reference-counting would cause the references to be
written, when a given Perl variable is copied.</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>The question is... does anyone have any insight into the
same CoW failings within the context of Perl 5 ?</div>
<div><br>
</div>
<div><br>
</div>
<div>Perl 6 uses a generational collector (does it *also* use
reference counting?), so the generational migration would
impact CoW. But given the rather smart people working on that
project, I can envisage that some solution may eventually get
implemented.<br>
</div>
</div>
<br>
<fieldset class="m_2670084075193268093mimeAttachmentHeader"></fieldset>
<br>
</div></div><pre>______________________________<wbr>_________________
Melbourne-pm mailing list
<a class="m_2670084075193268093moz-txt-link-abbreviated" href="mailto:Melbourne-pm@pm.org" target="_blank">Melbourne-pm@pm.org</a>
<a class="m_2670084075193268093moz-txt-link-freetext" href="http://mail.pm.org/mailman/listinfo/melbourne-pm" target="_blank">http://mail.pm.org/mailman/<wbr>listinfo/melbourne-pm</a></pre>
</blockquote>
<br>
</div>
<br>______________________________<wbr>_________________<br>
Melbourne-pm mailing list<br>
<a href="mailto:Melbourne-pm@pm.org">Melbourne-pm@pm.org</a><br>
<a href="http://mail.pm.org/mailman/listinfo/melbourne-pm" rel="noreferrer" target="_blank">http://mail.pm.org/mailman/<wbr>listinfo/melbourne-pm</a><br></blockquote></div><br></div>