[Melbourne-pm] caching CPAN dist files

[Toby Corkindale]

On 20/04/11 11:35, Mathew Robertson wrote:
> Hi Toby,
>
> I am not familiar with how this would do the dependency checking
> required -> how would building a package containing exactly the
> installed files, differ from simply creating a tarball?  ie: it appears
> this technique completely ignores the OS versioning problems...

My method is essentially an OS-versioned tarball.
In the Perl world, you have to accept that you're going to get the 
latest versions of modules, rather than a specific version XXX.
A modern application needs hundreds of modules once you include all the 
sub-dependencies - you would be insane to try and manually vet and 
hand-pick every one.
So my method says "take everything, latest as of now". Then you test 
this and ensure it works for you - if there are breakages (and there 
always are) then you fiddle with the packages involved until its good.

Once you have a "Stable" package, you keep using that until you need to 
upgrade, at which point you start the whole process over, until you have 
another "stable" release.

This gives you the benefits of tracking recent versions of modules from 
CPAN, without having the problems of random module versions introducing 
bugs or changes at difficult points in your development process.

Toby


> On 20 April 2011 11:27, Toby Corkindale
> <toby.corkindale at strategicdata.com.au
> <mailto:toby.corkindale at strategicdata.com.au>> wrote:
>
>     I install a custom Perl into /usr/local/$NAME/perl, and then install
>     the multitude of CPAN modules in there with it using CPANPLUS.
>
>     I have a script that can do the above automatically, and can also
>     update the CPAN modules.
>
>     After everything is in there, I add/update the files to a Git
>     repository, then build a Debian package out of the whole lot.
>
>     Just a single, monolithic package that contains everything.
>
>     Then that gets installed on development and production machines.
>
>     If new modules are required, the process is re-run, and a new single
>     package is built.. this gets tested in development alongside the
>     next upcoming production release, then release to production
>     alongside the next release.
>
>     This seems to work fairly well; I can't claim that the idea is mine
>     - it was inspired by a talk at OSDC a couple of years ago by Adam
>     Kennedy.
>
>     I have intentions to get this cleaned up enough to release as an
>     open-source project, but it's not quite there yet.
>
>     -Toby
>
>
>     On 20/04/11 09:51, Malcolm Herbert wrote:
>
>         I'm in the process of putting together a project that I'm
>         expecting to
>         use a fair number of CPAN modules directly rather than using the
>         system
>         native packages of them, for portability.  I have a few
>         development and
>         test instances and a few instances in production.  I'd like to
>         use CPAN
>         to manage the modules installed on each, however I want to make sure
>         that they stay in sync with each other and avoid potential
>         dependency
>         issues should my development and production environments drift
>         too far
>         apart.  Ideally I'd like to use the CPAN tools to update the
>         production
>         environment only from the set of modules I've collected and
>         tested on my
>         dev/test hosts.
>
>         Is there a good method for doing this?  I've discovered
>         Local::lib from
>         the Catalyst book which looks promising for installing modules into
>         non-system locations which means I can add and update modules
>         without
>         being root on the host, but I can't see how to get CPAN to only ever
>         look in local directories for dist files rather than fetching
>         from the
>         web.
>
>         Also, if down the track I discover a need for module xxx, the
>         current
>         version of which requires a later yyy than I have (ignoring
>         issues of
>         whether this is a good idea) how would I go about trying to
>         locate the
>         latest version of xxx that would be satisfied with the version
>         of yyy
>         that I already have rather than trying to update both?  How long
>         do old
>         versions of modules stick around in CPAN?
>
>         I've been able to avoid precisely this issue with pkgsrc by
>         pre-emptively downloading all the binary packages as they are
>         available
>         in one hit, which has meant I've been able to add new packages
>         to older
>         installs of hosts without issues, at the cost of downloading
>         around 10GB
>         of data ... do people do similar things with CPAN?  How big is
>         CPAN if I
>         were to attempt this?  Are there snapshot versions of CPAN out there
>         which don't change (apart from minor bug fixes)?  How long do
>         they last
>         in that state?
>
>         So many questions ... :)
>
>         Regards,
>         Malcolm
>
>
>
>     --
>     .signature
>
>     _______________________________________________
>     Melbourne-pm mailing list
>     Melbourne-pm at pm.org <mailto:Melbourne-pm at pm.org>
>     http://mail.pm.org/mailman/listinfo/melbourne-pm
>
>


-- 
.signature