[DFW.pm] Hackathon: architecture description

[Tom Metro]

As your contest entries will be reviewed by the judges, and in some
cases discussed at a Perl Mongers meeting, it would be appreciated (but
not required) that you include a high-level description of your
architecture. Either as a separate file (Architecture.pod?) or part of
the POD in your main code file.

It doesn't have to be much. Take for example the architecture summary
Tommy wrote for the reference design in his Perl Monks posting
(http://perlmonks.org/?node_id=1069338):

  1. Traverses the filesystem.

  2. Groups same-size files, tossing out the rest. (This is not threaded.)

  3. Takes each group and reads the first few bytes each file, creating
  sub-groups based on the bytes read. Then it removes sub-groups with a
  single element, thereby "throwing out" the non-similar files from the
  parent group.

  4. Makes a second pass at the above, but at the end of the file (the
  efficiency of this second pass is debatable but shows good results).

  5. Adds up the final N number of files to be processed in a :shared
  variable.

  6. Creates thread pool with worker threads and shoves 30 files at a
  time into their queues and waits until the threads have incremented
  the number of files they've processed to equal N
     -The threads digest the files in their queues in their entirety
     (this is bad?)

  7. Main thread signals to the threads that they are done by ending
  their queues and finally joins them.


It should cover the major steps your codes takes to solve the problem,
the order in which it does them, and in particular, highlight any spots
where you feel you are doing something atypical, like using parallel
processing.

 -Tom

-- 
Tom Metro
The Perl Shop, Newton, MA, USA
"Predictable On-demand Perl Consulting."
http://www.theperlshop.com/