[DCPM] Profiling

[Simon Waters]

Aaron Trevena wrote:
>
>> However when I switch YAML::XS for YAML the profiling figures still
>> claim the method that takes most time in YAML is taking a long time, but
>> it isn't assigning this time to YAML::XS.
>>
>> Is this typical?
> 
> It could be including the xs in the lowest level perl call ?

That is what I thought, but as far as I can see NYTProf is assiging it
to our routine "read_file", but not to any line in "read_file" (when
presumably YAML::XS::LoadFile is consuming most of the time - since it
doesn't do much else).

> Are you using version 3 ? ISTR it provided improved XS handling.

Version 3 of NYTProf - yes.

It seems to assign time as I would expect for most calls to regular
operations(?) that are implemented in C, just not here.

I'm fairly confident the YAML parsing time is a significant issue, but
aside from spending too long parsing it, we read the files too often,
they are bigger than they need to be, and as I mentioned we don't use
"LoadFile" and "DumpFile" (rather someone calls Load and Dump on data
loaded or written to files, which is almost but not exactly the same
thing - think UTF8 encoding).

The module YAML (not YAML::XS) has a regular expression at the heart of
it burning the CPU (with a comment to that effect next to it), is
written to a prerelease spec of YAML, and I think is probably best
avoided (says the guy with millions of such files to manage - ho hum).