[sf-perl] Testing a web crawler

Neil Heller nheller at silcon.com
Sun Dec 30 12:22:35 PST 2007

I have been asked to consider ways to test a web crawler (aka search

There are lots of "positive"-type tests I can think of that mostly deal with
"are the returned pages really relevant to the request".

How does one test for pages that are (or might be) relevant but were missed
by the web crawler?

What might be the best (is that a trick word?) or most optimized request
given someone's desire to find information?

More information about the SanFrancisco-pm mailing list