[Chicago-talk] Login

petemar1 petemar1 at perlmonk.org
Tue Jan 20 18:30:29 CST 2004


Uh...Ed:

1. I do use strict; # ...!

#!/usr/local/bin/perl -w

############################################################################
#
# FILE: ScrapeHandler.pm
#
# AUTHOR: Marc Peters
#
# PURPOSE:
# This is a "proof-of-concept" web-crawling application that performs
# "screen-scrapes" on ebay.com and stores info in a flat-file.
#
# USAGE:
#
#
# REFERENCES:
# ISBN 020-148-5672 (refactoring)
# ISBN 059-600-2890 (regex)
# ISBN 059-600-415X (regex pocket, NEW)
# ISBN 059-600-5776 (spidering hacks, NEW)
#
# REVISION HISTORY:
# Version 0.0.1, January 10, 2004: Started development.
#
# COMMENTS:
# Based on petemar1/ScrapeBay/ScrapeHandler.java,
# started November 26, 2003
#
############################################################################
#

use strict;
use warnings;
use FindBin::libs;
use IO::File;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;
use URI::Heuristic;
use WWW::Search;
use WWW::Search::Ebay;
use WWW::Search::Ebay::Completed;
use WWW::Mechanize;

# ...


2. # ...!!

# ...

    ###
    # scrape(regex)
    #
    # sub scrape {
    sub scrape(\$) { # :REFACTOR: MMP, JAN/18/2004

        unless (eval { # try

            my $oSearch = new WWW::Search('Ebay::Completed');

            $oSearch->login($user, $pass) || die "Can't login!";

            my $sQuery = WWW::Search::escape_query("canon digital
amera  -battery -case -cable -card -memory -camcorder");

            $oSearch->native_query($sQuery);

            print "\n searching  \n";

            open(TEMPFILE, "> tempfile.txt");

            my $n = 0;

            while (my $oResult = $oSearch->next_result()) {
                  #^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^HERE

                print TEMPFILE $oResult->url, "\n";

                print $oResult->url, "\n";

                $n++;
            }

            $oSearch->logout;

            close(TEMPFILE);

            snatch($_[0]) || die "SNATCH ERROR!"; # :REFACTOR: MMP,
JAN/18/2004

            getSeller();

            my $testvar = 0;

            1
        } ) # end unless()

        { die "\nSCRAPE ERROR!\n"; } # catch

    } # end Scrape()




3. # ...!!!

http://216.239.53.104/search?q=cache:ANqNscq6EYIJ:search.cpan.org/~mthurn/WW
W-Search-Ebay-2.17/++%22ebay+completed%22+site:cpan.org&hl=en&ie=UTF-8

Are you conspiring with "Kingpin" to undo me? DON'T WAIT FOR THE
TRANSLATION! ANSWER ME NOW! THE POWER OF GOOGLE COMPELS YOU! THE POWER OF
GOOGLE COMPELS YOU! THE POWER OF GOOGLE COMPELS YOU!

http://216.239.53.104/search?q=cache:WMP-OYph1X8J:search.cpan.org/src/MTHURN
/WWW-Search-Ebay-2.18/ChangeLog++%22ebay+completed%22+site:cpan.org&hl=en&ie
=UTF-8

Must have been removed within the past week.

Even using Mechanize to login causes problems. The request returns
successfully but I'm unable to scrape completed auctions.




-----Original Message-----
From: chicago-talk-bounces at mail.pm.org
[mailto:chicago-talk-bounces at mail.pm.org]On Behalf Of Ed Summers
Sent: Tuesday, January 20, 2004 4:08 PM
To: chicago-talk at mail.pm.org
Subject: Re: [Chicago-talk] Login


petemar1 wrote:
> # :UNRESOLVED: login and scrape with WWW::Search::Ebay::Completed, MMP,
> JAN/20/2004
>
> my $oSearch = new WWW::Search('Ebay::Completed');
>
> $oSearch->login($user, $pass);
>
> my $sQuery = WWW::Search::escape_query("canon digital
> camera  -battery -case -cable
> -card -memory -camcorder");
>
> $oSearch->native_query($sQuery);
>
> print $oResult->url, "\n";
>
> $oSearch->logout;

First off, I don't know what Ebay::Completed is. Shouldn't that be

    my $oSearch = new WWW::Search( 'Ebay' );

The results of a search (your $oSearch object) can have many results. This
is why the docs [1] tell you to use the next_result() iterator to fetch
them. So instead of:

    print $oResult->url, "\n";

You need:

    while ( my $oResult = $oSearch->next_result() ) {
        print $oResult->url(), "\n";
    }

If you were using 'strict' you would've noticed that your $oResult
variable was never declared, because your program wouldn't compile!. There
is
rarely a reason not to have this at the top of your Perl program.

    use strict;

//Ed

[1] http://search.cpan.org/perldoc?WWW::Search::Ebay
_______________________________________________
Chicago-talk mailing list
Chicago-talk at mail.pm.org
http://mail.pm.org/mailman/listinfo/chicago-talk




More information about the Chicago-talk mailing list