Phoenix.pm: keep-alives with RobotUA
Scott Walters
scott at illogics.org
Fri Dec 5 12:45:43 CST 2003
Not to mention you've stopped giving them links to try and follow
if they are just a niave robot.
Re: Sun E450s, that's on the NetBSD supported hardware list, so there is a good
chance that Linux supports it also (or that support could be added). Given a
choice between running Solaris on E450s and running Linux on a Dell, I'd have
to go with the E450s. Consumer hardware sucks indescribeably bad. I keep
trying to replace slowass.net's 180mhz PowerPC machine but
each machine dies. So far it has outlived a 300mhz K6II, a 450mhz K6II, a
200mhz PII, and now I've got a 1.2ghz Athlon that suddenly decided that it is
going to report its clock speed at 1.2ghz odd but run about as fast as a 300 on
the bytebench. Consumer hardware is nothing but bonepile fodder.
-scott
On 0, Michael Friedman <friedman at highwire.stanford.edu> wrote:
>
> David,
>
> On Dec 5, 2003, at 7:04 AM, David A. Sinck wrote:
> > I'm guessing you're not using a pentium 90 too. :-)
>
> Heh. No, not Pentium 90s. More like 20 Sun E450s plus a multi-terabyte
> disk server. We've looked into Linux, though, and it could actually
> handle our traffic pretty well, if we had enough standard Dell (or
> similar) machines to host everything.
>
> > \_ Now, we've taken steps to avoid such behavior. If you hit one of our
> > \_ sites that fast, we'll block you from getting any pages within a
> > \_ second.
> >
> > This sounds like a juicy bit, if you care to illucidate?
>
> We wrote an Apache module (in C, but the prototype was in mod_perl)
> that keeps a log of traffic by user. If the module sees too many
> requests too fast from a single machine, single browser, or single
> proxy server, it aborts the Apache delivery process and sends back a
> warning page instead. Then it keeps a list of people who have been
> blocked and slowly lets the blocks expire over time.
>
> Since the module is high up in Apache's chain, it takes less effort to
> block someone than to serve their requested page, thus reducing the
> load on our machine and (since the warning page is small with no
> graphics) reducing the network traffic.
>
> You can still flood the server if you try hard enough with a DDOS, but
> it takes more effort than we have enemies, if you know what I mean.
>
> -- Mike
> ---------------------------------------------------------------------
> Michael Friedman HighWire Press, Stanford Southwest
> Phone: 480-456-0880 Tempe, Arizona
> FAX: 270-721-8034 <friedman at highwire.stanford.edu>
> ---------------------------------------------------------------------
>
More information about the Phoenix-pm
mailing list