Phoenix.pm: keep-alives with RobotUA

[Scott Walters]

Not to mention you've stopped giving them links to try and follow
if they are just a niave robot. 

Re: Sun E450s, that's on the NetBSD supported hardware list, so there is a good 
chance that Linux supports it also (or that support could be added). Given a 
choice between running Solaris on E450s and running Linux on a Dell, I'd have 
to go with the E450s. Consumer hardware sucks indescribeably bad. I keep
trying to replace slowass.net's 180mhz PowerPC machine but
each machine dies. So far it has outlived a 300mhz K6II, a 450mhz K6II, a 
200mhz PII, and now I've got a 1.2ghz Athlon that suddenly decided that it is 
going to report its clock speed at 1.2ghz odd but run about as fast as a 300 on
the bytebench. Consumer hardware is nothing but bonepile fodder. 

-scott


On  0, Michael Friedman <friedman at highwire.stanford.edu> wrote:
> 
> David,
> 
> On Dec 5, 2003, at 7:04 AM, David A. Sinck wrote:
> > I'm guessing you're not using a pentium 90 too.  :-)
> 
> Heh. No, not Pentium 90s. More like 20 Sun E450s plus a multi-terabyte 
> disk server. We've looked into Linux, though, and it could actually 
> handle our traffic pretty well, if we had enough standard Dell (or 
> similar) machines to host everything.
> 
> > \_ Now, we've taken steps to avoid such behavior. If you hit one of our
> > \_ sites that fast, we'll block you from getting any pages within a
> > \_ second.
> >
> > This sounds like a juicy bit, if you care to illucidate?
> 
> We wrote an Apache module (in C, but the prototype was in mod_perl) 
> that keeps a log of traffic by user. If the module sees too many 
> requests too fast from a single machine, single browser, or single 
> proxy server, it aborts the Apache delivery process and sends back a 
> warning page instead. Then it keeps a list of people who have been 
> blocked and slowly lets the blocks expire over time.
> 
> Since the module is high up in Apache's chain, it takes less effort to 
> block someone than to serve their requested page, thus reducing the 
> load on our machine and (since the warning page is small with no 
> graphics) reducing the network traffic.
> 
> You can still flood the server if you try hard enough with a DDOS, but 
> it takes more effort than we have enemies, if you know what I mean.
> 
> -- Mike
> ---------------------------------------------------------------------
> Michael Friedman                  HighWire Press, Stanford Southwest
> Phone: 480-456-0880                                   Tempe, Arizona
> FAX:   270-721-8034                  <friedman at highwire.stanford.edu>
> ---------------------------------------------------------------------
>