Conditional http get for dynamic pages
scottp at dd.com.au
Sat Nov 23 19:50:51 CST 2002
On Saturday, Nov 23, 2002, at 16:13 Australia/Melbourne, David Dick
> G'day all,
> This is slightly off topic, but i figure there is some interest in web
> stuff in the perl-mongers. I've been reading the http rfc over the
> last week and realising just how cool conditional gets are for dynamic
> pages. It seems to reduce network traffic by a good amount. Is anyone
> else using them for dynamic pages? Any experiences on this?
Are you talking about standard IMS request (If Mod Since). If so that
has been a part of the standard for many years and built into most
proxy and web servers. For example every time you make a request
through SQUID it will do an IMS request to the up stream server to only
get the response if has been modified.
However it is of course extremely difficult to do with dynamic pages.
My preference is along the lines of this. If it is a dynamic page that
changes only occasionally then you are best off generating the page. A
good example of this would be the slashdot home page. Generate the page
off line and let the web server deliver it. This has many advantages
the most of which is that you don't have to write special code.
However, if it is a dynamic page which changes depending on the user
logged in etc, then you are not going to get any advantage out of an
IMS header response. There are a few reasons for this. A proxy cache
generally does not cache these pages (it can do but is almost pointless
so generally don't). Another is that things change in too complicated a
way. Theoretically you should be able to get cookies and stuff with
your IMS, but all this overhead generally slows things down quite a bit
To reduce network traffic for dynamically generated pages I recommend
the following strategy:
- Generate the pages and put them somewhere sensible if they can be
cached. If they can't be cached (eg: aways changing) then you gain
nothing from IMS anyway.
- Use a reasonable expiry time
- GZIP the pages. If you use the right settings in apache they only
get compressed once so it is reasonably efficient. You can even go
further and compress yourself and then NOT check if the file is
changed, just always send the current GZ file if GZIP is supported.
Open source developer
scottp at dd.com.au
Dismaimer: Open sauce usually ends up never coming out (of the bottle).
More information about the Melbourne-pm