I think I’ve just found what seems to be a nasty bug in Drupal.
You can specify a page to display if a requested URL isn’t found. However, if you request a URL that starts with “/node/” this page doesn’t seem to be displayed - you just seem to end up back on the referring page, but with a seemingly new URL which is a product of the existing URL plus the requested URL.
Why is this bad? Because if you inadvertantly miss out the “http:” bit from a link and therefore create a local link instead of an external one, you can end up creating an infinitely recursive sequence of “pages” that link to each other. In reality they are all just the same page, but each URL for each one is different.
This isn’t an issue for humans, but it is for you if a dumb crawler stumbles across your site. It will just sit in a loop hammering the same page… forever. Nasty.
Needless to say I discovered this bug by making the mistake myself. And I only spotted it (a long time later) when looking through the logs wondering why on earth a page on my site was continuously being requested that contained the URL www.apple.com in the middle of it!