Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

What is this thing spidering my site?

I would generally hate to turn away legit crawlers, but a particaulr address
( sls-gc8p11.dca2.superb.net ) has been harvesting one of my sites for 4
days, taking up about 300 MB of pages per day. What can it be? Using HTTP on
that address gives the default Apache installtion page. Should I just block
all such mysterious requests? Any idea what it might be?

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index