Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: deny access for search bots

__/ [bastardx] on Friday 16 September 2005 11:40 \__

> How to stop search engines indexing for all websites.
> Is there any  trick to do that with apache?

In the home directory's public area (e.g. ~/public_html) of all Web sites,
add the file robots.txt containing the text:

User-agent: *
Disallow: /


All the details in: http://www.robotstxt.org/wc/robots.html 

Hope it helps,

Roy

-- 
Roy S. Schestowitz      | chmod a-r *.mbox
http://Schestowitz.com  |    SuSE Linux    |     PGP-Key: 74572E8E
 11:40am  up 21 days 23:54,  4 users,  load average: 0.22, 0.22, 0.29

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index