Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: Site owners check your site for robots.txt file!

__/ [ softwarelabus@xxxxxxxxx ] on Tuesday 08 August 2006 19:31 \__

> Big Bill wrote:
>> On 8 Aug 2006 07:16:01 -0700, softwarelabus@xxxxxxxxx wrote:
>> >Hi,
>> >
>> >I wanted to warn all website owners that some evil web hosts like
>> >vistapages will periodically place a robots.txt file on your site that
>> >disallows all search engines.  It happened to me.
>> >
>> >Over the last several months I've noticed my web traffic dropped to
>> >nearly zero.  A few days ago I noticed a new file, robots.txt.  As most
>> >of you know, if your site has a robots.txt in your websites home
>> >directory then all search engines will look at it for possible
>> >instructions. The robots.txt file tells search engines what to do or
>> >what not to do. In my case, it had simple instructions to disallow all
>> >user-agents; i.e., telling all sites they cannot come here.
>> >
>> >How to check:
>> >If you web site is called www.mywebsite.com  then you want to check the
>> >following web page:
>> >www.mywebsite.com/robots.txt
>> >
>> >You should also look for this file when you ftp to your site in case
>> >your web host places a sneaky server script to make robots.txt
>> >invisible only to you.
>> >
>> >Paul
>> I guess your web hosts don't like you, do they? Did you ask them about
>> this?

To put it politely, your Web host got you f**ked. This is unacceptable unless
you received some form of notification and an opportunity to rebutt. I have
heard similar stories before though (Cat, are you there?).

> BB, I had several domains with vistapages, but only switched one domain
> so far.  It was a very important domain-- business related.

Now that you mention this, I suspect this affected Darren as well. When the
shared server is getting overloaded, rather than upgrading or
redistributing, the host filters out visitors (indirectly -- humans too).
It's laziness; or an attempt to cut down running costs.

> As far as contacting vistapages, I can't bear yet another conversation
> with them.  And call me cheap for not yet switching all my domains to
> another host because they still have my money and laugh whenever I ask
> for a partial refund.

I suggest you get a host you can trust. If you can't speak to your host and
if dread an argument with the host, then the relationship is unhealthy. It's
time to break up.

> BTW, so far I've had good luck with bethehost.com-- knock on wood.
> Except they've been under severe attacks from a hacker this week and
> had to take the entire server down for about 4 hours.  I still give
> them an A grade. They always email everyone with server updates, etc.
> As for vistapages, I can't recall a single informative email from them.
> Anyhow, I would appreciate other web host recommendations.

I have been happy with http://catalyst2.com for the couple of years that I
was with them (still going strong). I have 3 domains on their servers and
communication with them is a snap (spoke to them last night).

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index