[Xitami] Xitami has slowed to a crawl

[Ron Ackerman]

That will exclude all robots, I'm not sure I want to do that because I would
like to be included in the Search Engines. Is there a time setting that can
be applied. Like Excluding the robots between 7am and 11pm?

-----Original Message-----
From: xitami-admin@lists.xitami.org [mailto:xitami-admin@lists.xitami.org]On
Behalf Of Jeff Hamblin
Sent: Tuesday, January 11, 2005 12:39 PM
To: xitami@lists.xitami.org
Subject: Re: [Xitami] Xitami has slowed to a crawl


>I think I'm finding the source of my slow downs. Crawlers and the worst
> right now is msn bot. But I'm also seeing Yahoo Slurp. Which normally I
> woudn't mind them indexing my websites but when it is affecting the server
> then it's a problem.


Try putting a "robots.txt" file in the root of your website with the
following:

# robots.txt file to stop indexing of site
User-agent: *
Disallow: /


--
Xitami Users Mailing List -- For Xitami support
To unsubscribe: http://lists.xitami.org/mailman/listinfo/xitami
--
No virus found in this incoming message.
Checked by AVG Anti-Virus.
Version: 7.0.300 / Virus Database: 265.6.10 - Release Date: 01/10/2005

--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.300 / Virus Database: 265.6.10 - Release Date: 01/10/2005