does Crawl-Delay affect seo for robots.txt

**anexsia** · 12-04-2011, 11:46 PM

I'm curious...what would be the benefit of setting a crawl-delay?

**AllAboutCams** · 12-04-2011, 11:49 PM

Originally posted by anexsia

I'm curious...what would be the benefit of setting a crawl-delay?

apparently that is why my hosting keeps crashing

**AllAboutCams** · 12-05-2011, 12:10 AM

google ignores Crawl-delay anyway

**anexsia** · 12-05-2011, 12:21 AM

"You can change the crawl rate (the time used by Googlebot to crawl the site) for sites that are at the root level—for example, www.example.com and http://subdomain.example.com. You can't change the crawl rate for sites that are not at the root level—for example, www.example.com/folder.

Note: Crawl rate refers to the speed of Googlebot's requests during the crawl process. It doesn't have any effect on how often we crawl or how deeply we crawl your URL structure."

**AdultKing** · 12-05-2011, 01:01 AM

Originally posted by xxxupdate

apparently that is why my hosting keeps crashing

If your hosting can't deal with Google crawling then fuck knows how you handle real visitors.

**Black All Through** · 12-05-2011, 02:40 AM

Originally posted by AdultKing

If your hosting can't deal with Google crawling then fuck knows how you handle real visitors.

**AllAboutCams** · 12-05-2011, 04:09 AM

Originally posted by AdultKing

If your hosting can't deal with Google crawling then fuck knows how you handle real visitors.

i have 9 blogs running on Amnoc-r2 with about 1000 hits per day and i keep getting the error can not connect to data base
problem seems to have gone know i went in to web-master tools and set all the crawl rates to the lowest they do
ill monitor the uptime and see what happens
or i could try iinet vps

**AdultKing** · 12-05-2011, 05:19 AM

Originally posted by xxxupdate

i have 9 blogs running on Amnoc-r2 with about 1000 hits per day and i keep getting the error can not connect to data base
problem seems to have gone know i went in to web-master tools and set all the crawl rates to the lowest they do
ill monitor the uptime and see what happens
or i could try iinet vps

Dude. 1000 hits a day is fuck all.

if by iinet vps you mean iiNet in AU, forget it , you can't host porn anywhere inside AU geographically.

edit: it seems to me you have a configuration or other issue with setup.

**BIGTYMER** · 12-05-2011, 05:25 AM

Find a new host.

**AllAboutCams** · 12-05-2011, 06:17 AM

Originally posted by AdultKing

Dude. 1000 hits a day is fuck all.

if by iinet vps you mean iiNet in AU, forget it , you can't host porn anywhere inside AU geographically.

edit: it seems to me you have a configuration or other issue with setup.

they all work fine know ive never had a problem before could have been the rss feed i had syncing it did add 15,000 post on 3 of the blogs that maybe why i was over using mysql

**AdultKing** · 12-05-2011, 04:18 PM

Originally posted by xxxupdate

they all work fine know ive never had a problem before could have been the rss feed i had syncing it did add 15,000 post on 3 of the blogs that maybe why i was over using mysql

With all due respect, you really don't have a clue do you ?

**CyberHustler** · 12-05-2011, 04:24 PM

Tell them to increase your rams...

**AllAboutCams** · 12-05-2011, 04:28 PM

Originally posted by AdultKing

With all due respect, you really don't have a clue do you ?

im telling you what happened how can i not have a clue about that

does Crawl-Delay affect seo for robots.txt

does Crawl-Delay affect seo for robots.txt

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment