Thread: Y'all ever had problems with google crawl rate?
View Single Post
Old 11-26-2019, 10:56 PM  
wankawonk
Confirmed User
 
Industry Role:
Join Date: Aug 2015
Posts: 1,018
Quote:
Originally Posted by rowan View Post
A 429 Too Many Requests error code could help, although you would need to be careful not to overdo it.

According to this page, Google sees 429 as being the same as 503 Service Unavailable.

https://www.seroundtable.com/google-...ode-20410.html
Perhaps I will try this. Perhaps it will work better than what I have been doing. Thanks.

But shouldn't I just be able to easily tell them, "don't hit my site more than 100k times per day" and they should respect that?
wankawonk is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote