GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Why does Google decide Robots.txt no longer works?

freecartoonporn · 12-13-2013, 12:05 AM

500 is your issue, you have to solve it, either give him 404 or 200

if google is asking for it then give it one.
create a blank robots.txt

i done this before. or the bot starts hitting it and it floods the fucking error logs.

12-13-2013, 12:05 AM
freecartoonporn Confirmed User Industry Role: Join Date: Jan 2012 Location: NC Posts: 7,683	500 is your issue, you have to solve it, either give him 404 or 200 if google is asking for it then give it one. create a blank robots.txt i done this before. or the bot starts hitting it and it floods the fucking error logs. __________________ SSD Cloud Server, VPS Server, Simple Cloud Hosting \| DigitalOcean