GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Does Google bot follow directory listings?

WiredGuy · 10-05-2003, 04:07 PM

If google can crawl your site and find a list of urls to spider, then yes. Google might not index it though if there's a problem with the page like a 404 or 401.

WG

10-05-2003, 04:07 PM
WiredGuy Pounding Googlebot Industry Role: Join Date: Aug 2002 Location: Canada Posts: 34,491	If google can crawl your site and find a list of urls to spider, then yes. Google might not index it though if there's a problem with the page like a 404 or 401. WG __________________ I play with Google.