Hey, we're using directories to list our sample images for each site we list, usually around 8 sample images per site, but some upwards of 16.
http://www.pornsumer.com/sites/8th-s...pics/picture4/
That's an example.
So we have like picture1, 2, 3, to 8 usually per site. We expect to list thousands of sites using this system. Is this too many URL's for google?
My feeling is that we should add them to the robots.txt, but I'm not sure the syntax we'd use considering the dynamic directory structure.
Perhaps we should leave them out of the robots for some other reason? We could still leave this page in. This is an extreme example with 19 pics = 19 directories.
http://www.pornsumer.com/sites/8th-s...nas/free-pics/
Thoughts? Comments? Syntax?
