Quote:
Originally Posted by SmokeyTheBear
lol well i hate to do this but....
Have you given consent for google to scrape you ? yahoo ? bing ? etc.
|
Yes, he's given Google permission to index, not scrape, the site, and thereby promote it.
If you've been a webmaster for more than a few days, you know about robots.txt.
By choosing not to put up a "no indexing" sign (robots.txt), you've given implied
permission for Google to promote you by adding you to their index. I'd bet the people
scraping (not indexing) the site don't check for robots.txt.
Besides, use a ounce or so of common sense. Obviously webmasters want their
porno sites listed in search engines. Duh.
__________________
For historical display only. This information is not current:
support@bettercgi.com ICQ 7208627
Strongbox - The next generation in site security
Throttlebox - The next generation in bandwidth control
Clonebox - Backup and disaster recovery on steroids