03-20-2010, 06:56 PM
|
|
|
So Fucking Banned
Industry Role:
Join Date: Apr 2001
Location: the beach, SoCal
Posts: 107,089
|
Quote:
|
Your site could be on a virtual host with a lot of other web sites on the same IP address. In theory, you can run into limits on how hard we will crawl your site. If we can only take two pages from a site at any given time, and we are only crawling over a certain period of time, that can then set some sort of upper bound on how many pages we are able to fetch from that host.
|
Who needs dedicated IPs? |
|
|