07-22-2008, 04:29 AM
|
|
|
Confirmed User
Join Date: Jun 2006
Posts: 974
|
Quote:
Originally Posted by SmokeyTheBear
exactly.
basically if it finds 200 new pages it records the urls , then schedules the order they will be spidered by length.
|
Anyone tested this?
|
|
|