Quote:
Originally Posted by dazzling
I dont understand the tech behind all this but am I correct in guessing that ixquick.com is using a similar structure?
|
Yes, basically what DuckDuckGo does is collects structural information which it uses to order search results, so it grabs information from places like wikipedia, bing, etc and reprocesses them to display results. Duck Duck Go doesn't actually independently crawl the entire web like Bing and Google do. It is dependent on other search engines to do that and store the huge amounts of data. Apparently he uses less than 1TB of disk for the entire platform.