|
We believe we have solved the bug with adding sites.
Please feel free to add your sites now and please reply back to the thread if there are any more bugs encountered.
Do not expect your entire site to appear in the search results immediately, the web crawler uses a number of complex algorithms to determine how it crawls
To see if the crawler has been to your site look for these user agents in your web server logs
"User-Agent: Mozilla/5.0 (compatible; PornoBug/1.2; +http://www.pornobug.com)"
or
"User-Agent: Mozilla/5.0 (compatible; PornSpider/1.0; +http://www.pornspider.net)"
Once your site is crawled it takes around 5 hours for results to start appearing in the index.
As with any real search platform, the databases are huge as is the traffic, we also try to limit our crawler from overloading your site.
Please let me know if you have any questions.
|