tobedeleted

One of the sites I manage does not appear in Bing. It's not blacklisted, it's 'filtered'.

If I type the URL in Bing it does not appear. If I change my setting to allow adult content it will appear. So at that point I thought it would be nice to bypass their default settings, but I can see why they are there and I don't want to deceive any users, so perhaps it is best to leave it 'as is'.

However if someone is searching for adult content and the search engine default restricts the return results, then I can't trust that the average user will know how to remove the filter.

So I checked other adult sites with SafeSearch back on (default is moderate). And those sites did appear. Like pornhub, xvideos, etc.

I thought, WTF? I even checked similar competition and they appeared.

So next I started seeking out an official 'banned' word list as there must be something in the titles, meta-tags, descriptions or even posts that have triggered the block. But so far I have had no luck.

Does anyone know how sites like pornhub can pass Bing's SafeSearch? Is there a robots.txt trick?

I checked their file and found references to:

User-agent: Bingbot
Disallow:

User-agent: googlebot
Allow: /

But even listed in the search queries were profane words, so I don't know how some of these sites are not blocked by SafeSearch.

Any insight would be greatly appreciated.

Thanks