The best way to test a 'new' niche is to do what we did with
http://www.dilf.com.
Send more general traffic to the site first then, once you have some figures from that traffic take complimentary niche traffic, in our case, mature gay traffic and send that to the site.
Then, once you have tested those types of traffic, send traffic to the site from an unrelated niche so again for Dilf we sent twink traffic to the site also.
This should give you a good idea of what type of traffic converts on the new niche as well as to how 'in demand' the niche is not only to the surfers but also to the webmasters who will be promoting the site.
Of course, you need to send in excess of 100k-500k of all types of traffic or more to get a realistic figure which will enable you to work with.
Another idea would be to build a semi-free site and get that in the SE's and see how well the targeted keywords do for you also.