I wonder about many of the same things, to be honest here. This is why I am an anonymous cartoon character instead of, say, Robbie, who flaunts his work and lifestyle and is proud to be what he is: a successful pornographer (and more). I do admire that about Robbie (and others), how they don't seem to give a fuck what anyone else thinks about them.
THIS is true "freedom", BTW: Not giving a fuck what other people think of you, or what you do, how you dress, look, etc.
Anyway, if you're thinking this way then you should probably just fade out as gracefully as you can.

For me, because of my network and "success", it's a little harder to just walk away with no consequences. I'd be giving up a lotta moolah if I did so, with nothing to replace it (at the moment). But if you're not financially tied to Adult and are feeling this way then you should split, or at least take a long break and see how you feel about things a few months from now.
Cheers and best of luck either way!