VPS is typically a dedicated server running some kind of virtualization e.g. XEN, OpenVZ, Vmware so that you're sharing resources (HDD, CPU, RAM, connection) with others. Some providers do oversellling in which case one dedicated server runs hundreds of virtual servers (VPS), but if you're lucky you can get a decent VPS which is not overloaded.
For static content, page loading time usually depends on connection latency of your server (assuming connection / hdds are not overloaded). Since all users will not have the same routing to your server, it may seem slow for one person but fast for another. You can check latency of your server from various locations here:
http://lg.he.net/ (select ping, lower ping = better).
It's hard to say what kind of hardware you need without knowing what kind of software you're using (apache, nginx, mysql, some 3rd party soft maybe?) or how many clients you have.
Any professional hosting should be good for hosting adult content if you choose a dedicated server option (just check TOS to be sure). I can recommend OVH, leaseweb.