Without knowing the specific coding environment at play here, I don't think this is a server issue. I think we're just talking about different implementations of technology.
A pre-recorded video, or a split HLS stream, with the proper moov atom will tell the browser to download the chunk necessary to start the video now. Nothing is being processed on the server side, so we're down to just how fast the connection is between user and server. The browser does all the heavy lifting interpreting the video, showing it, and asking for the next chunk.
Depending on a livestream, you might need to connect to a socket and perform a handshake before the video loads. There could be checks that need to be performed based on session data. All of that takes more time, because you could be interacting with more than just the server. They might need to check a database, or poll for live status, etc. 25-40 seconds seems like an excessive amount of time, so there might be a server issue in conjunction with this, but ultimately the difference stems from the differences in technology.
I think if you hosted with either of the top two adult hosts, you would be fine hardware wise. Both
Vacares and
MojoHost are perfectly capable of serving assets quickly.
I built a library that I now sell as a SaaS for a client that integrated Streamate/Cambuilder, NATS, and payment processors. For the cam part of this equation, I used their API to grab a list of live performers. The only assets that were served were static images, but the round-trip time to the API took longer than simply loading a predefined asset. Speaking of, I also needed to connect to the APIs of NATS and the payment processor, and those took time too. You might be seeing something similar here. Nevertheless, it was way faster than 25 seconds (we're talking differences between milliseconds here).
Ultimately I think the answer to your question depends on the tech stack, and whether it has been optimized. If there is no limitation on their end, then a properly configured and optimized server and coding stack on your end should not present any noticeable delay.