Hello Cyberdudex,
The IP should not make any significant difference in the speed of your site as 100's of domains can operate on the same IP. The resource demands that you and the other clients load the server with, as well as the hardware specifications of the server, are going have a greater impact on what the server can handle.
In some instances it might not be the server! If everything has been working fine and all of a sudden you start receiving complaints you might inspect the trace routes that your customers can supply you, this can be an amazing source of information.
If your host does move you to a different server they should be able to make it seamless for you and invisible to your clients. If you would like some additional assistance hit me up on Monday when I get in to the office and I can get a System Administrator on the line to help with the technical side of the move.
In the meantime, goodluck!
