You can add your sites and servers to
Web-Tech-Center.com
There you'll be able to monitor your site(s) download speed from Europe and the US, plus have an close eye on your Servers CPU, memory and bandwidth usage, plus Server Load.
And in case anything goes wrong you can also add your email or Mobile Phone Number and you'll receive an Alert Notification, in case you're on vacation etc.
Actually we're in early Beta, so it's completely free, we only ask that you let us know about any bugs you come across
BTW, and you should do a Traceroute at
http://www.traceroute.org for your site and post it here, maybe it's not your server but it's connection that slows something down.
Or could also be that you're running a buggy Client based script on your site (i.e. JavaScript, Flash) that slows everything down when your page is already fully loaded.