Brad,
A rather insightful post.
However, unfortunately the FCP is not the be-all end-all of network performance - it's not a panacea, nor is it even adequate on its own. It is certainly not a replacement for qualified, top-tier network engineers.
As an example for you, here are some sample traceroutes from T1 in Miami (close to NOTA) taken earlier today.
You have higher latency than several other hosts when you are geographically substantially closer. All of these other hosts which are FARTHER away (NYC or Atlanta) and do in fact have rather dramatically lower latency than you.
The routing technology you've employed is not doing anything to improve this route, and the other hosts are already naturally getting better routes, or are tweaking things themselves better than the FCP is.
While I'm sure Internap is pleased you've implemented the FCP, I do hope that you don't feel that this makes your network superior in and of itself - and don't become too reliant on its feature set. Anyone can drop an FCP into their datacenter - or achieve similar or better performance without one.
Also, certainly not all of your peers show up in fixedorbit, no one's does, but the bottom line of it is that other hosts lose about the same percentage as you and have more visible. At the end of the day - the FCP can only work with the routes you give it - sure, it might make 3 "poor" networks less poor - but it can't make them good.
The traceroutes:
2 tpa-core-01.inet.qwest.net (205.171.27.101) 4 msec 4 msec 4 msec
3 dcx-core-02.inet.qwest.net (67.14.1.250) 36 msec * 36 msec
4 dcp-brdr-01.inet.qwest.net (205.171.251.62) 40 msec 40 msec
dcp-brdr-01.inet.qwest.net (205.171.251.98) 40 msec
5 Te-4-3.car3.Washington.Level3.net (4.68.110.177) 40 msec 40 msec 40 msec
6 vlan89.csw3.Washington1.Level3.net (4.68.17.190) 40 msec 44 msec 40 msec
7 ae-81-81.ebr1.Washington1.Level3.net (4.69.134.137) 44 msec 48 msec 40 msec
8 ae-2.ebr3.Atlanta2.Level3.net (4.69.132.85) 64 msec 64 msec
9 ae-78.ebr2.Atlanta2.Level3.net (4.69.134.53) 56 msec 64 msec 56 msec 10 ae-4-4.car2.Miami1.Level3.net (4.69.133.9) 64 msec 64 msec 64 msec
11 ae-24-24.car4.Miami1.Level3.net (4.69.133.26) 64 msec 64 msec 68 msec
12 EASY-ONLINE.car4.Miami1.Level3.net (4.71.208.6) 336 msec 208 msec
256 msec
13
www.mojohost.com (208.122.213.2) 64 msec 64 msec 64 msec
2 tpa-core-01.inet.qwest.net (205.171.27.101) 8 msec 8 msec 8 msec
3 dcx-core-01.inet.qwest.net (67.14.32.2) 40 msec 40 msec 40 msec
4 dcp-brdr-01.inet.qwest.net (205.171.251.58) 36 msec
dcp-brdr-01.inet.qwest.net (205.171.251.94) 40 msec
dcp-brdr-01.inet.qwest.net (205.171.251.58) 40 msec
5 ge-6-9.r04.asbnva01.us.bb.gin.ntt.net (129.250.8.173) 40 msec 44 msec 40 msec
6 ae-1.r21.asbnva01.us.bb.gin.ntt.net (129.250.2.180) 40 msec 40 msec 40 msec
7 as-0.r21.nycmny01.us.bb.gin.ntt.net (129.250.2.8) 56 msec 44 msec
44 msec
8 po-4.r03.nycmny01.us.bb.gin.ntt.net (129.250.2.221) 176 msec 228 msec 328 msec
9 xe-2-2.r03.nycmny01.us.ce.gin.ntt.net (129.250.192.202) 44 msec
44 msec 44 msec
10 76.9.8.53 48 msec 52 msec
64.111.213.153 44 msec
11 host.isprime.com (64.111.215.152) 44 msec 44 msec 44 msec
2 tpa-core-01.inet.qwest.net (205.171.27.101) 8 msec 4 msec 8 msec
3 atl-core-01.inet.qwest.net (67.14.14.2) 20 msec 20 msec 16 msec
4 atl-brdr-04.inet.qwest.net (205.171.21.174) 16 msec 20 msec 20 msec
5 0.so-3-0-0.BR1.ATL4.ALTER.NET (204.255.169.81) 20 msec 20 msec 16 msec
6 0.so-1-1-0.XT2.ATL4.ALTER.NET (152.63.86.174) 16 msec 20 msec 16 msec
7 0.so-6-0-0.XL2.EWR6.ALTER.NET (152.63.17.109) 44 msec 44 msec
0.so-1-0-0.XL1.EWR6.ALTER.NET (152.63.20.69) 44 msec
8 POS7-0.GW2.EWR17.ALTER.NET (152.63.28.49) 44 msec
POS6-0.GW2.EWR17.ALTER.NET (152.63.28.45) 44 msec
POS7-0.GW2.EWR17.ALTER.NET (152.63.28.49) 44 msec
9 voxel-ewr17-gw.customer.alter.net (157.130.8.162) 44 msec 40 msec 40 msec 10 827.ae0.cr1.lga2.us.voxel.net (69.9.184.249) 44 msec
826.ae0.cr1.lga2.us.voxel.net (69.9.185.93) 44 msec
827.ae0.cr1.lga2.us.voxel.net (69.9.184.249) 44 msec
11 5.ge0-1.esr1.c8.lga2.us.voxel.net (69.9.185.99) 44 msec 44 msec
44 msec
12 208.122.14.210 52 msec 44 msec 40 msec
13
www.voxel.net (208.122.6.98) 40 msec 44 msec 40 msec
2 tpa-core-02.inet.qwest.net (205.171.27.105) 8 msec 8 msec 8 msec
3 dcx-core-01.inet.qwest.net (67.14.32.2) 40 msec * 36 msec
4 dcp-brdr-01.inet.qwest.net (205.171.251.94) 36 msec 44 msec 40 msec
5 ge1-4-1000M.ar2.DCA3.gblx.net (64.208.110.29) 40 msec 44 msec 40 msec
6 64.210.29.178 52 msec 52 msec 52 msec
7 national-net.com (66.115.135.38) 56 msec 52 msec 52 msec
Interesting that you mention this - given that the one of the FCPs main selling points is the ability to make routing decisions based on cost (you enter cost per megabit per provider, and what you want your total cost to be) rather than the best route necessarily. E.g. it intentionally facilities not necessarily using the best route and utilize a cheaper "acceptable" route.
Best,
Sam