I checked out
http://tailoredservers.com/ and wasn't able to come up with pricing anywhere near what you are posting here. Are you sure you aren't mixing up the monthly price with the setup fee?
Like on the machine you listed :
Dual Xeon 2.4GHz 1MB Cache, 2GB ECC DDR, 2 * 36GB 10K DRPM SCSI HDD, RAID1 - $99/mo. + $249 setup
The price I come up with is actually $359/mo and $199 setup.
If you are looking to get into this for the long haul, and want reliability, then don't even think of going with "cheap". Cheap ends up being just that in the end. Garbage, and a nightmare hosting experience.
If you want to consider something truly reliable, feel free to hit me up. But the price will be more then what you are listing above.
Do you own a cheap car, cheap house, cheap cloths? No? Why not? Think about it.