It sounds like a bad monitor to me. I have one that was acting similarly screwy. The resolution was set at 1024x768 for years but the thing started acting whacky with horizontal lines and 'squishing'. So far, I'm still using the same monitor but have it set to 800x600 (then everything is normal).
Until you can switch monitors to check that or take it back, just keep playing with your settings, light some candles, hum a chant and good luck.
