Forget about what I said about hooking the white and black up to both ceiling blacks. I thought I had that working before, but I guess not.
Now I just tried every combination possible. The only one that "works" is black light to right ceiling black and white light to right ceiling white. However, it stays on all the time even if the switch is OFF.
There has to be something wrong between this and the switch. There is only one switch, yet four of those wires. Why?
In the same box as the switch there is a socket. On that there appears to be an extra line connected to the bottom two screws. Two wires on each screw (plus one wire in each of the top screws). I know that's not right.