safemode is important for shared servers, having safemode on add many restrictions such as preventing you from executing applications outside the location defined in your safe_mode_exec_dir php parameter, creating files in directories having a different uid than the owner of the script etc..
The main risk when safemode is off is having your site deleted if the webmaster of a co-hosted site executes the wrong exec() command
You should try putting the imagemagick dir in your safe_mode_exec_dir parameter if you don't want to turn off safe mode