In some parts of london I've seen black guys stealing ppls cell phones on the open street who didn't neccecary look like thugs or tried to pull you in a side street and steal the shit out of you if you see that happening a lot you might get scared
And in the US there are so many black people the only reason might be the mass media,I don't think you see gang stuff and robberies in every small city.the prisons are full of blacks because many can't afford good lawyers that might add something to the fear too.And of cause gangsta rap
If you life in LA or New York maybe ppl had bad experience with blacks robbing them on the street or whatever,I have no clue never been in the US,in germany I haven't seen anybody beeing scared of blacks cause you don't hear anything about blacks robbing the shit out of you,in some big cities like hamburg nigerians and other africans are the big groups of drug dealers but nobody is afraid of them cause they don't rob or kill you.
Long story short sense the mass media makes the biggest image after that personal experience and in rich ppl areas in the US the ppl just might not like non white skin color