Isn't there also the issue of "self fulfilling prophecy" to consider? If someone is constantly being labelled as stupid, lazy, bad, deceitful, etc., etc., aren't they more likely to internalise that characterisation and behave accordingly? Also, doesn't such labelling of groups of people generate hostility and lead them to turn in on themselves and have little regard for those that are not part of their group?
I realise it is natural for humans to assess risk based on their experience, and I understand why, if a person has had one or more bad experiences of a particular group, they tend to expect the worst of that group. But I also think that if you do expect the worst of people, that is what you are more likely to get - and vice versa.