Yeah, generalizations and assumptions really suck when they're inappropriately and unfairly applied. I think they're an evolutionary holdover - quick decision making is essential to animals surviving in the wild, so we evolved to lump things together to facilitate that and ensure our own safety. It's often not such a good thing when it comes to anything more complex than the question of "is that other being likely to harm me," however.