Funnily enough, empathy doesn't always apply to all other people! Many often learn to only utilize their empathy to gain social advantages by leveraging an understanding of others, or practice selective empathy in that they have functioning, typical, empathy - just only within their bubble. It's why some of them get uncomfortable when they're forced to perceive poor people in a more humanizing way - they have empathy, they're just choosing not utilize it in a way that benefits larger society. I feel I should also point out that a number of people fit the qualifications for being considered low-empathy, but are regarded as leading admirable lives by otherwise choosing to be good to their fellow human beings - James H Fallon comes to mind.