I guess you only met cat people who never air out their house and/or don't clean the litter boxes frequently enough. Also cats don't smell nearly as much as dogs, they literally wash themselves constantly. Which does make them smell like their (terrible) breath for a few minutes but it doesn't stay.

And no, I'm not just used to the smell, I've lived with and without cats for long periods of time, and have experienced the terrible smells at some people's houses (and never at some others, even though they had several cats)