Nah, there is definitely a truth to this. I grew up in a working class family who moved into a wealthier region at some point and I would never trade places with people who grew up wealthy. Pretty much all wealthy people are constantly unhappy, are obsessed with control to the point where they alienate their families, they are constantly scared of losing their control, status and wealth, constantly paranoid towards everything and everyone and often engage in self-destructive behaviour.
Of course, not having enough money sucks, it generates stress and restricts autonomy. But a similar thing happens at a certain level of wealth.
Or just oligarch or power addict. In the eyes of most people, wealth is about luxury, material goods and fancy toys. Of course that's part of it, but at a certain point, wealth is no longer about luxury and toys, it's about power and having control over resources everyone else depends on.