Merely not owning a car really only saves you money and gives you exercise, yeah.

The much, much bigger difference is in urban design: living somewhere where you don't need a car vs living somewhere where you do.

I've lived in a few places where you don't need a car, with populations ranging from 2.5k to 4 million. I currently live on a small hobby farm just outside the edge of suburbia, and grew up in suburbia.

Living somewhere where you don't need a car is great. Being able to walk to a dozen restaurants in 5 min or to the grocery store in 5 min is amazing. Owning a car in a city is a pain - needing to find parking, having to move for street cleaning, etc. There's little advantage unless you need if for your commute.

Living somewhere car centric honestly kinda sucks. It takes longer to drive to anything in suburbia than it does to walk to it in a small walkable town. That 5 minute walk to the store? How about a 10 minute drive instead? Walking 5 minute to a restaurant? Why not drive 15 instead? Walking to a bar? No, get a designated driver or Uber instead. It also leads to really ugly scenery - giant parking craters are literally no one's idea of beauty. There's a reason people will willingly walk a few miles down a small town's mainstream, but will literally drive to the other side of a strip mall rather than walk it. And driving in bumper to bumper traffic is soul sucking. Suburbanites will literally drive to a gym to get on a treadmill. I think that says something.

I'm guessing this is assuming you live somewhere that is built with the infrastructure to support people without cars?

It would be way more expensive for my family not to own a car, unfortunately.