The key number here is the dependency ratio. So the number of retired and underage people to working age. The intressting part is that there are fewer younger people and the old ones tend to die. The other part is built up infrastructure. If you are a country with a falling population, you do not need to built a lot of housing, as there is already enough around. Public transport infrastructure just has to be maintained and things like the electricity grid work as well. Even many factories do not need massive upgrades all the time.

So for the most part it tends to work rather well, unless you have a really bad ratio. Obviously innovation helps as well.