Nobody complains about big cars whilst they're towing, and if they were doing it everyday you would see them.. well.. everyday towing, but they typically are not.

Fuel efficiency that you lose whilst towing you would gain on the other 99% of your kms.

I live in a country where everyone buys used cars from western Europe and it's semi-common knowledge among car people that you should avoid Dutch cars with tow hitches (and the used car yards that bring their cars from Holland tend to have the worst reputation).

That said, if you only tow heavy loads maybe a thousand, tow thousand kilometers a year, it doesn't really matter. It's prolonged heavy towing that kills the small car.

Anyway, my midsize diesel car can tow way more than I personally am legally allowed to and I prefer throwing a tent in the trunk to towing a camper, so my car sees maybe <500 km of light-weight towing a year and under a metric ton you can barely feel the hit to fuel efficiency or performance (because diesel torque is ridiculous)

(for instance, earth moving equipment)

That's quite some reach.

Only very few people regularly pull caravans. I worked for one, but he a) sold caravans and b) had the car to pull them.