My main issues with vacancy taxes are three-fold:

1. The cities with the worst housing crises are typically the ones with the lowest vacancy rates. This makes sense, as if vacancy rates are super high, potential tenants have a lot of negotiating power against landlords, so they can demand lower rents. When there are very few vacancies relative to the number of prospective tenants, landlords have all the negotiating power and can demand high rents.
2. Vacancy tax focuses on shuffling ownership of existing units and doesn't do anything to encourage densification and development. Own a detached single-family home right next to a metro station in the middle of Manhattan? So long as someone lives in it, you pay no vacancy tax, despite the fact it's clearly a massive waste of some of the most valuable land in the world.
3. It's easier to evade and thornier to implement. For instance, there are a lot of "statistics" thrown about regarding "millions" of vacancies, but many on-paper vacancies aren't what you or I imagine. For example, "vacant" technically includes student apartments where the student lists their parents' address as their permanent address. Getting back to the point, if you can just on-paper claim a unit is occupied, you can evade the tax, which means the government then needs to actually go out and check if someone us actually living there at least 180 days out of the year, which is way harder to enforce.

Altogether, vacancy taxes are a pretty marginal solution, and I think our focus is much better spent on land value taxes and YIMBYism (e.g., zoning reform).