It's an interesting little gotcha, but I wonder if it wouldn't be preferable to just implement a type trait like is_expensive_conversion that is false by default and true for specific cases, and just static_assert this stuff.
Relying on obscure implicit type conversion mechanics for this feature feels like being too clever for your own good.