The optics in this case are bad enough that the downsides of sending a candidate chosen in a gender-neutral fashion outweigh the upsides, but I'd definitely advise being cautious about assuming that's always the case. If anything it's the exception, not the rule.

That's fair, but also beside the point. This isn't a conference on equality (which should have a fair number of male representatives). It's a conference on female empowerment. Yes most conferences of this nature shouldn't be female only, but this is a pretty obvious exception imo. So obvious, it's not really worth bringing up the alternative scenarios.