Actually ~~not~~ it's complicated.
Fascinating as it may sound, transvestites were mostly allowed in the third reich, when they weren't homosexual.

The even got their own pass:
https://en.m.wikipedia.org/wiki/Transvestite_pass

After reading the German and English version I am a little puzzled, since the German version says, that they were kept and transvestites stayed unharmed when not homosexual while the English version says:

After the Nazis came to power, most passes were revoked or German police refused to recognize them.

The German original source says the following:

Sofern Trans*personen den „gegen sie erhobenen Homosexualitätsverdacht entkräften konnten, lässt sich in keinem Fall eine Strafverfolgung nachweisen. "

Which translates to:

If trans*persons "could invalidate the suspicion of homosexuality, there is no evidence of criminal prosecution."