Why don't you go ask the Nazi sympathisers we executed for treason after the war about that?

If a state permits its citizens to betray the country in favour of an adversary in the event of a war, it's incapable of protecting itself. The most important task of any society is to keep its members safe. A crucial aspect of that is accepting the social contract that everyone on the society will help keep each other safe, even in the event that an outside adversary invades and threatens to kill you. If you break that social contract- guess what? The rest of society will typically (at least historically) brand you as a traitor and imprison or execute you. Why? Because you've shown that you're willing to put their head on the block for your own benefit, so they see you as a threat (perhaps the worst thinkable threat) to the security their society provides, and decide to remove that threat to protect themselves.

No matter what oath you have or haven't taken, societies obligation to keep you safe only extends as far as your willingness to protect the society. This is why treason, in most societies, is seen as one of the worst, if not the worst, crime you can commit. It's literally stabbing strangers that are willing to die for you and your family in the back.