OneNote does not have export to plaintext files. However I found this cool little program to export all my notes to markdown in one go.

But, turns out that OneNote only lets you export to .docx if you have Word installed too, so you need to have that (the program then converts the .docx to .md with Pandoc).

Anyway, just sharing it here in case someone finds it useful to get away from vendor-locked notes. There are a few different projects on GitHub that do this, but this one was the one that worked best for me.