I think even deeper than that, they just conflate Capitalism with economics. There's a good bit to what you're saying too, though. They see anything outside traditional notions of economics as utopian.

It's a fundamentally different economic system at the principle aspect. For starters, public ownership does not mean production goes straight into the pockets of gov officials, they are paid salaries. Secondly, publicly owned services are usually not for profit, or even at cost, through taxpayer money or otherwise. Finally, Capitalists are a specific type of Capital owner, small handicraftsman, feudal lords, etc aren't Capitalists but do own Capital. Even further, gov officials aren't the owners of publicly owned industry, but indirect administrators. Managers and accountants in businesses aren't the owners.