just because the (corporate) internet works this way now, doesn’t mean it should

The web worked this way before there was a large corporate presence. Scraping was common during the blogosphere period and robots.txt was the solution everyone at the time agreed on and that's been the standard ever since.

I’m happy that you’re comfortable with this model, but I don’t want people who operate like this to intrude on the spaces we’re building to get away from it

We're not intruding on this space. We've been in the fediverse for just as long or longer; the fediverse has been scrapable since 2008.