Using someone's preferred pronouns is a sign of mutual respect, your refusal to do so is a sign of disrespect to those around you. It's really that simple, bud.

Do you call people Johnny when they tell you their name is John? It's literally the same thing, they've explained how they'd like to be addressed, and deviating from that uninvited is just rude.

I get that they matter a lot to some people, and of course it's super annoying (if not worse) to be referred to in the wrong way

It's dehumanizing and disrespectful, it's not annoying. I've had family members refuse to use an individual's pronouns, but in a heartbeat correct themselves for referring to a pet by the wrong pronoun. I've had people go out of their way to call me "man, guy, dude, bruh" when I'm fem presenting, and I'm the only woman they're speaking to that way while I get the "I talk like that with everyone, bruh," excuse.

and if one group of people can try to force a change they prefer, I'm as much in my right to resist it if I don't like it.

Correct, but then you don't get to complain, like you are, that people get upset with you about it. You're not free from the consequences of those around you simply because you have the right to feel differently on something like basic human respect for your fellow people.

I don't get to complain that no one wants to have dinner with me just because they don't like me taking food off their plates, even though I don't agree with that societal norm.