And you absolutely could not avoid a fight and just walk away from the situation with plausible deniability because you “only insulted them”.

If you offend someone so hard that there's a possibility of physical harm and/or death from it, and that person survives that insult, I'm quite sure most people wouldn't just let that slide.