My experience as a dude growing up in a rural region has been harsher.

There's this subconscious dick-measuring contest between males, especially among teenagers, which means if there's any suspicion of you doing something different in a way that's 'better' health-wise or moral-wise, the more insecure folks will feel personally attacked and become aggressive. Especially so, if they're drunk.

And particularly in rural regions, you're much more limited in the social circle you get to choose. The twats who have a problem with you not drinking, they're inevitably your peers.