I notice a lot of vague/personal language being used. "Men", which men? "Society", which one? "We", who is we? So I can only give vague responses.
I rarely use my gender to identify myself, but when asked I say I am a man. Proudly.
I understand why some men/boys would feel shame, my own shame came from my failure to treat others fairly/respectfully.
I don't experience that shame anymore now, back then at that stage in my life I was guided by men who had my best interest in mind, but cared little about the values I deemed important.
And now I believe that if you behave in a way that is in accord of your own values, you should have no shame.