You aren't complicit because you're a man, not to belittle your feelings though. I get uncomfortable around generalised terms in regards to gender issues, because I was always told never to do that, but it's something I similarly have to deal with.

Sometimes the conversation doesn't require your voice, and that's okay. Ideal even. If there are people with more skin in the game, they're more than welcome to have their say in my eyes.