Something people don't realize is it's quite easy to photoshop something else onto that sign. Something you don't want your face associated with. Then publish that with the name they got from reverse image search and some text saying they photoshopped the pictures on their social media because they got backlash.

If you wear a mask you interrupt this cycle. You can drop your pictures from social media and deny it was ever you.