As far as I know they're mostly harmless, but they will emit an unpleasant smell if disturbed, hence the name. Vacuuming them up works reasonably well.

I dislike them because they put holes in my peaches before they ripen.