Honestly, they should put the downvote next to the boost and renamed to something more meaningful if it's not the opposite of an upvote. Everyone is bound to eventually get some amount of upvotes/downvotes, but boosts are not something given as easily (some people might not wanna use boosts at all!).

Then if something opposite to the upvote is wanted, add a different method. So we'll have upvote/downvote with no (or little) effect on reputation, and then something like boost/penalize.

The way it is now it kind of discourages participation in any popular topic if your opinion can be polarizing (even if it's totally respectful and valid) or if there's a significan minority of people who dislike what you said (even if most people upvoted it). Your reputation will be penalized, even when you are being polite, citing valid sources and your vote balance is still positive.