I can imagine a few scenarios in which a flower would be rightfully downvoted:

1. Blurry/out of focus/too much jpeg. You know, generally potato quality.

2. Sob/stupid story title.

3. Obvious stock art.

4. Photo of cherry blossoms on a community about fungal fruit bodies.

5. My father died from anaphylaxis due to severe pollen allergy.

Kbin gives me the ability to see up/downvotes, and this is usually what I've used it for in the rare cases I've cared to check. To satisfy your curiosity, it's usually accounts who do absolutely fuck all except downvote seemingly at random. Never comment. Sometimes they've never even upvoted anything.

Twice only has it been someone I recognized and knew to be generally a normal human user. One was and still is a fan of melodrama, so they were probably having One Of Those Days. The other, I'm still a little confused because I knew them to be of above average intelligence, but I think I have to chalk that up to fat thumbs.

Sorry if I'm ruining the fun. Can confirm the only haters your hypothetical flower has are absolute weirdos, though, and questionably human.

Severe allergy perhaps. Or maybe the flower is also a popular name, such as Rose or Jasmine, and they hate absolutely hate someone named like that