Considering that M'Benga is demoted, it's likely he did something that is consider attrocious, but not to drum out of Starfleet entirely.

Considering the Cliffhanger, what if M'Benga developed some kind of permannent genetic engineering that changed Gorn from reproducing with Paralystic Eggs being pregnant with their own paralystic eggs?

The reasoning I say is this: with Paralystic Eggs, they are consider direct threat to well being of the various Species within Federation due to nature. However, in Kelvinverse Gorn can get pregnant; and by the time Lower Decks come about Gorn is "friendly enough" for them to do business within Federation, something may have changed - voluntary or not voluntary. I will say not voluntary.

The technology exist: Klingon's smooth forehead is a combination of genetic manipulation with virus, leading to the spread of the geneitic engineering beyong those in the experiment.

Now, if M'Benga did it, it actually put Starfleet in a dilemma:

• On one hand, M'Benga commited permanent genetic engineering without authorization, as well as cultural/genetic genocide. Grave crime in Federation
• On the other hand, it's Gorn, who attacked Federation space and colonies.

Thus, the demotion - if it's just the first part, he will be dishonorably discharged. But since under the table he effectively saved Federation, they decided to just demote him if he choose to stay on Enterprise, or assign to some random outpost. He chooses the former.