Having worked in computer graphics myself, it is spot on that this shit is uncontrollable.

I think the reason is fundamental - if you could control it more you would put it too far from any of the training samples.

That being said video enhancements along the lines of applying this as a filter to 3d rendered CGI or another video, that could (to some extent) work. I think the perception of realism will fade as it gets more familiar - it is pretty bad at lighting, but in a new way.