I'm not sure what you're planning to discover by that. What makes modern finance (particularly stocks) so hard to manage is that it's very emotional. People invest based on what they believe in, and when they get scared they sell. Is the goal to anticipate this and see if the AI can replicate it exactly, or are you expecting it to do it better somehow? If it's the latter, it would be very hard to measure success, because you can't measure how the market would react without involving the market.

I think people try to make prediction models all the time, but if it really worked I feel like it would become quite obvious when someone basically never misses.

If it's just a matter of running a simulation to see how far they diverge, I'm not sure what kind of insights you could gain from that. I think it would be a bit like running a weather simulation for years. Very soon after (likely a matter of days), unpredictable events would fork reality from the simulation, and they would only diverge further.

To answer the question, it might be possible to set up an imperfect, incomplete simulation.