The problem is the unearned confidence in the answer they know nothing about

that is a different problem, which i agree is significant.

there is no reason to use AI to answer the question I asked you

if you ask me "what is your view on X" than I will not tell you what AI said. If you ask "Is there something we haven't considered about X", then I could answer something like "according to AI X has property Y, which afaik we have not considered ".