Yeah, honestly its worse at "how many characters are in this word" kinds of questions than just about anything else. Asking what time it is in other places sometimes yields similar nonsense

And, at the same time, sometimes its able to find things that exist somewhere on the web that I simply cannot find because search engines are a broken disaster. I was looking for short, wide, aquarium stem plants like starugyne repens, but that would be better for a low tech aquarium- low tech plant alternatives to high tech ones feels like a thing that 100% exists somewhere in discussion on the web. Search engines found nothing. ChatGPT found me hygrophila corymbosa 'compact'. Which is exactly the kind of thing I was looking for, but that search engines were just incapable of finding. And honestly thats largely an indictment of search engines. It wouldn't surprise me if google in part was making a cost benefit analysis on allowing search to be broken since it probably makes people more dependent on ai results.

But thats an easily verifyable thing, where the information exists out there somewhere, and neccisarily I need to look it up to use the information in anyway. I'm more "looking for something" that I know probably exists, than trying to learn something or get information in a conventional sense.

I find its more successful when its organizing existing information around a question that isnt the exact same blog post or article that every relevant source has written because it has optimal SEO and appeals to the widest number of people within that niche (eg: "low tech stem plants" except I'm looking for one with an atypical growth pattern so none of those articles help and google just shows me a million useless pages because theyre almost what I want, and there are millions of them)

On the other hand I went looking for an old quote from a book about antisemitism and it seemed to just keep making shit up and saying it was from other translations of the book, even as I gave it more and more of the actual quote when I found it without ai. Which was quite a while back now, so maybe I need to try that again.

But the more functional it gets the more risk it carries, and the more it can worm its way into people's lives and reshape social contracts that society has always been built on, while doing lots of environmental harm :/