Yes, thinking involves signals firing in your brain. But, not just any signals. Fire the wrong signals and someone's having a seizure not thinking.

Just because LLMs generate words doesn't mean they're thinking. Thinking involves reasoning and considering something. It involves processing information, storing memories, then bringing them up later as appropriate. We know LLMs aren't doing that because we know what they are doing, and what they're doing is simply generating the next word based on previous words.