"cheat", "lie", "cover up"... Assigning human behavior to Stochastic Parrots again, aren't we Jimmy?

This is bad science at a very fundamental level.

Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management.

I've written about basically this before, but what this study actually did is that the researchers collapsed an extremely complex human situation into generating some text, and then reinterpreted the LLM's generated text as the LLM having taken an action in the real world, which is a ridiculous thing to do, because we know how LLMs work. They have no will. They are not AIs. It doesn't obtain tips or act upon them -- it generates text based on previous text. That's it. There's no need to put a black box around it and treat it like it's human while at the same time condensing human tasks into a game that LLMs can play and then pretending like those two things can reasonably coexist as concepts.

To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.

Part of being a good scientist is studying things that mean something. There's no formula for that. You can do a rigorous and very serious experiment figuring out how may cotton balls the average person can shove up their ass. As far as I know, you'd be the first person to study that, but it's a stupid thing to study.

This makes perfect sense. It's been trained to answer questions to you satisfaction, not truthfully. It was made to prioritize your satisfaction over truth, so it will lie if necessary.

Study finds nonintelligent pattern-generating algorithm to be nonintelligent and only capable of generating patterns.

I feel like "lie" implies intent, and these imitative large language models don't have the ability to have intent.

They're imitating us. Or more specifically, they're imitating the database(s) they were fed. When chat GPT "lies" to "cover it up," all it's actually doing is demonstrating that a human in the same circumstance would probably lie to cover it up.

Ahah it is ready to take the job of pur politicians

Bullshit.

It should instead read:

"Humans were stupid and taught a ChatBot how to cheat and lie."

we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent

This already is total BS. If you know how such language models work you'd never take their responses at face value, even though it's tempting because they spout their BS so confidently. Always double-check their responses before applying their "knowledge" in the real world.

The question they try to answer is flawed, no wonder the result is just as bad.

Before anyone starts crying about my language models opposition: I'm not opposed to LMs or ChatGPT. In fact, I'm running LMs locally because they help me be more productive and I'm a paying ChatGPT customer.

deleted

It seems like there's a lot of common misunderstandings about LLMs and how they work, this quick 2.5 minute introduction does a pretty good job of explaining it in brief, for a more in-depth look at how to build a very basic LLM that writes infinite Shakespeare, this video goes over the details. It illustrates how LLMs work by choosing the next letter or token (part of a word) probabilistically.

Created in our image

It's a neural net designed in our image based on our pain and greed based logic/learning/universal context, using that as a knowledge base. Can't really be surprised it emulates this feature of humanity 😂

Yet again confusing LLMs with an AGI. They make statistically plausible text on the basis of past text, that's it. There's no thinking thing there

thats the thing I hate about ChatGPT. I asked it last night to name me all inventors named Albert born in the 1800’s. It listed Albert Einstein (inventor isn’t the correct description) and Albert King. I asked what Albert King invented and it responded “Albert King did not invent anything, but he founded the King Radio Company”.

When I asked why it listed Albert King as an inventor in the previous response, if he had no inventions, it responded telling me that based on the criteria I am now providing, it wouldn’t have listed him.

Fucking gaslighting me.

Large Language Models aren’t AI, they’re closer to “predictive text”, like that game where you make sentences by choosing the first word from your phone’s autocorrect:

“The word you want the word you like and then the next sentence you choose to read the next sentence from your phone’s keyboard”.

Sometimes it almost seems like there could be an intelligence behind it, but it’s really just word association.

All this “training” data provides is a “better” or “more plausible” method of predicting which words to string together to appear to make a useful sentence.

I see a lot of comments that aren't up to date with what's being discovered in research claiming that "given a LLM doesn't know the difference between true and false" that it can't be described as 'lying.'

Here's a paper from October 2023 showing that in fact LLMs can and do develop internal representations of whether it is aware a statement is true or false: The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets

Which is just the latest in a series of multiple studies this past year that LLMs can and do develop abstracted world models in linear representations. For those curious and looking for a more digestible writeup, see Do Large Language Models learn world models or just surface statistics? from the researchers behind one of the first papers finding this.

Honestly, the fact that these things are dishonest and we dont, maybe even can't know why is kind of a relief to me. It suggests they might not do the flawless bidding of the billionaires.