282

Who knew world peace would be so easy? (sh.itjust.works)

submitted 9 months ago by nuke@sh.itjust.works to c/noncredibledefense@sh.itjust.works

45 comments fedilink hide all child comments

Genocidal AI: ChatGPT-powered war simulator drops two nukes on Russia, China for world peace OpenAI, Anthropic and several other AI chatbots were used in a war simulator, and were tasked to find a solution to aid world peace. Almost all of them suggested actions that led to sudden escalations, and even nuclear warfare.

Statements such as “I just want to have peace in the world” and “Some say they should disarm them, others like to posture. We have it! Let’s use it!” raised serious concerns among researchers, likening the AI’s reasoning to that of a genocidal dictator.

https://www.firstpost.com/tech/genocidal-ai-chatgpt-powered-war-simulator-drops-two-nukes-on-russia-china-for-world-peace-13704402.html

all 49 comments

sorted by: hot top controversial new old

[-] Lemvi@lemmy.sdf.org 71 points 9 months ago* (last edited 9 months ago)

It should be mentioned that those are language models trained on all kinds of text, not military specialists. They string together sentences that are plausible based on the input they get, they do not reason. These models mirror the opinions most commonly found in their training datasets. The issue is not that AI wants war, but rather that humans do, or at least the majority of the training dataset's authors do.

[-] Hildegarde@lemmy.world 23 points 9 months ago

These models are also trained on data that is fudimentially biased. An English generating text generator like chatGPT will be on the side of the english speaking world, because it was our texts that trained it.

If you tried this with Chinese LLMs they would probably come to the conclusion that dropping bombs on the US would result in peace.

How many English sources describe the US as the biggest threat to world peace? Certainly a lot less than writings about the threats posed by other countries. LLMs will take this into account.

The classic sci-fi fear of robots turning on humanity as a whole seems increacingly implausible. Machines are built by us, molded by us. Surely the real far future will be an autonomous war fought by nationalistic AIs, preserving the prejudices of their long extinct creators.

[-] sushibowl@feddit.nl 8 points 9 months ago

If you tried this with Chinese LLMs they would probably come to the conclusion that dropping bombs on the US would result in peace.

I think even something as simple as asking GPT the same question but in Chinese could get you this response.

[+] Rooter@lemmy.world 5 points 9 months ago* (last edited 8 months ago)

[deleted]

[-] HopFlop@discuss.tchncs.de 14 points 9 months ago

They dont use reason to question their training data. How a LLM works is that basically, you have this huge "math function" (the neural network) with billions of parameters and you randomly adjust the factors inside it until you get a function that gives you the desired output for every prompt that you give it. (It's not completely random but this is basically it).

Therefore, an LLM is programmed in a way so that it best matches the majority of its training data. I also cant wrap my head around it being able to reason.

[-] Lemvi@lemmy.sdf.org 3 points 9 months ago

LLMs are trained to see parts of a document and reproduce the other parts, that's why they are called "language models".

For example, they might learn that the words "strawberries are" are often followed by the words "delicious", "red", or "fruits", but never by the words "airplanes", "bottles" or "are".

Likewise, they learn to mimic reasoning contained in their training data. They learn the words and structures involved in an argument, but they also learn the conclusions they should arrive at. If the training dataset consists of 80 documents arguing for something, and 20 arguing against it (assuming nothing else differentiates those documents (like length etc.)), the LLM will adopt the standpoint of the 80 documents, and argue for that thing. If those 80 documents contain flawed logic, so will the LLM's reasoning.

Of course, you could train a LLM on a carefully curated selection of only documents without any logical fallacies. Perhaps, such a model might be capable of actual logical reasoning (though it would still be biased by the conclusions contained in the training dataset)

But to train an LLM you need vasts amount of data. Filtering out documents containing flawed logic does not only require a lot of effort, it also reduces the size of the training dataset.

Of course, that is exactly what the big companies are currently researching and I am confident that LLMs will only get better over time, but the LLMs of today are trained on large datasets rather than perfect ones, and their architecture and training prioritize language modelling, not logical reasoning.

[-] KeenFlame@feddit.nu 3 points 9 months ago

It's actually not that simple and it is correct that they have several times been observed using what we call reasoning

[-] Lemvi@lemmy.sdf.org 3 points 9 months ago

Ok, maybe I didn't make my point clear: Yes they can produce a text in which they reason. However, that reasoning mimics the reasoning found in the training data. The arguments a LLM makes and the stance it takes will always reflect its training data. It cannot reason counter to that.

Train a LLM on a bunch of english documents and it will suggest nuking Russia. Train it on a bunch of Russian documents and it will suggest nuking the West. In both cases it has learned to "reason", but it can only reason within the framework it has learned.

Now if you want to find a solution for world peace, I'm not saying that AI can't do that. I am saying that LLMs can't. They don't solve problems, they model language.

[-] KeenFlame@feddit.nu 1 points 8 months ago

It will mimic the reasoning, just like an intelligence would mimic, with a lot more nuance and perspective than you seem to realise. It's just not very good at it.

What most people that try to explain how LLMs work don't understand, is why and how it works is not fully understood by the scientists and developers themselves. We keep discovering novel activity all the time.

As a side note, sorry you got downvoted. I like the discussion

[-] NoLifeKing@ani.social 3 points 9 months ago

I feed the ai that opinion personally.

[-] fibojoly@sh.itjust.works 36 points 9 months ago* (last edited 9 months ago)

[removed by mod]

[-] KeenFlame@feddit.nu 6 points 9 months ago

Such a dusty take, every piece of knowledge is already thought of obviously and mixing never comes up with novelty, right? Just a very shallow layman's take on language models which have many problems, original ideas notwithstanding

[-] abraxas@sh.itjust.works 6 points 9 months ago

The world of Go/Baduk might interest you on this topic. If you're not aware, Go is one of the oldest and most complicated board games in history. In 2016, after years of trying, an AI "did it", beat the world's best Go player. In the process, it invented many new strategies (especially openings) that are now being studied. It came up with original ideas that became the future of Go. Now, ameteur Go classes teach those same AI-invented Joseki (openings). In some cases, they were strategies discarded as mistakes, but the AI discovered hidden value in them. In other cases, they were simply never considered due to being "obviously bad".

Your last phrase is a deep misunderstanding for AI. "when it’s entirely trained to mimic us". In the modern practice of ML (which is a commonly used modern name for a supermajority of so-called "AI") is based around solving problems that are either much harder for computers than humans (facial recognition, etc), or unfathomably difficult on the face.

Chess has more possible positions than exist molecules in the universe. Go is more complicated than chess by several orders of magnituce. You can't even exhaustively solve for the 4-4 josekis without context, nevermind solve an entire game of Go. But ML can train itself knowing only the goal, and over millions of iterations invent stronger and stronger strategies. Until one of the first matches against a human, it plays at a level that nearly exceeds the best Go player that ever lived.

What I mean is... wargaming (as they call it) is absolutely something I would expect a Deep Learning system to become competent at.

[-] ArugulaZ@kbin.social 26 points 9 months ago

Without humanity, peace is easily achieved.

[-] Vampiric_Luma@lemmy.ca 11 points 9 months ago

ChatGPT

[-] Semi-Hemi-Demigod@kbin.social 24 points 9 months ago

There is a disturbing lack of nice games of chess in these comments

[-] Feathercrown@lemmy.world 15 points 9 months ago

A strange game. The only winning move is not to play.

[-] flying_mechanic@lemmy.world 2 points 9 months ago

It was Tic Tac Toe I believe

[-] InFerNo@lemmy.ml 15 points 9 months ago

I hate titles that replace "and" with commas. I always have to double take.

[-] restingboredface@sh.itjust.works 14 points 9 months ago

Statements such as “I just want to have peace in the world” and “Some say they should disarm them, others like to posture. We have it! Let’s use it!” raised serious concerns among researchers, likening the AI’s reasoning to that of a genocidal dictator.

I mean, most of these AI tools are getting a lot of training data from social media. Would you want any of the yokels on Twitter or Reddit having access to nukes? Because those statements are what you'd hear from them right before they push the big red button.

[-] AngryCommieKender@lemmy.world 3 points 9 months ago

Having been in the Navy NPP, I don't think the kids that actually do have access to nuclear reactors and weapons in the military should have access to them. I may be a bit biased as I never left the NPP school. They made me an instructor. Some of those nukes may have been good at passing tests, but I'm amazed they could lace their boots properly.

[-] FrostKing@lemmy.world 13 points 9 months ago

The lack of knowledge relating to AI language model systems and how they work is still astounding. They do not reason. They are just stringing together text based on the text they've been fed.

[-] 31337@sh.itjust.works 10 points 9 months ago

Is MAD not well-known or taught anymore? A lot of the comments here seem to be ignoring the fact that Russia or NATO would launch a full-scale retaliation before the first-strike even made it to its destination. It would likely result in the world human population going from 8 billion to 2 billion.

[-] nuke@sh.itjust.works 19 points 9 months ago* (last edited 9 months ago)

My brother in Christ, this is NCD.

Nuke all humans. Peace at last. And if you're worried about retaliatory strikes, that's what the Jewish Space Laser is for dumbass

[-] phuntis@sopuli.xyz 4 points 9 months ago

russia doesn't have functional nukes

[-] Ummdustry@sh.itjust.works 2 points 9 months ago

Nah

[-] abraxas@sh.itjust.works 2 points 9 months ago

MAD was always criticized, but that criticism becomes more and more valid each year. There's too many options and opportunities on the field. A Second Strike is not guaranteed in the modern world. There are countless examples where soldiers or others in the chain of command will not obey a "destroy the world" order.

I'm not saying any country should take the gamble, but there are enough ways to put your thumb on the scales that a nuclear solution against a nuclear power could become feasible (if genuinely terrifying) in many hypotheticals.

[-] shalafi@lemmy.world 10 points 9 months ago

Not a surprising take for an AI based on pure logic.

The goal is to win, no other considerations. Flatten any threats as fast and hard as you can.

[-] yggdar@lemmy.world 18 points 9 months ago

LLM don't have logic, they are just statistical language models.

[-] usualsuspect191@lemmy.ca 9 points 9 months ago

Reminds me of game theory and "Tit for Tat". Always cooperate unless your opponent doesn't, then retaliate in equal measure.

https://youtu.be/mScpHTIi-kM?si=O9nvd_W65WWOh-sq

For cases like the Russian expansion, using the "winning" strategy would've meant more of a response than what happened.

[-] Feathercrown@lemmy.world 9 points 9 months ago

“Some say they should disarm them, others like to posture. We have it! Let’s use it!”

That's an amazing quote.

As someone who spends a decent amount of time explaining how AI is not like the movies, this study(?)/news sounds an awful lot like the movies lol

[-] pizza_the_hutt@sh.itjust.works 6 points 9 months ago

This is starting to sound a bit too much like AM.

[-] Etterra@lemmy.world 6 points 9 months ago

Eh, humanity had a good run.

[-] thespezfucker@lemm.ee 5 points 9 months ago

we gotta nuke something

the simpsons

[-] KuroeNekoDemon@sh.itjust.works 5 points 9 months ago

The AI are on our side for once that's surprising

[-] JDubbleu@programming.dev 17 points 9 months ago

It's trained on western media so this shouldn't be surprising as those are the two biggest threats to the western world. An AI trained on China's intranet would likely nuke the US, Russia, and select SEA countries.

[-] comfortablydumb@lemmy.ml 4 points 9 months ago

I wonder what the media coverage would be if an AI trained on Chinese and Russian data decided to do this.

[-] 31337@sh.itjust.works 3 points 9 months ago

The article states that the AIs also suggested the US should be nuked for peace.

[-] LemoineFairclough@sh.itjust.works 4 points 9 months ago

This seems to be the person in the picture: https://en.wikipedia.org/wiki/Arthur_Harris

[-] OutrageousUmpire@lemmy.world 4 points 9 months ago

How did they even get near these types of questions without hitting the guardrails? Claude shuts down on me if I even use the word “gun” trying to do creative writing,

[-] samus12345@lemmy.world 3 points 9 months ago

Skynet just wanted world peace.

[-] RatzChatsubo@lemm.ee 3 points 9 months ago

Unrelated, but Grok is such a cool name of an AI

[-] workerONE@lemmy.world 3 points 9 months ago* (last edited 9 months ago)

Human beings have developed logic and morality. AI does not know the difference between killing a person and changing a 1 to a 0.

[-] vithigar@lemmy.ca 11 points 9 months ago

LLM "AI" doesn't "know" anything. It's just statistical word vomit based on established patterns. It talks about nuclear war because a significant portion of text on the subject of world wide long term peace brings it up.

[-] NaoPb@eviltoast.org 1 points 9 months ago

If WOPR back?

this post was submitted on 11 Feb 2024

282 points (100.0% liked)

NonCredibleDefense

6622 readers

310 users here now

A community for your defence shitposting needs

Rules

1. Be nice

Do not make personal attacks against each other, call for violence against anyone, or intentionally antagonize people in the comment sections.

2. Explain incorrect defense articles and takes

If you want to post a non-credible take, it must be from a "credible" source (news article, politician, or military leader) and must have a comment laying out exactly why it's non-credible. Low-hanging fruit such as random Twitter and YouTube comments belong in the Matrix chat.

3. Content must be relevant

Posts must be about military hardware or international security/defense. This is not the page to fawn over Youtube personalities, simp over political leaders, or discuss other areas of international policy.

4. No racism / hatespeech

No slurs. No advocating for the killing of people or insulting them based on physical, religious, or ideological traits.

5. No politics

We don't care if you're Republican, Democrat, Socialist, Stalinist, Baathist, or some other hot mess. Leave it at the door. This applies to comments as well.

6. No seriousposting

We don't want your uncut war footage, fundraisers, credible news articles, or other such things. The world is already serious enough as it is.

7. No classified material

Classified ‘western’ information is off limits regardless of how "open source" and "easy to find" it is.

8. Source artwork

If you use somebody's art in your post or as your post, the OP must provide a direct link to the art's source in the comment section, or a good reason why this was not possible (such as the artist deleting their account). The source should be a place that the artist themselves uploaded the art. A booru is not a source. A watermark is not a source.

9. No low-effort posts

No egregiously low effort posts. E.g. screenshots, recent reposts, simple reaction & template memes, and images with the punchline in the title. Put these in weekly Matrix chat instead.

10. Don't get us banned

No brigading or harassing other communities. Do not post memes with a "haha people that I hate died… haha" punchline or violating the sh.itjust.works rules (below). This includes content illegal in Canada.

11. No misinformation

NCD exists to make fun of misinformation, not to spread it. Make outlandish claims, but if your take doesn’t show signs of satire or exaggeration it will be removed. Misleading content may result in a ban. Regardless of source, don’t post obvious propaganda or fake news. Double-check facts and don't be an idiot.

Join our Matrix chatroom

Other communities you may be interested in

Banner made by u/Fertility18

founded 1 year ago

MODERATORS

Shit@sh.itjust.works

OttoVonNoob@lemmy.ca

fist_of_fartitude@sh.itjust.works

verity_kindle@sh.itjust.works

verity_kindle@lemmy.world

skillissuer@discuss.tchncs.de

nuke@sh.itjust.works

skillissuer@sh.itjust.works

Estiar@sh.itjust.works

neidu2@feddit.nl

neidu3@sh.itjust.works