155
submitted 2 weeks ago by cm0002@lemy.lol to c/linux@programming.dev
you are viewing a single comment's thread
view the rest of the comments
[-] Franconian_Nomad@feddit.org 1 points 2 weeks ago

They take code snippets and copy and paste them? Or do they create own code based on what they’ve learned by trawling?

[-] Barbarian@sh.itjust.works 16 points 2 weeks ago* (last edited 2 weeks ago)

LLMs don't "create". Under the hood, they're tokenizing the queries, looking for "clouds" of tokens that are similar to the query, then returning a sequence of tokens (with some random noise thrown in) that match what their training data says the answer should be.

In short: all LLM code is an amalgamation of their training data by definition. If there's nothing similar in there, it's literally not possible for it to be part of any response.

[-] Franconian_Nomad@feddit.org 3 points 2 weeks ago

You’re exactly right. I should have used „generate“ instead of „create“.The point is I don’t think LLMs normally use copyrighted code in a way that would hurt open source projects.

Under the hood, they're tokenizing the queries, looking for "clouds" of tokens that are similar to the query, then returning a sequence of tokens (with some random noise thrown in) that match what their training data says the answer should be.

Lol, so how do humans code in comparison?

[-] prole 6 points 2 weeks ago

You’re exactly right. I should have used „generate“ instead of „create“

Did you purposely respond like an AI?

[-] Franconian_Nomad@feddit.org 3 points 1 week ago

Damn, I forgot the em dash.

[-] Barbarian@sh.itjust.works 4 points 2 weeks ago* (last edited 2 weeks ago)

The point is I don’t think LLMs normally use copyrighted code in a way that would hurt open source projects.

I don't know. I'm not a lawyer, and copyright for code was a hot mess even before LLMs got involved. With how many opportunistic copyright/patent trolls there are and how easily convinced judges have been in the past, it could go either way.

Lol, so how do humans code in comparison?

The good programmers normally code by breaking down the problem into constituent parts and logically working through the problem, step by step. What differentiates this from tokenization is that instead of just looking for code that is similar for a similar problem, programmers can usually understand the effects of each line of code, visualize what the state of each variable will be in that step (or dump out the variables to look directly if unsure), and then move on to the next step. This logical problem-solving approach is fundamentally different from a tokenization+noise looking for a similar-looking problem approach. For one thing, you can solve problems that haven't been solved before.

[-] vanillama@programming.dev 3 points 2 weeks ago

Human programmers at least can tell you where they got a snippet they copied, whether it was in the docs, stack overflow or elsewhere, and you can try to keep attribution if you care about compliance. Not only that, but most of our skills are related to designing stuff and recognizing which pattern to use, the specific implementation isn't necessary the same unless we go look for whatever we saw in the past, as our memories don't just record everything and repeat it word by word. And after picking up a new language or framework I only need to look around when using a third party library or some API I'm less familiar with, or when something breaks.

this post was submitted on 16 Jul 2026
155 points (100.0% liked)

Linux

14476 readers
456 users here now

A community for everything relating to the GNU/Linux operating system (except the memes!)

Also, check out:

Original icon base courtesy of lewing@isc.tamu.edu and The GIMP

founded 3 years ago
MODERATORS