461
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 18 Sep 2024
461 points (100.0% liked)
Technology
59253 readers
2515 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
Let's go, already!
How you can help: If you run a website and can filter traffic by user agent, get a list of the known AI scrapers agent strings and selectively redirect their requests to pre-generated AI slop. Regular visitors will see the content and the LLM scraper bots will scrape their own slop and, hopefully, train on it.
This would ideally become standardized among web servers with an option to easily block various automated aggregators.
Regardless, all of us combined are a grain of rice compared to the real meat and potatoes AI trains on - social media, public image storage, copyrighted media, etc. All those sites with extensive privacy policies who are signing contracts to permit their content for training.
Without laws (and I'm not sure I support anything in this regard yet), I do not see AI progress slowing. Clearly inbreeding AI models has a similar effect as in nature. Fortunately there is enough original digital content out there that this does not need to happen.
Absolutely. It's more a matter of principle for me. Kind of like the digital equivalent of leaving fake Amazon packages full of dog poo out front to make porch pirates have a bad day.
Well it means they need some ability to reject some content, which means they need a level of transparency they would never want otherwise.
AI already long ago stopped being trained on any old random stuff that came along off the web. Training data is carefully curated and processed these days. Much of it is synthetic, in fact.
These breathless articles about model collapse dooming AI are like discovering that the sun sets at night and declaring solar power to be doomed. The people working on this stuff know about it already and long ago worked around it.
I mean, we've seen already that AI companies are forced to be reactive when people exploit loopholes in their models or some unexpected behavior occurs. Not that they aren't smart people, but these things are very hard to predict, and hard to fix once they go wrong.
Also, what do you mean by synthetic data? If it's made by AI, that's how collapse happens.
The problem with curated data is that you have to, well, curate it, and that's hard to do at scale. No longer do we have a few decades' worth of unpoisoned data to work with; the only way to guarantee training data isn't from its own model is to make it yourself
But that's exactly my point. Synthetic data is made by AI, but it doesn't cause collapse. The people who keep repeating this "AI fed on AI inevitably dies!" Headline are ignorant of the way this is actually working, of the details that actually matter when it comes to what causes model collapse.
If people want to oppose AI and wish for its downfall, fine, that's their opinion. But they should do so based on actual real data, not an imaginary story they pass around among themselves. Model collapse isn't a real threat to the continuing development of AI. At worst, it's just another checkbox that AI trainers need to check off on their "am I ready to start this training run?" Checklist, alongside "have I paid my electricity bill?"
It was, before we had AI. Turns out that that's another aspect of synthetic data creation that can be greatly assisted by automation.
For example, the Nemotron-4 AI family that NVIDIA released a few months back is specifically intended for creating synthetic data for LLM training. It consists of two LLMs, Nemotron-4 Instruct (which generates the training data) and Nemotron-4 Reward (which curates it). It's not a fully automated process yet but the requirement for human labor is drastically reduced.
But that guarantee isn't needed. AI-generated data isn't a magical poison pill that kills anything that tries to train on it. Bad data is bad, of course, but that's true whether it's AI-generated or not. The same process of filtering good training data from bad training data can work on either.
They'll just start using a chrome user agent
Only if enough people do it. Then again, loads scrapers outside of AI already pretend to be normal browsers.
You can validate that against user telemetry data expected from a browser.
It’s kinda interesting in how it actually roughly parallels the dawn of the nuclear age in some specific ways. Namely, that there’s a clear “purity” line established by the advent of the technology - and I mean that literally, not figuratively. Content on the internet is going to have a very similar dividing line. But it’s also going to be way harder to definitively source data from before that line, unless someone clairvoyant happened to offline and archive a huge storage array with a complete internet snapshot right before ML made its public debut. And I know exactly what the scale of that storage commitment would be, and how much it would cost. So I’m certain nobody has done that.
Are there any good lists of known AI user agents? Ideally in a dependency repo so my server can get the latest values when the list is updated.
Okay but I like using perchance cus they dont profit off anything 👉👈
a large chunk of that site is some dudes lil hobby project and its kinda neat interacting with the community and seein how the code works. Its the only bot I'll ever use cus they arent profiting off of other people shit. the only money they get is from ads and thats it.
Dont kill me with downvotes, I like making up cool OC concepts or poses n stuff and then drawing em.