Bruh, this is literally like scientists scavenging metal from shipwrecks that predate the era of nuclear bomb testing, cause they need steel that isn't contaminated by nuclear fallout for certain measuring equipment 😵💫😵💫
Great analogy for the shit we're currently saturating our digital world with
Low background bullshit training data.
Bruh, this is literally like scientists scavenging metal from shipwrecks that predate the era of nuclear bomb testing, cause they need steel that isn’t contaminated by nuclear fallout for certain measuring equipment
Except in this case, these scientists are working at a nuclear weapons lab. They need metal that hasn't been contaminated by nuclear bombs so that they can invent better nuclear bombs.
A guy I used to know who would go to estate sales looking for blacksmithing equipment once told me the market for old metalworking stuff is disappearing because too much stuff is being bought by companies én masse for this reason.
I don't know if I believe that companies are buying small lot auctions and estate sales for old anvils, but he was sure convinced.
I like to joke I'm sitting on a pile of cash because my anvil isn't radioactive. (it's not a very good joke, but it carries some weight!)
According to Wikipedia, while a market for old, low-background steel was real phenomenon for a few decades, that's pretty much over.
World anthropogenic background radiation ... began to fall in 1963 ... and by 2008 it had decreased to only 0.005 mSv/yr above natural levels.[9] This has made special low-background steel no longer necessary for most radiation-sensitive uses, as new steel now has a low enough radioactive signature.
So you may consider your friend's estate-sale theory in light of that time-line.
but it carries some weight
Did you know about !dadjokes@lemmy.world?
If a big company needed a lot of something specific, they'd probably pay a pretty penny and let people going to small lot auctions accumulate a sizeable amount and then buy it from them.
It's not impossible to buy stuff from small lots at scale, as long as you're willing to pay a premium
Tell me more about nuclear steel.
It’s actually the low radiation stuff from before the first nuclear bombs were detonated that we want. Turns out those kinds of bombs spread fallout and now literally everything on the planet has a trace amount
nice
The worst part is that they are destroying the books afterwards.
We also have no guarantee these books will be available at a later date in these digitized formats, nor that they won’t be altered to fit certain narratives which is of course trivial to do with digital copies. At a minimum these need public verifiable checksums to be relied upon.
I don’t like this and don’t trust it one bit.
These AI companies already showed their hand in trying to corner the PC component market with an end goal of users no longer being able to own their own PCs. Now they’re potentially trying to do this with all written recorded information.
How long until Altman’s Firemen show up at your door to burn your books?
They even hve an economical incentive to destroy the books, because by doing so, they will get the data and prevent the competition from getting it for them
Literally Fahrenheit 451
Is it really destroying though? They're digitizing them, and publishers still have the digital copies ready to print more at any time. So it's not like they're destroying the texts, they're just shifting them.
Nobody complained when Google did this over a decade ago 🤷
When you say they're "destroying the books" you make it sound like they're erasing one of the last known copy of some important work when in reality, most of these books were purchased in bulk from bookstores and libraries that were planning on discarding them anyway.
Almost all these books were either headed to the dump or the recycling center. They're just being digitized on the way.
Digitized for private consumption.
Nobody cares when Google did this or when archive.org does this, because they’re sharing the results with the world. (Idiotic shortsighted lawsuits from the authors guild notwithstanding)
in this case they often are actually destroying them. they take them apart, beause it's easier to scan than to use a proper book scanner
When they destroy the physical copy they remove them from the antique stores market which often rely on circulation.
Edit: Also physical copies don’t require electricity, a device and internet access plus they are something you can own.
Almost all these books were either headed to the dump or the recycling center.
My wife works at a law firm that contracts with "Books By The Yard", which provides books purely for office aesthetics. For a few hundred bucks you can plaster a bookshelf full of material nobody will ever read, because they're such a commodity.
It's so crazy to see people ingesting this news from the conspiracy-brain perspective of "The AI companies are stealing all the knowledge!" without recognizing the more pressing reality of decade upon decade of publisher overproduction, resulting in a total devaluing of physical media for its academic importance.
Imagine going into hysterics because a warehouse full of shitty airport books went up in smoke, like it's the Library of Alexandra that just burned down.
Are you sure they're digitizing Book By the Yard quality material?
Consider this. Purchasing, shipping, and physically scanning a book is the single most expensive way for an AI company to acquire the text of a work. It's been widely documented in court cases that these companies engage in mass IP theft. They're literally running massive torrenting farms, grabbing copies of every film, song, book, etc. that they can get their hands on.
What kinds of texts are most likely to be found pirated on the internet? It's the mass market stuff. The common stuff was digitized long ago. They can just download that. They can pirate it. They can buy the ebook. There's no need for OpenAI to purchase and destructively scan the works of Steven King. No shade on the man, but his works aren't exactly hard to find. I'm sure they can just find a torrent.
There's little value in scanning the mass market books that are produced in enormous quantities. They probably don't buy "Book by the Yard" books, because they already have digital copies of those.
But the rare, long out-of-print stuff? The stuff that you actually cannot find a legal or illegal copy of online? That's only stuff worth paying money to buy, ship, and scan.
Doing anything physical, especially at scale, is slow and expensive. I would guess a good portion of these books, perhaps an outright majority, have simply never been digitized.
I imagine they were talking about destruction in a practical sense. Disassembling the book and scanning it like that is faster and cheaper than purpose build book scanning machines.
Additionally, court documents indicate they generally just throw them away afterwards. So the knowledge is retained, but that book is destroyed.
https://tagteam.harvard.edu/hub_feeds/3415/feed_items/14341220
The knowledge is retained and owned by a private corporation who now does not have to share what may have been a still under copyright, but now the corporation owns it? Make it make sense.
Make it make sense
A robber barron class of psychopaths are running the whole damn show!
Reminds me of Judge Holden in Blood Meridian, who sketched ancient artifacts in his personal notebook and then destroyed the originals.
Damn AI ruined AI.
Hee hee.
They're doing everything they can to avoid it, but it's supposed to be tHe FuTuRe
Destroying the books too :(
Not to defend the AI crowd, but ever go to a Goodwill Outlet store? They usually have at least 1 gaylord box (the giant box on a pallet watermelons come in) full of books. Not sure how often they cycle those out, but I'm guessing pretty regularly. Multiply that by every semi major city Goodwill has a presence in. Add in other thrift sores with a similar quantity of books. You're talking ALOT of books. SO MANY books are thrown out everyday. Including by libraries. I don't really have a problem if some of those get diverted for a bit to be scanned. Yes, it's sad to see books getting destroyed. But there's alot of stuff people just really don't want. And, if you REALLY want a villain, maybe you can blame copyright law? That's part of the reason AI companies destroy the books - it helps to make their scanning efforts more legal. Absent that, they could probably be convinced to spend just a little more for nondestructive scanning, assuming they could find a home for the books they scanned (the sellers of these things couldn't, so good luck).
If the AI companies were simply picking up books that were already going to a landfill, I don't think that would bother anybody. That isn't what we're talking about here, though. These books are coming from booksellers, so it's fair to guess that most of them are at least a notch or two above what you'd find in a Goodwill bargain bin.
Modern book burning
Anna's Archive: ...uh, well okay, then.
You can only stave off model collapse so long.
It’s only a matter of time before most of the data going in to LLMs came out of one too.
Like I’m not even good at math or anything, much less computer science, but that seems obvious and inevitable to me.
Maybe someone smarter than me can chime in.
It's kind of sort of already happening. Emdashes are fragments of it, a token saving strategy introduced by already trained on old books in their data sources. Model collapse is inevitable.
I'm waiting for the day when desperate LLM companies start paying people to post real human content, only for those people to just ask ChatGPT to do it.
Already happening for years.
Amazon mechanical turk, Outlier.ai, scale ai, are all gig platforms that pay people to produce real human content for LLM training.
At first they were paying people to actually produce content, like worked solutions to maths problems or translations, then they pivoted more to rating LLM output.
And as with every enshittification cycle, wages and work rapidly dried up, so people responded by asking LLMs for the answer just to meet deadlines and get by.
"now you only posted 4 times today Jenny, do you just want to do the minimum?"
Nice that they're actually buying them this time instead of just downloading torrents of book collections again.
I imagine they already burned through literally all of the torrentable books
Ah fuck... No wonder old maths texts are a nightmare to get hold of recently...
This is a most excellent place for technology news and articles.
