297

A machine learning librarian at Hugging Face just released a dataset composed of one million Bluesky posts, complete with when they were posted and who posted them, intended for machine learning research.

Daniel van Strien posted about the dataset on Bluesky on Tuesday:

“This dataset contains 1 million public posts collected from Bluesky Social's firehose API, intended for machine learning research and experimentation with social media data,” the dataset description says. “Each post contains text content, metadata, and information about media attachments and reply relationships.”

The data isn’t anonymous. In the dataset, each post is listed alongside the users’ decentralized identifier, or DID; van Strien also made a search tool for finding users based on their DID and published it on Hugging Face. A quick skim through the first few hundred of the million posts shows people doing normal types of Bluesky posting—arguing about politics, talking about concerts, saying stuff like “The cat is gay” and “When’s the last time yall had Boston baked beans?”—but the dataset has also swept up a lot of adult content, too.

you are viewing a single comment's thread
view the rest of the comments
[-] foremanguy92_@lemmy.ml 76 points 2 days ago

If you post something publicly, that thing will be used to train AI. Nevertheless the privacy speaks of the company.

[-] Brumefey@sh.itjust.works 26 points 2 days ago

I don’t know why social media are used for training. It’s like the worst quality of data ever and it results to answers like « go kill youself » when prompted about something sad…

[-] EngineerGaming@feddit.nl 10 points 2 days ago

IDK if this is accurate, but it's absolutely my headcanon as to why Glados decided to murder everyone almost immediately after she was turned on. She just vacuumed up the collective stupidity of the Internet.

[-] Infomatics90@lemmy.ca 1 points 1 day ago

Its good headcanon, but it omits the fact that GLADOS isn't aware of the combine, and for some reason the combine haven't even found Aperture labs.

[-] EngineerGaming@feddit.nl 2 points 1 day ago

I was under the impression that she knew and was deliberately laying low... Also, wasn't she built before these events?

[-] Infomatics90@lemmy.ca 1 points 1 day ago

I could be wrong. I just don't remember her mentioning anything about them or seeing anything about them in the game.

[-] EngineerGaming@feddit.nl 2 points 1 day ago

There have been suggestions that this line references them: "Are you trying to escape? Things have changed since the last time you left the building. What's going on out there will make you wish you were back in here. I have an infinite capacity for knowledge, and even I'm not sure what's going on outside. All I know is I'm the only thing standing between us and them. Well, I was".

She might just be lying like she usually does, but it still wouldn't be surprising if it indeed was that time period.

[-] Infomatics90@lemmy.ca 1 points 1 day ago

oh, i missed that. been awhile since i played.

[-] TriflingToad@sh.itjust.works 1 points 2 days ago

oh my God you are so right

[-] foremanguy92_@lemmy.ml 6 points 2 days ago

They are used because they are "real life" (not really but you know) conversation example

[-] Pandemanium@lemm.ee 4 points 2 days ago

But why do we need to recreate "real life?" Don't we already do this relatively well in books, TV, and movies? People keep saying we won't use AI to replace creative writing, but this (and propaganda, making bot conversations seem like real people) are the only use cases for this kind of data. LLMs don't need to improve their conversation skills. What they really need is to stop hallucinating, and this kind of data won't help with that.

load more comments (10 replies)
this post was submitted on 27 Nov 2024
297 points (100.0% liked)

Technology

59689 readers
2581 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS