801
LOL
(lemmy.world)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 19 Feb 2026
801 points (100.0% liked)
Fuck AI
5999 readers
694 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
Does robots.txt really work in the fediverse? At least on lemmy, the content can be retrieved on different hosts, all of which have different robots.txt files. Unless it is somehow "baked" into the protocol.
Major search engines respect robots.txt, but as you said some instances allow them but this is not a scalable way