[-] mirrorwitch@awful.systems 12 points 1 week ago

While you all laugh at ChatGPT slop leaving "as a language model..." cruft everywhere, from Twitter political bots to published Springer textbooks, over there in lala land "AIs" are rewriting their reward functions and hacking the matrix and spontaneously emerging mind models of Diplomacy players and generally a week or so from becoming the irresistible superintelligent hypno goddess:

https://www.reddit.com/r/196/comments/1jixljo/comment/mjlexau/

68
submitted 3 weeks ago* (last edited 3 weeks ago) by mirrorwitch@awful.systems to c/techtakes@awful.systems

The other day I realised something cursed, and maybe it's obvious but if you didn't think of it either, I now have to further ruin the world for you too.

Do you know how Google took a nosedive some three-four years ago when managers decided that retention matters more for engagement than user success and, as this process continued, all the results are now so vague and corporatey as to make many searches downright unusable? The way that your keywords are now only vague suggestions at best?

And do you know how that downward spiral got even worse after "AI" took off, not only because the Internet is now drowning in signal-shaped noise, not only because of the "AI snippets" that I'm told USA folk are forced to see, but because tech companies have bought into their own scam and started to use "AI" technology internally, with the effect of an overnight qualitative downstep in accuracy, speed, and resource usage?

So. imagine what this all looks like for the people who have substituted the search bar by the "AI" chatbot.

You search something in Google, say, "arrow materials designs Amazonian peoples". You only get fluff articles, clickbait news, videogame wikis, and a ton of identical "AI" noise articles barely connected to the keywords. No depth no details no info. Very frustrating experience.

You ask ChatGPT or Google Gemini or Duck.AI, as if it was a person, as if it had any idea what it's saying: What were the arrows of Amazonian cultures made of? What type of designs did they use? Can you compare arrows from different peoples? How did they change over time, are today's arrows different?

The bot happily responds in a wise, knowledgeable tone, weaving fiction into fact and conjecture into truth. Where it doesn't know something it just makes up an answer-shaped string of words. If you use an academese tone it will respond in a convincing pastiche of a journal article, and even link to references, though if you read the references they don't say what they're claimed to say but who ever checks that? And if you speak like a question-and-answer section it will respond like a geography magazine, and if you ask in a casual tone it will chat like your old buddy; like a succubus it will adapt to what you need it to be, all the while draining all the fluids you need to live.

From your point of view you had a great experience. no irrelevant results, no intrusive suggestion boxes, no spam articles; just you and the wise oracle who answered exactly what you wanted. Sometimes the bot says it doesn't know the answer, but you just ask again with different words ("prompt engineering") and a full answer comes. You compare that experience to the broken search bar. "Wow this is so much better!"

And sure, sometimes you find out an answer was fake, but what did you expect, perfection? It's a new technology and already so impressive, soon¹ they will fix the hallucination problem. It's my own dang fault for being lazy and not double-checking, haha, I'll be more careful next time.²
(1: never.)
(2: never.)

Imagine growing up with this. You've never even seen search bars that work. From your point of view, "AI" is just superior. You see some cool youtuber you like make a 45min detailed analysis of why "AI" does not and cannot ever work, and you're confused: it's already useful for me, though?

Like saying Marconi the mafia don already helped with my shop, what do you mean extortion? Mr Marconi is already beneficial to me? Why he even protected me from those thugs...

Meanwhile, from the point of view of the souless ghouls at Google? Engagement was atrocious when we had search bars that worked. People click the top result and are off their merry way, already out of the site. The search bar that doesn't work is a great improvement, it makes them hang around and click many more things for several minutes, number go up, ad opportunities, great success. And Gemini? whoa. So much user engagement out of Gemini. And how will Ublock Origin ever manage to block Gemini ads when we start monetising it by subtly recommending this or that product seamlessly within the answer text...

[-] mirrorwitch@awful.systems 12 points 3 months ago

I find it impressive how gen-AI developed a technology that is fine-tuned to generate content that looks precisely passably plausible, but never good enough to be correct or interesting or beautiful or worthwhile in any way.

Like if I was trying to fill the Internet with noise to ruin it, on purpose, I couldn't do better than this. (mostly on accounr of me not having massive data centres nor the moral calousness to spew that much carbon, but still). It's like the ideal infohazard weapon if your goal is to worsen as many lives as you can

[-] mirrorwitch@awful.systems 12 points 3 months ago

Seems like everybody got that email, my account is semi-abandoned and still got it. I love the reek of desperation in the morning

[-] mirrorwitch@awful.systems 14 points 3 months ago

OK so we're getting into deep rat lore now? I'm so sorry for what I'm about to do to you. I hope one day you can forgive me.

LessWrong diaspora factions! :blobcat_ohno:

https://transmom.love/@elilla/113639471445651398

if I got something wrong, please don't tell me. gods I hope I got something wrong. "it's spreading disinformation" I hope I am

[-] mirrorwitch@awful.systems 15 points 5 months ago* (last edited 5 months ago)

What I never get about this stuff is how unfun all of it is. The characters in character.ai don't sound anything like their model characters, at all. ChatGPT necromancy is terrible, the séance table in my hometown sucked but the medium on a lazy day was still significantly better at producing some sort of impersonation that felt at least a little bit like the dead person, a skill I've come to appreciate a bit when compared to ChatGPT's attempt at it. Everything that ChatGPT writes, no matter who it's trying to imitate, has the exact same flavour, and the flavour is slop.

[-] mirrorwitch@awful.systems 17 points 6 months ago

translate technically fiddly instructions of the type where people have trouble spotting mistakes, with patterned noise generators. what could go wrong

[-] mirrorwitch@awful.systems 23 points 6 months ago

The representative of the fascist party in Germany says she's "lesbian but not queer". I think it's the same case.

[-] mirrorwitch@awful.systems 18 points 6 months ago

I also fear that said collapse could be ruinous to big tech, deeply damaging to the startup ecosystem, and will further sour public support for the tech industry.

Yes... ha ha ha... YES!

[-] mirrorwitch@awful.systems 15 points 6 months ago

I find the polygraph to be a fascinating artifact. most on account of how it doesn't work. it's not that it kinda works, that it more or less works, or that if we just iron out a few kinks the next model will do what polygraphs claims to do. the assumptions behind the technology are wrong. lying is not physiological; a polygraph cannot and will never work. you might as well hire me to read the tarot of the suspects, my rate of success would be as high or higher.

yet the establishment pretends that it works, that it means something. because the State desperately wants to believe that there is a path to absolute surveillance, a way to make even one's deepest subjectivity legible to the State, amenable to central planning (cp. the inefficacy of torture). they want to believe it so much, they want this technology to exist so much, that they throw reality out of the window, ignore not just every researcher ever but the evidence of their own eyes and minds, and pretend very hard, pretend deliberately, willfully, desperately, that the technology does what it cannot do and will never do. just the other day some guy way condemned to use a polygraph in every statement for the rest of his life. again, this is no better than flipping a coin to decide if he's saying the truth, but here's the entire System, the courts the judge the State itself, solemnly condemning the man to the whims of imaginary oracles.

I think this is how "AI" works, but on a larger scale.

[-] mirrorwitch@awful.systems 33 points 7 months ago* (last edited 7 months ago)

A note for the unawares that Nanowrimo also tried to cover up a scandal when one of their mods was found to be referring minors to an ABDL fetish site. To my knowledge Nanowrimo never tried to own up to it, never even admitted anything was wrong until the FBI got involved, and still blocks any discussion of the situation.
https://xcancel.com/Arumi_kai/status/1760770617073082629
https://speak-out.carrd.co/

Reportedly they're now shilling AI hard on their Facebook (I don't have Facebook to check). I consider it 100% likely that, from this year on, everyone who uploads their 50k words to the organisation to prove completion will have their work promptly fed to the hungry algorithms.

At least one writer in the board has already resigned over the AI blog post https://xcancel.com/djolder/status/1830464713110540326

68

We also want to be clear in our belief that the categorical condemnation of Artificial Intelligence has classist and ableist undertones, and that questions around the use of AI tie to questions around privilege."

  • Classism. Not all writers have the financial ability to hire humans to help at certain phases of their writing. For some writers, the decision to use AI is a practical, not an ideological, one. The financial ability to engage a human for feedback and review assumes a level of privilege that not all community members possess.
  • Ableism. Not all brains have same abilities and not all writers function at the same level of education or proficiency in the language in which they are writing. Some brains and ability levels require outside help or accommodations to achieve certain goals. The notion that all writers “should“ be able to perform certain functions independently or is a position that we disagree with wholeheartedly. There is a wealth of reasons why individuals can't "see" the issues in their writing without help.
  • General Access Issues. All of these considerations exist within a larger system in which writers don't always have equal access to resources along the chain. For example, underrepresented minorities are less likely to be offered traditional publishing contracts, which places some, by default, into the indie author space, which inequitably creates upfront cost burdens that authors who do not suffer from systemic discrimination may have to incur.

Presented without comment.

[-] mirrorwitch@awful.systems 12 points 7 months ago

I really hate it that I already see in my mind the linkedin types going like "80% of AI projects fail—how you need to sleep in the office 7 days of a week to reach the 20%". I mean Y Combinator used to take pride on the fact that over 90% of startups fail, do you have what it takes to be a ten percenter??

[-] mirrorwitch@awful.systems 13 points 7 months ago* (last edited 7 months ago)

I wanted to torture myself so I looked into that person's posting history. They have made the exact same point that "Monty Python in fact used to mock the gays, how ironic huh?!?" three times in a row in different threads, really showing it to them wokes

view more: next ›

mirrorwitch

joined 7 months ago