358

A Telegram user who advertises their services on Twitter will create an AI-generated pornographic image of anyone in the world for as little as $10 if users send them pictures of that person. Like many other Telegram communities and users producing nonconsensual AI-generated sexual images, this user creates fake nude images of celebrities, including images of minors in swimsuits, but is particularly notable because it plainly and openly shows one of the most severe harms of generative AI tools: easily creating nonconsensual pornography of ordinary people.

you are viewing a single comment's thread
view the rest of the comments
[-] guyrocket@kbin.social 127 points 2 years ago

This is not new. People have been Photoshopping this kind of thing since before there was Photoshop. Why "AI" being involved matters is beyond me. The result is the same: fake porn/nudes.

And all the hand wringing in the world about it being non consensual will not stop it. The cat has been out of the bag for a long time.

I think we all need to shift to not believing what we see. It is counterintuitive, but also the new normal.

[-] kent_eh@lemmy.ca 111 points 2 years ago

People have been Photoshopping this kind of thing since before there was Photoshop. Why "AI" being involved matters is beyond me

Because now it's faster, can be generated in bulk and requires no skill from the person doing it.

[-] ArmokGoB@lemmy.dbzer0.com 42 points 2 years ago

I blame electricity. Before computers, people had to learn to paint to do this. We should go back to living like medieval peasants.

[-] 0x0@programming.dev 15 points 2 years ago

Those were the days...

[-] KISSmyOS@feddit.de 2 points 2 years ago

This, but unironically.

[-] Bob_Robertson_IX@discuss.tchncs.de 32 points 2 years ago

A kid at my high school in the early 90s would use a photocopier and would literally cut and paste yearbook headshots onto porn photos. This could also be done in bulk and doesn't require any skills that a 1st grader doesn't have.

[-] ChexMax@lemmy.world 29 points 2 years ago

Those are easily disproven. There's no way you think that's the same thing. If you can pull up the source photo and it's a clear match/copy for the fake it's easy to disprove. AI can alter the angle, position, and expression on your face in a believable manor making it a lot harder to link the photo to source material

[-] Bob_Robertson_IX@discuss.tchncs.de 14 points 2 years ago

This was before Google was a thing, much less reverse lookup with Google Images. The point I was making is that this kind of thing happened even before Photoshop. Photoshop made it look even more realistic. AI is the next step. And even the current AI abilities are nothing compared to what they are going to be even 6 months from now. Yes, this is a problem, but it has been a problem for a long time and anyone who has wanted to create fake nudes of someone has had the ability to easily do so for at least a generation now. We might be at the point now where if you want to make sure you don't have fake nudes created of you, then you don't have images of yourself published. However now that everyone has high quality cameras in their pockets, this won't 100% protect you.

[-] nudnyekscentryk@szmer.info 18 points 2 years ago

But now they are photo realistic

[-] driving_crooner@lemmy.eco.br 13 points 2 years ago

And it's looked as realistic as AI jobs?

[-] nednobbins@lemm.ee 9 points 2 years ago

As much skill as a 9 year old and a 16 year old can muster?

https://en.wikipedia.org/wiki/Cottingley_Fairies

[-] Dkarma@lemmy.world 8 points 2 years ago

Not relevant. Using someone's picture never ever required consent.

[-] Vespair@lemm.ee 1 points 2 years ago

no skill from the person doing it.

This feels entirely non-sequitur, to the point of damaging any point you're trying to make. Whether I paint a nude or the modern Leonardi DaVinci paints a nude our rights (and/or the rights of the model, depending on your perspective on this issue) should be no different, despite the enormous chasm that exists between our artistic skill.

[-] echo64@lemmy.world 88 points 2 years ago

I hate this: "Just accept it women of the world, accept the abuse because it's the new normal" techbro logic so much. It's absolutely hateful towards women.

We have legal and justice systems to deal with this. It is not the new normal for me to be able to make porn of your sister, or mother, or daughter. Absolutely fucking abhorrent.

[-] AquaTofana@lemmy.world 55 points 2 years ago

I don't know why you're being down voted. Sure, it's unfortunately been happening for a while, but we're just supposed to keep quiet about it and let it go?

I'm sorry, putting my face on a naked body that's not mine is one thing, but I really do fear for the people whose likeness gets used in some degrading/depraved porn and it's actually believable because it's AI generated. That is SO much worse/psychologically damaging if they find out about it.

[-] brbposting@sh.itjust.works 17 points 2 years ago

It’s unacceptable.

We have legal and justice systems to deal with this.

For reference, here’s how we’re doing with child porn. Platforms with problems include (copying from my comment two months ago):

Ill adults and poor kids generate and sell CSAM. Common to advertise on IG, sell on TG. Huge problem as that Stanford report shows.

Telegram got right on it (not). Fuckers.

[-] 0x0@programming.dev 12 points 2 years ago

Because gay porn is a myth I guess...

[-] Jrockwar@feddit.uk 12 points 2 years ago* (last edited 2 years ago)

And so is straight male-focused porn. We men seemingly are not attractive, other than for perfume ads. It's unbelievable gender roles are still so strongly coded in 20204. Women must be pretty, men must buy products where women look pretty in ads. Men don't look pretty and women don't buy products - they clean the house and care for the kids.

I'm aware of how much I'm extrapolating, but a lot of this is the subtext under "they'll make porn of your sisters and daughters" but leaving out of the thought train your good looking brother/son, when that'd be just as hurtful for them and yourself.

[-] echo64@lemmy.world 11 points 2 years ago

Sorry if I didn't position this about men. They are the most important thing to discuss and will be the most impacted here, obviously. We must center men on this subject too.

[-] SharkAttak@kbin.social 9 points 2 years ago

It's not normal but neither is new: you already could cut and glue your cousin's photo on a Playboy girl, or Photoshop the hot neighbour on Stallone's muscle body. Today is just easier.

[-] echo64@lemmy.world 9 points 2 years ago

I don't care if it's not new, no one cares about how new it is.

[-] cley_faye@lemmy.world 7 points 2 years ago

How do you propose to deal with someone doing this on their computer, not posting them online, for their "enjoyment"? Mass global surveillance of all existing devices?

It's not a matter of willingly accepting it; it's a matter of looking at what can be done and what can not. Publishing fake porn, defaming people, and other similar actions are already (I hope… I am not a lawyer) illegal. Asking for the technology that exists, is available, will continue to grow, and can be used in a private setting with no witness to somehow "stop" because of a law is at best wishful thinking.

[-] Ookami38@sh.itjust.works 4 points 2 years ago

There's nothing to be done, nor should be done, for anything someone individually creates, for their own individual use, never to see the light of day. Anything else is about one step removed from thought policing - afterall what's the difference between a personally created, private image and the thoughts on your brain?

The other side of that is, we have to have protection for people who this has or will be used against. Strict laws regarding posting or sharing material. Easy and fast removal of abusive material. Actual enforcement. I know we have these things in place already, but they need to be stronger and more robust. The one absolute truth with generative AI, versus Photoshop etc is that it's significantly faster and easier, thus there will likely be an uptick in this kind of material, thus the need for re-examining current laws.

[-] HubertManne@kbin.social 1 points 2 years ago

typical morning for me.

[-] EatATaco@lemm.ee 22 points 2 years ago

I suck at Photoshop and Ive tried many times to get good at it over the years. I was able to train a local stable diffusion model on my and my family's faces and create numerous images of us in all kinds of situations in 2 nights of work. You can get a snap of someone and have nudes of them tomorrow for super cheap.

I agree there is nothing to be done, but it's painfully obvious to me that the scale and ease of it that makes it much more concerning.

[-] T156@lemmy.world 4 points 2 years ago

Also the potential for automation/mass-production. Photoshop work still requires a person to sit down to do the actual photoshop. You can try to script things out, but it's hardly an easy affair.

By comparison, generative models are much more hands-free. Once you get the basics set up, you can just have it go, and churn things at rates well surpassing what a single human could reasonably do (if you have the computing power for it).

[-] AstralPath@lemmy.ca 21 points 2 years ago

This kind of attitude toward non-consensual actions is what perpetuates them. Fuck that shit.

[-] Assman@sh.itjust.works 20 points 2 years ago

The same reason AR15 rifles are different than muskets

[-] daddy32@lemmy.world 11 points 2 years ago
[-] SharkAttak@kbin.social 8 points 2 years ago
[-] EatATaco@lemm.ee 3 points 2 years ago

The irony of parroting this mindless and empty talking point is probably lost on you.

[-] SharkAttak@kbin.social 1 points 2 years ago

God, do I really have to start putting the /jk or /s back, for those who don't get it like you??

[-] EatATaco@lemm.ee 1 points 2 years ago
[-] SharkAttak@kbin.social 1 points 2 years ago

Okay, okay, you won. Happy now? Now go.

[-] EatATaco@lemm.ee 1 points 2 years ago
[-] HubertManne@kbin.social 2 points 2 years ago

This is something I can't quite get through to my wife. She does not like that I dismiss things to some degree when it does not makes sense. We get into these convos where Im like I have serious doubts about this and she is like. Are you saying it did not happen and im like. no. It may have happened but not in quite the way they say or its being portrayed in a certain manner. Im still going to take video and photos for now as being likely true but I generally want to see it from independent sources. like different folks with their phones along with cctv of some kind and such.

[-] Pretzilla@lemmy.world 33 points 2 years ago

Ok so pay the dude $10 to put your wife's head on someone agreeing with you. Problem solved.

[-] roscoe@lemmy.dbzer0.com 6 points 2 years ago

I didn't expect to get a laugh out of reading this discussion, thanks.

[-] HubertManne@kbin.social 4 points 2 years ago

lol. there you go. hey you cheated on me. its in this news article right here.

[-] A_Very_Big_Fan@lemmy.world 2 points 2 years ago

Why "AI" being involved matters is beyond me.

The AI hysteria is real, and clickbait is money.

this post was submitted on 29 Mar 2024
358 points (100.0% liked)

Technology

77341 readers
1568 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS