446

Rep. Joe Morelle, D.-N.Y., appeared with a New Jersey high school victim of nonconsensual sexually explicit deepfakes to discuss a bill stalled in the House.

top 50 comments
sorted by: hot top controversial new old
[-] Blaidd@lemm.ee 58 points 8 months ago

Creating fake child porn of real people using things like Photoshop is already illegal in the US, I don't see why new laws are required?

[-] Bgugi@lemmy.world 38 points 8 months ago* (last edited 8 months ago)

Well those laws clearly don't work. So we should make new laws! Ones that DEFINITELY WILL work! And if they don't, well I guess we just need more laws until we find ones that do.

[-] NotMyOldRedditName@lemmy.world 16 points 8 months ago

Since we need a rule explicitly for AI related cases, even though it's already covered by others, lets ensure that we also make a 100 page law for if the material is explicitly made in Photoshop, and also another 80 pages if it was made in Gimp. If you use MS Paint to do it, we need a special 200 page law that makes the punishment even harsher, because damn you got skillz and need to be punished more.

[-] LWD@lemm.ee 8 points 8 months ago* (last edited 8 months ago)
[-] Bgugi@lemmy.world 11 points 8 months ago

No, I'm not criticizing the bill's content. If you don't enforce laws, new ones won't work either. The new ones are, at best, an opportunity for people to huff and puff and pat themselves on the back at the cost of actual victims. At worst, it's smoke and mirrors for what the new law actually does.

load more comments (1 replies)
[-] General_Effort@lemmy.world 12 points 8 months ago

This is not at all about protecting children. That's just manipulation. In truth, kids are more likely to prosecuted than protected by this bill.

There are already laws that could be used against teen bullies but it's rarely done. (IMHO it would create more harm than good, anyway.)

This is part of an effort to turn the likenesses of people into intellectual property. Basically, it is about more money for the rich and famous.

This bill would even apply to anyone who shares a movie with a sex scene in it. It's enough that the "depiction" is "realistic" and "created or altered using digital manipulation". Pretty much any photo nowadays, and certainly any movie, can be said to "altered using digital manipulation". There's no mention of age, deception, AI, or anything that the PR bullshit suggests.

[-] rabiddolphin@lemmy.world 11 points 8 months ago

Regulatory capture. OpenAI wants to kick down the ladder

[-] fine_sandy_bottom@discuss.tchncs.de 49 points 8 months ago

I really wonder whether this is the right move.

This girl, and many others, are victims and I don't want to diminish that, but I for better or worse I just don't see how legislation can resolve this.

Surely deepfakes will be just different enough to the subject to create reasonable doubt that it depicts the subject.

I wonder whether, as deep fakes become commonplace, people might be more willing to just ignore it like any other form of trolling.

[-] galoisghost@aussie.zone 35 points 8 months ago

It’s not trolling it’s bullying. You need to think beyond this being about “porn”. This is a reputational attack that makes the victim more likely to be further victimised via date rape, stalking, murder. These things already happen based on rumours, deepfakes images/videos will only make it worse. The other problem is that it’s almost impossible to erase once it’s on the internet, so the victim will likely never be free of the trauma or danger as the images/videos resurface.

load more comments (1 replies)
[-] andrew_bidlaw@sh.itjust.works 23 points 8 months ago

I hope it won't overregulate technology itself but instead would be ruled by already existing means about defaming people and taking photoes without their consent, sharing them. Plus, if she's a teen, it's a production of CSAM. This person had an illegal intent, just used a new tool not unlike others, just more efficient.

[-] loki_d20@lemmy.world 15 points 8 months ago

Surely deepfakes will be just different enough to the subject to create reasonable doubt that it depicts the subject.

That's a major assumption. Do people really think a school board will really consider that when a student creates a fake Only Fans of a teacher? A random University or Company doesn't even give reason for denying an application when they see any form of online nudity? People are lazy as fuck and will just move on to the next candidate or let someone go to save their own image rather than that off the victim.

load more comments (1 replies)
[-] Overzeetop@kbin.social 12 points 8 months ago

I think it doesn't go far enough. Straight up, no one should be permitted to create or transmit the likeness of anyone [prior to, say, 20 years following their death] without their explicit, written permission. Make the fine $1,000,000 or 10% of the offender's net worth, whichever is greater; same penalty and corporate revocation for any corporation involved. Everyone involved from the prompt writer to the work-for-hire people should be liable for the full penalty. I can't think of a valid, non-entertainment (parody/humor), reason for non-consensual impersonation - and using it for humor or parody is a slippery slope to propaganda weaponization. There is no baby in this tub of bathwater.

[-] fine_sandy_bottom@discuss.tchncs.de 15 points 8 months ago

I'm not sure this is practically possible.

A $1m penalty is more or less instant bankruptcy for 99% of the population. It's probably not much of a deterrent for, say an 18 year old. In my jurisdiction I don't think there are criminal penalties higher than a few thousand dollaridoos. It doesn't matter whether you think this act is so aggregious that it deserves a penalty 1000 time higher than any other, my point is that it would be unenforceable ineffective.

Secondly, how do you determine whether an image is someone's likeness? Create any random image and surely it will look like someone, but that doesn't mean that creating that image violates that someone.

[-] Shazbot@lemmy.world 8 points 8 months ago

The missing factor is intent. Make a random image, that's that. But if proven that the accused made efforts to recreate a victim's likeness that shows intent. Any explicit work by the accused with the likeness would be used to prove the charges.

[-] TimeSquirrel@kbin.social 12 points 8 months ago* (last edited 8 months ago)

Yeah, just like the FBI warnings on VHS tapes about massive fines and jail time stopped us from copying them in the 80s and 90s...

[-] AtmaJnana@lemmy.world 10 points 8 months ago

no one should be permitted to create or transmit the likeness of anyone [prior to, say, 20 years following their death] without their explicit, written permission.

I dig the sentiment. I do. And If this were my own fantasy world, I'd agree. But unfortunately, we don't live in the timeline where that is considered even close to reasonable.

load more comments (1 replies)
[-] flipht@kbin.social 10 points 8 months ago

I think you're right if the goal is to stop them all together.

But what we can do is stop people from sending them around and saying that it's true/actually the person.

Once they've turned it from a art project into a weapon, it should have similar consequences to "revenge porn."

[-] HubertManne@kbin.social 12 points 8 months ago

I would think this would be covered by libel, slander, defamation type laws. The crime is basically lying about a persons actions and character.

[-] BrianTheeBiscuiteer@lemmy.world 8 points 8 months ago

I don't know how strong the laws are on the topic but I feel this falls under harassment or libel. In most cases this will cause emotional distress and harm to a person's reputation. If you're trying to show off your AI skills you can use a subject that isn't real or depict a real person wearing clothes. This is clearly an attack in my mind.

[-] AlteredStateBlob@kbin.social 10 points 8 months ago

My dude there are people out there thinking they're in a relationship with Johnny fucking Depp because some Nigerian scammer sent them five badly photoshopped pictures. Step out of your bubble, maybe. This shit isn't easy to spot for the vaaaaaast majority of people and why would this lie with the victim to sort of clear their name or hope that idiots realize it's fake?

Especially with and around teenagers who can barely think further than their next meal?

Good lord.

WDYM "step out of your bubble"?

It's not a question of being able to detect whether or not a video is fake. When deepfakes become so prevalent that everyone's grandma understands that they're prevalent, it won't matter whether you can identify the video as fake.

load more comments (1 replies)
[-] ShellMonkey@lemmy.socdojo.com 40 points 8 months ago

Just wait until them tech savvy folks in Congress try to understand the difference between 'deepfakes' in the sense of pasting a new face on existing footage and whole cloth generative AI creating the entire scene, and then someone tells them that the latter is derived from multiple existing media sources. Gonna be some smoke pouring out of their ears like in the cartoons trying to slice up all the specifics.

[-] SpruceBringsteen@lemmy.world 9 points 8 months ago

Just tell them the 2nd amendment guaranteed your rights to Gaussian Splatting.

[-] uriel238 32 points 8 months ago

FOSTA is still in effect and still causing harm to sex workers while actually protecting human traffickers from investigation (leaving victims stuck as captive labor / sex slaves for longer). And it'd still regarded by our federal legislators as a win, since they don't know any better and can still spin it as a win.

I don't believe our legislators can actually write a bill that won't be used by the federal Department of Justice merely to funnel kids for the sake of filling prison cells with warm bodies.

We've already seen DoJ's unnuanced approach to teen sexting which convicts teens engaging in normal romantic intercourse as professional producers of CSAM.

Its just more fuel for the US prison industrial complex. It is going to heavily affect impoverished kids caught in the crossfire while kids in richer families will get the Brock Turner treatment.

This bill is wholly for political points and has nothing to do with serving the public or addressing disruption due to new technology.

Until we reform or even abolish the law enforcement state, anything we criminalize will be repurposed to target poor and minorities and lock them up in unconscionable conditions.

[-] henfredemars@infosec.pub 31 points 8 months ago* (last edited 8 months ago)

This is just the tip of the iceberg of the threat AI poses to human social structures. We have yet to appreciate the gravity of what these new technologies enable. It's incredibly dangerous yet equally naive to think that AI-generated porn laws will keep us safe.

Firstly, the cat's out of the bag. We can ban the technology or its misuse all we like, but can we really practically stop people from computing mathematical functions? Legal or not, generative AI can and will be used to generate content that hurts people. We need better planning for identifying, authenticating, and responding when this misuse happens.

Secondly, we have an already huge, huge problem with fake news and disinformation. What is such a law for this special case of AI porn going to do for our inability to address harmful content?

It's a shame, but it strikes me as more feel-good than actually doing something effective.

[-] kibiz0r@lemmy.world 9 points 8 months ago

People said the same thing when, after the printing press, there was rampant plagiarism and reverse-plagiarism (attributing words to someone who never said them).

After a period of epistemic chaos, the result was several decades of chartered monopoly and government censorship to get it under control.

I hope we won't need heavy-handed regulation this time around. But that will only happen if we learn from history. We need to get this under control now, while we have the chance to start a framework for protecting our fellow human beings from harm. Complaining that it's hard is not an excuse for doing nothing.

load more comments (1 replies)
[-] ZILtoid1991@kbin.social 5 points 8 months ago

Just ban any instance of generative unconsentual generative nudity, AI or not.

[-] henfredemars@infosec.pub 21 points 8 months ago

I just drew a stick figure of Johnny Depp and it's indeed naked. And I'd do it again, too.

[-] OsrsNeedsF2P@lemmy.ml 28 points 8 months ago

More laws that will only affect people who follow laws

[-] Candelestine@lemmy.world 24 points 8 months ago

I mean, they do usually effect the people that break them and go to prison too.

[-] webghost0101@sopuli.xyz 5 points 8 months ago* (last edited 8 months ago)

You mean the poor and disadvantaged?

[-] Laticauda@lemmy.ca 15 points 8 months ago

Ah yes, the poor and disadvantaged who can afford a computer and an ai program and who make ai csam.

[-] Candelestine@lemmy.world 5 points 8 months ago

When working properly, everyone equally. I will admit we do not nearly reach that standard, though.

load more comments (3 replies)
[-] henfredemars@infosec.pub 17 points 8 months ago

It's like how DRM only hurts people who purchase content legally.

It's been very illegal to pirate games for decades, and still pirated content is quite common in the wild. What's banning it (defamation) harder going to practically achieve?

[-] Phil_in_here@lemmy.ca 10 points 8 months ago

Isn't that... all laws?

[-] kibiz0r@lemmy.world 9 points 8 months ago

Might as well not make any laws then.

[-] Laticauda@lemmy.ca 5 points 8 months ago* (last edited 8 months ago)

How? How will this only effect people who follow laws? If you aren't making ai porn of real underage girls how would this affect you, a law abiding citizen?

load more comments (1 replies)
[-] General_Effort@lemmy.world 18 points 8 months ago

I would have thought that deepfakes are defamation per se. The push to criminalize this is quite the break with American first amendment traditions.

If I understand correctly, this would put any image hoster, including Lemmy, in hot water because 230 immunity is only for civil suits and not federal criminal prosecution.

[-] TORFdot0@lemmy.world 7 points 8 months ago

the text of the bill exempts service providers from any liabilities as long as they make a good faith attempt to remove it as soon as they are aware of its existence. So if someone makes AI generated revenge porn on your instance as long as you take it down when notified, you want be in trouble.

load more comments (5 replies)
load more comments (2 replies)
[-] Coskii 17 points 8 months ago

There is no world where a law aimed at this type of thing will ever be used for its intended purpose.

[-] hydration9806@lemmy.ml 14 points 8 months ago

I feel we are in need of a societal shift here, just like another commenter said about the printing press. When that first came out, the pushback was from the worry that the words would be attributed to someone who never said them (reverse plaigerism). The societal adjustment to this was the universal doubt that anyone said that thing without proof.

For generative AI, when it becomes widespread, photos will be generateable for literally everyone, not just minors but every person with photos online. It will be a societal shift; images will be assumed to be AI generated, making any guilt or shame about a nude photo existing obselete.

Just a matter of time so may as well start now!

load more comments (1 replies)
[-] guyrocket@kbin.social 13 points 8 months ago* (last edited 8 months ago)

How different is photoshopped fakes from AI fakes? Are we going to try to bad that too?

ETA: *ban that too. Thx phone kb.

[-] kibiz0r@lemmy.world 13 points 8 months ago

What does the method matter? If the result is an artifact that is convincing enough for the average person to believe that the subject knowingly posed for sex acts that never occurred, the personal experience and social stigma is traumatizing no matter how it was made.

As the sociologist Brooke Harrington puts it, if there was an E = mc^2^ of social science, it would be SD > PD, “social death is more frightening than physical death.”

[-] guyrocket@kbin.social 5 points 8 months ago

What does the method matter?

That's my point. If we're going to ban AI fakes should we then ban ALL fakes? Where do we draw the line and how do we do that without limiting free speech? I'm not sure it is possible.

And the days of believing everything you see are over but most don't know it yet.

[-] kibiz0r@lemmy.world 7 points 8 months ago* (last edited 8 months ago)

Where do we draw the line

It's ever-changing. We're social animals, not math equations, so it's all according to the kind of society we want.

how do we do that without limiting free speech?

All freedoms are in tension between "freedom to" and "freedom from". I can have the freedom to fire my gun in the air. I can have the freedom from my neighbor's randomly-falling bullets. I can't have both of those codified in law (unless I'm granted some special status over my neighbors).

I think that, many times, what we run into is a mismatch between a group thinking in terms of "freedom to" and a group thinking in terms of "freedom from".

The "freedom to" folks feel like any restriction on their ability to act is a breach of liberty, because they aren't worried about "freedom from". If, for example, I live in the middle of nowhere and have no neighbors, what falling bullets do I have to fear except my own?

The "freedom from" folks feel like having to endure the effects of others' actions is a breach of liberty, because they aren't worried about "freedom to". If I spend my life dodging falling bullets, I'm not likely to fire more into the sky.

And the days of believing everything you see are over but most don’t know it yet.

We said the same thing about the printing press. And it plunged us into a long period of epistemic chaos, with rampant plagiarism and reverse-plagiarism (attributing words to someone who never spoke them). The fallout of this led the crown to seize presses and allocate exclusive printing rights to a chartered monopoly (with some censorship just for funsies).

We can either complain it's too hard and do nothing, eventually leading to an overreaction to a policy that is obviously not sustainable... Or we can learn from history, get our heads in the game, and start imagining a framework that embraces the transformative power of large-scale computing while respecting the humanity of our comrades.

C2PA is a good start, but it's probably DOA in the hacker zeitgeist. We tend to view even an opt-in standard for proof of authenticity as a gateway to universal requirements for proof of authenticity and a locked-down tyrannical internet forever and ever. Possibly because a substantial portion of us are terminally online selfish assholes who never have to spend a second worrying about deepfakes of ourselves. And also fancy ourselves utilitarian techno-solutionists willing to sacrifice the squishy unquantifiable touchy-feely human emotions that just get in the way of objective rational progress towards a transhuman future. It's a noble sacrifice, we say, while profiting disproportionately and suffering none of the fallout.

load more comments (1 replies)
[-] autotldr@lemmings.world 11 points 8 months ago

This is the best summary I could come up with:


A teenage victim of nonconsensual sexually explicit deepfakes joined Rep. Joe Morelle, D-N.Y., on Tuesday to advocate for a bipartisan bill that would criminalize sharing such material at the federal level.

In addition to criminalizing the nonconsensual sharing of sexually explicit deepfakes, the measure would also create a right of private action for victims to be able to sue creators and distributors of the material while remaining anonymous.

Mani said her school administration told her on Oct. 20 that male classmates had created and shared sexually explicit deepfakes of her and more than 30 other girls.

After he heard about what happened at Mani’s high school, which is in his hometown, Rep. Tom Kean, R.-N.J., became the first Republican co-sponsor of Morelle’s bill.

The lack of legislative movement around deepfakes has raised concerns about the technology’s potential to disrupt the 2024 election cycle.

A legal expert who specializes in nonconsensual intimate imagery, Mary Anne Franks, who Morelle said helped inform the bill, said deepfakes have already targeted female politicians.


The original article contains 457 words, the summary contains 169 words. Saved 63%. I'm a bot and I'm open source!

load more comments
view more: next ›
this post was submitted on 18 Jan 2024
446 points (100.0% liked)

Technology

58199 readers
2731 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS