Instagram Advertises Nonconsensual AI Nude Apps ( www.404media.co )

Instagram is profiting from several ads that invite people to create nonconsensual nude images with AI image generation apps, once again showing that some of the most harmful applications of AI tools are not hidden on the dark corners of the internet, but are actively promoted to users by social media companies unable or unwilling to enforce their policies about who can buy ads on their platforms.

While parent company Meta’s Ad Library, which archives ads on its platforms, who paid for them, and where and when they were posted, shows that the company has taken down several of these ads previously, many ads that explicitly invited users to create nudes and some ad buyers were up until I reached out to Meta for comment. Some of these ads were for the best known nonconsensual “undress” or “nudify” services on the internet.

uriel238 ,
@uriel238@lemmy.blahaj.zone avatar

It remains fascinating to me how these apps are being responded to in society. I'd assume part of the point of seeing someone naked is to know what their bits look like, while these just extrapolate with averages (and likely, averages of glamor models). So we still dont know what these people actually look like naked.

And yet, people are still scorned and offended as if they were.

Technology is breaking our society, albeit in place where our culture was vulnerable to being broken.

echodot ,

I suspect it's more affecting for younger people who don't really think about the fact that in reality, no one has seen them naked. Probably traumatizing for them and logic doesn't really apply in this situation.

Grandwolf319 ,

I think half the people who are offended don’t get this.

The other half think that it’s enough to cause hate.

Both arguments rely on enough people being stupid.

postmateDumbass ,

How dare that other person i don't know and will never meet gain sexual stimulation!

inb4_FoundTheVegan ,
@inb4_FoundTheVegan@lemmy.world avatar

My body is not inherently for your sexual simulation. Downloading my picture does not give you the right to turn it in to porn.

CaptainEffort ,
@CaptainEffort@sh.itjust.works avatar

Did you miss what this post is about? In this scenario it’s literally not your body.

inb4_FoundTheVegan ,
@inb4_FoundTheVegan@lemmy.world avatar

There is nothing stopping anyone from using it on my body. Seriously, get a fucking grip.

inb4_FoundTheVegan ,
@inb4_FoundTheVegan@lemmy.world avatar

Wtf are you even talking about? People should have the right to control if they are "approximated" as nude. You can wax poetic how it's not nessecarily correct but that's because you are ignoring the woman who did not consent to the process. Like, if I posted a nude then that's on the internet forever. But now, any picture at all can be made nude and posted to the internet forever. You're entirely removing consent from the equation you ass.

Drewelite ,

Totally get your frustration, but people have been imagining, drawing, and photoshopping people naked since forever. To me the problem is if they try and pass it off as real. If someone can draw photorealistic pieces and drew someone naked, we wouldn't have the same reaction, right?

inb4_FoundTheVegan ,
@inb4_FoundTheVegan@lemmy.world avatar

It takes years of pratice to draw photorealism, and days if not weeks to draw a particular piece. Which is absolutely not the same to any jackass with an net connection and 5 minutes to create a equally/more realistic version.

It's really upsetting that this argument keeps getting brought up. Because while guys are being philosophical about how it's therotically the same thing, women are experiencing real world harm and harassment from these services. Women get fired for having nudes, girls are being blackmailed and bullied with this shit.

But since it's theoretically always been possible somehow churning through any woman you find on Instagram isn't an issue.

Totally get your frustration

Do you? Since you aren't threatened by this, yet another way for women to be harassed is just a fun little thought experiment.

Drewelite ,

Well that's exactly the point from my perspective. It's really shitty here in the stage of technology where people are falling victim to this. So I really understand people's knee jerk reaction to throw on the brakes. But then we'll stay here where women are being harassed and bullied with this kind of technology. The only paths forward, theoretically, are to remove it all together or to make it ubiquitous background noise. Removing it all together, in my opinion, is practically impossible.

So my point is that a picture from an unverified source can never be taken as truth. But we're in a weird place technologically, where unfortunately it is. I think we're finally reaching a point where we can break free of that. If someone sends me a nude with my face on it like, "Is this you?!!". I'll send them one back with their face like, "Is tHiS YoU?!??!".

We'll be in a place where we as a society cannot function taking everything we see on the internet as truth. Not only does this potentially solve the AI nude problem, It can solve the actual nude leaks / revenge porn, other forms of cyberbullying, and mass distribution of misinformation as a whole. The internet hasn't been a reliable source of information since its inception. The problem is, up until now, its been just plausible enough that the gullible fall into believing it.

nandeEbisu ,

Regardless of what one might think should happen or expect to happen, the actual psychological effect is harmful to the victim. It's like if you walked up to someone and said "I'm imagining you naked" that's still harassment and off-putting to the person, but the image apps have been shown to have much much more severe effects.

It's like the demonstration where they get someone to feel like a rubber hand is theirs, then hit it with a hammer. It's still a negative sensation even if it's not a strictly logical one.

UnderpantsWeevil ,
@UnderpantsWeevil@lemmy.world avatar

So we still dont know what these people actually look like naked.

I think the offense is in the use of their facial likeness far more than their body.

If you took a naked super-sized barbie doll and plastered Taylor Swift's face on it, then presented it to an audience for the purpose of jerking off, the argument "that's not what Taylor's tits look like!" wouldn't save you.

Technology is breaking our society

Unregulated advertisement combined with a clickbait model for online marketing is fueling this deluge of creepy shit. This isn't simply a "Computers Evil!" situation. Its much more that a handful of bad actors are running Silicon Valley into the ground.

uriel238 ,
@uriel238@lemmy.blahaj.zone avatar

Not so much computers evil! as just acknowledging there will always be malicious actors who will find clever ways to use technology to cause harm. And yes, there's a gathering of folk on 4Chan/b who nudify (denudify?) submitted pictures, usually of people they know, which, thanks to the process, puts them out on the internet. So this is already a problem.

Think of Murphy's Law as it applies to product stress testing. Eventually, some customer is going to come in having broke the part you thought couldn't be broken. Also, our vast capitalist society is fueled by people figuring out exploits in the system that haven't been patched or criminalized (see the subprime mortgage crisis of 2008). So we have people actively looking to utilize technology in weird ways to monetize it. That folds neatly like paired gears into looking at how tech can cause harm.

As for people's faces, one of the problems of facial recognition as a security tool (say when used by law enforcement to track perps) is the high number of false positives. It turns out we look a whole lot like each other. Though your doppleganger may be in another state and ten inches taller / shorter. In fact, an old (legal!) way of getting explicit shots of celebrities from the late 20th century was to find a look-alike and get them to pose for a song.

As for famous people, fake nudes have been a thing for a while, courtesy of Photoshop or some other digital photo-editing set combined with vast libraries of people. Deepfakes have been around since the late 2010s. So even if generative AI wasn't there (which is still not great for video in motion) there are resources for fabricating content, either explicit or evidence of high crimes and misdemeanors.

This is why we are terrified of AI getting out of hand, not because our experts don't know what they're doing, but because the companies are very motivated to be the first to get it done, and that means making the kinds of mistakes that cause pipeline leakage on sacred Potawatomi tribal land.

UnderpantsWeevil ,
@UnderpantsWeevil@lemmy.world avatar

This is why we are terrified of AI getting out of hand

I mean, I'm increasingly of the opinion that AI is smoke and mirrors. It doesn't work and it isn't going to cause some kind of Great Replacement any more than a 1970s Automat could eliminate the restaurant industry.

Its less the computers themselves and more the fear surrounding them that seem to keep people in line.

uriel238 ,
@uriel238@lemmy.blahaj.zone avatar

The current presumption that generative AI will replace workers is smoke and mirrors, though the response by upper management does show the degree to which they would love to replace their human workforce with machines, or replace their skilled workforce with menial laborers doing simpler (though more tedious) tasks.

If this is regarded as them tipping their hands, we might get regulations that serve the workers of those industries. If we're lucky.

In the meantime, the pursuit of AGI is ongoing, and the LLMs and generative AI projects serve to show some of the tools we have.

It's not even that we'll necessarily know when it happens. It's not like we can detect consciousness (or are even sure what consciousness / self awareness / sentience is). At some point, if we're not careful, we'll make a machine that can deceive and outthink its developers and has the capacity of hostility and aggression.

There's also the scenario (suggested by Randall Munroe) that some ambitious oligarch or plutocrat gains control of a system that can manage an army of autonomous killer robots. Normally such people have to contend with a principal cabinet of people who don't always agree with them. (Hitler and Stalin both had to argue with their generals.) An AI can proceed with a plan undisturbed by its inhumane implications.

UnderpantsWeevil ,
@UnderpantsWeevil@lemmy.world avatar

I can see how increased integration and automation of various systems consolidates power in fewer and fewer hands. For instance, the ability of Columbia administrators to rapidly identify and deactivate student ID cards and lock hundreds of protesters out of their dorms with the flip of a switch was really eye-opening. That would have been far more difficult to do 20 years ago, when I was in school.

But that's not an AGI issue. That's a "everyone's ability to interact with their environment now requires authentication via a central data hub" issue. And its illusionary. Yes, you're electronically locked out of your dorm, but it doesn't take a lot of savvy to pop through a door that's been propped open with a brick by a friend.

There’s also the scenario (suggested by Randall Munroe) that some ambitious oligarch or plutocrat gains control of a system that can manage an army of autonomous killer robots.

I think this fear heavily underweights how much human labor goes into building, maintaining, and repairing autonomous killer robots. The idea that a singular megalomaniac could command an entire complex system - hell, that the commander could even comprehend the system they intended to hijack - presumes a kind of Evil Genius Leader that never seems to show up IRL.

Meanwhile, there's no shortage of bloodthirsty savages running around Ukraine, Gaza, and Sudan, butchering civilians and blowing up homes with sadistic glee. You don't need a computer to demonstrate inhumanity towards other people. If anything, its our human-ness that makes this kind of senseless violence possible. Only deep ethnic animus gives you the impulse to diligently march around butchering pregnant women and toddlers, in a region that's gripped by famine and caught in a deadly heat wave.

Would that all the killing machines were run by some giant calculator, rather than a motley assortment of sickos and freaks who consider sadism a fringe benefit of the occupation.

iquanyin ,
@iquanyin@lemmy.world avatar

hmmm . i’m not sure we will be able to give emotion to something that has no needs, no living body, and doesn’t die. maybe. but it seems to me that emotions are survival tools that develop as beings and their environment develop, in order to keep a species alive. i could be wrong.

Nobody ,

It’s all so incredibly gross. Using “AI” to undress someone you know is extremely fucked up. Please don’t do that.

MxM111 ,
@MxM111@kbin.social avatar

Can you articulate why, if it is for private consumption?

KidnappedByKitties ,

Consent.

You might be fine with having erotic materials made of your likeness, and maybe even of your partners, parents, and children. But shouldn't they have right not to be objectified as wank material?

I partly agree with you though, it's interesting that making an image is so much more troubling than having a fantasy of them. My thinking is that it is external, real, and thus more permanent even if it wouldn't be saved, lost, hacked, sold, used for defamation and/or just shared.

InternetPerson ,

To add to this:

Imagine someone would sneak into your home and steal your shoes, socks and underwear just to get off on that or give it to someone who does.

Wouldn't that feel wrong? Wouldn't you feel violated? It's the same with such AI porn tools. You serve to satisfy the sexual desires of someone else and you are given no choice. Whether you want it or not, you are becoming part of their act. Becoming an unwilling participant in such a way can feel similarly violating.

They are painting and using a picture of you, which is not as you would like to represent yourself. You don't have control over this and thus, feel violated.

This reminds me of that fetish, where one person is basically acting like a submissive pet and gets treated like one by their "master". They get aroused by doing that in public, one walking with the other on a leash like a dog on hands and knees.
People around them become passive participants of that spectactle. And those often feel violated. Becoming unwillingly, unasked a participant, either active or passive, in the sexual act of someone else and having no or not much control over it, feels wrong and violating for a lot of people.
In principle that even shares some similarities to rape.

There are countries where you can't just take pictures of someone without asking them beforehand. Also there are certain rules on how such a picture can be used. Those countries acknowledge and protect the individual's right to their image.

scarilog ,

Just to play devils advocate here, in both of these scenarios:

Imagine someone would sneak into your home and steal your shoes, socks and underwear just to get off on that or give it to someone who does.

This reminds me of that fetish, where one person is basically acting like a submissive pet and gets treated like one by their "master". They get aroused by doing that in public, one walking with the other on a leash like a dog on hands and knees. People around them become passive participants of that spectactle. And those often feel violated.

The person has the knowledge that this is going on. In he situation with AI nudes, the actual person may never find out.

Again, not to defend this at all, I think it's creepy af. But I don't think your arguments were particularly strong in supporting the AI nudes issue.

CleoTheWizard ,
@CleoTheWizard@lemmy.world avatar

In every chat I find about this, I see people railing against AI tools like this but I have yet to hear an argument that makes much sense to me about it. I don’t care much either way but I want a grounded position.

I care about harms to people and in general, people should be free to do what they want until it begins harming someone. And then we get to have a nuanced conversation about it.

I’ve come up with a hypothetical. Let’s say that you write naughty stuff about someone in your diary. The diary is kept in a secure place and in private. Then, a burglar breaks in and steals your diary and mails that page to whomever you wrote it about. Are you, the writer, in the wrong?

My argument would be no. You are expressing a desire in private and only through the malice of someone else was the harm done. And no, being “creepy” isn’t an argument either. The consent thing I can maybe see but again do you have a right not to be fantasized about? Not to be written about in private?

I’m interested in people’s thoughts because this argument bugs me not to have a good answer for.

Resonosity ,

Yeah it's an interesting problem.

If we go down the path of ideas in the mind and the representations we create and visualize in our mind's eye, to forbid people from conceiving of others sexually means there really is no justification for conceiving of people generally.

If we try to seek for a justification, where is that line drawn? What is sexual, and what is general? How do we enforce this, or at least how do we catch people in the act and shame them into stopping their behavior, especially if we don't possess the capability of telepathy?

What is harm? Is it purely physical, or also psychological? Is there a degree of harm that should be allowed, or that is inescapable despite our best intentions?

The angle that you point out regarding writing things down about people in private can also go different ways. I write things down about my friends because my memory sucks sometimes and I like to keep info in my back pocket for when birthdays, holidays, or special occasions come. What if I collected information about people that I don't know? What if I studied academics who died in the past to learn about their lives, like Ben Franklin? What if I investigated my neighbors by pointing cameras at their houses, or installing network sniffers or other devices to try to collect information on them? Does the degree of familiarity with those people I collect information about matter, or is the act wrong in and of itself? And do my intentions justify my actions, or do the consequences of said actions justify them?

Obviously I think it's a good thing that we as a society try to discourage collecting information on people who don't want that information collected, but there is a portion of our society specifically allowed to do this: the state. What makes their status deserving of this power? Can this power be used for ill and good purposes? Is there a level of cross collection that can promote trust and collaboration between the state and its public, or even amongst the public itself? I would say that there is a level where if someone or some group knows enough about me, it gets creepy.

Anyways, lots of questions and no real answers! I'd be interested in learning more about this subject, and I apologize if I steered the convo away from sexual harassment and violation. Consent extends to all parts of our lives, but sexual consent does seem to be a bigger problem given the evidence of this post. Looking forward to learning more!

CleoTheWizard ,
@CleoTheWizard@lemmy.world avatar

I think we’ve just stumbled on an issue where the rubber meets the road as far as our philosophies about privacy and consent. I view consent as important mostly in areas that pertain to bodily autonomy right? So we give people the rights to use our likeness for profit or promotion or distribution. And what we’re giving people is a mental permission slip to utilize the idea of the body or the body itself for specific purposes.

However, I don’t think that these things really pertain to private matters. Because the consent issue only applies when there are potential effects on the other person. Like if I talk about celebrities and say that imagining a celebrity sexually does no damage because you don’t know them, I think most people would agree. And so if what we care about is harm, there is no potential for harm.

With surveillance matters, the consent does matter because we view breaching privacy as potential harm. The reason it doesn’t apply to AI nudes is that privacy is not being breached. The photos aren’t real. So it’s just a fantasy of a breach of privacy.

So for instance if you do know the person and involve them sexually without their consent, that’s blatantly wrong. But if you imagine them, that doesn’t involve them at all. Is it wrong to create material imaginations of someone sexually? I’d argue it’s only wrong if there is potential for harm and since the tech is already here, I actually view that potential for harm as decreasing in a way. The same is true nonsexually. Is it wrong to deepfake friends into viral videos and post them on twitter? Can be. Depends. But do it in private? I don’t see an issue.

The problem I see is the public stuff. People sharing it. And it’s already too late to stop most of the private stuff. Instead we should focus on stopping AI porn from being shared and posted and create higher punishments for ANYONE who does so. The impact of fake nudes and real nudes is very similar, so just take them similarly seriously.

KidnappedByKitties ,

What I find interesting is that for me personally, writing the fantasy down (rather than referring to it) is against the norm, a.k.a. weird, but not wrong.

Painting a painting of it is weird and iffy, hanging it in your home is not ok.

It's strange how it changes along that progression, but I can't rightly say why.

Max_P ,
@Max_P@lemmy.max-p.me avatar

Seen similar stuff on TikTok.

That's the big problem with ad marketplaces and automation, the ads are rarely vetted by a human, you can just give them money, upload your ad and they'll happily display it. They rely entirely on users to report them which most people don't do because they're ads and they wont take it down unless it's really bad.

alyth ,

The user reports are reviewed by the same model that screened the ad up-front so it does jack shit

Max_P ,
@Max_P@lemmy.max-p.me avatar

Actually, a good 99% of my reports end up in the video being taken down. Whether it's because of mass reports or whether they actually review it is unclear.

What's weird is the algorithm still seems to register that as engagement, so lately I've been reporting 20+ videos a day because it keeps showing them to me on my FYP. It's wild.

space ,

That's a clever way of getting people to work for them as moderators.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • [email protected]
  • kbinchat
  • All magazines
    •