'Deepfake face swap' app ZAO is making people afraid of the future

Originally published at: https://boingboing.net/2019/09/02/deepfake-face-swap-app-zao.html


Wow, the quality of the example using Dicaprio is very impressive. This is kind of scary.

This app is the best argument to convince anyone who still needed convincing to always be sceptical of what any media outlet, photo or video seems to show us.


What’s the problem here? All we need to do to ensure honesty amongst ourselves is precede each true pic we send with “Simon Shows:” and omit it when we don’t mean it.


Given that this has always been true (even pre-Photoshop photography was manipulatable in ways both subtle and gross, even just by virtue of the fact that one could take a moment out of its context), the people who needed to be convinced are often the people who want to be credulous. (Even if just for stories that confirm their prejudices.) The people who really need to learn this lesson still won’t.

I was just a minute ago seeing some obviously altered images being passed around Twitter to advance a pro-Brexit narrative. No one was much bothered they were clearly fake.

I fear the biggest change will become the ability to deny that which is real - anything that disagrees with what one wants to believe can now be easily dismissed (even if the source is credible, even if it can be demonstrated that it isn’t faked - even if it’s clearly beyond the software’s ability to fake). Reality is now fully partisan.


Exactly right.


My fear of the future pre-dates ZAO by at least two decades.


Where have you been? This has been true for almost two decades. The next great war will be over the fact that we can’t agree on basic fucking things like what a fact is anymore.


The word “fully” is deliberate here. It’s not been fully true for the last couple decades - I’ve sporadically watched Fox during that time, which was a primary driver of the dynamic in the US. It’s been somewhat gradual and there are still vestiges even now. Not for much longer.


That people consider reality negotiable isn’t half as discouraging as how much most seem to like it just fine that way and don’t see it as something to overcome on our collective march down the drain not facing facts.


I live in a blue state that is one of those ‘80% red but for the city’ - and hey - land can’t vote so yea! However I can tell you for a fact that that the ‘talking heads’ on fox are only just catching up - the real change happened with radio - Limbaugh and Hannity - you think Hannity is bad on Fox - try his radio show - the dittoheads were developing in the early 90’s - the ‘that aint true because I don’t feel like it’s right’ has been an idea percolating in the ignorant red for a long time.


This scares the shit out of me.

But if all we need to do to counteract this is teach all humans to always distrust what they see and hear—as we’ve done since approximately forever—then…

1 Like



Sure since having a darkroom and xacto blading lee Harvey Oswald’s head was something anyone could do. Even photoshop took some level of access and skill. Doing this with 2 clicks (and in MUCH more convincing fashion) is in fact orders of magnitude more dangerous and playing the ‘but it’s always been like this’ card is either disingenuous or defeatist. I agree with the ‘some people like their bubble’ sentiment but that doesn’t mean we give up on needing a shared reality if humanity’s not gonna rip itself apart.


Er… even without this tech, I was already “afraid of the future,” given all the insane fuckery that’s currently happening in the present.



Don’t fear this.

Humanity has operated just fine and built amazing things with nothing more than slow face-to-face and oral truths before and we can do so again.

1 Like

Has someone already looked at what usage rights the company whose server those pics are uploaded to retain for themselves?

Sorry, can’t really be bothered to read the whole topic, or the EULA. Too scared already.


I tend to underestimate the impact of right-wing radio, as I’ve never listened to it. I only started watching Fox after 9/11 because I heard they were selling specific narratives and wanted to see what they were saying (or not saying). Subsequently, a lot of clips of Fox get presented elsewhere, so I have some notion of what narratives they’re pushing. That doesn’t seem to happen so much with radio for some reason.

Not even (or just) that, though. The very act of taking a picture is to create a narrative that might not be true. (E.g. photographing a crowd so you can’t see how many people are in it; photographing the act of one person in a crowd, even if they’re the only person doing that particular thing) And not just taking a picture, but just presenting it - I frequently see photos of one event passed off as another, for instance. But yeah, absolutely - taking a photography, distributing a photograph and substantially altering a photograph have all gotten orders of magnitude easier now, changing the dynamics.

Oh absolutely. I just don’t know how to get (back) there, given where we are now.

Unfortunately that also involved bloody tribalism (and/or bloody conquerors of empires, held together with more blood).


I was really hoping someone would transpose Trump’s head over Thanos’ during the final scene of Endgame

1 Like

This emphasises what’s been obvious for a while.

Once it took special support services to lie in public. The support infrastructures of politics or journalism for example.

Now it’s democratised.

There’s an awful lot that probably won’t stand up in court, and whole new lines of defense. Be interesting to see where that goes.

(Like being tricked into making porn movies, the possible defenses and retaliations …)