00:00Dear Fellow Scholars, this is Two Minute
Papers with Dr. Károly Zsolnai-Fehér.
00:04Today we are not going to marvel at neural
network-based AI systems, but we are going
00:10to defeat them. What? How? Well, of course,
not with brute force, but with trickery.
00:18You see, this is what we call an adversarial
attack. And few know about it, but many modern
00:24AI techniques are quite vulnerable against
them. This is the “You Shall Not Pass” game,
00:30where the red agent is trying to hold
back the blue character and not let it
00:35cross the line. Here you see two regular
AIs duking it out, sometimes the red wins,
00:41sometimes the blue is able to get through. Nothing
too crazy here. This is the reference case which
00:48is somewhat well balanced. Now, look closely,
because here comes the hacker adversarial agent.
00:56Ha! Yes, you are seeing correctly, this chap
it doing nothing. Absolutely nothing. But it
01:04is doing nothing in a way that reprograms
its opponent to make mistakes and behave
01:10close to a completely randomly acting
agent! This paper was absolute insanity.
01:16A different adversarial attack paper showed an
interesting case where we were able to take an
01:23image of a horse. The attacked neural network
indeed recognized that this was a horse. However,
01:29when changing just one pixel, an otherwise quite
competent AI now thinks that this is a frog. Note
01:38that this does not mean that we can just change
any pixel anywhere. This is a sophisticated attack
01:45that knows who it is attacking and how this
one pixel difference will reprogram its brain.
01:52Similar attacks also exist that do this
with a little more flavor. Look. This
01:58is a bus. And this is noise. And when we
add this together, do we get a bus+noise?
02:05Nope, what we get is an
ostrich. So does the AI think.
02:10Now, let’s have a look at
new papers with new attacks,
02:14first against AIs that can
play Go. Now, wait a second,
02:19what is really new here? We have heard
players reporting before that with AlphaZero,
02:25they have experienced little hiccups where the
AI made suboptimal moves in an otherwise really
02:32well played game. Why is that news? Why write
a paper about this? How is this one different?
02:38What makes this attack interesting
is that this is not a one-off fluke,
02:43this is an AI that finds systematic flaws in
other neural network-based systems. This means
02:50that they are not only able to exploit
their weaknesses and win against them,
02:55but they can do it on a consistent basis. In
other words, they can win over and over again.
03:02In this case, hold on to your papers, because
this paper describes an attack that is able to
03:08defeat KataGo in 97% of the games. That number is
unreal. Let me explain why. From what I have seen,
03:17KataGo seems to be even stronger than AlphaZero
and AlphaGo Zero, DeepMind’s legendary systems
03:25that are able to beat the best human players
in the world. They authors note that it works
03:31on many AlphaGo-based systems, and it
likely works on AlphaGo variants too.
03:36What makes it even more impressive is that
this adversary was trained from scratch and
03:42did not use any human knowledge.
It found this out all by itself.
03:48And finally, here is another attack against
a competent image recognition AI. When
03:55showing it the Starry Night painting
by van Gogh, which it will, of course,
03:59recognize as shown the by red frame. However, as
we start adding carefully crafted noise to it,
04:06nothing happens. But wait just a little
more, as we continue the process,
04:11bam! Now this noise looks nothing like Starry
Night, but not according to this AI. It would
04:19swear that this is exactly the Starry Night
painting. And this works for other examples too.
04:25The goal of all this is to show you some
of the weaknesses of recent AI systems.
04:30This is a really interesting new field
where we have these powerful new tools,
04:35but this also means that they
have their own limitations too,
04:39and sometimes, it is not at all obvious
what they are. More research is required.
04:45Now, one word about the LK99, the
room-temperature superconductor
04:50project. I see that many of you Fellow
Scholars would love an episode on it,
04:55however, I do not have the required knowledge
to comment on it. Obviously it would be a very
05:01good thing for views, but that doesn’t
matter. What matters is that you Fellow
05:06Scholars get the quality of videos that
you expect here. That is what matters.
05:55Thanks for watching and for your generous
support, and I'll see you next time!
