I want you to imagine walking into a
room control room with a bunch of people
100 people hunched over at depths with
little dials and that that control room
will shape the thoughts and feelings of
a billion people this might sound like
science fiction but this actually exists
right now today I know because I used to
be in one of those control rooms I was a
design ethicist at Google where I
studied how do you f eclis scare
people's thoughts because we don't talk
about is a handful of people working at
a handful of technology companies
through their choices will fear what a
billion people are thinking today
because when you pull out your phone and
they design how this works or what's on
the slip on the feet its scheduling
little blocks of time in our mind if you
see a notification it scheduled you to
have thoughts that maybe you didn't
intend to have if you swipe over that
notification it's scheduled you into
spending a little bit of time getting
sucked into something that maybe you
didn't intend to get sucked into when we
talk about technology we tend to talk
about it as this blue sky opportunity
could go any direction and I want to get
serious for a moment and tell you why
it's going in a very specific direction
because it's not evolving randomly
there's a hidden goal driving the
direction of all of the technology we
make and that goal is the race for our
attention because every news site Ted
elections politicians game even
meditation apps have to compete for one
thing which is our attention and there's
only so much of them and the best way to
get people's attention is to know how
someone's mind works
and there's a whole bunch of persuasive
techniques that I learned in college of
that lab called the persuasive
technology lab to get people's attention
a simple example of YouTube YouTube
wants to maximize how much time you
spend and so what do they do they
autoplay the next video and let's say
that works really well they're getting a
little bit more of people type well if
you're netflix you look at that and say
well that's shrinking my market share so
I'm going to auto play the next episode
but then if you're Facebook you say well
no that's shrinking all of my marketers
that now I have to autoplay all the
videos in the newsfeed before waiting
for you to click Play so the internet is
not evolving at random the reason it
feels like it's sucking us in the way it
is is because of this race for attention
we know where this is going technology
is not neutral and it becomes this race
to the bottom of the brainstem of who
can go lower to get it let me give you
an example of snapchat if you didn't
know snapchat is the number one way that
teenagers in the United States
communicate so if you're like me and you
use text messages to communicate
snapchat is that for teenagers and
there's like a hundred million of them
that use it and they invented a feature
called snap streaks which shows the
number of days in a row that two people
have communicated with each other in
other words what they just did is they
gave to people something they don't want
to lose because if you're a teenager you
have a hundred and fifty days in a row
you don't want that to go away and so
think of the little blocks of time that
that schedules in kids minds this isn't
theoretical when kids go on vacation
it's been shown they give their
passwords up to up to five other friends
to keep their snap streak going even
when they can't do it and they have like
thirty of these things and so they have
to get through photos taking photos of
just pictures or walls or ceiling just
to get through their day so it's not
even like they're having real
conversations we have a temptation to
think about this as oh they're just
using you know snapchat the way we used
to gossip on the telephone it's probably
okay but what dismisses is that in the
1970s when you were
gossiping on the telephone there wasn't
a hundred engineers on the other side of
the screen who knew exactly how your
psychology work and orchestrated you
into a double-blind with each other now
if this is making you feel a little bit
of outrage notice that that thought just
comes over you outrage is a really good
way also of getting your attention
because we don't choose outrage it
happens to us and if you're the Facebook
newsfeed whether you'd want to or not
you actually benefit when there's
outrage because outrange doesn't just
schedule a reaction in an emotional time
space for you we want to share that
outrage with other people so we want to
hit share and say can you believe the
thing that they said and so outrage
works really well at getting attention
such that if facebook had a choice
between showing you the outrage speed
and a calm newsfeed they would want to
show you the outrage speed not because
someone consciously shows that but
because that worked better at getting
your attention and the newsfeed control
room is run by is not accountable to us
it's only accountable to maximizing
attention it's also accountable because
of the business model of advertising for
anybody who can pay the most actually
walk into the control room and say that
group over there I want to schedule
these thought into their minds so you
can target you can precisely target Ally
directly to the people who are most
susceptible and because this is
profitable it's only going to get worse
so I'm here today because the costs are
so obvious I don't know a more urgent
problem than this because this problem
is underneath all other problems it's
not just taking away our agency to spend
our attention and live the lives that we
want it's changing the notion the way we
have our conversations it's changing our
democracy and it's changing our ability
to have the conversations and
relationships you want with each other
and we and it affects everyone because a
billion people have
views in their pocket so how do we fix
this we need to make three radical
changes to technology into our society
the first is we need to acknowledge that
we are persuadable once you start
understanding that your mind can be
scheduled into having little thoughts of
little blocks of time that you didn't
choose when we want to use that
understanding and protect against the
way that that happens I think we need to
see ourselves fundamentally in a new way
it's almost like a new period of human
history like the Enlightenment but
almost the kind of self-aware and
lighting this that we can be persuaded
and there might be something we want to
protect the second is we need new models
and accountability systems so that as
the world gets better and more and more
persuasive over time because it's only
going to get more persuasive that the
people in those control rooms are
accountable and transparent to what we
want the only form of ethical persuasion
that exists is when the goals of the
persuader are aligned with the goals of
the persuading and that involves
questions big things like the business
model of advertising lastly we need to
design Renaissance because once you have
this view of human nature that you can
steer the timelines of a billion people
just imagine there's people who have
some desire about what they want to do
and what they want to be thinking what
they want to be feeling and how they
want to be informed and we're all just
tugged into these other directions and
you have a billion people just tugged
into all these different directions
we'll imagine an entire design
Renaissance that try to orchestrate the
exact and most empowering time
well-spent way for those timelines to
happen and that would involve two things
one would be protecting against the
timelines that we don't want to be
experiencing the thoughts that we that
we wouldn't want to be happening from
that ding happens not having the ding
that sends us away and the second would
be empowering us to live out the
timeline that we want so let me give you
a concrete example today let's say your
friend cancels dinner on you and you are
feeling a little bit lonely and so what
do you do in that moment you
open up Facebook's and in that moment
the designers in the control room want
to schedule exactly one thing which is
to maximize how much time you spend on
the screen now instead imagine if those
designers created a different time line
that was the easiest way using all of
their data to actually help you get out
with the people that you care about
think just imagine how alleviating all
loneliness in society if that was the
time line that Facebook wanted to make
possible for people or imagine a
different conversation where let's say
you wanted to post some things super
controversial on Facebook which is a
really important thing to diligent to
talk about controversial topics and
right now when there's that big comment
box
it's almost asking you do you want to
what what key do you want to type in
other words it's scheduling a little
timeline of things you're going to
continue to do on the screen and imagine
instead there is another button there
saying well what would the most time
we'll spend for you and you click host a
dinner and right there underneath the
item it said who wants to RSVP for the
dinner and so you'd still have a
conversation about something
controversial but you'd be having it in
the most empowering place on your
timeline which would be at home that
night with a bunch of friends over to
talk about it so imagine we're running
like a Find and Replace on all of the
timelines that are currently steering us
towards more and more screen time
persuasively and replacing all of those
timelines with what do we want in our
lives it doesn't have to be this way
instead of handicapping our attention
imagine if we used all of this data and
all this power and this new view of
human nature to give us a superhuman
ability to focus and if superhuman
ability to put our attention what we
cared about and a superhuman ability to
have the conversations that we need to
have for democracy the most complex
challenges in the world require not just
us use our attention individually they
require us to use our attention and
coordinate it together
climate change is going to require that
a lot of people are being
to coordinate their attention the most
empowering way together and imagine
being creating a superhuman ability to
do that sometimes the world's most
pressing and important problems are not
these hypothetical future things that we
could create in the future sometimes the
most pressing problems are the ones that
are right underneath our noses the
things that are already directing a
billion people's thoughts and maybe
instead of getting excited about the new
augmented reality and virtual reality
and these cool things that could happen
which are going to be susceptible to the
same rage for attention we could fix the
rates for attention on the thing that's
already in a billion people's pockets
maybe instead of getting excited about
the most exciting new cool fancy
education apps we could fix the way kids
minds are getting manipulated into
sending empty messages back and forth
maybe instead of worrying about
hypothetical future run away artificial
intelligences that are maximizing for
one goal we could solve the runaway
artificial intelligence that already
exists right now which are these news
feeds maximizing for one thing it's
almost like instead of running a way to
you know colonize new planets we could
fix the one that we're already on
[Applause]
solving this problem is critical
infrastructure for solving every other
problem there's nothing in your life or
in our collective problems that do not
require our ability to put our attention
where we care about at the end of our
lives all we have is our attention and
our time what will be time well spent
for ours thank
[Applause]
- son thank you hey that paragraph yes
that's all thank you I know we asked you
to this talk I'm pretty short notice and
you better you know quite a stressful
week getting this thing together so
maybe the date thank you um some people
listening might say well look what what
you complain about is addiction and all
these people doing this stuff for them
it's actually interesting all these
design decisions have built use the
content this is fantastically
interesting well it's more interesting
than it ever has been what's possibly
wrong with that well I think it's really
interesting it's like one way to see
this is if you're just faced by YouTube
for example you want to always show them
more interesting next video you want to
be better and better it's just in that
next video but even keep it propose the
perfect next video that everyone would
want to watch it would just be better
and better at keeping you hooked on the
screen so what's missing in that
equation is figuring out what's our
boundaries would be right you would want
YouTube to know something about say
falling asleep the CEO of Netflix
recently said that our biggest
competitor is our Facebook I mean
Facebook YouTube and sleep right and so
we what we need to recognize is that the
human architecture is limited and that
we have certain boundaries or or
dimensions of our lives that we want to
be honored and respected and the
technology can help do that I mean could
you make the case that you know part of
the problem here is we've got a nice
model of human human nature so much of
this is justified in terms of human
preference we're only we've got these
algorithms is an amazing job of
optimizing the human preference but
which preference there's there's the
preferences of things that we already
care about when we think about them
versus the preferences of what we just
instinctively click on it like if you
could implant that more nuanced view of
human nature and every designer would
that be a step forward absolutely I mean
I think right now is if all of our
technology is basically only asking our
lizard brain what's the best way to just
impulsively get you to the next tiniest
thing with your time instead of asking
you in your life what would be most time
well-spent for you what would be the
perfect timeline that might include
something later will be time well spent
for you here at Ted in your last day
here so Facebook and Google and everyone
said to us first up hey would you like
us to optimize your reflective brain or
your lizard brain you choose right I
will go
yes um when you said you said persuade
ability that's an interesting words
because this to me there's two different
types of persuade ability there's the
persuade ability that we're trying right
now of reason and thinking and making an
argument but I think you're almost
talking about a different kind of more
visceral to persuade ability of being
persuaded without even knowing that
you're thinking exactly yeah when I the
reason I care about this problem so much
is I studied at a lab called the
persuasive technology lab at Stanford
that taught people exactly these
techniques there's conferences and
workshops that teach people all these
covert ways of getting people's
attention to orchestrating people's
lives and it's because most people don't
know that that exists that this
conversation is so important son you and
I you know we both know 70 people from
all these companies they're actually
many here in the room and I don't know
about you but my experience of them is
that there is no shortage of good intent
people people want a better world that
they're actually they really want it and
and I don't think anything you're saying
is you're saying is like evil people
it's the system does these unintended
consequences that really got out of
context raised for attention yeah it's
the classic rates to the bottom that
when you have to get attention and it's
so tense the only way to get more is to
go lower on the brainstem you know to go
lower into outrage go lower into a
motion to go lower into the lizard brain
well thank you so much for helping us
all get a little bit wiser about this
first on hands thank you thank you