Transcript
30:55
please do contact them directly rather than introduce or interrupt what’s going on on the stage. Um it will be kind of
31:02
uh helpful for everyone um so we can proceed through the evening. Um and also I was uh reminded just to note everyone
31:09
that um note to everyone that uh there is a zero tolerance um for personal attacks or any sort of discriminatory
31:15
behavior. Um we are very aware of that the content of uh tonight’s evening can
31:22
bring up a lot of um feelings and strong opinions. So please do reach out to any of the people in the awareness team um
31:28
if anything comes up. Um, also, uh, there’s going to be a lot of, uh,
31:34
incredible content. Um, we will be taking questions at the very end. So,
31:39
you know, take a note on your phone if, you know, you have any questions or anything you want to hear more about. Um, there’s a full half an hour at the
31:46
very end um, to ask any of our speakers any questions whatsoever. [snorts] Um, so on that note, um, an introduction
31:53
to myself. My name is Gia Clara Brekie. I’ve been working um kind of I suppose in the area of technology and ethics for
32:00
at least 15 if not 20 or more years um and that kind of cuts across academia um
32:07
policy also the arts um both kind of in theory and in practice right now working
32:13
very much in that theme very much in practice um as the chief strategy officer at a software company called NIM
32:19
where we produce um privacy software um and in fact that was the first time where I at Snowden. We had a launch
32:26
event with him about 5 years ago or so. Um I also currently run a a discussion
32:32
series at an institute called Ibraz in London on the topic of technology and power um where we um also deal with a
32:39
lot of the subjects like we’re going to be discussing tonight. Um so this event is hosted by ETH Zurich Ethics Forum.
32:46
Um, and before introducing the incredible speakers, um, by way of
32:51
setting that context a little bit further, I would like to start off with a short introduction to the type of
32:57
ethics that we’ll be discussing discussing today. Um, and uh, I won’t hesitate to say that it is a very real
33:04
form of ethics. It’s a very practical form of ethics. It’s something that the speakers are doing in practice. um as
33:11
opposed to a hypothetical or let’s call it a corporately convenient one. Um this
33:17
is real life. Um there’s real stakes and tonight we’re going to be grappling with the complicity of the tech sector um uh
33:25
causing harms at an infrastructural scale. Um and the agency that each of us have to change that. Um so the last
33:33
month or so um we’ve seen of course you know a lot of AI in the news headlines a
33:39
lot of panic um not just from the kind of usual tech pundits let’s say but also
33:44
among ordinary politicians and regular people um and the latest wave of concern
33:50
follows you know especially kind of the three or three big media stories one that broke just today um the hack of
33:56
open AI and hugging face um uh this the machine learning um platform And of
34:02
course, the story of anthropic employee Jacob Coxen um who quit his job because he says, you know, AI companies are
34:08
gambling with our lives and the people developing AI sincerely believe it could kill us all before the end of the
34:13
decade. Um I don’t know if anyone saw the headline today, but Open AI also in
34:18
fact um hacked uh the Australian government health website. Um, now it’s
34:24
not to say that I don’t think that any of these, you know, dangers are, you know, real, um, or that, you know,
34:30
agents doing unpredictable things is real. Um, but I would like for us given
34:36
the kind of subject matter of tonight to question a little bit the framing of those stories. Um, so to kind of get us
34:43
warmed up uh with a bit of ethics or critical thinking 101, um, let’s talk a
34:48
bit about the framing. Um, so when we’re looking at the specific story or the framing of Jacob Coxin’s, you know,
34:54
line, um, that, uh, we sincerely believe AI is going to kill us all before the end of the decade. Um, you know, I would
35:01
like to first focus on, um, you know, the this this theory that through recursive learning, we’re going to
35:07
somehow arrive at a kind of super intelligence that’s going to kill all of humanity, right? Um so first of all if
35:13
we look at the frame the frame you know and this cuts across not just media headlines but also the technical reports
35:19
of the company companies themselves the frame sits tightly on the agents
35:25
themselves. It’s not looking at all the decision- making all the hypotheses the
35:31
ideas the funding the you know corporate strategy everything that led up to this
35:36
particular moment. um but really just on what are the agents doing, right? Um and
35:42
this limited frame means that agency so to speak, the agency to do things is
35:47
limited to the agents. Um and it also kind of clearly sits within the interests of uh the corporate narrative
35:54
who are in fact trying to sell a technology that proclaims to be you know super intelligent. So the more that you
36:00
know we have these stories that feed into that narrative um the the more it benefits the the kind of product that
36:07
they’re trying to sell. Um and of course it also helps deflect uh you know accountability off the the corporates
36:13
themselves. Um and so uh secondly I would also note that you know when we’re
36:19
talking about this idea of a kind of coming super intelligence um it also shifts focus from the real harms that
36:26
are happening here and now um to a kind of hypothetical danger right
36:31
[snorts] um and you know there’s especially two main talking points around this hypothetical danger uh it is that this
36:37
super intelligence might be out of alignment with humanity um or that it will pose an existential threat to
36:44
humanity. [snorts] Um if we take a slightly expanded focus I think it’s very clear that um AI is
36:51
already out of alignment with very very many people in the world today um and
36:57
out of alignment of their best interests um and in fact are causing existential harms um to people already um and you
37:05
know we can note that with the involvement of the AI industry with the military in the United States um and of
37:12
course the military in Israel Um and we’ve seen the things that have happened in Iran where 120 school children were
37:18
killed and of course um more than 20,000 children and you know hundreds more than 100,000 people killed in the genocide in
37:24
Gaza. So we’re already in a situation where there’s existential threat to certain people. We’re already in in a
37:30
situation where um uh you know the systems that we’re building is out of alignment with the welfare of humanity
37:37
as a whole. Um but the and the issue never was and never will be that machines per se are going to be against
37:44
people per se, but rather that some people and their companies, their
37:49
investors and so on are putting their effort and their talent towards optimizing machines to dominate and kill
37:57
other people. You know, for example, shortening the kill chain. Um, now Cox
38:03
is not, you know, by far not the first uh employee to speak up um and lose their job over AI developments and the
38:09
harms um that it’s causing. We have another person here that’s going to speak in a minute. Um, and I also want
38:16
to kind of point towards um uh point towards uh the work of Timnit Gabbru um
38:23
especially if you want to kind of read um more intelligent commentary about the recent exploits that have been going on.
38:29
Um Tim Nich was uh the technical lead of Google’s ethical AI research team. Um published the famous stocastic papers
38:36
paper alongside um sarcastic parrots paper alongside some other excellent authors. Um and was in fact fired,
38:42
right? And if you look at the history of AI, it is riddled with people who have been silenced and fired for speaking out
38:47
against um this uh agenda of building a kind of you know super large model at
38:53
the cost of everything else. Um so this is what we mean when we talk about
38:59
freedom of speech and this is indeed the very real ethical uh context that we’re in um today. Um and so to round up uh
39:08
kind of zoom out a little bit again um when we’re talking about technology we’re talking about um developing ways
39:13
to automate, distribute, speed up and multiply processes. And when we’re
39:19
talking about ethics or the question that ethics poses of that is in fact whose needs and what processes we’re
39:25
speeding up. Is it in fact the killchain or is it something that perhaps you know benefits life and and people um what
39:32
exactly is being made more efficient at what cost and to who [snorts] um and how
39:37
indeed our tools um change and affect our relationships to one another as
39:42
people living together on this planet. Um so now that we’ve cleared the air of
39:48
uh you know any ethical confusions or confusions around ethics that we’re going to be discussing today um I would
39:54
like to turn to our amazing speakers. Um the title of today’s event is of course
40:01
um AI surveillance and freedom of speech. Um that’s the subject matter but I would like to reveal already now that
40:07
um the real theme of tonight um is in fact agency and moral integrity. Um that
40:14
is the core and that’s the ambition of this event. Um for us all to re re-reme remember that it is completely normal
40:20
and it’s completely innate to all of us to have a moral sense of what’s right and wrong to have a compass. Um and in
40:27
fact that we all in different ways um do have the agency to act on that. Um and that’s very much embodied by um the
40:33
incredible lineup today. [snorts] Um so to start off I would like to introduce Margaret Meyer um who’s a freelance
40:40
investigative journalist um who worked with the Swiss news organization Republic um and the research collective
40:47
web uh to publish a series of articles in 2025 about how Palunteer a US spytech
40:54
company um had been failing to win uh contracts um in Switzerland with the
40:59
Swiss government. [snorts] Now the Swiss authorities cited uh national security as one of the reasons why um they
41:04
refused to adopt Palanteer software. Um and this series of articles had a huge impact all across Europe because in fact
41:12
Palanteer does have a lot of contracts with authorities all across Europe um ranging from local authorities to
41:18
national um and ranging from policing to electric grids um health uh and so on.
41:25
Um and uh let’s say after that art article series it gave a real boost to campaigners who have been trying to
41:31
raise issues with you know the the nature of Palunteer in the company for for quite a long time. Um and we’re
41:37
starting to see some significant changes. Um uh there’s five police forces just in the past week I think in
41:43
the UK that have dropped their contracts with Palunteer. Um the London mayor has blocked a contract with the Met Police.
41:50
Um and uh we see the NHS uh 50 million uh no $330 million uh dollar contract
41:56
also under review. We’ve got the French prime minister who’s looking at their national security contract reviewing
42:02
that. Same goes in Germany and so on and so on. So um really a kind of change in mood there. Um Palanter tried to sue
42:10
Republic um also also worth noting. Um they failed. I just heard uh actually earlier today that um Republic won the
42:17
court case which is fantastic. Um so I’m going to give the floor now to Margaret who’s going to speak um a little bit
42:23
about the case but mostly um about uh AI um and its use on vulnerable
42:29
populations, how this is now becoming mainstream practice and um and the agency of journalists and tech students
42:36
um and workers to do something about it. Um and yeah, so reminder, note down any
42:41
questions you have. We’ll take them at the very end after all the speakers have had their turn. Margaret,
42:48
applause
42:59
thank you very much uh Ja for the nice introduction. Um I will move to my Yes,
43:08
there’s my slide. um investigating AI as a journalist. I’m as Ja said I’m a
43:13
freelance investigative journalist based in Zurich. I have uh worked a lot on
43:19
various topics on migration, on human rights, on politics, on organized crime
43:24
and in the last few years uh shifted a lot of my work towards defense, security, AI and tech. Um and as Jia
43:32
mentioned before uh the palunteer uh investigation which of course you all are here for um was
43:40
also a collaboration. So I mostly do collaborations in the last few years also crossber collaborations with
43:47
colleagues from different countries. I really think that collaborations are the future of journalism. Um so today I’m
43:54
here to talk a bit about of course that case but I will give a short overview of
44:00
three different investigations I’ve worked on with different colleagues in the realm of AI and tech and I will also
44:08
try to find some communalities and common factors that we’ve come across in these different investigations.
44:16
So the first one um was home of drones in 2023
44:21
together with my colleague Arian Luti. Um we were working together uh as colleagues and um stumbled across um
44:29
this website called home of drones because Switzerland was trying to push its um location as a location where
44:38
drones can be um you know uh developed and it was this kind of you know drones
44:44
for good, AI for good um you know ctography in in agriculture,
44:51
search and rescue that was kind of the narrative. So we tried to dig a bit deeper. Um and we sort of dived into
44:59
this ecosystem of uh research institutions, universities in Switzerland and startups and companies.
45:05
Um we very quickly found that um basic research that was done in Zurich was
45:13
also the kind of technology that was used in drones, lethal autonomous
45:19
weapons. Um and basically one of the things of our story that we published
45:24
with SUSB Blick um a large uh large publication in Switzerland was that we
45:32
broke the story that um a group of scientists and researchers was actually
45:38
asking the Swiss government for regulation because this threat of the killer robots was kind of looming. Um 3
45:47
years later, 2026, we’re here. There have been the first um proven cases of
45:54
actually killer robots in Ukraine by Russ used by Russia um where there
45:59
allegedly was no human in the loop anymore at all, not even a little bit at the end. Um we’ve seen it in Gaza. We’ll
46:06
talk about Gaza a bit later. So this is uh kind of three years back and now
46:11
we’re suddenly in the full-blown topic. We didn’t think it would go that fast.
46:19
The next investigation I would like to talk about is um in the realm of AI and
46:25
migration. So basically with this international team together with Yorgos Christidas, Danna Maraya, Florian
46:32
Schmidtz and Katie Fallon, we um did two investigations. Both of them were one
46:37
year long across nine countries um in Europe. And um we talked to a lot of
46:43
sources. We talked to a lot of people working inside border control um in in
46:49
police. We talked to people um on the move. And in the first investigation
46:55
which uh was called invisible walls, we tried to find out what kind of tech is
47:01
used in um border control and migration. So we actually were able to trace from
47:09
the Evos border between Turkey and Greece all the way through Europe to the English Channel the different type of
47:16
tech that was used. It’s a topic that keeps on giving. So we
47:22
decided to do a second investigation with the group which was um published last year. We tried to follow the
47:28
question who actually really benefits from this push in a in of AI in
47:33
migration. What we saw is that um basically it’s a massive shift to
47:38
militarization of any kind of migration tech um against people on the move who
47:44
are a super vulnerable group. Um we saw that basically the benefits are um big
47:50
defense contractors like Shield AI for example um research institutes um and
47:57
also of course the whole you know startup landscape that were
48:02
like subcontractors to to the big contractors and research institutes. I
48:07
focused on the Swiss part of the investigation and uh highlighted one example of IDOP uh the private um
48:16
private research institute but of national uh importance that is also partly publicly funded who got a uh
48:23
Frontex grants to Frontex is the um European Agency for Border Control um
48:30
that basically enabled um a large simulation project on um biometrics with
48:37
gate recognition. So basically not just faces but also how to recognize an
48:42
individual based on how they walk. Um
48:48
we’re working on a third investigation, the same team. Um you’ll hear about that probably later this year when we’re
48:54
publishing. Then of course, Palanteer um a project
49:01
that was collaborative but within Switzerland together with my colleagues Adrienne Fa, Jennifer Steiner, Lawrence
49:08
Nagali and Balti from Republic Magazine and uh the
49:13
research collective VAF. Um we had had Palunteer on our radar for quite a long
49:20
time. Um and then we learned that a few years ago they had set up shop in Switzerland as well. So we asked
49:27
ourselves what are they doing in Switzerland and um do Swiss authorities also have contracts with them?
49:35
So we uh investigated for a long time and actually got a lot of documents. We
49:41
had one of the main parts of the investigation was an extensive um FOYA
49:48
strategy. FOYA is freedom of information access request. So basically Switzerland
49:53
has a law like many other countries that’s any citizen has the right to know
49:58
what’s going on within authorities. So authorities balance that of course with a right to privacy etc. But basically
50:05
you can request um ad documents from the government um and from administration.
50:14
So we foyaded as we say um dozens of Swiss authorities and we found that
50:20
Palanteer had over several years tried to sell their products to various authorities but they couldn’t succeed.
50:29
So basically it was a thanks but no thanks. Um we also visited their Zurich offices
50:36
for an interview later in the investigation. One of the government authorities that we also foyed was the
50:42
Swiss army and there we found they had tried to sell to the Swiss army a product. Um
50:49
one of the documents we got was a report. It was an assessment of um the
50:55
software. A very normal thing like you know institutions basically they get a
51:00
new software or they’re thinking of getting a new software and they take a look at it and say like is it suitable for us? Is it reliable? Does it meet our
51:07
needs? uh what are the capabilities? But then there was also of course a small
51:14
paragraph which in the end was crucial for the whole investigation that was um
51:19
that stated the Swiss army stated that because of the proximity of Palunteer to
51:25
US intelligence it was not suitable and they stated national security concerns
51:31
because of course uh armies don’t necessarily want their own intelligence going to uh other intelligence uh
51:38
organizations. So that really was the crucial part of the investigation. Uh we published last
51:44
year in December in 2025 and it relatively fast got picked up by some um
51:52
some other outlets mostly in Germany. It started in Germany then the Guardian picked it up in the UK and suddenly our
51:58
investigation was quoted in uh the UK parliament as well and we were like okay
52:04
um nice. we always want impact and there finally you get it and then like woof.
52:09
Um yeah, so basically people were starting to pick
52:14
up that very small detail of the investigation saying like if it’s not good enough for the Swiss army, why is
52:21
it good enough for us? Well, Palanteer didn’t like that. I think they mostly didn’t like the not
52:28
good enough part. Um because you know they’ve been called evil and etc and been criticized which we didn’t do. We
52:34
didn’t call them evil. We just showed that they failed to sell. Um,
52:42
so they requested uh corrections to the article. I mentioned before that we had actually in uh done an interview with
52:50
them in their offices in Zurich. They had given like the green light to all the quotes we used um as is standard
52:57
journalistic practice. Like we didn’t just you know meet them and and sneak background information into an article.
53:04
So we really made sure that this was really really correct. Um but so they asked for corrections a whole list of uh
53:11
over 30 corrections. Um and they wanted a counter statement.
53:18
Now under Swiss law that is uh standard practice. I mean it’s it’s nothing
53:23
unusual that the subject of an article has the right to ask for you know
53:29
corrections or their point of view. And the idea is that the reader can um
53:37
make up their own mind, right? But the law basically only applies to facts,
53:44
right? So our facts were safe. But of course, as journalists do, we do
53:50
uh assessments of things. So basically, we have opinions that are based on the interpretation of these facts.
53:57
Um they they didn’t like that because we said a few things that they contested.
54:04
Um so they decided to take Republic to court, not us individually
54:10
but uh the magazine asking for an extensive list. So it was
54:16
in the end it was like over a 100 points which is longer than our original article was. Um it went back and forth
54:24
five for for five months. five months. Um, you know, there’s different rounds like we had to answer to all their
54:31
points, they had to answer to ours, the court was sending it back and forth. Uh, it took a lot of time uh and a lot of
54:39
work on our side. Um, but finally after 5 months the um the court decided and we
54:47
won 95%. We had to like change one small thing. Um but some of the main things
54:54
that we were allowed to continue to say was that they do surveillance technology
55:02
which is an interpretation of their actual like technical stuff that is not technically surveillance but it works as
55:10
surveillance technology. So other media also Spanish media now has written uh surveillance technology. So basically we
55:16
set a bit of a precedent. Uh we also were allowed to write uh lethal weapon
55:22
that their products serve as lethal weapons. They tried to um uh like not
55:28
allow us to to write that even though um some of the executives had actually bragged with that that you know our
55:37
products kill people. Yeah. Okay. Um time is running out so I’m going to
55:43
try to found find like um the commonalities uh around over these three
55:48
um investigations. So AI what we see it’s being um used like in the
55:55
development chain of of AI technology um vulnerable populations are used as a
56:02
testing ground. So migration in migration uh people on the move refugees
56:07
civilians in war um we’ve seen like ICE I mean Palanteer has a contract with ICE
56:14
in the US and why is that the case well of course there’s a lack of transparency
56:20
there are often legal exemptions for example um in migration in border
56:26
control um there exemptions for national security reasons right so that’s exempt
56:33
from uh some safeguards. There’s a bit of a gray area. Um there’s often not
56:39
much public interest because it’s marginalized populations. Um what we can see is an absolute trend
56:46
to militarization and it really is a playing ground and a
56:51
testing ground um and a development ground. So basically optimization ground for companies and research in research
56:59
institutes to scale up. there are a lot of research uh exemptions as well.
57:04
So basically they can optimize their products and then sell it to a bigger market.
57:10
So we see the shift from vulnerable populations then into a larger market
57:15
into big defense uh contracts into uh
57:21
general population like yeah
57:27
what I find really important is to look behind the narrative. I mentioned
57:32
before, uh, companies are called evil, but they’re often not called, uh, like
57:38
not that great. Um, when it comes to quality, what we see is a tech
57:43
solutionism. This is especially referring to migration. Um, there’s basically a problem, a political
57:50
problem. This is not not my assessment of it, but like people want solutions
57:57
for certain issues and basically we’re throwing tech at it, right? doesn’t change anything about why people migrate
58:04
for example. Um a lot of the things we found was wording issues. So for example
58:10
a lot of um use in in border areas was um classified as pilots or tests but
58:18
they were actually being used in real life uh situations. And that also means
58:23
with real life consequences for the people you know they be they they’re using it on. Um, also like hallucinate.
58:32
That’s a bit of a different uh label, but I find it very interesting because every every other industry you would say if your product makes a mistake, you
58:38
would call it a mistake, but here it’s kind of hallucination. Um, I know the
58:44
like more technical scientists among you will probably say like ah but
58:50
so what we see in general is that pilots and deployments blur the lines between research and operations.
58:56
um and that the private sector when it comes to lobbying uh increasingly dictates the military and security
59:03
demands and not the other way around instead of responding to a defined public need. Um basically they say this
59:11
is your need and we have a solution and research exemptions and that might be more interesting to you in your daily
59:18
work um that are made to keep the information out of public reach.
59:26
How to get behind the narrative uh as a journalist speaking uh stick to accuracy, avoid speculation, follow the
59:33
function and not the form. Don’t trust what it’s labeled as, but see what it does. Um, really understand the
59:40
function. What does it do? Why is it being done? Who benefits? And who is
59:45
affected? And why and how.
59:53
How do we demystify AI? I think we’ll talk a bit more about it later. Um, in
59:58
the end, AI companies are just corporations and they should be treated as such. they have corporate
1:00:05
responsibilities. There’s also human responsibilities of the builders and the users. Um we need accountability on the
1:00:14
regulatory side which is very slow. Um but also on the individual side um each
1:00:21
one of us has a moral agency be we builders be mainly be builders or users.
1:00:27
So briefly what can I do? Really try to understand what is really happening. Um,
1:00:33
always keep pushing for an ethical debate at work, at uni, with friends. Um, really ask like, okay, which
1:00:40
guidelines, safeguards, ethical principles are we working with here? Um, and I think the most important thing
1:00:47
is strengthen and defend uh basic human rights, uh, strengthen the freedom of
1:00:53
the press, support independent journalism, uh, and really emphasize the importance of the rule of law. And when
1:01:00
it starts to get uh you know decrease really push against that and most
1:01:06
importantly as with everything center humans I think that’s the crucial bit I
1:01:12
would say and if you want to get in touch that’s my signal. Thank you very much. [applause]
1:01:33
Right. So the next speaker is Vikram Subramanan um who is an exg Google
1:01:39
employee and co-founder of Notech for Pythu. Um now I have heard that that’s kind of
1:01:46
renaming that group is renaming itself to Orca. Um but uh yeah, V kernels be
1:01:52
we’ll be speaking a little bit more about um kind of Google as a company and
1:01:57
what they’ve been up to which um is a kind of um interesting contrast to
1:02:02
Palunteer. I mean Palanteer likes to present itself as an evil company, right? Um Google on the other hand has
1:02:08
for a long time had you know very these very kind of lofty um ambitions of organizing the knowledge of the world
1:02:14
and don’t be evil and so on. Um but when it comes to the topic of surveillance um
1:02:21
Google is probably the one that the single company that really changed the game um and moved the kind of internet
1:02:28
space into an era of mass surveillance um which we’ll also hear a bit more about later um uh and in fact there’s
1:02:36
been a kind of long history of a bit of a tension between um the talented engineers that Google does manage to
1:02:42
recruit um and the leadership of the company um already in 2018 18 there were large-scale protests and organizing
1:02:49
against um project maven which was a contract with the Pentagon um to develop
1:02:55
AI for for guiding drone strikes. Um that shouldn’t be uh mistaken for the Palanteer Maven system. They’re I don’t
1:03:01
know that name seems to be very popular. Um but uh these protests back in 2018
1:03:07
forced the company to also adopt a series of principles um in fact on AI
1:03:13
some ethical principles on AI. Um but if we fast forward to the year 2025 um last
1:03:20
year uh Sundar Pichai attends the inauguration of Donald Trump’s second
1:03:25
term um and very soon after in February um 2025 Google does change its ethical
1:03:30
principles on AI um and it withdrew its commitment to not pursue certain applications including creating
1:03:38
technologies that cause or are likely to cause harm um including creating
1:03:43
technologies principally designed ed as weapons or facilitating injuring to people. Um and the use of information
1:03:51
for the purpose of surveillance against accepted norms, not the law but norms.
1:03:57
Um and whose purpose the which purpose contravenes international law and and human rights. Um so uh Vicram is going
1:04:06
to give a little bit of um insight into just how far Google has in fact gone um
1:04:11
in recent times um into the realm of providing military systems um and what
1:04:17
uh employees and exemp employees are doing about it. and welcome Vicrim.
1:04:22
applause
1:04:33
Hello everyone. Uh thanks for having me here. Uh I recognize that I stand between you and somebody called Snowden
1:04:41
after this. Uh so I’m going to keep this uh quick, personal, and controversial.
1:04:46
So I’m here to talk about Google who is the new age like AI digital arms dealer
1:04:52
company right so um about myself very briefly uh I studied in Indian institute
1:04:59
of technology which is like the premier like tech institute not very unlike uh
1:05:04
the ETH and other like students joining on the live stream uh your universities
1:05:10
um I worked in big tech for more than 20 years uh and um as a Google engineer for
1:05:19
uh 13 years and the last four years of my uh stint at Google I was in Google
1:05:25
Zurich uh and I parted ways with Google in 2025
1:05:30
um and since uh last two and a half years I’ve been organizing against uh
1:05:37
use of uh tech for human rights violation actually uh as it applies to
1:05:42
uh uh its application in Palestine, Right. So, um I’m I’m giving this
1:05:47
background to say like this is a this is a path many of you are probably going to
1:05:52
like take or can identify with and and and I hope I can short circute some of
1:05:59
the routes that I took that you more immediately arrived to a more like hey we can do something about fixing the
1:06:06
industry. Uh so of course uh for a long time I bought this whole don’t be evil
1:06:11
thing from Google. I had found my place. I I was working Microsoft before Google. I hated it. I came to Google, lunch was
1:06:19
free. Um uh and I thought I thought I found my place like uh in the tech
1:06:25
industry, right? So um um little did I know um and uh there was uh this project
1:06:33
called uh like contract called Project Nimbus. It was signed in uh 2021. Uh and
1:06:39
it’s a $1.2 2 billion contract that Google and Amazon jointly signed to provide uh AI and big data tool to the
1:06:47
Israeli government as a call but it as it turns out it was mostly for the military. Um right and Google PR was
1:06:56
always like this like uh this contract is not directed uh towards military and
1:07:02
surveillance use and that they are very serious about the acceptable use policy
1:07:07
uh which included things like you cannot use our our platforms for human rights violation or in general causing harm to
1:07:16
uh any humans. Right? So and for for some time I I I I did buy it right. uh
1:07:22
uh it was uh already like uh October after October 2023 uh I started getting
1:07:29
more uh involved with what was happening in Palestine. Of course like when like Israel started it siege of Gaza. Uh but
1:07:38
it wasn’t until April 2024 that I made the connection really in my mind uh when
1:07:46
Time magazine essentially wrote an article that said um Google signed a new
1:07:52
contract directly with the Israeli Ministry of Defense uh to provide big
1:07:57
data and AI tools. uh right and this hit me really hard because I was working in
1:08:04
a team that was building tools uh for the big data uh team uh in Google uh and
1:08:11
everybody around me were were working on things which I could see that it was finally ending up uh in the hands of uh
1:08:20
the Israeli um occupation force. Right? So um and to me uh by that time it was
1:08:28
already clear that a genocide was uh unrolling. Uh the international court of
1:08:33
justice had already said there’s a possible genocide uh and that companies and states have an obligation to
1:08:40
disengage. Uh and at that point that Google would sign a new contract made me
1:08:47
finally realize that of course Google completely knew uh what they were doing
1:08:52
and they were using it more as a business opportunity uh right and so that was like a turning point for me uh
1:08:59
that I wanted to mention uh and I want to mention like what is the sort of the army that you’re talking about when
1:09:05
you’re talking about the Israeli army right I mean this is not like the Swiss army uh right like so these are uh uh
1:09:12
these are like essentially um an army that is heavily invested in uh using AI
1:09:19
and surveillance for targeting. Uh right I want to talk about uh two specific
1:09:24
systems here just to give a more concrete uh example on how these technologies were used uh in the
1:09:31
genocide. Uh right. So the first one is Lavender AI. uh it’s a AI based human
1:09:38
targeting uh system and it used uh signals uh all sorts of surveillance
1:09:43
signals uh like uh your cell phone location or even whatsApp group uh that
1:09:48
you are part of and then would in some sort of a pre-rime uh kind of uh thing
1:09:56
would say what is your score uh as to uh whether you’re a target uh or not right
1:10:02
and once it uh a person cross a particular threshold that person would
1:10:07
be marked for targeting. Um and this whole system was run in a minimal human
1:10:13
intervention way. Uh and according to like uh sources in the IDF itself itself
1:10:21
um in many cases it was just check whether the target was male or not. Uh right and
1:10:27
and the only thing that the system was being optimized for is that the target
1:10:32
should be never ending. Right? So they would decrease all sorts of thresholds uh and the scores and of course the the
1:10:40
collateral damage uh that would be caused to just generate more and more of these targets. Uh the second one is uh
1:10:47
vars daddy which is like by its name itself like says like how bad uh that
1:10:53
system is. Um and it’s a system where the once a target is identified by
1:10:58
lavender uh would tell uh the military when uh the target entered their home uh
1:11:05
because it is when uh when they are with their with their family that they had
1:11:11
the uh highest possibility of essentially dropping a dumb bomb on the
1:11:17
building and and and according to them neutralized target. Of course they knew
1:11:22
again from the name itself they knew what sort of collateral damage that would involve and of course um initially
1:11:31
they were uh according to the reports uh from uh inside the Israeli military
1:11:36
itself uh initially they were uh just feeding few of the targets from lavender but by December 2023 uh they were
1:11:44
feeding almost everything from lavender into virus daddy which essentially meant
1:11:49
entire family and bloodlines were wiped out leading to the uh genocide. Right.
1:11:54
So um and uh let me quickly go through like yeah I mean most of our information
1:12:01
that we got out are is actually from the IDF itself uh right and uh that
1:12:08
basically uh neutralized the two Google PR points that they had uh where they
1:12:13
said it’s not directed towards military but IDF was like hey Google, Microsoft and Amazon are really good and we
1:12:19
couldn’t have done what we did without their help. uh right and uh again from
1:12:25
the sources inside IDF we also know that Google and Amazon essentially waved any
1:12:31
restrictions based on like acceptable use policies. uh right and so uh so this
1:12:37
is the kind of uh thing that we are dealing with in in big tech in general
1:12:42
and this is just an example right like Palestine is a is a laboratory like like like how even the traditional arms
1:12:50
industry uh deals with Palestine uh right and project Nimbus is just a start
1:12:56
um as Ja also mentioned uh Google uh dropped its AI principles uh right after
1:13:04
uh Trump’s inauguration uh they are working with Loheed Martin for their AI factory uh they’re expanding partnership
1:13:11
with Palunteer uh providing the cloud backbone for ICE and also doxing students like uh to ICE and providing
1:13:19
them information on uh on students uh leading to their deportation and of course they have now
1:13:26
contracts with the US uh department of uh war. So um and just to give an idea
1:13:32
of again what is the environment that you are entering right probably a decade
1:13:38
back there was some uh interplay between these AI companies and government you
1:13:43
could expect some meaningful regulation to come or on the on the flip side
1:13:50
companies to actually say no what the government is trying to do is wrong uh right but now we are in an environment
1:13:56
where there’s a higher collusion of these AI and big tech companies and government. And again, this is the
1:14:02
picture of all those CEOs lining up shamelessly for um uh the um
1:14:09
inauguration of Trump. Um right. Uh I I mean it doesn’t stop here, right? I mean
1:14:14
there are many institutions which could help us here. Uh universities sometimes
1:14:20
they do okay, sometimes they don’t. Uh for example uh this is just mentioning a
1:14:26
UN report which said uh one of the premier uh Swiss universities uh
1:14:32
essentially um took like criminal charges to handle
1:14:38
peaceful protests against the genocide right so uh so we have institutions that
1:14:43
are failing uh and and what I want to emphasize uh regulations are not working
1:14:49
it’s really up to each one of us and you to essentially uh take back the agency
1:14:55
in charge. Uh right. So one of the ways is worker organizing. Uh this is
1:15:01
something that’s not taken hold very much in the tech industry. Uh because again they give us free lunch, make us
1:15:07
feel good uh and actually make us feel bad for unionizing uh right
1:15:12
historically. uh but there is more and more uh like urgency to organize right
1:15:18
and one form of organizing is within your workplace irrespective of which company you end up with please uh think
1:15:25
about this inside uh Google and Amazon there is no tech for apartheite
1:15:30
specifically that organized uh against project nimbus uh and and I want to again emphasize
1:15:37
these are these are cloud systems which cannot stay up without uh like a
1:15:43
worker’s constant input, bug fixes and maintenance, right? Hardware, software,
1:15:48
everything. Uh they make they make you feel as if these are autonomous systems that just keep working and you don’t
1:15:54
matter. But really like all of us tech workers have a big role to play but also
1:16:01
means uh we have a big complicity in everything including the genocide uh
1:16:06
that’s happened and happening in Palestine. uh right uh the challenges are our own mindset too. Uh we need to
1:16:14
also change that around uh like our opinions about unions because this is
1:16:21
again we should consider as more as organizing for like uh making ethical
1:16:26
change as as uh as the forefront issue and that’s something different from unions in other sectors right and of
1:16:34
course there’s company repression uh right that we have to counter against but again it’s much easier when we get
1:16:41
together uh I also want to mention community organizing uh again like
1:16:47
talking with folks like you and activating you to think about what uh
1:16:52
you are going to be part of right and and u like really question the companies
1:16:59
that come here be it Google or Palunteer uh and ask them the hard questions before before you sign up for them um
1:17:06
and but also we work like uh try to do cross-disciplinary organizing with
1:17:11
academics lawyers jour analyst, activist, activists and whistleblowers. Um many of the information that you we
1:17:19
saw are from whistleblowers uh from within Google or outside Google uh from
1:17:25
different organizations. So uh like it forms like this pact or this um like
1:17:31
cooperation between whistleblowers and the organizers and journalists essentially are are what is uh um uh
1:17:40
bringing out all these issues, right? So um again what can we achieve? I just
1:17:45
want to mention three uh quick things. Uh we have definitely like documented
1:17:50
from all the information that have come out uh at a UN report level about Google’s complicity in the Gaza
1:17:57
genocide. Uh right uh and more specifically uh this year we have also uh essentially
1:18:04
worked with Republic here in Switzerland to capture Google Switzerland specific
1:18:09
role but we also uh work with uh uh community members here. We have a actually a climate activist disrupting
1:18:16
an Amazon CTO’s talk in a conference called AI for good. Um but uh like very
1:18:24
ironically named right. So uh so I I just want to end up end there and say uh
1:18:30
all of you can make a difference and don’t wait 25 years. Please start now and see the appro. Thank you. [applause]
1:18:47
applause
1:19:01
Thank you so much, Vicram. Um, and now to introduce the person that needs no
1:19:07
introduction, um, Edward Snowden. Um I am nevertheless going to do a bit of an
1:19:12
introduction because I think it’s um good to just remind ourselves um kind of uh what happened um many years ago um
1:19:19
and the kind of interim period. So, um, Edward Snowden is a former US, uh,
1:19:25
former contractor for the US National Security Agency, um, and of course a worldwide famous whistleblower, um, who
1:19:32
in 2013 worked, um, very closely with journalists, um, to leak, uh, thousands
1:19:38
of documents, um, that effectively gave evidence of a mass surveillance program
1:19:43
that the National Security Agency was running um, alongside uh, or in partnership with the UK, Australia, New
1:19:49
Zealand, and Canada called Five Eyes. Um so, uh of course there were, you know,
1:19:55
kind of a a huge attention around, um this case. There’s been countless films and and articles and things that have
1:20:01
come out since. Um but really what the revelations showed was that the very
1:20:06
nature of surveillance had changed in very fundamental ways. Um so to put it in the the words of NSA director or then
1:20:15
um NSA director General Keith Alexander um he said you need the hay stack to
1:20:20
find the needle. Um and what that means is that surveillance instead of targeting someone who is a suspect um
1:20:27
targets everybody aka the haystack um and then uses the data about everybody
1:20:33
to surface anomalies and to discover targets instead. So a kind of a flipping
1:20:40
of the kind of uh the way that um that surveillance actually um works and kind of effectively turning us all um in the
1:20:46
eyes of the machine into potential sub um subject um suspects sorry.
1:20:52
Um so uh with that I’m going to hand over um to Edward Snowden who’s dialing
1:20:57
in um remotely um who will give us a little bit of a kind of uh I guess some
1:21:03
thoughts on what’s happened um in the interim years. um since those
1:21:08
revelations and what’s what’s new in the landscape of surveillance um today with AI. So um I think we can get Edward up
1:21:16
on the screen. [applause]
1:21:30
You can hear me right? I got to check. All right. Uh
1:21:37
so I I have to say I haven’t been doing public speaking in a number of years. So
1:21:43
I might be a little bit uh rusty. I have become a parent since. And so instead of
1:21:49
being out there authoring tweets uh to nobody who was really paying attention
1:21:55
like the feeling that I have struggled with over the years, one of the reasons that I took a step back uh was it feels
1:22:03
like and I I think you guys probably understand this in the world uh the same way independently in your own uh selves
1:22:09
everybody is yelling and nobody is listening. There’s not much conversation happening anymore. Our AI certainly
1:22:14
hasn’t helped that. But coming away from that and kind of coming into a family
1:22:21
life has helped because I’m able to begin to look at things again through how children understand it and then hate
1:22:28
myself for recognizing that within all of us as an sort of tyrannical patriarchal dictator who has to say no,
1:22:36
it is dinner time. No, you have to stop what you’re doing right now and actually impose the rules. And that’s an
1:22:41
unnatural state of uh being for me. Uh but it’s been quite instructional because ultimately somebody has to do
1:22:48
that. But what this means is that I’ve been spending a lot of time reading fables. And maybe just two nights ago when I was
1:22:55
thinking about what I wanted to say tonight, I came across one of the stories that I I think you might recognize. I I I just want to share. Um
1:23:03
and that is The Emperor’s New Clothes. Remember that one? Uh
1:23:08
it’s been probably a while for all of you so I’ll go back through it. Uh the
1:23:13
idea is that uh you know once upon a time long ago these conmen bride into
1:23:21
town uh that is you know overseen by this king who is a famously vain ruler
1:23:28
if you can imagine very very uh concerned with appearances and this person’s squanders the treasury on
1:23:34
apparel you know buying fancy capes fancy hats things like that no one can be more beautiful no one can be more
1:23:41
interesting than this person or they just lose it and they do everything they can to make the attention in the story
1:23:46
all about them. Uh now these conmen
1:23:52
uh see a profit opportunity. So they spread this rumor uh that they have a
1:23:58
magic fabric uh that only the best people can see. Anyone who is unwise or
1:24:06
unsuited for their offices, that’s the language, unwise or unsuited to their office, cannot see the clothes. And so
1:24:13
the king, who actually does have some interest in governing still despite their their vanity, thinks, "Oh wow, that would be fabulous. I’ll be able to
1:24:20
tell uh who doesn’t deserve to be in their job." Uh so I’ll send my most
1:24:25
trusted minister uh to go look at these clothes, you know, order them to be be
1:24:30
made for me. Uh, and everybody in the country hears about these clothes because the uh, emperor, you know,
1:24:38
spreads this around about how wonderful they are, how they’re getting made, and it’s the most expensive thing in history, whatever. And the conmen put
1:24:45
uh, this great effort, this great show into showing how hard they’re working on
1:24:50
a loom, making this fabric that no one can see. So, what happens? The minister goes to see uh how the progress is going
1:24:59
and all of the con make a big deal of showing the fabric and go what do you think about this and everyone in the
1:25:05
country knows if you say you can’t see this fabric you don’t deserve to have
1:25:11
your job you’re an idiot you’re a philistine you can’t see these things uh and so what happens everybody instantly
1:25:19
adapts to the lie and the minister goes oh it’s so fabulous
1:25:25
Oh my goodness, his you know royal highness will be delighted with this. uh
1:25:30
and they bring in the you know chief of finance, the foreign affairs minister, everyone like this. They all come in.
1:25:36
Everybody from the party comes in and one at a time they all instantly without pre-coordinating, without a conspiracy,
1:25:43
without agreeing to do this, they all see the other person pretending to see it and they go, "Oh, instantly I know
1:25:51
what to do. I can see it. It’s wonderful." And you know, yada yada yada. You know how the story goes from
1:25:57
there. The story ends, of course, when the emperor in his new clothes gallivanting around in front of the
1:26:02
mirror then steps out onto the street. Uh, and all of the town’s people, all of the adults, all of the wellrespected
1:26:09
civilized people see the emperor walking around butt naked and they say, "Oh my
1:26:15
goodness, how glorious his reignment that everyone sees. How dazzling, how wonderful." until a child who is of
1:26:22
course ignorant socially acceptable lie that we all tell ourselves as adults
1:26:27
points out that there is in fact a genocide occurring in Gaza that its exterminationist intent has been openly
1:26:34
vowed uh by a number of uh Israeli ministers both on camera and in writing
1:26:41
and that for western democracies to continue arming such an operation given
1:26:47
the totality of evidence is not only in unethical uh but it’s most probably
1:26:52
criminal and there’s actually in fact nothing anti-semitic or or even anti-Israeli
1:26:58
in saying that uh we should expect better. We all decided the international
1:27:05
law together in the same time period. We all had a hand in making these rules and
1:27:10
so we should abide by them. And it’s of course you know only
1:27:16
children who recognize that we can live in accordance with the ideal just as
1:27:23
easily as we can live in accordance with reality as it is today.
1:27:29
The moral of the story more or less uh is that there will come a moment in your life when everyone else in your office,
1:27:38
in your lab, in your city, in your state, in your country is wrong.
1:27:44
Uh they may know they’re wrong, they may not. uh we we build up these big and
1:27:50
complex systems of law and belief and behavior and we all want to be good and
1:27:56
decent people who do the right thing. But you know sometimes these these
1:28:02
requirements to fit in and these requirements to behave in a humane
1:28:08
manner they come into conflict. What do you do when the moral is in
1:28:15
conflict with the legal? In my case, I I had a contradiction, a
1:28:22
conflict amongst my oaths, right? Um, and this is why any of you, you know,
1:28:27
know me or recognize me. I struggled with this inside. I talked to other people in my workplace. I saw the
1:28:33
director of national intelligence at the time, you know, our highest intelligence authority in the United States, uh,
1:28:39
General James Clapper, uh, actually uh, raise his hand, swear
1:28:45
an oath to tell the truth in front of the US Congress. Uh, and he was asked under oath by a senator who knew what
1:28:51
was going on because he was, you know, sitting on the oversight committee. Uh, is the NSA direct quot is the NSA
1:28:58
collecting uh communications on millions or hundreds of millions uh of Americans
1:29:07
and uh he thought instantly realized the trouble that he was in. He rubs his head
1:29:14
and he begins to sweat and he says, "No sir, no uh not wittingly. There may be
1:29:20
cases inadvertently perhaps but not wittingly.
1:29:26
And so now here I am the IT guy effectively a systems administrator in
1:29:32
the office of information sharing. It’s what they hired me to do. [laughter]
1:29:37
Didn’t really realize how good I would be at that job. Um
1:29:43
realizing that I have sworn an oath to the Constitution to defend it against all enemies. it’s language foreign and
1:29:50
domestic. Uh but then I have also signed standard form 312
1:29:56
uh the classified non-disclosure agreement. It says you’re not going to tell anybody about what it is you do. Uh
1:30:02
so people can disagree about the hierarchy between these two documents of course
1:30:07
but in my case I felt that the constitution was owed a little bit more weight. Um that actually turned out to
1:30:14
be a somewhat controversial decision. Uh but there are moments in life like that,
1:30:20
right? Um and you have to decide what to do and how you’re going to go about doing it and how much to risk. Like one
1:30:26
of the things is that I disagree with strongly is people come out and they introduce me, they they kind of paralyze
1:30:33
me. Uh and I want you to understand that I’m not a hero and I don’t believe in heroes. I’m not a perfect person. Do
1:30:39
terrible things all the time. And sometimes I feel bad about them and sometimes I don’t feel bad about them. I
1:30:46
am human chiefly. Uh and frankly it’s been a a struggle to get to that point.
1:30:52
Uh but but that’s where I’m at now. The thing is when we heroize people when we say oh that person is a hero like one of
1:30:59
my personal heroes in this this context whistleblower Daniel Ellberger repeal
1:31:04
revealed the Pentagon papers in the 1970s in the United States and the president then at the time called the
1:31:10
most dangerous man in America. Um,
1:31:17
when you call someone a hero, you distance them from yourself. You think they did something exceptional. You
1:31:24
think they did something that you can’t do, but you can, right? Uh, if the IT
1:31:29
guy, if the guy who fixes your printer can do it, you can, too. And frankly, guys, I didn’t graduate from university.
1:31:36
uh anybody sitting in this room in Zurich right now uh is probably a lot
1:31:42
further down the road of intellectual development than I was at the time even at 29. You know, I was a little bit
1:31:48
younger. Obviously, I’ve got a few more years on me now. And I I think I understand things a little bit better
1:31:53
than I did back then. But generally, I think I made the right call. I might do it a little bit differently. I frankly
1:31:59
might have done it a little bit more aggressively than I did back then, but I wanted to be overcautious. I wanted to
1:32:05
be more careful. I wanted to trust journalists to check my instincts to see
1:32:10
if maybe I to determine the public interest uh and balance my beliefs
1:32:16
against their judgment against the government’s sort of argument against
1:32:21
them. That’s becoming increasingly difficult to do because there is a new sort of uh
1:32:31
trend is not quite the right word. Um but there is a new prediliction for even
1:32:40
western democracies to engage in information control that frankly came out during the co era where they went
1:32:46
questioning sort of the lockdown policies and the mandates is dangerous to the public and you can understand
1:32:52
where they were coming from right uh but what they went with uh was fundamentally
1:32:59
in contradiction with our ideals as an open society And I think in a lot of
1:33:04
ways it’s gotten worse. We spend a lot of time now talking about misinformation. We talk about disinformation. We have Western
1:33:11
officials who are very blatantly lying to the public, which is not a new thing. That’s happened in every government throughout time. Uh but now we see
1:33:18
journalists sort of going along with it. Not because they believe them. The journalists definitely don’t believe
1:33:24
them. They have the savvy. They have the understanding. They have the back to understand. but because they think that
1:33:29
is the uh sort of pro-social activity to engage in and that’s dangerous.
1:33:36
And I want to get to the the real point of where I’m going at tonight because we we’ve got a lot of things to cover and
1:33:42
we’ll get to the panel. We can discuss a little bit more. Um artificial intelligence sort of these
1:33:49
large language models as we understand them to be artificial intelligence today. uh which is you know there’s
1:33:55
distinctions and nuance there but this is how we experience it. It’s the average person
1:34:01
uh has changed a lot of our understanding because they are centralizing power in the same way but
1:34:08
on a much greater scale. Uh we used to endlessly panic about the authority of
1:34:14
Google and how they controlled things like search results. Uh but a lot of people aren’t even using search anymore.
1:34:20
there just asking the LM and these these standards of behaviors sort of these motifs of how we
1:34:27
understand and experience the law how we believe our society to be organized and
1:34:32
governed has uh given rise to I think more contradictions that need to be resolved
1:34:39
and they will be resolved by your generation right like I’m out of
1:34:44
the workforce now uh the CIA and the NSA will no longer see fit to employ me I
1:34:50
heard heard in the introduction uh that I was introduced as a contractor which is actually correct for that last uh
1:34:56
office I was in the office of information sharing. Uh but originally I was actually a field officer, a
1:35:01
government uh a government employee directly for the Central Intelligence
1:35:07
Agency working overseas uh undercover in Switzerland
1:35:12
down in Geneva um and and many other places. you know, you go back and forth, but this was the
1:35:18
point uh sort of the understanding of the government employee and the private
1:35:24
company employee was beginning to become blurred in the United States in the post 911 era. Uh and that distinction has
1:35:32
only become more difficult to differentiate as corporations have
1:35:38
acrewed more and more power and we start to see these palunteer type activities
1:35:43
where the government has access to the silo the data repository but they don’t
1:35:49
actually have the manpower or want to have the manpower or want to regulate and know and control all of these things
1:35:56
in house because that’s a lot of housekeeping. That’s the love of bureaucracy. They would rather spend
1:36:01
their budget in, you know, different ways, however they want to do it. Instead, they want to put the manpower
1:36:08
liability on a private company. They say, "You come in, you construct our silo. Uh we will then run and operate
1:36:15
even though you built it, you understand it, you control
1:37:22
That’s probably
1:37:42
two one two one two um so that’s not suspicious at all [laughter]
1:37:51
um let’s see uh if we can get um connection it sound looks like um not
1:37:57
just the audio is gone now but
1:38:16
[snorts] In all seriousness, this was um a little bit of a concern [laughter] that we had.
1:38:44
All right. Great. [applause]
1:38:51
So, sorry about that everyone. I I have to say I use this system. It’s a known good system. I’ve validated the
1:38:57
baseline. And I’ve used this in many talks before. I never have issues like this. Uh suddenly when we’re talking
1:39:02
about these things tonight, uh I just had a blue screen crash where it started
1:39:07
with my audio driver. Uh it dropped out. I looked at it and then I you know the whole system blue screen. This is a
1:39:13
Windows machine which I hate but you know it’s for that. It’s just remarkable
1:39:18
remarkable that uh you know these kind of coincidences occur in these contexts. Anyway, can you uh just tell me where
1:39:26
you lost my audio so I know where to pick up from? Just a couple minutes. Um
1:39:33
yeah, just the last Yeah, literally just a couple minutes back. Yep. No, I I mean the last thing that I said
1:39:39
because I went on talking for a minute.
1:39:45
Silos. Silos. Ah, right. So these um these uh private
1:39:52
companies are basically operationalizing the plumbing and they’re becoming
1:39:59
partners to government. They’re becoming in a way agents of government which actually without actually being
1:40:04
employees of government. Now this agents of government is a uh important term in
1:40:11
US legal ease uh because the constitutional issues that we were talking before that I was struggling
1:40:17
with uh the constitution can’t be applied to bind uh private companies in
1:40:23
the US Google doesn’t have to respect your freedom of speech or association or or anything else like that um because
1:40:31
the constitution binds the state except in the context of what we call the state
1:40:37
action doctrine and this is if someone is directed to act as an agent of
1:40:43
government then the constitution now binds it’s very rare to see this applied
1:40:49
uh but the idea is that uh the idea for me the new idea is that
1:40:56
somebody has to own the liability of the action the government can’t just point
1:41:02
to the company the company can’t just point to the government and have it go away. Well, that’s what we’re doing with LLMs. You see a lot of talk in the news
1:41:09
today about open AI in anthropic who have uh developed these very largely uh
1:41:16
advanced models uh that are, you know, state-of-the-art relative to their peers who are constantly playing job. But they
1:41:22
did this by scraping the commons. Uh they did this by training on my book as came out, a lawsuit, my writings,
1:41:28
everything like that. Uh and everyone else is out there, right? Like I’m not special. This is, you know, anybody
1:41:33
who’s commented on an internet forum ever, anyone who’s been engaged in these things. Um, they trolled the corpus,
1:41:41
they distilled the commons, and they trained a model off of that. And actually, I’m not bothered by that. Uh,
1:41:47
I I think it’s, you know, fine. It’ll they violated the copyright, and I want
1:41:52
them punished for that. Mostly just because I want to hinder them enough that everyone else can catch up more so than I care about the fact that they’re,
1:41:58
you know, studying from from my notes or whatever. Uh but the idea is that now
1:42:07
they’re talking about agentic AI which I think is a huge mistake that they’re going to be discovering very soonly.
1:42:12
should not brand this thing as your agent because if the government uses it,
1:42:18
you are an agent of government and that means the constitution applies to you. And if they the government is using your
1:42:25
model running on your server uh where you are submitting queries they are then
1:42:32
intercepting your query applying their own system prompt to it which recomposes
1:42:37
your query running through their systems and it makes a mistake and people die as
1:42:44
or even if it’s correct people die as a result as they have in Iran if reporting is to believe be believed where we’re
1:42:50
bombing the wrong uh buildings. things, you know, because oopsie, the LLM made a mistake. Uh, and tremendous numbers of,
1:42:58
you know, children, the most innocent that we can imagine are being killed.
1:43:03
Well, who owns the liability there? And of course, naturally, they would say the
1:43:09
government owns the liability because you’re never going to successfully uh sue the government for this, even though
1:43:14
they they should eat it. Uh, but maybe that doesn’t work anymore because maybe they are an agent of government. And I I
1:43:21
don’t know where we’re ever going to see standing to sue on this, but we we should. And people in this room, I
1:43:27
think, are frankly going to be called upon in the course of their careers, in the course of their studies, in the
1:43:32
course of whatever you do to contend with these kind of issues. Where are we going to draw a line? Because an algorithm cannot make mistakes in the
1:43:41
way that we understand it as uh you know, you making because the errors in
1:43:46
its outputs are not the computers. It’s the slave to its instructions. The responsibility lies with its developer,
1:43:53
its programmer, its creator, with its architect. Which is to say, responsibility lies with a person. And
1:44:00
we cannot hide that. We cannot araid that away. Right? A gun maker is not
1:44:05
liable. It’s true for the misuse of the gun. Uh but if during the testing of
1:44:11
their latest piece of artillery they mistakenly obliterate, you know, a nearby village, it’s not the cannon that
1:44:17
should go to jail. And you know, just because you put out a press release describing how surprised
1:44:23
you were that your new model, you know, hacked the neighbor, uh or you know how
1:44:29
impressed you are by the destructive power of your missile. That doesn’t release you from the liability of
1:44:35
launching. this little lodge itself h you
1:44:40
uh and so there’s a lot we can get into but the idea that I want people to
1:44:45
contend with is is how we deal with this and right now these closed models are saying AI safety AI safety safety as
1:44:52
they call it where the idea is these things are so dangerous oh gosh they’re so dangerous but they’re so dazzling
1:44:58
they’re so important they’re so valuable we can’t get rid of them but you know oh it’s going to kill some people but it’s all right you know that just happens oh
1:45:04
it’s going to hack some companies Oops. You know, whatever. We’ll invest in them. The idea here is
1:45:11
who operates this? And this should apply to our devices more broadly around the world. Right now, you buy a cell phone
1:45:18
from Apple and before you’re allowed to use it. Uh, you have to click okay to continue. And they go, well, you’re
1:45:24
assuming liability for that. You you clicked okay to continue. You know, Google sells you an Android phone.
1:45:29
Samsung sells you an Android phone. You click okay to continue. And now that’s only you. that can’t be possibly how it
1:45:35
works because you can’t inspect the system. You can’t understand how it’s used and particularly in the context of
1:45:40
LLMs where you’re submitting queries to a model that they are running uh where they are for your safety and everyone
1:45:47
else’s safety intercepting the prompt and recomposing the prompt saying we’re doing uh applying safety rules and
1:45:53
transformations to it. Uh that’s on you but it’s not. I can’t even see what you
1:46:00
did. I asked you a question, you gave me something back in return. And if it’s an agentic AI where I’m putting this thing
1:46:05
in a harness, right? The harness being just a little program that basically uses these large models to feed it
1:46:12
instructions that it can then provide to tools that are existing on your local
1:46:18
system. So now your computer that is connected to the internet is being frankly remotely controlled by a model
1:46:25
that’s far away that you don’t understand how it works, but you know, you hope it works pretty well. And frankly, they do work reasonably well,
1:46:32
but every now and then it might hack, you know, some random company or whatever out there. Uh,
1:46:39
I think we need to put Sam Alman in jail. And I think it’ll be an instructive experience.
1:46:45
[applause] I think it’ll be an instructive
1:46:51
experience uh for the industry and for the world. Um, [laughter]
1:46:57
and you know, it wins me friends in this uh this audience I appreciate, but it
1:47:02
might not throughout the world. I don’t know. It depends on I haven’t been following the public mood recently. Uh,
1:47:08
but the idea is that it doesn’t have to be like this. Uh, there is an alternative model that’s that we have
1:47:14
rights. We run literally these large uh language models through an open-source
1:47:20
manner and we see this through competitors. They’re still imperfect. Uh but it’s no longer incumbent on these
1:47:26
companies uh to try to play hall monitor and interject them between the operator
1:47:34
the user uh as they should be and the operator currently the status quo now
1:47:39
which is the company and the data center. When you run these things locally when you push them to the edge when the user is the operator that’s a
1:47:47
more fair model. When Apple sells you a phone and you are operating it, right,
1:47:52
if you own it, if you control it, if you can see what’s going on it, if you can inspect it, it there’s a much more fair
1:47:59
legal argument to be made that you are the true operator. And that’s the way it should be. If you are buying something,
1:48:06
it is yours. It should work for you. And you guys are going to be involved in the architecting decisions about how future
1:48:12
paradigms of people’s interaction with software actually works. And I I would like you to remember this. We need a
1:48:19
kind of first law of technology something like this. Uh which is that a device will never act contrary to the
1:48:28
interests of its operator. Uh it’s good to think of you know how we do these
1:48:34
things make them in a safe way right but if you tell the computer to do X it
1:48:39
should not do Y and then tell Z it should do X.
1:48:45
Um, a device should be designed to serve the operator’s will, right? The owner’s
1:48:52
will, assuming they’re the same thing. This becomes a little bit contrary when you have sort of the owner is nebulism,
1:49:00
like they’re trying to lease out a device or whatever, but I think that kind of shows the problem. If a device
1:49:07
is sitting in your pocket that tracks you, that understands you, that interprets you, that connects you to the
1:49:13
world, uh, that interjects itself into all of these different things, that intermediates
1:49:18
everything, but you are dependent on to live, to work, to eat, to engage, to
1:49:24
travel, to buy tickets, to establish yourself, to prove yourself, uh, to permission yourself in the world. Uh,
1:49:31
this must be yours. And if it’s not yours, if it’s somebody else’s and you
1:49:39
can’t engage them, you can’t travel, you can’t establish who you are, you can’t spend, you can’t trade, you can’t
1:49:44
interact, you can’t speak, you can’t publish without these things. Well, that
1:49:50
says something very dangerous about where sort of the allocation of power in society has become. And I know I’ve gone
1:49:57
over time, so I’ll I’ll shut up and and and try to cut this short here. And you know, if anybody has any questions, we
1:50:02
can come back in here. But the idea is that if open AI were anthropic were trying to
1:50:09
say they’re for public safety and they’re interposing themselves between your will and the actual output and they
1:50:17
are transforming these things. Uh they are saying that they know better than you how to operate your technology. And
1:50:24
perhaps that is true. But that is a very dangerous proposition um that they are knowingly entering into
1:50:31
millions of times a day including in these like military paramilitary governmental um contexts.
1:50:40
Uh where if you were a passenger in someone’s car uh and you go, "Oh, the
1:50:46
driver’s acting somewhat unsafe." you grab the steering wheel and you pull it and you in fact cause an accident rather
1:50:52
than avoid an accident. [clears throat] [laughter] If the police show up at the scene of the accident, we go, "Well,
1:50:58
well, I only pulled the wheel a little bit." The liability is still fully yours
1:51:04
and I think it should be theirs as well. The thing that I want to summarize here
1:51:10
and bring it back for for this audience that’s important is to understand that that
1:51:16
in a lot of these contexts there aren’t very clear bright line answers. Uh like
1:51:22
in the emperor’s new clothes there is a very clear very well
1:51:28
understood socially expected answer but that doesn’t mean that it is the
1:51:33
correct answer. And a lot of times we in developed
1:51:38
societies like to substitute the law for our own judgment. We go it’s the law.
1:51:45
It’s the law. Slavery was legal, right? I was wrong. Um
1:51:52
we need to understand that the law doesn’t protect us. We protect the law,
1:51:57
right? We invest the law with meaning through our acts uh through upholding
1:52:02
sort of these legal traditions. When the law begins acting in ways that are contrary to the protection of people and
1:52:09
society, the law needs to change. If people can’t see that,
1:52:15
it sort of like the morning bell. Someone must strike it to wake us and
1:52:21
bring us to attention. Right? The spirit of the law might itself be slumber, but when it hears the bell, everyone pays
1:52:28
attention. Now, that doesn’t mean everything gets better. That doesn’t mean you save the world. God knows. you know, I tried to tell people something.
1:52:35
Uh, and frankly, a lot more people heard it than I ever expected, but that didn’t fix the problem. It’s still out there.
1:52:42
It’s continuing. It’s getting worse in many different ways. But it also got better than others. And when people have
1:52:48
awareness, when we’re aware of the danger, we can begin to steer around it. We can begin to correct. And that’s a
1:52:54
long process. It might take longer than our lifetimes. Uh, but I hate to say it,
1:53:00
but that that’s kind of normal. Information by itself is not power. No
1:53:06
matter how much we want to repeat the slogan, organization is power. Our will
1:53:14
is power. Our willingness to act, right? The information does nothing by itself.
1:53:19
It can even pacify. It can disempower people. It can sedate people. We see the dumerism. We see the endless scrolling.
1:53:25
You see the atrocity in the newsfeed. You click the play button and then go, “Oh, I didn’t want to see that.” You scroll along without saying anything.
1:53:32
You go to the next thing that makes you feel good. Self-certain in the illusion that you know you couldn’t do anything
1:53:37
about it. It’s, you know, inevitable. We have no time to act in a lot of
1:53:43
cases. And people want to feel like they can’t do anything, but you can. You just have to look for where in your place,
1:53:50
your office, everybody else is wrong. And you know, most of the times it’s you
1:53:55
who’s wrong. Maybe I was wrong. Uh and again that is normal. We try and we
1:54:02
fail. We start out poorly. We are constantly making mistakes as people.
1:54:08
But we can get better. I think humanity in a large way is about failing. It’s about failing well,
1:54:16
failing by less, failing by smaller degrees, failing better, failing big,
1:54:21
and striving farther. you know, failing deeper and faster until you’ve gathered
1:54:26
such skill and confidence in correcting your errors and in facing the fall, the
1:54:33
uncertainty that you can begin to bend the path of your decline into a launch,
1:54:40
you know, and suddenly now you’re moving. Headlong dives accelerate and
1:54:46
they build upon themselves with an arc and dignity like
1:54:54
like a bird, you know, for whom the tyranny of gravity no longer holds fear.
1:55:02
And that’s the idea that I want you guys to remember. We are not born human,
1:55:07
but we do have the chance to become it through the choices that we make every day.
1:55:13
Thank you. [applause]
1:55:26
applause
1:55:39
applause
1:55:51
Thank you so much. Um, thank you so much for that, Edward. There was um, that was
1:55:56
a beautiful note to end on. Um, and in fact kind of a great segue into my first
1:56:01
question for the panel as well. So, if I could invite Margarit and Vikram um to join me here on the stage
1:56:09
um please [applause]
1:56:23
uh so um we’ve heard a little bit about uh the nature of AI surveillance and um
1:56:31
the threats to freedom of speech. of the past few speakers, three speakers. Um
1:56:37
there’s a great quote also that I really like that I enjoy from Meredith Whitaker, the president of Signal App,
1:56:43
who says that um AI or more precisely LLMs, um are a surveillance derivative.
1:56:49
Um and in many ways, surveillance has um become kind of the water that we’re all
1:56:55
swimming in. Um, and we’ve heard also about how, you know, people are calling, you know, instead of saying there’s
1:57:01
errors in software, we’re using words like hallucination and so on. Um, and also, you know, rather than, you know,
1:57:07
generally speaking, there’s not that much kind of focus on surveillance per se, we’re we’re hearing people talk more
1:57:13
about intelligence or um, you know, datainformed decision- making and so on.
1:57:19
Um so it’s become kind of part of the technical ecosystem that we’re all um
1:57:25
living in. So that can make it sometimes a little bit hard to um kind of notice
1:57:33
what is right and wrong in our everyday lives. Um so I thought maybe we could start off um with that point like at
1:57:40
what point in your kind of work as a journalist as uh someone working um in big tech um and in your case Edward Oso
1:57:49
um in in your work at what point did you sense that something was wrong and
1:57:54
decide to act on it? um what was the kind of wakeup call um given that you know the environment around us has kind
1:58:02
of normalized the situation or seeks to normalize the situation.
1:58:07
Um yeah I mean I can start. Um I mean as I said uh in my talk uh the turning
1:58:14
point uh was really uh information right? It was the starting point for
1:58:20
sure. Um,
1:58:31
[clears throat] do you would you like to respond,
1:58:37
Margaret, while I’ll try? Oh, we got it. Great. Great. Is it better? Yeah, perfect. Cool. Thank you.
1:58:44
Um, yeah. So um I mean for me it was that and that it
1:58:51
felt so close to what I was uh uh working in uh was really the uh turning
1:58:58
point uh for me and uh in in a way I was the thing also that struck me was
1:59:04
usually in some other situation I wouldn’t be the only person saying
1:59:10
something but in this case for various reasons Right. It felt like I was saying
1:59:17
something that probably somebody else was hesitating to say even though it was
1:59:22
probably clearer uh to them too. Uh which made me also more responsible for
1:59:29
continuing to talk about it. Right. So yeah. Well, for me as a as a journalist, there
1:59:36
wasn’t really a moment where I was like, you know, this was my my moment where I was like, I know what is right and
1:59:42
wrong. Um, I mean, I think there’s like journalistic practice that you learn,
1:59:47
you you develop, uh, you look back at mistakes you’ve made or things that you could have handled better. Um, you grow,
1:59:55
as Edward said before, like you become hopefully better at what you do. Um, and
2:00:02
like you also learn to sort of bit hard to explain, but you get like a
2:00:08
sense, a gut feeling for some things. So that’s often the starting point for a lot of things. So most investigations
2:00:15
start with a question like, yeah, what is actually that that that thing we’re
2:00:21
seeing or does that smell fishy? I don’t know. I don’t know what what’s up there. Like quite an innocent thing. Like most
2:00:28
most of our investigations start over a beer or something, you know, where you’re talking with each other or over
2:00:34
coffee and you’re, you know, just bouncing ideas back and forth. Um and
2:00:41
yeah, I I think learning over time that um
2:00:50
people have agency. Um different peoples have agency. I don’t believe in proxies
2:00:55
when it comes to populations. I think it’s very important to to note that
2:01:01
individuals have agency uh and to take them seriously. Um, and then yeah, um, I
2:01:09
think you sort of just try to build your journalistic practice and integrity that
2:01:14
way. Um, trying to do things correctly, talking to everyone you can, getting your quotes greenlighted, giving
2:01:21
everyone the right to reply. I do believe in the rule of law. Um, I like though what you said about um, that we
2:01:28
uh, we protect the law. Um also the investigations we do couldn’t be done
2:01:35
sometimes or not as easily be done um in different contexts. So I do also believe
2:01:40
in democratic values that support the things we do. Yeah. Um and speaking of democratic
2:01:47
values, I mean I suppose you already mentioned one moment when you were speaking um Ed about uh witnessing a
2:01:53
government official lying under oath. I can imagine that that was a big kind of um moment for you. But um is was there a
2:02:00
specific tipping point uh at what what moments do you kind of remember of kind of realizing that something was out of
2:02:07
line with your moral compass? I mean it was it was actually [laughter]
2:02:13
I had been thinking about it for a long time and you know you have conversations in the office with people and you’re
2:02:18
like hey you know what do you think about this and they’re just like ah you know
2:02:24
it is what it is but that there’s only so much it is what it is. Uh and there’s
2:02:29
not anymore. What the reason I say the the the clapper anecdote uh is actually being played in the cafeteria at the
2:02:37
facility was at is called the the tunnel in Kuna, Hawaii. It’s an old abandoned World War II aircraft facility. It’s
2:02:43
closed now, so I don’t think even though I’m sure they’d love to add more charges to my charge sheet for it, I don’t think
2:02:50
it really matters. Um, but sort of buried under a pineapple field, you’ve
2:02:55
got this old like threestory uh facility which used to be used in
2:03:00
World War II to build the aircraft and then they would fly it out and be bombed before it uh
2:03:06
got uh got made and put into service. But uh it hadn’t been used in a million
2:03:12
years. It was like leaking this ramshackle facility. But you’re kind of trapped as employees there because it’s
2:03:18
so far away from everything else. There’s like no place to get lunch. So, they’ve got their own little cafeteria. You go in there and they’ve got like,
2:03:24
you know, CNN, Fox News, things like that playing on the TV in constant loop. Uh, and it’s not just that I saw it.
2:03:31
It’s that everybody in the cafeteria saw it. Everyone saw that testimony who was
2:03:37
getting lunch at the time. I was there with my buddy and I just kind of looked at him, you know, raised my eyebrow and
2:03:42
he looked at me and he just shrugged. And it’s the shrug really that that gets
2:03:47
me. like it’s I was supposed to shrug and for the
2:03:52
longest time I did shrug like people again this is fine but don’t hold me up as a hero. Um everybody in the room is
2:04:00
doing better than me. I volunteered to work for the CIA and I thought that was
2:04:05
a good and noble thing and I understand the reasoning like at the time honestly it was cool it was interesting. It was
2:04:11
whatever but it wasn’t that interesting and you realize that it’s just government work like any other work and it’s just you’ve got different goals. We
2:04:17
have a a a a problem now in the media. It’s every every country, right?
2:04:23
Typically, I’m talking about Western society because that’s the one that I understand the best. I’m native in. But we see this everywhere where like spy
2:04:30
chiefs are being quoted on like policy issues and things like that as if these
2:04:35
guys don’t have an agenda, as if they’re a reliable source, as if they’re telling you what’s actually going on. And it’s
2:04:40
[laughter] like in no work is it more clear than an intelligence. Like these
2:04:47
guys are constantly lying. They’re supposed they’re expected they are required by law in many cases to lie to
2:04:55
protect you know classified work or whatever. Um and there is some need for
2:05:00
that times of conflict and when you don’t want the enemy to know the military’s coming over the hill until they’re over the hill. Um but we don’t
2:05:09
have a history of that frankly going back that far outside of very narrow times of war. Uh except just like the
2:05:17
last 75 years the post-war period. Uh and in the US the state secrets doctrine
2:05:22
was really only established uh because a widow was suing the government for
2:05:28
compensation for her husband who had died in the crash of a test aircraft.
2:05:34
Right. And then the government was like, “Oh, you know, aircraft designs are we don’t want to admit this is we’re working on this program, whatever.” And
2:05:40
so they just shut it out. They’re like, “State secrets, you know, we don’t we don’t want to talk about this.” Instead of just compensating this lady, quietly,
2:05:46
make it settlement, whatever, because they were afraid of the president. Um, but we have followed that. We just
2:05:52
don’t really question it and we go, this is what we we need to do. It’s important. But do it. Do it. You know,
2:05:58
they said when I came forward in 2013, the sky was going to call fall. The, you know, oceans were going to boil off, the
2:06:04
atmosphere was going to collapse. We’re all going to die because now we know how these guys were breaking the law to spy
2:06:09
on everybody, not just in the United States, but around the world to the maximum extent they could. Uh, and now
2:06:16
everybody knows about. And you know, the the world didn’t collapse,
2:06:22
it [clears throat] became a business, right? Um, yeah. [laughter]
2:06:27
And the thing is if we don’t question it and constantly that’s the thing. It’s not a bright I don’t want people to
2:06:33
think you’re sitting in the room and it’s like spark of genius. It’s exactly like Margarite was saying you’re just
2:06:38
talking just thinking but you don’t stop thinking. Don’t try. That’s all it
2:06:44
takes. So and picking up from there. So you’re
2:06:49
you’re in the cafeteria. you’re seeing your colleagues or you’re you’re out having drinks with your colleagues discussing, you know, what’s going on in
2:06:56
the world and um you come upon, you know, an issue. Um
2:07:03
I’m curious about kind of [snorts] this this uh question of how you find your
2:07:09
allies, like who do you know like is on your side and how do you even start having kind of meaningful conversations?
2:07:16
There is of course a very very long history of the relationship between the whistleblower and journalists, right? Um
2:07:22
and that’s kind of like let’s say a a set of actors that we’re somewhat familiar with now. Um we kind of know
2:07:30
what it means to be a whistleblower one might kind of consider becoming a whistleblower in a situation like that.
2:07:36
Um and maybe even also have the thought that as a whistleblower I must get in touch with some journalists that I that
2:07:41
I trust and and know and so on. Um but uh could you speak a little bit to the
2:07:46
kind of um this moment of becoming when you realize something is wrong and you decide to take action? How do you even
2:07:53
know who to start talking to in order to to to actually take action? How do you know who your allies are? Who are the
2:08:00
kind of characters that that one can call upon in moments like that
2:08:06
for me? Yeah, please do go ahead. We we’ll take I think the question is for the whole panel in fact, but um do go ahead. I
2:08:12
I’ll try to go quickly uh because I know the last one. I I think you’ve got to read uh when like you get the that
2:08:21
feeling that everybody on the panel about uh that that something is wrong. Uh but you you still have your
2:08:27
understanding of it and you want to broaden that. Uh so I bought books. I looked at things online and frankly this
2:08:33
was the interception period that the NSA could have caught me and realized this guy’s having an ideological turn. uh
2:08:40
[laughter] as you see the person’s reading interest which I know now they’re probably monitoring much more than they were before and they shouldn’t
2:08:46
be but they will be and you know the models and whatever we are increasingly transparent
2:08:53
uh to these algorithms to these monitoring systems to these behavioral metrics and tracking so they’ll they’ll
2:08:58
see these terms and you got to be aware how’s that going to look in a subpoena uh this is the nice thing you can still
2:09:04
go to like an actual bookstore and buy an actual book with actual money instead
2:09:10
of using a card. Hard to believe that people still do that. Uh but you look at who’s written on this. You look at who’s
2:09:16
criticized this. You look at who’s written on it and looked at it from other angles. And that’s actually kind of making the list yourself. That’s why
2:09:22
I think journalists are so important is you see who has skin in the game, who’s actually put their name on it and and
2:09:28
made an argument and who can persuade you and maybe you think you can persuade them.
2:09:35
[snorts] Vicram. Yeah. I mean in my case I would say the easier part was as I said like there
2:09:43
were already people organizing uh inside Google even though in a smaller way like uh there was no tech for apartheite
2:09:50
which was already there right so when I wanted to uh like uh talk more about
2:09:56
what is happening in Google organize more there was sort of uh that uh
2:10:02
already there even though uh it was just a couple of us in Zurich who had started
2:10:08
off uh doing that. Uh right. So uh for me I would say there was already some uh
2:10:16
thing I could like uh uh stick to to like get like uh like saying like the
2:10:24
group that I trust in some sense and and that also led us to build something similar in Zurich and in other parts of
2:10:32
uh Europe also, right? So that when the next person wants to get organized, do something uh there is already already uh
2:10:40
us for that they can approach to do uh the next step right. So
2:10:45
yeah. Yeah. So visibility and and campaigning. Yeah. And Margaret for me it’s a bit less about allies. I
2:10:52
wouldn’t say that. Um as a journalist I try not to have like allies but I try to
2:10:58
have a variety of sources. And not every source is a whis or what something we call a whistleblower. But uh a lot of
2:11:05
information that we get comes from people giving you like a a so-called tip off or just telling you about something
2:11:11
you didn’t know or like hey check out that. Um so basically you try to sort of
2:11:16
talk to as many people as you can. Um try to really wrap your head around a matter and a question. Try to be open in
2:11:25
where you’re you’re heading. So don’t assume something and then just try to sort of prove that but uh really you
2:11:32
know investigate in an open way and also think about um like what voices are not
2:11:39
heard very often um and who is accountable.
2:11:45
Yeah. Um at this point I want you to start thinking about your questions because I’m going to ask two more questions of
2:11:51
the panelists and then we are going to open it up. Um, so, uh, so to kind of,
2:11:58
um, I guess get a little bit more into, you know, okay, so you’ve realized something is wrong. Um, you’ve kind of
2:12:04
started to identify who might be potential allies to kind of work on this topic. Um, the next question is, of
2:12:11
course, uh, what do you then do? what are the forms of action um that are available uh to a person that is um kind
2:12:19
of keen to shift the dynamic um in their workplace or even um as a student as
2:12:24
well. Um yeah maybe we can start with you Vicram. Uh I mean uh I think uh we
2:12:31
have sort of tried many different things uh and at this point we feel that uh
2:12:37
there needs to be a variety of strategies also and which is why I talked about both organizing internally
2:12:43
at your workplace but also using all the resources externally uh too uh right to
2:12:50
because uh uh I mean there might be workplace dynamics which are shifting
2:12:55
more like employees have lesser But these big tech companies still worry
2:13:01
about their reputation and uh in those cases u approaching like people outside
2:13:08
like forming like like bonds with community to apply pressure from the outside could also be uh things that
2:13:14
work. For example, uh when I wanted to uh raise awareness, first it was just in
2:13:19
the form of just sharing articles uh because at that point uh when I had
2:13:25
started 50 people had actually been fired at Google uh just uh for uh like
2:13:31
doing a sit in to like get an answer from the executives uh right and so there was an environment
2:13:37
of like uh um repression right which made it difficult to talk. So uh at that
2:13:44
point even sharing something like uh companywide uh was something we needed
2:13:50
to figure out how to do safely and things like that started out like that but as um Snowden was also saying like
2:13:56
it doesn’t stop with uh information right so that is just a a starting point so we were also looking at different
2:14:03
ways to apply pressure so uh that’s why we work with journalists and activists
2:14:09
and lawyers and try to find like what are the different ways we and bring these companies to account, right? So,
2:14:14
yeah. No, great. Um, you might be able to tell, uh, Ed that I’m trying to kind
2:14:20
of pick up on your, you know, no one’s a hero here and everyone can can do something and that’s kind of part of the
2:14:26
the framing of these particular questions of the panelists. So, um I don’t know if you have any thoughts on that in terms of um in terms of
2:14:32
meaningful action or even if you just kind of think back um Ed on your history. Um what was how did you kind of
2:14:39
think what action was meaningful to do in that moment? Like what were the ideas in your head? Because there’s obviously
2:14:45
a variety of one can protest, one can set up a petition. Um you chose kind of
2:14:50
uh providing evidence um to the press. Yeah. I mean it’s it’s a tough one
2:14:55
because I think it’s different. It’s also different in people’s lives. One of the things that I said when I I came forward in the first interviews in the
2:15:02
the hotel room that was featured in the documentary Citizen 4 uh was, you know,
2:15:08
not not everybody can do this, but it’s not about uh sort of personal makeup or where
2:15:15
you’re at or anything like that. It’s also life circumstances. Uh I was in a
2:15:21
committed relationship, but you know, as a as a young man, we weren’t married. I didn’t have family in the context. So, I
2:15:26
had kids. I had people. I would provide for they they wouldn’t make it. And now I do, right? And so I wonder back then
2:15:34
if I was in the same situation if I would have been able to do the things I did. One of the things that I I try to
2:15:40
talk to people about when they’re doing this is I I don’t want to see martyrs, right? A lot of people think like I went
2:15:46
out, you know, blaze of glory or whatever. And it was incredibly risky what I did. And frankly, the chances
2:15:52
were always greatest that no one was ever going to hear my name. you know, I was just going to step out of the office
2:15:59
and be clapped in handcuffs and then, you know, you’d read some story in A12 of the paper that was talking about
2:16:04
arrest of some contractor, you know, hoarding material or something like that. There actually been a number of cases like that in the NSA since they’re
2:16:11
like, oh, this person has a disorder, this person has an issue or whatever, disgruntled employee. We’ve seen all
2:16:16
that before. But uh in terms of like how you do it, how you think about how how I
2:16:23
thought about it was like what what what should you do? What is the corrective? and they thought well
2:16:30
the government has a constitutional obligation here to respect our privacy u
2:16:40
and if that’s not the case uh this should be you know and they are arguing they’re
2:16:46
arguing that what they’re doing is legally authorized uh but in my reading
2:16:52
it couldn’t be legally authorized even if you passed the law to do it without a constitutional amendment which obviously
2:16:57
was absent the law was even questionable [clears throat] you know wake that hadn’t been authorized what’s the
2:17:03
corrective and you know what should people know what should you do so I thought well maybe it’s just a journalists know about this they can do
2:17:09
it but how can I do it uh without you know destroying my own life rotting in a
2:17:15
prison or whatever and it wasn’t clear to me that that I could because being a tech I understood the audit trails I
2:17:23
understood that all the fingers were going to point back at me eventually now it was going to take time to reconstruct
2:17:29
and do these things like that. Uh so could I get journalists in the room with
2:17:34
the documents with enough time to go through and basically help them understand this and get the story out
2:17:41
before the door gets kicked in. And then it becomes a question of balancing jurisdictions and logistics story and
2:17:48
protection and OPSACE and how do you make sure this doesn’t fall into the wrong hands and a lot of other things. But frankly those are details. uh the
2:17:55
key was what is happening what do you know about it should other people know
2:18:01
about and what can you do basically the way to do it now there’s also balancing act of how far you go with this and I
2:18:08
actually you know there’s been a tendency a lot of people in the room may have already forgotten this but Julian
2:18:13
Assange the founder of Wikileaks right uh was sort of the OG in this area for
2:18:19
publishing classified documents and the government argument again they persecuted this guy had him in a dungeon for five years uh was they that he went
2:18:26
too far, he went too far, he went too far. And they would say, “Oh, Snowden was good. Snowden was careful. Snowden was cautious.”
2:18:33
Assange was reckless. You know, endangered people did all these things. And I don’t think that’s true. I mean,
2:18:39
at trial, they weren’t able to show harm. They were asked to show harm to any person. Surely couldn’t do it. Um
2:18:45
but more broadly, it’s a question of this balancing. I there’s a question. Was he actually closer to right than I
2:18:51
was? Was I too cautious? And there there’s no correct answer here. These are the kind of things that you think
2:18:56
about. You do the best you can in the context you have. There is no ideal. There’s no right answer. Just try to do
2:19:04
as well as you can with what you have. Yeah. No, thank you. Um you mentioned uh
2:19:10
the documentary Citizen 4 um that captured those moments when you’re um in the hotel with Laura Portric and and
2:19:17
Glenn Greenwell. Um what kind of struck me when I watched that documentary uh many years ago now,
2:19:24
but um was really how calm you were in that situation. You seemed so incredibly
2:19:30
calm. So this is a question both for you but also the other panelists because
2:19:35
you’ve each of you been in in quite intense situations when it comes to um standing up for you know what you
2:19:42
morally and ethically and and also legally believe is is correct. Um, what
2:19:47
does it feel like, you know, when you finally are taking action on this? Like, do you feel just as kind of zen and calm
2:19:54
as um Ed seemed to be in in that documentary? Um, or what what is the kind of um the feeling of of doing that?
2:20:03
I mean, I’ve never been in that kind of intense situation. Um, but like maybe
2:20:08
talking about the lawsuit, uh, a lot of people asked us if we were scared or if we were, you know, Yeah. how we felt
2:20:15
about it. Um, we kind of were so it could have happened with any
2:20:21
investigation, right? [clears throat] This can happen, you know, that and you kind of think of that as a journalist
2:20:27
when you’re investigating a company or, you know, powerful individuals. You always kind of think of of that
2:20:33
beforehand what what could happen. You think in strategies. Um, we were
2:20:38
relatively calm. It was uh quite a bit of a pain. uh lot of work uh tire tiring
2:20:46
but also we knew that we were in quite a privileged situation. Um we we are in
2:20:52
Switzerland. We have a rule of law. We still trusted the um the the legal
2:20:58
system. Colleagues in other countries cannot uh trust that. So we knew that we
2:21:04
were in quite a good position and we knew that we had worked really well and um I I really one of the things that
2:21:12
struck me most in that time was we getting so much so like so many emails
2:21:18
and messages of solidarity from journalistic colleagues from activists from everywhere like you know emails
2:21:24
from Lithuania hey just wanted to say you know support and we’re like okay
2:21:29
thanks um But I think knowing that you’re, you know, not not being a hero,
2:21:35
but you’re building a body and maybe also building um uh you know, when you
2:21:41
when you win a court case, you you build a point of reference. And I do think that points of reference are really important, like the Universal
2:21:47
Declaration of Human Rights that states that everyone is born free and with the same kind of dignity or that uh unwanted
2:21:54
um interference into privacy should uh not be allowed. Um I think that’s really
2:22:00
important. So knowing full well, I think it’s really important to know
2:22:07
in what way you can play the best role possible. Um, and in general maybe how
2:22:13
you want to spend the 80some years you might have on this earth. It doesn’t always [clears throat] have to be loud. Um, yeah. But from the
2:22:21
sounds of it also this feeling that by stepping forward and putting this story out you guys were also building a
2:22:29
community uh public in a sense um that was interested and and supportive I guess so but it’s a risk you kind of
2:22:35
always factor in. Y uh how about you Vicram? Yeah. Uh I mean
2:22:42
uh in my case I mean definitely there is um there is a extraordinary part of this
2:22:48
whole thing that we are in midst of a live stream genocide that we are seeing.
2:22:55
Uh right that is definitely a part of like I knew again reading more about
2:23:01
history like what uh are the individual responsibilities or at least what they
2:23:07
said right uh from from when you read these historical records that definitely
2:23:14
was part of it and that that we are like in in in the middle of genocide. uh
2:23:20
right and for me while taking the action it was it wasn’t clicking to me I would
2:23:27
say at any point but the thing I would think of uh is uh am I setting it up
2:23:32
right for my daughter for the next generation and when I look back 10 20
2:23:39
years from now can I say that I did everything that I could do to stop this
2:23:44
genocide right so I I think it’s really important that we um in whatever role we have and whatever
2:23:52
jobs we have um we think about like what kind of society do I want to live in and
2:23:57
how do I like add to that. Um yeah and
2:24:02
principles like not every everyone has to make the biggest sacrifice ever but
2:24:08
principles are there to you know keep them when it gets a little hard. Yeah.
2:24:14
you maybe turn down the job with a high salary because you have a principal or
2:24:20
you know you decide not to take that trip to somewhere or whatever.
2:24:25
Yeah. Yeah. I I I think for me it’s first first off
2:24:32
the camera right [laughter] you’re not always cold there were there were moments where I was unusually calm
2:24:40
and detached like a vague state dissociative kind of thing uh and I mean computer people are a little weird to
2:24:46
begin with uh [laughter] that’s just how it is um but uh
2:24:54
you become something that’s that’s bigger than the self and you have to understand that at the time that was
2:24:59
filmed I had already accepted the worst possible outcomes and by the time that
2:25:05
camera is rolling in terms of what I was willing to accept I had already won right like we were
2:25:12
over the finish journalists were in the room they were filing stories like there was no stopping this and so that gave me
2:25:19
like a measure of respit but towards the end of the movie you see I’m like taking my glasses off trying to you know escape
2:25:25
the hotel I’m worried about being recognized I don’t think it’s on film But, you know, 5 minutes before that shot, uh, the hotel phone had rang and
2:25:34
this is the room I’m staying in in Hong Kong and be like, “Hi, Mr. Snowden. I’m calling for the Wall Street Journal.” And I’m like, "Uh, you have the wrong
2:25:41
number. Why?" So, actually, [laughter] I am uh, you know, panic, but
2:25:48
I think the anxiety comes through a little bit in that that scene. But, uh, it’s there’s peaks and valleys, right,
2:25:55
and moments of call. The main thing is that you realize you’re becoming a part of something bigger than yourself. Uh
2:26:02
and so you can kind of feel a sense of satisfaction regardless
2:26:07
of what happens to you as the individual. It becomes part of the becomes part of the solidarity. You have a love on the line, but you know other
2:26:13
people will continue to work no matter what happens to you. And that that is a comfort.
2:26:19
Yeah. Thank you. Um, I think we can open it up to audience questions at this
2:26:25
point. Um, can we put the lights on so I can see all the hands that are um
2:26:31
popping up into the air right now. Um, and we will start uh way at the back um
2:26:40
over there. A person with glasses. I think I saw your hand uh first. Yeah, you um is there I think there’s mics on
2:26:47
the chairs themselves. If you do those work, can I get a yes from the tech
2:26:53
team? Yep, they do. Press a button.
2:26:59
Firstly, thank you to all of you. Now, I feel like an unfortunate truth of being in this room is knowing that
2:27:05
there’s a significant portion of people that despite everything that was said today would still take an internship at
2:27:11
Google. And I think it goes back to one small bullet point on the presentation
2:27:16
you gave Vicram and that’s on kind of this idea of the tech worker mentality.
2:27:22
You know this idea that you have to prove yourself the idea that you have to be the best, get the highest marks, do the coolest research, whatever that
2:27:29
might entail, even when that entails genocide. I think my question is is kind
2:27:34
of whether it’s to yourself or whether it’s to the people around you, how do you go about kind of unmaking this tech
2:27:42
worker mentality? Yeah, I’m a tech worker by the way. Sorry. [laughter]
2:27:48
Unmaking or making a new one, a different one that thinks. Yeah. Uh yeah. uh I mean I I I understand that
2:27:55
also different people have uh different like they’re in a different uh situation
2:28:02
uh right that I I wouldn’t actually judge somebody for taking an internship at Google or not but rather irrespective
2:28:10
of whether you take the internship of at Google or not you should be doing something to bring Google to account
2:28:17
right and that is a responsibility for all of us right and and the person who
2:28:22
takes the internship and goes inside. Google has a specific role whereas people who also like turn down the
2:28:30
internship have to make a statement to Google that they are doing it because of
2:28:36
their moral convictions and make sure Google hears it and other people hear it. Right? So uh irrespect of that I
2:28:44
think there is uh there is something to be done number one. Number two, the tech
2:28:49
uh um uh mentality is one I think of individuality in some sense. Everybody
2:28:55
thinks that they are the future CEO in some sense, right? So I mean that and that that’s how the tech tech uh like
2:29:02
industry works, right? Everybody is like thinks they are in the path to become the next executive. Um and would rather
2:29:10
identify themselves as the executive and would even say when we take some
2:29:15
actions, oh let’s not put the executive in like a delicate portion, right? Uh that really has to change. They really
2:29:22
have to identify themselves as a worker. uh right and and that uh and that that
2:29:29
they have they they have a role to play in in the sense of uh bringing uh
2:29:34
bringing back a moral clarity to the company which is not going to come uh from the executives. So uh I think it’s
2:29:41
a work in progress. It’s also political in nature. uh we have to politically educate uh students and workers and to
2:29:51
like know where they stand and whom they identify themselves with and why they
2:29:57
should they should do these things, right? So, uh there’s a lot of work that needs to be done. So,
2:30:04
yeah, I mean I I would say again I I I signed up for the CIA. Um
2:30:11
I’m not going to tell anybody not not to sign up for Google. The question is what you do with it, right? Uh like there
2:30:18
there is a level of economic procarity at which it’s very difficult to have any
2:30:23
political influence in which it’s very difficult to organize in which case it’s very difficult to sacrifice anything
2:30:29
because you don’t have much in the first place. If Palunteer is offering you an internship for $400,000 a year or
2:30:35
whatever and you go into Palunteer and you see what’s happening and then you know two years later you turn around and
2:30:42
you write a book or whatever that exposes it. Was that net good or net bad? I don’t know the circumstance. I
2:30:48
don’t know what they’re doing. I don’t know what your involvement is. Uh but I have a suspicion that typically, you
2:30:55
know, from having seen what I do, they don’t put you on the most sensitive project from the start. So, [laughter]
2:31:02
you go into these things and you get a chance to look around and see what’s going on before they, you know, put you at the keyboard of the orphan crushing
2:31:09
machine. Uh, and then you have choices. You put you at the keyboard of the
2:31:14
orphan crushing machine. It makes it a lot easier to decide what to do. Um, but yeah, I I mean I I don’t know. The main
2:31:21
thing is I don’t want to say judge people based on where they work. Uh,
2:31:26
judge people on what they think, what they do, what they say. There are a lot of people at DNSA um who I know are
2:31:32
absolutely irredeemable but then I know there are a lot of people who are actually good people to do the right
2:31:37
thing trying to make the organization better trying to shape it from the inside and I think the world needs all types uh but the main thing that I will
2:31:44
caution you on for anyone in the room who’s going into one of these organizations write down what you
2:31:51
believe and why you believe it before you get there because institutions are
2:31:57
designed to shape people to fit whereas institute or individuals are not really
2:32:02
designed to shape the organizations that they go into. That’s not where the the the power imbalance is sort of spread or
2:32:09
allocated. Uh so you have a lot of people who go into an organization believing one thing and then you know
2:32:17
400 mandatory trainings and agency all emails other afterwards they believe
2:32:23
something very different. Uh, and it’s important to say not just what you believe, but why you believe because
2:32:28
it’s very easy to look back on what you thought about years ago and go, "Oh, if I just if I understood what it was
2:32:34
really like back then, it would have been different." Uh, you want to
2:32:39
understand the reasoning and connect with the future self. Um, so that you actually remember just a little core
2:32:46
about yourself because people change. Remember, I went into these things. I was basically a true believer uh with
2:32:52
far lesser ethics than everybody who’s in the room right now uh because you’re attending these things I wouldn’t have
2:33:00
uh but uh yeah I would say just keep the sense of self and remember that the
2:33:06
organization will araid that and try to replace that.
2:33:13
That’s a really great very practical tip um that we could all take with us um if nothing else. Yeah, that’s something I
2:33:20
think uh one of the like biggest things like one of the things that I really value a lot is when
2:33:27
people in hindsight also get to a different conclusion like grow and
2:33:33
sometimes I feel like as a society if someone gets to a certain conclusion a little later than others um we don’t
2:33:40
value that as much as we should because it’s one of the hardest things to say uh
2:33:45
in hindsight like I was wrong. Um, yeah, that’s more of a meta, not not that meta, the other meta. [laughter] Yeah.
2:33:51
Comment. Um, do we have a question from online?
2:33:57
Uh, that’s any any questions that popped up on this is uh streaming to loads of
2:34:02
viewers, by the way. Um, otherwise we’ll take one more from the room before I wanted to go oneonone.
2:34:11
Oh, here we go. So for everyone on the panel, what are the most powerful ways we can organize
2:34:16
together at a grassroots and community level to resist mass surveillance and build collective digital defense?
2:34:25
That’s a very small question. [laughter] I actually want to jump in because I I go long and I can do this one very very
2:34:32
briefly. Uh, I’m out of touch, so I don’t want to advise people on on most things. But the most powerful way to
2:34:39
organize together for resisting mass surveillance or anything is to organize in the first place. You need to build
2:34:46
networks. You need to talk to people. The thing that made this hardest for me was I was alone and I couldn’t talk to
2:34:51
anyone. I literally couldn’t tell my romantic partner because if I did, they would be an accessory to the crime. Uh,
2:34:57
so they learned about this from seeing it on the news and that was very difficult, very isolating. Most people
2:35:02
aren’t like that, right? If you’re not at the CIA, if you’re not at the NSA, you’re not worried about going to prison
2:35:09
forever, you’re worried about maybe there’s a lawsuit or whatever, or maybe you lose your job. The stakes are lower.
2:35:15
Uh but there’s still real stakes. and to not be alone, to have people that you could talk to, to have people that you
2:35:21
can trust, to have people that can help you, to have people that you can help in turn, even if you’re not doing anything,
2:35:27
but you know, sitting on the couch earning a salary, working for other people, doing whatever uh that
2:35:32
organization uh to avoid the isolation, just to reach out and connect and to talk, that’s everything that matters for
2:35:39
anything. Super. So community organizing or
2:35:45
talking talking to people um community can help us all stand firm um in what we
2:35:51
believe in. Anything you would like to add Vicram? Uh I mean yeah I mean we try to also
2:35:57
organize with uh other groups with intersectional interest be it like
2:36:02
against building data centers in general or or try to even learn from other
2:36:08
groups that are trying to bring down flock cameras for example, right? Like so there are
2:36:13
different groups who are trying different things and there’s definitely things that we can learn from each
2:36:18
other. Yeah. Yeah. And [clears throat] I and I guess a broad variety of kind of appetite for risk also. So you can build an alliance
2:36:25
that maybe has different functions in that way also. Yeah. Yeah. I’d just like to add that building a
2:36:31
community doesn’t necessarily mean uh that you have to be friends with everyone um or you don’t have to like
2:36:37
everyone or you don’t have to agree with everything they think. like you know this purity cult that sometimes happens
2:36:44
in in organizing I think that sometimes is a little hard to actually then
2:36:50
scale up and move forward. Yeah, super important point. Um back to the audience. Uh we have a question
2:36:57
right there in the middle. Not sorry not you with a white shirt. You were a little bit slower than the woman behind
2:37:02
you. Please go ahead.
2:37:07
Um, if you guys don’t mind, I wrote down my question, so I’ll just read it from that. One of the effects of surveillance
2:37:14
AI being used on mass by governments is the erosion of public trust, particularly in democratic societies.
2:37:20
And one of the popular ideas being promoted now is a push for the slowdown in AI development to monitor the
2:37:26
progress to monitor the usage and evaluate exactly how AI and democracy and human rights can exist handinand.
2:37:33
But AI slowdown is also widely perceived as an impossible endeavor partially
2:37:39
because of this lack of trust in the local government as well as lack of trust in the international community to
2:37:44
adhere to the sentiment. I think many of us here and I’ll try not to speak on behalf of everyone um here either work
2:37:51
with AI or even work directly on AI and are not sure what to do and what to advocate for in the pace of the
2:37:58
development of AI. What would you what would be your response?
2:38:04
[gasps and laughter] I have so much to say on that topic also. You can totally answer this one. Uh
2:38:11
I’m gonna bust out of my my moderator hat um for a second. I mean the the
2:38:18
slowdown of AI has been something that people have been arguing for for a very long time before the the kind of tech
2:38:25
the leaders of the big tech companies um themselves have started to call for it. Um and it kind of feels a little bit I
2:38:32
mean there’s obviously a lot of theories circulating online about why now why they’re doing it can be tied to all
2:38:37
kinds of financial things happening as well can be tied to you know regulatory capture as a lot of people want to
2:38:43
definitely want to argue um so that’s you know for sure something to kind of look at a little bit more critically
2:38:49
right is it is it really coming from the right place here um and also look a
2:38:54
little bit at kind you know what’s the kind of background for the the slowing down right they
2:39:00
um let’s say one narrative which is this idea of a super intelligence that if we don’t prevent AI from recursively
2:39:07
learning um and therefore you know all of a sudden exploding into this or becoming this AI god in some way or
2:39:14
another um uh then we’re going to face a bad situation. I mean a different way to
2:39:19
kind of understand it is um you know they’re headed that there is this kind of obsession with scale right to kind of
2:39:26
achieve scale at all costs because they have this idea that scale in an LLM
2:39:31
model equals super intelligence one can definitely question that
2:39:36
hypothesis as well um but that is what’s driving a lot of this kind of like a
2:39:41
movement towards um towards speeding things up at all times. Um going back to Tim Nick Gabru the the um tech lead at
2:39:49
ethics the ethics um AI lab at Google um what she’s been arguing for for a very
2:39:54
long time um is actually to start you know rewind from that kind of pseudo
2:40:00
superstitious direction and actually come back to sound scientific practice which is to actually analyze the kind of
2:40:07
causes and effects and get better understanding of that and also set very clear R&D goals what are we trying to
2:40:14
achieve with these models that we’re developing and if they’re not achieving that then something’s wrong with them
2:40:20
and we need to do it better right so kind of really come back to a bit more of a rational approach to to how we’re
2:40:26
developing this tech um I think Ed made some good points on on that topic as well during in in his talk earlier um so
2:40:33
yeah a slowdown but I would say for a diff from my perspective a very different reason than than the kind of
2:40:39
narrative that that uh the big tech companies are trying to put out there because that clearly benefits them um in
2:40:45
many ways. But um I don’t know if anyone else has anything to add or if we I mean I feel like currently the
2:40:51
discourse around AI and especially like rogue AI um is is very much the the
2:40:56
discourse of acts of god. We talk about it like you know a natural catastrophe that’s just happening to us. Um, and it
2:41:03
really helps to me. It really helps to go back and think like, okay, it’s a product. They’re they’re products. We’re
2:41:08
talking about products here. And I heard an analogy a few days ago, I think, in a podcast. I really don’t remember by whom
2:41:15
but like if we see in other industries you know someone building a construction
2:41:20
company building a bridge and the bridge crumbles and they’re not like we’re not going to
2:41:26
be just satisfied with them saying like oops sorry it did it all on its own. No,
2:41:33
we’re going to say you built it and they’re going to say we built it. So yeah, in the end take take down that
2:41:39
sort of um mystic magic thing and just talk about products again.
2:41:45
Um so in terms of what should be done you know like slow down like the
2:41:51
companies are not going to kind of self enforce that right but there is existing laws and if the existing laws were
2:41:56
applied properly we wouldn’t be in this situation um from my perspective but Ed please do go ahead I interrupt you.
2:42:02
Yeah. No, no, no. You were you were speaking. It’s um what I was going to say is um
2:42:08
basically that like we’re they are playing a shell game and the intelligence community used to do this
2:42:14
famously which is why I smell this and it kind of irritates me. Uh they want to talk about the
2:42:22
hypothetical threat of a you know future possibility
2:42:28
as opposed to the concrete harms that are happening right now. And intelligence agencies love this.
2:42:34
They did it when I came forward. They were like, "Oh, again, if you talk, if you print our secrets in the newspapers,
2:42:40
all of these bad things can happen." Did they actually happen? No. Were they ever going to happen? No. Uh because everybody involved is trying to avoid
2:42:47
the harm. Like if you see it, if you know what’s going
2:42:52
on, it’s easier to control. Now, you know, more complex the different context. It’s
2:42:58
not directly transferable. uh but they are trying to avoid the liability
2:43:03
conversation uh which is really what this is like. I
2:43:08
do believe they are reasonably concerned about future
2:43:14
capabilities of future models. I do think the improvement is going to come
2:43:19
much more rapidly than people are prepared for and we’re going to have a lot of disruption and trouble as a
2:43:24
result of that. Uh I don’t think the proposal is realistic uh or that it
2:43:33
would advantage any in terms of the public so much as it would advantage the
2:43:38
companies that are currently the market leadersh which is what they want. It’s about where where do we concentrate
2:43:43
power right the power is coming the power is growing how will it be distributed uh and what they are seeing
2:43:50
right now is that the frankly Chinese research community is about to seize the
2:43:58
means of induction [laughter] and uh they are making models that are
2:44:06
80% as good for 20% of the cost uh and they are very worried about that
2:44:12
because they don’t know if that dynamic is going to continue. They are scaling things 10 times, 100 times, thousand times greater because China is compute
2:44:19
constrained. Um, and China is compensating by basically being more efficient, being more careful, making
2:44:25
trade-offs where it advantages them. And I think that’s going to continue. It doesn’t need to be a nationalist
2:44:31
argument because other people are going to catch up. I uh think we are going to
2:44:36
see these models and these capabilities spread. we’re going to see them become more advanced. Um, but we actually want
2:44:43
that. We want this to be distributed widely. Uh, what we need are we need advanced models with advanced
2:44:49
capabilities to come forward quicker that are focused on securing against the
2:44:56
predictable threats that we can all anticipate things like in a software vulnerability. And we actually have seen
2:45:02
some positive pro-social uh projects sort of come out of these companies to address those things. Is it enough? is
2:45:08
it fast enough? Not really. Uh because they’re they’re focused on other things. Um but there’s complexity here and I
2:45:14
think we need to understand that. But I also think and this is where it gets a little bit wacky. Um I think humanity is
2:45:21
changing and I think it’s about to change. I think during my generation we got the internet and then we got
2:45:28
smartphones. And when you put the two together and you abstract them out from the device, what we’re talking about is
2:45:34
functional telepathy, right? You know everything that’s going on in the world all the time. You’re connected to everybody. You have a yelling machine
2:45:41
shouting in your head, in your pocket, right around with you all day. Uh that changes what the human experience is uh
2:45:47
in a meaningful way. Does it make us a different species? No. But you see a lot
2:45:52
of these labs giving these things access to the internet. You see them giving them
2:45:58
access to sensors. They’re talking about giving them access to laboratory equipments to run their own scientific tests. uh all of these things are easy
2:46:06
and they will be done even if you ban it. Uh so the question is where does it go from there and the real functional uh
2:46:13
limit is we can’t interface with these uh machines with these technologies with
2:46:20
the internet right now without this external interface that we can just put
2:46:26
away that we could walk away from um but I think within human lifetimes today uh
2:46:32
particularly for the younger people in the room we are going to start seeing these technologies basically beginning
2:46:37
to interface with our brains. Uh, and it’s just like we’ve seen with these LLMs. Originally,
2:46:44
when we talk about the safety conversation, nobody thought anybody would be stupid enough to hook one of
2:46:49
these things autonomously up to the internet because it’s very obvious that they would start hacking things and coordinating and doing things because
2:46:56
why would they? They’re modeled on our behavior. They learned from us and that’s what we do. Uh, so of course the
2:47:04
mimicry machine is going to mimic our behavior. Uh, but are people going to be
2:47:09
able to resist this when they have the capability? And will they have the capability? I think we’re running out of time. Sorry, Ed.
2:47:15
We’re running out of time. And, uh, I I would like to get just another couple questions, one from online and one in
2:47:21
the room just so we have a chance to answer a few more people because there’s so many hands up in the air here. So,
2:47:27
but thank you. Um, one more question from online before we go back to the room.
2:47:35
Um, what is one thing everyone in this room should do differently tomorrow? Is
2:47:42
there some little habit that helps make make us helps us make better decisions
2:47:47
or be a better person?
2:47:54
Talk less, read more. [laughter]
2:48:03
Uh we’re we’re taking questions from the audience. Um one last question and then
2:48:10
we’re So um there was one person that I saw that has had their hand up the entire time. I think you from the very
2:48:16
beginning. Yeah. Yeah. Uh all right. So not only do I like to
2:48:24
use two in one shampoos, but uh it applies to the same principle applies to questions. So is it okay if I use uh
2:48:32
make two questions? Let’s go. All right. Uh first of all um after your
2:48:38
speech person I feel pretty pretty disgusted to even use Google now with
2:48:43
all the things that they have done etc. What are certain measures that we can do
2:48:50
as students or people that don’t have as much power button? Oh sorry. Uh yeah. Nor
2:49:01
Ah, keep a press. Yeah, keep a press. All right. Uh or I will [laughter]
2:49:09
No. All right. Perfect. So I will pause it uh again. Yeah. [laughter]
2:49:15
All right. Uh what can we do as students or as people with no power individually
2:49:22
in order to make a difference besides like uh organizing etc. And how can we
2:49:28
balance speaking up or you know uh speaking up the truth
2:49:35
without uh you know ruining our lives or ending up in Guantanamo or so please
2:49:41
please excuse my English but [laughter] all right uh I mean I’d like to answer
2:49:47
by saying I mean you you did answer it yourself like I mean you should organize you should you should speak up uh form a
2:49:54
community I mean those are the things that we are saying like will lead to like a better society and more better
2:50:03
outcomes, right? Uh and be strategic, right? Uh trying to boycott Google is
2:50:09
probably not as like not using the product is not going to be that
2:50:14
effective like try to find like that’s why we try to target things like their reputation. If these companies end up in
2:50:21
a conference called AI for good, uh that’s a target that we can go and uh
2:50:27
confront them. Uh right and and I would just like to say it’s practice, right? I’m not saying any of us are perfect in
2:50:35
doing or organizing or speaking up. Uh right and it’s a practice. So start
2:50:41
today, right? So start practicing.
2:50:46
Um, Ed, would you like to respond? No, no. I mean,
2:50:53
I I don’t like to prescribe things for other people. Basically, you have to care. You have to look for the problems.
2:51:00
Uh, and you have to always have a plan. Uh the the reason what I did worked uh
2:51:06
in a meaningful way as much as getting the story to the journalists was I have
2:51:11
to tell you the journalists loved the escape story. That’s what the global media drove 24-hour coverage of. They’re
2:51:19
like, "Oh, he’s in the airport. Oh, he’s traveling. Oh, he’s going to get away with it. Oh, it’s terrible. What are we gonna do about it? They’re gonna break
2:51:25
down the plane. Obama is in Africa, you know, at some summit." And they go to him for comment. says, "I’m not going to
2:51:32
be scrambling jets for some 29year-old
2:51:38
haler." Uh, and then like three days later, he downs the jet of the Bolivian president,
2:51:44
right? There’s drama in that and people love it. But the main thing is nobody is
2:51:49
going to be able to save you. Uh, ideally, you’re not going to need saving
2:51:56
uh because you’re going to have an idea what to do. It’s going to go wrong. things are going to go sideways. You
2:52:01
react, you adapt, you do what you can. And maybe like maybe things end bad for
2:52:07
you. But it’s that thing that I was saying about what are we here for? We
2:52:12
become part of something greater in these acts. Why am I calm in the hotel room? I’m about to go to Guantanamo.
2:52:18
Uh it’s because you understand like there’s a baton pass. And if we have at least some sense of some kind of belief
2:52:26
in these uh systems of government, the way we protect ourselves,
2:52:31
we’re we’re part of a larger human. Uh and your voice will be heard and won’t
2:52:37
be forgotten. On that note, um we unfortunately do
2:52:43
have to wrap up the evening. Um Vicram and Margaret are going to be at the appro so you know everyone who had more
2:52:49
questions um do come up and chat. It’s a shame that Ed can’t join us for the appro after the event. Um, but it was
2:52:56
incredible to have you in the room, Ed. Thank you so much for joining us. Um, and I think the kind of big takeaway
2:53:03
here really is um, we do all have some element of agency to do something that
2:53:09
sits um, more correctly within our moral compass. Um we might not have the agency
2:53:15
to fix everything ourselves but as a community we can do things that make a real impact and make a real difference.
2:53:22
Um and in fact we have the responsibility to do that as people who are um either tech students or entering
2:53:27
the tech workforce specifically because the tech sector is so implicated um in
2:53:33
large-scale harms that are happening um today including in the genocide in in Gaza. So, um, with that, I would like to
2:53:40
give a huge thanks to all the speakers, um, and especially for, of course, your
2:53:46
own, uh, bravery and standing up for what you believe is right. Um, [applause]
2:53:59
applause
2:54:43
and and and and of course a huge thank you also to the
2:54:49
organizers of this evening, the ethics forum here at ETHZurich. Um I believe you also want to
2:54:56
applause
2:55:08
Thank you very much for coming here everyone. It is such an incredible honor to see this of this much of an
2:55:14
overwhelmingly positive response to this event. I’m struggling to find the words to express just how amazing it is to
2:55:21
stand here in front of you and to know that we managed to do this within a relatively short time frame to show to
2:55:27
everyone that we can believe in ourselves and undertake these great things like organizing this incredible
2:55:33
event. Uh first of all, we would like to thank our incredible speakers, every one of you for sharing your knowledge, your
2:55:39
experiences and your expertise with the crowd today. And unfortunately, we can’t really um give flowers to to Mr. know
2:55:46
them. But we have flowers, the bookcase for the rest of the for the panel. Thank you so much for joining us here today.
2:55:52
[applause] Thank you so much. Thank you very much.
2:56:06
[applause] And while I’m standing here, I would
2:56:11
also like to take a moment to thank the many volunteers and organizers that helped realize these events. Many of
2:56:17
them might be invisible, but they helped with the catering, the crowd control. You must have seen the human barrier
2:56:22
standing outside that uh informed you where to go and with the many other things that were required to organize
2:56:27
this event. Uh so, another round of applause for our organizing. [applause]
2:56:41
We would finally also like to thank GZnner LA for having our back throughout the organization of this event and the
2:56:46
many private donors whose uh sponsorship actually helped organize this event financially. Uh without their help we
2:56:52
could not have been standing here. Um but before I give the word to our co-president I’d like to very quickly say what we as the ethics forum are. uh
2:57:00
we are a platform intending to cultivate critical discussions at our university. We believe ETH and the broader Turk
2:57:06
ecosystem require these kinds of critical confrontations with how our uh
2:57:11
technologies that we develop contribute to the crisis we see emerge today but also how our skill sets can be leveraged
2:57:18
to mitigate them. We should believe in ourselves. I think many of you here will recognize yourself in the story that Mr.
2:57:24
Snowden told us about people shrugging or simply saying it is what it is. It might not happen in an underground
2:57:30
secret cafeteria. It’ll happen in the polymensa next door instead. But I think
2:57:35
we are all familiar with this experience and it was exactly this experience that made a group of students gather together
2:57:41
and found the ethics forum roughly a year ago just to give you an idea of how quickly you can get the ball rolling. Uh
2:57:47
but without further ado, I would like to give the word to our co-president Julia to share some inspiring words with us
2:57:52
tonight and then you all can go and enjoy the appro. Thank you. [applause]
2:58:05
Now I have the difficult role of standing between you and your upper ro. Um but it’s only three minutes I swear.
2:58:11
Maybe you can uh give them to me. I brought a quote from Robert Kennedy senior. Um
2:58:19
our future may lie beyond our vision but it is not completely beyond our control.
2:58:26
All of us will ultimately be judged and as the years pass we will surely judge ourselves on the effort we have
2:58:32
contributed to building a new world society and the extent to which our
2:58:38
ideals and goals have shaped that event. For the fortunate among us, there is the
2:58:43
temptation to follow the easy and familiar path of personal ambition and financial success so grandly spread
2:58:51
before those who enjoy the privilege of education. Few are willing to brave the disapproval
2:58:57
of their fellows, the sensor of their colleagues, the wrath of their society.
2:59:03
Moral courage is a rarer commodity than bravery in battle or great intelligence.
2:59:10
Yet, it is the one essential vital quality for those who seek to ch to
2:59:16
change a world that yields most painfully to change. And I believe that those with the courage to enter the
2:59:22
moral conflict will find themselves with companions in every corner of the globe.
2:59:29
We believe so too. That is why we founded the ethics forum because we all
2:59:34
are among the fortunate ones who do enjoy the privilege of education and we’re so close to innovation that
2:59:41
changes the world. The technology we develop literally changes the world. So
2:59:46
let us discuss how we are creating a space at ETH where you
2:59:52
can bring the questions that move you and you will hopefully leave the room with even more questions.
2:59:59
We are creating a space where you can discuss, listen, change your mind and maybe change it back again.
3:00:07
A space where you can bring your own hot takes and brave the disapproval of your fellows and where you can think about
3:00:13
topics that you have never thought about before. And maybe most importantly, a space
3:00:19
where you find friends who are not just academically ambitious, but morally ambitious, who you can call when after
3:00:26
following the temptation of financial success, you find yourself being the IT guy of some company and the emperor
3:00:33
looks quite naked to you. [laughter] friends who encourage you to use your
3:00:38
talents to contribute to this new world society in a way that makes you proud,
3:00:44
who inspire you while you shape your values and ideals. Because isn’t that what defines who we really are? Thank
3:00:51
you. [applause]
3:01:01
applause
3:01:07
[applause] So that wraps it up for tonight. Thank
3:01:13
you all so much for coming here. Uh please follow us on our socials, come to our weekly ethics forum. Reach out to us
3:01:19
over email or send us an Instagram DM, whatever way you prefer if you want to work with us. We’re always open to
3:01:24
collaborations or organizing these events. Again, but please go out and enjoy the APO with our amazing speakers.
3:01:30
And thank you again for coming here. [applause]
