Weaponized Design and the Digital Identity Event Horizon with Cade Diehm
The Blockchain Socialist | 2025-10-21 | 1:39:07
I spoke to Cade Diehm, Head of Research at the World Ethical Data Foundation and founder of New Design Congress, an independent research group confronting the gap between what is said to be happening and what is actually happening in digital societies. His career also involves working at Deloitte Australia, as leading design for CoinJar, the largest crypto exchange in Australia and Tactical Tech. I had him on to talk about his work on Weaponised Design (when an interface or system harms...
Top Keywords
No salient keywords identified yet for this episode.
Transcript
Speaker 0
0:00 – 0:14
We were seeing things like clients who would big clients who would have databases that we had full access to, that we shouldn't, in fact, had access to. And the attitude throughout tech at the time was that encryption was just too hard. Web The
Speaker 1
0:27 – 0:44
things that we're finding appalling about technology are not simply technology's fault, I guess, but is one of the symptoms of socioeconomic inequalities, atomization, basically all these downstream effects of our economic system.
Speaker 0
0:44 – 0:59
This element of, like, bad outcomes as a result of a technology introduced into a society. Digital identity would create brittle societies because the societies would inherit the social engineering vulnerabilities of these digital identity systems.
Speaker 1
1:00 – 2:34
This episode is sponsored by NIM, the world's most private VPN that protects your Internet traffic and metadata. Unlike traditional VPNs, NIM uses a decentralized mix net to scramble your Internet data, hiding who you're talking to, when, and how often. You can switch between full mix net mode for maximum anonymity or a faster VPN mode for everyday use. Pay in crypto or fiat, and even your payment stays anonymous thanks to z k powered anonymous credentials. Take back control of your online life at nim dot com. Sign up today using the code blockchain socialist and get an extra month for free. Hi, everyone. You're listening to the Blockchain Socialist podcast, and I'm here with Cade Diem. He is the head of research at the World Ethical Data Foundation and founder of New Design Congress and Perea Limited, which is research and design, research lab. And I met Cade, a couple of weeks ago at the Cypherpunk retreat, where we were exploring and talking to some of the, I guess, biggest people who are working in the privacy world or working on technologies around privacy. And we had a really fun and interesting conversation while there, and I had to have him on. He had just amazing rants that at me, I really respect someone when they tell me that I made them angry when they listened to my podcast, and they wanted to criticize me. So I had to have him on so he can criticize me to my face and see Oh, no. Tell me what everybody. Why I'm wrong.
Speaker 0
2:36 – 2:49
Thank you for having me. I appreciate it. And, also for indulging me in the in the cyberpunk retreat rant. That was extremely gratuitous on my behalf. So I appreciate the fact that that I had an audience that didn't judge me so harshly on it. Yeah.
Speaker 1
2:50 – 3:46
No. It's fine. Honestly, I'm I I really enjoy it when, like, at least what I could tell from your rants is that you knew what you were talking about, and it wasn't, I've had many people rant at me where they didn't know what they were talking about, I felt. So I enjoyed I enjoyed being ranted at. But maybe before we get into the rants and what they were about, if you want to give your own proper introduction to to who you are because you have a pretty extensive history of working in the tech world, both from the private sector and, I guess, you can call the pry the public sector, the think tank, nonprofit side. And, yeah, if you want to give an introduction to that, one of the things that I thought was really interesting looking at your resume was that you also, just like me, used to have a a past corporate life at Deloitte in which we both, at this point, have have turned away from that life. So we'd love to hear your maybe if you wanna tell your story a little bit. It's a long,
Speaker 0
3:47 – 13:11
long, long time ago. One of the first jobs I had out of university. So I went and did a master's degree after my undergraduate, in generative interfaces at the Monash University in Melbourne, Australia. And I got a research scholarship for it, but did not finish for a variety of reasons. Made it all the way through to the end and then didn't make it because at the number of, like, logistical reasons, which is something I'm not happy about to this day, but that's fine. And one of the first jobs I got out of my university that I had to start I ran out of money. I had to start working with. I got the corporate job at Deloitte through a friend of mine as an analyst, which was equal parts Deloitte had just bought a a major Deloitte Australia had just bought, like, a major digital services firm and a few years prior and had integrated it into their offerings. So it was sort of partly research based in looking at digital systems as they were developing around the time of the first iPhone combined with executing on web properties, early applications for, like, large blue chip companies, these sorts of things. And there was nothing specific about Deloitte itself that turned me away from it. I mean, the work that I do now is extremely it's much more nuanced than what you would be able to do. There's no space for the kind of work I do at a place like Deloitte or any of those, like, larger consulting firms, I don't think. But in terms of the thing that made me leave was that we were seeing this is, again, super early in in in the realm of iPhone apps. We were seeing things like clients who would big clients who would have databases that we had full access to that we shouldn't, in fact, had access to. And I'm not suggesting that there was any misconduct or anything here, but even just the fact that that was seen as a normal thing across not just Deloitte, but all of the larger consultancies that were doing the same kind of big digital work at the time was really was really something that bothered me. And this was, you know, around the time that the the Assange stuff had happened with collateral murder, the the video that they leaked through Chelsea Manning. The was right on the cusp of the Ashley Madison hack. There was the iCloud photo hack coming up. All of these things were happening. And the attitude at the time, again, which is not specific to Deloitte and not specific to any one team, but rather was pervasive throughout the throughout tech at the time, was that encryption was just too hard. Securing sites and things like that was just too hard. Again, this is a time period where, like, let's encrypt didn't exist. Right? We're talking, at this point, signal doesn't exist. Telegram doesn't exist. I mean, not that Telegram's secure, but whatever. Like, not this is another world. This is like a pre right at the cusp of the of the you know, WhatsApp was, like, 99¢ in private and not owned by Facebook and not end to end encrypted. And that was really something that bothered me substantially. And so my first real collapse of of trust in the system was not being able to reconcile those two things at the same time. Like, the idea that we had access to, in many ways, a lot of expertise that wasn't available in other places. And yet, for some reason, I was, you know, a junior in this massive white collar meat machine, and I couldn't I couldn't get people to pay attention to it. And, of course, like, again, like, there's lots of reasons for that. It's not specific to you know, I'm I sound like I'm hedging, and in some ways I am because I think that it would be it would be the wrong thing to take away from this to think that if it was, like, unique, that there was something uniquely bad about my time at Deloitte. It wasn't. I had it. I learned a lot there, and I learned a lot about navigating very, very complex structures socially as well. But Mhmm. It lot of sense in many ways, although at the same time, you know, I'll I'll I'll I'll immediately dismantle. I'll I'll do against myself immediately after this. It makes sense because I was a junior straight out of university. I had no idea what I was doing. At the same time, I was trying to unionize the design team there, which was, like, probably not a good idea to do at the same time. And so everything was, like like, I was seeing sort of, like, I I wasn't doing things the right way in that place. And as a result of that, like, I was, you know, they I was not paid, like, I those concerns were not paid attention to. And and also, you know, quite frankly, I was young. I was super young. I was in my very early twenties. I thought I knew everything about very specific things around encryption, things like that. And if I were me in that situation now, and I had some asshole 20 coming to tell me that, like, things were wrong, I would probably be as defensive as some of the people who I spoke to as well. Right? So it's it's really it it was a formulary moment realizing that I couldn't, like, make people care about private data in that universe, but it was, like, consistent across clients, across adjacent firms, across contractors, across everything. And I think that really, although I wasn't in a position at the time to do that gosh. I have a larger story about this too. It was really this is why I'm paying paying so much attention to it. It was really something that was right there all the time. And the reason why I'm saying that is because prior to that and how I got the job at at Deloitte was that when I did my undergraduate and my master's degree, I had full fee scholarships all the way through from my from my work. And that meant that when I that meant that my my work and my interest in in my academia was picked up by Apple, for example, and I got a full fees full full ride to the world worldwide developer conference in, gosh, 2011, I wanna say it was. Something like that. And there I you know, when you when you go in on the student scholarships at the worldwide developer conference, again, like an incredible experience from someone very much on the outside. You go into this world and you're being courted all the time. Right? Because Apple is you're getting like this pre stamp of approval from Apple. I mean, they don't know much about you except the application you put in and, you know, they go through, you know, they go through some of what you've done and things like that. But they don't really like, it's an early stage approval. Like, you know, I I guess in some ways maybe akin a little bit to like, you know, other accelerator programs like y Combinator and stuff. But in this case, you know, you're also sort of testing for even maybe getting jobs at Apple and things like that. And so I'm in I'm in San Francisco in the Bay Area watching, you know, Jobs do his last keynote as it turns out. And and I'm being given, like, these rides through, you know, meeting different early stage startup apps and things like that. I'm meeting, you know, senior executives at companies like Uber and and Airbnb back when they're first starting out, you know, not these exact companies, but companies of this caliber. And the whole time I'm sitting there, and I'm like, wow. This is actually there's something really really wrong about this. So like I have this tension point where on the one hand, I feel like I've been accepted into this world. It's the same with with Floyd. Mhmm. And I'm, you know, I could go anywhere with this stuff, and I could, you know, ride this wave that I could see coming, which was smartphone tech. And on the other hand, there's just something really, like, the the cavalier nature of everybody towards the the user is is really throwing me off. And at the time, I don't really know at the time, I don't know what what I'm what I'm even talking about. I just know that there's something in my gut that's telling me that, like, there's something really wrong here. And so from there, I I don't end up getting hired. I I do a bit a bit of work remotely for some of the people I meet in SF and one of the apps that I worked on I did a bunch of wireframing for this thing called Guitar Toolkit which like ended up winning an Apple award, but I did I mean I just did wireframe sport, but you know touched touched an Apple design award winner, which is pretty funny. And then I end up, like, deciding once I'm, like, realizing I need to leave Deloitte, I end up working at, one of the first crypto companies in Australia, which is a company called Coinjar. I was their first hire. So, like, the beginning of my career is is this period where there's so much going on that felt really off and kind of being able to and and just by sheer fluke, completely not being caught up in any of it while sort of knowing that something was up at the same time. So then I joined CoinJar. That was my first serious role. I was their first hire and, you know, for all intents and purposes that chief product officer on paper working on design and creative direction, but also helping engineering. There was a very good engineering director there too, but and we worked very collaboratively and we built with a team of like eight people, really talented group of people. We built like an entire early stage crypto platform for Bitcoin. And this is the era where like it's 2012, and it's super early crypto. And I'm of the opinion that this is not a speculative asset. I'm, like, trying to build things that allows you to create allows you to pay your rent. Like, in Australia for a period of time, you could pay your rent with Bitcoin, which was kind of cool. And I was really all in on the idea that this was around about this was about solving the issues of being unbanked.
Speaker 1
13:12 – 13:20
And This this was, like, through a separate platform you could pay your rent in Bitcoin? No. So there was this platform called Satoshi at Home,
Speaker 0
13:21 – 17:31
something like that. And it was it yeah. I mean, this is so it's so lame now. It's so it's so cringe, but it's like it was super early. Right? And and this is when Satoshi was like, the mystery was really there. You know? It was super early. I mean, to give you a tie an idea of the time frame because people memory hold this stuff, Silk Road was hadn't started yet. Like, Silk Road started while I was at while I was at like, it was it was really coming into being, like, like, Reddit had dark web subreddits. Right? Like Mhmm. This is all This is very early. Things have changed a lot in this time period. Right? And so, you know, we do this work, and we we put together and launched its entire platform. It's an Android app and iOS app. They're both feature complete and written in their native languages and not because, like, cross, cross platform software development sucks. PWA is not really a thing properly yet. And so we write we have a team of two working on Android, team of two working on iOS. And then, like, these absolute monster CTO, like, cofounder of CoinJaa, who is working on the back end of the thing. And then there's a small design team, and there's me and the engineering lead. And we, like, basically build this entire platform from nothing in, like, a matter of together in a matter of, like, twelve weeks. They have, like, a an exchange where you could, like, exchange between Australian dollars and crypto, but, like, that was it. And we turned it into, like, an entire, like, bank like a bank account equivalent. And then immediately after that, we went and built this thing called Coinjar swipe, which was one of the first bank cards that you could use. This is, like, at the same time as, like, Zapor was promising Zapor, whatever it's called, was promising their their debit cards in The US. I don't know whether we beat them or not, but we were like very very close. We were right there at the very beginning. But of course, it's only Australia, and it's EFTPOS, which is like an Australian payment and money moving system that's not really used in many other places in the world. Mhmm. But I have this extremely fond at Terminator two opening moment where very much in the same vein as John Connor, Edward Furlough furlough and it's John Connor. We go to a bank ATM machine with a totally blank, like, bank card, and we withdraw, like, a thousand Australian dollars out of a out of an ATM connected to a Coinjar account. And that fuck. That was, like, that was a moment where you're, like, wow. We're really, like messing with things here. Felt like particularly subversive. Right. And then yeah. And then basically everything that happened happened. You know? Like the unintended consequences of that. People thought that Bitcoin was private at the time, and they were you know, we had instances, not many, but enough to really bother me that we had instances where people were, like, posting on dark dark web subreddits, dark net marketplaces. The subreddit saying, oh, yeah. You can sell your, like, whatever it is that you're manufacturing. Like, you can sell your drugs through a dark net subreddit. Are you a dark web, like a Silk Road kind of marketplace? And then, like, you you know, get the Coinjar swipe card, you can cash out. And then we started getting, like, the Australian the feds coming in, like, you know, taking user data. And I'm sitting there, and I'm like, wow. These people really think that this is private. No way have we said anything is private. They have to submit their driver's license or their passport code. Right. And they get one of these cards. What what is going on here? And so what I discovered was that there's like this gap between what is said to be happening and what's actually happening in in any kind of digitized society and that that gap itself was like it could be it could have no real, like, malice involved in this process whatsoever. But like that just the fact just the fact that digitization was inserted into a previously human only system would lead you to these unintended consequences and also misrepresentations or misunderstandings of how things were actually going in the world. And that then that was like the moment I think where I realized that, like, I had to do something. I couldn't just stay within the realm of of product or work within this sort of startup up scene that I had to do something else. Should I keep going on this? It's like
Speaker 1
17:31 – 19:36
I mean I mean, I think that's, like, pretty interesting already. I think the you know, just hearing that, I think it's easy to fill in fill in the the rest of the story of, like Right. And then now, you know, focus on privacy, focus on a lot of the other work that you did. So maybe from there, I think it would be interesting. We could talk about, I think, one of your more, I think, looked at pieces of of work or concepts, which is weaponized design. Mhmm. So you've written quite a lot about weaponized design. You've presented quite a bit, and I think that'd be a great place to to dig into it more because I think, I mean, there is an easy connect, I think, from that everyone is aware of of, like, you know, addictive social media feeds or, like, even in in crypto, I think we're seeing it more and more weaponized design in terms of, like I think, you know, how easy and, like, frictionless it is to, like, buy speculative assets, which, you know, if you look at the marketing of, I don't know, Coinbase or, you know, Base, their their layer two blockchain. They make it sound like it's this is, like, an amazing thing how frictionlessly you can invest in things and speculate on meme coins and whatever else and that this is the future. And I think there is something to say for, you know, this being ultimately framed in a way that is neutral, which is, like, kind of, I think, the story of a lot of tech things that are framed as neutral as, like, an inherent good kind of Mhmm. In a design perspective as almost like a an appreciation of, like, wow. They designed a product really well. But in actuality, when it comes out to the public, when it comes to, you know, people actually start using it, it has all these unintended consequences. So Mhmm. Maybe to start off, I think if you want to give just kind of, like, a high level, what is weaponized design and what do you mean specifically Right. When talking about it. Yeah. So weaponized design is a system or an interface that harms the user whilst performing exactly as it's it's intended to. So whereas
Speaker 0
19:37 – 24:44
it's great that you brought up the the prevalence of dark patterns inside crypto web three properties, especially more recently. That is adjacent to weaponized design, but it's not necessarily specific to it. It's it's it's part of it sits alongside weaponized design, and when used together, it, you know, very effective. In the context of weaponized design, though, is things like when a platform begins to just broadcast locations of people and it's an opt out system. By by design, the unintended consequences of that are things like, you know, real world stalking and harassment and things like that. Now, of course, like this that example, which is one I use in in the 02/2017 essay, which is when it was produced. It's my first essay actually, which is yeah. It's, you know, a couple years after I've been working in this field for a while. The the example one of the first examples I use is Snapchat turning on this, like, always on location sharing system, which immediately freaks out its user base. Because its user base is not just people sharing their location to Coachella or whatever millennials are doing. It's also, you know, there's a huge sex work early pre only fan kind of community working there. Right? There's a marketplace for that stuff there. And so immediately, those people are at risk. Because the kinds of relationships that they have between what it means to follow someone when you're going to Coachella with them and when you're, like, following a performer of some kind is a completely different relationship with the dynamics that are extremely different. And so by flattening that through a design choice into the same threat model means that the outlet the risk becomes extremely outsized. I mean, it's risky for for the people at Coachella anyway because who knows, you know, what else is going on there with the location data. But for somebody in that kind of who is, you know, using the platform as something akin to, like, OnlyFans. It's very, very risky. And the denial of the team to understand that they have that diversity of use case on their platform is that ignorance is is a major part of that that kind of manifestation of weaponized design. I used in the in the essay, I used that. I used the Facebook timeline view and then how the timeline view had been made to be as addictive as possible, and then it was then used by a social sciences a psychology team at Facebook to see if they could create, like, emotional contagions where you could make people happy or sad based on what you've fed them through their timeline, which that paper was published. I can't remember in which journal it cited in the article on the new Design Congress website, but it's I can't remember exactly where it's from, but it was retracted as a result of ethical concerns about the the study really is a pre to everything that follows with regards to psy ops in the world. It's like the OG digital psy ops paper is this, like, unbelievably unethical weaponized design of the Facebook timeline. And so these sorts of things were what I was seeing, and I was was like I mean, the iCloud photo hack, which I mentioned before, is the other really big one that I used in the piece where I talked about how the the issue here wasn't wasn't that people that people broke into accounts. They were very easy to break into at the time because, you know, Apple hadn't fully grappled with the sensitivity of what they were dealing with. But the weaponized design component comes in the form of the treatment of every single photo in a camera roll as being equal and deserving of being on the Internet in in a so so the acclaimed to be private space. You know, the issue really here was and and one that's still to this day underexplored is that people at the time when I I when Apple turned on iCloud photo uploads, I had no idea that that meant all camera photos. Right? Like and I know that we know that. Mhmm. You know, especially if you're, like, deeply embedded in tech, you know that that's, like, the thing that we mean. Like, that when you say you're gonna back up your photo library, it's gonna be the whole thing. But, like but normal people don't work in those terms. Like, a lot of people if you're not familiar with the system, you're compartmentalizing your photos very differently to how people do how an engineer does that. And different photos in the taxonomy in your head have different they mean different things. They have different weightings. They have different levels of of of sensitivity. And it's it's that lack that flattening, which is the weaponized design component of it. Mhmm. Yeah. So that piece, I think, was it came out right around the time when people were talking about dark patterns, and it sort of filled in the gap of something else that was missing there. And that was really the start, I think, of what then became NDC in, firstly, in 2018 and then more so broadly into into this decade. Yeah.
Speaker 1
24:44 – 27:05
Mhmm. Mhmm. Yeah. I mean, I think if you what's interesting is that, at least for for a lot of people, of course, like, for example, the Cambridge analytical stuff or just, like, a lot of these things that these concerns around privacy kind of blasted off, I think, a bit during the first election of Trump, more or less, around that time. But there is, like, a, you know, a very clear line you can draw from from then to just, like, years to decades of research on how to do all these things that then were used or weaponized for, you know, by firms like Cambridge Analytica or whatever else via, you know, private markets. These are all private companies that were using data that they could leverage in order to, you know, coerce people into particular behaviors. And so, you know, I think it's, yeah, I I think it's a good time to maybe kinda move on to my next question, which is the particular presentation that you gave. I think you gave it in Osaka during Dev Connect as well. I saw a different recording of it, but it was called Ethical Design. No thanks. So I was curious to hear your thoughts on what is the issue with ethical design to explain a bit about that from that presentation because I think it's interesting that the kind of, I guess, I don't know if this is the right term, but kind of, like, almost like centrist solution to dark patterns is ethical design or just, you know, designing ethically is kind of like, some sort of creed to to follow when you're when you're designing systems. But, I think you you take a few problems with that. And one of I think one of the one of the examples I just wanna point out that you had in this presentation is where you showed a baby, a video of a baby crying after parents took away the smartphone from them. So the baby was just kinda, like, looking at the smartphone and then the parents took away and the baby just, like, kinda went ballistic and started crying like crazy. So I thought that was a really interesting kind of exposure of kind of the, again, I guess, unintended consequences of simply making what seems to be neutral design of products like smartphones and how that can, you know, intrude on, you know, the developmental capacities or the experience of life for babies.
Speaker 0
27:05 – 32:57
Yeah. So the that's the that piece, by the way, is, like, the starting point, the real moment where I knew that new design congress was gonna be a real thing, and I gave it at Osaka. To the credit of the web three community, the web three community has always been very, very, very quick to embrace and fully engage with the work that I do and my team does. And the starting point there in, like, October 2019 is absolutely no exception. For starters being invited to talk about that topic at all, but then also the subsequent discussion that happened as a result of that was only really clipped by the pandemic, which follows, like, in, you know, 2020 almost immediately afterwards. The the the talk makes a central claim, which is that by focusing on ethical design, both in terms of decision making processes that design teams have go through and also the makeup of those teams. What you're doing is you are almost in a sense running distraction against the structural problems of digital systems. And that that's because all of the discourse around that time and this is like your Tristan Harris Center for Humane Tech kind of part of the world. It's also to a great degree related to the the ethical algorithm algorithm work that was going on in that decade, that sort of last half of the decade as well with teams in, like, you know, Google and Twitter and things to like, putting out papers around the ethics of recommendations engines and stuff like that. And the problem the criticism that I had of it all was that, like, it was technology is an accelerant for existing conditions. Right? Which is sort of the kind it's almost tiptoeing dangerously close to a Nickland kind of accelerationist theory. But the context here isn't so much that that is celebrated, but rather it is true of any technology in any context in any point in human history. That the the introduction of a technology, whether it's computerized or not, accelerates the conditions, the good or bad, of that society. And there's nothing more to be mythologized or read into or pathologized from that statement. In the context of, you know, the second half of the decade, we had, you know, Tristan Harris putting out material about the dark patterns of Netflix and YouTube making kids into, like, zombies. We had the QAnon stuff that was, like, all the rage at that point in time. And, you know, we thought was gonna be, you know, the peak of it before COVID hit. We had all of these systems in place, and and the the the talk really dives into that work. It talked talked about smartphone addiction as a kind of a symptom of time poor parents. You know, this is an example, and don't read this as a traditionalist argument, but this is an example of societies where single income households become dual income households, and then that isn't squeezed by a runaway economic cost in terms of not even necessarily inflation, just the amount of things that one has to pay for in a family of four accelerating over the course of, like, twenty to thirty years, and how the smartphone offers, like, a a simple way to for time poor parents to ensure that their kids have any kind of stimulation without being totally, like, neglected. It. Right? And the point is is that the the baby that you see who, like, screams for the cell phone, babies have, like, babies will scream. This is, like, used in in various arguments at the time of, like, well, this is an example of how, like, terrible the smartphone is. And, yes, there are components of that where this shiny you know, there are studies that show how, like, infant cognition changes over time when introduced too soon to smartphones. That that's a real thing that, like, people have written about and argued for. But to sort of say that that you look at a a video of a of a of a child an infant with a smartphone, and when the phone is taken away from them, they, like, lose their minds. To say that that is you like, a design ethics problem, it also it ignores things like the infant psychology, you know, such that you take away a toy elephant from them and they have the same reaction. It's a it's a it's an expression of frustration. But also that, yeah, as I said before, that, like, the the reasons for this cannot be held specifically by the team designing the tools because it's a symptom of a wider a wider problem within a society. I argued the same. There's three sets of videos that I showed. The first one that you identified is the baby with a smartphone. The second one is a live streamer in an ex like a Chinese live streamer in a really disgusting bedroom. And the third one is a woman on q and a on on a live stream, like, screaming about q and on. And in each of those, I then show here's the claim that, you know, this information makes you like this. People are addicted to their phones to the point where their their houses are messy. And then I like then I bring in a central argument that dismantles that so in the case of the live stream with the messy room I show how like they there's like interviews with twitch streamers really famous twitch streamers who say like if you don't stream you'd go broke the second later like this is the economy of the situation and then with the QAnon person, there was like a news report showing a deradicalized person, a young woman who says, like, I was totally lost until I found a conspiracy theory system, and then that was what brought me in was my total atomization. So really, the idea is, like, that the things that design ethics had you know, the the popular things that design ethics said were were caused by platforms and the design teams within platforms are actually symptoms of economics and atomization. And that by focusing in the last half of the decades of the twenty tens, by focusing on these as as design problems rather than structural problems of capitalism, was was to miss the point entirely at a time when we could still do something about it. And that's really what design ethics no thanks is about.
Speaker 1
32:58 – 34:03
Yeah. Yeah. No. I think it's, it was it's an important reminder that, like, you know, the things that we're finding appalling about technology are not simply technology's fault, I guess, but is a a larger is is one of the symptoms of socioeconomic inequalities, atomization, basically, all these downstream effects of our economic system. So I think it's it was it was a nice reminder of that. And I think that's it's an important reminder in a time whenever, like, I feel like, especially in this moment right now with around regards to AI, regards to I mean, even in crypto or just the in the past few years that, like, a lot of the problems I think stem out of these technologies are ultimately ones of of of capitalism, I guess. And then technology kind of on on top of that simply, they they ride on the already existing trends if you want to call them that or the the they socially reproduce faster the effects of of capitalism, I guess, is kind of how how I see it, I guess.
Speaker 0
34:03 – 35:50
I would call it so in my more recent work, I've started calling this a boom extract enforce cycle, where you start with a technology that promises liberation of some kind, the automobile, digital systems, whatever, even industrialization itself is presented and delivered as, like, a radical economic opportunity. And then it once it entrenches, whether through laying down roads or the investment into factories or the investment into cabling and infrastructure for digital systems, you then start an extraction process in that society where you reconfigure the society around exactly what you want with that technology. And then once you reach the plateau there, you move into an enforcement agency. So in the context of of of automobiles, it's the the rise of, you know, license plate scanners, speed guns, all these things that that make sense. Right? But are still, like Mhmm. Still a massive enforcement system proxy that didn't exist in the you know, before this system became ubiquitous. The same with digital systems. The enforcement systems follow as everything from surveillance all the way through to, the more insidious versions of this now, which is like, authoritarian regimes sort of cracking down on people speaking through the promise of free speech interfaces that the digital world was supposed to be. And, of course, in industrialization itself, in force is literally the formation of modern policing itself. Bedrock of all of this. And I think, like, what you're sort of saying here is precisely that system when you develop a new technology and you place it under this particular structure, you get those those cycles in repetition, I think. I think, structure, you get those those cycles in repetition, I think.
Speaker 1
35:50 – 36:28
Mhmm. So I had a I think your work kind of remind me of a conversation that I had with my friend Anuj Das Gupta from Smart Transactions, probably, I think, an interview about a year ago, where kind of like his his hot take was that designers shouldn't be allowed to work in the private sector, that they should only be allowed to work through their own guild or a union or some nationalized, like, entity as a way to kind of, like, prevent these types of things or that, like, to take seriously, like, the ethics of design. Designers shouldn't they shouldn't work in the private market. I'm curious what you think about that.
Speaker 0
36:30 – 39:21
I think so I'm not familiar with that conversation. Like, that was on the podcast. Right? Yeah. Yeah. About a year ago. Yeah. So so the I think there's a deeper thing that I would I mean, it would be so cool to talk directly about this. It'd be really great. The there's a deeper point here to be made how the tech scene, which has enabled a particular kind of crystallization of an economic and power structure was built off of the back of a deliberately atomized workforce Mhmm. That I think transcends not just design, but travels into the the wider tech worker field. And I I think very, very, like, directly back to the late nineties, early thousands. I can't remember exactly when it was, but there's this period of time where no. It was it was in, like, the mid two thousands when Apple's leadership and, like, all the big tech leadership in Palo Alto and and and the surrounding area had, like, a gentleman's agreement that they wouldn't hire each other's stuff. So they kind of artificially suppress the wages of this entire region. Right? At the at the exact moment when it was, like, when that area was just popping off. Right? And, like, I think that is a really underestimated factoid of the story of how Silicon Valley's power rose and how complicit the workforce was in it while simultaneously being like having the boot to the neck the entire time. And I think you could extend the idea of all designers should not work in the private sector to basically anyone who worked in tech. Because this entire industry until until Meredith Whitaker and her associates pulled together the Alphabet Union and and sort of fought against Apple. Nobody could do anything. Like, every everything there was no such thing as collective bargaining en masse in that industry, despite the fact that everybody was paid very well. So the risks were quite low because your salaries were high enough that you could withstand potentially withstand, retaliation from your employer. Yet there was no there was no large scale movement for many years. I mean, there were smaller efforts done in the Bay Area and things like that, but then nothing nothing at the level of, you know, what was eventually the Alphabet Union, Alphabet Workers Union. And I think that I think yeah. I think that that's a I think it's a great idea. I think that I just think you could extend it. Like, I think you could say the entire tech industry could could have Doesn't go far enough. Done. Yeah. It doesn't go far enough. That's right. That's me. It doesn't go far enough. Right. That's fair.
Speaker 1
39:21 – 40:45
So the next kind of, you know, you I think you mentioned it maybe earlier, but kind of a lot of your work more recently has been around digital identity. Yes. And you wrote this piece through new new design congress called, I think, digital identity, the event finding the event horizon, something like this. And this was also, like, a big part of discussions in at the Cypherpunk retreat. I think identity is, like, overall, like, this giant boondoggle of a problem in the tech connected to the state, connected to even private actors' world. Like, it it before it was you know, Bill Gates said that on the Internet, nobody knows you're a dog. But I think now on the Internet, everybody knows, like, your ZIP code, you know Right. Like, basically, at this point. So and digital identity is just kind of, like, I think this holy grail that a lot of different types of institutions, not just the state, really. I think, really, private actors really want this as well. So I kind of I mean, just to say, like, kind of the I think oftentimes digital identity can kinda be framed as this, like, the state trying to capture you or, like, or, like, only in a kind of, like, somewhat conspiratorial right wing type of way that I often see. But it's also, I think, something that private actors would love as well. But, yeah, digital identity, finding the event horizon, what is the digital identity event horizon?
Speaker 0
40:47 – 49:01
So we were a over the last four years, one of the core pieces of work we did at New Design Congress was this deep study into digital identity. And it the finding is part of a broader report called the digital identity event horizon. The finding I think you're referring to, if I just click this link here, is the m yes. The The identity accountability gap, which is one of key findings out of 10 that we had. I will jump back to that. But before I do, I'll justify what this project is and why we did it. So Mhmm. As we talked about throughout this interview so far, there's this huge focus on the unintended consequences of tech. Right? Whether there is consequences that have been downplayed or whether they truly people truly know what they were. There's, this element of, like, bad outcomes as result of a technology introduced into a society. And one of the things that's really prominent across all of those is the role of the representation of the self in those systems. Be it from the iCloud photo hack through to the Facebook news feed. The idea is that what that study I mentioned earlier on the contagions of emotional contagions within Facebook users, it hinges on this idea of using your identity in Facebook as a way of of carrying the contagion to other people in your timeline. Right? It's using the social graph of identities to create the mapping that that allows for their experiments on emotional contagions to, like, travel through the Facebook network. So what we so what I began to realize was that, like, digital identity is actually the most important thing and the most important battleground of of of any of the the discussions or debate around around digital systems. Because ultimately, what is the point of a digital system if it's not to represent people within that system? There are only a few use cases. You can narrow it ex to an extremely sort of select number of applications if you're not representing people in a digital system. So at the same time, if you look I mean, throughout, I mean just through tracing from my career itself. Like, if you look at the rise of of of the economic damage alone of social engineering attacks through digital identity systems, it's exponential and it's global. Right? So yeah. So, like, you know, digital identity is the primary vector for attacking digital systems and that's through social engineering. Just in the five year period alone up to 2017, US companies paid an estimated $1,600,000,000 as a result of social engineering attacks. And in that time period the the success rate for social engineering attacks jumped by 15% which meant that like by the time 2017 like this is almost ten years ago Three out of four social engineering attacks were successful. And US companies alone, just companies, were paying $1,600,000,000 per year in in, like, recovery for, like, lost economic damage as a result of social engineering attacks. Right? And the thing is that the twenty twenty twenties themselves have been, like, basically revealed themselves to be a decade of fragmentation and conflict. And with that comes blockchains, emergent biometric systems, and machine learning technologies, and all of those have this major impact on digital systems. And the basic premise of this report that we conducted over four years, the research, was that because we were worried that no current or emerging digital identity system were was capable of fully fighting against social engineering attack, that they would all remain vulnerable to social engineering. And as a result, as we began to insert these digital identities into deeper and deeper into our society, digital identity would create brittle societies Because the societies would inherit the social engineering vulnerabilities of these digital identity systems. And that's largely what we found in this in this report from four years worth of work. Now this goes from this goes for this covers everything from emerging web three identity systems all the way through to nation states scale ones such as Adhaar or the Estonian digital identity system, all the way through to existing other systems like the the Australian robo debt enforcement, which in which they the conservative government tried to recoup alleged lost or fraudulent welfare payments through automated identity identity construction, which was illegal and led to the suicides of, you know, multiple people as a result of in innocent people being targeted for these things. The the the the scope of what we cover in this is is is extremely broad and then extremely detailed. And the one of the main things I would say is that you mentioned the second finding, the accountability gap. This is like the the two first finding for the most foundational in terms of why we have this problem. The first finding is that we are in a situation where digital identity means different things to different people. There's no universally accepted identity definition. And that people tend to understand digital identity through either their first encounter with it, conceptually, or through their economic interests, which is fine because that's how a lot of things happen. Right? Like, that's fine. But the problem is is that if we're then using this as the first principle of, like, systems design, it has to have an uncontested start point, and it just doesn't have that. If you're somebody who believes that you're working on on a digital identity system because you're a web three person, you might think of the cure like, and you're you're sort of working in performative or game spaces. You might think that an identity system prioritizes the user's curation of themselves. If you're somebody who works in in fraud, you might think of digital identity as like a series of behavioral markers that allows you to understand whether someone is who they say they are or not. It like, we had a number. We had maybe 40 to 50 expert participants give qualitative contributions to the work. And what we found is that even those people who which ranged from people in the military, like, high level military people all the way through to CEOs, nobody had a, like, an agreed identity definition, and everybody's followed the same pattern, which is that it was specifically tied to their either their current work or their their education history with with regards to digital identity. That then leads to the second finding, which is accountability. If you can't identify something properly, you can't then create accountability around it. Because if it's amorphous and difficult to understand, you can't build any kind of legislative, punitive, or corrective measures to know what what what its responsibilities are and its liabilities and what they are not. And so you end up in the situation where a digital identity system means different things to different people. And then when it fails on us, whether through weaponized design or actual weaponization, nobody really knows who's responsible for that. And so it leads to these situations where, for example, biometric systems, when you pay for stuff with your bank account, are used to skirt if you then are a victim of fraud. Those biometric authentications are used to to bypass legislation that requires consumer financial institutions to be responsible for fraudulent payments. Right? And these are the laws that go back, in some cases, hundreds of years. And because digital identity and and what a biometric identity is and is not is not fixed, then you end up with these gray areas where companies are able to offset the cost of, you know, billions of dollars worth of fraud onto the public. And these are, like, tangible real world consequences that happen right now, happening like, as we speak. It is primarily the rise these sorts of accountability dodges that leads to everything from Musk walking in the front door with Doge and, like, wrecking havoc with Palantir on the federal government in The United States, all the way through to the insane amount of fraud that happens is on on everyday people as a result of of digital identity circumvention.
Speaker 1
49:02 – 52:03
Yeah. Yeah. I mean so the the thing that kind of popped up for me whenever I was reading that work is that I during my previous life as a corporate consultant at Deloitte, I actually did I thought that this was just an analogy. You actually worked with Yeah. Them. I did. Uh-huh. I was a strategy consultant at Deloitte. Wow. One of the projects that I worked on was at SWIFT, the giant banking cooperative that runs, you know, international payments. And they we were exploring digital identity solutions for them and, like, all these different banking for their banking partners because it's such, it's such a big problem. I mean, there's I mean, I could I could rant for for an hour about about Swift, but, like, the entire architecture that they have is, like, from the nineteen seventies. It was not really made in a way that, like, kind of predicted or could have possibly predicted the the situation that we're in now. It's so convoluted. It's so fragmented and divided. So they need some sort of, like, unifying identity solution because, like, so much fraud keeps happening through the existing international banking system set aside. You know, nothing to do with crypto. Crypto's not involved at all. There's already a ton of fraud in in in international banking and do the normal, you know, quote, unquote banking system. So they wanted a an a digital identity system, but, you know, these are all private actors that wanted something. And I don't know. I guess through the experience of that project, you know, I wasn't leading it. I wasn't the manager or anything. I I, you know, was still fairly early. It was to me, it just felt it just felt strange kind of watching them flail a little bit. I'd like trying to find a possible solution and just, like really what they wanted was a third party that they can put the blame on for if, like, for the actual identity solution because they can't they can't make their own. Like, to make their own would be like, you know, they would be blamed for it. Like, they the the the liability issues are just too big. But trying to pick out who's that third party was also, like, a a huge mess. Yeah. But maybe focusing a little bit on the crypto aspects of identity because this has been, like, a big a big part of crypto and kind of, like, also considered a holy grail in crypto for being able to unlock a whole bunch of different use cases and however you want you want to put it. There was an interesting example that you put in that article about the World Food Programme using a digital identity system, I think, based on a blockchain of some sort, not really sure, but called building blocks for meant for refugees. And this is something that, like, is a often, I mean, depending on what it is, it's like a little bit of a a a common use case, like, brought up as to how, you know, blockchain can help the global South or whatever. But I'm curious to hear, you know, from your research, what were some of the issues with this type of program?
Speaker 0
52:05 – 55:41
So what are some of the issues? Sorry. It's it's it's like I I'm I'm laughing not because of the question, but because I'm like, god. Where do I start? It's more about me being like, how do I even answer this? Less than the question. The the the issue there's an entire chapter within this work. It's actually, I think, online. The the report is currently delayed for a reason that we will maybe be able to talk about a little later, like, in the next couple of months or so. But there's a the the the dynamic of identity and refugees and crisis is, like, something that has been a flashpoint for, like, a hundred and fifty years in particular. It begins with the plight of individuals in the first World War. And I'm gonna skip over a lot of stuff here. So what I'm about to say is slightly inaccurate, but it's like, otherwise, we're gonna be here all day. Essentially, this the movements of of of large populations from conflict in the first and second World War is the the starting point for the modern passport that we understand it today. This is like the overarching again, skipping lots of lots of lots of history around this. The the point being that refugee movement and the the idea of escaping from violence comes with it in a sort of, I believe, a xenophobic way, a distrust of the the person in flight and the claim that they are who they say they are. It's made way worse by the fact that, like, oftentimes the system that read them, whether that was a social system or whether it was a state, is also fractured or destroyed. So you end up with people who have to make a claim about themselves in another society system and are unable to prove themselves by any metric that satisfies this other system. And what when you when you partner that with these these ideas around resource distribution and the align the allocation of decision making or resources to individuals who have fled conflict, it provides a perfect incubation point for identity systems that that have, quote, unquote, accountability tied to them. So the idea of you have somebody who's entirely disempowered, who nevertheless, you're going to say, we need to ensure they are who they say they are. And also that they have they only take what is they can rightfully have. These two sort of absurd concepts. Right? That there's not enough to go around, so they must only have what the rightfully there is under our own definition of this in the system that we've created. And that they must, despite being completely disempowered in this moment, they must subject themselves to we must validate them in their entirety. And so programs like the building blocks program are these early stage systems that allowed for this promise of what we can actually, like, use iris scans and other biometrics to determine that a person is who they say they are over time and provide them with the exact amount of food that they require in the as they stay in this exact amount of food that they require in as they stay in this temporary space in a refugee camp. Now, of course, the problem is multiple. I mean, aside from the fact that I find that to be a deeply dehumanizing system
Speaker 1
55:41 – 55:43
that is outrageous in an in an era of absolute abundance, it's also it also, like, has
Speaker 0
55:47 – 60:48
particularly devastating consequences potentially as a result of where the system is inserted and its immutability. Right? So in the context of the world of the world through programs building blocks, pilot project in Jordan, the Houthi authorities resisted attempts project in Jordan, the Houthi authorities resisted attempts at what they called ventriloquizing for the poor, and they resisted it from within Yemen as they watched what was unfolding in in Jordan. And what they what ended up happening here is that when the project was rolled out, then the Houthis the power's balance shifted in the favor of the Houthi government. They then the aid program then had to operate within the context of the data sovereignty demanded by the Houthis. So this giant biometric system then was given act like, they gave had to give access to the Houthi authorities as a result of that. Now, you know, I'm not making a statement one way or another about, like, the actors in this situation, but rather, like, this is a an immutable blockchain tied to to together with iris scans that, like, changes who whose, you know, governance and access changes radically over the over the course of two years. That's an astonishing Mhmm. Situation. Right? And it's even more astonishing when you realize that at this point in, like, 2019, you know, we're still I mean, we're headed for at this point, we're headed head first into a decade of authoritarianism, but we're not far that far away from the historical connotations of, like, the IBM collaboration with the Nazis. Right? Like, in the nineteen forties. Like, we are we are, you know, well aware of of the consequences of of large scale data sets falling into the hands of people who we don't want them to or who never the systems were never designed to accommodate for in terms of the threat model. And yet here we are in 2019 with this exact situation control within a contested region. What you saw in in Jordan and in, in Yemen happens again at, like, a much greater and more dramatic scale. Again, when, like, Elon Musk and the Doge, the Department of Government Efficiency Project walks in the front door of the federal government. These are systems that have huge amounts of citizen data that were built in many cases by people who followed the design ethics kind of thing. You know, like, people I left at my job at Airbnb as a head of design in order to work with The US design the digital core and build these systems for for civility and for for the institution. Suddenly, these are they're completely dismantled. They're fired from their roles. And then what comes in is, like, this giant data, like, accumulation system that over the course of a number of weeks just vacuums everything that's available and completely recontextualizes it under a Palantir algorithm. Right? Like, this is precisely what happened in 2019 with the world, you know, world through programs building block system, but on a much larger scale. And so it's I'm so grateful that you brought this up and you've let me to ramble about this on your on your podcast because this is the sort of situation that we find ourselves in. And and blockchains themselves, whether they're zero knowledge, whether they regardless of the designs of them, these are something that we have to grapple with. Because central to the concept of a blockchain system is the idea of a record, a a ledger that is visible in some way or another to a to a to a participants, and its consensus is key to the the the viability and operation of the ledger and of the blockchain itself. This means that each of the blockchain systems that we have built is a potential world food program disaster in waiting. It's the same it is a potential Elon Musk, Doge, federal government, US situation waiting to happen. All we need to do is find Monero's blockchain, and it's all over. Right? And and, yes, this is a fatalistic way of looking at it. Yes. I also agree that we need secure and safe ways, especially as things slide towards authoritarianism. You know, there's something like 36% of the world's population is not living under some form of increasing authoritarianism, which is a bleak number in 2025. These questions, these designs matter. And they we are just not prepared, I think, to grapple with that in any meaningful sense. And digital identity, in my view, plays a central role in that. Both in terms of how we psychologically as systems designers create systems in our own image through this lack of shared understanding what digital identity is and isn't. And then the dangers that are then emerged through, you know, essentially the weaponized design, the weaponizable design of these systems. And that's like Mhmm. One of the key and argued over 270 pages. One of the key arguments of the of the report. One of the key findings of the report.
Speaker 1
60:51 – 62:36
Right. Yeah. I mean, this is, you know, kind of a, I think, for a lot of people, disaster in weight type of situation when we combine weaponized design with digital identity system. I guess, like, in in the context in which weaponized design is kind of the norm for a lot of tech applications or applications using technology and such. So, yeah, it kind of reminded me there is this I wrote about this in in my book. I'm now forgetting the name of the guy, but he kinda critiqued, Lawrence Lessig on, like, code is law type of stuff in that he was saying that code is not necessarily law. Code is actually its own modality, and it's more akin to architecture. But it has its own thing in the sense that, like, code, unlike architecture, which by architecture kind of means, like, in a abstract way, like, you know, the architecture of of the world, how that kind of regulates society in a certain way. Yep. Yeah. Like, almost like the the physical world. But instead, it's in on digital systems, you have this kind of, like, immediate effect, a zero to one, you know, as soon as something has changed in the code, there is, like, this immediate effect on everything. That this is kind of, like, its own modality and kind of is not we are still not, like, really remotely have the systems built up in place to kind of understand the consequences of all these things, or we or we are kind of, like, the ramifications of its consequences whenever things go bad. But I guess the the ideal situation would be to where we don't have to experience the bad things before we before we understand what those consequences are.
Speaker 0
62:37 – 68:30
The that's I love that you brought up, Leslie. It's so great. Like, the biggest issue with all of this, it from that perspective is that there is a massive disconnect between, people who believe that code is law or that the coin is decentralized. And the reality of the situation, which is to say that, like, the thing that's missing from all of this and the late David Columbia does a great job of explaining this both in the context of coin itself, but then also more widely cyber libertarianism as a as as something that has inherent right wing historical roots and and core ideology alignments, which is that what it ignores is the skill cost in order to actually affect change. Whereas a democratic process, you just need to, like, raise votes. Ideal I mean, obviously, those are perverted by, you know, things like money and social engineering connections, etcetera. But at the end of the day, if you wanna participate in a so called democratic or participatory or peer to peer system in the code, you have to know how to read the code. And that is like the the the power centralizes around those who know the code and who can accept changes on behalf of the network into the code. And that is, like, a kind of anointed role. Right? This is, like, not to, like, name drop yet another fucking sorry. I can't probably swear on your podcast. Not to not to No. We're believers in free speech here. We're believers in free speech here. I'm a believer in, like, light discourse as you can tell. So I will rephrase that if you want. But I keep dropping right wingers and weirdos in your podcast, but Maldus Curvas Yarden is like Yavin is like another example of this. This. Right? Where the anointed is Mhmm. Like, there's a disconnect between this is somebody who who claims that the institutions are, if not directly collaborating in order to produce consensus, naturally doing so via their positions in order to produce a controlled consensus. Lastly, you must capture the democratic systems and run them as a as a hierarchy kingdom system blah blah in order to I mean, essentially, this is why, you know, Urbit exists. And these sort of, like, the the projects that they worked on exist. They, they have projects that embrace without any kind of I mean, they do have rhetoric, but they they they baked into their design is the understanding that, like, the people who know how to code are the people who are worthy in this in this way. You either you either know how to code and you get access as a result or you invest into it. You provide capital. And those two above all else are the participatory those who have participatory voices. And and then you have other projects. I can name a lot of them who are on the left who who deploy the same systems and not those structures and say that it's and then, like, yes. This is participatory because, yes, you can run a Mastodon server that makes it participatory. But, no, you can't run a Mastodon server because it's hard. You can't modify the code base. You can't I mean, the the level of exposure I mean, I've written about this extensively too. The level of exposure that one has from running infrastructure in a decentralized environment is extreme. From Tor nodes to Mastodon accounts. It's all dangerous in ways that aren't accounted for by the people who build the systems in the first place. And so at least the the the terrible irony of the situation is that for all of the things that I think are disgusting about land and Yavin, at least the one thing that then that they didn't hide behind is the absolute belief in the systems that they designed because they're the only ones. Like, these this thinking, this area of philosophical thought is the only one that truly embraces the the the the limitations of the digital and and the kind of power structures that emerge naturally from them, which is that if you are digitally illiterate, if you cannot code, if you do not have the prowess to be involved in the the code merging and the the the deployment of these systems, you do not have a voice in it. Right? That is the reality of the situation. That is a form of neo feudalism that that we now have mistaken in our own project on the left as something that doesn't exist or they can be routed around by governance whether it's DAOs or the Commons or whatever. None of that has actually proven to produce a worthy alternative to the fact that all you need is a bunch of assholes with high like, good coding prowess, and you can shape the system any way you want. Even the very basics of the tooling that we use. For example, the Linux kernel is is a hierarchical system maintained by a benevolent set a small group of benevolent dictators who are aging, who are polarizing figures, who just happen to be more ideological aligned to someone like me than to a right winger. And yet the structure is still the same. And the reason why Linux continues to improve and get better is because that's exactly it. It's not pretending to be governed in another way. And I say this as somebody who doesn't want these systems to exist in this way. But the the evidence after twenty years is undeniable. There are so few examples of of cooperatively owned systems that truly have made a dent in the world versus the ones that are controlled entirely vertically. And then when you then insert those systems into a society, you end up with a really dangerous precedent. And that's sort of where we're at right now. That is that the the digital identity event horizon that we're about to cross is literally that problem right now.
Speaker 1
68:33 – 69:56
Yeah. I mean, I think there's I mean, there's some maybe a little bit of pushback I would provide is that, you know, the the cooperative economy being one of like, the the sector of the cooperative economy is much larger than people realize, at least, on the in the world's scale. But I will say that, like, definitely in the tech world, in particular, like, they tend like, Oregon, there aren't many examples of large, like, cooperatively owned institutions that involve technology. I think there's something I've been thinking about writing about, you know, at some point about, like like, how it's easier than ever to be a king or something like that. Like, the, you know, thinking of technology as the, you know, extension of the self, like, the kind of, like, original intention of what, you know, technique or technology meant that with, you know, these highly networked digital systems, you know, the power of one person who is the admin of a particular system, therefore, has more power and control than, like, any dictator or king previously ever could have possibly hoped to have. And I mean, there's I guess there's, like, plenty of examples of for dictators or authoritarian governments using the technology. But I think in particular, the direction that we've been going as technology gets better or progresses, the more and more power a single person has the potential to exert,
Speaker 0
69:57 – 77:07
if they're able to wield it. I really I really love that you said that because I have there's actually a section in the report that nails that entirely. And I wanna I wanna just say, like, very specifically that you haven't read the report. So I'm not, like, coming after you for it. You've only read what's available on the website. But there is an area here that will be very you you will find very interesting. It's around what we call kill switch sovereignty, which is this idea that I mean, you see this in Europe from the beginning of the year. The thing that's the thing that I would push back on there, I think you're exactly right. But the thing that's missing there is that that power is boxed. Right? It is boxed within the constraint of the supply chain all the way through to the operating system, all the way through to the network itself. Like, a a month within that fiefdom that you've just described, that that that king has to operate within a set of within a supply chain stack that it has no con that that person has no control over. So, yes, they have this, like, surface level full control. And we even within the context of sort of your taking that analogy away from, like, the admin into, like, the real world where you have states deploying surveillance architecture in the name of COVID surveillance or COVID response, which then calcifies into control systems. Even those rely on a set of, like, a stack that is completely out beyond their control, right to immediately the interface in which that system runs on is outsourced. And so the concept of kill switch sovereignty is this idea that as you digitize a system of power, it begins to answer to external actors and legislation that it previously didn't. So in this this actually is is, in my view, very exciting because it shows you, I think, the first hints as to how to fight truly fight against global right wing authoritarianism that's, like, addicted to technology, which is that this stuff is brittle, as I've said before. This stuff doesn't last under any kind of sustained assault. And more importantly, it's a mess of jurisdictions. And it is so straightforward if there we if if there were actual countries that were the targets of these things that wanted to push back properly beyond something like the European Digital Services Act in which strong letters and fines are sent to US companies. If there was there were actual ways in which you can intervene and actually, like, trigger these kill switches that compromise the sovereignty of of another of another country. I mean, The United States has been doing this actively to the European Union during the minerals negotiation a little while early, a couple of months oh, gosh. Probably a while ago now. Between Ukraine and the European Union and The US and Russia, Musk stepped into the whole thing. I keep picking on this guy because he's like, he's not overrepresented in this report, but I think he's a figure that most of your listeners will be familiar with. Musk's Starlink was used as a bargaining chip within Ukraine as a way of getting a better deal on minerals in the post war reconstruction for the Americans. Like, that is the literal definition of kill switch sovereignty. And so, yes, while a person can run, some kind of fiefdom on top of a star link infrastructure, there's always someone underneath it who can do something about that. And that, I think, is the the tension point that exists that so many of us are unwilling to confront. The second thing I wanted to mention is that, like, very, very clearly, I'm not saying that, like, cooperation is not the right model. What I'm saying is specifically that, like, it is rare to find that working in tech by the very nature of the topologies of how technology is organized. However, there are examples of this. There's a topic on on New Design Congress called the parareal, which we haven't talked about and isn't in our notes. And I'd be happy to either direct people in this podcast to go to our website and read up on the parareal, or we could talk about it either way. I don't mind. It's just very much off the grid in terms of what you had planned. But the parareal is this is it the consequence of the par no. The parareal just briefly is a state of being in which a user, someone sitting in a computer or being in virtual reality or otherwise interacting with a digital system, their real world their material conditions that they exist in at that point in time and the digital feedback of whatever it is that they're working on that point in time eclipse each other like a moon and a sun. And as a result of that, they're in perfect sync for just a moment and a third space opens for just that period of time. A great example of this that has been documented, like, consistently for years is when someone, a a male identified individual putting a VR headset on inhabits a female presenting avatar. And there's a certain number of people, enough that it's been written about extensively, that realize that that's the trigger for them of of of body dysmorphia or like the I the understanding that they may actually, in fact Mhmm. Struggle with gender identity issues as a result of seeing their body mapped to a digital avatar that doesn't match the gender that they presented as to this point. That moment where they're in that space, having that experience and and realizing that that is the power real. It is a it is a fleeting moment that exists in in in in real time. That system and there are entire communities across gaming, across subcultures that basically live and try to cultivate that in real time. Not as a form of a drug, but as a way of creating connection. But also out of that creating economies and marketplaces and things like that. The Sims is a great example of the parallel at scale. The idea of, you know, hundreds or thousands of players, millions of players creating societies, real world reflected back, not just by creating their own selves inside the sims, but rather finding, like, analogies between the narratives in the game and things that are happening to them in real life through emergent narrative play is an example of this. And what is an example of this. And what has that led to? One of the things we're studying right now is the multimillion dollar market cap of Sims four's the the Sims four's economy of teenage girls writing GPU shaders and selling them on Patreon to make the Sims look more like a k pop soap opera, right, at runtime. Like, this is this is what we live with. Right? These are these are systems that exist that that that deny, that disprove exactly what I've just said of the of the totalitarian nature of technology. But they are fleeting. And more importantly, they're also underappreciated by people on the left. And that's the problem, I think. Like, we still think that, like, what we need is Twitter but not racist. But actually, what we need is something completely different. Right? That's the thing. And so many of these communities have starting points for that. And I think that's that's my counterargument to my own work, and that borrows very heavily from what you were just saying then around, like, cooperatives and things like that.
Speaker 1
77:08 – 77:38
And that's interesting. I think there's something maybe to say for, like, what people are missing when it comes to identity online is not that people even want to represent themselves one to one, you know, from from the meat space into the digital world, but actually, it is a space for, like, experimentation with identity that helps them kind of realize more about themselves through that play with identity rather than by
Speaker 0
77:38 – 81:42
trying to map it one to one. Indulge me for, like, two seconds. I'm gonna read a small excerpt from the Paralleal Manifesto because it is precisely what you just said. Sure. Would you mind if I just read, like, one little thing from it? Okay. So this is about Yeah. Go for it. Against, like, what the parallelism. So we sort of say, like, that the yeah. The first popularized by the dystopian science fiction writers, like William Gibson and Neil Stevenson, and embraced by web three and the likes of Meta. The metaverse is peep pedestrian, a capitalist desire to complete the process of financialization of what remains of the untouched self. This is what you're talking about with the one to one relationship between people and their digital selves. The desire to inform and perform one's own image for themselves and others. Whether deployed as a lazy sci fi shorthand or presented as progress to share or crypto hodlers, the metaverse exists to put down creative complexity and lubricate this process of flocking the future by making the possibilities of self actualization as narrow as possible. Min maxing power, clout, or investor return in the decaying digital wasteland depends upon the fabrication of new territories to plunder. But the lucrative models of software as a service lives or dies on its promise of no surprises. The immersive future they propose is set on rails and made as linear as possible. Paired with this is the reduction of the restraint of the plurality of the first person persona, regressing the potential of the self in the mediated digital real world hybrid back to user. The metaverse, Ready Player One, Fortnite, Roblox, Decentraland, Call of Duty, Facebook, Horizons, are all passive interfaces masquerading at Frontier. No more meaningful than the web two point o platforms they proclaim to replace. There is no meaning or value to be found in a Zoom call, a Facebook account, or an NFT. Nobody seriously puts on a VR headset because they want to puppet their low poly digital clone, complete with adjustable waistline thickness or optional wheelchair accessory in a $2 replica of a white collar prison slash conference room. Nobody truly yearns to inhabit the image of a billion dollar superhero franchise. Nobody wants your fucking DLC. So, like, the point here from this perspective is to say that, like, the closer that we get to digital identity that's supposed to map one to one, the less valuable it becomes despite the fact that we financialize that substantially. And I think that that's the you know, again, to quote from this piece is, like, what Gibson and Stevenson failed to anticipate is the metaverse and by definition digital identity, the true horror. Despite the colossal infrastructure costs, the metaverse is really fucking boring. And that's the the ultimate problem is that what you find in places on the left primarily, although not always, but, you know, that are leveraged by the the left in these sorts of what I would call, like, parareal economies is they are first and foremost massive creative and performative places, first and foremost. And then downstream from that comes the the identity the identity itself is, like, central to that. Like, it takes this idea if we have, like, a Cartesian idea of identity, you know, you take a concept of I think, therefore I am with Voltaire. Descartes' like, claim of the self versus the other, and then you, like, repurpose it into a digital identity sense. What we have is, like, I authenticate, therefore I am. Right? And what what these what these systems produce, these these alternatives, they produce this concept. This is like a person is a person to other people. It is assemblage of temporary identities and performative identities, which are then fed through a creative process that is central to that as, like, a starting point. And I think that's where those then create the kind of strengths and the, I guess, the stakes required for them to persist over time. And that's, I think, is what miss what's missing as as opposed to, like, yeah, just any kind of Twitter but not racist kind of left wing project in that sense. We went very far off of your your question script. I'm very
Speaker 1
81:43 – 83:26
I'm very thankful for you allowing me to indulge that. I wanted to ask, this question just to hear your thoughts on it. Getting back to to the crypto aspect. So we were both at the Cypherpunk retreats, which was this, pay for a protocol labs, included a lot of people from various privacy oriented projects like Tor, Internet archive. There were, like, lots of different, projects represented both in Web three and totally not in Web three Yeah. But doing important work. That, you know, experience, maybe not even just from that experience, but what are your thoughts on what are interesting positive light in the maybe also, I think, the things I think is interesting just for my own, you know, curious to if if you have somewhere to point me and what what is interesting that, like, new cryptographic primitives coming out. As as far as I understand, mainly because of funding from the crypto industry. So things like zero knowledge, cryptography, fully homomorphic encryption is another one that a lot of people are into where you can, you know, do computation on some dataset without knowing what is in the dataset. And, this one that I'm forgetting exactly what it stands for, IO interchangeable something, but where you can do computation without knowing the algorithm. Anyways, lots of very interesting cryptographic printout, being funded by the crypto industry. I don't really know necessarily what the unintended consequences could be of them. They're oftentimes framed in the positive, but I'm curious if you do find things coming out of out of these spaces. So the thing to, I think, remember is that, like, broadly speaking, my starting point with digital identity
Speaker 0
83:26 – 89:04
and this goes from everything every single type of digital identity. Right? Including the organizations who have funded this from the Signal Foundation to Worldcoin to members of, like, Aspiration from civil society through to radical the Radical Foundation. A range of different groups have have given us in like, have have supported this particular work. We my baseline begins and ends with the idea that digital identity is a net bad in its current form. Right? It is also, unfortunately, a necessity, especially given the fact that that the idea of proving yourself on as a person in and also being able to computationally produce something, whether it's identity related or something that you want to produce that's, you know, sensitive that requires external compute in any way, shape, or form. Like, these are the realities. I can't take a purist approach and say, like, let's just not do this because nobody will listen to that, and it's not realistic. But, like, my starting point with all of this is that this is inherently dangerous. It doesn't go far enough towards solving what needs to be solved. And broadly speaking, it needs to be paired with a complete rethink of digital identity or at least serious grappling with the with the actual issues that are at hand. The the interesting thing about encryption again, this is coming from somebody I mean, I worked on signal one point o. That was the thing that I did after Coinjar, was when I worked on signal accidentally before it was signal. And the the what I learned about at all of those experiences was that encryption is absolutely necessary. It is also not the only solution that we need because because at the end of the day, you have a pipe, if you like, that nobody else can see the insides of that goes between person a and person b and then back again. At some point, you can still access the contents of that pipe no matter what. That's the nature of the fact that at the end of the day, it has to be converted into some kind of signal that a human can read in many cases, whether it's like looking at it with your own eyes or something that is stored somewhere that you yourself can access. So there's always a way to access stuff even with all of this defense that built. The risk that cryptography produces that I did not really anticipate in the previous decade of my career is that it too is a kind of formalization of power and is in fact by itself, deeply authoritarian in a in a kind of more neutral sense of the word. Not necessarily in a political sense, but it is authoritarian in the sense that the cryptographic algorithm that one uses in a system determines what is reality and who gets to access that reality or that dataset. And as a result of that, that in itself is an appeal to a certain kind of finality that doesn't exist in a lot of, like, other parts of human society. Right? Like, the idea of ambiguity and uncertainty is central to how humans build understanding. So everything from innovation all the way through to emergence of storytelling and things like that. The idea of discrepancy between how person a sees an event versus person b is what leads to everything from the arts to courtroom complete courtroom understandings of of of events. And and I think that this is probably a very unsatisfying answer to the the listeners of this podcast, but I think that each of these systems, you know, the benefits themselves are, extremely obvious. Like, you know, a zero knowledge system, the it creates the ability for an identity system to to produce credentials or identity attributes is what I'm looking for without actually, revealing the larger identity system is a huge deal. Right? It's a massive deal. It also comes with massive negotiations around power and and like and these are systems that haven't been tested in those ways yet. Right? The same with, like, the idea of computing datasets. I was just having a conversation last night with someone about doing exactly this. And I I'm avoiding saying the word homomorphic because the person that I was saying it to was being very vulgar. And it was, like, quite difficult for me to to, like, not, like, say that. So these are systems that are relatively new. They introduce this sort of enforcement through cryptography that's necessary for the for the privacy or the integrity of what's being computed, and yet still don't fully go all the way towards resolving the tension points that we identify in the digital identity event horizon. And as a result of that, are they then used to as, like, an anesthetic in order for a society to think that we've resolved some of these tension points or these weaknesses? Or do they accelerate certain things? I don't know. Like, these are this is, like, the work that we do that we're gonna do in, like, 2026 and onwards is really examining these systems and seeing how their immutability, how their their inaudibility, the like, the in an uninterpretable nature, ininterpretable nature of some of these systems, how we actually measure those against their claims versus their out their actual outcomes. Right? What is the weaponized design vector of a system that you cannot fully see? How do you even understand what the difference is between abuse and system flaw in a system that you cannot, like, watch compute something? These sorts of things. Again, I'm frustrated even talking about this because I feel like my answers don't go anywhere near as like, they're not satisfactory answers. But to me, they're really interesting questions. Yeah.
Speaker 1
89:05 – 90:14
I mean, it's it if if that's the truth of of of what you think, that's I think, you know, I think that's what it is. I mean, I don't have answers either. Like, I think, for me, I keep up with these cryptographic primitives because there are promises being made, and I'm I want to keep up with whether those promises are actually kept. And they seem, like, powerful in on the surface, and that I, like, definitely agree. I think, you know, photography over a digital space is kind of like the expression of power in many ways. And there is, I think, a lot of, a lot missing in terms of, like, understanding cryptography from the left. I think it's almost completely missing in the broader left, I guess. Cryptography can and cannot do. But, yeah, I mean, they're they these are the types of things that that I want to keep up with or I'm choosing to keep up with in order to see whether or not we can, if not, fix things, at least limit the damage of of some of these systems that seem to be coming our way.
Speaker 0
90:15 – 95:21
What I think I you're exactly right. And I think the the the pairing that I would give to that to kind of consolidate what I was just saying there is that, like, straight up, they don't solve the issues that we raise in any of the work that we've done, right, over the last five years. They just don't because that requires a structural there is a there is a straight line between, the design ethic no thanks criticism in 2019 through to the the the gap between what is said to be resolved as a result of zero knowledge versus what old right and it's really important that I say that and then qualify that by also saying that the point of this isn't that they're useless. Right? The point is or that the people who are building these systems are in any way miss misrepresenting them deliberately, but rather that we have this colliding mess of, interests, digital structure, that already exists that has demonstrated the vulnerabilities to it. And so there are really two outcomes from that. You can either start again, which is like, you know, not really a thing. I mean, it could be. Who knows what happens in the next couple years? I'm not saying that for an apocalypse level. I'm just saying it could get to a point where, like, digital systems are just completely unbearable to use for a variety of reasons. Or it you know, the other way of doing it is to intervene more and there are there are definitely ways you can intervene smartly. Like, simply moving down the stack and intervening in different layers of the stack is super interesting. Again, like, in our work, we've talked about how, like, Blender and Godot, Blender being the open source three d software that basically has toppled all of the expensive for profit. And Godot being the open source game engine that has emerged in the last three years to challenge Unity and Unreal as sort of a cultural force. These are examples of interventions that are a little bit slower in the stack that have resulted in, you know, know, economic diversion, like forking in a meaningful way. And I think that's really, I think that's the important part here with with this. It's, like, it's about crit is, like, it's about critiquing these systems through the lens of the fundamental flaws that we have in digital societies. Of course, you know, the main one being digital identity itself. And then realistically evaluating the promises against those. And I would say just to wage this, I'd like to hedge what I'm saying, is is that, like, most of the claims being made across these systems, wouldn't dare say that they solve for social engineering because most of them won't be able to. Right? So it's not that I'm putting I don't want to put words in those mouths and just say, oh, yeah. These these are being made about, you know, very serious structural problems with digital identity. And, you know, it's not gonna I'm not saying that. What I'm saying is that there exist a handful of these, like, first principle issues that nothing can resolve without serious reboot of that particular first principle. And it's it's about ensuring that in the process of convincing people that things like zero knowledge and, homomorphic encryption and and, the entire sort of even proof of personhood as another example of these kinds of technologies. As we start to roll these systems out, proof of personhood is something that we abs because, I mean, we just spent four years looking at the the action of deep fakes as a scam vector. It is scary how serious that has become. How easy an essay and then, like, set the entire like, it did an audiobook version of an essay on the collapse of civil society in 2025. And, and then set the entire JC Denton's voice, the lead character from the Deus Ex franchise. And it's convincing like it it like it's a forty minute book. I mean, an essay recording that sounds Right. Like it came straight out of Deus Ex came out of the video game. Like, no, that's a stupid sort of claim, a stupid example, but the point I'm trying to make is it, like, that took me an afternoon and a GPU. That was it. Right? And, like, this is sort of the world that we we touch on this again in the July Identity Report. This is, like, what we're up against. And I think this is why I'm so hesitant to like, firstly, why we need these systems, place place of condemnation, but really proactive. I think that's extremely important. Again, one of the other problems, of course, with this is that we're not gonna really know where they fall short until they're actually tested in the real world. That's one of the things that we are really keen to work on, and in fact, are working on, you know, from the end of this year onwards. Yeah. These are the it feels like an unsatisfying answer, but I feel like I've given you something more than what I just did when, you know, based off of what you what you just
Speaker 1
95:26 – 95:52
said then. Yeah. Sure. I think you've definitely given me and I think listeners a lot to chew on. Okay. And I think thinking about the right questions and at least thinking about the right principles to be holding when thinking about these things and when being supposed solutions for, you know, the world's problems through, through zero knowledge proofs. So may I think I think definitely a conclusion to be made. No single cryptographic primitive is going to fix
Speaker 0
95:53 – 96:02
our our digital The one I am. I am. I think that's fix everything. Don't you worry about that. I'll make that. That's a joke. Sorry. Sorry. I interrupted.
Speaker 1
96:03 – 96:29
So there there there is one cryptographic primitive that someone is making secretly somewhere that is going to solve everything. We just haven't found it yet. But, yeah, I appreciate you coming on, and I appreciate the, the rambles and the rants. Yes. The cryptographic primitive is turning off the Internet, actually. 100%. But, yeah, if you want to just finish it off with any plugs, people keep up with you and your work and your research. Great.
Speaker 0
96:30 – 98:28
Well, NDC is currently dormant at the moment. And, you know, who knows where that goes for now, but you can indefinitely find the work that's been done so far. If you do a little bit of googling about my name, you'll be able to find, view of the report that's floating around. But we will be publishing that, I hope, kind of soon, pending, some events unfolding the way they need to unfold. Broadly speaking, though, I think, like, I mean, just keep an eye out. Like, I'm on I'm on Mastodon. Unfortunate you can find me there at, horace.lurk. That's like shiba the dog, shibacomputer, or like my web, shiba.computer. And I think more than anything, like you mentioned at the start, we've started an R and D lab back in March. We're doing a number of really interesting projects there, including, like, an encrypted alternative to Substack that we're working on at the moment that does, like, ephemeral, like, publishing through any CMS that you currently have under any structure that you want and can run, like, basically out of your laptop or on a headless server headless on a server somewhere. That's one of our early projects that we're working on. And then in terms of plugging stuff, I mean, World Ethical Data Foundation is incredible. They have a forum that, we're sort of working on, comes up every year. That's worldethicaldata.org. They're also, they they do a lot of pro bono work for years on, digital security and are in the process of security business, which is trying to build a research institute inside, which is one of my primary goals for the end of this year and next year. So worldethicaldata.org is where to go there. I think alongside the the continued publication of the work that we're doing in NDC, I think that is the major effort in 2026 is is that particular project with World Ethical Data. Crazy organization. Super cool. I think that's it Mhmm. For now.
Speaker 1
98:29 – 98:44
Sure. That works. Thanks. I appreciate it. I I definitely encourage people to check out those links. Those plugs, I'll put them in the show notes. The New Design Congress is also a very pretty website, I will say. You guys have really nice animations for, the different articles. But yeah. Thanks, Kate.
Speaker 0
98:45 – 98:49
Thank you so much. This has been great. Thank you. I really appreciate you having me on.