Welcome to Tech Talk. Bye.

Yes. So there were three new features introduced, that Apple's talking about as a package of tools that will help fight, child sexual abuse and child exploitation on its platforms and online. And I'll talk about each of the three. But quickly, they are going to make a change to iCloud. It makes a change to iMessage, and then it looks at, Siri and in search. So with the first one, with respect to iCloud, that is going to be able to scan the new feature will scan photos that would be stored in iCloud, and it scans them on the user's device, using something close to, hash matching is one way of describing it, which, you know, it against known child sexual abuse material. So it's there's different lots of different parts of that, so I'll try to break it down. There exists a lot of known content that, depicts child sexual abuse, and that is held by an agency. And there there are several around the world that are mostly, related to government functions, but but those databases are elsewhere to say that. And and they're trustworthy, and they, you know, they they sort of determine that. Now that content can't really be stored anywhere. So when, there are any variety of services out there operated by a variety of tech companies that use this database, to make sure that that content isn't spreading on their platform. What they're really doing is they're they're comparing, the content's hash, not the content itself. And a hash of a content is simply just a way of taking the content and making a unique ID based on that content. So it's sort of reducing the data down to a certain number of bits. It's a very small number. And then that can that reduction, those bits can be compared to, the same hash of other images to see if they compare if they're the same. And if they match, then that's what we called hash matching. So that computation is going to be done on a user's device. The hash database is going to be stored on the user's device. This is either on a Mac or an iPhone, and and that will only be done, for users in The United States for now. And it will only be done on photos, not videos, that are stored on the iPhone that would be destined to be stored in the cloud on iCloud services. So, you know, if a person's not using iCloud backup, they this won't work on their phones. And and it will be rolled out for all iPhones, once that change is made, in a few months' time. But the feature, like I said, won't be turned out. So that's the first one. And then second one is, meant to work in iMessage. So right now, iMessage on iPhones and Macs is an end to end encrypted, chat platform. You can have groups. You can do one to one. And so this feature is introduced to, warn, children or young people about sexually explicit material and to also, help parents intervene. And that works in a couple of different ways. So, it has to the account the the user of the device has to be designated as a child in a family sharing account. So not all devices are part of family sharing, not all people that are part of family sharing are designated as children. But if you are a child, then these features will apply to you. And also then if you if the parent has turned it on, this will apply. So what happens is that, it's really and it's also focused on, sexually explicit content or just nude photos. We just call it that. It's probably the simplest way to think about it. So if a a person who's been subjected to this change, is sending or receiving a nude photo, there would be a notification pop up what the industry terms like a speed bump. Right? Slows the user down, gives them some information about what's about to happen. If they choose to view it or if they choose to send it, then that will notify, actually, the parent, or the person who's designated a parent in the family sharing account of that, material and also allow that the parent, to to see it. So that's, that's one. There are I'll probably sweep back through these and talk about some of the concerns. I know that's coming in our in our discussion later. But I wanted to get to the last one as well in terms of how it works. So the last one, I think, is covered less in the press because it may be a little bit, less alarming. But, essentially, anyone using Siri or search, there are gonna be some things built into it. So there Apple is going to be creating some guidance, around how to how families and children can avoid, child sexual abuse online and a variety of things like that. Apple's gonna create that. It's gonna be like guidance for how to use iPhone safely, how to be online safely. The only other thing they're gonna do is that, when we use search and Siri that is server side service, meaning that we type in our search query or we speak to our, assistant voice assistant, and then the server processes what we're asking for and sends us back a result. That sort of, search lookup is going to intercept whenever it appears that someone is searching for child sexual abuse material. It's going to intervene and not going to allow that,

affect end to end encryption? Well, so let's focus on, the two most important ones. The change to iCloud photos. So when you as as Mallory said, when you upload a photo to the iCloud, it'll be scanned for the hash of CSAM, images. Now iCloud, is encrypted, but Apple holds the keys. So iCloud, interactions, if you will, were never end to end encrypted. If something's end to end encrypted, that means that only the sender and the intended receivers have the keys and can access the the content can understand the content. So end to end encryption is really important because we all rely on it every day when we use the Internet. A lot of, services are encrypted end to end, so it only the again, so only the sender and intended recipients can view the message or the material that's being, sent. And it's really essential to, financial transactions, any sensitive business dealing because you can't have people in the middle get access to the information that's being sent. Where end to end encryption is most compromised in the announcement that Apple made is on the iMessage side. And for iMessages in family accounts, the child on the account no longer enjoys the benefit of end to end encryption. That means that the child, is going to be receiving notices that an algorithm thinks that the content, which in an end to end system would never be accessed, by anyone but the, sender and the receiver. They'll be they'll be receiving notices that the content may include something that's sexually explicit. And, in some cases, the parent on that account will be able to access the content. Those cases are when the person on the account is 13 or at least, is signed up to be 13, and that person proceeds to view the image about which they were warned. When that happens, the parent gets a notice that the child has moved forward and, has viewed an image that the algorithm thinks is sexually explicit, and the and the parent can access the image themselves using a different feature, of the iOS system. So, that's not what we think of when we think of end to end encryption. And I know that Mallory has done a lot of work at the technical level, to get a good definition of end to end encryption in place. So let me, turn it back to Mallory and ask her to explain, where that process is.

Definitely. And I can just say that there's a lot of features, of end to end encryption, that are not always incorporated into every single system. So we when we're thinking about a definition for what it means to have, one, two, or many endpoints talking to one another, completely encrypted where there's no knowledge of that conversation by inner in any intermediary, including the server of the service that's providing the the messaging, we're trying to encompass a lot. We're trying to encompass video and, you know, audio and email as well as messaging. So that definition is really, really broad. But I will say there are some things that are really important to think about, you know, when, considering how these architectures work. Things like, you know, if you can have access to one message now, like, say that I've, you know, taken a screenshot of a message in an end to end encrypted chat and shown it to somebody. That's actually not as much of a violation as, you know, being able to intercept, messages, you know, maybe as they're being sent. So sort of like a persistent, capture. Then, you know, if you have one message in its entirety, would you be able then to decrypt messages from the past or messages going forward? So that's something called perfect forward secrecy. There are things about, like, deniability. Can you actually, you know, hide some things about the messages so that, you know, a user's client or something wouldn't be able to verify whether or not they were the one who sent it? This is like these are probably issues like metadata. You know? Can you you can say you can you can see who said what and who they set it to and when they set it, but you can't see what's inside. So there's a lot that encryption end to end encryption tries to achieve in reducing all of that data, making sure that, it's time bound and that any sort of access, exceptional or otherwise, wouldn't reveal all of the access. So there's a lot of really hard problems in it. So, Mallory, can you explain client side scanning? And and is this a backdoor? So there are many reasons why I think it's a backdoor. I won't just say yes, although yes. So there are a few things that we are promised in end to end encryption, and I think four ways that client side scanning sort of doesn't meet those expectations. One is that when we're in an end to end encrypted conversation, we expect that no other person would be able to see what we're saying. In the case of the iMessage feature that Apple's introduced, they are notifying parents. So that's a person that's a person who can see, you know, the message that you've sent even though you were told, you know, in you're you're you're given a notification in iMessage that, you know, if you send this, your parents gonna see it. So it might be consensual, but, you know, a person who can see what you've just sent breaks the promise of end to end encryption in a really obvious way. I think the other one that client side scanning proposals sometimes, have, although I will be clear that the iMessage proposal doesn't, other client side scanning proposals and end to encryption do where there's review. So an important part of notifying or of flagging or actioning problematic content, whether it's CSAM or it's terrorist content or something, is that it might not be, actually. The computer might have gotten it wrong. Computers make mistakes all the time. And so you have to sometimes bring a person in or review process to make sure that if you do something very, severe in terms of actioning it, that there's been proper review. So review is a way that you're actually inviting a person or an entity in to see that message. And so that's one violate. That's another very obvious violation of a person, seeing that content. Do you other ways or a bit more about what I think of as, way ways the computers are supposed to be in the way so end to end encryption is not just about, a person not being able to see. It's also about not a a computer not either. So so even though, you know, iMessage or Signal or Facebook Messenger are providing the end to end encrypted service, the point is that the servers can't see what you're saying. That gets broken in client side scanning in two ways. One is that sometimes there have been proposals where the clients the client is actually not powerful enough to do the processing of the scan because either the database that it's scanning against is too big or whatever. And And so the the client side scanning actually is server side scanning. So that's one problem is when there is actually a computer that's receiving those messages and actioning and doing something with it. And then the other thing is about the resource itself, the the database. So the database of hashes of content or or it could be a resource such as, like, a machine learning library or some third party resource that's being updated. It's being changed. It's learning. It's it's shifting. And no and the user doesn't have control over it. It's very opaque to the user, and it's an external resource that the application is relying on, in order to do the scan. I think those four things really mean to me that, an end to end encrypted system that includes a scan is incompatible with the confidentiality that we expect from end to end encrypted systems.

As I was saying earlier, people rely on end to end encryption without even knowing it, and it's what enables the most sensitive transactions and interactions that we have on the Internet. When you're doing online banking, when you're paying bills, you have to be confident that the information that you're sending and receiving is seen only by your bank and by the payee. If that's not the case, if you can't be confident of that, you're not gonna use that service. And companies get this. They they know that, they have to offer security. They have to offer privacy. And if they don't, then people will hesitate to use their services. So, again, we use end to end encryption all the time, and we use it for the most sensitive, interactions that there are. And increasingly, we're using it for daily interactions because it also helps deal with the problem of, malicious hackers and people gaining access to information that they shouldn't have access to. In fact, CDT recently helped found the Global Encryption Coalition, which consists of NGOs and technical experts from all over the world, and some companies as well. And we're dedicated to promoting end to end encryption and resisting governmental mandates that would compromise it. So

But I wanna caution us that these are really big problems, and what is being asked of these systems, the engined encrypted messaging systems, are beyond what we really should expect from a messaging system, and that they're they are essentially tech solutions. I think sometimes we have this mental model in our heads that, you know, backdoor encryption, breaking encryption, decrypting messages is what is being asked for, so that those, messages can help in the fight. But, actually, it's more than that. These are these are very robust feature requests. And, as we've seen with Apple and the ways that it's taken a lot of time for cryptography experts, like those that seems he works with to even understand what they've presented, they're really robust features. So are what what can we do though in in the sense of, you know, going to the point where we're not actually breaking in encryption? To me, I think as you as I've mentioned, it's not just about, can you do it without decrypting a message. It's can you do it without interrupting the expectations and the architecture of an end to end encrypted system. And that's why things like client side scanning are incompatible or, something called traceability is also incompatible because it creates these sort of this robust metadata that we don't really need, and is extra. There are some things that work though. Things that are that are really user empowered. So we should always expect our messaging or social media apps and platforms to allow us to block and report content. So that's a user empowerment feature that means, you know, I don't consent to seeing this or talking to this person, or this is a this is a problem. Somebody needs to look at this. There's something wrong, with what I've been sent. So those are really important features, and you can build out from there. So one of the good examples of things that most platforms do without breaking confidentiality or user expectations around privacy is spam management. So spam is pervasive. A lot of ways that this is done is, through, like, machine learning around, behavioral patterns of the content itself. And I think that could be possible. So without enhancing the amount of data about the data or metadata, without sort of interrupting the way the systems work, you can actually learn a lot about what kinds of patterns might be abusive or might be problematic. And then, you know, I think that there are some things that, there's some ways to hash match content that would stay within the application. So, for example, when we're using an encrypted messaging application, one of the really nice convenient features would be, like, predictive text input. Right? Like, we all kind of are used to that now where our phone gives us the right spelling or, you know and I don't think anybody feels like they're being spied on when their keyboard suggests, like, the right spelling of a word. That's an example of, like, machine learning and natural language processing that's built into apps, that for the most part, like, isn't relying on, you know, possibly nefarious external resource to come into the application. Things like that. Things that are, like, able to be contained within an application, that's not relying on a third party resource, that's not pulling in a database or doing server side processing, those think those could really work, you know, like the sort of speed bumps we mentioned before. The only the the issue then, I think, with the speed bump becomes not a technical one and not a concern for end to end encryption, but, you know, how do you culturally do that? How do you sensitively nudge the user to do to take a certain action or have a behavior, especially when it comes to something sensitive like sexual expression that doesn't, you know, shame them or that is translatable and localized across different contexts, that's actually the hard part there. But, you know, an application that has robust features that that empower user behavior in one direction or another are not necessarily objectionable to end to end encryption.

on companies in the future? Sure. Let's talk about the latter one first and think about the hash matching that, we've been talking about and describing. So right now Apple is going to start doing hash matching of CSAM images and that means it'll be comparing hashes of the images that are in a pre existing database hashes of of, content being handled by the phone. So, what if a government sees what Apple has done voluntarily there that we would like you to scan for in the iCloud when people are uploading it. And, we want you to close their accounts when they are uploading, over a certain threshold of images of terrorist content or of copyright infringing content. And there are proposals now in Europe to require companies to scan for this kind of content and to keep it off their platforms. Well, if Apple can do this with CSAM as it has demonstrated that it can and will, why couldn't it be compelled to do it for other types of content? And these other types of content, you know, what is, terrorist content? In, in some countries, it could be one thing, and in other countries that terrorist content could be, just protest information that the government is labeling terrorist content because it wants to squelch it. So, I think that there in a way Apple has really led with its chin and it's gonna be getting some demands from governments that it says it will resist, but at some point, one could expect those demands to be rather insistent. Another thing on the CSAM matching is that current, proposal, Apple will start doing the CSAM matching and sending the results when a certain threshold of CSAM images is met in one account, looking at that material and then sending it to the authorities, the National Center for Missing and Exploited Children, which then passes it on to law enforcement, if the, images are in fact sexually explicit. Well, there's no reason why, there couldn't be a decision or a compelled decision by Apple to send that information directly to law enforcement and to send the other information that I talked about earlier discovered on its platform, to law enforcement. So I think that we're that Apple has opened, the door to a lot more, potential government abuse than perhaps it's accounting for in deciding to go down this road. With respect to the search for sexually explicit images, there are probably some more immediate impacts that we should all be concerned about. First, these classifiers, the things the machine learning tools that are gonna scan for sexually explicit images are significantly less than perfect. They have identified as adult content, art, advocacy material, memes, photos of people's dogs, posts about design patents. I mean, there's a lot they make a lot of mistakes. And a mistake can be really costly in this context because remember, there's a lot of parties that could be involved. You got two kids, and they're, texting to each other. And one kid texts something to the other kid, and the first kid gets a note saying, hey. That might be sexually explicit. So does the recipient. And then the first kid says, oh, I'm gonna send it anyway. And the second kid says, oh, I'm gonna receive it anyway. Well, now there's four parents. Well, two parents, two owners of the account who now are going to have access to that information. They're gonna get a notice that their child sent or received sexually explicit material. Some parents will act on that notice right away. Some parents and children do not have healthy relationships, and, the action of the parent might be very detrimental to the health and well-being of the child. And, there's going to be occasions when, I think particularly for LGBTQ children where they are, put in awful situations with unsympathetic parents who are notified that an image may have been sexually explicit. So I think that we're going to see some some real abuses come out of this out of the changes that Apple has announced.

Yeah. And as Greg said, that might be more possible now than ever. I guess what I've said previously, which is that there's actually quite a bit one can do without breaking end to end encryption or just coming short of that. And so thinking about ways to focus on features that really empower users. And, also, I would say that whatever features they are, that they aren't too complicated. I think one of the things that I notice a lot is, when we when, you know, we're saying that, okay. It's gonna work for you with these three conditions met, but if you do this, if you don't store your photos here or if you don't turn this on, then it's fine. I mean, that's too confusing. I think that's that's not helpful to users. If it's a if it's a very onerous opt out, feature, then that's gonna be a a problem. And we can't rely on configurations, to save us from criticism. Right? I I think, you know, Apple shouldn't think that because it's built in configurations that it's now absolved from breaking into encryption for everyone because I think that that is what it's done because its system now is built that way and we know that everybody uses defaults. That's just a blanket statement that's almost always true. I think that companies really need to stand firm and resist these feature requests from law enforcement for a few reasons. I think that we shouldn't think of our messaging apps as the tools, the tech empowered tools that we need to fight these social problems. There are there's a lot that technology can help with, but I think trying to reappropriate our common public spaces where, like Greg said, we're just trying to do our banking and talk to our loved ones. Those aren't the tools we need to fight this. There may be other ones, but I think it's a mistake to think that we can have, these huge monolithic platforms that do everything for everyone, including government agencies in a variety of different countries. That's that's, I think, just not a great way to design, a technology, especially a technology that's meant to be lightweight and secure and working for users' privacy. The other thing that I would note is that a lot of the rhetoric around, you know, should I should I message or should Facebook Messenger others, like, should these be private places? It's actually not that conversation. It's not that question. The question that people are really asking is it's not so much, you know, can this be a private online place? It's can there even be online private space? Because I think once we start eroding all of these common platforms where most people are, there is nowhere to go. There's nowhere to have a private conversation anymore. And it just feels like these messaging apps, and there aren't that many and that are using this, actually. They're the only places we have to have a private conversation. I don't think we wanna be in a situation where there's nowhere to be private online. That that would be a mistake. Because at the other end of the spectrum, you know, there are folks who certainly will use it. I would mention that, you know, the definition of end to end encryption that I'm working on that was mentioned earlier in the podcast, there's a whole effort that I'm only partially involved in to actually standardize an end to end encrypted protocol at the Internet engineering task force. And this means that, hopefully, there will be a robust ecosystem of end to end encrypted messaging. So if we take away the popular ones, then most of the public is going to miss out on intent encrypted messaging while it will still be available for those motivated enough to use it. And that tends to be powerful people because they wanna keep their conversation secret, like folks who work for government and so on, and then folks who want to break the law potentially. So we also might be in a situation where, you know, those committing crimes, and then those fighting those crimes are the only people who have end to end encrypted messaging and the rest of us don't. So I would say, please, to companies, don't let that situation play out. Like, we need a robust ecosystem of end to end encrypted messaging. And just because Apple has taken this path doesn't mean that everyone can. And I think we can hopefully rely on many companies, to do the right thing together so that it's harder to feel like this is a an inevitable trend.

Yeah. I would I would say this. The trajectory of end to end encryption has been pretty strong on an on a upward trend. Many services that used to not be encrypted are, and, the numbers and the quantity of communications that are being sent over encrypted services has been increasing dramatically over the last few years. Apple is really an outlier here in what they've decided to do. They're going in the opposite direction of a lot of, companies in a lot of trends. It's troubling to us because we're concerned about, the safety of people who are using Apple products, and we're hopeful that they will reverse course and see that what they've, put on the table is not going to, have the the beneficial outcomes that they are hoping for. And we're also encouraging companies to continue the march toward end to end encrypted products and services because we really need those in order to continue to be confident when we access the Internet and use, the devices that companies manufacture. Who wants to be who wants to pick up their cellular phone and look at it and think, I wonder what this phone is gonna be doing. I wonder what it's gonna be reporting to the authorities about what I'm doing. We don't wanna have that lack of trust in our devices and in the services that we use.