Welcome to CDT's Tech Talks brought to you by the Center for Democracy and Technology and the Center for Democracy and Technology Europe. This week, we are launching a special series of conversations bringing together policy experts from CDT Europe and CDT US to discuss how digital rights issues are unfolding from both sides of the pond. I'm Aimee Dupre Macabrese, and it's time to talk tech.

C t t. Today's conversation is all about AI, equity, and data. As regulation ramps up and public awareness of AI's risks grows, the question remains, how do we ensure fairness, transparency, and human rights all left behind in the race to deploy AI? To talk through this, we're joined by Laura Lazaro Cabrera from CDT Europe who has led our engagement on the EU's AI Act and the European Commission's recent moves toward deregulation, and Miranda Bogan from CDT US who focuses on AI governance and technical mechanisms to ensure safety. Let's join their conversation.

Hi, Miranda. Great to have you here. Great to chat to you. It's always great to talk to you, Laura. Fantastic. So, Miranda, let's dive right in. I know in The US, you're focusing on practitioners working on development and governance of AI.

We hear a lot of that false dichotomy being peddled here in Brussels as well. Very often, we hear innovation and robust governance or regulation being placed as two things that are completely at odds with each other. Very often, a big part of the conversation that's completely missed is the fact that businesses have a real interest and incentive in making sure that they have trustworthy products and that whatever they're developing is of interest, not just to individual consumers or people, but also to bigger entities that they're chasing after, for example, the public sector. So it's interesting to see that you're hearing the exact same thing across The Atlantic. Yeah. So, Laura, you work out of CDT's EU office. So what is going on there? How is that conversation translating into the regulatory and legislative side of things? So, interestingly, based on what you mentioned, we are in a different stage where we're not so much talking about private incentives for with governance, but more thinking about the rule setting and standard setting on AI. And here, we see three key emerging trends. So the first one and the biggest one is the implementation of the law and artificial intelligence in the EU, also known as the artificial intelligence act. It was adopted last year, but its implementation has staggered. Some of it's foreseen because the law itself sets different milestones over time for different things to come into force within the law. But but some of it has also been unforeseen on account of some of the pushback that we're seeing towards regulation. And this is quite unusual in that, typically, when European law is enacted, the focus is a 100% on implementing, and there is no discussion on whether or not the law is good or bad. Once it's set in stone, it's set in stone, and people move on to the next thing. But the AI Act experience has been quite peculiar in that there is several voices, mostly from industry, calling out the legislation as a whole and essentially pointing out things that should be removed or changed or even suspended pending further clarity on what obligations in the law should look like. So it's a bit of a pivotal moment, both in terms of what the implementation of the AI act looks like, but also in terms of the European Union's credibility on the global stage in terms of following through with the rules that they themselves set after several years of difficult negotiations. So it hasn't been a smooth road for the AI Act, and there's a lot of open questions at the moment as to whether key victories in the law, especially from a fundamental rights perspective, will be preserved. So this talk of simplification, of reopening aspects of the AI Act is getting a lot of traction, and, we're quite concerned to hear that this is something that has some credibility at least in some quarters

incredibly fraught, complex, but it was also the first of its time in that it brought together different stakeholders to try and sit down and agree on what should be the nitty gritty rules that general purpose AI model developers should follow in order to be considered to be complying with the AI Act. As a process, it was quite unique. And in the end, we ended up with a code of practice that was made up of, three chapters, notably, chapters on transparency, safety and security, and mitigations. And the conversation has now evolved from a place where we thought we were only going to have one code of practice ever to seeing the whole code of practice and the specific chapters within it as different texts that could be subject to change and amended in the future. So could be subject to evolution, and we're just the beginning in terms of setting down standards for future conversations on AI act implementation. That's been quite interesting from a civil society perspective because it's meant from the city to Europe side that we were involved in conversations and effectively negotiations with all sorts of people in the AI governance spectrum ranging from, well established academics to think tanks to industry representatives. And hearing what folks' different perspectives were in different issues and what were the red lines for people has been incredibly interesting. There has been another call for a code of practice, process, this time focusing on transparency obligations And whether or not that process will follow the structure of the first one is going to be a very interesting question to to dig in. It just goes to show what sort of dynamic approaches to policymaking around AI might be needed just because it's moving so quickly and the best practices are are evolving. So thanks for staying really plugged into that work. Thank you. And, I mean, we think that the multi stakeholder process is the is the best way forward. Having different folks in the room with, different areas of expertise and interest is really crucial to coming up with a framework that ultimately will be a compromise, but, ultimately, one that people can live with, including fundamental rights advocates such as ourselves. The second trend that we're seeing in the EU is a renewed effort to bet on AI as a technology of the future, which will fulfill the promise of European competitiveness. So we're seeing a lot of investment in the AI industry, a lot of new infrastructure being created and supported. And we perhaps got the first teaser about it during the French AI summit where European commissioner, president Ursula von der Leyen identified supercomputers, public supercomputers specifically as a core strength of the EU and announced a €10,000,000,000 investment in them. And that was going to be just the beginning of the EU's, latest foray into investing in AI adoption and AI development. Very recently, we saw the release of the apply AI AI strategy, which is a road map that brings together the different strategies that the European Union is thinking of developing in order to make sure that they're making the most of AI as a technology. And they were very explicit in that document where they said that they were going to apply an AI first approach. So it's a change of tune in terms of the conversation that was happening a few years ago when the AI act was being negotiated, and there was a dual track thinking around it. On the one hand, the risks and and the safety mitigations needed, and on the other hand, the need to enable, at least to some extent, some level of AI development and AI growth. Now the conversation is very much focused on the latter. And perhaps the third and and final theme that we're seeing emerging is AI as strategic technology for the European Union in terms of sovereignty and independence. There's been a growing realization, by EU policy makers that there is a strong reliance on critical materials, coming both from The United States and China that is increasingly proving unacceptable to several quarters, in the EU institutions. And so the AI conversation has been recast as a strategic critical conversation for the EU in in the months to to come forward. And very crucially, AI has been identified as a core technology for defense. And the strategy that I was just mentioning on applying AI, again, reiterates this and the promise that AI has to make a key difference for the EU,

like this is really being normalized in the defense conversation and more importantly in the war theater. But, unfortunately, it seems that's a trend here that's here to stay. And we're seeing a lot of, presence from, industry that is involved in the defense field settling, in Brussels or growing expanding their teams. So it it will be a key part of the equation, for civil society organizations going forward. If there was ever a time that human rights were important, it seems like it's now. This is it. This is it. And people shouldn't shy away from engaging with militarization topics, securitization conversations. Even if they're not always part of the traditional fundamental rights wheelhouse, now is certainly the time to start looking into that and seeing what the strategies that governments are are thinking about are. Because very often, the technologies that tend to be the most, intrusive or infringing of fundamental rights start in the defense area. So this is this is just the beginning. Miranda, part of your work is following how AI is being created and sold in new ways. And we're hearing a lot about agents. Could you tell us a little bit more about what they are and how they operate? Yeah. Oh, man. AI agents.

Language is so important when it comes to this conversation. And often, I think it is either used deliberately or even weaponized to keep some levels of expertise outside of the room. And then, in fact, it turns out that the many things that we should be worried about are very traditional concerns that have been emerging around technology for a very long time. And at the same time, sometimes a new frame can actually

What you're saying right now is really bringing to the fore for me many of the conversations that were had in the context of the AI act being negotiated and even during the implementation and the code of practice process under non purpose a models where the generality of a particular artifact or model was almost seen as being fundamentally in a position to the idea of regulating that particular model because of the variety of potential use cases of context of deployment. And this was a strategy that was very often used to say, well, if somebody else gets to run with this and do whatever they like with it, then why should we, as the developers, be responsible for any of the consequences? Or why should we be casting our minds into the future and try to imagine every possible reasonably foreseeable risk that could attach to a product like this? And it was a real challenge because this is a new type of technology. And while we've had new technology for a long time, some aspects of the generality that can be inherent to some of these advanced models are really tricky to grapple with in terms of regulation.

earlier this year on that. So, to maybe go back in time a little, from the very beginning that the commission started thinking about regulating AI as a whole, they were also thinking about specific liability rules for AI systems and potentially even AI models. So the AI act as a proposal emerged with a sister proposal that was strictly about AI liability. And the liability proposal stalled in parliament for a very long time until the final text of the AI act was agreed, which at the time was a decision that made sense considering that a lot of the content of the liability proposal would ultimately be influenced or decided by the text of the AI act itself. And earlier this year, during the French GA summit, the commission officially pulled the proposal, as you mentioned, officially owing to a lack of consensus or agreement between member states. And the liability law wasn't the only one to be pulled. There were several others that were cut at that time, but it certainly sent a very clear signal that the time for regulation in the EU, at least when it came to AI, was done. And at the moment, when it comes to the AI agents conversation, there's a lot of enthusiasm, but there is also a lot of unanswered questions. The AI act doesn't explicitly address them, and there's a lot of theory and discussion about how they would be regulated. Increasingly, the consensus is that at least commercially available AI agents will both be an AI system on the one hand and a general purpose AM model on the other hand. So they will, in fact, be what is known as a general purpose AI system put together, and that has a lot of implications in terms of what obligations attached to providers and how they're ultimately governed. So the taxonomy here will matter a lot, and there are still a lot of open questions as to what the ultimate definition of these AI agents, at least in EU law, will be. The liability question also needs quite a bit of unpacking. Since the AI liability law was shelved, there is no dedicated proposal or live legislative instrument that will specifically look at liability in the context of AI. What we do have, however, is general product liability rules, which recently what underwent their review process and were revamped to be more adapted to the digital age. And they cover AI, or at least that's the the product they're sending that we also share. But it's not tailor made for AI, and so there will be a few challenges there in terms of how the law grapples with some of the challenges that are simply inherent to AI, including opacity and and complexity. So how different actors in the AI supply chain map out to onto existing liability taxonomies in EU law, which remains very manufacturer focused, that's going to be quite an interesting question. And when it comes to agents, the challenge is only exacerbated because it's very difficult to trace harm to specific actors, specifically when it comes to multi agent systems and delegation of tasks. So this is gonna require a lot of intellectual flexibility, and, ultimately, a lot of research and legal discussion to try and figure out whether or not these systems are even subject to liability. And if so, how? It will definitely raise questions of human oversight, human in the loop.

So, Miranda, let's now turn to privacy. The conversation in the last few years has been about privacy and copyright implications of massive training datasets to create advanced AI models.

is incredibly concerning in that we've also seen that memory is being sold as a feature, not a bug. So the days of outrage back when the Cambridge Analytica scandal happened where the extent of micro targeting of people in terms of political advertising on social media platforms was revealed, That seems to have now been forgotten completely by the folks who are trying to to sell this this technology and these different elements that might make it, in their view, more user friendly and for which there might actually be a demand. And that is a real question. To what extent will consumers, everyday people decide to activate these features and derive any real benefit from them? I think the companies are framing these features as

So our EU, data protection law, also known as general data protection regulation, or for those of us more familiar, with it, more affectionately known as the GDPR, has been the crosshairs, for quite a long time. And something that people tend to forget when having this conversation is that in reality, data protection did not come about with GDPR. It is a fundamental right that can be found in the Charter of Fundamental Rights of the European Union, which is a regional, fundamental rights law or human rights law. And it's also found in the treaty on the functioning of the European Union, which is one of the foundational treaties, that came to build the EU as such and as we know it today. So very often, that the of the GDPR will focus on the GDPR as the source of all evil, forgetting that the GDPR is merely the operationalization of the fundamental right to data protection. And they seem to think that you can have that right unaffected without GDPR, but there have been many verse versions of what is now known as GDPR in the past. GDPR is just the latest currently valid iteration. So I always try to remind people of that when they start going on a rant against GDPR because of the many, many obligations that it imposes on different folks in the data pipeline. I'm always telling the people that complain that, actually, we're very happy to have GDPR here. And if we're unhappy with the core purpose, the core mission of GDPR, then we're saying we're unhappy with the level of fundamental rights guaranteed in the European Union. And that's a completely different conversation, but the two are inter related. So it's always useful to recall that. Going back

for civil society and just people more generally, whether or not they have an interest in data protection law or not. Because the core victory that the GDPR has is give people way more agency about the way their data is handled in the digital age and more control. So the GDPR will give you, avenues to directly interact with the entities that are processing your data, ask them for information, ask them for access to your data, ask for restrictions of processing of your data. So allow consent for one use, but then object to another use. But at the same time, it also gives you a few avenues to complain to different authorities that exist that you known as data protection authorities if you feel that your rights haven't been respected or that the GDPR as a whole has been infringed. And it lastly gives you access to judicial avenues as well, which is, an aspect of the GDPR enforcement that we don't often talk about because of how costly it is and daunting it can be for both civil society folks, but also regular people. But it's there. And so if somebody considers that their rights in their GDPR have been infringed and they've suffered some level of harm, they can go to court and they can ask for compensation. And as you rightly mentioned, companies now have to factor that in in terms of their compliance cost, which is one way of having to think proactively about what they should be doing to to avoid these certifies. But when it comes to GDPR, it really wasn't created with, AI as such in mind, but it did foresee the challenges that new technologies would bring to the protection and enforcement of fundamental rights. For example, the lack of transparency involved in some of these processing activities, the many hands problem, the the diversity and number of actors that could be involved in any given processing operation, and also the mass use of of data and profiling down to the most intimate, intricate, unexpected personal characteristics. So in that sense, the GDPR was always built to be future proof, and it has to the test of time whether people like it or not. And it's a conversation that we've been repeatedly having, unfortunately, when it comes to the AI context. So there's several implications on AI, when it comes to data protection law. So one consideration, for example, is the extent and sensitivity of personal data that's fed into a model, but also going into automated decision making territory, as you were mentioning, the extent to which outputs can be used to take decisions on a person's life with next to no human oversight or control. And, also, when it comes to large language models or generative AI, the role of personal data is even more complex For a while in the EU, there was an ongoing debate as to whether even models could process personal data in the traditional sense. That debate was fortunately resolved by the EU data protection regulator that sits above all member states, And they answered that question in the affirmative. They said, yes. A model can absolutely process personal data if they can, for example, regurgitate information or personal data can be extracted from the model, and that's subject to a threshold of effort level needed to be able to extract the data to have it regurgitated. But for a while, we even saw data protection authorities arguing that the way that models work meant inherently that they couldn't be processing data in the traditional sense. Enormous implications for data protection law so as to make it irrelevant in the age of AI. And we're very happy that that opinion, didn't fly and that we got some very, decisive arguments in favor of models being subject to data protection law, basically. It was a really laughable analysis. It was. And the fact that it was coming from a data protection authority was all the more puzzling. But for a while, it seemed that this was a plausible line of argument or at least a line of argument that could be successful. And it was a very scary time. So we're very happy to have had it confirmed that, no. Indeed, models can process personal data. And, of course, with models, the challenge is you can't be sure at any given time that a model won't regurgitate personal data ever again no matter how many attacks are thrown at it or strategies are developed to try and make personal data essentially extractable. The one caveat is that it's all based on training data. So European data protection law, when it thinks about, how models process personal data, they will be thinking about how training data is extracted. But the personal data in question needs to have been in the training data to begin with, which is an interesting nuance, that, that I'm sure we'll see tested through the courts. But more generally, there's a lot of relevance to data protection law even in in traditional settings or in the traditional sense when it comes to personalization, which as we've discussed is not new and frankly is not at odds with data protection. So something that we were often at pains to demystify is this notion that data protection prevents all sorts of processing of personal data. I mean, the simplest, easiest legal basis to have under GDPR when it comes to processing of personal data is consent. If you can get someone to consent within the terms of the meeting of consent, set in data protection law, you're fine. And that is honestly the best way we think for any technology, to to lawfully process personal data. Because otherwise, you're getting into more uncertain territory of what the legal basis is. Is it because the commercial interest that the entity has enables them to use this particular type of data? Is it because there's another, legal basis in national law for the entity to perform a public task that legitimizes this use of data? All of these questions can be really thorny and really intricate. So really with consent, you're just taking those concerns out of the window, and you're making it much easier on yourself if you are an entity that processes things. But there's challenges at the same time. So for example, under GDPR for consent to be valid, it needs to be specific, informed, and explicit among many other objectives that I'm not mentioning here to make, the audience's life easier. But this means that for the consent to be valid, you need to fulfill a lot of requirements that are very rarely fulfilled in their entirety. And so it's unclear if even if a person were to give consent, whether the model, for example, would be able to observe the scope of the consent that was given. For instance, if you gave an instruction that I want you to process information that I'm a vegetarian specifically for the purposes of looking up restaurants in the neighborhood, we can't be sure that that information will be used for anything else. And in fact, I I don't know that this option to restrict the type of personal data that is processed about a person is even available. I mean, either it's memorized or it's not. But I don't know that there is a sort of gradation of types of purposes that you could use it or not use it for. We just had a workshop with with experts talking about that exact topic,

which wouldn't be compatible with these legal frameworks or expectations, it sounds like. Absolutely. Because when you have so much information or so many personal characteristics together, then it's difficult to know whether any particular recommendation or output is being received because it corresponds to this distinct type of data or data category, for instance, dietary preferences, or whether it's a result of these different memories or personal attributes or traits that you've fed into the model over time. And so it it becomes more than just the sum of the parts. It becomes something else entirely. And from a data subject agency perspective, that's a nightmare because then it's really hard to exercise consent in respect of each of these different elements. And this brings me to the question of data subject rights. The whole premise of data protection law is, yes, you can process the data so long as you have the legal basis that allows you to do it lawfully. But at the same time, you need to be able to comply with a full range of data subject rights. Some of which are conditional, but some others are absolute. So this will mean that if a model is processing personal data, for example, they will need to be able to guarantee that they can fulfill data subject rights, such as restriction of processing, which I mentioned, or access information, so access to the full range of personal data held by the model and the purposes for which it is used. And this is going to be incredibly difficult to fulfill satisfactorily, at least to the standard of the GDPR. And there are real questions at the moment as to whether this is even possible, whether a model is even susceptible of having achieving this high level of compliance. So, I mean, where we're landing and where many privacy advocates are landing in this conversation is increasingly viewing AI, as a normal technology and basically not falling into the trap of AI exceptionalism when it comes to data protection compliance. So instead of making data protection fit the reality of AI, really what we're thinking about is how do we do we make AI fit the reality of data protection law as it is and has been developed over years, even decades, based on core rights that were foundational

conscious of the different narratives that are taking place and how we should not get duped.

That's it for today's episode of Tech Talks. Check out more of Sidity's work by visiting us online at sedity.org and cdt.orgeu. You can also find us on various social media at sendemtech, that's c e n d e m tech, and CEDT EU. Thanks for listening, and see you next time.