Technology, which tends to centralization, is never a good thing for democracy. If the technology tends to, to be, you know, centralized, and that is clearly the ambition of the developers of AI, then that we must insist on decentralization.

Thank you.

Sure. I'm a visiting professor at the College of Europe in Brussels, in in in Bruges, in Belgium, near Brussels, where I teach a post grad seminar on AI law and data protection law. I do this as a retiree. I was director in the European Commission for fundamental rights, and citizen rights and special adviser, also principal adviser, on the digital transition. And in this context, I learned about the digital. I had my own playful approach to the digital as a lawyer, because I was the lead director on the GDPR, bringing GDPR onto the law books. I I led the whole conceptual preparation and and the negotiations. And and that basically, was such a learning experience that I stayed on as, let's say, the principal adviser on on digital transition, and all the following legal acts, I I was also involved in. Like, the AI act, the data act, and so on. Not as a lead director. Only, the DGPR was, I was in the lead because my director general, DG Justice, in the European Commission was in the lead. All the later acts, giving shape to the digital sphere, were led, by a different director general, namely, DG Connect, which I would say is, let's say, the more engineering oriented, director general.

Well, it is important, that, the digital transition and also now, the transition to a world shaped by AI, it does not, lead to a world where democracy doesn't function anymore, where fundamental rights count for nothing. I think we have to engage and revive, both fundamental rights and, democracy. And, and I think that's that's a very, very important challenge because, otherwise, technology and the companies behind the technology, they will just, take over. They will rule us and our lives and our societies.

And, yes. And and there's a lot of concentration of power, and history probably has never seen, such, efficient mix of power, both over individuals and societies. If you think about it, there are eight sources of power, which are concentrated in the big tech companies. First of all, they are the richest and best capitalized companies, in the world. They're all in the top 10, of this US stock exchange. So they have a lot of money. Second, they have all the personal data about us. Every individual in the world is profiled by these companies with up to 3,000 parameters, so they basically know everything about us. About every individual, they know the leanings, the political leanings, the sexual leanings, religious leanings. They know our fears and our ambitions, our dreams, and they know all our relationships. And this is, of course, very, very dangerous that, these profiles are at the core of the danger to democracy of fundamental rights and fundamental rights. And then, these companies have the power of innovation on their side, and not only, are they strong innovators and innovate in secrecy in... To such an extent that public universities These companies impose, gagging orders on their contracts, work contracts with the gagging clauses, which means that people can't talk about, what what they are researching, inside the companies to such an extent that really, the good old tradition that, PhD, students return to their professors from time to time once they start working and talk with the professor about what they are now working on, it's just not happening anymore. They have completely... These companies have completely cut off in academia and pushed forward technology in secrecy, which is, I think, a a great problem. And it's a it's a huge change, let's say, relating to thirty or forty years ago still where there was, still a very lively dialogue, and between universities and, and private research. Okay. So we are talking about, the eight elements of power, which are concentrated in big big tech and which gives them, a power in society and democracy over people, but all... Also over whole countries, which really never existed before. And I was saying that, of course, the key element of this, aid is, the personal data which they collect. Because this means they know everything about every individual. They know our dreams, our ambitions, our fears, and, they know our political leanings, our religious leanings, our sexual leanings. They know our relationships. And, of course, in the first place, they use it to sell for commerce, but this information can and is being used for political purposes. And, it's not only, about understanding people. It is used to manipulate people in their choices, and not only commercial choices, but also democratic choices. And we have seen this before. It is documented. So this is at the critical core of democracy. This concentration of knowledge in the profiles about people, I think that is, a a core problem. And, honestly, if we want democracy, we have to get rid of these all encompassing profiles. And I'm not talking about your playlist on Spotify or your loyalty card, with a supermarket or, with an airline. You know, I'm talking about profiles which consolidate all the information which is in all what I just mentioned, plus everything else you can imagine. Every time you're on the Internet and often also offline, you are being profiled through a collection of your activities of your personal data. And this has to end this this consolidation into all encompassing profiles is a huge problem, and I will explain later why it's why it's even a bigger problem in terms of AI. But let's continue on, the elements of power. They have the power of innovation, I mentioned already. And they have the power because they have the money to buy up competing innovation. They are able to buy up, the next logical development of their own, technology and just integrate it, but they can also buy up smaller competitors, start ups, new market entrants, and just kill them in order to, not have competition anymore. And they do not only, own, the power of data and network technology and now AI. They also own and control the networks, the hardware, in terms of the cables which transmits, information. And they are very, very influential in politics. In particular, in American politics, you can see how they are trying to buy votes, how much money they are spending in every state and in Washington to avoid legislation which is in the public interest to control the content of legislation and to bend it to their will with money. So I think, we have to see all this together. And the final, and also important thing is they use their money to spread narratives, to buy up philosophers, sociologists, political scientists, to develop stories, about the future, which place technology at the center of everything. And, this combination of technological power, data power, network power, power to influence the public, sphere, the political choices which are made in countries by lobbying politics, but also by manipulating individuals before elections, and the power of narrative. All of this together is a huge danger, to democracy in itself. And now we enter the age of AI, and everything gets even worse. Because with AI, not only the collection of data becomes much more intense, because people tell to, the AI systems, the large language models, much more than they were revealing in, the normal use of social networks and, in in their search behavior or watching videos on TikTok or YouTube. They talk to the LLM like it's a friend. So they reveal even more, and all this is being used for profiling. But also, based on the profiles, the targeting of people, the individual manipulation from the strongest computers of this world with AI, of course, gets much more intense. It's quite logical. You can use AI to manipulate people in a much, let's say, more sophisticated way than you could do it before AI. So, our theme today is, is democracy being supported by AI, or is it being undermined? And my view on that is very clear. There is ample evidence already that AI undermines democracy. And there's a lot of talk and a lot of wish that it supports democracy, but that's not real. We don't have large scale examples of empirics where we can already show that AI serves democracy. But we have large scale empirics already that AI undermines democracy. So, we have to look at the empirics and not get lost in theoretical ambitions and theoretical dreams about how great AI could be for democracy if we would just be using it in the right way. Because the likelihood that AI is being used without the companies using all the other elements of power which I just mentioned in their interest, and also an AI in their interest. After all, they are... These companies which have the data about us are also among the top players of AI is very small. It's, very unlikely that suddenly, these huge, corporates... And I would call, also now, OpenAI and and and Antropic among these, you know, why should they suddenly turn to be good citizens and supporters of democracy until now? They have basically stalled any type of legislation, and, they have resisted, any type of strict obligations being posed on them. And we have seen the fight they were fighting in Europe, and this fight is by far not finished because, obviously, it continues also into, enforcement, activities. So in short, it's crunch time for democracy, and it's time for individuals to engage on this issue.

Well, I think, we... There are a number of, constellations we have to think through. If it is true that AI takes over big part of our work, it means that people do not have, the context anymore in which they normally have their political debates. Because the classical political opinion building takes place at the workplace. This is where people talk with each other. This is where they meet. This is where they make their social connections. And, if that falls away because AI either takes over the job or AI allows people to work on their own at home or as digital nomads, or as crowd workers, it's very difficult for them to engage in democracy and have, the type of dialogues which you have at the normal workplace. So I think AI as a technology already on this point, is something which will become a problem for democracy. The second point where I think AI as a technology becomes a problem is that it disqualifies us. You know, how are people going to be motivated to learn if everything is... Seems to be known, by AI? And how will people be able to check on the political bias of the answers they get when they ask questions on AI? You know, that is much more difficult than if you read your daily newspaper where you know these guys are on the right, these guys are on the left. You know, newspapers have a line normally, and it is known. And we make choices on this basis. The ideology of AI is everything is scientific, Everything is objective. We give you the one best answer. That's, of course, a huge lie. The Max Planck Institute of Intelligence Systems in Tubingen has a website where you can ask your questions, and you get 21 different answers. And in the first step, the people at at the Max Planck, they don't tell you who's answering. And I have tested, on this website, the political bias of AI answers. I have asked political questions. For example, I've asked the question, what is the evidence that AI undermines democracy, and what is the evidence that AI supports democracy? And the answers from the AI system were hugely different. The large majority of AI systems, gave an answer, which I think, is really not a good answer. It said, oh, AI is a technology. It's neutral, and it can be used for good purposes and for bad purposes. It all depends on the governance. Well, honestly, I don't believe that's true. AI is not a neutral technology because it is produced by people who have interests for their purposes. And, so it's too simple and under complex to say, oh, it's like any other tool. It's not any other tool. It's, it's a technology of domination. And that's the difference to the hammer and the knife. The people who develop AI want to dominate us, and many of them want to overcome the human. Many of them are post humanists. They think that humans are basically failing information treatment systems, and they believe that AI is the better information treatment system. And therefore, it's absolutely okay in their eyes to push the humans out and have it all done by AI. Certainly, Elon Musk is one of those. So what I'm saying here is, you know, with a little bit of thought, one discovers very easily that already today, AI has huge potential and does actually undermine democracy. I I would go as far as saying that the business model... And that's not an AI, only argument. That's also business model argument. The business model of social networks combined with AI even more than before, polarizes society, drives people into the extremes, and that's what we see in America, and that's what we see in Europe in the elections. More and more elections worldwide lead to a rise of extreme right wing and extreme left wing. And that's not a coincidence. But, that is, I would say, empirical evidence of the fact that the communication environment, which social networks combined, with AI create, is good news, for the extremists because it, the the business environment to make money, to keep people on the screen, to maximize clicks favors extreme views. On the other hand, you know, your question, well, you know, let's just think about the potential, of the technology. Yes. We can dream, and we can tell great stories. And that's part of the hype to tell to business, you know, everything will be more efficient when you do it with AI, and to tell to societies, oh, you know, all your problems will be solved with AI, including the problem of consensus building. I mean, just invention. DeepMind had to to build a machine which they called Habermas machine. Habermas, the great philosopher, who, at the center of his academic work, explained to us how communication in the political environment in the public space work and how consensus is built. And they really went out without the agreement of Habermas. This is in my book, because we have talked to Habermas about this. Without his agreement, without ever talking to him, and claimed they've created a Habermas machine which simulates the consensus building fi... Building, which he describes in his philosophical theory, which is total hubris. And, we wrote to them, and, they said, okay. We will not use this name anymore. But, of course, the scientific, quote, quote, scientific, I say, I don't think it's scientific, but but but the publication is out and is being quoted as an example of how AI can, help the democracy. Well, you know, I have heard from, Justin Hendricks, the editor of tech policy press and who's a professor of, informatics at Cornell University that they have tried to replicate the Habermas machine, and it didn't work. So replication was not possible. And that's just one of the many examples which I could give you where, you know, a story is created claiming that AI can help democracy, but in reality, it's not happening and it's not working. In the end, you know, democracy is about people talking with each other, and I would even go as far as saying people physically talking with each other. And because, you know, democracy means sustainable long term engagement. It's often a fight. You need to rely on the others. So the type of solidarity and the type of sticking together, which you need in protracted long term political fights, is only built if you know the people, if you see them physically, if you speak and discuss with them, not just once, but many times. So it's a genuinely human thing, and I don't think, mediation helps, helps a lot in that context. Sorry.

Just a moment. I have to take this once.

Francesco, no problem. Tell me... Let's be quick because I'm in a video conference. Yeah. Send me the Just send me the address or the name. I'll find it. And 01:00 is fine for you? Yes. Okay. And 01:00 is fine for you? Okay. See you later. Yeah. It's good. Okay. Bye. Sorry. Now I switch it off.

Sorry. Sorry. Sorry. Okay. On we go.

No. I think, there was, let's say, historic experience and evidence in media like television and radio. Of course, television and radio have become essential parts of public communication and the public space, like the newspapers. But if they are not regulated properly, they kill democracy. And the example is, in, you know, the times of Hitler, in the end of the Weimar Republic, the radio, and the monopoly in the printed press in Germany, helped, Hitler to rise. And certainly, for the fascists, the radio was the key instrument of domination. And the consequence which was drawn from this after the second World War in Germany... Well, in in Germany and in Europe, and in fact, was imposed on Germany, by the allies, was a separation of powers, vertical and horizontal, including on media. Radio was by law and TV, by law, completely decentralized. So there's no, federal, TV domestically. Like in Italy, Rai as a national station, it doesn't exist, in Germany as a result of the experience of fascism, but it has been decentralized. So that's my first point. Technology, which tends to centralization, is never a good thing for democracy. If the technology tends to, to be, you know, centralized, and that is clearly the ambition of the developers of AI, then that we must insist on decentralization decentralization by law. Because the market is not on its own delivering this. The private TVs, you have seen this also in Italy, you know, with Bella... Bellosconi, you know, national TV, started decentralized but became very centralized. So, that's the first point, and there are many other reasons why media need to be the public space, needs to be regulated for democracy. If you leave it just to the market, it doesn't deliver democracy. And you can see this in many countries of the world. So that is my comment on the technology, which is probably closest to social networks and AI. But AI is a special case. Because so far, when we talk about telephone, or we talk about social networks, or we talk about book printing, These are distribution technologies of content. AI pushes people out of the production of content, of the thinking. And that is a completely new quality. And, so I think that the danger, to, democracy, which needs thinking people. Democracy only works which people which think themselves, which are responsible, which take part, and engage and inform themselves and get information that they can act, as citizens, if they are... If if if the technology agenda is to push people out and to overcome them and to produce all the content they previously produced in their minds through the machine, well, where does that leave democracy? Where does that leave thinking people? Where where does that leave, responsible people? So I think, it is not correct to discuss theoretically about this technology only, but we always have to see this technology also together with the ambition which is behind it. The ambition behind AI is to demonstrate that the machine can do everything better than humans, and therefore, humans should be pushed out. They should be pushed out of work, out of business processes. Mister Altman said it very clearly. The ideal company for me is me plus 1,000,000 agents. AI agents, no human. And, of course, push them out of democracy. And let us govern. We want to govern. We are the new oligarchs. We have the technology. We want to govern. That is the ambition, and that is the agenda of most of these technology actors, and that's why they also happily line up with Trump.

Yes. I think on both, we have to be realistic. They were political compromises, at the time between a grand coalition of social democrats, conservatives, liberals, and greens. So they are halfway. You know? Some others, you know, if you come from the more, market neoliberal side, you would say, oh, we have avoided the worst, in terms of intervention in the market by by by the legislator. I am more on the social democratic side. I would say we have not gone far enough. So, you know, there are many things which could be done better in a second generation of these, laws in the public interest. The second problem is enforcement. For me, it's really the main point of weakness of GDPR enforcement is that none of the enforcement actions so far has gone for these all encompassing personality profiles, which are in the hands of Facebook, Meta, also Instagram contributes to that, of course, and Alphabet through, the search, now also through Geminis and through YouTube. These two companies not only cash in 80% of all Internet based advertisement, which previously was money for the free press for journalism, and they are killing journalism. But they also constitute the core of data collection, which is then used to create these all encompassing profile of... About all peoples. And I am of the firm opinion, These profiles are absolutely illegal under GDPR, but the problem is the enforcement authorities are so small and so weak, and none of them dares to go for this central element of problematic data processing. Because it's not only about the collection, it's also the fact that they consolidate the data, that they process it to to learn about people, and, that they, thus, do things which are completely unproportionate. And they collect too much, and keep the data too long. And so, we have huge enforcement problem in Europe also because of the fragmentation of the enforcement authorities. And, we have a very specific enforcement problem with the AI act, which is that in contrast to GDPR, people have no rights, it seems, to invoke the AI Act when they feel that they have a problem, with how AI is being used in relation to them. Why do I say this? In GDPR, it's very clear that an individual has the right to complain to a data protection authority, and the data protection authority has the duty to follow-up this complaint. In the AI act, it's not clear whether the authorities have a duty to follow-up or whether they can just say, yes. Yes. We take this into account. Thank you very much for your letter. But then do nothing specific. You'll have to see how things develop. The proposal of the commission of the European Commission was clearly intended to exclude to exclude individual rights. The European Parliament added to, the AI Act a democracy impact assessment, but only for the use in the public sector, so not generally, maybe one can draw honey from this addition of the parliament, namely the democracy, the fundamental rights impact assessment, and argue, well, if an act contains a fundamental rights impact assessment, it serves the protection of fundamental rights. And therefore, people should have an individual right to complain, and there should be a duty of the enforcement authorities to to, follow-up. But for that, we need jurisprudence. And I think this argument can be made. I would be willing to make it, but I'm not sure the jurisprudence would confirm that this is the right way to interpret the AI. My view is at least the commission proposal at the outset was clearly intended to exclude individual rights. And that's a huge problem because it means nobody can force the enforcement authorities to do anything. And that's the difference to GDPR. You see, GDPR, there have been cases already where civil society and individuals have forced enforcement authorities, which, you know, have a natural inclination not to do anything. I can say it. I was myself a civil servant. You know, the natural inclination of the civil servant is to do nothing. But they... But in GDPR, they can be forced. And that's important because the enforcer needs to know that also if he or she does nothing, he or she will be taken to court. Not only when he or she takes a tough decision. That's the big fear. Yeah. You know, I take a tough decision, and then I'm in court for years because, big tech companies are so rich. You know, they can go always to last instance. They put all the best lawyers of the world at this. And, so my work as a regulator is bogged down for years. That is the reason why many of the enforcers don't do anything hard. At least, if an enforcer knows, if I don't act, I will also be taken to court. You know, there's an incentive to do something.

Yeah. Well, I mean, there is, of course, the famous book, called Code as Law, and, and the author, Harvard, constitutional law professor, he said, 200 engineers rule how we live. And, of course, that's exactly not democracy. That's exactly what this company wants. That they set the rules in the community rules, how we communicate, and how we live. So I'm not a great friend of of of this, this idea. And, you know, code is written for stupid. Who are the stupid? The stupid are the computers because they cannot think themselves. That's why we always need a revision of the code, update. We see it on our mobile phones, you know, every two months. We we need an update because the computer can't sync itself. Law is written for humans, and humans can think themselves. A good law can be reinterpreted in light of developments of business model, of technology, of society by people, namely judges, who can think for themselves. And so I'm actually not convinced that to give shape and control, and and direct technology, or for that matter, behavior in society, it's an advantage to put it all in code. I would say the legal system, you know, is a pretty efficient system exactly because you don't need to rewrite the code, the law, all the time when there is a change of circumstances. The trick is here that law must be, of course, technology neutral. And it must be not too specific, not using the buzzwords of the day, but written at a certain level of abstraction so that it keeps meaning also if circumstances change. But there's another argument against code as law. And that is, if we mean it with democracy, the laws must be understandable to people. Now I granted to you that many laws today are complicated, and if you read it as a normal person the first time, oh god, it's difficult. But at least, it's written in the language which you understand. And if you make an effort and if you research a little bit, you will understand the law. But with code, I don't think we can hope for that people understand, you know, and that we have parliamentary debates about the code and and this and that. That is much more removed from the normal individual understanding code than understanding language. And, so, you know, I would say, let's rehabilitate the normal human language before the code. I think it is fundamentally wrong to say children must learn coding in school. No. I would say they must learn to express themselves properly in the language. Also, by the way, in order to live together peacefully and, you know, in a way, which is, you know, a good way of living together. You must master the human language. And, and that should be our priority. We have to rehabilitate, the normal language before the code. And also for democracy, the language is key. So, I don't believe... I know, also in Germany, there are, and and all over Europe, also the OECD has projects, to digitalize law. Yes. You know, that can work on some types of law, like traffic law and law, which is very numerical, like tax law and so on. That can work. But digitalizing constitutions or digitalizing, take AI act or or GDPR. You know, honestly, I think that is again hubris of the engineering world of computer science to think they can do everything. They cannot do everything. For example, Stuart Russell, who is a computer scientist, says it very clearly in his book, Human Compatible, The Problem of Control. He says, every AI must be programmed in such a way that when a value judgment is necessary, the program must stop and return to the human because AI cannot do value judgments. Value judgments can only be done by humans. So I would say, you know, we have to fight the narrative that humans are useless. Everything can be done by AI. And, we have to insist that democracy can never be done by the machine, and that there are many things which human can do with with human language,

Yes. I would say, the challenge is to democratize the digital and democratize the AI rather than the other way around to digitalize democracy and, you know, make democracy the object of AI consensus finding or or whatever. So, let's, work together to democratize AI, to decentralize AI as a technology. Let's develop. And I call on you because I can't develop. I call on people who have the technological knowledge to develop AI, which is the central, which doesn't allow power concentration, which, spreads wealth, in different countries and different regions, and, which is also controlled by different entities and, you know, not just by five or six or 10 companies in the world. I think that's what we should be aiming for. And I'm happy to hear from time to time that edge computing and maybe also open weight, AI models may go in this direction. Every type of decentralization of the technology itself and of the control of the technology is a step forward and is good for democracy. That's the first point. The second point is democracy needs the technological knowledge of the new technical intelligentsia, the people who work with digital and AI, which develop it, which use it as professionals, and so on. And, I would invite all of you to participate in one way or the other through a party, through an NGO, through a trade or or a professional organization in democratic dialogue and democratic process. That's what our society needs also from technologists, not just the technology as such.

Thank you, Alessandro. It was, good to talk to you.