Hello, everyone, and welcome to Medigov seminar. These are our weekly, research presentations that we host at the Medigov community. We invite folks researching digital governance from all around the world to come and share their work with us, and it's awesome to have you all here. And I'll pass the mic to Liz Berry, our executive director, to introduce today's speaker.

Thanks very much, Liz, for that very generous introduction. And apologies from Aviv as well. He's just completely swamped at the moment, but he would love to be here. Let me just share my screen. So this talk is to give you all an an overview of what we're calling the democracy levels framework, which was published recently in an ICML position paper with all these wonderful coauthors. I think the talk will be relatively quick, certainly under half an hour, so there should be heaps of time for questions. But the the goal is just to sort of motivate it and orient you to the different parts of it. While it's published, it's still, I think, fair to say, a work in progress, and we're iterating on the maybe exactly how to factor up the different concepts and dimensions of democratic quality and so on. So any any feedback is very much welcome. But with that context, let me walk you through it. And I'll just say also sorry. I'm presenting within a a browser window. I just there's a couple of other tabs I wanted to flash to at certain points, so that's the the motivation for that. So the motivation for this is that we are broadly interested in AI governance and ensuring AI alignment and governance stays broadly human centered. And to do that, we sort of see that there's, like, these three levels at which you can steer outcomes, the AI system itself, which is governed by the various alignment processes. There's the organizations that create and operate AI systems, and then there's some sort of external regulator that governs the the the those organizations. And each of these are sort of subject to varying degrees to democratic infrastructure, which feeds back information about what people want when they make decisions of varying degrees of democraticness. And so, like, at the AI system level, you could think of the the the training data is in some sense an aggregate of what people express preferences to be in some form. They're very imperfect, but that is some sort of aggregation process. And then at the regulator level, there's existing, obviously, democratic institutions that, to some extent, influence what decisions regulators make. But we think improving this democratic infrastructure is critical for, yeah, ensuring how AI impacts society and how society changes to adapt to AI stays human centered. There's lots of examples of versions of this being made explicit in various ways with to varying degrees of quality. So at the AI system level, there's things like collective constitutional AI that the Collective Intelligence Project did with the Anthropic, where they used POLIS to sort of collectively author a or what generate clauses which would go in the constitution, which is then used to train a large language model. At the regulator level, there's various forms of public input into and public consultation into to what regulations of AI should be, and there's a spectrum in between. Another big example is Meta's community forums that use, Stanford's deliberative polling, platform to to conduct large scale deliberative polls on what Meta's approach to sort of policy of how they they design their Azure product should be. So that, yeah, there's a there's a spectrum here. But we think to sort of ensure that these efforts continue to build on each other and to improve and don't peter out, but progressively become more substantive over time. We need a framework for sort of articulating what success looks like, which was so that was the motivation for for this work. It is more general than just the, like, AI alignment and governance context, but that's the the context in which we're most interested in applying it in the in the short term. So with that in mind, the we propose this framework, which we think, as I just said, can be used to sort of milestones or a measure of or a a way of articulating what the goal might be. It can also be used more practically to help, like, say, specifically, what is the next step if you've if you've run one of these processes? How do you do a bit better next time? And then, equally, it can be used as a evaluation framework to sort of sort of, yeah, provide a language for talking about how just how democratic is this thing that we've done. We've run some public input, public engagement, collective decision making exercise, but how how substantive is that really both by organizations who want to improve themselves, but also by sort of outside observers wanting to hold them accountable to their their stated level of democraticness. There's another way of sort of looking at what this framework provides, which is that we think or to to a large extent, the the framework as written is kind of agnostic on the degree to which any particular decision should be made democratically. The framework as a whole is more about helping people ensure that to the degree that a given decision should be made democratically, that the decision sorry. The the the process that's used for making that decision, the democratic process or process or democratic system, is good enough to handle the level of black decision making responsibility that it's handled, and that both of those things are sort of well calibrated to the context in which that decision is being made. Maybe the degree to which a decision should be democratic differs depending on the the political context. Maybe the the level of democraticness of or or the quality of a a democratic system, making a certain decision necessary for being handed that level of responsibility also depends on the context in in different ways. So the framework aims to scaffold how to think about those kinds of compatibility issues. Very much welcome on on that as well, but let me talk you through it. So the setup is somewhat abstract, but, hopefully, general in application. We basically assume that there's these two entities for for some given domain of decision making. We assume that there's a unilateral authority that is sort of the the status quo decision maker who has the power to make that decision, and then there's some alternative democratic system which could be used to make that decision. We have for the moment, we abstract away exactly what that democratic system looks like and how how good it is and how it's designed. But the the core of the framework is about the the transfer of decision making power or responsibility from this unilateral authority to the democratic system. And this is inspired a little bit by some of you might have seen there's these in a completely different context. But so for self driving cars, this is the Wikipedia page for self driving car. There's a fairly widely adopted standard on autonomy levels or automation levels for self driving cars, which is autonomy levels or automation levels for self driving cars, which is used for basically characterizing how much decision making responsibility has been transferred from the the driver to the to the vehicle. And, obviously, it this is not a perfect analogy because in in the self driving car case, it's sort of going from the the person to the machine, whereas in our case, we're going from one one actor to more people, a more human oriented approach. But the the basic structure of this was an inspiration, and as you increase automation levels progressively more or, like, the domain of decision making responsibility that the the the alternative handles gradually steps up. And so in our case, we were sort of the the core of the framework answers that question, but for going from some unilateral authority to democratic system, and we've structured it somewhat analogously. So we have five levels. Level zero, there's no decision making roles performed by the the democratic system, so it's just a unilateral decision by the decision maker. At l one, some democratic process is used for informing the decision, but there's no nothing more than that. And the unilateral actor doesn't have any they're they're not required to sort of take into account that information in any particular way, but they that's sort of the process the democratic process is conducted, and it does notionally inform the decision. At level two, the democratic process doesn't just sort of inform it, but it provides specific options which could be taken and implemented without any sort of editorialized action by the the unilateral authority. At l three, the democratic process actually makes the decisions, and those decisions are are binding without the ability to to veto. At l four, the process doesn't just make decisions, but it has the power to initiate when there should be some democratic process run to make a a certain decision. And maybe that's also not just sort of at the discretion of the democratic body or system, but it happens at some regular intervals or it happens based on some trigger for when a decision needs to be made. So you could think of maybe when a a a lab is deciding whether it's safe to release a model given a particular set of benchmark trade offs that they've observed, that might be a trigger that initiates some some deliberative process. And then at l five, the the democratic system or process is responsible additionally for its own meta governance. So it sort of has the ability to make binding decisions within a given domain, but can also democratically determine how its own structures and processes evolve over time. So that text on the right is basically what I just said. You might be thinking most of the systems in the world are pretty low on the next levels, which is true, but it it's meant to be not too constrained by what we have at the moment, but to, like, yep, provide a a a framework which stands what might be possible in the future or might be desirable in certain context. So that's the levels. A second question you might have about this general setup is is what does it mean for a decision making system to be democratic or or or be good in a democratic way. So for that, we have this second part of the framework, which we call the dimensions, which are broadly dimensions of democraticness or or things that contribute to democraticness in the system, which are loosely clustered into three groups. So there's the quality of the decision making process itself. There's the dimensions related to the degree and the way in which the unilateral authority delegates or transfers the power to the democratic system, or, I guess, more the way they do that rather than the overall degree because the degree is captured by the levels. But I can talk through each of those if there's interest. And then trust, which or what we're calling trust, which is dimensions which relate to, like, the external conditions or or things that are not in the the sole control of the either the unilateral authority or the democratic process, but the the sort of external conditions in which they both exist and help support the success of that, that democratic exercise. So each of these dimensions has more detailed definitions, which we get to in the paper. And happy to answer questions about any of those. I think if there's one part of the framework which might reasonably evolve a bit over time, it's probably these dimensions, but happy to to to discuss that. And then there's two more questions which you might reasonably ask about this setup. One is for a given context and a given domain of decision making, how much power should be transferred or, like, to what degree should a given decision be democratic? And then the second question is, given that sort of normative prescription, is a given democratic system that you might be considering, is it good enough to handle that level of decision making responsibility? And so we don't have nice clean answers for either of these questions because they're obviously contested and very complicated and context specific, but we do have some tools for helping think them through, which are namely the levels decision tool and the democratic system card or what we're calling those two things. So the levels decision tool don't worry if you can't read this. I'll show you a closer version in the section, but in a second, but this is what it looks like in the paper. It's basically a a list of considerations for the or or reasons why you might think more democracy or less democracy is desirable normatively desirable in a given context. So let me just I'll I'll just mention now there's a website, democracylevels.org, that has a bunch of resources related to this paper, including the ones I'm about to show you. But from there is linked a template for the both the both the tools. Let me zoom in a little bit. So this is the the levels decision tool template, and you can see it's basically just a a cluster list of considerations. But so for example, if if the decision involves values laden trade offs, if it involves significant public interest concerns or externalities, and there are reasons why you might want it to be more democratic, if there's if it's a private technical or operational matter and doesn't involve too much in the way of normative considerations, that's probably less less likely to need democratic legitimacy and and so on. So there's a a long list of considerations here, many of which are quite sort of practical in in nature if you're, like, leading an organization who who your the actions you take have an impact on society, but you don't obviously, not every decision you make as an organization can be made perfectly democratically or or seeking significant external engagement because there's significant costs involved to that. This is a set of considerations that can help you think through, you know, a particular domain whether a given decision should be democratic or not. And then so that's the level levels decision tool. The the second tool is a democratic system card, which is to answer that second question about is a given democratic process good enough to handle the level of decision making responsibility that is handed to it at a given democracy level. And so, again, this is the sort of formatting of that in the paper, but we have linked on the website a more usable version of that. And, basically, what this is is a set of questions for helping you both describe how each of these dimensions that I introduced earlier, how each of those is operationalized in your system, and then evaluating the degree to which they not not just how they operationalize, but the degree to which those dimensions are are met or fulfilled by by your system. So, for example, informedness we define as the extent to which those making decisions understand the information critical to making that decision. So there's a spot to describe how that works in your context. And then there's a bunch of sort of questions to prompt whether or, yeah, how well that's that's implemented and could it be better, and what are the limitations and so on. So this doesn't lead you to a, like, a definitive quantitative score on how how democratic is the system, but it does help you flesh out ways in which the system can be more or less democratic and and reflect on that and reflect on whether the the system is is sort of good enough to handle the the level of responsibility that's being handed to it. There's a couple of examples on the website as well. There's an example in this doc as well for a hypothetical UK citizens assembly where you can scroll through and see, like, what what what some answers to these questions might might look like in practice. So I'll finish there, but happy to answer any, you know, questions about this. These are the wonderful coauthors from many different places in the the general community. Thanks so much for having me. Looking forward to questions from this guy.

Yeah. I I'll preface this by just saying I'm I'm speaking for myself. I can't claim to speak for all those authors, obviously. But I I think it would be interesting to evaluate existing attempts or at, like, examples of so some of those I've flushed up of of ways that, say companies or, regulators or or so on have tried to run these public input exercises, which they often frame in democratic terms, to, like, evaluate them on some consistent standard, which could be the the levels. And I I think often you would find that they're all quite low down. But that that's the sort of that like, a a useful signal, I think, as a like, you can just show the gap between maybe there's domains where you think there's a convincing case that a high level of democracy is required, and in practice, that's that's not being achieved. So there's that sort of accountability mechanism. That's pretty general. Did did you have, like, a more specific context or or communities? Or

connects to the public AI work if that's what how how I think it connects. I I guess in in my head, even if public AI is super successful at getting up public funding for foundation models, there's still a lot of decisions about how the use of that model impacts society, and it would probably suffer from a lot of the standard limitations of Oh. There just being a relatively low sort of information low bandwidth information channel between the public, that model, and the use of that model is meant to represent and the things it actually does in the world. So there's probably lots of implicit decisions that are being made in the design of that whole paradigm and the the way it's operationalized, which within each of those, you could, like, identify particular domains of decision making. And maybe some of those decisions someone is actually making or a regulator or or someone or some series of committees, others decisions might be being made just sort of implicitly or by default in the way the system has been set up with no one sort of explicitly expressing their view on on that particular thing. But I yeah. I I think, I guess, just because something is publicly funded, it doesn't mean that the, like, the critical decisions about its design are highly democratic in the terms of the dimensions that we, flesh out here. And, obviously, it creates the possibility for that level of input and contestation and accountability and so on. But I I think there's, like, even in the the world where public AI is very sort of successful, I I think there's still room for, like, evaluating to what degree is public AI and the way it impacts the world democratic in in various different ways, which is it's a different way.

Yeah. Just I mean, I'm very much along the lines of Joseph's question, and I but I do have a particular viewpoint I wanna express afterwards, perhaps.

am. That is

Yeah. Completely fair. Like, it it didn't it it didn't play a significant role in our developing the framework. It was mainly well, I guess so a lot of the talk about democratization of AI, that term is often meant to pull it. It's used in lots of different ways, but by a lot of actors, it's used to mean increasing access to use of models or or access to open source models or or so on. Whereas, I guess, the, like, implicit view in this framework is that, like, more substantive meaningful democratization means there needs to be some sort of transfer of power from some unilateral or or small small group of decision makers to some sort of democratic decision making process or system. And the place the or the role that the self driving car analogy played is purely in just some inspiration for how people have, in another context, sort of carved up the degrees of transfer of decision making power. And, like, it it was really sort of in just, like, the general factoring of that space and how how people talk about degrees and breaking it up into, like, different decision making roles. That's that's the extent of the influence it had. But, yeah, I I I completely agree. There's lots of well, like, don't don't think about it too much because there's lots of flaws in the analogy. It's more Okay. Information design. You know?

Yeah. I thought it might come up. And and we do cite this in the paper as well. It was also inspiration, I guess. Yeah. Like, I this this is also doing a very similar thing. I think ours is maybe sort of well, it's obviously factored slightly differently. I think it's probably, like, just framed slightly more neutrally and that it I don't know. There there's a lot of sort of, like, connotations or loading to the terms in this letter, which in some domains of where you're trying to change the world, that's useful. In other domains, it's less useful. So, yeah, I I see them as complimentary, but happy to discuss further with if anyone has questions.

Oh oh, okay. That's not 09:28 for me. So it's twelve how am I babbling about? Oh oh, yeah. Yeah. That's interesting. Right? So the idea is I think this framework seems detailed enough. Obviously, I haven't looked at the paper yet, but you could feed that and any sort of procedural documentation you have from an organization and just let the LLM figure it out, how well it conforms to your levels and your dimensions and so on and so forth.

For sure. Yes. That's how we

Cool. You know, I'm gonna do it immediately.

Great. That's it.

Yeah. Well, Sabeet, thank you for the super thoughtful comments. And, yeah, I'll be be thinking more on each of them. I broadly so two aspirations for this framework are both to connect it better with political theory as we iterate on it, and we do like, we our aspiration, which is probably imperfectly achieved at the moment, is that it does apply to democracy in general even though, I guess, a lot of the other things that many of the coauthors are working on are more in the deliberative democracy space. So that definitely had an influence. I I I think we were probably we left out a definition of representation mainly to try and, yeah, abstract away from like, we're aware of the the different conceptions of it in in political theory, but it it was yeah. It was an it was just like an area of contestation we we didn't want to get into, but I I think it would be good to sort of grapple with that more substantively. Your example of a a case where everyone shows up to make a decision and that being sort of absent of representation. In my head, that sort of, like, maximal represent or, like, optimal representation where everyone's representing themselves if if you allow for people to represent themselves. I I guess in correct me if I'm wrong because I'm not a political theorist or a democratic theorist, but my understanding of, like, a contrast that people might often draw between more representative notions of democracy and other forms of democracy might be something like, the the contrast might be with something that's more agonistic, and there's a lot more sort of contestation that no that, like, it's that's less focus on electing representatives or or descriptively representing through sortition any particular polity. So I yeah. I I think the framework could do a better job of certainly handling that that alternative conception of democracy. I'd be curious for your your thoughts on that. And if anyone else wants to chime in on the conflation of representation of democracy, I'd be curious for thoughts as well. Steve?

I I mean, like, I'm as outs I don't you know? I don't you you have a very good box, but I don't even know where the box is. So but so seeing being able to find your box that is so, you know, intricate and ornate will give me the exact platform I need to then innovate off of those assumptions and, you know, live in those interstitial spaces that aren't accounted for. So I I look forward to diving into this in-depth and, you know, getting into all the nitty gritty details. For example, you know, I have conceptions of democracy as something that can also be built from the ground up. I'm gonna be putting up immediately after this meeting my first draft of, my fiduciary framework for AI agents, and that's a way that any set of individuals can aggregate into a democratic process. So I think for the most part, top level democracy is lost. It's gonna just wither on the vine, hopefully, if we can rebuild from underneath. But looking at this framework is a way, to contrast my work very effectively with the status quo. So I look forward to digging into it.

Why is that tension there, do you think? In other words, is there an incentive to keep this sort of dumbed down version of representativeness that's being forwarded by certain people?

I think that was to you, Joseph or Luke.

Yeah. That that's super helpful. Thank you. I as you've been talking, I I think what we were trying to get at with representation abstracted from the particular conceptions of it is I I think of the dimensions, it's the only one that sort of is a tether between the decision that's made and what the people in that public or polity want in some sense that I I think you could, like, you could score highly on the rest of them. You could have a very deliberative process. You could have a very informed process. You could have a very tight coupling between the the output of the decision making process and the the, like, system in which it is implemented or operationalized, but still have the decision be something that's completely out of left field and far removed from what people actually want in in some sense or some aggregation over what people want. So I I think that's the role it's playing. But, yeah, we we need to make that more rigorous and grounded for sure.

Yeah. I thanks for your comment, Joseph, as well. I I haven't personally been super closely involved in running processes in this space of sort of learned about various ones that have been run, and I'm pretty familiar with them. But I I'm not sure I'm necessarily the best person to speak to this, but, like, I I think it's a it's a challenge in general to the extent possible. There are certain types of decisions around impacts from AI and how it's adopted that you can talk about the, like, normative trade offs without getting into the details of how the technology works in some cases. I I think that said, often having a, like, a deep intuition for how the tech works is often a bottleneck. So, like, for example, I think we've all encountered the fact that a lot of people still have the intuition that ChatGPT, say, is well, kinda, like, it it's more confusing now than it was when it first came out. But the intuition that it's doing some search over a database when it comes back with an answer to you, which sometimes it is now searching the web, but oftentimes, it's not. It's just sort of confabulating based on its the the training data it's seen. And it's it's not like there's some human readable database anywhere that it's consulting. But that's an intuition that a lot of people have about how these systems work or just, like, the the risk of hallucination or ascribing intent or belief or there's there's a lot of, like, intuitions people have about how generative AI works that aren't that close to the the reality of the technology. So I think it's it's going to be a challenge for a long time and probably always. But I yeah. I think and, also, none of these dimensions also are like, we don't think you can do perfectly on any of them. You can do better or worse. They're all operationalized to degrees, and there's a cost involved to scoring better on each of them, which is why, like, the the framework is sort of framed in. Is the is the overall quality of the or democraticness of the system. Is it good enough to handle the level of decision making in a given context? It's all sort of trying to be pragmatic in the the trade offs and the costs involved in in doing better or worse on these divisions, including in faultness. Yeah. I'm not sure if that answers your question, but that's that's kind of it. I don't know.

I know. It's fine. For sure.

I think, Liz, what you're saying kind of gets out what I wanted to hear Luke maybe speak on a little, which is, like, the actual use of the framework and, like, if and how you envision, like, organizations maybe, like, on the implementator side who are, like, being pressured to adopt AI to automate some processes, which is like, okay. You know, sure. There is, like there could be a helpful implementation of AI to make their jobs easier, especially in, like, nonprofit context or government context, but, like, kind of doing, like, a slower adoption of these tools, like, where your tool might fit into that, like, process that is actively happening in these organizations and, like, how to like, do you envision this being, like, a workshop that you could like, you could use this tool in some kind of workshop with these organizations? Or yeah. Like, what kind of environments do you envision people using this framework in?

Yeah. That's a great question. I I think there's a lot of often, there's, like, one advocate for more participatory democratic approaches within a larger organization, which has other interests and incentives. So I I think often we're targeting those advocates as people who need a bit more, like, an armory of language they can use to sort of articulate what better or worse looks like. And, yeah, like, sort of draw some hard distinctions around just how how good a system is, but also a sort of like, I I think the framework can not our sort of hope is that it can be a common language that when when companies or organizations adopting AI or deploying AI do do well according to the term like, the the terms of the framework, but that's something that they can be sort of rewarded for and recognized externally as well. I I think the there's obviously more levels of detail that the framework doesn't get into, which I think would also be super helpful for people. And this also goes a bit to the political theory point. So, like, I think please correct me if I'm wrong on this, Joseph, but in in my interactions with the literature, it it was something that might be super helpful for someone in an organization making decisions would be, like, just tell me which decision should be democratic and which ones is it okay for them not to be democratic? I don't think existing democratic theory is sort of structured in a way that just blank provides a a checklist of criteria that helps someone who's not a democratic theorist make those judgment calls and to sort of defend them to other people in the organization. And that's something I would love to do moving forward is, like, help sort of just concretify and, you know, definitely not universally accepted. But, yeah, like, sort of translating political theory and democratic theory in a way that makes it a bit more concrete and actionable even if it's not universally accepted. If you you could sort of build enough of a consensus around that those kinds of sort of tools for making normative decisions as well, I think that would be super helpful.

Oh, thank you.

Mhmm.

No. It's a great question. I yeah. I think this is getting into the domain of what decisions should be made democratically, which I've thought a bit about. I would like to think more about, but I I guess in at at least from my own sense, I'm more convinced that the sort of the externalities and I I feel like there's a bunch of sort of massively consequential decisions, which are sort of like forks in the road that society could go down as we adapt to AI. So, like, what I don't know. Should agents have legal personhood, or to what extent should they like, many people will think the answer to those questions is obvious, and maybe there's a a massive consensus on them already. But they they're certainly not decisions that should be made by one guy in a room somewhere that they should have those sorts of, like, big trajectory questions. I think there's a a strong case for them being made using some sort of democratic system. As you get to the more technical level of, like, particular systems and control over inputs and outputs for particular system, I I'm very uncertain in general. But you can imagine cases where, like, so like the ARIA in The UK, some of you might be aware of Dufferdad's safeguarded AI vision of sort of deploying advanced systems in particular constrained domains with formal guarantees that they will adhere to some formal safety specification, and his proposal is that safety specification which formally specifies a set of trade offs between sort of observables or metrics in the world that capture things that people care about. That specification would be something that's developed with significant deliberative input. So that's that's one way in which you could the more, like, individual system level constrain a system and use deliberative or democratic processes to make the the normative the questions about normative trade offs. But What's

the particular paper you're referencing?

It's I'll give you the keywords to search to look for a bit. ARIA safeguarded AI. I I yeah. It'll come up here from Google if you search that. But, yeah, that that doesn't quite factor it into inputs and outputs, but it it factors it into what is the effect that this system has on the world when it's making decisions in a given domain.

You can unmute on top for our speaker really quick before we log off. Thank you so much, Luke.

Yep.

Have a great day, everyone.

Thank you.