Everyone. So hi, everybody. Welcome to the third week of our four week, seminar series called, On and Off Responsibilities and Governing AI. Today is 02/14/2024, and we're joined by Vara Shankar, who my cohost Manan Raval will be introducing briefly. This is a seminar series that's hosted by Medigov, and it is conducted in collaboration with the Responsible AI Institute. You will be able to, later on, find recordings from our first seminar session as well as some recaps of the second. And, yeah, I think that's usually all we need in terms of introduction. So without a pass over to Minh on, who will give a kind of overview of the session today and introduce our guest speaker. Minh.

Great. Thanks so much, Mano, and and thanks to the Medigov team for for hosting us. Would love to keep I I have a presentation, but would love to keep this conversational. I also am recovering from whatever my older son brought home from school, so I will kind of be taking breaks to to sip some hot water, so might might be a little bit slower. So I'll just elaborate a little bit on my background. So as Mano mentioned, I'm a lawyer by training. I joined after practicing law at a big law firm here in New York City. I joined the provincial government in British Columbia, which is where I grew up to work on kind of citizen data initiatives, and user centered design for government services. They had kind of developed, something they call the services card, which is a digital version of a driver's license and a health care card, and it opened up a possibility to kind of provide all kinds of digital government services. Of course, it also raised a whole host of questions about the digital divide access across the province, and it really only picked up as an unintended result of the COVID-nineteen and pandemic response, which kind of forced the question of, yes, you do have to provide all of these services remotely. And then while in government there, I also worked on the AI strategy for for the province. In 2021, I co developed with a friend at Google the AI ethics course for Kaggle, and that work was so interesting to me that I kind of joined this organization shortly after that to work on this full time. So I will just quickly share my screen. Let me know when you can see. And also I can't see you, so let me know by seeing something.

Yes. We can see your presentation

on you. So the Responsible AI Institute is a nonprofit organization. We are we work with policymakers and regulators. We take a community focused approach, and we represent subject matter expertise both in kind of, the the sectors that we focus on, but also in in the jurisdictions of, The UK, Canada, and, The US primarily. We work closely with a host of members from different industries, and our approach is to help these members develop their responsible AI programs kind of at at inception in a responsible and scalable way and then to kind of provide ongoing hand holding and that bridge to civil society and policymakers. So the kind of just to situate where where we find ourselves, we're seeing kind of quick improvements in, in AI, and then we're also seeing a drop in adoption time. So, I was surprised to see the statistic that ChatGPT grew way faster than TikTok or Instagram and had a 100,000,000 active users active monthly users within two months. And now, of course, you know, you you don't even have to wait for the next drop. It it can just directly be dropped into your Google search, which is, you know, the the kind of retirement of Bard and the introduction of Gemini, has shown us. And then we're also seeing kind of a policy response, that's equally well, I'll say it's complicated, it's rigorous, and it's kind of fragmented globally. So both in The US and The EU, we've seen major action. We've seen other jurisdictions kind of strike strike a balance between the two as well as kind of focus on providing high quality expertise in specific areas as we've seen with Singapore. So what we do is we convene policymakers, practitioners, and researchers to align on what responsible AI means in practice, and the types of researchers that we love to work with are researchers that are really interested in kind of the social, in sharing the social aspects and implications of their research. Excuse me. I think Mano is the really the perfect example of the type of researcher that we love to work with because, you know, she obviously brings that kind of deep technical expertise in statistics and in machine learning, but then also is kind of grounded in political philosophy and interested in kind of the community conversations surrounding the issues that she's focused on. We also have a number of advisors kind of more thematically focused on healthcare or employment or specific kind of research areas. We also provide intuitive resources for responsible AI practitioners. So this could be something as simple as policy temp enterprise AI policy template that you can can download and start using within your organization or an AI impact assessment template that's you can similarly kind of adapt to your purposes. We maintain current AI assessments and an ecosystem map, and then we promote AI certification at the application level, which is gonna be kind of the topic of of the rest of my remarks.

So

in terms of the I I think I'll just touch on this super briefly. This is the sort of event that we love to convene where you get where you bring kind of policymakers, researchers, civil society represented by us, and then, maybe open it up to, like, small and medium businesses and talk about, how how to kind of wrap, flexible policy instruments for responsible AI, into, you know, regulatory requirements, in a way that's easy to communicate. I'll skip this slide in the interest of time. So we've already seen kind of significant guidance come out at the organizational level, so the NIST AI risk management framework that was published last year and then the ISO 42,001 that was published at the end of last year are both examples of kind of guidance at the organizational level. We map all of our materials very closely to this guidance, but we focus more on the application level. And there are a number of different kind of stakeholders that we're that we are that certification kind of helps serve, so I think, you know, organizations themselves kind of get an idea of what what what good looks like. The public knows that their rights are protected. Compliance teams within businesses are are happy that they have kind of a standard that they can move forward with. And then regulators and investors who kind of wanna come in and and take a quick look, get get to see that that the organization is the system or application is being developed with an eye towards responsible AI considerations. And excuse me. So there's three ways that our our assessments can be used. There's the internal self assessment, which is kind of like a checklist, so you can use it as at the design stage or for internal documentation. There's second party, which is, you know, investor or close partner, and then there's the formal certification that's delivered by an accredited auditor and independent third party. So we've been quite successful in kind of in having our certification and rating system incorporated into contracting requirements, sorry, into government frameworks in Assurance and in the Department of Commerce's request for comment last year. We've also piloted it with Standards Council of Canada in an automated lending use case, and we have kind of ongoing automated skin disease detection work using our assessments with Memorial Sloan Kettering Cancer Centre. So we've done a fair bit of work on kind of convening all of the all the right organizations around this this idea and and making sure that it it aligns with every everybody's kind of different approaches. So the certification ecosystem is comp excuse me, complicated. So we we've tried to be rigorous in kind of teasing out the different swim lanes. So we focus on conformity assessment schemes and certification, but we can't do the work that we do without, accreditors, testing and evaluation companies that kind of provide documents, auditors that review the documents, and kind of providers of the best practices. Similarly, the ecosystem kind of varies by by sector, and so it's it's a different stack, for example, for health. And then for a different view of this, this is sorry. This is the autonomous vehicle view of of the stack kind of in a different way. So in even at the at the at the top two pieces, which are, like, providers of best practices, you have everything from really high level guidance down to kind of industry best practices. So this this is all of the authoritative guidance that you would need to incorporate in into AI governance for autonomous vehicles. So our approach to date in developing a conformity assessment has been to convene a group of of subject matter experts to complete an AI impact assessment with them to establish controls, test them, and validate them in real use cases, and then to develop an audit guide. Given the pace at which kind of the technology policy response and different pieces of the ecosystem are moving, We're kind of rethinking this process with an idea to focus more on making sure that our core is is the core of the impact identification is always up to date, always mapped to all of the regulations and kind of best practices, but to create working groups that kinda do this work for specific verticals. So excuse me. So I'll just share two kind of lessons, and then I would love to open it up for conversation. First the first view is, like, my my lessons for me. So this is my takeaways having worked at the Responsible AI Institute for two and a half years. I think it's really important for certification or any kind of assessment to be intuitive and simple, easy to use. It's been really important to meet companies where they are if if we get too kind of far off in the weeds or if we kind of aren't aligned with how organizations are structured, then we tend to not not be able to get any traction in developing a responsible AI practice. It's really important to keep in mind the behavior the ability to drive behavioral change at the design phase. So it's not all about documentation. It's more about, like, are you making the right decisions to begin with? But at the same time, you also need to be responsive to regulatory and assurance regimes so you need to strike a bit of a balance and ideally serve both of those purposes. We need to align with how auditing firms work in their timelines, and so this has been difficult because the pace of change in best practices is quite fast. Auditing firms like to plan in kind of five year chunks. Right? So they wanna train auditors to be able to use certain standards or certain kind of set sets of products, like, standards or frameworks, and they want stability in kind of how planning and how it's delivered. So having something that's as dynamic as this is is a bit of a challenge. And then requirements for auditors, If if we really kind of make the best AI, you know, AI AI impact assessment and kind of document requirements, as rigorous as we can be, it's difficult to get a large number of auditors trained up in being able to audit and certify it. And so you need to kind of strike a balance between being rigorous but also not having an achievable bar in terms of the training required for an auditor. Process elements are easier to certify than application elements because you can just say, you know, this person looked at this thing on this date and signed off, whereas an application element is more kind of you know, the is with the dynamic nature of AI systems, it's more difficult to kind of pin pin down and document what that application element is. It's not impossible, but it's we we do find ourselves sometimes kind of relying more on process than than on application elements. And then finally, we've we found it helpful to focus on the core, and to and to kind of empower industry task forces to manage extensions. So I'll quickly move to another view which is our certification working groups view, this is a multi multinational interdisciplinary group of experts on AI certification, We co chair this group along with the World Economic Forum and the University of Toronto, and we have a white paper draft that we'd love to get your comments on. It's going to be published soon. So kind of verifiable claims that kind of can be verified across the chain. AI regulatory regimes particularly rely upon development of a AI certification ecosystem, but that ecosystem is not yet fully formed. And so the the group the gaps that the group has has kind of identified are clarity about what we're solving for. So, of course, there's a number of different, you know, sources of of true of guidance on this, and there's a number of different objectives, so how do we prioritize these objectives in different contexts and who gets to decide that? Another gap is the reference architecture, so that's going back to kind of the certification roles and responsibilities slide that I showed earlier. For AI, it's not exactly clear who's in which lane, and and so it it would be good to have some authoritative guidance on what the swim lanes are and what the requirements are to be in each swim lane. And the recommendations that the group kind of came up with is the government should lead the way in establishing fundamental objectives because going through the democratic process is the best way to kind of prioritize objectives in different sectors and different use cases, and the levers that they have at their disposal are policy and procurement. So we're already seeing lots of action on this. And then for all ecosystem participants, it's really important to, you know, get get the foundations in place, get the reference architecture in place, form the offline and online communities that we need to kind of have meaningful conversations in each of these areas and respond quickly to new developments and then move quickly to advance the state of the art from those foundations with kind of well a well funded and and directed research agenda, ideally government funded research agenda to advance AI verification and validation tools. So I'll stop there and kind of hand it back over to Manu for, you know, discussion questions as well as the rest of the event, but wanna thank everybody for for your time today.

Yeah. And we'd love to also hear, Amanda, your thoughts on this. So I think it really helps that we we're kind of a not independent nonprofit and and occupy a more technocratic space just given our kind of risk reduction within organizations and and helping like, giving them the tools they need to scale their programs. I think it's it's a fairly tame kind of area of of the responsible AI world. I would agree with you that there's, like, like, any other field, you know, there there's and especially now with all the attention that it's getting with all the kind of funding that's flowing, you know, you saw the $7,000,000,000,000 ask from Sam Altman for chips. There's definitely a lot of kind of grandstanding in the space, but I think that by being around, like, first is, I think, visibility and track record. So just the longer you spend in the space, the more you get to know people, the more you've worked with their organizations. You kind of have an idea for, like, who who, like, talks themselves up and, like, doesn't have a lot of kinda content behind that versus who's kind of very thoughtful but, like, not really a self promoter. And so we've we've got a decent idea of that in the space. The other thing is, like, I think in and Manu, I would also love your reviews on this point. There is kind of an element of, like, harnessing those forces. Right? So, like, you wanna create those those, like, roles and responsibilities, and then you want somebody to come in and say, yeah. I'm the I'm like, you know, this auditing company is, like, the first auditing company to, like, audit to this standard, and then you want somebody else's ego to, like, get hurt by that and come and say, no. Like, we're we we can do a better audit. And and, like so to some extent, like, we do wanna we do we don't wanna, like, just shirk away from that. We wanna be thoughtful about kind of aligning the incentives in a way that promotes our objective, which is to develop a culture within organizations of being thoughtful about responsible AI.

I think that,

really well, Var. Maybe the only thing yeah. It's just that's why we take the approach of trying to, we don't audit, but applying an approach of looking at evidence requirements, getting clear on the specific questions that we can ask, and looking for what is the proof of walking the walk, not just talking the talk. Like, if you say that you consulted stakeholders, that sounds great, but who, how many, how did you engage them, to what extent, was it a one off, you know, all of the ways that we can try to measure, the tough intangible things, that's the approach that we take, and we're always evolve evolving as the field is and looking at how you measure these sorts of things. But to the fullest extent possible, we try to be quantitative and understand the role that that ego and emotion plays in it and try to look for the the proof. Manu, anything you wanna add?

So I'll pause.

Yeah. So I'll just kind of on the on the question about and I kind of agree agree with all those perspectives. I think when a question we often get is, like, do what's the bar? Like, are we trying to make AI systems less harmful? Are we trying to do better than a human does? If it's slightly better than a person does, then that's arguably better than the present state. And at least from policymakers, the question is, like, what's different about AI? Why should we care more about this? And their usual kind of responses to this are it's it can't always explain itself well, and it can amplify bias based on data availability and and data quality and kind of other kind of things that historically skew one way or the other. But I think that one one kind of piece that scares me a little bit is, like, if you if you look at, like, automated employment systems as an example, yeah, we know that humans are biased. The, you know, auto automated employment systems, as long as they're slightly better you could make the case that as long as they're slightly better than all the biased humans, they're fine. But I think it's really the structure of the market and the ability of the systems to scale that gives me a little bit of pause about that being the standard because you could say, yes, this is better than a human. But if every company is then using that same system or, you know, there's, like, two or three major providers and every company is using them and it has some some flaw that you don't yet know about, that scalability is really something that, you see it'll, like, kind of manifest itself across society in a way that human bias won't because everybody is kind of slightly biased in a different way. Right? So that's I just wanted to contribute that thought that the standard should probably be way higher than just, like, is this better than a human given the ability of it to scale.

Okay. Thank you, everybody. It's like like Minoan said, it's always so amazing to watch everyone work in a shared document. I mean, it's a time tradition, like, established, like, collective practice, but it never ceases to be amazing to watch. So we'll we'll wrap with the the cowriting portion. And and our remaining time, I just wanna highlight the, what we'll be doing next week. So the idea of this seminar series was to, go through a different a couple of different modalities, exploring this topic of responsible AI and different methods of governing it. And we wanted to ensure that we, centered the voices of the people who have been coming to these meetings and participating. So next week, we have a we have a call for presenters. We'll be taking two presenters, to give, five minute lightning talks, and then, people on the call can break out into rooms and, go and speak with the presenters whose lightning talk they found the most interesting or resonant with the things that they're also exploring in their research. So if you are interested in presenting, please do reach out to me at Sent on the MediGov Slack. And as a as a speaking of compensation, as a form of compensation for your time, we're offering a $200 honorarium to, our presenters, which has been graciously funded by the MediGov membership program. If you are interested in supporting MediGov's community initiatives, there's also a link in the agenda where you can contribute to our open collective. And with that, we also have a tradition of thanking our guests with a round of applause. So if you would like to come off of mute and give our yes. Yes.

Yeah. Yeah. They're in the React. There you go. In the soundboards.

So, yeah, thank you so much for for coming. Thank you for Manan for organizing this guest presenter, And thank you everyone for the really, really interesting and stimulating discussion today. We'll be meeting again for the final session next week, and hope to see all of you there. Great. Great. And with that, we'll go ahead and close the room. Thank you all so much.