Talking Tech with Grace Gedye and Matt Scherer on AI Legislation
Recently, legislative bodies across the United States have exploded as lawmakers in over 10 states introduced several closely related bills to tackle AI- systems and their impacts on various facets of society. From hiring practices to education, insurance, housing, lending, government services, and even criminal sentencing, the scope of these bills is vast and far-reaching and will likely have a rippling effect across the country. <br><br> <br>Here to talk about these systems and the decisions they are responsible for making. Grace Gedye, Policy Analyst for Consumer Reports, and Matt Scherer, Senior Policy Counsel for CDT.
Top Keywords
- bill 0.020
- bills 0.018
- consumers workers 0.012
- colorado bill 0.010
- groups 0.010
- grace 0.009
- consumer 0.008
- consumers 0.008
- labor consumer 0.008
- consumer groups 0.008
- colorado 0.007
- workers 0.007
Transcript
Welcome to Tech Talk. Bye. CTT.
Welcome to CDT's Tech Talk, where we dish on tech and Internet policy while also explaining what these policies mean to our daily lives. I'm Jamal Magby, and it's time to talk tech. Recently, legislative bodies across The United States have exploded as lawmakers in over 10 states introduced a number of closely related bills aimed at tackling AI systems and its impacts on various facets of society. From hiring practices to education, insurance, housing, lending, government services, and even criminal sentencing. The scope of these bills is fast and far reaching and will likely have rippling effects across the country. Here to talk about these systems and the decisions they are responsible for making is Grace Geddy, policy analyst for Consumer Reports, and Matt Scherr, senior policy counsel for CDT. Matt and Grace, it's a pleasure to have you here today. Thank you so much for having me. Glad to be on, Jamal. To kick us off, and and, Grace, I'll point this to you first, what types of decisions are AI systems making, and what are the concerns you have about them?
Yeah. So AI tools and AI software is already being used to make all manner of decisions. I'm particularly concerned about kind of higher stakes decisions for consumers and workers. So, you know, we're seeing AI used in the hiring context, reviewing and filtering out people's resumes or maybe analyzing video footage of a virtual job interview and making recommendations to hiring managers. We're seeing it being used to prioritize who gets extra medical care. We're seeing it being used to screen potential tenants when they're applying for apartments or even, you know, it it more broadly than AI specifically, but algorithmic decision making software, setting car insurance premiums, setting your rent. And so these are all really high stakes scenarios where if the tool has issues, maybe it's imperfect and it has errors, maybe it's biased, or maybe it just doesn't quite you know, the recommendations it's making aren't fully backed by science. It's not living up to its marketing claims. Those are all kind of really concerning, especially in these higher stakes realms of housing, employment, insurance. And when we get into the policy conversation later, it's really these higher stakes scenarios that early bills are targeting.
And something that I'll just add to that is that what's really kind of makes this a pressing issue is that a lot of the time, consumers and workers have no idea that AI systems are being used in these decisions. And they certainly don't have any idea about how they are being assessed or evaluated by the AI systems. So there's really just a major problem, not only that these decisions are being made, but often workers and consumers don't even know that they're being made, much less why they might be rejected for a particular job or health care claim or apartment or whatever it might be.
Yeah. And to build on that, I mean, in the scenarios where we do have some transparency mechanisms, we know that there are problems happening. So there have been some lawsuits against companies that have tenant screening algorithms where folks have applied for an apartment. Their their profile their application was confused with someone else who had the same or a very similar name, and who had a criminal record. And then on that basis, they were denied the apartment. And that was, you know, the software's fault, the algorithmic software's fault. And that was only discovered because of existing consumer credit laws. And that is what gave people the opportunity to kind of catch that error and take recourse. But if those kinds of errors are happening in other parts of people's lives where, you know, existing consumer credit laws don't apply, right now, there's no way to discover those errors and get them remedied. So that to me is also a a high concern. One thing I wanna ask about is we touched on it earlier, but is the concept of, bias in AI. And I'd like to just hear a little bit more about how these biases are built in to these AI systems and how they,
impact results.
Yeah. So, really, when it comes to biases, they can crop up in any number of ways. It can be the result of having training data that is dominated by people who have, you know, who are from particular racial or ethnic or gender groups, or it can be from the fact that it is trained on particular past human decisions that are themselves biased. And kind of a connecting thread through all of it is that a lot of the times these tools are just looking for patterns in what past decisions have been based on. And there's very frequently not any real effort to kind of dig down and think about, okay, how is it that we measure what makes somebody, to use the example of employment, a good worker in a particular job? So it's you're very likely to end up kind of just repeating the same demographic patterns that have taken hold already in a particular job or industry. Or if you're talking about housing in particular housing communities and when you're talking about health care, you know, past biases, systemic biases in health care will tend to be repeated. And for a variety of reasons that I won't get into, it's very difficult to eliminate those biases. You know, a lot of the efforts to kind of make people selected at an equal rates can kind of backfire, and you end up just, having a tool that is either less accurate or less able to identify the best candidates from disadvantaged groups in the past. So it's a very difficult problem to overcome.
Wow. I want us to zoom out a bit, and I and I would like to hear from both of you a little bit about what's been going on in the policy world this year for AI driven decision systems? What what are we seeing in the space?
Yeah. So we've seen lots of action at the state level, specifically looking at how to regulate AI driven decision making systems with with an eye towards the issue of bias and transparency. You know, Matt and I have looked at at least 10 bills in different states. Some of them are pretty similar to each other, and a lot of them are looking across issue areas like employment and housing and insurance and government services. And I think one takeaway from how many bills we're seeing is there's this positive sense of urgency among state lawmakers. I feel like there's this narrative, and it's right that with a lot of other areas of tech policy, lawmakers, especially in congress, have been a bit slow to respond, and the harms have kind of accumulated. You know, the rest of us have taken the toll. And so I think there's a lot of leadership at the state level where, you know, state lawmakers don't wanna see that happen again and are ready to act while this technology is still relatively novel or at least is getting a lot of public attention, which I think is is good.
One thing that I do think is interesting, even though it's a good thing that state policymakers are tuned in right now, one unfortunate thing is that the bills that are currently progressing most quickly through legislatures are not those were that were written with input from public interest groups. And as a result, there's kind of, I think, an unduly tilted discussion towards proposals that are backed by tech companies and business groups. And, obviously, they are major stakeholders in this process, and they should have a seat at the table. But so far, there has not been, I think, adequate input from labor and consumer groups when these bills are first being drafted. And that kind of puts us on our back foot when we are having to respond to proposals that are primarily written
by industry groups rather than being part of the process from the beginning. I'd like to hear your take on the recent bill that's passed in Colorado.
How how are you feeling about it at this moment? I think that it's fair to say that consumer and labor groups have mixed feelings about that bill. The that was an example of a bill that, it's it's kind of part of a cluster of similar state AI bills that started to pop up last year and then really accelerated at the beginning of this year. And the Colorado bill was derived from one in Connecticut that the Connecticut bill was based on that was an example of one of those bills that had a lot of input from industry and not any really from consumer or labor groups. The Colorado bill improved a lot by the time that it passed because the sponsors did take feedback from public interest groups seriously, but there are still a lot of provisions that are ambiguous or that have loopholes in them that would undermine accountability, and the enforcement provisions are not great. And just in general, it doesn't kind of provide the strength of protection that I think, you know, Grace and myself and representatives from labor unions and other public interest groups would prefer. It's not a bad bill. I think that it's right on the border of, you know, being acceptable. And I've heard arguments on both sides from labor and consumer groups of which side of that line it falls on, whether it's
not quite good enough or just barely good enough. Yeah. And I'll just add to that. You know, for folks who aren't familiar with the bill, there are a couple of whole new rights for consumers and workers that I think are worth flagging just because I hope they'll be incorporated into any future legislation. And those are, you know, when AI is used or assists in making one of these high stakes decision about you, you know, whether or not you are accepted to rent a property, whether or not you qualify for a home loan, whether or not you have access to a specific insurance product or a slot in school. Under the Colorado bill, you if you were denied in kind of one of those decision scenarios, you would get a right to an explanation after the decision. You would get the right to correct any incorrect personal information that was used, in that decision. And hopefully, the explanation would kind of make clear, you know, give you enough information to tell if any incorrect information was relied on. And then you'd also get the right to appeal the decision with, you know there are some exemptions, and exceptions to the right to appeal that make it much weaker than I would like to see. But those are all, you know, what I put in the in the positive bucket about this bill. Those are kind of important steps forward because right now, as Matt said earlier, you know, consumers and workers have essentially zero insight into how these systems are influencing their lives. So, a bit of sunlight is is a really positive aspect of this bill. Always need a bit of sunlight.
Are are there any other bills you you two are keeping your eyes on this session? Yeah. So
there's another bill, and, you you know, I'd refer to it as a cousin of the Colorado bill that's pending in California. That's a b twenty nine thirty. And that one, you know, again, it's it's it's kind of part of this cluster of similar bills that have popped up over the last year. That one hasn't been, improved in a lot of the ways that the Colorado bill was before it got passed. So right now, even though it has provisions that, you know, if you read through the bill at first glance, it would provide some transparency, There are some really major loopholes that would make it very difficult to enforce the law. Like, companies would be able to get out of the disclosure requirements, I think, way too easily. And on the other hand, the California bill does have somewhat stronger enforcement provisions, which would be which is another kind of critical area of accountability. So that that that's one that's on my radar. There's another one that's pending in New York, that actually has the backing of a whole lot of labor in consumer groups. But, the New York legislative process is
very complicated compared to other states, so it's kind of hard to tell how far that bill will will go this session. Yeah. And one other bill that might be of interest to folks, Leslie, along it's kind of on a different subject, but I think it's gonna gonna be an interesting one to follow, is a bill in California that's actually a bit more focused on AI safety and competition. It's being carried by senator Scott Wiener. I believe the bill number is s b ten forty seven, And its focus is really more on, these these kind of societal threats, like threats to infrastructure, public safety, and then it also would kind of create funding for, public funding for public interest AI research. So that's, you know, a kind of a different subject, but, an interesting bill being carried by a a really influential California lawmaker and one that I'm watching. And I should mention that the bill numbers for the other ones that I mentioned, the California cousin to the Colorado bill is a b twenty nine thirty,
and the New York bill I mentioned is, s seven six two three. So those are bills that we're keeping an eye on, during the remainder of this year's session. And in addition to keeping eyes on particular bills,
what what have you learned from some of your work on state AI bills this year? Yeah. I mean, one of my takeaways
is that every state legislature is really is different both in terms of process, you know, how how just in the most literal sense, how the bill makes its way through the legislature when it gets hearings, when it could potentially die. Also in terms of political priorities and, how, you know, who which actors have the most political power and how they're weighing different priorities. My other takeaway is that especially when it comes to the state policy landscape, it's just so important to prioritize. There are too many states and way too many bills. I think there are something like over 400 bills on AI introduced in the states this year. And so I've been really trying to focus on bills that I think could, you know, be a first in the nation bill and really set precedent for other states or kind of have higher stakes for one reason or the other. Yeah. I I think that's right. I'd I'd also add on that
given the number of bills that are happening nationally, it's important and difficult for labor and consumer groups to keep up. I mentioned that a lot of the bills that are pending right now were kind of developed without really much input or any input really from labor and consumer groups. And, obviously, when you're in that position, it's very difficult to kind of play catch up after the bills have already been introduced and are potentially going through committees. And, you know, one of my worries is that during this session, there are bills that are just not even on Grace's or my radar that may slip through, without really significant scrutiny from public interest groups simply because the resources of our organizations are not strong enough to keep up with the deluge that has happened over the past few months. And, hopefully, in future years, we'll, we and other groups will kind of be better positioned to, have our ear to the ground on what's happening. But this year, it's kind of been drinking from a fire hose from,
from almost right after New Year's. What should people be watching out for in the future when it comes to state AI legislation? Is there anything that that the average consumer should be should be looking for, or is it all kind of a big open wide open for for consumption? I think that right now,
it's very difficult to kind of project out what the what the policy landscape is gonna look like in one year, much less five years. I think that we are going to keep seeing legislative proposals being made and legislation being introduced across the country. Part of the issue is because, you know, I I I hate to kind of keep banging this drum, but, you know, due to the lack of transparency and the lack of just disclosure regarding when companies are using AI tools to make decisions, it's difficult for consumers to know where there is a need for potential regulation and disclosure. So it's one of the things that I do hope, though, is that in the future, public interest groups, again, to go back to my last, last answer, I hope that we will be at the table, either by design or because we get wind of legislative proposals before they get introduced so that when legislation does make it to a committee, it's already had people take a look at it who are thinking from the perspective of how to protect consumers, workers, and, you know, society at large from the impacts of these decisions, which, as Grace mentioned, you know, can be both error prone and biased.
Yeah. I think that's spot on. You know, when we when we spot a bill and it's a couple weeks away from passing, there's only so much you can do to try and get some improvements in. And so if we're able to be helping to shape bills from the outset, I think the outcome will be better for consumers and workers, and the the process will be smoother. The legislative process will be smoother. So totally agree with everything Matt just said. So before we close, any final thoughts? To me, I think that
what we're kind of caught in a lot of ways, in a catch 22 when it comes to how to demonstrate to regulators, to policymakers, I should say, that there's a need for regulation in this space. One of the things that's kind of been interesting is that there's been a lot of industry pushback on the Colorado bill that just passed. Again, despite the fact that if you look over kind of the long arc of how that bill came into existence, industry had greater input at the front end than labor and consumer groups. And one of the arguments against it is that that I've heard from industry groups is that while the disclosure requirements are too much and that we need to wait to see how these tools are having an impact on consumers and workers before we jump in and try and start to regulate it. But that kind of leaves, again, consumers and workers in a catch 22. Without regulation, we are not going to get those insights into how companies are using these tools. And so to me, there's kind of like a threshold need. You can there you know, I think that there's an argument to be made. I don't I don't agree with it that the time is not ripe to establish outright bans on any particular technologies or even maybe to impose very strict requirements before companies can deploy them. But I do think that there is an urgent need for transparency and disclosure in this space. And that without having that, both consumers and workers are going to be subjected to decisions that alter the courses of their lives without any idea of what's happening and could potentially, be subjected to biased or erroneous decisions. And policymakers won't have the information about these tools that they need in order to develop good policies about things like impact assessments. And, what sorts of particular types of tools, like, should personality tests or tools that, you know, use facial recognition and facial analysis to decide whether or not somebody's good at the job, whether those tools have the scientific backing necessary, for them to continue to be used. I think before you, you know, in order to make well informed policy decisions about that stuff, at a bare minimum, you need transparency that is lacking right now.
Yeah. I totally agree. It it's, you know, it's it's so opaque right now that even researchers who wanna conduct research on how these products are working in society and in people's lives don't have access to the basic information. They would need to have some findings that, you know, might might really build the case against some of these claims that these these technologies don't need to be regulated at all. But as Matt said, it's kind of a a catch 22. And, you know, it it's just it's just very hard from the outside when when there's no information about when these tools are being used, what factors they're taking into account, who they're being applied to, to get a little insight into how they operate in people's lives. And,
yeah. So I I agree. And with that, I think we're gonna leave it there. Grace and Matt, thank you both so much for being here today. It's been a pleasure having you. Thank you so much, Jamal. This was fun. Thanks, Jamal. And, Grace, always a pleasure. And to all our listeners, to keep up with the work that CDT's policy teams are doing, please visit us at cdt.org and follow us on Mastodon, Facebook, LinkedIn, and the social media company formerly known as Twitter at Sendem Tech. I'm Jamal Magni. Thank you for talking tech.
