CDT EU and 2018, Natasha Councils NYC -- Talking Tech w/ Jens, Laura, Natasha Duarte
[NOTE: No, that's not a typo in the title – we're having funs with puns. NOTE NOTE: You make notice this EU segment is the same as one posted on it's own. It is.]<br><br>It will be a very busy year in tech and internet policy in Europe, with a number of major policies rolling out, including the General Data Protection Regulation and the ePrivacy regulation. In this episode, our Brussels-based EU policy team, Jens Henrik-Jeppesen and Laura Blanco preview the year ahead. <br><br>Then we talk about how New York City is leading the way in addressing issues of fairness, accountability, and transparency when it comes to using automated decision-making technologies. Yes, when a government deploys AI to make civic decisions, the public should be very informed and involved and CDT’s Natasha Duarte sheds light on how.<br><br>More on CDTEU: twitter.com/cdteu<br>More on Jens: twitter.com/JensJeppesenCDT<br>More on Laura: twitter.com/Blaunco<br>More on Natasha: https://twitter.com/Natasha_Duarte<br>More on our host, Brian: bit.ly/cdtbrian<br><br>Attribution: sounds used from Psykophobia, Taira Komori, BenKoning, Zabuhailo, bloomypetal, guitarguy1985, bmusic92, and offthesky of freesound.org.
Top Keywords
- systems 0.009
- decisions 0.007
- laura 0.007
- jens 0.006
- automated 0.006
- content 0.006
- automated decision 0.006
- make 0.006
- data 0.005
- companies 0.005
- task force 0.005
- making 0.005
Transcript
Welcome to Tech Talk. Bye.
CT. Tea.
Welcome to CDT's tech talk where we dish on tech and Internet policy while also explaining what these policies mean to our daily lives. I'm Brian Wojlowski, and it is time to talk tech. In this episode, we will welcome Laura and Jens from our CDT team in Brussels, and they're gonna preview the very busy year ahead in The EU. Then we'll be talking about how New York City is leading the way in addressing issues of fairness, accountability, and transparency when it comes to using automated decision making technologies. Yes. When governments deploy AI to make civic decisions, the public should be very informed and engaged. In our last episode, we previewed the year ahead in tech and Internet policy with a focus on The US. And in this episode, it's all about Europe. Many of the same issues are being debated on both sides of the Atlantic, but the policy approaches, courses of action, and public pressures are very different. CDT, of course, has a wonderful team in Brussels that is actively engaging on the full range of policy debates, including copyright privacy, free expression, and much more. Jens Henrik Yepsen and Laura Blanco from that team in Brussels are our guests this week. Welcome, Jens and Laura.
Thank you, Brian.
Hi, Brian. Hello. They're on the phone, so apologies to our listeners. They're gonna do the best to deal with with me on this one. So let me just kick it off in a broad way. You know, what are the biggest challenges, of 2018? What are gonna be the biggest debates happening, in Europe?
Yeah. Thanks, Brian. So we begin with data protection and privacy, which is of course one of CDT's core areas of focus. And in Europe, 2018 is going to be a busy year on this front. So first of all, in May, the new European Data Protection Law comes into force. It's called the General Data Protection Regulation or GDPR. It's going to replace existing rules dating from 1995. And so while it maintains and updates those rules, it also imposes new and stringent obligations on companies that collect and store and share personal data. And it gives new rights to people whose data is processed. And also it's important to note that the GDPR will apply to any processing of data of people residing in the EU also if that processing is done by companies that do not have a presence in the EU such as an app developer for example. Another important change is enforcement. The new regulation gives European regulators powers to fine companies at a scale that we have not seen so far. So today European data protection authorities can impose administrative fines in the range of a few €100,000. But with GDPR penalties for companies in breach can go as high as 4% of global turnover for the gravest type of infringements.
Wow. That's a big jump.
It is. It is. And so, right now, both companies and regulators are working overtime to get ready for the the new rules. It's very important for companies to be able to demonstrate and document that they are putting in place all of the safeguards and processes and structures to be able to cope with the new rules. And of course the regulators are creating a new European Data Protection Board which will be set up to ensure that the rules are applied in a consistent way across the EU. That should be of interest to companies who if the system works will be able to rely on a consistent line of application which has not necessarily been the case so far. Okay.
And what else is going on?
So further on privacy and data protection, there's a new law on its way in addition to the GDPR. So right now legislators in the EU are working on a piece of legislation which will complement and particularize the GDPR in the field of electronic communications. It's going to succeed existing e privacy rules from 2002. And the main intention of the new e privacy regulation is to apply the similar rules to all kinds of electronic communications, including what's known as OTT services over the top services, so Skype and WhatsApp and other Internet based messaging and communication services. The existing rules only cover traditional telecom services. And so the commission published a draft in last year around this time. European Parliament has amended it and finished its work in October 2017 and now member states are working on a common position. So this work will take we think a good deal of the year. And CDT participates in these discussions. We would like the outcome to be as effective as possible in terms of protecting communications confidentiality. We're pushing for rules, affirmative rules that allow recognize the right of providers and users to apply encryption to communications for example. We're also working on finding good rules on controlling the kinds of tracking practices in for example the ad tech industry to the extent that those practices are causing people to lose trust and confidence in using online services. So there's a these rules have potential to address that situation in conjunction with the GDPR.
Okay.
Yes. So I wanted to talk now about law enforcement and encryption. Oh. This won't be brief.
It's your show, Yance. You can be as as not brief as you want.
Right. Okay. So further further in the in the area of data protection, there is a lot of work going on in the law enforcement context. So in the EU, security and law enforcement are big political priorities. And EU leaders are taking many initiatives to beef up law enforcement cooperation among member states. And in the tech policy space, two initiatives are particularly important. One is access to electronic evidence by law enforcement authorities. The European Commission is going to publish draft legislation that will enable law enforcement agencies from one EU member state to request data from communications and cloud providers in other EU member states and even probably from providers that are not based in Europe at all, but whose services are used by Europeans. So the argument here is that traditional MLAD systems are too slow and inefficient for investigations to be effective.
I mean, MLADs being those treaties between countries that kind of govern the request, yes.
This is exactly right, Brian. And in fact, some of the issues in play here are the same as in the Microsoft Ireland case that is before the U. S. Supreme Court right now and where CDT filed an amicus brief recently. So we'll be pushing for a strong privacy and procedural safeguards and court oversight. And we want to make sure that there are no conflicts of law with either The U. S. Or other jurisdictions. And then I'll just finish with encryption. This is an ongoing debate. There is a lot of pressure from the law enforcement community and from some member state governments for legislation that would enable them to access encrypted communications and data. So we've pushed back on this together with other groups and with the tech companies. And so far the commission has concluded that it will not move towards backdoors or key escrow or general weakening of security.
That's a good development. That's very good.
It is indeed. It is indeed. And there is good recognition that a high level of security is essential for the digital economy and society and for commerce and communications and so on. But at the same time law enforcement are going to strengthen their practices.
Great. So now to Laura. Sorry, I was premature in that. There you go.
There you go.
Yes. So, another of the biggest tech policy debates in Europe is in the field of online content moderation. Politicians are particularly concerned about terrorist content, hate speech, copyright infringement, and most recently, of course, the whole issue of fake news. This is obviously not only a debate that is taking place in Europe, but it's barely the start of the year. And here, we are seeing new initiatives. For example, there are plans in France to legislate to combat fake news. And more generally, there is the ongoing pressure from European legislators for Internet platforms to be more proactive in monitoring, filtering and taking down undesirable, possibly illegal content at a much, much faster pace. Some legislators are even suggesting timeframes of 120. Of course, this is impossible to assess in this unrealistic timeframe what content is legal and what is illegal. But more importantly, this push for companies to act as de facto sensors of online expression are a cause of great concern for us due to the potential implications on free speech. Just an example, the commission's voluntary code of conduct on hate speech incentivizes companies to take down more and more content regardless of whether it is illegal. So, generally, we find this, very dangerous approach.
Yeah. No. It sounds dangerous. So let's let's go a little bit deeper on that too. I mean, you mentioned this a bit with, like, fake news. But, certainly, you know, here in The US and, obviously, in the EU as well, you know, tech companies, you know, kind of the the darlings of the past have been receiving a little bit of, or a lot more negative coverage. Do you think it's gonna continue in 2038? And do you think there's going to be more policies like some of the ones that you just mentioned coming out because of that?
Yes. I think it's safe to say this trend of putting online platforms in the spotlight will definitely continue this year. And, while there are continue this year. And while there are real concerns about content online and on the use of social media, it is harder to work out good solutions to these problems. And we agree that with scale comes responsibility towards users and society at large. But in Europe, the issues that some initiatives have unintended consequences for free expression and access to information. The most obvious example at the moment being the German social media law also known as the NetStitchi. This law entered into force in October and is being implemented by platforms as of January. Just for background information, the NetSTG applies to social media companies and other providers with over 2,000,000 registered users in Germany that host third party content. And it basically threatens them with fines of up to €50,000,000 if they fail to remove what is known as obviously illegal speech within twenty four hours of it being reported. Now this is clearly creating a lot of pressure on this platform to censor speech. And to be on the safe side, these companies are likely to take down content that is perhaps offensive, but not necessarily illegal. In fact, there have already been examples at the beginning of the year of a clear content being taken down from social media platforms. And then another example of this type of legislation is the draft directive on copyright in the digital single market, DSM. This piece of legislation, if it would enter into force, would oblige Internet intermediaries that host third party content to use filtering technologies to monitor and prevent the upload of content that may infringe copyright. So overall the commission really wants to target through their legislation the largest content sharing platform such as YouTube. But an unintended consequence would be, that it would affect just all about all types of platforms including SMEs.
Wow. So, I mean, you know, in your hat as an advocate, what what would you tell politicians they should actually be doing here, as opposed to these these kind of troubling approaches that we're seeing?
Well, for example, talking about filtering technology and the automated content analysis tools, there seems to be a lack of understanding at technical level on what this type of technology can or cannot do. Policymakers here seem to consider it as a silver bullet to remove undesirable content. But our as our colleagues concluded last year in the paper mixed messages, this type of technology has many limitations. So companies can be for one can be more transparent about how their platforms handle content challenges and respect, we in this respect, we are quite hopeful in the whole fake news debate here in Europe because the commission recently set up an expert group and is requesting input from stakeholders across the board in a public consultation before deciding on any potential policy action. So we definitely encourage this type of multi stakeholder discussion. And then maybe finally, a more comprehensive policy solution could be for the EU to produce guidance on notice and action under the existing e commerce directive. This would maintain the principle of limited liability for intermediaries in that directive while making clear what actions the intermediaries should take and the transparency and appeal processes they should have in place. Actually, leading members of the European Parliament have recently they suggested this way forward as an approach that would deal effectively with online content problems, but in a way that respects the fundamental right to free speech and the rule of law. So we are of course following the same line in So we are of course, following the same line in that sense.
And this would be some ideas from our side. Well, I hope they're listening to you then. Those are great ideas. And that paper that you just referenced, Mixed Messages, that looks into automated content moderation is available at cdt.0rg if people wanna check it out. How about I go back to Jens for this last question here? Jens, you know, in addition to all you and Laura have already talked about, what are some of the the emerging or new issues that we might expect to pop this year that might not be on anyone's radar just yet?
Well, the debate around artificial intelligence, machine learning and automation, robotics, etcetera, that debate is really picking up this year. So politicians and officials are asking questions about the impact of these technological developments on people and on society in different ways. So the commission is working There are many different policy questions to be looked at. There are many different policy questions to be looked at including how do you make sure that Europe keeps up with other regions on developing these technologies such as China and The U. S. Of course. How do you make sure that Europeans have the right skills to deploy the technologies and use them? And how do we manage the various changes in the labor market that can be expected? Further, the commission is looking into questions about liability for various types of autonomous systems. One example is self driving cars. Another is the liability and responsibilities associated with the Internet of Things, smart cities, etcetera. Who's responsible when a piece of software results in significant damage to people or property? These questions are not necessarily clearly thought out at the moment. So the commission is looking at the regulatory framework to see whether any sort of legislative change is necessary. This is actually something that our colleague, Benjamin Dean, has been looking at in a recent paper that or in a Forthcoming paper. Exactly.
Yeah. No. It'll be a great paper, and that might be out as early as next week. No. And these are tough, tough questions. Well, Jens and Laura, thank you so much for joining Tech Talk. Obviously, you have a very, very busy year ahead. Keep up with the work, and it's always great to to have such great colleagues on the other side of the Atlantic, and we'll see you. I'm gonna do a shameless plug here. We'll see you in person in March for our wonderful annual event, Tech Prom. So people interested in coming to CDTs grand fundraising dinner. Another thing to do, you can meet Jens and Laura in person there. If that's not a draw, I don't know what is. Thank you for joining Jens and Laura. Thank you. Looking forward. It's the concrete jungle where dreams are made of. And it's also a leader in tackling the complex issues around the ethical use of decision making algorithms by governments. Yes. When governments deploy automated decision making systems for public programs or policies, the stakes are heightened and the public should be engaged. To help with this, the New York City Council has taken the proactive steps of forming a task force that will explore fairness, accountability, and transparency in the varied automated decision making systems operated by the city. This includes systems that assign children to public schools, identify potential buildings for fire inspections, and rate the effectiveness of teachers. Pretty big deals. Our guest today is CDT's very own Natasha Duarte, who submitted written testimony to the New York City Council Committee on technology about just how the city could best govern its use of algorithms to make civic decisions. Welcome, Natasha. Thanks, Brian. So in your testimony, I thought it started off really strong. You stated that decisions made by algorithms are not neutral. What do you mean by that?
So the algorithm itself might be math, but automated decision making systems are designed by humans to solve human problems. And so that means they incorporate the values of their creators. So if I wanna create a hiring algorithm to screen employees, I'm gonna make decisions about what I'm looking for in an employee. And so those, those ideas are going to be incorporated into the system. And they also, learn about the world machine learning models models learn about the world from the data on which they're trained. So whatever values or we might say biases are in the data, underrepresentations, overrepresentations, you know, potential biases of the people who've collected the data or the environment it was collected in. You know, whatever values are represented in those datasets will also be incorporated into the model. And we see that with, for example, pretrial risk scoring algorithms that, try to give a risk score to people who are, up for for bail or parole. And, in those, we see that, because there is, racial bias in policing data, black people are are arrested, proportionately more often than white people. We see that racial bias also reflected in the risk scores where, black accused people are more likely to be mistakenly identified as high risk and white people more likely to be mistakenly identified as low risk.
Oh, interesting. So go a little bit deeper. I mean, you've already kind of touched on this a little bit because that's one example. But, you know, in your testimony, you're really focused on why when governments deploy these systems and certainly what you just talked about is a government use of it. Why is it more problematic than when, say, you know, an organization or, you know, just kind of any company uses it?
So the decisions that governments make are more likely to impact people's rights, liberties, access to vital services and benefits like Medicaid, food assistance, education, and, of course, access to justice. And so while we may still be grappling with, what companies' responsibilities are in the commercial space when they it's pretty clear that when governments make decisions using automation, just as when they make decisions that affect people's rights without using automation, that they really need to constrain those systems carefully so that they comply with the law and so that they carry out, good government policy, and that they aren't discriminatory or or aren't improperly taking away people's benefits. And they also need to be, transparent and open to the public for scrutiny and auditing to make sure that they are performing as intended and not causing disparate impacts. And they also need to include clear and effective ways for people to appeal those decisions. So when it comes to government decisions, we have, you know, due process rights and we, need to be able to hold the government accountable for those decisions. And that is, much more clear in the public sector,
context than in the private sector context. Okay. So you you've also touched on this a bit, but maybe go a little bit deeper on this. Like, why vulnerable and minority populations
automation has been, replacing human decision makers for several years is in the, the and health care assistance and food assistance, and child welfare systems and unemployment benefits. And also a big area where automation is growing is criminal justice. And so, the people who rely on access to those benefits and to the criminal justice system for their survival and their freedom, are in a position of being more vulnerable to the decisions those systems make. And another reason is, as I mentioned, the potential for disparate impacts. So, minority groups are more likely to be disproportionately harmed by, biased decision making or incorrect automated decision making because, they might be underrepresented in the data that the system is learning from or they might be overrepresented, as in the criminal justice system, in a way that that can, bias the decisions,
against them. That makes a lot of sense. So, obviously, governments are gonna wanna do this. New York City is a great example of that. They they already have a few automated systems. You kind of touched on this a bit earlier, but if you were to give the top, you know, top three things, if a government wants to start to use these systems, what should they do? What's the checklist they should go through before they deploy any AI or automated decision making tools? I actually have three things. Oh, yes. Look at that.
This isn't my first time on the podcast.
I haven't given you a trick question yet, though. So
so the first thing is on the policy level. And I should say, you know, a lot of the things that governments need to think about when doing this are not necessarily super technical things. There are so many, nontechnical just sort of policy making considerations that need to go into making sure these systems are fair. And so the first thing is to make policies that serve the public interest and then build or implement systems that actually work to carry out those policies. So for example, you know, it's one thing to say, we're going to, introduce automation so that we have, we can, you know, cut back on human labor, so we make decisions more efficiently. Or, maybe we're doing it because we want to, you know, detect fraud to, you know, try and see if we can get some people off of certain benefits. But what we really want is for, governments to be engaging in, thoughtful, proactive policies that are in the public interest where automation could actually, potentially make things better for everyone. So for example, you know, reducing racial disparities and hiring. A lot of, proponents of automated decision making in these contexts say, you know, we can actually use algorithms to be less biased than humans, and cut back on racial disparities. Now I am not, taking the position that that is, true or that is really promising or that we can actually do that. But if that is indeed the goal, then the systems that governments implement, need to be ones where, they actually accomplish those goals. And so they need to develop tests to test those systems to make sure not only that they're, you know, accurate making the right decisions about people based on the data, but that they actually measurably do, cut back on discrimination or improve conditions in the way that, that they should be. So so number one is, you know, make good policies. It's a big number one. Yeah. It is it is big. Number two is involve the public. Hold hearings, reach out to communities for input, and continually seek the input from communities about how these systems are working for people on the ground. Because it's important that, we you know, these systems are not designed, you know, by some, engineer at a vendor company that, doesn't really understand how they're going to work on the ground. You know, governing, at the local level is messy. There are a lot of factors that, that vendors may not not be considering. Every city is different. The needs of every city is different. And the needs of every community is different. And so, community members who are going to be impacted in these systems need to be involved every step of the way from from the beginning of, you know, developing a plan for procuring these technologies through continually checking in to make sure that they're working properly and, not harming communities. And the third thing is to, use open, scrutable systems. So systems that can be scrutinized by the public and by outside experts, and where people can understand why decisions are being made about them and then challenge those decisions, appeal them if they think that they're wrong. And this may require some changes in the ways that, government agencies procure technology or services. So often, with these algorithms, they, you know, agencies will, procure them from from a vendor, a company, and the company will, protect the actual, algorithm through trade secrets. And that is problematic because it may preclude, the public from having access to how the government is making these vital decisions about them and from specific people who may be harmed by the decision or may be subject to incorrect decisions because the system had the wrong information about them or or just got it wrong, might be limited in how well they can appeal the system. And in fact, the actual employees inside the government agency may be limited in how much they know about the system and how decisions were made. So so cities really need to look at how they're procuring these things and, thinking about what, additional burdens they can actually put on vendors to make sure that, the decision making process and reasoning is open,
and that the systems are tested properly and are actually working. That's good. I'm sure a lot of these systems are not inexpensive, so you might as well put a few more burdens on the vendors to, make sure that they, seal that deal. So you touched on this a little bit earlier too, the role of citizens. I mean, as just kind of a normal citizen, some of this can sound really intimidating, not accessible. Like, you may not even know that decisions are being made about you in an automated manner. How should just a normal person, you know, normal citizen get involved with this? What advice would you have to them?
So right now, my best advice is if you're concerned or want to learn more about how your, local or state government is using automated decision making systems, you know, make some noise and seek out answers in whatever way you can. So specifically in New York, now that the mayor's office has to put together this task force to look into these systems, now would be a great time to actually tweet at the mayor, and, call 311 and, you know, file a complaint if you have an issue or seek out more information about these systems. Of course, a major barrier to seeking out that information or to, you know, complaining about it is that, people may not know where automation is is being used. And so I am hopeful that this task force and, hopefully, in other places if if they adopt similar measures, will help at least break down some of that barrier where people don't even know these systems are being used. And so,
I think that alone is a a good enough reason to to do this type of thing. Cool. So, yeah, what's what's your greatest hope that I mean, obviously, citizen engagement. What what's the biggest hope you have for New York City's task force? And, are you gonna be a part of that?
I hope to be a part of it. So,
awesome. The honor You just want more trips to New York, don't you? I see how this works.
I love that Acela train. So so the author of the bill, council member James Vaca, has recommended CDT along with other organizations, be represented on the task force. It's ultimately up to the mayor's office to, create that task force. So we hope to be on it. But if we're not, we will be very involved in making sure, you know, that oversight of the task force is happening. We'll be showing up to those oversight hearings. And so one of my biggest hopes for this task force is that it does increase public engagement and public involvement in the decisions around, implementing these technologies and these analytics systems because, as I said, the decisions they make, acutely impact people's livelihoods, and their their access to the things they need. And every city is different and every community is different and has different needs. And, people should be involved in major government decision making. You know, when government when, laws and rules are written, there are typically hearings and people show up and voice their concerns
they get this right. Hopefully, they they get this right. Hopefully, they will, you know, listen to this podcast, hear your thoughtful and wise comments, and put you right on that committee or someone from CDT. Thanks so much for joining Tech Talk, Natasha. Thank you. That's it for this episode of Tech Talk. If you wanna follow all the great work that our EU team is doing, check out their Twitter account, CDTEU. Also, the EU team has an amazing monthly e newsletter. If you want to get on their email list and I'm sure you do send Laura an email at Laura dot CDT dot org. I'm Brian Waslowski. Thanks so much for listening.
