It's quite scary. Part of the reason that coding agents are so utile is that you can give them a command, like, hey. Look at all this stuff on my computer and, you know, figure out the best way to make this app work for me. And they will just read all your files. With my cooperative and, like, the things that we're building, I mean, I built two features over a weekend, and I don't know how to code. Large language models are kind of bizarrely, uniquely one of the most collective technologies that we've ever invented. To threaten those capability levels via changes to their data actually becomes more powerful. You know, you could imagine sort of this comic book villain, but in this case, perhaps it's the hero, revealing their plan and saying, actually, you know, we've been poisoning your data for the last two weeks. And you have the antidote, and you need to, you know, give into our bargaining demands. And and in that case, it really is possible that the AI field automates itself first.

Yeah. Absolutely. First of all, thanks so much for having me. I'm really excited about this conversation. So just, yeah, to introduce myself, I'm Nick Vincent. I'm, as you said, assistant professor in computing science, so primarily, an academic. I've spent a while now sort of doing research, very focused around this idea that there's a large class of technologies that have been all over the news recently, which are these data dependent technologies, oftentimes just called AI or generative AI. And these are tools that are very unique in in the way that they rely on data contributions from large swaths of the public. And this opens the possibility that these are tools that can be sort of governed and bargained over via data. And this is a really powerful lever for letting people broadly have a say in how these tools ultimately impact society. And so, of course, nowadays, I don't have to make a long pitch about it seems like AI will have a big impact on society. Maybe five years ago, you had to do a little bit more hemming and hawing about that. But so I've been sort of working on this topic for a long time, and a lot of my projects are around sort of grouped into trying to measure the value of data. So auditing AI technologies, auditing search engines, doing sort of experiments or simulations where you're saying, hey. How would this technology work if we didn't have some portion of the data? Would it be better? Would it be worse? Normally, it'd be worse. And then also thinking about how to communicate that to people. So, should we write more blog posts? Do we need to make videos, interactive web pages, write better papers, go on more podcasts? And just, like, how do we actually operationalize this empowering people by letting them know about the value of their data? So lots of projects. Really excited to get into all of them. But I guess oh, the first question. So, yeah, coding agents. How does this relate to the broader data leverage agenda? I have lots of thoughts on this on on this that subsect that you mentioned. And so to give the short version of it, I think that first of all, I do think, like, the hype is real. Coding agents do legitimately provide a ton of utility to people. It's, like, a real step function jump in the utility from AI overall. And I think it marks like, you can definitely just sort of see this in consumer behavior that there's people who are previously kind of skeptical of the value of a lot of these AI tools and are now suddenly signing up for the 200 US dollar per month subscription. Right? So there's, like, quite a big jump in consumer behavior, I would say. In terms of data leverage, I think that it's actually I'm really excited about the move towards the coding agent paradigm versus these online chat bots where you just are having conversations with a server operated by the private lab. Because right now, for now, the coding agents by default keep a lot of data on your machine. And so you have this very rich transcript of all the stuff that they did. And this really opens the door to potentially exciting opportunities around pooling coding agent transcript data. So I'm thinking about that a lot. I'm also thinking about it's quite scary. Part of the reason that coding agents are so utile is that you can give them a command like, hey, look at all this stuff on my computer and figure out the best way to make this app work for me. And they will just read all your files. And I think that's not maybe super obvious in a lot of the if you just watched a YouTube video about Cloud Code and you installed it and you kind of clicked yes on all the permissions and, you know, you have your taxes that you're preparing right now on your desktop and you have your health records and you say, read all my and, like, sometimes it will legitimately that will make it useful. So happy to go in lots of different directions. What's the most interesting direction? Or your private keys.

Yes. This is a huge issue. And so I have a couple students I'm working with who are really interested in this as well and have a bunch of thoughts. And, like, ultimately, we are kinda coming around to the conclusion that you're gonna have to be running the average user experience if you are running agents all the time is to, like, have VMs or containers or have multiple pieces of hardware that you're, like, really air gapping and sandboxing in a serious way. But we can get into that later. Yeah. Yeah. No. I'm starting with this in part because, like, you know,

Yeah. This is they try to have the default settings will try to do this, and so there's sort of some operating system specific sandboxing settings that that you can use. I I do hope in the this is an area where I think that there's, like, a lot of sort of public teaching. There's a lot of low hanging fruit for creating tutorials and improving these tools. And I do expect I think that, like, at the end of the day, I think all the sort of organizations that are building these tools have incentives to do this. Right? They don't wanna be involved in these massive breaches. And so I think that there will be better there will be better default settings. There'll be better toolings. It's early days right now. I mean, I think it was quite notable. I mentioned this in a blog post as well that for the first many months of these coding specific LLM tools being available, it was the case that you couldn't really trigger deletion of your chats, like, at all. There was only an archive feature and there was no delete for a while, which was just quite striking, I think. And I believe there is deletion now if you're doing coding agent interactions via the web app. But it's actually still quite hard to trigger server side deletion of a coding agent transcript that you created via a CLI. So, like, again, this is something I expect to be worked out in the long term, but just, like, there's a lot of these this is a space that's moving so fast. It is there are potentially large stakes around accidentally leaking data, and this is something I basically I do think that it's worth my general recommendation is that for people who are sort of curious is that you should try these things out so that you get some of the, you know, visceral hands on feel. But definitely be really careful, and definitely we're not at the stage yet where you want to sort of turn everything over yet by any means. Right. Right. Right. My next question is also then, like,

Yeah. I mean, so let me say how I'm thinking about this right now. I'll also just say, like, this is definitely one of the most recurring conversations I think with, like, obviously, for computer science professors, this itself is a existential concern because I, you know, I sort of have, like there's a couple direct incentive streams that I have. One is to produce research, but one is, of course, to have students who sign up for your major and sign up for your courses. And so this is, like, actually quite scary from that side. And I guess I would say this. So there's maybe three factors that I think we have to try to balance here. So one, there's the question or sorry, three, like, distinct sub questions within that broader question. One is the question of the actual capabilities. Right? It's the question of, can you legitimately get the full sort of, like, junior engineer capabilities? Do you really need to not hire somebody? And in some cases, I I'll also note that, like, we're in the early days of AI evaluation more generally. Right? There's a lot of these works coming out that are sort of highlighting meta issues with benchmarking and evaluation itself. There's a big grand meta issue with the whole idea of, like, Internet scale AI models that use massive pre training datasets, which they sort of violate the original one of the most foundational concepts in all of machine learning is the notion of a clean test train split, which says that if you have a bunch of training data that you're gonna teach your model stuff, you need to have a separate test set that there's no overlap between those things. And if any of your training set gets into your test set, that contaminates your entire experiment. Right? And this is like in your ML 101 class, everyone has this moment where they train a model with 99.9% accuracy, and they think they've sort of, like, discovered a new foundational breakthrough, but then they realized they they just trained on their test set. And so in some sense, like, LLMs do this at a very grand scale. Is this because, like, you don't you want to evaluate that the model was able to figure out the problem without knowing the answer? Exactly. Yes. So it's really easy to write a computer program that memorizes a test bank of questions and then can answer them all with a 100% accuracy. Right? That's actually sort of just, like, trivially doable with the database and no machine learning at all, just via retrieval. So in machine learning, that's really important. And so when your approach to building a model is like, I'm going to sort of just get all the information I possibly can to train on it. And then you didn't necessarily document all that information. And now you're going to go test it. Right? Like let's say you're a company. You want to test, you know, can this new AI model actually replace my junior employees? It might be the case that the answer was on a forum post or a Stack Overflow post or in your internal documentation that you trained on. So all that is to say is, right, it could be the case that the model really can, really does have sort of the capability level to do a certain task, therefore replacing a job category. But you might not, like a company or an institution, might be unable to know that with certainty until they do all this laborious evaluation labor. So just sorry. Going back, making sure I don't lose my train of thoughts, I had three sort of sub questions. Number one is this capabilities question, which means that it's really hard for us to say, even if the capabilities are there, we can't necessarily say with certainty yet that they're there. So there's this big just issue of what are the capabilities? Can we measure them? There's a separate sub question about just like the social contract of education and junior employees. Right? So there's a world in which AI totally can do all of the basic coding that you might do in your first year as a software engineer, but there's a collective action problem. And some companies will want to try to kind of defect and not hire any junior employees. But if no one does it, then the whole industry collapses because there's literally no more ingestion. Right? And so what would happen is that you'd probably see eventually people will some organizations will recreate some kind of apprenticeship, mentoring, training style program. And I think there's lots of parallels, with this happening in other forms of automation and and, like, labor saving technology. And then I guess there's, like, a the third dimension is this is just like the grand question of do you gain some outside of the specific capabilities of the job, right? There's an argument that you should do a computer science degree or you should do a philosophy degree because you're learning logic and critical thinking and reasoning as a question in its own right. And so this the third question here is, will the models sort of do all of that eventually as well? Right? And so this is kind of a there's a big empirical debate just about the quality of reasoning, the extent to which reasoning is kind of is working because of memorization or because of data dependence versus there's like a separate reasoning capability that's being created. And like to some extent, maybe that doesn't matter. Right? But I think it's basically so just recapping three distinct questions are, can the AI actually do a specific job task? Questions are, can the AI actually do a specific job task? And then, of course, the economists would jump in here and say, well, actually, we also need to think about the distinction between tasks and jobs and how tasks and jobs relate. And I think that's, like, real. So can the AI actually do the capability? Plus, can we measure it? Secondly, like, what are just sort of the social contract concerns here? Like, separate from the capability, should we be making sure to maintain a training pipeline? And then third, is a CS degree, is education, is philosophy, is any other degree you know, teaching us some amount of reasoning or critical thinking that a model can't eventually get. And I think that is an open question just with scaling models. Maybe models will reason better when we do something that's not LLMs. Maybe we can just continue to scale the current pre training, post training practices and models will continue to get better at reasoning.

rant. Yeah. Well, two thoughts that I wanna throw in there. Also just, like, things I wanted to make sure to say during our conversation because I imagine they'd be interesting to listeners. Number one is that there's a lot of interesting information is kind of for a number of weird reasons. There is a lot of info that we might have hoped is in papers or in shared public information, but it's instead coming out via unsealing of documents in copyright lawsuits. So one such thing that I think came out about last week is that there's an essay from within Anthropic that was unsealed in one of the it was authors suing Anthropic about the use of pirated books, basically. And it's an essay about this is from 2021 and it's about what should we do, like how do we sort of achieve this new economic vision in a post AI world? And it's basically talking about a lot of these things, right? Oh, maybe we can figure out a way to you know, reduce labor and to distribute money via some kind of fund, and we'll sort of empower people while simultaneously doing wider distribution. And of course, I think just because of some of the competitive dynamics and the just a number of other factors driving the industry right now, like, no one in the labs has been given a lot of latitude to, like, execute on this vision. But, like, this is a, like, this is a vision that folks in the industry did have. And I think so that's where, actually, like, public pressure via some kind of leverage to kind of return back to that, I think, can be quite palpable. And I think the appetite is there. And I think that there are lots of people in the industry who do want to do this. There's also people who probably don't. You know, I don't know. So that's one thing I wanted to flag. Another thing that I wanted to flag that is really interesting to me or, like, a reason why I'm I remain optimistic about LLMs specifically being the kind of paradigm that is driving a lot of the hype and investment and activity in in AI right now is that large language models are kind of bizarrely, uniquely one of the most collective technologies that we've ever invented.

Yeah. So the pitch there is that when you use the Internet, when you do your job, when you create documents, basically, anytime any anyone is doing something that creates a digital record, which is a lot of for better or for worse, it's a lot of human activity these days. That creates a data record that potentially is improving a downstream data dependent technology such as AI. And so we can imagine, like, a number of counterfactual scenarios in which we group together, people band together, and they stop using a certain technology or they change their behavior or they retract or withdraw or modify their data in a way that, like, really seriously threatens a certain AI capability. And so this enables, like, all sorts of new potential for bargaining. And that bargaining could be used for this things that bargaining has always been used for. Right? You could imagine just sort of workers who do some sort of digital task and they are trying to bargain with their employer over wages and they use data specifically as one of the other just, you know, as another sort of item alongside other tactics they might use, like, like striking, like public relations, like just going to the bargaining table. So that's like one aspect of it. And but there's all sorts of other types of bargaining that we could do as well. You could imagine sort of like in the really extreme, you know, academic waving a magic wand version is that everyone on Earth comes together and says, we're gonna bargain. We're all gonna do this kind of global data strike until AI companies put a clause into their, you know, organizational constitution that says that they're going to distribute X profits via this financial instrument. Right? That's like another or, you know, create some sort of I think there is a group of folks called who are working on the windfall trust right now is, like, one specific proposal for doing this, but there's been a number of variations of, like, AI dividend, data dividend, etcetera. And so sorry. Just going back. That is data leverage, just to state it very simply, is the notion that we have access to a new bargain lever beyond just traditional labor leverage, traditional consumer leverage, traditional go and vote in your municipal election. Right? These are all sort of, like, actions that we have available to us that will try to give groups of people bargaining power. And the more powerful that AI becomes, the more that various organizations rely on certain AI capability levels to do what they're trying to do, our ability the ability of, like, the people broadly to threaten those capability levels via changes to their data actually becomes more powerful. So this is, like, a real source of hope for me is that as AI becomes more and more pivotal to more things, data leverage actually becomes more important. The paradox to this are like, why hasn't this happened already? Do I just sound like a kind of wacky academic spouting an idea? Is that the bigger a system becomes or the more people who are contributing data to it, generally, you probably do need a larger group of people to join your collective action to actually be impactful. Right? So if there's a data set with only 1,000 observations in it, 1,000 units of data, and you got 200 people, 20% of them, to all do what we might call a data strike or a data poisoning attack or do data contribution to a competitor. These are all various ways that you can enact data leverage. That would be really effective. Right? 20%, we think, is actually 20% is a lot. The problem is that, of course, 20% of 8,000,000,000 or a 100,000,000 is a lot more people than 20% of a thousand. With all that in mind, though, like, I guess it's just worth noting that the fact that LLMs have this massive collective dependence is itself like, I think for what on one hand, it's a reason why they're so good because, legitimately, there's just so much intellectual labor and knowledge contributions from so many people. And that's, like, what makes the tools utile ultimately. You can imagine a counterfactual world in which there's another version of AI, which actually, you know, was totally just grown in a clean room lab and molded to one individual's desires. Right? So, like, some CEO writes down this kind of, like, constitution style document that says, like, you're my personal AI, and you should help me acquire as much power as possible and do everything. And then that AI was unleashed on and let's, you know, assume we've solved all the neuroscience and neurosymbolic and all these sorts of things, which I think are exciting research problems, and I hope we do continue to progress on those things. But that tool would be hyper aligned to one individual in a way that the current models are not, and there would actually be no source of data leverage over that imaginary counterfactual tool, right, that was grown in the clean room. So the fact that LLMs are this just wildly extremely collective in a measurable sense. Right? In the sense that we can, like, write down the total number of people that contributed to this technology is just so much greater than any other technology that humans have ever created. That's exciting. Sorry. That was a not as concise as I hope, but I I just I wanted to make that point that this is this makes the technology unique, and that's actually that's really important and is hope makes me hopeful.

Yeah. I mean, I specifically agree with, like, this second the second clause about that there is, like, an intrinsic collectivist nature to the entire endeavor of machine learning that I think oftentimes as a field, you know, we're a little bit we maybe don't acknowledge enough. I mean, it's also there there's a number of sort of critical scholars who have noted just, like, the inherently managerial nature of ML and especially, like, early ML when a lot of the innovations were sort of gained by using, like, platforms like Mechanical Turk, right, to do to basically get a large number of gig workers to do your labeling for your data, and then you're able to build a new model. And there's sort of this impersonal way in which you might not have thought of yourself as, like, training for a managerial job. Right? You thought you were training for, like, to be a mathematician or be an algorithm designer, but then ultimately a key, a huge part of the value comes from just like delegating tasks to a, to a large number of people. So it's just like very managerial, but it's very collectivist. And that that is just like when you do an inference on a model that came from a lot of people, like you are inherently engaging in this sort of aggregation across massive preferences. And I mean, that being said, right, again, I think it's all I'll just, you know, mention that I when I pitch a lot of the collective bargaining for information stuff, I also do I do wanna bear in mind just like there's the value of, like, markets as a social technology. And so I think these things can simultaneously coexist that, like, AI as it exists right now, specifically massively data dependent large language models, has an extremely collectivist bent to it, but can benefit from, like, the deployment of markets and prices and these kind of social technologies that are, like, you know, of course, I'm sure in the context of these conversations, right, like, there's a much where we can go beyond, like, the two d sort of political axis of, like, markets versus non markets and so on. But that yeah. I think that's just like there's a really deep truth to that. It's really under acknowledged. And I would love to see a lot more discussion and a lot more exploration of, like, all the possibilities that that entails, such as this kind of large scale collective bargaining, other forms of, like, co ownership, other forms of kind of radical, wacky ways of managing, governing,

Yes. Okay. So I think that I would say there's kind of two two I'll I'll do two sub questions. So the first one is just, like, data as a lever specifically for governing AI technologies. And, like, if it is the case that in fact AI is kind of the upstream cause of a lot of these problems, which I think this is up for debate as well right now. Right? There's a lot of there's a lot of debate right now as to our is sorry. Our slush is Our leadership agents are the people who are leaders at companies just, like, using AI as a smokescreen right now to do stuff that they wanted to do anyways, such as layoffs or such as reorganization, reprioritization in their organization? That's a question that's, like, sort of, you know, not yet totally answered. And I my guess is that it's a mix of the two, that sometimes AI really is driving things and sometimes people are sort of getting out of their skis and, you know, laying off a bunch of their employees before before they ought to have, and they're gonna kinda pay the price for that. And then there's a separate question of, like, can AI itself, now that it exists, be used to empower people for other types of leverage gaining and other types of bargaining? And I like I guess, like, the sort of prototypical example there, I'm thinking off the this is off the top now, so it'd be a little less. Yeah. I'll be thinking a lot. So just caveat. I might contradict myself at the end of this long sentence. You could imagine sort of like a hackathon of let's try to use our new powers, our new superpowers from Claude Code to, like, create the best labor organizing web app that could possibly exist, right, that's never existed before. So a big thing, actually, going back to our first question, this hyper on Cloud Code, it really is the case, I think, that's changed in the last three months and maybe in the last six months that you can start to bring into existence small technologies that you might not have basically, it used to be the case that if you wanted to, like, make your own custom to do app or your note taking app or your database or whatever, that it would be a pretty extensive weekend project. You might have to go and do a c CS degree or just, like, kinda train yourself on the weekend, which is totally doable. I think that CS has always or computing and software has been unique in the sense that there are a lot of people who create tutorials and open source libraries, and there's, like, a big deep embedded culture of openness. But now it's faster than ever. Right? So, like, if we really wanted to just, like, hack over the next twenty four hours on building some kind of app to help organize people to sign cards or to, you know, vote on something or to write new petitions. It's easier than it's ever been in human history to do that. Yeah. So I think that's really exciting. The question there, though, is, of course, you know, what if there's another hackathon going across that's happening on the other side of town, which is the, like, let's use cloud code for twenty four hours to hack on this sort of like organization busting tactics. And will people be doing that? Yes, of course they will. So I think like at the end of the day, you do have this competitive dynamic where everyone can benefit from this the new ability to just, like, call into being, to invoke, you know, this ideal technology that you wanted to exist. So, like, I guess with that in mind, that's why I haven't I personally haven't, like, thought a lot about that second category. Like, I haven't personally done a forty eight hour intense vibe coding hackathon Yeah. On a, like, collective action solving technology. That's something that I'd like to do still, but I've been thinking more about the upstream issue, which is specifically doing collective action with the data itself. Like, doing specifically a data strike, doing specifically a cross industry, let's all try to bargain or license or otherwise, you know, get some sort of formal agreement so that we move beyond this rule lawless data free for all that is the situation right now.

I was able to. Yes. And so this is really exciting to me too, and this is, I guess, I'm trying to be cautious about I I think I have, like, a pretty large risk. I or I felt in the last couple months, I have a big risk of, like, just having a pretty major bias in my perspective on these things, right? Because this is so exciting. Because, you know, when I had the same experience that I think a lot of people in my circles had, which is that over December, you know, there was a people have time off from work and that coincided with some releases with, like, the Claude Code terminal app and they were giving out free credits and stuff. Right? And so everyone had this experience of, like, post holiday having a weekend to just, like, Claude code for ten hours straight and it being sort of like this childhood dream. And so I had that experience too, right? And I had this just like total crazy personal hackathon, mind blowing, wow. And of course, did I do sort of like the morning after cold, harsh light of daylight cleanup on on that? Not yet, I would say. Right? I think that there is, like, it takes even if you have your ten hours of manic coating, and then it's gonna take many months afterwards to surface the full set of, like, issues or bugs or user experiences. Right? And this is just, like, why creating real software for a wide diverse population of users is, like, very hard and requires a large organization to do typically. But anyway, yes, I think this is real. I think the ability to, like, create bespoke software, to call it up as you need it, to edit it for given contexts, that is that's really exciting and potentially can be very empowering and actually kind of promote bottoms up power versus, like, top down centralization.

I mean, I guess, I I think the viability of this actually it goes back to this question earlier of will how fast will progress be on specifically sort of, like, reasoning and critical thinking and managerial tasks? There is this, like, a big looming kind of threat over all of the AI hype is that we're already seeing that software engineer or, like, kind of the tech industry built this. And it seems quite likely that actually a lot of subsectors of tech are going to be impacted negatively first. Right? There's also this meta question of, well, what if it remains the case? For a long time, you had AI researchers who are sort of acting in this managerial role. They're delegating tasks to people via Mechanical Turk or similar platforms. Right? And then the people in Mechanical Turk are typically acting as the touch points with the real world. Right? Like, they're supposed to like, let's just take an example. In computer vision, I want to create a brand new dataset. And so what I need to do is I need to hire a bunch of people to go out in the world and take pictures with their actual cameras of stuff in the physical world, tell me where it is, you know, label it in some sense, maybe with sort of like geospatial data or their own kind of opinions about it. I'm sitting in my office and giving out these tasks. Right? And they're touching the world. And so there's this a a couple we can imagine sort of like a couple different progress bars of different parts of AI. Right? There's the progress on like reasoning itself and managing and delegation. There's the progress on like the robotic the core robotics needed to go and take photos and operate cameras and like kind of, again, touch, touch the real world, you know, touch grass, so to speak. And it's possible that you actually see progress in the managerial thing, that managerial little imaginary bar chart that I'm waving my hands for, That goes faster. Right? And and in that case, it really is possible that the AI field automates itself first. And for, again, for a lot of people who are not actually trying to they're not in the game for their own personal enrichment. Right? I really do believe in my heart there's this kind of there is a pretty core contingent of the essential folks in AI research who would sort of bounce around between academia and industry and so on and so forth. Right? And they have in their hearts, right, sort of this very sci fi, Star Trek y vision of promoting abundance. They think that's what they're working on. That's what they want to achieve. And I think they would be okay with that outcome. Right? So if they do build AI that basically automates away management consultancy and leadership and managerial stuff first, that's a thing that could legitimately happen. And I think there would be some subset of the industry that would be happy about it. Of course, there would be some subset that's not happy about it. Yeah. Yeah.

Exactly. Yes. So first, I would just say that in some cases, I think the early instances and arguably, like, there is some data leverage data collective bargaining that's already happened. And that is just, like, when you have labor disputes in knowledge heavy industries, like, almost certainly the content itself, the production of that content is a core is a core part of the bargaining. Right? So I think that, like, in the entertainment industry, we've seen several disputes recently around the use like, future plans for AI, kind of, like, contractual obligations to not use AI in certain contexts or to maintain people's likeness rights or other types of rights. So that's happening right now. Right? And I think that version actually looks it looks exactly like how bargaining used to happen five to ten years ago. It's just now the data component is like a more important is a more pivotal part of the overall negotiation. So I would guess like category one is you have labor disputes that were already happening. They continue to happen the same way. Data just becomes more important. Another sort of, like, exciting possibility is that data is uniquely is quite easy to move around and to change your practices or change your rights or flip your license just in the sense that there's no physical constraints. Right? Like if you are bargaining over shipping, actual boat full of physical goods, there's all this kind of, like, messiness of the real world. It's actually it's relatively you know, if you are going to change your license or if let's say just work through a little example here. Right? We have, like, a couple small organizations that all create content of some sort. You might imagine local news or a research cooperative or a writing group or a small software guild or something like that. Right? And right now, each of these organizations, you know, has their own sort of agreement with an AI company or a tech company for how they're gonna let their data be used. If they come together, right, they just have to get in a signal group chat or, you know, however they're going to coordinate, and they say, hey. What if we all sold our stuff as a big bundle? It seems like potentially, you know, we would have super linear gains or even we would have the sum of our parts will be worth even more. And they can just like, actually changing their licensing is flipping a switch. So the in some ways, the transaction costs or all of the messiness of the physical world is a lot lower in the data world. And so what that enables is you could sort of have more emergent coalitions and cooperatives that are not necessarily mapping to traditional geography or even traditional industry lines. And so what does that look like? That looks like organizations basically coming together relatively quickly and changing the bargaining landscape a lot. To your question about, like, what specifically, like, what do you need to do to, like, action to give some real edge to your data bargaining threats? In the kind of original data leverage work, we organize this, like, a taxonomy of three types of action. So action number one is data strikes, and a data strike is sort of exactly what you might imagine. Right? It's just when you withhold data. And you can sort of subdivide that as well into future looking data strikes. And that's the simplest data strike of all. Right? That we can sort of we can pick our least favorite social media app and agree to do a two person data strike right now, and that's where you just say, I'm not going to go to whatever social media platform I'm unhappy with starting tomorrow, and we're gonna commit to do it together for two weeks. That's a future looking data strike. In some contexts or jurisdictions, you could also do sort of retroactive data strikes where you use a deletion request or some sort of data, you know, editing tool. That's not always available. So that's option one. Option two is data poisoning. And that's where you say, well, I'm worried that just deleting or withholding my data won't give me enough leverage. So what I'm going to do is I'm going to get all the members of my bargaining unit and we're going to start spamming you with junk. We're going to do something more nefarious. We are all going to use an app to randomly spam you with junk once every three days. And so you're going to have a lot of this data that looks good, but we're going to be flipping it every so while. And perhaps we've actually hired a computer scientist to come and consult with us. And the we're not just gonna flip it to junk. We're gonna flip it to something that is meant to specifically, adversarially manipulate your AI. Right? And so this is there's a whole field of data poisoning and sort of, like, adversarial manipulation of AI systems. Traditionally, it was framed as, like, almost entirely a negative oftentimes around, like, geopolitical security concerns or con commercial manipulation concerns. Right? So anyone doing data poisoning is always a bad guy. But we're kind of starting to see that flip. And now you can imagine using that basically using these technical approaches to try to poison a model as a bargaining tool. Right? And you could even go so far as to say, like, you know, you could imagine sort of this comic book villain, but in this case, perhaps it's the hero, revealing their plan and saying, actually, you know, we've been poisoning your data for the last two weeks, and you have the antidote, and you need to, you know, give into our bargaining demands. So I I think, like, that's it's maybe sounds a little silly when I describe it like that, but that's a real legitimate possibility. And then the third option, the third sort of, like, lever in the data leverage canon is conscious data contribution or just threatening to give data to a competitor. And that's where you say that's where if you're worried that you can't actually delete, you can't withhold, and you can't manipulate to do effective poisoning, or maybe you're worried that you don't actually know what is the best way to poison this kind of new model, let's say, because no one's studied it. You just say, okay. Fine. What I'm gonna do is I'm gonna try to make it easier for your competitor to beat you. And that takes advantage of the fact that data the economists will say that data or information tends towards nonrivalry and nonexcludability. So it's actually like information wants to be free, you'll also hear people say. It's pretty easy. Even if you can't necessarily go in and edit your history, you can just download all of your articles or blog posts or code and give it to someone else. And it's, like, typically pretty almost zero cost to do that. So that's another form. So just to recap, three types of, like, specific actions that you can take strike, poison,

Idea. Idea. Legitimately, yes. That is very possible. I guess, just, like, two other just is it worthwhile is it useful if I go into some of the technical nuances here? Or is it Sure. Possibly too much? I mean, sure. I'll just go into it and then, you know, stop me. Yeah. Yeah. So in the data poisoning space, I guess we can actually oftentimes people would distinguish between things that are meant to poison training data versus things that are meant to manipulate a model via, like, runtime or inference time behavior. And so the training data manipulation is where I would, like, randomly flip a character every 10 words in my document that I expect to go into the pre training portion of the model, or could be the post training as well. And so, like, often AI people will distinguish this pre training realm. That's where you collect a bunch of data from the Internet, you filter it, you apply some quality heuristics to it, and then you get your quote, unquote base model weights. And then post training is there's a wide array of stuff that, like, you can sort of think of it as the the stuff that makes the model sound like an assistant. So this is where you have sort of, like, special curated data that's like, here is what a good assistant response sounds like, and you throw that into the post training stage so that your model is not literally just a autocomplete over the Internet. Mhmm. So you could try to poison the model for the pre training days phase. You could poison data for the post training phase, and that's where you could say, you know, you could sneak into that data set of here's what a good assistant sounds like and says a good assistant always says, you know, always reminds the user of their communal obligations to the broad, you know, human project or whatever. Right. And then you can do prompt injection, which is about it's not about trying to change the model's actual sort of like internals, that's the model weights or the parameters, but rather it's about sneaking in stuff so that the model itself isn't changed but acts differently at runtime. And so all these three things are viable and there's like a weird, weird overlap between these because actually you might not know as a user right now if the data if you're like putting content onto the Internet, is it gonna be used in pre training? Could it be even used in post training? So there's evidence that sometimes, you know, companies might want to scrape resources like research articles and then pay people to annotate them so that those that sort of, like, annotated research articles, this is what a good research article is, looks like, this is what a bad research article looks like. You could use that in certain parts of the post training process. Anyway, so all these three are options. They all overlap. And I actually do think that, like, it would probably it's probably not a bad idea if you are an organization right now trying to engage in bargaining to sort of take a bit more of a shotgun approach and consider all these things. Because you are playing this weird adversarial game where the actual model building process is a little bit of a black box. You don't know exactly what data sources companies might be might be drawing from. And so you do have to sort of hide your bets and try multiple tactics, I think. Right. Right. Right. Interesting.

Oh, yeah. Absolutely. So I would say yeah. The this like, kind of this is the first of all, like, just to say one really positive thing is that we myself with some other academics, we put together a workshop specifically on algorithmic collective action at the NeurIPS conference last December, which is, like, the big kind of flagship machine learning conference. And so it was really exciting for us to try to, one of my sort of just like longer term goals is to carve out more space in the computing research community for these ideas. Because again, I think that they're actually the number of folks who are like sympathetic to this stuff is like higher than you might believe if you only read sort of like the, you know, hype articles or the negative impact. So it it was really heartening to see that start to kind of emerge as like a real research space. Generally, yeah, there there's lots of overlap. The broader the growing, like, nascent literature on ACA, Algorithmic Collective Action in general, is kind of concerned with a lot of the the specific, like, technical details of what types of action are most effective under given algorithmic constraints. Right? So if we make some assumptions about a specific classifier or a recommender, how does that change, like, the exact number of people that we that we need to get to join our collective action? And then it also covers these folks who are maybe not doing the collective action because they're trying to bargain with someone ultimately, but are rather just trying to, like, directly achieve an outcome. So that's the classic example of, like, a type of ACA that hap that's happening right now, and it's been happening for quite a while since, you know, just humanity started to get a lot of our information via these algorithmically mediated, like, ranking feeds. It's like fandoms that are trying to promote the interests of the, you know, kind of, like, subject of their fandom. And so I think, actually, one one group that that there's lots of really interesting social dynamics here, actually, is the, like, online fandoms around k pop where people will sort of manipulate their viewing and listening behaviors or, like, intentionally listen to a song many times. And, like, this is the thing. This is not theoretical. This is not just, like, some Yeah. You know, sort of out there machine learning theory paper about what if people did this. Like, it's really happening right now. And it's, like, quite it really has a large effect on ultimately how these feeds deliver information to people. Mhmm. So anyway, all that is to say is that, like, I I guess to the question of, like, what's the difference here? These things are totally related. And I guess, like, in some sense, ACA is even more is a little bit broader. Right? And it's inclusive. Like, the Taylor Swift fans or the k pop fans aren't necessarily trying to bargain with anybody. They just want to make the model output the stuff that they want. Right? Which is, like, the thing that they think is good. But the thing is that they what they do have or what they're expressing

by direct Right. Right. They are using their they are using their important role as data creators to and basically, like, in some sense, you could view this another way, which is that when I if I'm kind of, like, running a tech company and I have a choice early on in my, like, sort of company's life cycle, which is, am I going to have some sort of curated feed? Am I gonna have some sort of chronological feed? Or am I gonna do one of these mixtures of collaborative filtering or other collaborative filtering, which is a type of recommender system or other approaches to recommender systems, or am I gonna build an LLMs? Right? And when I choose to do that, what I'm almost doing is I am acquiescing some power to my users. Right? If I make the technical decision to say, like, you know, okay. In week one, I curate all the content. I choose what shows up first on your For You page. Right? But we're scaling too much. We gotta do collaborative filtering now. Right? In doing so, I am acquiescing some of my power and I'm saying that I'm gonna, when I train my model on the user's preferences or what they watched or what they clicked or what they commented, I am saying in the future, if they change what they liked, commented, watched, or clicked, then my outputs are going to change. Right? So there's, like, this kind of very profound but implicit social contract going on where it's not just, like, it's not even the sense necessarily that when the ACA users did their collective action, that's what there's there's certainly, like, they're acting they're enacting their power then. But the power transfer actually happened when the company made the decision to use this type of technology, which has a social dependence. So I think that's I imagine that's, like, a concept that would be interesting to to to folks in this space who are listening to this. And I think it's actually it's really profound, and it's something that I hope that we these tech platforms start to reckon with a little bit more and maybe think about there there's again, this is repeating myself a little bit, but, like, there's a really strong case for some sort of collective ownership over these technologies. And you could say that these users are just, like, actually

Yeah. Absolutely. I mean, I think let's see. I am very hopeful about the potential for kind of these more out there, like, mechanism design flavored social technologies that are operationalized via Web three stuff. I don't have a good sense. I we were talking before. I was saying this is a space that I used to know more about, and I'm a little bit out of the game, so I don't know sort of the latest happenings. I also I kinda personally suspect that some governments or some tech companies are going to end up re engineering things that look like centralized versions of, like, Web three flavored coordination mechanisms. Right? So I think you could see companies start to do this. I guess, big open question for me as well is will they do this voluntarily or will they do this because they're forced to do it. Right? Like, one thing you could imagine is that there's a big maybe public outcry. I guess also something we didn't get to talk about a lot as well, which I think is really important to all of this is just like this polarization of the like, the anti the AI anti AI distinct polarization sort of dimension, which is, like, now becoming it's correlated with political with politics in certain ways, but also, like, is jumping beyond there there's some, like, weird groupings going on that's not 100% matching traditional political issue. It's sorry. Yes. It's a new issue. And so it has its own sort of dimensions. And I do think, like, that's going to affect the uptake of such technologies a lot. So, yeah, I think my thoughts on this, like, my contribution is that I am really I'm quite hopeful. I think that anything that lets people digitally action their leverage is gonna be really powerful in the short term. I think that there's a good chance that the thinking and a lot of the, like, early thought leadership from sort of cooperative funding and cooperation mechanisms and all sorts of kind of social experimentation that hasn't happened a lot outside of the Web3 space. I think that will either get sort of copied over and then ultimately maybe there's a jumping off point, right, where actually that the implementation of these mechanism style things in a Web2 context actually helps people onboard, mechanism style things in a Web 2.0 context actually helps people onboard to doing it, you know, using some sort of blockchain technology. Again, lots of caveats here, and I out of the game. Can't speak as confidently on this topic as I can about the data leverage and LLMs more generally.

Yes. I yeah. I agree with that. And I guess I'd I'd also add that I I think that in the short term, people one space that maybe will just again, you know, trying to appropriately not overstate my confidence on anything, but connect this in. I do think that the agents the use of sort of these increasingly autonomous agents is gonna make it pretty attractive to start doing cryptocurrency transactions for agents specifically. There's just, like, a number of reasons why this is quite convenient and you don't really wanna set up your, like, personal bank account or your personal Stripe account on your agent. Right? That's, like, quite a scary thing. And so there is, like, very real potential for in the next couple of months, like, a lot more overall human commerce that's being transacted via agents that are not using specific they're using, like, a wide range of different currencies mediated in in part by these kind of transactions. And so I think that's, like, on my list of something to look out for. And then also a great thing about, you know, a huge benefit of the generally, there's some level of transparency for a lot of these things, right, is that we can actually monitor that quite well. And and it kind of enables this new form of global scale economic monitoring that I think is, like, actually gonna be quite exciting to economists.

I mean, first of all, to just mention that all the kind of papers and research that I talked about here, right, almost always is a massive you know, getting that is a massive collaborative effort. So definitely, I mean, feel free to check out my website. You can read the paper, see all of my collaborators. Several of them, for instance, on this kind of simulation y measuring data work. Grad students who are, you know, near graduating, so definitely check out their work. You know, if someone happens to be listening to this and is hiring in this space, let me know. Happy to make some recommendations of great folks who are doing really exciting work. And more generally, I guess I I should also shout out there's a growing kind of public AI network, which is that's a place that I'm kind of digitally hanging out a lot, and that has become a, you know, a nice kind of network of friendly faces. And there's just I have benefited a lot from that group, so you can Google the Public AI Network. Check that out as well. But, yeah, I guess sorry. I didn't prepare a list, but generally speaking, whenever I do one of these things, I wanna make sure to just communicate just how immensely collaborative that this whole effort is and how these things are sort of, you know, big collective projects, of course. How could they not be? You know, shout out to all the people who contributed to the LLM so that I could, you know, use cloud code in codex. I make this out actually mean that quite seriously. I do think the labs should put, like, a thank you page on their website. I I I, like, legitimately think that there would be it would be meaningful if there was, like, a openai.com/thanks.html

Larry about Yes. No. I think we should do this. Okay. Yeah. So actually, just and really quickly, I'll just list off a couple other foods for thought pieces of food for thought slash maybe, yeah, we could carry on this conversation at another time. There is a growing technical field that's trying to do stuff like that. So specifically attribute, you know, people individuals or groups of people who specifically contributed to an AI model. That is a field that exists. It's really tough problem, but we're making progress there. And And then another thing that I didn't talk about much today is, I think I mentioned briefly, is that there is this whole emerging policy space around AI dividends, data dividends, windfall, things of that nature. And I think that is going to have some really deep interaction points with with a lot of the topics of your interest. So that's something I'm on the lookout for too. But I'll stop it there so we don't go too long. Obviously, so much to talk about here and really appreciate you providing the space and just chatting with me about this.