{ "metadata": { "transaction_key": null, "request_id": "metagov:metagov-seminar-coordinating-decentralized-research-with-lab-discourse-graphs-akamatsu", "sha256": null, "created": "2025-10-27T23:38:01.468137+00:00", "duration": null, "channels": 1, "models": [ "metagov-manual" ], "model_info": { "metagov-manual": { "name": "metagov-manual", "version": "2025-10-01", "arch": "manual" } }, "warnings": null, "summary_info": null }, "results": { "channels": [], "utterances": [ { "speaker": "Speaker 1", "start": 0.0, "end": 0.0, "transcript": "And hit the record button. Wonderful. Well, hello, and welcome, everyone. Today is Wednesday, August 21. And, without further ado, I'm just gonna pass it right off to, Matt Akomatsu who will be presenting today. So, Matt, please feel free to hop in and take over." }, { "speaker": "Speaker 2", "start": 15.0, "end": 15.0, "transcript": "Thank you, Eugene. I'm very excited to be presenting to all of you. I met Eugene at the DSi Denver conference in March. We had talked about discourse graphs as a way to synthesize knowledge, as pertaining to collective governance. And so I'll tell you today about how we use, this tool called discourse graphs to synthesize knowledge and push forward new research in my cell biology and biophysics lab. K. So let me share my screen, which you can see now. Okay. So in my day job, I'm an assistant professor of biology at the University of Washington, so I run a research lab. And now about half of my time and attention is is spent in applied metascience, developing software and approaches to to improve how we do distributed, collective, scientific research. And that takes the form of building, software, that helps us run our lab, as a collective system to do collective sense making and internal nano publishing. Okay. So the the main motivation for this project were that there are a number of systemic issues in science, that I encountered over the course of my of my training, that I thought we might be able to address with with, software that has a different, organizational approach. Okay. So the first barrier, that I, have encountered and noticed is that there's a high barrier to entry to scientific research. When you're entering a new field, when you're a trainee, when you're trying to make an original contribution to any research field, it takes a long time to acquire sufficient context and to maintain orientation to say, is the question that I'm asking one that's already been answered? Like, what is the state of knowledge about this question? You know, what studies have people done? What are the what what's the the hard evidence that they've collected that informs my question, am I at the edge of knowledge or not? This is also informed by some qualitative research suggesting that one of the factors that contributes to when a researcher starts to consider themselves a scientist is when they make an original research contribution that can be used by other members of the field. And traditionally, this happens when you publish a paper, but in our field, that takes five, six, seven years from when you start your your graduate training. And it creates this huge gap in when you are not sure what contributions you're making, and it's one of the factors that is sort of lowering the mental health of of research trainees and also contributing to an inequitable attrition from from the research ecosystem. Second, from an organizational perspective, I'm really interested in helping to develop a research environment that's fueled by intrinsic motivation. And I found that the hierarchical structure intrinsic to most academic labs is not conducive to that, which is part of the reason why I'm so interested in the decentralized science, movement. So some of the factors that, contribute to intrinsic motivation, based on other qualitative research are autonomy, sense of purpose, and expertise. And so to maintain the type of autonomy and and agency, it's and having all of the ideas and sort of action plan consolidated with the PI acting as a manager isn't really, conducive to, to the students or the trainees maintaining that type of, of agency. And so the alternative seems to be a a decentralized network of researchers, and they're excited about, DAOs and other ways to, organize research, in a in a decentralized manner. And to realize this this vision, it seems to me like we need a coordination layer, so that a given researcher sort of knows where they're at, how they can contribute, who to collaborate with, etcetera. Finally, the researcher roles tend to be uniform in academia. The the analogy that I like to make, which is initially created by Greg Nelson, who's a human computer interaction researcher at the University of Maine, is likening the current research ecosystem to unsustainable agriculture, where PIs sort of all have the same needed skill set, and it's basically the only way you can contribute to research after a certain point, which is really missing on a huge base of diverse intellectual contributions that we think from people who wanna be contributing to the research ecosystem. So our hypothesis is that we can address some of these systemic issues with new collaboration tools as a means to facilitate cultural and, organizational change. We do this by breaking research into modular parts that can be shared and cited, and those modular parts have a semantic element, which I'll describe in a second. Okay. So we aim to facilitate the following things, on onboarding into a new research group, bookmarking gaps in knowledge to motivate new experiments, the coordination and collective sense making between researchers interested in a shared question, and finally, the regular exchange of discretized results between researchers in a way that's much more flexible and reusable and sustainable than traditional research articles. Okay. So the the main approach we use is to decompose scientific arguments into questions, claims, and evidence. This comes, from a model of argumentation created by Stephen Toulmin, who's a science philosopher. And the main premise is that science, rather than being a series of monolithic truths, is instead a series of claims that are backed by more or less evidence. So the textbook has the best claims at the moment, but it evolves as more evidence arises. And this is part of a collaboration I've had with Joel Chan, a human computer interaction researcher. We started in in 2020 because we were interested in the same sort of distributed graph based note taking tools, and we finally met for the first time earlier this year at at DSIGHT Denver. Actually, the very first meeting was was through Eugene when we were when Joelle and and Eugene were creating this doing this discourse graph, workshop. Okay. And so the idea is to take the atomic units of a scientific argument or any research argument, questions, claims, and evidence, and connect them into a graph. So this is the schema for a discourse graph. The question is the motivation for your work. It's the thing that doesn't yet have an answer that you'd like to to answer. The claim is your current proposed answer to the question, and the evidence is a specific measurement or observation or outcome that supports or opposes the claim. And each piece of evidence is grounded in some source, in some study or some experiment, etcetera. And so every time that you make a claim, you support it with this discrete observation that has been made by most by a researcher of of some kind or another. And, of course, it's tied into a graph so that, the edges between these nodes can can be labeled as supports or opposes. So let me give a concrete example. Let's say your question was, what is the fastest land animal? And I claim cheetahs are the fastest land animal. And And the Internet and ChatGPT give me a bunch of a proper full evidence for informing that that question, so we need something more concrete to inform this this question. And luckily, a researcher made a rigorous measurement of the speed of a of a cheetah. This person published a paper where they tacked a piece of raw meat onto the back of a jeep, It had a stopwatch and a track of a defined length, and he measured the speed of this particular cheetah three times, to get the speed of 64 or so miles per hour. And the methods include information like, at the time, the researcher was an athletics coach and so was well trained in stopwatches. This is the stopwatch that I used. The wind was blah, blah, blah, blah. So it gives you all of the contextual information you need to evaluate for yourself the strength of this evidence towards this claim. And if you wanna make an alternate claim, this approach really encourages you to bring your own evidence and add it to the graph so that everybody can weigh the strength of one piece of evidence or another so that we can sort of increasingly arrive at a stronger and stronger claim. Let me show you an example in our lab. So let's say we wanna do we a journal club rather than about a paper, we do it about a question. We call that synthesis journal club. And for the sake of brevity, I will just show you what such a discourse graph looks like in a this sort of open canvas that we've created. So let's say you have this, research question about, like, how cells attach to their environment, which is something that we study. So you start with a bunch of sort of premise questions. We divide up the relevant papers between people in the lab that, might inform this question. Everybody pulls out a specific piece of evidence, which is, like, one observation from the paper, and you can see over here, like, the key information is, like, there's, like, some figure that has a a, like, single measurement that that gives you the outcome, and there's some contextual information about, like, you know, what were they measuring and what was it on so that this individual piece of information is sufficient to understand what the outcome was. And here, people in the journal club are sort of, like, arguing with each other about and bringing evidence for why they said that this protein interacts with this protein. And then you end up with some summary. So you're like, here's our diagram of this molecule interacting with this part of this cell, and we use it to create this claim, this claim, and this claim. And rather than just hoping that those are true, we have we have pieces of evidence that that support that claim that we can refer back to. So we consider this to be sort of getting to the edge of knowledge, the state of knowledge about this particular research question, and we can return to change the claims as we acquire new evidence. And I'll show you some more examples shortly to make that more concrete. So that approach has been great for getting up to speed on published literature and doing sort of a systematic review of the state of knowledge, but I realized that most of my time was actually spent trying to, be in the lab or run a simulation doing sort of original research, and I wondered how this schema could affect my ongoing research. And so the moment was when I realized that, like, what else am I doing in the lab except for trying to collect new pieces of evidence that are supporting a new claim, which I usually call a conclusion, and write that up in a paper. And I thought, well, if I organize our lab's research through the schema, this is gonna let me crowdsource a particular project where each person can be, testing a related how strongly a related claim is supported by running their own experiments to contribute a single observation or result that helps to support this claim. And when we think about decentralized science, I think that the schema could be a appropriate level of abstraction for people to make contributions. So everybody who's interested in the same research question could be make different claims based on the evidence or point us to what new experiments need to be done. So we think of each of these as the minimum shareable unit for collective sense making. How does this work in practice? So here's a plot of structure as a function of effort, where as you go up the the plot, there's increasing, structure that's afforded to your sense making. So in the normal course of our research, you're sort of going on and doing some sort of study or experiment. You get a new observation, which you say, okay. This is the, like, the take home, result from from my experiment. And now that informs my question. I can make a new claim. Oh, but I can't really make a stronger claim as I really wanted to, so I'm gonna call that a hypothesis because it's sort of an untested claim and go back, and I wanna make a new go carry out a new experiment as a result. And in practice, when you don't have time to run that experiment, you can save it for later, and you call it an issue. And you say, well, I or somebody else should really go do that experiment because it's gonna help inform our hypothesis. And so I'll show you some examples of how we how we do that. First, let's show you who's working on this. We have a great integrated team of developers who have been supported by Protocol Labs and Schmidt Futures and experiment.com who are building plugins currently in, Roam Research, this graph based note taking, software. And then the users are my cell biology and biophysics lab, and so we have a, let's say, weekly feedback loop where we're filtering for features that are specifically useful in this, research lab environment. Okay. So what does our lab graph look like, and what do we use it for? Okay. So let me just show a sort of inventory of the types of discourse nodes that people have created in the lab, recently. So some of these are results, new results from their experiments. Some of them is evidence from the literature. Some of them are new claims. Some of them are are questions. And the important thing is that each of these elements is tied to the source material. So in the case of our ongoing research, let's see what happened yesterday. So Valerie has a result that's tied to a ongoing experiment. This is a tissue culture experiment that lets you tie like, drill down into the nitty gritty details of on the lab notebook level, what exactly did they do? This is usually too much detail for somebody to make sense of, so you usually stay up at the level of this of this, result instead. But if you really want the the detail, it's accessible, in in the graph. And so this is the first 12 of, we of what are now 1,200 of these of these, discourse nodes that we've generated, as a research group over the course of the last, let's say, two years. So here's a graphical, this, sort of schema, showing, some of these discourse nodes where, like, each dot is one node that's like a question or a saved experiment for later, an issue, or a claim, or a piece of evidence, and gives you a sense of how interconnected this our research graph is. So it has on the scale of, let's say, 60 experiments, 80 hypotheses, 250 pieces of evidence from that are all tied to to source material. And so this is a wonderfully densely interconnected graph that essentially serves as the operating system for our for our lab's work. So how do you make sense of such a densely interconnected graph? There are a couple ways to do so. You can synthesize the your your research, on a canvas similar to the one that I showed previously. So here's another example. So that's this would have been created organically by by one such researcher. You would have the experiments, the results, the conclusions, and the follow-up questions. Current work now is this question, this experiment, and this result. And once again, the result has, like, some key, sort of plot summarizing what the outcome is, as well as the sort of the, like, lab notebook level, note taking that that researcher needs to do just for their own personal sense making. Let's see. Okay. And I'll show a couple more examples. One thing that I think would is we found to be especially useful for collective research is bookmarking and handing off ideas between researchers. And we do this with what we call an issue sport. So every time we're trying to make sense of our project and we say, well, there's one more analysis that needs to be done. If there were five of me, I would just do all of them. But but we've done the intellectual work. Let's just save this for later and and hope somebody else does that. So we collect all of those elements in this in this issues board, which you can filter by is it based on analysis? Is it based on the literature search? Is it based on the simulation? So that new researchers can come in and pick one of these and use it as their entry point into the lab. So we ran this experiment over the summer with a new undergraduate who joined in March or so, And we essentially gave her a query of a couple of these issues that sort of had a type issue type of an analysis, and we said, hey. Like, does this, like, piece of this this single page plus the sort of contextual information that you can click into give you enough background information to to make a substantial research contribution. So it contains the description. It contains the the, you know, graph connected hypotheses that the issue was trying to to address. And and so the question is, can you claim this issue and then make that your ongoing experiment? And she was able to do that. So we gave a little some background information. Of course, she was collocated in the lab so she could get more hands on information from us. But over the course of just several weeks, she figured out what analysis needed to be done, ran it, carried out, found some some results, and then put them together in this in this graph. And this so this is great as a question. Let's see what the literature says, results, etcetera. Okay. For time, which I think we are about out of, why don't I skip ahead and just advertise that large language models can synthesize this information too. We are sharing individual results both sort of in a peer to peer fashion through a search of my discourse graph and somebody else's discourse graph. This is ongoing work. And to individually which we're experimenting with sharing results individually with with the world and and having the and and how having the sense making happen outside of our of our lab through LLMs, etcetera. I wanna advertise that we're putting together a user experience pilot where we're going to have a number of other groups interested in a shared research question, build one of these graphs, either in room research or some related note taking software and give feedback in exchange for getting their their labs set up. So let me know if you're interested in participating. And we think that this lab notebook software can serve as the coordination layer for decentralized, research by creating scaffolding for trainees, creating a more decentralized organization network for researchers, and to create a more sustainable culture of research contributions by researchers of different types where the fruits in this schematic are the discoveries and the roots connected with each other are these, discourse graphs. So thanks for your attention. I'd be happy to talk more, with questions." }, { "speaker": "Speaker 1", "start": 30.0, "end": 30.0, "transcript": "Fantastic. Thank you so much, Matt. I, I definitely have a ton of questions, but I'm going to respect the order. This is super exciting, from this kinda collective research sense making. So I'm gonna follow the track of questions that came up, which I think Rick was first, to drop some questions in chat. Rick, do you wanna come off mute and ask about the NIH and AI related questions you had?" }, { "speaker": "Speaker 3", "start": 45.0, "end": 45.0, "transcript": "Yeah. One thing, I I used to run a a a faculty development program. And back then, this is maybe fifteen years ago, there were a lot of NAIH career development grants. So that's one thing. It all comes down to funding to do this work. But the one that I'm more interested in actually is the the issue of how we can develop AI skills development program and AI tools to conduct and generate research questions, help people develop their skills, design protocols. I don't know whether there's anything equivalent in academia. I do have connections in the innovation space where they are developing these tools, and I'm sure if you're not familiar with them, and I'm not very familiar with them. I just know the people who do it. What what can what can learn from the the sort of business innovation space that can be applied to academia?" }, { "speaker": "Speaker 2", "start": 60.0, "end": 60.0, "transcript": "Yeah. Absolutely. So we are taking the the sort of tact that we think that large language models and other AI tools are extremely helpful and are best used in collaboration with humans who are doing some validation, who are contributing knowledge. So we have a a sort of large language model that's baked into this discourse graph that sort of reads elements of the graph and has given instructions about how to how to how best to help with sense making and coordination. So for example, you know, based on this one experiment, hey. Give me the premise approach results in conclusion, And the assistant, you know, uses our discourse graph schema, so you can still click into the answers. And so let's convert that into a narrative that would substitute for a research article and then, you know, include hyperlinks so that somebody can actually ground the l m's responses in a result. This is a lot like perplexity.ai, for example, but we would imagine that the database rather than being papers would be these unpublished research results, which I think could really speed up the so sense making, be it, like, before papers are published and also ground the large language models, answers question generation in something that is verifiable and gives gives the LLM the right layer of scaffolding to contribute to." }, { "speaker": "Speaker 3", "start": 75.0, "end": 75.0, "transcript": "Yeah. Just like to echo one thing you said, it's really the cocreation of human AI synergies. And the complexity AI is a great tool. I use it all the time in my clinical work, and I'll hand off to the next question." }, { "speaker": "Speaker 1", "start": 90.0, "end": 90.0, "transcript": "Thank you so much for asking, Rick. The next one was from Steve, and feel free to hop off mute if you wanna add any color beyond what's in chat, which was, is there a Bayesian element to this? Steve, I don't know if you wanted to add any more color or, Matt, feel free to just take it if that's straightforward enough." }, { "speaker": "Speaker 2", "start": 105.0, "end": 105.0, "transcript": "I'd be happy to riff off of probabilities, but maybe he had something more specific that he wanted to ask about. I'll just start talking. He can interrupt me. But, that, of course, with the lang the large language model, there's a a probabilistic element to exactly how it's how it's responding and what, you know, what, what terms it it pulls up. These graph queries are deterministic right now and that there are you know, this issue is informed by these three hypotheses, so that's the highest level of connection between these these discourse nodes. There's another layer of, like, indirect referencing that you can use for for graph search, And we are starting to surface related discourse notes through, like, a semantic embedding. And so that sort of similarity search has some, you know, probabilistic elements." }, { "speaker": "Speaker 3", "start": 120.0, "end": 120.0, "transcript": "What I" }, { "speaker": "Speaker 4", "start": 135.0, "end": 135.0, "transcript": "what I was specifically asking about was generating confidences and hypotheses." }, { "speaker": "Speaker 2", "start": 150.0, "end": 150.0, "transcript": "Yes. So right now, as a pragmatic for pragmatic purposes, we are generating the confidences ourselves. So each person has their own sort of manual, description of how strongly a, hypothesis is supported by evidence. We haven't needed to formalize it beyond that, but as you as the graph gets larger and there's more people you don't know as well, it'll be important to either score or qualitatively assess those. So Right now, the qualitative assessment has been sufficient for us to answer the questions that are right at the edge of knowledge since we're not we don't need to represent all of knowledge with the discourse graph. It's really what's at the edge of knowledge and what people are interested in in working on right now." }, { "speaker": "Speaker 4", "start": 165.0, "end": 165.0, "transcript": "Yeah. Great. Yeah. It seems like it does work really well in your local environment, and they're just gonna be different things that have to be tackled when you do scale it. So, yeah, you've got that in your head. That's the right orientation, I think." }, { "speaker": "Speaker 1", "start": 180.0, "end": 180.0, "transcript": "Oh, thank you, Steve. Yeah. And it'll be definitely interesting in future conversations beyond today to explore what this kinda looks like in more open environments, where it might be multiple labs coordinating or multiple organizations coordinating and building coordinated research efforts around that. But I don't wanna skip the queue. So next was Brian's question of how does attribution of work happen in a crowd sourced research world?" }, { "speaker": "Speaker 2", "start": 195.0, "end": 195.0, "transcript": "Yes. So, one of the great strengths of this approach is that each person's contribution is attributable directly to them. So I don't know. Let's let's pick this this issue, for example. Like, each specific, like, piece of text that people created has a time stamp and a and a user stamp. And so you can see based on which of these discourse notes you created, not only a stream of, okay, like, what what did I accomplish over the past quarter, for example? How do I make sense of it? But also who used it and when and how. And so this has already contributed to researchers' sense of satisfaction and that they're contributing not just to their own advancement, but towards, you know, problems that their friends and colleagues are find important. So I my favorite, like, version of attribution right now is something equivalent to a GitHub contributor board where you get to see, you know, for the next CRISPR, the big, like, discovery, where in the timeline of of, like, attribution were you? Like, where did your experiment show up that was important for this hypothesis, which turned into blah blah blah blah blah. So you get you may not even need to have some ranked, you know, attribution contribution table, but rather you just have a very granular and accurate way to to trace where your contribution fit into the these these larger discoveries." }, { "speaker": "Speaker 3", "start": 210.0, "end": 210.0, "transcript": "Got it. Thank you." }, { "speaker": "Speaker 1", "start": 225.0, "end": 225.0, "transcript": "Thank" }, { "speaker": "Speaker 3", "start": 240.0, "end": 240.0, "transcript": "you. I" }, { "speaker": "Speaker 1", "start": 255.0, "end": 255.0, "transcript": "was just gonna see if there is anything else there on your end. So, yeah, Josh, your first question was, could the discourse graph representation be applied to produce meta analysis papers, or is there still not enough fidelity?" }, { "speaker": "Speaker 2", "start": 270.0, "end": 270.0, "transcript": "It can. So that original application is for, systematic reviews and meta analysis papers. And so, rather than those being one off massive efforts, the thought is that you can now make sort of updatable sustainable meta analysis reviews this way. So so Joel has filled out his discourse graph with enough, I would say, attribute level information to say, like, okay. I want this study, but only the ones that are, you know, controlled controlled studies, randomized controlled studies, or I want this one that, you know, just where the studies are just based in China versus The US. So you can do it sort of a more systematic comparison. The I think that the system the schema is really lets you figure out and, like, what the next element that would be needed for to to complete your meta analysis paper and make it much more feasible for other people to to make individual contributions to it. But it's possible that you could just feed in a whole subgraph of the LLM of the discourse graph, give it to the LLM, and they would do a decent job of making a meta analysis or at least, like, a first draft." }, { "speaker": "Speaker 5", "start": 285.0, "end": 285.0, "transcript": "Have you guys tried this already? On some has, you said." }, { "speaker": "Speaker 2", "start": 300.0, "end": 300.0, "transcript": "Yes. I mean yeah. He doesn't he doesn't do it through an LLM. He uses it through, like, human brain synthesis, and it it works works great. You mean talking? Yeah. Yes. Through writing. Yeah. Right? Writing writing based on this, you know, extensive, you know, graph based knowledge base. And we use it we use the LLM so far to to, like, summarize the state of, like, our answer from from our, like, general club on this targeted research question. And then the LLM can also, you know, suggest the next steps." }, { "speaker": "Speaker 1", "start": 315.0, "end": 315.0, "transcript": "Alright. Cool. And, Josh, we'll come back to your other questions. Thank you. And, George Maybe" }, { "speaker": "Speaker 5", "start": 330.0, "end": 330.0, "transcript": "just a" }, { "speaker": "Speaker 1", "start": 345.0, "end": 345.0, "transcript": "Oh, please." }, { "speaker": "Speaker 5", "start": 360.0, "end": 360.0, "transcript": "Just a super comment. I'll follow-up separately, Matthew, about this. Yeah. There's a there's a effort by a bunch of us to kinda, like, design data schemas for meta analysis kind of intake tail OEMs. I think this would be super interesting to follow-up on." }, { "speaker": "Speaker 2", "start": 375.0, "end": 375.0, "transcript": "Yeah. Absolutely. And we can share, you know, like, our whole discourse graph and see how how it how the LOMs would would do with it. Be very happy to be part of that." }, { "speaker": "Speaker 1", "start": 390.0, "end": 390.0, "transcript": "Great. Thank you. Next, we're gonna jump to Alex's question on how do you visualize evidence that supports other evidence. For example, the evidence of why is the earth around probably requires showing that it supports in turn by making truth and kind of bracketed by making a truth tree validated with cryptography. And, Alex, feel free to jump on if anything" }, { "speaker": "Speaker 6", "start": 405.0, "end": 405.0, "transcript": "Yeah. Sorry also if this is super obvious and you already said that in technical language or that I'm not explaining it a lot. But, yeah, basically, what I'm trying to say is that, like, in in the real world, the context that I see is that two sides, two polarities are actually not they are failing to agree in the most basic things. So I'm I'm guessing that in sometimes, the evidence that is supporting a claim actually needs to be supported by other evidence that needs to be supported by other evidence. So it's almost like you need to go to the root of, like, okay. In which things we do agree? Like, what is the, like, the layer one of truth? And then building from there until you even get to branches where evidence is more subjective, and and you cannot even prove it, like, fully. So you have different approaches to the same answer. So I'm just wondering because in the visualization that you showed, it it it seems like it's all, like, it's all connected, but, like, also spread. It's not like like a train of thought or, like, a tree of of truth, just to to speed it up. Thank you." }, { "speaker": "Speaker 2", "start": 420.0, "end": 420.0, "transcript": "Yeah. So you can make nested layers of claims for sure. And so we're we're experimenting with some some sort of, like, zoomable interface so you can sort of see different layers of abstraction. And there's, the work on argumentation mapping in sort of, like, public discourse is one of the original motivations for Joel for for creating this this software. And in practice, for us, the way that the path forward that I see is that you have a small community of people who have agreed to talk to each other or even, like, genuinely interested in the same question, and they sort of hash it out and they get to some unknown that they both have competing claims for that they both generally wanna try to find the answer to. And then this schema helps them find what the right level of granularity is, like you were saying, going up and down, probably mostly through informal methods, and then you formalize the part where there's really, like where you're really clashing. I my feeling and suspicion is that that will work more effectively than posting the, like, the objective, look, I've proven it to you, approach. But, of course, both both are going on out in the world." }, { "speaker": "Speaker 6", "start": 435.0, "end": 435.0, "transcript": "Just to explain, maybe what I'm saying is, like, crazy, but I'm I'm assuming as well that with cryptography, you could validate each level. So you don't need to actually show the whole line, but you could at least validate that they are, like, a proof of knowledge for that claim. Just as I'm I'm just saying it as a user experience tool." }, { "speaker": "Speaker 2", "start": 450.0, "end": 450.0, "transcript": "Yeah. Absolutely. And I think that's also true for if you wanna turn this into, like, a distributed peer review system for new science. You said, like, there's been some, like, attestations for this, this, and that, so don't you don't need to look at the details, but now we're at this stage. I think we should definitely try it. I also like leaving space for different people having different interpretations of the same evidence because I think that's extremely common once we're at the edge of knowledge, which essentially all of the questions in in our graph. Different people in the field have a different interpretation of." }, { "speaker": "Speaker 1", "start": 465.0, "end": 465.0, "transcript": "Sure. Thank you. So we're gonna jump to Josh's next question now, which is how hard it is to produce such graph representations and how much additional effort is it versus what a science naturally does. Going forward, are there components that can be captured through automations and or hooked into other extent?" }, { "speaker": "Speaker 2", "start": 480.0, "end": 480.0, "transcript": "Yes. Absolutely. So one of the reasons why these type of approaches have not taken off before is because they're too much work beyond the researchers' own sense making process. And so this is a key part of the design of this discourse graph software. So let's pull up what those design features were. So the first one is to enable the researchers to seamlessly create this formal discourse graph alongside our informal notes, and these is these are, like, some of the, you know, plug in tricks that we use to to make that possible. And second is that we, let users leverage their formal discourse graph to improve our thinking and writing. So it's genuinely useful to us week to week when we're trying to, like, save our knowledge for our future selves or to even share the, like, the TLDR for in our weekly updates. So that lets that has sort of shifted the trade off balance within our lab such that it's just it's helping people track their own project and and keep their own mental model, like, clear and traceable. And then the the side effect is that you have the searchable graph that's, like, good for the system of science. I think just the UX in room research is too clunky for another lab to just pull up off the bat. So this is what this user experience pilot is. Essentially, we just make it simpler, make it, like, minimal port in another, like, note taking software. But again and again, we I'd like scientists to say, well, this is what I've been trying to do anyway, but it, like, it it's it's, you know, storing it in my head didn't work, and, of course, it can't be transferred by other people. And we've been, pleasantly surprised by how good an LLM is at taking unstructured notes and plots and turning them into discourse graph elements if you give it a specific enough prompt. Interesting. Even just, like, the image itself, it does a pretty good job of giving you the first pass, like, results, which I'd be happy to to to show." }, { "speaker": "Speaker 5", "start": 495.0, "end": 495.0, "transcript": "I'll just say that, I think literally Eugene and I were joking about this a couple weeks ago, how it takes a transcendent being to you know, somebody needs to have already kind of transcended reality to use things like Rome and Obsidian as their, like, primary note taking vehicle. So it's like, I'm glad your lab is, you know, has effectively transcended, you know, this plane of existence and, you know, result ended up in Rome. Really quickly, I'll just say the this is yeah. This may not factor into your intrinsic motivation. Kind of, oh, you know, like, let's use this to kinda, like, help people, like, do better science. But the thread we've been kinda pushing, and I'm kinda curious, and we don't need to discuss this here, but the it's interesting to ask, like, there are external incentives where we can maybe force researchers into doing such things, especially if they feel like they're gonna get outcompeted if they don't do it. Right? If they're like, this is the thing that, like, all the journals are gonna use to produce, like, out of produce meta analysis papers. If you want it, like, a credit on that, you know, suddenly you need to put yourself into discourse, discourse graphs, or whatever representation. So I think that that would be super interesting to kinda develop. So" }, { "speaker": "Speaker 4", "start": 510.0, "end": 510.0, "transcript": "for scientists to do actual science, to actually use the method they're supposed to be using?" }, { "speaker": "Speaker 2", "start": 525.0, "end": 525.0, "transcript": "Yeah. Or to, like, make it it could, at the very least, encourage people to have their observations be reproducible because that if that was as important as it being double, then then you knew that your sort of, you know, history of contribution would be influenced by the extent to which other people could reproduce it. That would already that would all already change the sort of incentive structure for what you're contributing. But we should definitely have that conversation because I like, right now, I'm leaning towards this sort of, like, utopian open source internally motivated version, but we we should talk about the the versions where there's sort of more explicit incentives to to make these contributions to." }, { "speaker": "Speaker 1", "start": 540.0, "end": 540.0, "transcript": "Yeah. And I'll just quickly say, especially around one that I I very much hope to get to collaborate, both from my new role and from MediGov and with you, Matt, and with Joel in the future is potentially trying to build one around DAO governance related research, building off of some of the DAO science work that Josh and others have done and converting that potentially into discourse graphs. And can you, both from the funder and from the information sharing perspective, have kind of somewhat forced nudges and or strict motivation towards fulfilling that. Because if you also attach that to say, you know, the kind of retro funding rounds that we're seeing in the optimism round, it's like, hey. Well, if you wanna get credited for your ideas and in turn get compensated in them for the in the future for those contributions, like Josh was saying, where whether it's from the journal side or from the funder side there. I wonder what some of the more formal or, you know, extrinsic motivations are. But, George, sorry for keeping you waiting this long, but I wanna make sure to to hand it off to you. I know you had two questions, so I'll I'll leave it to you as to which one you wanted to to run with given time. But, yeah, please take it away, or I can read it out if that's better." }, { "speaker": "Speaker 4", "start": 555.0, "end": 555.0, "transcript": "I don't know if Matthew could, like, pick one that he seems to find more interesting or wants to answer more. Maybe that would be better." }, { "speaker": "Speaker 2", "start": 570.0, "end": 570.0, "transcript": "Sure. Yeah. So this question about, like, if you don't have a now have a facilitator in the middle, so I love that my role has transferred towards facilitator. I feel like the system will be fully successful if I can transfer my role out to a researcher again. I spend 80% of my time just plugging away at the at the venture or or coding. And this is a the facilitator is a it would be a great role for for an LLM, I think. They're not expected to do the whole work. They're just helping to point people one way or the other. Absolutely. And your second question about combining with quadratic voting and retroactive public goods funding, absolutely. That's that's in the our ideal road map. You know, when Protocol Labs was supporting us, it was part of the the, like, the grand vision that we haven't concretely attempted, but I think it works perfectly in, this sort of open source idea, open source ecosystem where you make the contribution and then in hindsight, the impact, the like, how important how how possible the discovery was if you, like, got rid of your contribution would be would correspond to how much money you got. So then you're you just do the work that's important to you, and and you get supported as a result. That's very much, the future that I wanna be doing research in." }, { "speaker": "Speaker 4", "start": 585.0, "end": 585.0, "transcript": "That's fantastic. Thank you." }, { "speaker": "Speaker 1", "start": 600.0, "end": 600.0, "transcript": "And I know I'm I'm very excited to as mentioned multiple times, you know, there's a couple of folks from the PBS Foundation, the Proposer Builders Operation Foundation here. And I know we're quite keen to explore this and maybe get to play around with adding some of those mechanism elements and see how we could potentially collaborate on multiple fronts. I guess given that we're getting to the last minute or two of the session, and I did drop a link in chat for anyone ask questions in Slack as follow ups after. But I guess, Matt, any thoughts and or suggestions for folks who are kind of walking away from this being like, hey. This sounds super cool. Now what do I do? Do you have any kind of suggestions of how someone can actualize this? And, of course, there's a QR code ready for just this scenario." }, { "speaker": "Speaker 2", "start": 615.0, "end": 615.0, "transcript": "Yeah. So so this talk itself is in an open notebook in Room Research that you can access and look through, with this this QR code. And so looking through that, it has, like, contact information and a a couple of places where we're documenting our ongoing experiments. And, again, if you have a use case for forming a research group that could use a discourse graph and are willing to give user experience feedback, these coming two years look like where we're gonna be running an extended series of pilots that we'd love to, have your contribution in." }, { "speaker": "Speaker 1", "start": 630.0, "end": 630.0, "transcript": "Amazing. Well, thank you, Matt. In keeping with the MediGov tradition, if folks can come off mute in a moment, I'll do a countdown from three to one and just give Matt a round of applause and sign of appreciation. So let's do that in three, two, one. Yeah. Thank you so much, Matt, and appreciate you sharing the QR code with a way to stay in touch. Yeah. We'll definitely be in touch from multiple perspectives. And, Yeah. I hope this is a session that everyone enjoyed. And, yeah, thank you all." }, { "speaker": "Speaker 2", "start": 645.0, "end": 645.0, "transcript": "Thanks a lot." }, { "speaker": "Speaker 1", "start": 660.0, "end": 660.0, "transcript": "Thanks the day." }, { "speaker": "Speaker 2", "start": 675.0, "end": 675.0, "transcript": "Looking forward to talking more." }, { "speaker": "Speaker 5", "start": 690.0, "end": 690.0, "transcript": "Thanks, everybody. Great session. Thanks, everyone." }, { "speaker": "Speaker 3", "start": 705.0, "end": 705.0, "transcript": "Thank you very much. Cheers." }, { "speaker": "Speaker 4", "start": 720.0, "end": 720.0, "transcript": "Thank you." } ], "summary": null } }