83 Discovery Sprints Re-release
Civic Tech Chat | 2023-01-24 | 1:02:19
We're pulling one out of the archives to start off 2023!<br><br>This time we talk discovery sprints! We're joined by Jenn Noinaj (https://www.linkedin.com/in/actuallyjenn/)} and Kathryn Jurick(https://www.linkedin.com/in/kathrynjurick/), authors of the United States Digital Service guide to discovery sprints. We'll chat about what these are, how they're run, and how they can help.<br><br>### Resources and Shoutouts:<br>- USDS Discovery Sprint Guide(https://sprint.usds.gov/)
Top Keywords
- sprint 0.025
- discovery 0.015
- sprints 0.012
- discovery sprint 0.009
- team 0.009
- usds 0.008
- agency 0.008
- might 0.008
- discovery sprints 0.007
- folks 0.007
- different 0.006
- talk 0.005
Transcript
Speaker 0
0:00 – 1:14
Hello. I'm Ryan Cook, and this is Civic Tech Chat, a show that looks at the way technology, politics, and policy impacts the world around us. The tools we use, the way services are delivered, and how we talk about and set policy all shape our society. We'll gather around and have a chat about these things together and more. Before we get started, I do wanna let you all know that we've started a Discord for the podcast. There will be a link with an invite down in the episode description. Do feel free to go check that out. It's a small community right now, but hoping to grow it. It's a great way to reach out to me and let me know things that you might want us to cover or to just hang out and talk about civic tech. Anyway, let's go ahead and start the show. This is an episode from the archives. So there may be things that are mentioned like events, links, email addresses, and the like that might be inaccurate or may have simply passed because this happened a while ago. So let's go ahead and sit back and enjoy this episode about discovery sprints. Kat, Jen, thank you both so much for coming on to Civic Tech Chat. Could you both, introduce yourselves and tell us a bit about what you do?
Speaker 1
1:15 – 2:07
Thanks, Ryan, so much for having us both on here. We're so excited to be here. My name is Jen Noina, and my background is in human centered design. I've worn many hats throughout my career. So I've been a user researcher, design strategist, product designer. And most recently, I was at the US Digital Service for two and a half years as a product designer. I am currently at the Beck Center for Social Impact and Innovation at Georgetown University, and I'm actually working on supporting the people in this space, so in the civic tech or gov tech or public interest tech space. And I'm looking at things like diversity, equity, and inclusion, and really trying to figure out how do we make sure that this industry that we're in doesn't turn into, you know, Silicon Valley tech. And we're more we are we can be as, inclusive as possible.
Speaker 2
2:08 – 4:02
Thanks so much, Ryan. It's great to be here with you and Jen today. My background is, primarily as a design researcher. I've been working in was working in private sector for more than fifteen years in places like Silicon Valley, in Seattle, in Boston, and, working at big companies, little companies, hardware, software, all kinds of different things. It's been, kind of a mixed bag for me, but it has meant that I've gotten to do a lot of different kinds of things in my career, and that's always really fun to get to talk about. Most recently, I've been at the United States Digital Service, which is how Jen and I know each other. I started out in the at the very end of 2016. I joined and was originally an an individual contributor as a design researcher on the Defense Digital Service team. After that, I had the opportunity to step into a leadership position at USDS HQ as design director, and I did that for about a year and a half. And then during that time, was responsible for doing a lot of staffing for our discovery sprints, which is one of the reasons that brought Jen and I to the conversations we started having that led into the work that we're gonna talk with you about today. Most recently, I am now in the process of heading back over to the defense department as director of design operations for a group inside, Space Force, which is another really exciting opportunity because they are setting up it's the first new military service in over seventy years, so they're setting up a lot of they have the opportunity to set up a lot of technical infrastructure differently than the way that the DOD has been doing things. And so I've been really enjoying that so far, but it's it's a very new thing for me right now, but it's super exciting.
Speaker 0
4:03 – 4:11
For each of you, what would you say is your personal why? The thing that drives you to get out of bed each morning and do what you do.
Speaker 2
4:12 – 5:10
Sure. Yeah. I think what has been really wonderful about finding a path into working inside civic tech is that there's such a strong sense of mission both for my personal self and the work that I'm doing and what drives me as well as the people that I'm working with. And I think that, you know, getting up every day and just looking for opportunities to make things, systems, technology experiences better for folks who, are dependent on government systems or on government agencies to really give them the support that they need is something that it's it feels really simplistic, but it's the thing that really helps me feel like I'm I'm doing something and I'm accomplishing something at the end of every day.
Speaker 1
5:11 – 5:51
I echo a lot of what Kat says. For me, it's really at the core of it is making a difference. I really, really care about helping others and contributing to society in some positive way. One of my mottos is, you know, how can I make the how can I leave the world a better place? And the way that translates, is, you know, that might be me being a supportive and reliable person to my friends and family and people who depend on me. And professionally, it looks like how can I use the skills that I have, for the greater good?
Speaker 0
5:52 – 6:04
Each of you has spent time both in the private and public sectors, which may differ in some interesting ways. As you think back on those experiences, what are some things that have stood out to you or surprised you?
Speaker 1
6:05 – 8:02
I think Kat sort of mentioned this a little bit in her personal why, the mission driven aspect of it. And I would say there is you know, I didn't really know what to expect coming into the public sector. One of the things that has stood out to me the most is how mission driven everyone is in this space. I think it's a very, like, infectious, optimism almost. And, it's it's really exhilarating, I think, being in this space and knowing that you're surrounded by other people who care just as much as you do. I would also say the scale, is one of the biggest differences. So similar to CAD, I also had a lot of experience in the private industry before I came into government. And, you know, the products that I was working on in the private industry or private sector were a lot smaller than I'm than the services and products I'm working on at government. And so, I think as a designer, that complexity is so so complicated and just such a fun thing to to work on. And then I think the last thing would be I I think it's like this question that I that's still unanswered for me. When we're working in the public space, sometimes we have to make the decision between fixing the system, and I mean, like, overall system, not like a technical system, versus fixing the symptoms. And I think that's something that I don't have an answer for yet, but, you know, there are these large problems that we're seeing in society today. Inequality, you know, health care access, just like basic human rights. And it's really it's just kind of a great place to be because we get to tackle those issues.
Speaker 2
8:03 – 11:28
Yeah. I would echo a lot of what Jen said about scale. The difference, you know, in working at technology companies, even ones that are I mean, a lot of technology companies stays in, say, in startup mode for a long time. And so you're working to your everything's around a million. Right? Like, you're working towards your first million dollars. You're working for your to your first million customers. You're working towards you're building on top of a base, that starts pretty much at the at the zero scale. In the government, it's almost entirely opposite. And so you have that that play of scale where you're on day one, on anything, really, you're looking at hundreds of millions of dollars involved. You're looking at hundreds of millions of people who are affected by these systems. Your customer base is different. I'm using air quotes for that. In private sector, you're looking for customers who are going to see value in what you are offering and pay for the opportunity to use your service or your product or whatever it is. In government, a lot of times, it's I don't wanna say it's a last resort, but oftentimes, whatever the solution that is being provided is you are the only way that somebody can do that. Right? Like, there's you can't file your taxes with some somebody that isn't the IRS. Right? Like, you have no choice but to interact with that system as a taxpayer. And so it's not only just a scale, but the idea of how you can meet the mission with so many different kinds of users and people who are coming from all different levels of technology experience. You can't really assume anything about who those who those users are going to be. And as a design researcher, that's really fun, right, because it means that you get to go out and interact with lots of people. And even if you're making incremental changes for the better, you still get to see the impact that you're having. I think that's a little more challenging to do on the private sector side. And I think the other thing that was I think you asked what was surprising. For me, what's been really surprising is how, design curious a lot of the federal government is. I mean, a lot of time we'll be talking a lot about federal government today. I think this is true for other aspects of government and other work that is happening in civic tech, but there is such a greenfield for design and and for human centered thinking. And everyone that I've had the pleasure of interacting with over the last four years is just super curious about how can we design better for the humans who are interacting with the systems that we're creating. Sometimes that's the systems. You know, it's somebody who's sitting in front of a keyboard who is then providing services to somebody who's sitting on the other side of that counter. And all of those people are humans that, you know, human centered design needs to be applied for when you're working at the federal level. And that's that was a surprise to me. It's just sort of how curious and how excited people were to to talk about this stuff and to give it space in a way that in the private sector, I think a lot of times design fights for a seat at the table with engineering and other disciplines that are often a little bit more transparent about the impact that they are having for on behalf of a company.
Speaker 0
11:29 – 11:57
I think this is a good time for us to take a shift over to our our main topic, which is something that I'm I'm pretty excited to dig in with with each of you. Folks listening might not know that each of you were involved in the creation of a discovery sprint guide that's hosted on The United States digital service, USDS on their website. For folks that might not be familiar with what a discovery sprint is already, or maybe even a sprint as a concept, could you talk us through what this is?
Speaker 1
11:58 – 13:31
Discovery sprints is something that I would say a lot of folks in the tech space have heard of before or the term discovery. And so at The US Digital Service, it was a pretty we we used it in a different way. I would say that for us at the federal level, we were you know, when whenever we get invited to partner with an agency or a team, on their problem, we would use discovery sprints as a method. Just a two to four week quick sprint for our teams to really dig into the agency, the team that's working with them, the people, the the landscape, the ecosystem, any of the root causes that might come out of that engagement, and really try to understand the the work that we're going to be supporting. I wanna iterate here that, you know, oftentimes at USDS, we would get invited to fix some sort of tech issue. And what we realized, and what we all know in this space is that tech is usually the solution, but it's not normally the problem. Right? And so it was mostly a way for us or discovery sprints was a way for us to really understand. Alright. Here's the technology that we may be looking at, but what is all of the stuff around it, that influences that technology? So the people and the processes and the culture.
Speaker 2
13:32 – 16:52
Yeah. I think, I just pivot off a couple of things that you said, Jen, that the United States Digital Service is lucky enough to be funded with money that means that they can provide folks or resources to any project across the federal government. So we have, a couple of different partner agencies, AT and F being the one that that people on this podcast will probably be most likely familiar with. AT and F is is fee for service, so it's essentially like hiring a consultancy. So if an agency wants to if an agency knows that they have something that they need work for, they can essentially hire they can hire out. So they can go through procurement and hire through contracting, or they can go to a group like eighteen f and pay for the services that eighteen f provides. For USDS, when we originally got stood up, we were doing a lot of emergency responses. And so, you know, something would fall down in the middle of the night, and we would send people to kinda go and figure out what that problem was. And there's a lot of kind of duct taping a bunch of servers back together, and that often was our first foray into understanding the needs of a particular agency where the things that were happening underneath the cover of darkness that were that were often the root causes of issues that were going wrong. And so that's kinda how we started is that we would go out. We would solve a complete we would solve a quick technical problem. And then when we were talking with the stakeholders, we would get invited to kind of stay a little bit longer. And Discovery Trends for us is a really great way to kind of do a quick investigation and just kinda see, like, does this seem like a kind of thing that USDS could actually help with? Right? Like, do we have the right skill sets on our in our group to be the people that are the right people to help them? And oftentimes, I would say 99% of the time, what one of the things the deliverable that we produce beyond the act of going and go doing a discovery sprint is a sprint report. And that sprint report lives with the agency. It belongs to them, and it has a series of recommendations and, observations that we have made. And oftentimes, that goes to their senior stakeholders. And then if work continues after that, either through USDS or through any other way, that report is is kind of a a a support system for kind of setting up the direct the right direction to head. USGS isn't always the right people to to try and and resolve those. Sometimes we would say, look. You guys are doing exactly all the right things. Just keep doing them. And here's our third party report that says that you're doing all those things, you know, great. Sometimes what they would need is is not the kind of support that the USDS could have good resources for, in which case we could say, this is what we recommend that you go and do. And then the third case is when it is a good match, in which case we start talking about, like, having longer term engagement with that organization, to try and bring them different resources, or a different perspective or the expertise that USDS has into that agency.
Speaker 0
16:53 – 16:59
For each of you, what do you think is the the why for for putting together a guide like this?
Speaker 1
17:00 – 18:36
So when Kat and I first started, Kat mentioned that she was responsible for staffing a lot of these engagements. And I had you know, at being at USGS, I had been in around been in and around, discoveries for a while. I had been on some. I had led, a handful of others. And, you know, I we we both got folks coming to us asking us for, you know, oh, how did you run this? How did you run that? And we realized, you know, we we first started putting together this guide as something that internally we could use, all the way down to the brass tacks of, you know, who should be on the team, how do you start an engagement, how do you write a sprint report, how do you conduct interviews, who who do you talk to when you go to an agency. And, as we started putting all of our resources together and writing the guide, we realized that this this guide we were writing could be very helpful for other teams just outside of USDS. And discovery sprints are definitely a method that, you know, we find value in, and we've seen other folks use it and also find value in it, themselves. And, so I thought that I think we both thought that it'd be really important for us to document, which is something that we could always do better on, for other teams as best practices for to them for them to use themselves.
Speaker 2
18:37 – 21:11
Yeah. I would echo a lot of what Jen said. I think in the early so USDS is about, seven years old right now. And in the early days of USDS, they did a lot of great documentation. So they wrote down what they were doing, why they were doing it, how they were doing it. And the landscape changes at USDS a lot because people come in on tours of duty, and so it's not the same people, who are staying. And so how do you pass on the information that you're learning and the things that you are doing to the next generation of people who come after you. A lot of times right now, especially I think it this is true this is definitely true for USDS. I think it's true for a lot of civic tech environments that a lot of that is just oral tradition. Right? That you are on a sprint You're on a discovery sprint with somebody else who's done a couple of discovery sprints before, and so you learn through the expertise of, you know, somebody who's done that before you. You get staffed on an on an agency team, and and you learn about how that agency works a lot from the people that are that you're partnered with on your team itself. And then over time, you build up a set of expertise that essentially walks away with you when you leave. And so, you know, we knew that there was a need for us to have internal documentation. And I think Jen and I both, like, felt like we had done good work while we were at USDS, but this idea of, like, wanting to leave something behind for our other US future USDSers. And so that was that was a piece of it. I think the other piece is that we when we talk to other civic tech groups, every civic tech group is is struggling with a lot of the same core things, right, and trying to figure out how to how to do what they're doing in spaces that that often haven't in had teams like this or had people that, have these different types of expertises. And so especially, when I was design director, we would talk to other organizations or leadership at other, you know, at civic tech organizations or digital services all over the world. And everybody was like, what are what are you guys doing? How did you solve this problem? Like, how are you thinking about this? Like, what is how are you, you know, what are you doing, and how are you doing it? And we were all doing that with each other all the time. And I think this is sort of our, our small gift back to that, that discipline or that group of folks who are are trying to solve some of those same things or tackling some of those same issues.
Speaker 0
21:12 – 21:28
When we were having our, preparatory chat, that like, that prior call, one of the things that came up is this notion that discovery should be thought of as a method, as a deliverable, and also as a process. Could we dig into and expand upon that that notion a bit for folks listening?
Speaker 1
21:29 – 25:10
We mentioned that discovery sprints are typically two to four weeks of an engagement with a stakeholder. And so at the US digital service, that would typically be at the federal level. And so when we first start, we typically would put together a team of a really small team, like four to five folks. And we make sure that this team is a cross functional team. There is, you know, someone who has a product management background. There's someone who has design and research as an someone who, you know, is an engineer by trade, someone who has expertise in data and procurement. And we also make sure to staff the team with a lead. And so this, the sprint team lead is responsible for really the engagement with the stakeholders and, really managing expectations of what to expect, as they engage in this process with us. And so, you know, it's the discovery sprint is really quick. We go in, and for two weeks, the team sits with the agency. You know, we try to get access to as many things as we can, systems, documents, people. And we the entire team just, you know, spends a lot of time digging into documentation and interviewing stakeholders and interviewing users and interviewing, everyone from leadership all the way to the folks, you know, typically, like, at a call center or something like that. And that being said, out of these two weeks, Kat mentioned that, we typically result in a, a sprint report and presentation. And so, you know, the two weeks for us is really for us to uncover all of the root causes and, you know, the understanding of an agency's problem. It's really not for us to say, hey. This is how you're gonna fix it. We might have some ideas, and we might provide some of those recommendations, but it really is to dig into, hey. You know, it might just one of the things might be, you know, there's a person in this department and there's a person in this department, and they might need the same data, but they're getting data from different sources. Right? And so we might identify, because we do have access to the entire, ecosystem of the organization where someone else might not. We might be able to say, hey. You know, person a might just need to talk to person b, as one of the things that we uncover. And so, kind of setting that stage, when we think of discovery as a method and process, that's what I would typically say. Right? Like, it's this process that we go through to really, really understand the problem even before we start solutioning. And I think that that's key because I think in in a lot of these spaces that we're in, folks already have an idea of what the answer might be. Right? They might say, hey. We need a new database to do this thing. And it it might actually just end up being a communication problem like in the situation that I outlined. And so Discovery Sprint as, like, a method and a process is really, to uncover to really, really figure out what is that problem space that we're working within.
Speaker 2
25:11 – 28:35
What can be really nice about Discovery Sprint and I think this is one of those things that it makes what we makes this guide that we have written more applicable or adaptable for other organizations out, like, beyond just people who are working at the federal level or beyond, folks who are even working just in civic tech. I mean, I think the idea of a design sprint or a sprint zero has been around for a really long time. In organizations that work inside agile, the idea of a sprint is is pretty common. There are places inside the federal government that are doing waterfall or doing other kinds of development methods. And so the idea of a of a sprint might be, a little bit new to them. But one of the nice things about it being so quick in terms of an as an experience is that it's pretty easy to get a yes to, to try out doing a discovery sprint. And so as a deliverable, you know, I think and, Jen, you touched on this too that, like, most of the folks who are working inside these federal agencies, they are the subject matter expert in their agency. They are the subject matter expert in their, in their end users or in the people who are on the receiving end of the services that are being provided. They know all of those things cold. And 02/2001, they've probably had some group of people come in every couple of years that say, we're gonna help you do things better. We're gonna help you, you know, change it up, or we're gonna teach you a different method to do things. And they are incredibly patient, and are often willing to, engage and to to try to adopt these methods to see if they can, result in a a different approach or a a different kind of solution that may not have been available to them prior. As a deliverable, what's nice about having an external third party come in, it's not so much about the expertise that we bring as it is just the opportunity to shake things up a little bit and to give the folks inside different organizations, or different groups a voice. And so, you know, as I said, they're the experts. They know the stuff cold. They often have maybe been filing tickets or they have been, you know, writing or talking to the folks above them for a long time about whatever symptoms that they are seeing that might be causing larger issues. And for us to come in and just to get people into a room together and get folks, explaining things to us as a third party can sometimes just help open up levels of communication or opportunities that have been there the whole time. But this is just the catalyst to to make that happen. And so I think, like, as a deliverable, it's the communication piece that supports and encourages and just gives the opportunity for that to happen in a way, that is really useful. And, you know, when it's done quickly, it can still instill a lot of opportunities for that to continue to happen long after we're not there anymore.
Speaker 0
28:35 – 28:56
One section of the report that, kinda stood out to me when I when I was reading through it is the one that's titled how sprints fail. Seems like this gives some sense of the things the team might seek to watch out for as they're going through their their own sprint. And maybe often, like, kind of knowing, like, what things to watch out for is just as important as knowing, like, what are the good things to do.
Speaker 2
28:56 – 30:41
Are there any potential problems from that that kinda stick out to each of you that you'd like to talk about? I mean, I think what is what's interesting I mean, Jen got to participate in a couple of different sprints, both within the agency that she was working at and then as, part of cross agency work. I got to see and staff a lot of different sprints, during the time that I was at USDS. And there are definitely patterns that repeat themselves, and so this was that section was an was an effort for us to kind of pre identify those things. I think what's crazy about a discovery sprint is that you're going in to discover. You don't always know what you're gonna find, and you don't always know, what is going to be needed in terms of different skill sets. And you don't always know what, you know, what the environment is gonna be like. And so there's there are some things that you, as the creator of a sprint team, can control, and there's a whole lot of stuff you can't. And because it's discovery, it's different every time. And so figuring out which things we could kind of distill down into good advice to give people was really challenging because we wanted to give enough information for people to make to make it useful if they wanted to try using some of these methods. But we didn't wanna be so prescriptive that if you ran into something that wasn't on that list, that you wouldn't know what to do. And so I think that from a when and how sprints fail, we do see patterns. And I think, that's something that we tried to raise up and, you know, give some some hints as as to the kind of direction that those could be.
Speaker 1
30:42 – 34:45
The things to me that really stand out in this section, would be, you know, managing scope and managing expectations. Right? So we have to always be realistic about you know, for the most part, discovery sprints are only going to be two weeks. We really, really I think it works so well because we time box it. Right? So that's, like, the first takeaway for anyone who's thinking about doing a discovery sprint. Two weeks we find is, like, the sweet spot. You know? We've done sprints in the past that have been one week. We've done sprints that have been three or four weeks. And we find that two weeks isn't just enough time to spend the first week going really broad and uncovering all of the different rocks, talking to so many different people, and then spending the second week, you know, having the team decide, okay. We've gotta go even broad broader or, you know, starting to zero in on a path and say, hey. We're gonna we're going to focus on this one rock we've uncovered because it seems like it's going to make the most impact for this space. And so scope is huge. Right? There's, there's so many different opportunity areas to fix when we go into an agency or when we partner with anyone. And I think that even stakeholders sometimes will come to us and say, hey. We want you to do this and this and this and this, and we realize it's only gonna be five of us, and we've gotta be really realistic about what our North Star is going to be. And so for me as a you know, in the past when I was a sprint team lead, it was always at the end of every day. Right? There's only ten days. At the end of every day, syncing up with the team and understanding. Alright. Is our North Star still our North Star? And what I mean by North Star is the reason why we're here. The problem that you know, the general problem scope, is that still the thing that we feel is the right problem to focus on? Sometimes it changes mid sprint through, but I think it's important for the team to really rally around what that scope is like. And then for the sprint team lead as well as well as the other sprint team members to manage expectations of what that is. Like Kat mentioned before, you know, sometimes when we partner with an agency, it is a little bit daunting and is a little bit scary for the folks that are, you know, see us. We're, like, kind of walking in and they're like, oh, what's going on? Who are these people? And, you know, there's always this sort of connotation. And so it's so important for anyone who's doing the sprint to just recognize that. Right? So they can start to mitigate that. For me, personally, and I think anyone who's ever done a sprint, we never wanna be seen as an outsider coming in, telling what people to like, telling people what to do. And so, making sure that whenever you're interacting with anyone, that you you exude the right, values that you want to stand for. And so for me, that always was, you know, be really humble, just curious. Right? And really just recognizing your privilege when you get to come into these spaces. And then I would say the last, like, tactical thing from, like, a logistical perspective is that, you know, because sprints are so short, these two week sprints, not having, like, fully dedicated team members is really tough. Right? So for anyone who's thinking about doing this, just being aware that, hey. You know, we we can only have, like, four to five people on the sprint. Let's make sure to have let's make sure that everyone can be fully dedicated on the sprint. It just becomes kind of a nightmare when you're trying to, you know, in one week, right, interview 40 different people and, half of your team might have other meetings they have to go to. So we found that, you know, if we're able to, like, put boundaries and timebox lists, sprints are much more successful.
Speaker 2
34:46 – 36:06
I think I'd add one more to that just as that with that scope and the time, when you when you are uncovering issues that you it's really easy to jump into solutioning. Right? It's really easy when you're starting to talk about people and you're like, oh, I know I know a good answer for this, or I know how they could solve this problem, or I've seen this before at another agency, and, like, I want I want to to I want to help solve this problem that, you know, folks are facing, staying in that our point is discovery. Our point is to uncover these things. Our point is to write them down. Our point is to start the conversation with within that agency about which things, are going to be their priorities, and that's completely up to them. When you start driving towards solutions too quickly as or at all as part of a discovery sprint, you tend to get really, boxed in, and those things tend to to become larger than they might be in the whole ecosystem of just looking at all the different pieces that are getting put together because they are familiar or because you you know what the answer might be for that particular one. And so I think that that's another one that when we've seen when we've seen teams struggle, it's because they are they're trying to jump too quickly into to solving some of the things that they're uncovering.
Speaker 0
36:07 – 36:34
Related a bit to that, and as far as, like, if you're taking a team who's done a sprint and you're looking back, they might go through an exercise called a retrospective, where they kinda go and try to evaluate what happened during their discovery sprint. I imagine the way one goes about facilitating that and and how that's handled is is really important. If there's someone out there listening and they're going through this and they're kinda looking for advice on how to go about that, what what tips might y'all give someone like that?
Speaker 2
36:35 – 39:35
I think I definitely have a couple of tips off the top of the bat. Don't don't skimp on doing a retrospective, Especially if you've put together a team that hasn't worked together with, each other very like, if you've brought people from different orientations to work together, it's really useful to have a retrospective because at least you're getting a sense of closure and you're sort of giving everybody the space to mark what they did, to talk about it, and then to go back to whatever it was that they were doing before. So that part is really important. I think from a facilitation perspective, get a third party to come and facilitate the retrospective so that everybody who participated in the sprint can talk. It's really easy to say, oh, well, the sprint lead was leading, so the sprint lead can lead the facilitation. But that leaves out the, you know, incredibly robust feedback that the sprint lead might have to give. And so, those are sort of the the quick hits that I would give. When I have run sprint retrospectives in the past, I usually do ninety to ninety minutes to a hundred and twenty, which seems like a really long time, but taking the first forty five minutes to an hour. And I just ask the team to tell me the story of the sprint. Right? What happened? What did you do? And then what did you do? And then what happened? And who did you talk to at this stage? And I will whiteboard the story that they're telling me, in bullet points. And that's really great because there's no emotion involved. So if this was particularly stressful or if there was, if there were differing opinions on the team about what happened when or or, you know, how something was handled, you take all of that out. And so if people are just telling you the story, what happened, and then you take a break, and then you come back. And for the second piece, it's sort of talking through why some why those things why do we think those things happened and taking time to say, okay. Well, like, this like, looking back, you can say, oh, this was a this was a turning point. This was a pivotal turning point in the sprint. We didn't know it then because we didn't realize it because we we're standing at a fork in the road. Now we can look back and we know that. And, again, it takes some of the emotion out of the, you know, who said what when, and it puts people into a more positive perspective of, like, it happened the way that it happened, and so now we're reflecting back. And then I think the the last half an hour or so, I always like to be, you know, what advice would you give for the next team that is doing a sprint? So, actually, two things. One, at USDS, what advice would you give if USDS continues to do work at this agency? What are the what would what things do people really need to know? And then the second piece is, like, what is it that you learned about doing this sprint that is useful for other people? And so I had photographs lots of photographs of whiteboards, from having done these retrospectives when Jen and I started talking about writing, this guide, and a lot of that feedback fed into, what ultimately became this discovery guide.
Speaker 0
39:36 – 40:09
So whereas, like, a retrospective, you know, we're we're kinda looking at that that's kind of like a norming activity that you sort of do at the end of an activity, and then maybe you end up kind of going on to another. But one thing I've picked up, through some of y'all's answers is that oftentimes or maybe not often, but sometimes these teams are kind of new teams that are brought together. Maybe it's five people who haven't done a lot of work together previously. And in those situations, are there any kind of norming activities that y'all have experienced that, like, really kinda help that initial hump you have to get over in order to, like, work well cohesively?
Speaker 1
40:10 – 43:04
I remember the first sprint I led. Everyone on my sprint team knew of each other. Right? Like, everyone knew that we all you know, they knew what, what discipline everyone was, but they had never worked with each other before. And I as corny as this sounds, right, like, team building activities exist for a reason. And so, I think that it's so important when you're doing this really intensive work in a short amount of time to build psychological safety for the team. Right? To be vulnerable, to be honest, to to feel like they can really thrive and build trust with one another. And so the way I've done that in the past is, you know, we we do a sprint kickoff with the stakeholder, and that's really to set the scope, give ourselves the chance to ask, that stakeholder, any questions we might have and set our expectations and their expectations for the engagement. And then we also make sure as a team internally to really just get to know one another. So things like, what are your strengths? What are your weaknesses? What are you know, what's something that you really, really don't wanna do, so don't ask me to do it on this sprint? So I had a few folks that were like, I am so bad at writing. I'm so sorry I'm not gonna be good at writing this sprint report, but I'm so happy to help edit and revise and do grammar. And so I think just building that trust within the team just by asking each other questions, getting to know each other. And another key piece for that I found in norming is, is just bringing snacks. As silly as it might sound, like, we had a designer on our team go to Trader Joe's, and she bought, you know, $30 worth worth of snacks. And so we just always had snacks in our the space that we worked in. And people could come in and, you know, like, agency folks would come in and say, oh, can I get can I get an Oreo? Or I think they're, like, called Joe Joe Oreos from Trader Joe's. We we could, like, start a conversation that way. Right? Or, you know, we need an afternoon pick me up, and, we we'd have chips that we could munch on. So just really being intentional about building a good positive culture and asking each team member to contribute to that as well as, what what's important to them really, really helps with that team building.
Speaker 2
43:05 – 44:25
Yeah. I think what the one thing that I would add that I've seen when teams really gel, I think one of the things and it's, you know, hard to say this at month whatever we are in the midst of a global pandemic, is oftentimes, you know, one of USDS's core values is go where the work is. And a lot of times, these agencies a lot of agencies are located in DC, which is where the the core of USDS is. But a lot of times, sprints involve going out on the road to talk to external stakeholders, of various types and kinds. And so the act of traveling together with the team is a real uniting force. Right? Like, the even just figuring out who's going where, when, and and the opportunity to have dinner together at the end of the day after a long day of of user interviews, or stakeholder interviews is something that can really pull a team together. And sometimes when teams don't travel, it's it become it's a little harder to kind of create that that space for a team to be all kind of all in the same way that you are if you're working out of a hotel lobby together, starting to write the beginning of the the report. And so that's it's something to maybe aspire to even though that's it's not something we can actually do right now as much.
Speaker 0
44:26 – 44:56
No, Kat. That is a a great segue to the the very next question I had. Your mention of that principle about going where the work is. You know, as you mentioned at the end of your answer there, it's it's a little it's a little harder to do that now. You know, we're kind of in a position where for folks' safety, we kinda have to work at a distance, do things over distributed technologies. As you've kind of, like, seen folks trying to kinda still get to that same goal with these different tools, how have those folks been kinda, like, working around the pandemic for for lack of a better way to put it?
Speaker 2
44:57 – 46:34
You know, when we first started writing the guide, we didn't have a remote section, because it was sort of pre or very beginning stages of the pandemic. And so at a certain point, we were like, oh, we really if we're gonna make this useful for people who are living in the now, then we can't talk about it as sort of aspirationally. We need to talk about the way that that that things really are. I think one of the things that has been surprisingly smooth from a running discovery sprints at the federal level is that so many of our federal partners are also doing telework right now or working remotely. And so when a situation when everyone is remote, it's a different kind of experience because you're all having you know, it's not it's not great. It's not ideal. You miss a lot of the surrounding environment of what you would see when you were going to somebody's, you know, place of work and interacting with them. You know, as a researcher, you always wanna do that. Right? Because you get to see, like, what are the posters that you have on the wall, and what are what does this post it mean, and, you know, why do you have three monitors, and, you know, kind of talk me through what it looks like to be you at your desk. You miss some of those things. But during this past year, I think because everybody has been remote, it's just sort of we're all in the same boat. And that has enabled us to sort of get into the core of what people's paths or roles or issues or the things that they're dealing with because the the things that we are looking to discover all still exist even in a remote space.
Speaker 1
46:35 – 49:11
Yeah. I think that's a great overview, Kat. For me, it does it is a little bit harder because, those water cooler conversations are so key. Just, you know, seeing something and then just following up on those things. And so the way I've overcome this is kind of using, like, a chain reaction type of thing where, you know, we identify handful of folks that we might wanna talk to first. That, like, stakeholders or leadership might say, oh, these are, you know, the people that are involved in this project. They're the a good place to start. And from there, you know, as a researcher, it's important to build trust. And I usually do that through, like, jokes and humor. I feel like, especially now, it's hard because we are living in a pandemic, and it is really stressful. And so as much as I can, just kind of bringing in some playfulness into the space that we're in, really, like, breaks down those walls. And, I always make sure that whenever I'm talking to, anyone to to make sure that, like, I'm, you know, respectful and, building that trust so they can really get the value or understand, oh, you know, like, this team is here to help us. Right? And, like, we like, there's a lot of value in chatting with them. And so kind of using that chain reaction metaphor. Right? Like, having that conversation and then asking simply, like, anyone else we should talk to? And more often than not, if those you know, if you've been a good interviewer, if you've, you know, been a good human and, like, really welcoming of that person, that person's going to say, oh my gosh. You should talk to Bob. You should talk to Paul. You should talk to Gloria. Right? Like, it kinda just expands that way. And so, I think we've had to work a lot harder to reach across the aisle during this time. But like Kat said, because everyone's operating in this remote way, it has become the norm to a certain extent. And, you know, any way you can kind of stand out when you're talking to the person, the better. So for me, like I said, it's like joking around and, like, that humor of, like, bringing our personal selves into this work a little bit more, and not not saying like, oh, this is just professional me. Like, I can be both in this space and still, do really good work.
Speaker 0
49:12 – 49:34
As both of you kind of think back on your time, doing this sort of work, are there any anecdotes that stick out to you that you might be willing to share, whether it's something that, like, taught you a lesson about the process, something you thought was, like, particularly successful, or or maybe the opposite, something that didn't go so well that folks might wanna watch out for. Dovah, did you have anything like that you'd like to, like to share with folks?
Speaker 1
49:35 – 51:55
I'm not sure if this is necessarily an anecdote, but, you know, I've been on a handful of sprints, and those sprints have kind of had very, very different outcomes. So, you know, one of the sprints I was on, we had identified this huge chunk of an issue, for the agency we worked with. And at the end of the day, that agency was really only interested in doing, like, a small piece of that work. And, you know, that was, to us, like, a really great opportunity to start engaging with that agency more. And so, you know, sometimes folks might think, oh, we're gonna do this discovery sprint, and it's gonna be this huge big outcome. But like Kat said, it's a relationship with the agency and with the stakeholder. And so sometimes it might just be a smaller piece. And sometimes, you know, it might not be a piece at all. Right? Like, you hand the report in, and it gets put on someone's desk somewhere. And, and that's totally fine. We've had, we've also had instances where, you know, a team did a discovery sprint, two years ago, right, at an agency, and then that agency stumbles across this sprint report two years later and is like, hey. There there's some pretty good points here. Should we reach out to US Digital Service and get them to chat with us, and, like, start an engagement with them? And so I think my anecdote, or, like, my kind of lesson learned through this experience has always been, we're really planting the seeds. Right? We never know what environment we're going to step into. And so for me, discovery sprints are a really good way to help an agency understand and define their problem. And, and then whatever comes out of that is is really sort of just dependent on that environment. And so sometimes it might be the best time to undergo that. Sometimes it's going to be a small piece of it. And other times it might be two years later, but it's still work that's worth doing, and work that's worth exploring.
Speaker 2
51:56 – 57:55
Yeah. I think when we were having our pre conversation for this, you had asked Ryan for some examples. I think it's hard for us to give examples because that work belongs to the agency, and it's, you know, sort of proprietary for them, and as well as, like, whatever they decide to do or not do, given the outcomes of of any particular report. But I've been thinking about it since you asked, and I think I can tell a little bit of a of a story about a sprint that I, personally was part of where we were invited into an agency that was having, the the problem that they had identified. And this is something I think that we didn't really touch on very much, before now in this conversation. But a lot of times, a problem has been identified that we are asked to come in and look at, and it's that's not always the root cause. And so as a sprint team, it's a little bit of a delicate dance because, like, you want you want to look deeply into whatever it is that you have been asked to go and look at. But if you discover other things, you need to have enough space and enough trust with the stakeholders to be able to say, you can solve this thing, but you might want to spend some time or effort solving this other thing first because you may continue to have this this issue, or this this issue isn't is being caused by something else that is a little bit left field. And so just to to kind of give an example of that, I worked for Discovery Sprint where we, were talking to an agency that had call centers that were overwhelmed. And so lots of people calling in, the call volume super high, people are waiting on hold for a long time. And this is a an agency that was providing benefits that folks were entitled to, but that could be kind of confusing to navigate. And so they said, you know, our call centers are incredibly busy. We're trying to figure out, like, what to do about this. And when we went in and talked to the call centers, the call centers were very busy. We got to sit down on some calls, and we got to talk with the operators, and we got to talk with the stakeholders, and we got to talk with the IT folks, and and everybody who was doing their absolute darnest to make sure that the that the system was, working as well as it could be. But come to find out that the reason that a lot of people were calling in and waiting on hold and the call centers were overwhelmed is that they had attempted to do whatever they were doing, by going to the website first. And in order to make the changes that they and use the website as a service as opposed to calling in and talking to someone, they they needed to input a username and a password. And if you forgot your password I I may be misremembering this, but if you forgot your password, there wasn't an easy way to change it, and so that's why people were calling. So you have lots of people in, you know, calling this call center to get services, and then you have a whole lot of other people who are calling in because they can't change their password themselves on the website. And this was something that people didn't do on a regular basis, and they needed but the website required them to be changing their password every ninety days. And so this was sort of, like, the self fulfilling problem. Right? It didn't matter how many people were calling. There were always gonna be people waiting on hold because it was the the people who were responsible for solving the problem on the on the website and the people who were solving, you know, their set of problems on the call center side reported to different silos, and it was, like, different infrastructure. And so we the website folks weren't in the purview of the scope of what we were there to look at, but we had, you know, the immense privilege and ability to be able to go to, you know, several layers higher when we were delivering our report to say, you know, what you really need is probably a program manager that's looking at all the different ways that you are providing service to these folks. And, you know, our advice in our report is you put one person in charge who has both the responsibility and also the authority to kind of look at these different pieces as a whole instead of looking at them at separate parts. And, oftentimes, you know, everybody's doing their best to solve their issues, but there is there's systemic things that are connected that are hard to see if you're in the throes of, you know, managing a call center that's overwhelmed all day, every day. And so a lot of times, the outcomes that we come up with are just bringing a different perspective. Right? They're they're not it's not, it's not that we are bringing anything different to the table with the exception of a different perspective. And so I think that when we're talking about how other audiences or how other groups could adopt doing discovery sprints, like, could you do your own discovery sprint for your own team? And I think the answer is yes. I think it's useful when you can have a third party and a fresh set of eyes. But if you can't do that, I think sometimes just taking that time to look at a problem from end to end or to look at what the root causes of of some of the things that you're struggling with are, you can get pretty far that way. And so I don't I don't want anybody to feel discouraged about trying this themselves if they're listening to this and trying to figure out, like, how do I take what is here and do something to help my own environment learn or be able to run a discovery sprint?
Speaker 0
57:56 – 58:10
A tradition we have on civic tech chat is that we like to leave some space at the end for our guests to give us an idea of what we they'd like us to leave the conversation thinking about. So for each of you in this conversation we've had today, what would those concluding thoughts be?
Speaker 2
58:11 – 59:42
So when we first started thinking about how to reach a broader audience with writing this discovery guide, we were really unsure how people were going to be able to use it. And so we did as much as we could to write for as broad an audience as possible, But we left room for other voices who come after us and for things that, for other things that people learn. Like, we know we've just barely started to scratch the surface. But what we did was lay some groundwork for other people. And so if folks are out there using the guide to do discovery sprints or if they are doing discovery sprints of their own, we do have a section in the guide for case studies. As we mentioned a couple minutes ago, sometimes it's hard for USDS to share some of our case studies, but we do have a section there for people to be able to contribute and, to be part of a larger conversation. And I think Jen and I would love to see and hear about what other people have done or tried. And so this is this is something that we wrote, to kind of give back to, you know, our organization and for folks who might be, discovering this stuff later. But I think that this there are always learnings happening, and there are always people who are coming after us. And there's like, the stuff this work is cyclical. So we'd love to encourage folks to contribute and, to tell us their stories as well.
Speaker 1
59:43 – 61:32
Yeah. I definitely echo Kat there with sharing, you know, your own stories of your experiences with Discovery Sprints. Documentation is so important, and we really see this as a living document. The other thing I would add is I think when we talk about what it means to understand the problem, sometimes we find that teams might be a little bit narrowed in their scope. And so something that that's really important to keep in mind is that, you know, to look at the problem from all different perspectives. Right? And one of the things that we really try to explain in the discovery sprint guide is who should be who you should talk to, right, and who you should consider in this space. And so, looking at it from, like, a systems level perspective, right, it's the people who are supporting the work, the people at the call center, leadership of the agency, the people who are at the receiving end. Sometimes their family members. Right? Like, we know that a handful of times, if it's someone who's elderly or someone whose language isn't first, first language isn't English. Right? It might be it might the the service might have to fall onto a family member or a neighbor. So to me, I I really just wanna leave with, like, this work is so important, but but it's so important just to understand everyone that, we're doing this for and to not forget that as we, as you kind of undertake this method and process.
Speaker 0
61:34 – 61:47
Jen, Kat, I wanna I wanna thank you both for taking the time out of your day to come and be on Civic Tech Chat. I I have no doubt folks are gonna find some nuggets of wisdom to take into their own days as they listen to this episode later.
Speaker 1
61:47 – 61:49
Thanks so much for having us, Ryan.
Speaker 2
61:49 – 61:57
Thank you. It was great to be here and and to get to talk to Jen, who I don't get to see as much anymore, and also to just to talk about this project.
Speaker 0
61:58 – 62:10
You can follow us on Twitter using the handle at civic tech chat. Visit us on the web at civictech.chat, or subscribe to us for content updates wherever it is you download your podcasts.