Hello. I'm Ryan Cook, and this is Civic Tech Chat, a podcast about the civic technology movement. We seek to harness the power technology has to improve the delivery of public services to people everywhere. Kat, Jen, thank you both so much for coming on to Civic Tech Chat. Could you both, introduce yourselves and tell us a bit about what you do?

Thanks so much, Ryan. It's great to be here with you and Jen today. My background is, primarily as a design researcher. I've been working in was working in private sector for more than fifteen years in places like Silicon Valley, in Seattle, in Boston, and, working at big companies, little companies, hardware, software, all kinds of different things. It's been, kind of a mixed bag for me, but it has meant that I've gotten to do a lot of different kinds of things in my career, and that's always really fun to get to talk about. Most recently, I've been at the United States Digital Service, which is how Jen and I know each other. I started out in The at the very 2016. I joined and was originally an an individual contributor as a design researcher on the Defense Digital Service team. After that, I had the opportunity to step into a leadership position at USDS HQ as design director, and I did that for about a year and a half. And then during that time, was responsible for doing a lot of staffing for our discovery sprints, which is one of the reasons that brought Jen and I to the conversations we started having that led into the work that we're gonna talk with you about today. Most recently, I am now in the process of heading back over to the defense department as director of design operations for a group inside, Space Force, which is another really exciting opportunity because they are setting up It's the first new military service in over seventy years, and so they're setting up a lot of they have the opportunity to set up a lot of technical infrastructure differently than the way that the DOD has been doing things. And so I've been really enjoying that so far, but it's it's a very new thing for me right now, but it's super exciting.

For each of you, what would you say is your personal why? The thing that drives you to get out of bed each morning and do what you do.

Sure. Yeah. I think what has been really wonderful about finding a path into working inside civic tech is that there's such a strong sense of mission, both for my personal self and the work that I'm doing and what drives me as well as the people that I'm working with. And I think that, you know, getting up every day and just looking for opportunities to make things, systems, technology experiences better for folks who, are dependent on government systems or on government agencies to really give them the support that they need is something that it's it feels really simplistic, but it's the thing that really helps me feel like I'm I'm doing something and I'm accomplishing something at the end of every day.

Each of you has spent time both in the private and public sectors, which may differ in some interesting ways. As you think back on those experiences, what are some things that have stood out to you or surprised you?

Yeah. I would echo a lot of what Jen said about scale. The difference, you know, in working at technology companies, even ones that are I mean, a lot of technology companies stays in, say, in startup mode for a long time. And so you're working to your everything's around a million. Right? Like, you're working towards your first million dollars. You're working for your to your first million customers. You're working towards you're building on top of a base, that starts pretty much at the at the zero scale. In the government, it's almost entirely opposite. And so you have that that play of scale where you're on day one, on anything, really, you're looking at hundreds of millions of dollars involved. You're looking at hundreds of millions of people who are affected by these systems. Your customer base is different. I'm using air quotes for that. In private sector, you're looking for customers who are going to see value in what you are offering and pay for the opportunity to use your service or your product or whatever it is. In government, a lot of times, it's I don't wanna say it's a last resort, but oftentimes, whatever the solution that is being provided is you are the only way that somebody can do that. Right? Like, there's you can't file your taxes with some somebody that isn't the IRS. Right? Like, you have no choice but to interact with that system as a taxpayer. And so it's not only just a scale, but the idea of how you can meet the mission with so many different kinds of users and people who are coming from all different levels of technology experience. You can't really assume anything about who those who those users are going to be. And as a design researcher, that's really fun. Right? Because it means that you get to go out and interact with lots of people. And even if you're making incremental changes for the better, you still get to see the impact that you're having. I think that's a little more challenging to do on the private sector side. And I think the other thing that was I think you asked what was surprising. For me, what's been really surprising is how, design curious a lot of the federal government is. I mean, a lot of time we'll be talking a lot about federal government today. I think this is true for other aspects of government and other work that is happening in civic tech, but there it's such a greenfield for design and and for human centered thinking. And everyone that I've had the pleasure of interacting with over the last four years is just super curious about how can we design better for the humans who are interacting with the systems that we're creating. Sometimes that's the systems you know, it's somebody who's sitting in front of a keyboard who is then providing services to somebody who's sitting on the other side of that counter, and all of those people are humans that, you know, human centered design needs to be applied for when you're working at the federal level. And that's that was a surprise to me. It's just sort of how curious and how excited people were to to talk about this stuff and to give it space in a way that in the private sector, I think a lot of times design fights for a seat at the table with engineering and other disciplines that are often a little bit more transparent about the impact that they are having for on behalf of a company.

I think this is a good time for us to take a shift over to our our main topic, which is something that I'm I'm pretty excited to dig in with with each of you. Folks listening might not know that each of you were involved in the creation of a discovery sprint guide that's hosted on The United States Digital Service, USDS on their website. For folks that might not be familiar with what a discovery sprint is already, or maybe even a sprint as a concept, could you talk us through what this is?

Yeah. I think, I just pivot off a couple of things that you said, Jen, that the United States Digital Service is lucky enough to be funded with money that means that they can provide folks or resources to any project across the federal government. So we have, a couple of different partner agencies, 18 f being the one that the people on this podcast will probably be most likely familiar with. 18 f is is fee for service, so it's essentially like hiring a consultancy. So if an agency wants to if an agency knows that they have something that they need work for, they can essentially hire they can hire out. So they can go through procurement and hire through contracting, or they can go to a group like AT and f and pay for the services that AT and F provides. For USDS, when we originally got stood up, we were doing a lot of emergency responses. And so, you know, something would fall down in the middle of the night, and we would send people to kind of go and figure out what that problem was. And there's a lot of kind of duct taping a bunch of servers back together, and that often was our first foray into understanding the needs of a particular agency where the things that were happening underneath the cover of darkness that were that were often the root causes of issues that were going wrong. And so that's kinda how we started is that we would go out. We would solve a complete we would solve a quick technical problem. And then when we were talking with the stakeholders, we would get invited to kind of stay a little bit longer. And Discovery Sprint for us is a really great way to kind of do a quick investigation and just kinda see, like, does this seem like a kind of thing that USDS could actually help with? Right? Like, do we have the right skill sets on our in our group to be the people that are the right people to help them? And, oftentimes, I would say 99% of the time, what one of the things the deliverable that we produce beyond the act of going and go doing a discovery sprint is a sprint report. And that sprint report lives with the agency. It belongs to them, and it has a series of recommendations and observations that we have made. And oftentimes, that goes to their senior stakeholders. And then if work continues after that, either through USDS or through any other way, that report is is kind of a a a support system for kind of setting up the direct the right direction to head. USGS isn't always the right people to to try and and resolve those. Sometimes we would say, look. You guys are doing exactly all the right things. Just keep doing them. And here's our third party report that says that you're doing all those things, you know, great. Sometimes what they would need is is not the kind of support that the USDS could have good resources for, in which case we could say, this is what we recommend that you go and do. And then the third case is when, it is a good match, in which case we start talking about, like, having longer term engagement with that organization, to try and bring them different resources or a different perspective or the expertise that USDS has into that agency.

For each of you, what do you think is the the why for for putting together a guide like this?

Yeah. I would echo a lot of what Jen said. I think in the early so USDS is about, seven years old right now. And in the early days of USDS, they did a lot of great documentation. So they wrote down what they were doing, why they were doing it, how they were doing it. And the landscape changes at USDS a lot because people come in on tours of duty, and so it's not the same people, who are staying. And so how do you pass on the information that you're learning and the things that you are doing to the next generation of people who come after you? A lot of times right now, especially I think it this is true this is definitely true for USDS. I think it's true for a lot of civic tech environments that a lot of that is just oral tradition. Right? That you are on a sprint you're on a discovery sprint with somebody else who's done a couple of discovery sprints before, and so you learn through the expertise of, you know, somebody who's done that before you. You get staffed on an on an agency team, and and you learn about how that agency works a lot from the people that are that you're partnered with on your team itself. And then over time, you build up a set of expertise that essentially walks away with you when you leave. And so, you know, we knew that there was a need for us to have internal documentation. And I think Jen and I both, like, felt like we had done good work while we were at USDS, but this idea of, like, wanting to leave something behind for our other US future USDSers. And so that was that was a piece of it. I think the other piece is that we when we talked to other civic tech groups, every civic tech group is is struggling with a lot of the same core things, right, and trying to figure out how to how to do what they're doing in spaces that that often haven't in had teams like this or had people that, have these different types of expertises. And so especially when I was design director, we would talk to other organizations or leadership at other, you know, at civic tech organizations or digital services all over the world. And everybody was like, what what are you guys doing? How did you solve this problem? Like, how are you thinking about this? Like, what is how are you, you know, what are you doing, and how are you doing it? And we were all doing that with each other all the time. And I think this is sort of our, our small gift back to that, that discipline or that group of folks who are are trying to solve some of those same things or tackling some of those same issues.

When we were having our, preparatory chat, that like, that prior call, one of the things that came up is this notion that discovery should be thought of as a method, as a deliverable, and also as a process. Could we dig into and expand upon that that notion a bit for folks listening?

What can be really nice about discovery sprints, and I think this is one of those things that it makes what we makes this this guide that we have written more applicable or adaptable for other organizations out, like, beyond just people who are working at the federal level or beyond, folks who are even working just in civic tech. I mean, I think the idea of a design sprint or sprint zero has been around for a really long time. In organizations that work inside agile, the idea of a sprint is is pretty common. There are places inside the federal government that are doing waterfall or doing other kinds of development methods. And so the idea of a of a sprint might be, a little bit new to them. But one of the nice things about it being so quick in terms of an as an experience is that it's pretty easy to get a yes to, to try out doing a discovery sprint. And so as a deliverable, you know, I think and, Jen, you touched on this too that, like, most of the folks who are working inside these federal agencies, they are the subject matter expert in their agency. They are the subject matter expert in their, in their end users or in the people who are on the receiving end of the services that are being provided. They know all of those things cold. And two zero one, they've probably had some group of people come in every couple of years that say, we're gonna help you do things better. We're gonna help you, you know, change it up, or we're gonna teach you a different method to do things. And they are incredibly patient, and are often willing to, engage and to to try to adopt these methods to see if they can result in a a different approach or a a different kind of solution that may not have been available to them prior. As a deliverable, what's nice about having an external third party come in, it's not so much about the expertise that we bring as it is just the opportunity to shake things up a little bit and to give the folks inside different organizations, or different groups a voice. And so, you know, as I said, they're the experts. They know this stuff cold. They often have maybe been filing tickets or they have been, you know, writing or talking to the folks above them for a long time about whatever symptoms that they are seeing that might be causing larger issues. And for us to come in and just to get people into a room together and get folks, explaining things to us as a third party can sometimes just help open up levels of communication or opportunities that have been there the whole time. But this is just the catalyst to to make that happen. And so I think, like, as a deliverable, it's the communication piece that supports and encourages and just gives the opportunity for that to happen in a way, that is really useful. And, you know, when it's done quickly, it can still instill a lot of opportunities for that to continue to happen long after we're not there anymore.

One section of the report that kinda stood out to me when I when I was reading through it is the one that's titled how sprints fail. Seems like this gives some sense of the things the team might seek to watch out for as they're going through their their own sprint. And maybe often, like, kind of knowing, like, what things to watch out for is just as important as knowing, like, what are the good things to do.

Are there any potential problems from that that kinda stick out to each of you that you'd like to talk about? I mean, I think what is what's interesting I mean, Jen got to participate in a couple of different sprints, both within the agency that she was working at and then as, part of cross agency work. I got to see and staff a lot of different sprints, during the time that I was at USDS. And there are definitely patterns that repeat themselves, and so this was that section was an was an effort for us to kind of pre identify those things. I think what's crazy about a discovery sprint is that you're going in to discover. You don't always know what you're gonna find, and you don't always know, what is going to be needed in terms of different skill sets. And you don't always know what, you know, what the environment is gonna be like. And so there's there are some things that you, as the creator of a sprint team, can control, and there's a whole lot of stuff you can't. And because it's discovery, it's different every time. And so figuring out which things we could kind of distill down into good advice to give people was really challenging because we wanted to give enough information for people to make to make it useful if they wanted to try using some of these methods. But we didn't wanna be so prescriptive that if you ran into something that wasn't on that list, that you wouldn't know what to do. And so I think that from a when in house prints fail, we do see patterns. And I think that's something that we tried to raise up and, you know, give some some hints as as to the kind of direction that those could be.

I think I'd add one more to that just as that with that scope and the time, when you when you are uncovering issues that you it's really easy to jump into solutioning. Right? It's really easy when you're starting to talk about people and you're like, oh, I know I know a good answer for this, or I know how they could solve this problem, or I've seen this before at another agency, and, like, I want I want to to I want to help solve this problem that, you know, folks are facing, staying in that our point is discovery. Our point is to uncover these things. Our point is to write them down. Our point is to start the conversation with within that agency about which things, are going to be their priorities, and that's completely up to them. When you start driving towards solutions too quickly as or at all as part of a discovery sprint, you tend to get really, boxed in, and those things tend to to become larger than they might be in the whole ecosystem of just looking at all the different pieces that are getting put together because they are familiar or because you you know what the answer might be for that particular one. And so I think that that's another one that when we've seen when we've seen teams struggle, it's because they are they're trying to jump too quickly into to solving some of the things that they're uncovering.

Related a bit to that, and as far as, like, if you're taking a team who's done a sprint and you're looking back, they might go through an exercise called a retrospective, where they kinda go and try to evaluate what happened during their discovery sprint. I imagine the way one goes about facilitating that and and how that's handled is is really important. If there's someone out there listening and they're going through this and they're kinda looking for advice on how to go about that, what what tips might y'all give someone like that?

I think I definitely have a couple of tips off the top of the bat. Don't don't skimp on doing a retrospective, Especially if you've put together a team that hasn't worked together with, each other very like, if you've brought people from different orientations to work together, it's really useful to have a retrospective because at you're getting a sense of closure, and you're sort of giving everybody the space to mark what they did, to talk about it, and then to go back to whatever it was that they were doing before. So that part is really important. I think from a facilitation perspective, get a third party to come and facilitate the retrospective so that everybody who participated in the sprint can talk. It's really easy to say, oh, well, the sprint lead was leading, so the sprint lead can lead the facilitation. But that leaves out the, you know, incredibly robust feedback that the sprint lead might have to give. And so, those are sort of the the quick hits that I would give. When I have run sprint retrospectives in the past, I usually do 90 to ninety minutes to a hundred and twenty, which seems like a really long time. But taking the first forty five minutes to an hour, and I just ask the team to tell me the story of the sprint. Right? What happened? What did you do? And then what did you do? And then what happened? And who did you talk to at this stage? And I will whiteboard the story that they're telling me, in bullet points. And that's really great because there's no emotion involved. So if this was particularly stressful or if there was, if there were differing opinions on the team about what happened when or or, you know, how something was handled, you take all of that out. And so if people are just telling you the story, what happened, and then you take a break, and then you come back. And for the second piece, it's sort of talking through why why those things why do we think those things happened and taking time to say, okay. Well, like, this like, looking back, you can say, oh, this was a this was a turning point. This was a pivotal turning point in the sprint. We didn't know it then because we didn't realize it because we we're standing at a fork in the road. Now we can look back and we know that. And, again, it takes some of the emotion out of the, you know, who said what when, and it puts people into a more positive perspective of, like, it happened the way that it happened, and so now we're reflecting back. And then I think the the last half an hour or so, I always like to be, you know, what advice would you give for the next team that is doing a sprint? So, actually, two things. One, at USDS, what advice would you give if USDS continues to do work at this agency? What are the what would what things do people really need to know? And then the second piece is, like, what is it that you learned about doing this sprint that is useful for other people? And so I had photographs, lots of photographs of whiteboards, useful for other people? And so I had photographs, lots of photographs of whiteboards, from having done these retrospectives when Jen and I started talking about writing, this guide, and a lot of that feedback fed into, what ultimately became this discovery guide.

So whereas, like, our retrospective, you know, we're we're kinda looking at that that's kind of like a norming activity that you sort of do at the end of an activity, and and then maybe you end up kind of going on to another. But one thing I've picked up, through some of y'all's answers is that oftentimes or maybe not often, but sometimes these teams are kind of new teams that are brought together. Maybe it's five people who haven't done a lot of work together previously. And in those situations, are there any kind of norming activities that y'all have experienced that, like, really kinda help that initial hump you have to get over in order to, like, work well cohesively?

Yeah. I think what the one thing that I would add that I've seen when teams really gel, I think one of the things and it's, you know, hard to say this at month whatever we are in the midst of a global pandemic, is oftentimes, you know, one of USDS's core values is go where the work is. And a lot of times, these agencies a lot of agencies are located in DC, which is where the the core of USDS is. But a lot of times, sprints involve going out on the road to talk to external stakeholders, of various types and kinds. And so the act of traveling together with a team is a real uniting force. Right? Like, the even just figuring out who's going where, when, and and the opportunity to have dinner together at the end of the day after a long day of of user interviews, or stakeholder interviews is something that can really pull a team together. And sometimes when teams don't travel, it's it become it's a little harder to kind of create that that space for a team to be all kind of all in the same way that you are if you're working out of a hotel lobby together, starting to write the beginning of the the report. And so that's it's something to maybe aspire to even though that's it's not something we can actually do right now as much.

Nekat, that is a a great segue to the the very next question I had. Your mention of that principle of going where the work is. You know, as you mentioned at the end of your answer there, it's it's a little it's a little harder to do that now. You know, we're kind of in a position where for folks' safety, we kinda have to work at a distance, do things over distributed technologies. As you've kind of, like, seen folks trying to kinda still get to that same goal with these different tools, how have those folks been kinda, like, working around the pandemic for for lack of a better way to put it?

You know, when we first started writing the guide, we didn't have a remote section, because it was sort of pre or very beginning stages of the pandemic. And so at a certain point, we were like, oh, we really if we're gonna make this useful for people who are living in the now, then we can't talk about it as sort of aspirationally. We need to talk about the way that that that things really are. I think one of the things that has been surprisingly smooth from a running discovery sprints at the federal level is that so many of our federal partners are also doing telework right now or working remotely. And so when a situation when everyone is remote, it's a different kind of experience because you're all having you know, it's not it's not great. It's not ideal. You miss a lot of the surrounding environment of what you would see when you were going to somebody's, you know, place of work and interacting with them. You know, as a researcher, you always wanna do that. Right? Because you get to see, like, what are the posters that you have on the wall, and what are what does this post it mean, and, you know, why do you have three monitors, and, you know, kind of talk me through what it looks like to be you at your desk. You miss some of those things. But during this past year, I think because everybody has been remote, it's just sort of we're all in the same boat. And that has enabled us to sort of get into the core of what people's paths or roles or issues or the things that they're dealing with because the the things that we are looking to discover all still exist even in a remote space.

As both of you kind of think back on your time, doing this sort of work, are there any anecdotes that stick out to you that you might be willing to share? Whether it's something that, like, taught you a lesson about the process, something you thought was, like, particularly successful, or or maybe the opposite, something that didn't go so well that folks might wanna watch out for. Dua, do you have anything like that you'd like to, like to share with folks?

Yeah. I think when we were having our pre conversation for this, you had asked Ryan for some examples. I think it's hard for us to give examples because that work belongs to the agency, and it's, you know, sort of proprietary for them, and as well as, like, whatever they decide to do or not do, given the outcomes of of any particular report. But I've been thinking about it since you asked, and I think I can tell a little bit of a of a story about a sprint that I, personally was part of where we were invited into an agency that was having the the problem that they had identified. And this is something I think that we didn't really touch on very much, before now in this conversation. But a lot of times, a problem has been identified that we are asked to come in and look at. And it's that's not always the root cause. And so as a sprint team, it's a little bit of a delicate dance because, like, you want you want to look deeply into whatever it is that you have been asked to go and look at. But if you discover other things, you need to have enough space and enough trust with the stakeholders to be able to say, you can solve this thing, but you might want to spend some time or effort solving this other thing first because you may continue to have this this issue, or this this issue isn't, is being caused by something else that is a little bit left field. And so just to to kind of give an example of that, I worked on Discovery Sprint where we, were talking to an agency that had call centers that were overwhelmed. And so lots of people calling in, the call volume super high, people are waiting on hold for a long time. And this is a an agency that was providing benefits that folks were entitled to, but that could be kind of confusing to navigate. And so they said, you know, our call centers are incredibly busy. We're trying to figure out, like, what to do about this. And when we went in and talked to the call centers, the call centers were very busy. We got to sit down on some calls, and we got to talk with the operators, and we got to talk with the stakeholders, and we got to talk with the IT folks, and and everybody who was doing their absolute darnest to make sure that the that the system was, working as well as it could be. But come to find out that the reason that a lot of people were calling in and waiting on hold and the call centers were overwhelmed is that they had attempted to do whatever they were doing, by going to the website first. And in order to make the changes that they and use the website as a service as opposed to calling in and talking to someone, they they needed to input a username and a password. And if you forgot your password I I may be misremembering this, but if you forgot your password, there wasn't an easy way to change it, and so that's why people were calling. So you have lots of people in you know, calling this call center to get services, and then you have a whole lot of other people who are calling in because they can't change their password themselves on the website. And this was something that people didn't do on a regular basis, and they needed but the website required them to be changing their password every ninety days. And so this was sort of, like, the self fulfilling problem. Right? It didn't matter how many people were calling. There were always gonna be people waiting on hold because it was the the people who were responsible for solving the problem on the on the website and the people who were solving, you know, their set of problems on the call center side reported to different silos, and it was, like, different infrastructure. And so we the website folks weren't in the purview of the scope of what we were there to look at, but we had, you know, the immense privilege and ability to be able to go to, you know, several layers higher when we were delivering our report to say, you know, what you really need is probably a program manager that's looking at all the different ways that you are providing service to these folks. And, you know, our advice in our report is you put one person in charge who has both the responsibility and also the authority to kind of look at these different pieces as a whole instead of looking at them at separate parts. And, oftentimes, you know, everybody's doing their best to solve their issues, but there is there's systemic things that are connected that are hard to see if you're in the throes of, you know, managing a call center that's overwhelmed all day, every day. And so a lot of times, the outcomes that we come up with are just bringing a different perspective. Right? They're they're not it's not, it's not that we are bringing anything different to the table with the exception of a different perspective. And so I think that when we're talking about how other audiences or how other groups could adopt doing discovery sprints, like, could you do your own discovery sprint for your own team? And I think the answer is yes. I think it's useful when you can have a third party and a fresh set of eyes. But if you can't do that, I think sometimes just taking that time to look at a problem from end to end or to look at what the root causes of of some of the things that you're struggling with are, you can get pretty far that way. And so I don't I don't want anybody to feel discouraged about trying this themselves if they're listening to this and trying to figure out, like, how do I take what is here and do something to help my own environment learn or be able to run a discovery sprint?

A tradition we have on Civic Tech Chat is that we like to leave some space at the end for our guests to give us an idea of what we they'd like us to leave the conversation thinking about. So for each of you in this conversation we've had today, what would those concluding thoughts be?

So when we first started thinking about how to reach a broader audience with writing this discovery guide, we were really unsure how people were going to be able to use it. And so we did as much as we could to write for as broad an audience as possible. But we left room for other voices who come after us and for things that, for other things that people learn. Like, we know that we've just barely started to scratch the surface, but what we did was lay some groundwork for other people. And so if folks are out there using the guide to do discovery sprints or if they are doing discovery sprints of their own, we do have a section in the guide for case studies. As we mentioned a couple minutes ago, sometimes it's hard for USDS to share some of our case studies, but we do have a section there for people to be able to contribute and, to be part of a larger conversation. And I think Jen and I would love to see, and hear about what other people have done or tried. And so this is this is something that we wrote, to kind of give back to, you know, our organization and for folks who might be, discovering this stuff later. But I think that this there are always learnings happening, and there are always people who are coming after us. And there's like, the stuff this work is cyclical. So we'd love to encourage folks to contribute and, to tell us their stories as well.

Jen, Kat, I wanna I wanna thank you both for taking the time out of your day to come and be on Civic Tech Chat. I I have no doubt folks are gonna find some nuggets of wisdom to take into their own days as they listen to this episode later.

Thank you. It was great to be here and and to get to talk to Jen, who I don't get to see as much anymore, and also to just to talk about this project.

You can follow us on Twitter using the handle at civic tech chat. Visit us on the web at civictech.chat, or subscribe to us for content updates wherever it is you download your podcasts.