Legacy Lockpicks
We're joined by David D'Silva, Principal Product Manager at Nava, for a conversation about legacy lockpicks: the reusable tools, playbooks, and patterns that help government teams move out of legacy systems faster and with less risk. We talk about why trust is the real currency of modernization work, why so many efforts stall at the data layer, and how to choose between patterns like strangler fig and lift and shift. We also dig into where AI is genuinely useful in modernization today and where it's mostly hype.
- David’s LinkedIn: https://www.linkedin.com/in/daviddsilva/
- Prior episode on Strata: https://civictech.chat/episodes/nava-strata-open-source-government-technology
- Strata page: https://www.navapbc.com/strata
- Legacy modernization page: https://www.navapbc.com/legacy-modernization
- Our prior episode covering the strangler fig pattern: https://civictech.chat/episodes/the-specops-method
Top Keywords
- modernization 0.013
- legacy 0.009
- work 0.008
- around 0.007
- strangler 0.007
- teams 0.006
- system 0.006
- nava 0.006
- data 0.006
- legacy system 0.005
- talked 0.005
- time 0.005
Transcript
Hello. I'm Ryan Cook, and this is Civic Tech Chat, a show that looks at the way technology, politics, and policy impacts the world around us. The tools we use, the way services are delivered, and how we talk about and set policy all shape our society. We'll gather around and have a chat about these things together and more. This episode is sponsored by NAVA Public Benefit Corporation. They're a mission driven organization in the government technology and public interest technology spaces. I'm grateful and excited they were able to support this conversation about legacy modernization, where we'll cover some tools and techniques that many of you will find useful in your day. So with all that said, let's go ahead and enjoy the show. David, thank you so much for joining us here on Civic Tech Chat. Could you introduce yourself and tell us a bit about what you do?
Yeah. Absolutely. I'm David De Silva. I'm a principal product manager, at Nava. For those who don't know, Nava is a public benefit corporation whose mission is to help government agencies make services more simple and effective. Inside of Nava, I work in our technology solutions and services team with a particular focus around legacy modernization. So we think a lot about how to help teams leverage our learnings from projects,
and, you know, not have to so much reinvent the wheel every time that they start a legacy monetization. And what would you say is your personal why? That thing that drives you to get out of bed each morning and do what you do.
Yeah. For for me, it's really about improving the way that the public interacts with government. So, you know, we think about, obviously, a lot of cases around sort of benefits or receiving benefits, but there are a lot of interactions that, go beyond that, whether it's, you know, like health care or hospital quality reporting, which is like another project that a team works on. And so I think it's really interesting because those challenges work across a number of levels. You have technology, people, policy. And and, ultimately, for me, like, it's those interactions and the quality of those interactions that lead, I think, to, like, higher trust and confidence in government. And so that's, for me, really the the so what and and why I'm interested in this space.
That statement you made about the quality of the interaction, having a relationship to folks' trust in government, is interesting. It sounds like very personal thesis. What inspired you to kinda land there?
For me, it's it's, I think, a couple there's a couple there's a couple, like, personal reasons, and then there's some there's, like, a macro reason in there too. You know, on on the personal level, it's I a lot with, like, my own just interactions with government, right, as as a citizen, And I think of, like, family interactions with government. I was talking with my dad. This actually happened, like, a few weeks ago where, you know, he was, trying to get in touch with someone at Social Security for help with his benefits. Right? And he had a really positive interaction, and that led to a really much more positive sort of, like, thinking about that agency and the and the mission in the space. And so at that level and then I think about some of the macro things, like, almost like it's kind of like a silly example, but if you ever seen the movie, Zootopia, there's a moment there where they're, you know, the DMV, and and they represent that interaction with sloths. Right? And and it it it, I think, speaks to a really, like, specific kind of perception of government, right, in the public and which which is unfortunate and I and I think could be improved and and and create a a positive cycle there. So so, yes, so so some very personal things, but also just sort of seeing kind of, like, the the general cultural, like, discussion, right, around interacting with government. Folks that have been looking at the episode description already before they hear us might have already spotted this legacy lock picks term.
When we use that in our conversation today, what are we referring to?
Yeah. So, you know, Nava's definition of a lockpick in in this case is really, like, a reusable tool, a reusable playbook, a framework, a process. Basically, anything that's going to help a team transition, you know, out of a a legacy system either more quickly or or with less risk. You know, we kinda use that lock picks term somewhat intentionally here. So the, you know, the idea is not that you just wanna get rid necessarily of everything in that legacy system. There's a lot of a lot of value that's built up in those systems. But either maybe because the system is super brittle or there's or there's some sort of, like, vendor entanglement, it can be hard to access that. And so that's where that sort of, like, lockpick metaphor kinda comes in. And as I kinda touched on, like, it can be like, these can look and feel very different based on, like, the particular challenge that we're facing on on any particular project. Right? So it might be a tool that helps teams sort of compare the outputs of a legacy system to a new system. It might be a tool that, helps, you know, convert policy into requirements. The the point really is that we want to find the reusable assets, reusable ideas, right, that, again, help teams from having to reinvent the wheel whenever they're engaging these problems on a on a modernization.
The, part of my brain that, you know, comes from that software background, loves the concept of things that are reusable. Right? I think that that usually brings a lot of good to projects. What's an example of one of these lock picks that you've experienced and seen succeed in being reusable like that?
Yeah. That's that's a great question. I think so, you know, we talk about reusable here. Like, the example I'm gonna give is around, actually, like, change I'll I'll sort of cite change data capture and data replication as as a form of a lockpick. Now is it is it a is everything a 100% usable around that? No. Like, every, right, every time you use that that approach, right, you have to sort of have things around the edges that fit to that, you know, particular system that you're working with. But at the end of the day, like, why I kind of give that as an example is, like, when you've got two systems that you're trying to maintain in parallel, you have to keep those in sync. Right? And if you if you can't sort of figure that out, if you can't lock pick that data, then you're gonna be stuck. And so, I think that's a great example of a pattern or an approach that teams can reuse, that, you know, greatly derisks a lot of modernization efforts.
How does this concept of legacy lockpicks reflect on and fit in with the broader approach that you take to the work?
Yeah. So, you know, LockPix kind of really fits in a lot with the underlying sort of, like, Nava principles around open architecture. So the idea of, like, we want to, help agencies build systems out of modular components that are easy to evolve over time, right, and sort of avoid that kind of, like, slide back into lock in. So, that that's, I think, really connected to why we launched Strata last year as Nava. Right? Because those are those are those sort of predefined, standard open architecture components that agencies can kind of build, their systems with. And so for lock picks at the end of the day, this is about sort of connecting into delivery. Right? It's a way to take that kind of, like, high level strategy, right, and then kind of, like, push that into delivery, so that teams are able to kind of embed that on their day to day work.
You mentioned, Strata as as one of those kind of examples of an of an open architecture thing that y'all have latched on to for use. For folks that maybe haven't seen our prior episode, where we kinda do a little bit of a deep dive into what it is, what it's about, what are some things that Strata might get used for kind of like in in early days that would help accelerate a project?
Yeah. So I think a good example of that is is some of the work we actually did with, Minnesota PFML. So that was a a new system that got stood up recently, in the state of Minnesota, in our work with them. And so, the way that Strata kinda got involved with that is that, is was in two ways. So one is that there's sort of a base application inside of Strata that we are able to to leverage. And then two, there is a set of infrastructure templates, that allowed, you know, the the team to spin spin up and and deploy those deploy that new application, you know, in a matter of of weeks. So, in particular, like, that accelerated that early stage of just getting something into production so that we could then have that positive momentum. I think at the end of the day, something like, we estimated save something like seventeen weeks over the course of that. So, that I that I think is a a a pretty good example of where, you know, something like Strata can help accelerate the overall, you know, delivery motion.
And to get to that seventy weeks, what do you think is, like, the primary saver activity? Is it kinda, like, decisions that you don't have to make because they're like, a like, the thing is opinionated? Is it, the infrastructure itself, the setup that's easier?
Where do you kinda see the the most value coming out of that? It you know, I I think it's there's probably not any, like, single layer where that comes in and and delivers value to the team. I think certainly, you know, it saves, in in my opinion, it saves a lot of kind of, like, the repeat effort. Right? So, in terms of, like, deploying infrastructure, right, we can have that ready to go. You can have that infrastructure as code, and then it you know, it's about sort of, like, clicking a button, as opposed to kind of rewriting that from scratch every time. I think similar with at the component level then too for the actual application. Right? So you so, we've got ways to make it easier to build those forms, as an example there. Again, just so teams don't you know, keep teams can start at sort of, like, the 80% mark as opposed having to start at the zero, you know, at the at the complete, like, blank blank page or blank screen.
Often, technology is not the most significant bottleneck for this kind of work. There's organizational challenges to surmount, like change management, the need to shift policies and practices, gaining trust and credibility to push work ahead. Of these threads, which one do you find yourself spending the most energy on? What's an approach to address that that's worked for you?
So for me, it all comes down to trust, especially in in legacy modernization. So if if you don't have that trust, I really think it's only a matter of time until, you know, the politics of the organization catch up with you or, you know, the resources that were initially sort of dedicated to your efforts start to get pulled, you know, off to other engagements or get pulled onto the shiny new object. Right? And so that that trust factor, I think, also really works across, like, all of the layers that you just mentioned that are part of these holistic modernization efforts. And so for us, a lot of our focus is therefore then on, like, how do we really help teams get off to faster starts? If you think about, you know, the start of start of a big project. Right? There's gonna be a lot of there's gonna be a lot of goodwill. Usually, there's gonna be a lot of excitement. There's gonna be a lot of energy. And so it becomes really important to convert all of that into forward momentum, so you can start to demonstrate the value that that team is going to be able to to deliver. And so one like, clicking clicking on this, like, a little bit further down, you know, one of the areas that we're, spending a lot of time exploring is is, like, how do we compress the amount of time that teams spend in this discovery phase or, like, often referred to as, like, archeology phase. Right? So by by compressing that time because because we still have to do those activities. Right? We still have to understand what's happening in the current system. But if we can compress the amount of time that teams spend doing that archaeology, we can really allow them to focus on much higher leverage, higher value work. And so that's, I think, a key way to kind of build that that trust that we're talking about. The other the other layer that we think about is around and this this starts to get a little bit more into, like, back into the technology layer, but I think it's important, kinda worth calling out, but around the the data layer itself. So, talked to a few folks in the last couple weeks who sort of cited examples of modernization efforts that completely stalled and died because, not because they couldn't figure out the logic, not because they couldn't, you know, rewrite the application, but because ultimately they couldn't migrate the data out of their legacy system. And so they just sort of, you know, ended up wasting, like, a couple of years of effort. So for us, again, it's a lot about thinking about, like, okay. What is what is the existing shape of that legacy data? Like, what do you have today? And then how do you understand kind of, like, what that future state model is and how do you how do you connect those two? When we talk about patterns like Strangler Fig two and, you know, I mentioned a second ago about sort of, like, running systems in parallel and being able to sort of derisk efforts by doing that, That also becomes, like, really dependent on, like, being able to manage that and unlock that data layer. And so, you know, for those reasons too, I I I see that as, like, a really kind of, like, key aspect of a successful modernization effort.
On the the first layer of your answer, I heard you talking a lot about trying to compress time from, like, a certain level of value activities up to the next one. And I presume maybe you can think about that, like, stepping up. And it almost sounds to me like there's this almost like a timer that a team has, like, to put something in a real person's hand, lest patience run out with the project. Is that is that pressure I'm kinda sensing from your description there and something you're seeing
with teams in this kind of work? Yeah. I I I think that's I think that's absolutely right. Like, you know, I think the you know, these modernization efforts are, like, high high investment. Right? There is there is typically, like, very high cost with these projects. Right? There's a lot of typical like, typically, like, the sponsors of these projects are spending a great deal of of, like, organizational like, their own organizational capital, right, to to get these projects approved. And so so, yeah, I think there's there's, you know, it just it just human nature to want to be able to show kind of quickly, right, that you're able to,
deliver deliver value, right, and show that you're kind of on the right track. You talked about how, data can get stuck and how that impacts the modernization method you're using. I'm curious if in your lived experience, if you've kinda gone through a project there or either, you know, got stuck and was a problem or a situation where it happened to work out quite well because you managed to work past that.
Yeah. So one one of the one of the projects at Nava that I think has been, I think, really interesting for me to, like, kind of, like, learn from and see what they're doing at the data layer is, some of the work that we've been doing at CMS around, like, data modernization. And so, you know, there's a a mainframe system there. And and in that case, right, there's really no desire to get off the mainframe at least anytime kind of soon. But there's a lot of value that can kind of be unlocked by by essentially, like, replicating that data out, to the cloud so that other, like, newer applications can consume it. And so I think it's a really great example of of a way to modernize that isn't necessarily, you know, we have to deprecate this mainframe, but but really being intentional about how to unlock value from what's there, and and kinda strike the right balance between, you know, what's the appetite for risk versus what do we need from, like, the needs of of the business. So so, yeah, I I would point to that as, like, I think really good example around modernizing that that data layer.
It sounds like kind of the success there is not replacing the mainframe, but kinda adding, like, another level of abstraction so that if you want to later, you're already relying on these other things in the cloud. It's just the way the
the data gets to those would have to change in order for you to reach that step. Yeah. Exactly. Right? So so, you know, I think, you know, it's it's been interesting in the space around particularly around, like, you know, when you when typically, right, when when someone mentions legacy modernization, most people think immediately about the mainframe. Right? And, like, sort of that's kind of, you know, the only thing that people think about. But, you know, I think there's a lot of you know, if you dig into, like, the chatter in the space, like, there's there are a lot of folks that are out there saying, like, well, one of the reasons that mainframe is still with us because it it's really good at a lot of things that it does. Right? And so being, I think, intentional about where where does this technology work for us. Right? Where does it not work for us? And then using that to inform, like, how you modernize, is, I think, I think, kind of key and, like, I think underline some of the points that you're making there. These projects ultimately
exist to serve people. We're talking Medicare beneficiaries, claimants, families navigating social safety net programs. And a lot of the daily work is about wrangling stakeholders, contracts, old code bases. When you're kind of in the middle, in the bubble of all that, how do you keep those end users that we're thinking about present in the room with you?
Yeah. Yeah. So, you know, for us, I think the the the goal is to keep end users kind of in like like, present throughout that entire process. So, you know, you know, human centered design, is is a major tenant of Nava's approach. Right? And and so one of the ways that we do that is not just have, you know, kind of like research be an input at the kickoff that you sort of, you know, use once or refer to once and you move on, but, right, the goal is to kind of continually pull that into our our sprint cadence and be able to pull that into decisions as we go. I think the other thing about this is is, you know, not designing in a vacuum. Right? So I think it's one thing to sort of, you know, think about design as, like, what is what is the most elegant design versus, like, a a more maybe again, like, maybe to overuse the term human centeredness here. Right? But to be more grounded in sort of, like, what's what's the real world experience going to be around this? Right? If someone's filling out benefits and they're trying to do this at 11:00 on a Sunday, they're gonna be in a very specific sort of mode, right, to try to get that work done. And so how do we how do we think about what does it mean to design for that specific type of of interaction? Like, it's also connected to something that we were talking about a a minute or two ago. Right? You know, I was talking about, like, compressing compressing discovery. Right? Compressing, like, the archeology phases of of these modernization efforts. And this is a big reason to me why it's important to do that because teams spend so much time thinking about or trying to understand, like, what was this system designed to do 40 ago, and then they they miss the opportunity or they run out of time to actually think about, like, well, how should this actually work going forward. Right? And so, again, like, the the less time that we spend sort of unpacking the past, the more time we can think about what's the right way to have this service exist going forward.
I, like the connection you're making, back to that because, you also, in your answer, talk about this being, like, an ever present thing in the process. Like, the research step isn't just like, alright. I test it with some users. I'm done. You know, Their the nature of their circumstances change. The technology they use has changed. The systems and policies change. Like, you need to be constantly. And, you mentioned there at the end, like, the the goal to try to, design something that works for them. Well, it's, like, really hard to learn enough to do that if they can't touch it. Right? So, like, as you talk about compressing time, it seems like these ideas connect together to again, getting in that place where it's like, hey. Can I get this to a person's hands so I can learn if it's any good at all and if it's useful?
Mhmm. Yeah. Yeah. Absolutely. I I you know, the the whole idea of, like, shipping and it's it's also partly why it's important to ship early. Right? Not not only you building trust with you know, we talked about building trust with stakeholders, right, and and showing positive direction and positive momentum. It it is also, of course, to still, like, validate that what you're building is actually working for your intended users.
Caseworkers and program staff who use these systems often hold deep institutional knowledge that it might not be written down in documentation. How do you make sure to engage them in the work, and how do you handle it when their look workflow expectations have some conflict with what, the opinion is that the modernized system of what, like, what it should do?
Yep. Yeah. So, caseworkers are, like, a a tremendous source of insight, right, in terms of how systems actually work. So, you know, you could have written documentation. You could have written policy, but oftentimes, right, that can kind of drift from what's actually there. And so caseworkers are are are great at understanding that reality. And so, because of that, right, we wanna treat them as, you know, co as co designers really, not just sort of like passive stakeholders. And so by incorporating them into our design, I think the other element here is, like, you know, we've we've been talking a lot about, like, redesigning, experiences for beneficiaries. Right? We wanna, like, change how services are delivered. But a lot of our work is actually about how do we reduce the burden for caseworkers themselves. Right? How do we make it easier to process a case? Because if we can do that, then we can make, you know, processing times for the for the end beneficiary go down. And so in that case, it's really thinking about those case workers, you know, not just as, like, information sources, but actual users that we want to be able to design around. In that case, you know, there's, like you get into, like, some of the change management stuff that we've talked about, right, where case workers, as an example, right, can have, like, very specific ways of working and and, like, their their processes and their ways of working can sort of grow around the legacy system. And so when we're thinking about how should this thing change or behave going forward, right, we wanna be really intentional about, okay, what's the what's the why behind there? And is it something that actually, you know, is is a preference that, like, we can sort of change because there's a better way to do it, or it's actually, no. This is actually, like, a hard requirement. We we have to keep doing it this way for some, you know, specific reason. So, you know, at the end of the day, it's, like, just as we wanna be intentional with the experience that we're creating for end users, we wanna be intentional about the experience that we're creating for caseworkers and and other internal staff.
What's the time when you saw the direction or design shift after you kinda discovered something as you were working with, caseworkers and learning about their workflow?
So I think I think, I think kind of riffing off that, I think there's a recent example actually from some of the Nava Labs work that that kinda fits in well here. So, Nava Labs, was recently working with, Maryland around sort of snap work requirements, right, and all those changes that are coming up. And so, you know, as as part of that, there have been sort of like, the typical ways that a beneficiary would sort of submit their documentation. Right? And Novel Labs was sort of or not sort of. They were piloting a new way to do that, leveraging OCR and AI to, essentially, like, improve the quality of the images that are coming in because that's a there's a key pain point in that overall process. Right? And so I think that's I I think this is a great example of, like, one of those tensions where the the stated process is that there are, you know, require there are, you know, requirements to gather certain documentation or certain evidence, and we wanna obviously work within that overall requirement. But then there are going to be, like, different ways that we can innovate within that space. Right? And so pre like, creating this new pathway through Document AI, kind of allowed allowed that sort of, like, slight process shift that still work within the overall, like, end to end goal, that the agency had. So I think I think I would, like, point to that as as an example there.
And that example sounds like there's was potentially the possibility of having some, like, stakeholder conflict between caseworkers and beneficiaries in this case, which are kind of, like, a different coming from different perspectives. Is that something you ran into? And if so, how did you try to navigate that?
Yeah. That's a great question, Ryan. I I wasn't on that project so much day to day, so I don't have, you know, I think first hand view of the stakeholder dynamics, but it totally, I think, a fair a fair flag and and question to ask.
I guess if not that project, then, one that you have that lived experience in, is that something you've run into where it's kinda like two very different, like, maybe folks kind of on two sides of an application running into conflict? Are there strategies that you kind of have seen successful for that kinda conflict resolution?
Yeah. I think I think in that case, like, it it gets back to kind of, like, the research, right, and that we were kind of talking about a little bit before. So, you know, the like, being able to, actually, also sort of hearkening back to what we were talking about with sort of, like, change management, sort of, like, the nontechnical aspects of a lot of this work is, like, it's it's it's natural for people to not want to change. It's natural for people to be kind of, like, used to the way that they're working and want to be able to keep that. And so I think the way forward through that challenge, right, like what you're talking about, where there is that sort of, like, tension or conflict is, like, is not about the technology, right? It's about like the more of like the just working, like how do you build and work through this as like humans, as a team? And I think to that extent, like it comes down to like showing, like, true, like, empathy for the people that you're trying to serve. Right? And, like, true and, like, trying to get to, like, a true understanding of what is it that they're doing. And so if you could do that and then if you can reflect that back to people and show them that, yes, you are listening. Yes. We hear you. But also help them understand with, like, a fairly rational case about why things have to move forward. I think most of the time you can find that right compromise for folks, and nav you know, navigate the right path forward between those tensions.
We've talked a bit so far around different ways that, patterns for approaching legacy modernization can kinda come into the work. What sort of legacy patterns do you tend to favor with with your client work?
Sure. So patterns wise, you know, I I think when people think about legacy modernization, they tend to think about, like, re re you know, generally rebuilding is a is a major pattern that comes up. And within that, like, strangler fig seems to be the one that, like, almost everybody can, you know, name, right, which I think is this interesting data point. You know, the the the whole idea behind that strangler fig approach, right, is that you have some sort of, like, front door, or facade that you put up so that you can essentially keep the user experience consistent while you kind of migrate or, you know, modernize behind the scenes more incrementally. You know, to to use, like, a little bit maybe more of, like, a nontechnical metaphor just to you know, this concept is so prevalent. I think it's worth, like, spending a second on, but to maybe have, like, a nontechnical metaphor to to kinda walk it through. You know, imagine you were eating at, like, a very small restaurant where there was only one one person working there. Right? And so they're, you know, they're gonna greet you at the door. They're gonna sit you down. They're gonna take your order. They're gonna go cook it. They're gonna bring it back to you. If that person hits the end of their shift and they need to, you know, switch out, like, it's gonna be a very disruptive experience for you as a diner. Right? Now, like, that's a very fictitious example. But if you contrast that with, like, a more typical dining approach, right, or, you know, situation where you have, like, a waiter who's sort of they're they're now there to sort of, like, create that continuity for you. Right? Create that same experience. That's that's the facade. That's the front door that we're talking about in that Strangler Fig approach. So so in that case, right, if the entire kitchen staff turns over during the course of your meal, like, you're you shouldn't really know. You should never notice, and you shouldn't really care, right, at the end of day because, like, your experience is not being disrupted, right, because of of that waiter. So that's what we're trying to do. Like, you know, that metaphor isn't perfect, but, you know, hopefully helps. And, like, that's basically what we're trying to do with with Strangler Fig there. Right? Now once you have inside of that pattern, there are, I think, lots of other things that you can start to do around, like, comparison testing and behavioral testing to make sure that systems are actually getting to the same sort of outputs. So I think a lot of interesting, like, things like a like a click down from there. Stepping back out, like, there are other patterns, that folks can name, like lift and shift. Right, that's, you know, for me, like, as we talk about a lot about, like, AI approaches, I think a lot of a lot of those AI driven approaches to modernization sort of, like, tend towards this side of the spectrum. Another one is kind of leave and layer, which is, in this case, an idea that, you you don't need to get fully off of the legacy system, but you're gonna sort of build around it. And then a a version of that is is actually what we were talking about a few minutes ago with with data replication and data modernization. The idea that, you know, we're gonna leave the existing system of record, but we're gonna find a way to make that data available so that new applications can, like, consume it and create experiences around it. So those are those are a couple of different examples, I think, that we've seen firsthand. Ultimately, at the end of the day, like, the right pattern is going to come down to whatever the specific, like, business need is, right, for that agency. And and based on that, you can kind of choose the pattern that kinda best best fits their purpose.
I hear you making a a strong connection in your answer between, the lift and shift pattern and AI kind of tooling and techniques in used in modernization approaches. To hear you say that to you, are they kind of, like, becoming one and the same? It's just it's lifted and shift, just new tools. Or or is it maybe a bit different than that? Yeah.
I I I mean, you're right. I did I did associate those. So I I, when I say that, like, AI, a lot of these AI approaches are are like lift and shift, I don't mean it I guess, I don't mean it in, like, the strict definition of lift and shift of, like, you know, the the rehost, like, we're not gonna touch anything. We're just gonna, you know, take it on prem and drop it into the cloud. Like, that that's a very, like, strict interpretation of lift and shift. What I'm more trying to convey, I think, is that these approaches, they're not really trying to, like, for the most part, like, reimagine or rethink what this system should do going forward. They're they're very much about, like, we're gonna take what's here now. We're gonna try and we're gonna try to move it as quickly as we can to some sort of future state, and then we're gonna iterate from there. Right? And so I think that that that to me is why I kinda, like, put it more on that side of the spectrum. But, yeah, there are sort of wrinkles inside of, like, okay. This is a rehost. This is a replatform, etcetera, which is, like, a fair a fair call out there. Yeah.
You talked at length of that metaphor on Strangler Fig, and it's actually one that we've, covered a bit in a prior podcast episode, which I'll also link in the description for folks if they wanna do some additional diving there. As as you think about that method and your own personal experience with it, what's an example where that's either, like, worked out really great or one where maybe it didn't work out so great and that experience really stuck with you? And what was kind of, like, the lesson you picked up?
I think from a modernization standpoint, like, Strangler Fig, it really works well when there are kind of, like, I would say, like, three there kind of need to be three criteria in place for it to kinda work really well. So one, you know, it's gotta be a a fairly large system because, like, you kinda know that you're not gonna be able to just rip it in one go. Right? It's it's big enough that you're like, okay. We need we need to kind of, like, chunk this up. Two, it's also a system that you know you need to get off of for whatever reason. Right? So we talked about, like, the leave and replace where, like, it's just like, oh, you know what? We're gonna innovate around this. Like, if if you're doing Strangler Fig, like, your goal has to be, like, we're gonna get off of this system. And then three, you have to be able to sort of find, like, seams or, like, you know, clear clear places where you can sort of, like, incrementally pull, logic and processing out of that legacy system. So if you've got those three things, I think then it'll work, like, pretty well. Right? And I think we've seen, one of the one of the projects that we're working on now, like, essentially, like, kind of, like, built the, like, new application portal, and then, essentially, that became the facade, that front door. Right? And then they were able to start kind of, like, working behind it. So I think that works really well. I think we've seen other examples or, like, even, like, inherited other examples, right, where you start that process, but you never actually get off of the legacy system. And so now you're in now you're in a much worse spot where you're kind of like you've got two systems and you're trying to straddle them and some some, you know, some your your system of record is partially in the mainframe and partially in this new system. That and that becomes really that becomes really, really messy. And so I think that's where, you know, we certainly don't wanna get stuck in that position. But if you've ever worked in that like, those types of environments, like, that that becomes, like, a very difficult situation to kind of work out of.
I I think I'm hearing, like, a a little, like, hidden constraint slash requirement to Strangler Fig, particularly where it's like, you need to have not just political will for the project, but over, like, the long haul. Like, there's needs to be some amount of, like, continuity from, you know, program policy coming down that this project's important, and this method is what we're what we're doing. We're committed.
Yeah. Abs absolutely. I I think that's that's totally right. Because, like, you know, if if almost by definition, if you're employing the strangler fig pattern, right, like, you're dealing with some kind of, like, long running modernization effort. So, yes, like, absolutely need that sort of, like, top down, top down buy in. And then but it's also what we talked about. Again, like, so much of this conversation has been about trust. Right? And, like, what do teams do with, like, a nontechnical perspective? But, like, teams also need to sustain that energy. Right? And they sustain that energy by delivering. So that, like, that it it's it's both. Right? Like, both both sides of that of that spectrum need to be able to to kinda keep keep the buy and keep the the willpower to keep going.
How can we use solid practices to ensure we get a a return value on our investment early in the work rather than waiting all the way to the the tail end? And how does that affect our approach to risk management?
Yep. Yep. So yeah. So, you know, we've been talking a lot about this today. I think so, you know, we've talked about, I think, the momentum parts of this challenge. Right? So, like, thinking about what's something that we can actually ship to production that's going to create value for a beneficiary, for a caseworker. You know, we we think a lot about it in terms of, like, you know, not just shipping the or not shipping, but we don't just think about, like, prototypes or demos. Right? But we wanna actually have working software in production. And so okay. We talked a lot about the momentum. I think the other sort of piece here that we haven't yet talked about is that, like, shipping to production does a double duty of creating that value, but also derisking the longer term because, like, you essentially shrink the unknowns that you're dealing with. Right? So just by that act of trying to get production, you learn a lot about the system. You learn about also, like, nonfunctional requirements that can often be a a drag on these efforts, whether they're security, ATO, right, compliance. And so by kind of getting into that pattern early of shipping, you know, to production, you could really, I think, derisk these efforts because you you just learn more and more about what you're dealing with at the end of the day.
I imagine a lot of the work we've been talking about today involves some parsing between decisions that are easy to undo and those that are much harder or maybe even impossible to pull back. How does that influence the way you tend to approach these modernization projects?
Yeah. I so I, I really like the language around two way doors and one way doors as sort of like a framework for thinking about these decisions. Two so two way doors are ones that you can essentially kind of, like, walk through and then and then walk back through if if needed. Right? And so those types of decisions are really you know, should be cheap and easy to change. In that case, like, for those types of decisions, right, you wanna be, like, pretty quick in making that decision, ship something, and then see what the data says. Right? So, you know, thinking about, like, AB experiments or other just data driven decision making. I think contrasting this with one way doors, which is, you know, these are gonna be decisions like like, for example, like a data model where, you know, changing that later is going to be pretty painful and costly to the team. And so in those cases, you wanna kind of you do wanna spend a little bit more time upfront doing that analysis. Maybe it's a spec or a spike, right, to get more information. You know, get get to that understanding and then make that decision. So the, you know, the the biggest sort of kinda trap in all this is just, you know, not understanding which decisions are one way or two way doors. Right? But if you can get that right, I think you can really move through these decisions, like, much more much more rapidly and much more effectively.
I hear maybe a a little bit of, like, a shared worry between kind of the answer. And this question is also the one we talked about earlier about data getting stuck when you're kind of talking about the the data layer. Are the kind of the worries about, one way decisions and kind of data gains that kinda coming from the same place?
Yeah. Absolutely. I I I think, you know, that, like, the examples that we talked about, right, where, you know, the modernization efforts kind of completely stalled out because that data was was locked in, I think is a perfect example of this where, like, the team, you know, should have spent more time kind of upfront thinking through what that strategy is going to be so that then when they get to the end, they actually have a path forward. So, yeah, I think it is a perfect example of, like, you can't you know, some of these things you can't defer to the end because you you're just you you paint yourself into a corner and then you get stuck.
And, something I think you mentioned that's rather important is you can't treat every decision like a like a one way door. Right? So some of them have to be two way doors and and treated as such. And I imagine in the product space, having to try to facilitate identifying those and then agreement that, hey. This one actually is a two way door, and here's how we're gonna control for that is maybe a challenge. What's your personal experience been like trying to navigate that kind of conversation?
Yeah. I I you know, in some cases, you can sort of use the you know, with some teams, you can sort of use the two way door, like, terminology explicitly. Right? You could, like and, like, you could even think about, like, that being a a mental model that the team shares. I think, you know, for teams that I've worked with, you know, I don't necessarily use like, I don't necessarily call something a two way door, but it'll it'll I focus more of my language and, like, the team around, like, experimentation or data. Right? And so if they're you know, a a common question that I'll ask is, like, what what's a, you know, what's a quick experiment that we can run here? And if the if if they come back pretty quickly and be like, yeah. We can go run this experiment. Like, great. Like, let's go do that, and then and then we'll make then we can make a decision. Right? So I think it's just framing like, it's a lot of the framing around it as, like, an experiment, I think, is, like, basically if if you could structure it as an experiment, it's probably a two way door. You're probably pretty pretty safe.
AI tooling has been showing up in our space in a few different ways now from helping translate policy into testable rules to assisting with code understanding with legacy systems. Where are you finding it genuinely useful for modernization work, and where do you think it's more, like, hype machine kind of stuff?
Yeah. Yeah. So, you know, I think for us where we've seen it is like specific like particularly around like trying to understand legacy systems and getting your arms around it, like certainly is has been like very valuable there, right? So it's, you know, these tools are able to either generate diagrams, dependency maps, especially in situations, right, where, like, the original developers are long gone. There's maybe only a handful of folks that have any context on what the system actually does. Like, that that is that is truly valuable. I think in a you know, adjacent to that is some of the, like, the, like, documentation generation that you can do with that. So, you know, helping teams, like, rebuild that knowledge base because, again, oftentimes, like, it's it's not there or if it is there, it's it's really stale. One area that we've been experimenting quite a bit with is is around, like, thinking about how to go from, like, policy to actual, like, working software and, like, compressing the amount of time that teams spend kind of, like, typically in that in that sort of, like, translation. So I think I think that is, like, something that we're hopeful. You can also imagine, you know, inside of policy, there are all sorts of, like, processes or procedures that are defined for these systems, and so trying to, like, quickly understand and unpack those. One, you know, you mentioned the, like, hype hype cycle around this stuff. I I will say I'm I am personally a little skeptical that, like, right now, at least you're gonna point an AI at a legacy system and just sort of magically get magically get that modernized system out on the other end. Like, I just I don't think that's where these technologies really are yet at the end of the day. And, like, for all the other reasons that we talked about too, which is, like, it like, modernization is is is about more than just the technology layer. Right? There's all the change management. There's all the process changes. There's the policy changes. And so it's like, once you even think about these efforts, like, as, like, holistically as they are, like, the the AI layer is just one is just one one component of that. I think the last thing I'll say on this is that, like, you know, AI is such a is such a compelling general purpose technology. Right? Like, you know, you know, I we started the call earlier. We talked about, like, change data capture. Like, that's a very specific technology that you use in a very specific time for, like, a very specific problem that you're having. But AI is is like the exact opposite, and so it creates a lot of noise and a lot of, like, you know, a lot of fog about, like, where can it actually be used. Right? And so one of the things that we're trying to do as as, like, our team in this space is to really be, I think, like, honest about, like, where are we where are we trying these tools, and then where are they where are they valuable, and and where are they not, right, at the end of the day. And so I think, you know, what we're learning is that, like, at least where these tools are right now, like, you have to have a human in the loop. You gotta you have to have someone there, to make sure that it's kinda staying staying on the rails. But overall, like, you know, like I said, like, our goal is just we wanna we wanna be transparent about kind of what we're learning as as we go on this journey.
At the end there, you mentioned this need for humans in the loop, which I think is a phrase folks hear a lot. But in your mental model of that, where do you tend to see the human kind of fitting along that loop?
Yeah. So I think we there are you know, so far, I think we've there are two areas where we've, I think, has seen the most value. So one is in sort of the, like, creating the mental model or the structure, for the AI to explore a given space. And so what do I mean by that? I mentioned this example that we've been playing around with with, like, policy as code. And so one of the, like, one of the early, like, examples of human in the loop that we had is, like, is, like, a policy strategist who would essentially, like, give the give the LLM some structure for, like, how to think through a given policy area. Right? And so, like, with that structure, we could improve the accuracy that of what the LLM was doing. So it's sort of like that upfront kind of, like, guardrails or guidance. And then the second is I I think as I think folks are like like, a lot of folks already into it is around evaluations. Right? Just, like, being there to understand, like, okay. This is this output actually does this output actually make sense? Right? Like and and all the judge all of the judgment that goes along with that. So that's those are the two areas that we've been kind of, like, most plugging in humans in the loop in our in our product develop.
And earlier in your answer, you mentioned the use case of documentation generation. And it's something it made me think about is on the software end, there's all these, like, open source projects. Right? And sometimes what can happen if it's a it's particular if it's a popular one, is folks really wanna help their favorite project out. Right? So maybe they, like, kick up their favorite LLM. They prompt it. They feed it context. Comes up with maybe an excellent PR or maybe not. You know, it depends on how well it was prompted probably. But then that happens once, twice, a 100 times. Meanwhile, the team that's managing the project maybe doesn't there's, like, three of them, you know, and they're volunteering their time on the side while they're working a job or you know? And it it's that it's that thing where it's like, at some point, is there a risk that the bandwidth to review gets overwhelmed by the the generation of stuff? Is is that something you you see and think about?
Yeah. I so, I mean, that's that is a that is a huge challenge. I don't I don't think we I don't think I have the answer to your problem yet because, like, I think we're see also seeing that firsthand in some of these tools where they they create so much documentation, and and so much analysis that like, there's real effort to get teams just to sort of be able to digest all of this, you know, information that it's producing and then let alone, like, assess whether or not it's accurate. So I think that's that's a huge point. I think some of the things that we think about as, like, mitigations around this. So one is a lot around, like, the testing and evaluation stage. And when I say evaluations here, I don't mean in sort of, like, the l m as judge kind of idea of evaluations, but more about, like, actual, like, deterministic, like, more like test driven development evaluation. So, early on, we talked about, like, comparing system behavior. And I think, like, where we can get to sort of, like, that type of guardrails, like, that can, I think, be a interesting check on on some some of this problem that you're talking about? The other thing the other thing that we're thinking about, which somewhat relates to what you're talking about, again, doesn't, like, exactly solve it, but we are thinking about sort of, like, through our development process as we're sort of using, you know, more AI tools in our development process, how are we making sure that, like, the documentation essentially, like, is updated as we go so that there isn't like, part of the part of the challenge of why there's this big influx of documentation that you talked about is because, like, so much has gotten out of date over time because it's it's hard to keep up with. Right? And so the more that we can sort of bake that in and, we were working on a I was working on a tool with a couple engineers just the other day, and we're changing sort of how one of the endpoints functioned. Right? And now those all those endpoint changes, like, automatically get updated into all of our API documentation. It's, I think, a great way to, like, reduce some of that burden over time,
even if it doesn't sort of, like, solve that, like, catch up problem that you were kinda talking about there. David, as we get to the the tail end of our conversation, you know, we've covered a fair bit of ground. We've talked about the lockpick concept and kinda how that becomes reasonable stuff between different efforts. We've talked about modernization strategies, the pros, the cons, the indifferent things between them. And we've even talked about where artificial intelligence can kinda fit in and around these kind this kind of work. If there is one or two things from what we've talked about today that folks would take into their day and beyond, what would you hope those would be?
Yeah. I think that's a that's a really it's a really great question. You know, I think the the one thing I would kind of, like, underscore, as we get to the end here is around and it's been a it's been a theme that we've kind of talked about through this entire conversation, but is is the idea that, like, legacy modernization is is more than just a technology effort. Like, yes yes, there's a technology component too, but when you think about modernizing government services, you have to work across policy. You have to work across organizational challenges. You have to work across business processes. And so these efforts are really about thinking about how all those things kind of come together to rethink how services should be delivered. And so at the end of the day, like, you know, just reducing it to a technology problem, I think, doesn't get to, like, the full essence or even, like, full challenge of of what modernization means, especially in in the government space.
David, thank you again for coming on Civic Tech Chat and joining us for this conversation.
Thanks so much, Ryan. I appreciated it.
Visit us on the web at civictech.chat or subscribe to us for content updates wherever it is you download your podcasts.
