Supply Chain Attacks
Ryan Koch discusses the importance of understanding supply chain attacks in cybersecurity, illustrating how they exploit trust in software components and build systems, and offering practical strategies to mitigate these risks.
Resources and Shoutouts:
- GRC Engineering Presentation by AJ Yawn: https://engage.isaca.org/events/event-description?CalendarEventKey=bd0c09c0-662f-4fe8-a0d2-019adaab6cc4&Home=%2fevents%2fcalendar
Top Keywords
- package 0.011
- supply chain 0.010
- software 0.008
- build 0.008
- supply 0.008
- maintainer 0.007
- vendor 0.007
- chain attack 0.007
- compromised 0.007
- attack 0.007
- chain 0.006
- attackers 0.006
Transcript
Hello. I'm Ryan Cook, and this is Civic Tech Chat, a show that looks at the way technology, politics, and policy impacts the world around us. The tools we use, the way services are delivered, and how we talk about and set policy all shape our society. We'll gather around and have a chat about these things together and more. Before we get started, I do want to let you all know that we've started a Discord for the podcast. There will be a link with an invite down in the episode description. Do feel free to go check that out. It's a small community right now, but hoping to grow it. It's a great way to reach out to me and let me know things that you might want us to cover, or to just hang out and talk about civic tech. Today, I wanna talk about something that I think is really important for anyone who works in, around, or with technology in the public sector to understand, supply chain attacks. I know that might sound like a niche cybersecurity topic. And if you're someone on the policy side or the program side, you might be thinking, okay. That sounds like an IT thing. But I'd argue this is one of those topics where the technology, policy, and real world consequences are so intertwined that everyone needs at least a working understanding of it. So let's build that knowledge up here together today.
What is a supply chain attack? The simplest way I can describe it is imagine one day you go out and buy a bottle of water, and you get real unlucky. The water makes you feel sick.
To you in that moment, the thing at fault must just be the water you drink. Right? A lot of things go into preparing it before it comes to quench your thirst. The labels were printed, the plastic was molded into some shape, and maybe the caps that get thrown at the end came from some other supplier before they were used. As those things all came together, the water was added to the bottle. What if those caps that came on the last step were contaminated? And what if that contamination wasn't noticed before you drank from the bottle? That kind of scenario is a kind of analog to what a technology supply chain attack might be like. Modern software is assembled from hundreds or even thousands of pieces, components. Those are open source libraries, third party services, vendor tools. Research suggests that open source code can make up somewhere between 7090% of a modern application. Every one of those components is a trust decision, much like the water bottle company trusting the caps. When a developer includes a library in the project, they're trusting that the people who maintain it, the systems they use to build it, and the channels they use to distribute
it haven't been compromised by some sort of malicious actor. A supply chain attack then exploits that trust. The attacker compromises
something along that upstream path. Maybe it's a popular software library, a build system, a vendor's software update mechanism, and it uses that relationship to arrive downstream and to potentially thousands of victim organizations. So that victims don't get hacked in the sense that you might see them in the movies, right? They just install a software update. They update a dependency.
They do the stuff they're supposed to do. And that's then how the malicious code finds its way inside. Now, why should we care about this? You might be wondering.
There is a few reasons. First is amplification, as we've been talking about. One compromise in one of these chains can reach a huge number of victims
very quickly. Second is detection. It can be really, really difficult
for these kinds of things because the bad code arrives through a channel that is meant to be trustworthy. It could be a signed software package update. It could be from a well known NPM package, you know, like, for you JavaScript folks. And these things then blend in with normal activity. Your security software, your scanning tools are, you know, looking for that to be something malicious. And so that's then something that can make this ever more dangerous. Third, the code that comes in through a supply chain attack may already have elevated privileges because of its entry point. It's running as the thing you built or in the build tools you have. So there's no need for someone to kinda take that step where they get into a lower level thing and try to escalate because
the nature of the attack got them there already.
The scale of this problem is also growing quickly. Supply chain attacks more than doubled in 2025. Global losses hit an estimated $60,000,000,000. Over 70% of organizations reported experiencing at least one supply chain related security incident. So hearing all that, you can, you know, I think, safely think that this isn't a theoretical concern that we're talking about here today. But how do these things tend to play out?
Let's walk through a couple of common patterns, where there's some interesting bits to pick up. One such common way is through a classic, you know, stolen credentials.
An attacker gets a hold of a package maintainer's login somehow and, or or, like, a long lived access token they use for a build system, and they use that to publish some poisoned version of what would otherwise be a legitimate software package. From the outside, it looks things look fine. The package's name is the same. They haven't, like, obviously taken it over. The version number probably changed, but it's not obvious that there's been tampering. Another way is social engineering, which can be, very sophisticated. A very striking example of this is, the x z utils backdoor that came up in 2024. Xzutil's is a compression library used across a lot of Linux distributions. So you can imagine it's something that ends up in a lot of really interesting and important infrastructure. The attackers spent two years making legitimate contributions to the project. They build up credibility. They sought to earn trust and eventually used sock puppet accounts to help pressure the maintainer to give them shared access of the project. And then once they have that trusted position after investing all that time, they used it to insert a backdoor that could have given them access to millions of SSH enabled systems worldwide. SSH is a remote access tool, so you can access computers that are far away. And what's interesting is it was caught by accident. A Microsoft engineer happened to notice a five hundred millisecond latency increase during some testing and chose to investigate why that change in performance happened, which is what led to the whole thing being uncovered. Another pattern is attacking the build system itself. A classic or a big example of that is the SolarWinds incident, which happened in December 2020. Russian affiliate attackers compromised the build infrastructure for the SolarWinds Orion platform and put a backdoor into their legitimate software updates channel. Those updates went out to 18,000 different organizations, including those within the, you know, US federal government, Fortune 500 companies, and the cybersecurity firm FireEye, which then happened to become the organization that discovered the breach. The attackers managed to operate undetected for nine months, the source code itself being clean. The malicious thing, the poison, for lack of a better way to put it, was introduced during the build steps in the build system. There's another example that I wanna spend a little bit of time on, both because it's recent and because it shows that ever increasing sophistication
that we're getting in this thread.
At the end of March twenty twenty six, just days ago as I record this, the Axios npm package was compromised. If you're not in JavaScript world, this Axios is a very widely used HTTP client library. It has, over 83,000,000 weekly downloads. It's all over the place. People's front end applications, their back end services, enterprise systems. It's very popular. The attackers compromised the account of the primary maintainer of this project and managed to publish not one but two malicious versions of it. What's interesting is that they injected a fake dependency that contaminated the kind of post install script process, the things that run after install. And what that process then did, the poisoned one, is drop a cross platform remote access trojan. And they have pre built payloads for macOS, Windows, and Linux. Both release branches were hit within thirty nine minutes, and after execution, the malware would delete itself, place its own package manifest with a clean version. It basically, you know, worked to cover the tracks, as it were. The level of planning here is what's super notable, though. It's not something that could have been done opportunistically. And security researchers at Elastic found that the macOS payload shared significant overlap with malware attributed to a North Korean threat group. So it's possible that this is something that is a state sponsored actor attacking one of the world's most popular JavaScript packages. And so folks might be listening to this thinking, well, my agency doesn't use JavaScript. That's not what's important. It's the the pattern and the fact that a build build process or a, maintainer account is attacked. The same thing can happen for projects that are using Python, Java, dot net, whatever tech stack you happen to use. So if this is the threat landscape, what can you realistically do? Is there someone at an organization listening to this? I'm not gonna pretend like there's some super easy obvious answer, where you just do this one easy thing and you're done, because there really isn't. But there are some meaningful practices you can do to reduce your risk a little bit.
First is the concept of a software or a bill of materials.
Think of it as an ingredients list for your software. It's a structured inventory of all the components, libraries, dependencies with their version numbers. Often package managers will have a lock file or something of that nature with this information in it. It's worth then keeping track of it. This is a kind of thing that can be, automated and potentially can interact with things like your,
GRC tools, your
governance risk and compliance tools. For example, there are incidents like the Log four j incident where organizations that had taken this practice were almost immediately able to identify their exposure, while those that weren't then had to kinda figure out, well, is this a thing we have installed in our tools, or is it not? Next is dependency management overall. You know, you can pin your versions. You can use lock files, like I mentioned. You don't need to automatically pull the latest version of all the things. There's even research that suggests that waiting seven days before adopting a new package version might prevent eight out of ten of the major supply chain attacks that have happened in 2025. That's a fairly simple thing one can do with maybe a big payoff retrospectively.
Third is pay attention to your CICD platform or your CICD platforms.
Third is give attention to your CICD platforms. Build systems are high value targets. You should enforce code provenance, use reproducible builds where you can, and review lifecycle scripts in your dependencies. The Axios attack, for example, happened entirely through a post install hook and a transitive dependency. If those hooks were disabled or reviewed by default, that attack vector is less likely to be exploited. Fourth, consider your identity and access management. You know, phishing resistant multifactor authentication is important on your developer and service accounts. You wanna get rid of long lived access tokens where you can.
If you can get rid of those, that's a a little bit of a tax surface that you can reduce.
Finally, for organizations that rely on a lot of external managed service providers, that sort of, vendor risk being a part of their cycle, You maybe wanna not rely just on annual questionnaires and contract clauses to manage that risk. It should be something where you're continuously monitoring what's going on with their solution, and you really want the ability to, you know, rapidly isolate a vendor integration if it becomes compromised. I wanna close with a few bigger picture thoughts because I think there are some themes in here that go beyond technical stuff. Everything we talked about today comes back to explaining trust. Trust in a package maintainer, trust in a vendor, trust in an update mechanism. That's a fundamentally, nicely different kind of problem than a traditional, hey. Here's a cybersecurity
vulnerability I need to mitigate it with a control. And it requires a little bit of different thinking. There can be real structural gaps between how fast these things move and how fast our governance and risk management processes operate. One, you know, governance can often be said to move to move at document speed, while attacks move at machine speed. Closing the gap is gonna take more than just, you know, finding a shiny tool. It takes changes to how we think about procurement,
vendor management, and what it means to trust the software you run on the systems at your organizations. I recently watched a talk given by Ajay Jong delving into the concept of approaching GRC, governance, risk, and compliance, as an engineering discipline, taking it beyond manual evidence collection, you know, building engineered systems, automations, integrations. I'll make sure I link it in the show notes as I think the ideas there are really valuable in this context and in others, and it's certainly worth the brain cycles for you all to take a peek at or listen to. This is an area that's evolving fast, and I think it's one that anyone working in civic tech, government IT, policy might wanna have on their radar. And that's gonna do it for today's episode. Thank you for listening in. If you found this useful, please share it with a friend, a colleague, your cat, or whoever in your life that enjoys listening to podcasts. Visit us on the web at civictech.chat, or subscribe to us for content updates wherever it is you download your podcasts.
