How AI Uncovers Silent Patches in the Open Source Supply Chain

TLDR: Many open-source maintainers fix critical vulnerabilities without issuing a CVE, creating a "shadow patch" gap that leaves downstream users exposed. By using LLMs to monitor public change logs and cross-reference them with vulnerability databases, researchers can now identify these silent fixes at scale. This research proves that a significant percentage of critical vulnerabilities remain undisclosed, and it provides a blueprint for using AI to automate the detection of these hidden risks.

The open-source supply chain is a house of cards built on trust, and that trust is often misplaced. When a maintainer pushes a commit to fix a security flaw without filing a formal CVE, they create a window of opportunity for attackers. While the fix is live in the repository, the lack of a public advisory means security scanners and automated dependency checkers remain blind to the risk. This is the reality of shadow patching, and it is a massive, unaddressed attack surface for any organization relying on third-party code.

The Mechanics of Shadow Patching

Shadow patching occurs when a maintainer identifies a vulnerability, implements a fix, and merges it into the codebase, all while bypassing the standard disclosure process. The reasons vary from a desire to avoid reputational damage to simple resource constraints or a belief that the issue is too minor to warrant a formal advisory. Regardless of the intent, the result is the same: a known vulnerability exists in the wild, but it is not tracked by the security tools that developers rely on to secure their applications.

The research presented at BSides 2025 demonstrates that this is not a fringe phenomenon. By scraping the change logs of the five million most popular open-source packages and using LLMs to analyze the commit messages and code diffs, researchers identified hundreds of undisclosed vulnerabilities. The process is straightforward: the AI monitors for keywords related to security, such as "fix," "security," or "vulnerability," and then evaluates the code changes to determine if they address a potential OWASP A06:2021 – Vulnerable and Outdated Components risk.

Automating the Hunt for Hidden Bugs

The power of this approach lies in its ability to scale. Traditional static analysis tools often struggle with the ambiguity of commit messages, but LLMs excel at interpreting the intent behind a code change. For instance, a commit that replaces a dangerous function call with a sanitized version is a clear signal of a security fix, even if the commit message is intentionally vague.

Consider the case of CVE-2022-0847, the Dirty Pipe vulnerability. While this was eventually disclosed, the time between the fix being committed and the public advisory being released is exactly where the risk lies. By automating the monitoring of these repositories, researchers can identify these patches as soon as they are merged.

The research team built a pipeline that performs the following steps:

1. Scrapes public package registries for the latest versions.
2. Uses an LLM to normalize change logs into a standard format.
3. Identifies security-relevant changes using a fine-tuned model.
4. Cross-references these findings against existing vulnerability databases.
5. Passes potential hits to a human security engineer for final verification.

This methodology has already uncovered critical vulnerabilities in widely used packages like Axios and Apache ECharts, which were previously unknown to the broader security community.

Real-World Impact for Pentesters

For a pentester or bug bounty hunter, this research changes the game. Instead of relying solely on public CVE databases, you can now monitor the repositories of your targets for these silent patches. If you see a commit that looks like a security fix but lacks a corresponding CVE, you have found a potential vulnerability that is likely still present in older, unpatched versions of the software.

During an engagement, this means you should not just check for known CVEs. You should look at the commit history of the dependencies in the application's package.json or go.mod files. If you find a suspicious commit, you can often reproduce the vulnerability by comparing the code before and after the patch. This is a high-value target for any researcher looking to find bugs that automated scanners will miss.

Defensive Strategies

Defenders must move beyond simple CVE-based scanning. If your security program only alerts on known CVEs, you are missing a significant portion of the risk. Organizations should implement a more proactive approach to dependency management, which includes:

• Monitoring the commit history of critical dependencies for security-related keywords.
• Using tools that provide visibility into the entire dependency tree, including transitive dependencies.
• Establishing a process for evaluating the security impact of updates, even when no formal advisory is provided.

The era of relying on public disclosure as the sole trigger for patching is over. Attackers are already using these techniques to find and exploit vulnerabilities before they are ever assigned a CVE number. If you are not looking for these shadow patches, you are leaving your infrastructure open to exploitation by anyone who knows how to read a commit log. The next time you are auditing a project, look past the CVE list and start digging into the code itself. You might be surprised by what you find.