A Framework for Evaluating National Cybersecurity Strategies
This presentation introduces a structured, evidence-based framework for evaluating the effectiveness of national cybersecurity strategies across multiple countries. The research analyzes 268 criteria spanning five key pillars: codifying responsibilities, protecting people and institutions, generating capacity, building partnerships, and communicating policy. The framework provides a comparative scorecard to identify strengths and weaknesses in national approaches, offering actionable insights for policymakers and security professionals. The talk emphasizes the need for measurable outcomes, specific timelines, and the inclusion of vulnerable populations in national security planning.
Beyond the Paper: Why National Cybersecurity Strategies Are Failing Your Infrastructure
TLDR: National cybersecurity strategies are often high-level, hand-wavy documents that lack the granular, measurable goals necessary to secure actual infrastructure. This research breaks down 268 criteria across nine countries to show that while governments are good at talking about "capacity building," they consistently fail to protect vulnerable populations or provide specific, actionable timelines for security improvements. For researchers and pentesters, this gap between policy and practice creates a massive, unmanaged attack surface in critical sectors.
Policy documents rarely make for good reading, especially when you are in the middle of a red team engagement or hunting for bugs. Most national cybersecurity strategies are written by committees for committees, filled with vague promises about "resilience" and "cooperation." However, the research presented at Black Hat 2024 by Fred Heiding and his team at Harvard provides a rare, structured look at what these documents actually demand—and where they fall flat.
When you look at the NIST Cybersecurity Framework or OWASP Top 10, you are looking at technical standards. National strategies are the opposite. They are the "why" and the "who," not the "how." But when the "who" is poorly defined, the "how" becomes a mess of misconfigured cloud buckets, unpatched IoT devices, and neglected legacy systems.
The Scorecard Problem
The research team analyzed nine countries, including the US, UK, Australia, and Singapore, using 268 specific criteria. They didn't just read the PDFs; they scored them. The most glaring issue they identified is the reliance on "absolute scoring" in previous attempts to quantify national security. If a country is a "7 out of 10," what does that actually mean for a pentester? It means nothing.
Instead, the team moved to a relative scoring model. They looked at whether a country is "Leading," "Meeting the bar," or "Lagging" in five pillars: codifying responsibilities, protecting people and institutions, generating capacity, building partnerships, and communicating policy.
For a researcher, the "Codifying Responsibilities" pillar is the most critical. If a strategy document doesn't explicitly state which agency is responsible for the security of the energy grid or the water supply, that responsibility defaults to nobody. When you are performing an assessment, you often find that the biggest security holes exist in the "seams" between agencies or departments. These strategies often fail to close those seams because they are too afraid to assign blame or mandate specific, measurable outcomes.
Where the Strategies Break Down
One of the most damning findings is the consistent failure to protect vulnerable populations. While these strategies talk a big game about protecting "critical infrastructure," they rarely address the end-user. From an offensive perspective, this is a massive oversight. If you want to compromise a government network, you rarely go through the front door of the hardened perimeter. You go through the vulnerable user—the citizen, the small business owner, or the non-technical employee who has been left out of the national security conversation.
The research highlights that while countries are getting better at "generating capacity"—essentially throwing money at workforce development and cyber education—they are failing to incentivize private companies to prioritize security. We see this in the wild every day. A company might have a "robust" security policy on paper, but if there is no legal or financial consequence for failing to implement basic hygiene, the policy is just a document.
The "Hand-Wavy" Language Trap
Technical practitioners know that security is found in the details. A strategy that says "the government will improve security" is useless. A strategy that says "all critical infrastructure providers must implement MFA by Q4 2025" is a target.
The research team found that most strategies are "hand-wavy." They use language that sounds good in a press release but provides no cover for the people actually doing the work. If you are a developer or a security engineer working in a critical sector, you need specific, measurable goals. Without them, you are just guessing at what "secure" means.
What This Means for Your Next Engagement
If you are a pentester or a researcher, you should be looking at these national strategies as a map of where the government thinks it is secure, which is almost always where it is most vulnerable. When a strategy document claims a country is "Leading" in critical infrastructure protection but fails to mention specific, enforceable standards for IoT devices or legacy industrial control systems, you have found your entry point.
The lack of specific, measurable outcomes in these documents is a feature, not a bug, for the people who write them. It allows them to claim success without having to do the hard work of enforcement. For us, it means the "national security posture" is often a facade.
Defenders need to push back against this. If your organization is operating under a national strategy, demand the specific, measurable criteria that the researchers found missing. If the strategy doesn't have a timeline, it doesn't have a deadline. If it doesn't have a deadline, it doesn't have a budget. And if it doesn't have a budget, it isn't security—it’s just noise.
The next time you are looking at a client in a regulated industry, don't just check the compliance boxes. Look at the national strategy they are supposedly following. If that strategy is as vague as the ones analyzed in this research, you know exactly where to start your testing. The gaps in the policy are the gaps in the defense.
Target Technologies
All Tags
Up Next From This Conference
Similar Talks

Kill List: Hacking an Assassination Site on the Dark Web

Counter Deception: Defending Yourself in a World Full of Lies




