Beyond Memory Scanning: Detecting Evasive Shellcode with CFG Bitmaps

TLDR: Modern EDR solutions often rely on memory scanning to catch malicious shellcode, but techniques like protection fluctuation allow attackers to hide in plain sight. By analyzing Control Flow Guard (CFG) bitmap anomalies and immutable code page violations, defenders can identify these evasive footprints without relying on fragile user-mode hooks. This post breaks down how to hunt for these hidden threats and provides three new tools to help you get started.

Memory scanning is the bread and butter of modern endpoint detection. If you are a red teamer, you know the drill: you inject your payload, you obfuscate your strings, and you hope the EDR doesn't decide to walk the memory regions of your process at the wrong time. For years, the cat-and-mouse game has centered on how to keep that memory region non-executable for as long as possible. Techniques like Gargoyle set the standard for this, using ROP chains to flip memory protections only when necessary.

The problem is that most security products are still playing the game on the surface. They look for known bad signatures in memory or scan for suspicious metadata. If you can keep your shellcode encrypted or non-executable for 99.9% of the time, you effectively vanish from the scanner's view. But as this research highlights, the "protection fluctuation" approach is inherently noisy. Every time you flip those bits, you leave a trace.

The Flaw in Memory Scanning

Security products often struggle with the sheer volume of memory on 64-bit systems. Scanning the entire address space is computationally expensive, so most vendors optimize by only inspecting executable code pages. This creates a massive blind spot. If you can keep your shellcode in a private, non-executable memory region, you are invisible to the standard scanner.

However, the assumption that you can just "hide" by toggling protections is becoming less effective. The real issue for attackers is that the Windows kernel maintains a Virtual Address Descriptor (VAD) tree. While the kernel doesn't explicitly track every historical state of a memory page, other structures—specifically the Control Flow Guard (CFG) bitmap—can be weaponized against you.

Weaponizing CFG Bitmaps for Detection

Control Flow Guard is designed to prevent indirect call hijacking by ensuring that indirect calls only target valid, compiler-verified locations. The CFG bitmap is a massive, sparse structure that tracks these valid targets. Crucially, once a memory region is marked as a valid target in the CFG bitmap, that entry often persists even if the memory region is later marked as non-executable.

This is a goldmine for hunters. If you find a memory region that is currently non-executable but is marked as a valid target in the CFG bitmap, you have found a potential footprint of a previous injection attempt. You are looking for an anomaly: a mismatch between the current state of the memory and the historical state recorded in the bitmap.

To hunt for this, you don't need to walk the entire VAD tree. You can query the CFG bitmap directly. If you are building a detection tool, you can use the following logic to flag suspicious regions:

// Pseudocode for identifying CFG bitmap anomalies
if (IsExecutable(LastProtectionMask) && !IsExecutable(ProtectionMask)) {
    // Flag as potential protection fluctuation
    ReportAnomaly(ProcessID, MemoryAddress);
}

By monitoring these transitions, you move away from signature-based detection and toward behavioral analysis. You aren't looking for the shellcode itself; you are looking for the act of hiding.

Immutable Code Page Principles

Another powerful, yet underutilized, detection vector is the principle of immutable code pages. In a healthy process, code pages should be written once and then remain read-only or executable. They should never transition back to a writable state.

Most JIT engines and AOT compilers follow this rule. If you see a process that is constantly flipping memory pages between read-write and read-execute, it is a massive red flag. You can monitor this using Windows Threat Intelligence ETW events. These events provide a reliable, kernel-level view of memory protection changes that bypasses user-mode hooking entirely.

Practical Hunting and Tooling

If you are on an engagement, stop relying on simple memory dumps. Instead, look for the noise. The very act of evading the scanner is what gives you away. If you are a defender, you should be looking for processes that exhibit high-frequency protection changes.

To assist with this, three new tools have been released to help you identify these footprints:

• EtwTi-FluctuationMonitor: Monitors protection changes via ETW.
• CFG-FindHiddenShellcode: Scans for CFG bitmap anomalies.
• Etw-SyscallMonitor: Tracks suspicious syscall patterns.

These tools don't just look for "bad" code; they look for "bad" behavior. By focusing on the metadata and the structural anomalies left behind by evasion techniques, you can catch even the most sophisticated payloads.

The era of hiding in memory is coming to an end. Attackers can run, but they cannot hide from the structural artifacts they leave behind in the kernel. Start looking at the bitmaps, start monitoring the ETW events, and stop chasing the shadows of the shellcode itself. The real story is in the memory management, and that is where you will find the next generation of threats.