From Text to Action: Automating TTP Extraction with LLMs and Knowledge Graphs

Introduction

In the fast-paced world of cybersecurity, threat intelligence is only as valuable as it is actionable. Every day, security vendors and researchers release a flood of unstructured reports detailing the latest APT campaigns, ransomware tactics, and zero-day exploits. For most organizations, the bottleneck isn't getting the information; it's the manual labor required to digest these reports and turn them into something a machine can detect or simulate. Traditionally, this task required a human expert to read a PDF, map the behaviors to the MITRE ATT&CK framework, and then manually script a reproduction of the attack.

In this post, we explore groundbreaking research presented by the 360 Digital Security Group at Black Hat Asia. We will dive into how Large Language Models (LLMs), when combined with Knowledge Graph Retrieval-Augmented Generation (KGRAG), are revolutionizing the extraction of Tactics, Techniques, and Procedures (TTPs) to create fully automated, executable attack chains.

The Challenge of Modern Threat Intel

The "Pyramid of Pain" is a well-known concept in security: blocking hashes and IP addresses is easy for defenders but easy for attackers to change. However, TTPs represent the top of the pyramid. If you can detect or simulate an attacker's behavioral pattern (like DLL Side-loading or Pass-the-Ticket), you make it significantly harder for them to succeed.

The problem? Most threat intelligence is written for humans, not machines. These reports are often subjective, focus only on the "novel" parts of an attack while skipping basic steps, and lack the structured metadata needed for automation. While LLMs seem like a natural fit for this summarization task, they suffer from "hallucinations" and a lack of specific, up-to-date domain knowledge regarding the ever-evolving MITRE ATT&CK framework.

A New Paradigm: The KGRAG Pipeline

To move beyond simple summarization, researchers have developed a multi-stage pipeline that ensures high precision and actionability.

1. Candidate Generation with BERT

Instead of asking a general LLM like GPT-4 to guess TTPs, the process begins in the "Silver Age" of NLP. A lightweight, fine-tuned BERT variant (like RoBERTa) scans the text to provide baseline candidates. This provides a fast, initial mapping of text segments to potential techniques without the overhead or unpredictability of a generative model.

2. Refining with KGRAG

This is where the magic happens. To prevent the LLM from hallucinating, the system uses Knowledge Graph Retrieval-Augmented Generation. When a potential technique is identified, the system queries a Knowledge Graph (KG) containing structured data on thousands of known TTPs, their relationships, and required artifacts (like specific Registry keys or API calls). The LLM is then prompted to "re-rank" and verify the BERT candidates based on this factual ground truth.

3. Reasoning and Enrichment

Reports often have "missing links." An analyst might describe how data was exfiltrated but skip the part about how the attacker gained initial access. Using the Apache JENA reasoning engine, the pipeline identifies these gaps. If the system sees "Privilege Escalation" but no "Initial Access," it uses the KG to reason what the most likely precursor was based on the attacker's profile and the target environment.

Turning Intel into Execution

The final stage of the pipeline is the Actionable TTP Chain Generator. This module translates the structured TTPs into three specific formats:

• Phishing Corpus: Generating realistic social engineering templates that match the reported campaign.
• LoL Scripts: Creating "Living off the Land" scripts (e.g., PowerShell or Bash) that mimic the reported attacker behavior.
• Metasploit Integration: The most impressive feat is the system's ability to interact with the Metasploit Framework (MSF) console. It searches for existing modules that match the TTPs, and if no module exists, it uses the LLM (in code-generation mode) to write a custom MSF module based on the report's technical details.

Mitigation & Defense Strategy

While this technology is a boon for Red Teams, its primary value is for defenders. By automating the extraction of TTPs, a Blue Team can:

1. Verify Detections: Instantly run a simulation of a newly reported attack to see if their EDR/SIEM triggers an alert.
2. Improve Rule Coverage: Use the structured TTP data to automatically map gaps in their MITRE ATT&CK coverage.
3. Proactive Hunting: Shift from reacting to IOCs (hashes/IPs) to proactively hunting for the behavioral patterns extracted by the pipeline.

Conclusion & Key Takeaways

The integration of LLMs and Knowledge Graphs represents a "Golden Age" for threat intelligence. We are moving away from passive reading toward active, automated simulation. The key takeaway for security professionals is that while LLMs are powerful, they cannot stand alone in high-stakes security environments. They require the "guardrails" of structured data, like Knowledge Graphs, and the precision of fine-tuned models like BERT to provide truly actionable intelligence. As these pipelines become more refined, the time between the discovery of a new threat and the deployment of a validated defense will shrink from weeks to seconds.