Why Your SIEM/XDR is Blind to Your Cloud Infrastructure

TLDR: Most security teams treat SIEM/XDR as a black box that magically secures their environment, but default configurations often leave massive blind spots in cloud-native and SaaS services. This talk demonstrates how to move beyond basic endpoint logs by building custom ingestion pipelines using AWS S3, Lambda, and Secrets Manager. Pentesters and researchers should prioritize these gaps during engagements, as they often provide the path of least resistance for lateral movement and data exfiltration.

Security teams often fall into the trap of assuming that because they have a SIEM or XDR platform, they have visibility. In reality, most of these platforms are configured for the lowest common denominator: endpoint agents and basic network traffic. While those are necessary, they are insufficient for modern, cloud-heavy environments. If your detection strategy relies solely on what comes out of the box, you are missing the most critical signals from your identity providers and cloud control planes.

The Visibility Gap in Modern Infrastructure

During a recent engagement, it became clear that while the endpoint coverage was solid, the cloud-native logs were effectively non-existent. We were looking at a classic Security Logging and Monitoring Failure, where the organization had plenty of data but no way to correlate it. The problem is that SIEM/XDR vendors prioritize the "easy" logs—the ones that fit neatly into their pre-built connectors. They rarely prioritize the logs that actually matter for detecting sophisticated Identity and Access Management (IAM) abuse.

When you perform a threat model, you have to ask yourself: where does the attacker actually go? They aren't just hitting your endpoints. They are hitting your Okta instance, your Office 365 audit logs, and your cloud storage buckets. If those logs aren't being ingested, normalized, and alerted upon, you are effectively blind to the most common attack vectors.

Building Custom Ingestion Pipelines

If your SIEM doesn't support a specific SaaS or cloud service out of the box, you have to build the bridge yourself. The most effective way to do this is by leveraging serverless architecture. For example, if you need to ingest third-party Web Application Firewall (WAF) logs that your SIEM doesn't support, you can route them to an S3 bucket and trigger a Lambda function to process them.

The mechanical flow for a custom integration typically looks like this:

1. Ingestion: The source service writes logs to an S3 bucket.
2. Processing: An S3 event trigger fires a Lambda function.
3. Normalization: The Lambda function parses the raw logs into a format the SIEM understands, such as Common Event Format (CEF).
4. Delivery: The function pushes the normalized data to the SIEM API using OAuth for authentication.

Here is a simplified conceptual example of how you might handle the parsing logic in a Go-based Lambda function:

// Simplified parser logic for custom log ingestion
func parseLog(data []byte) (NormalizedEvent, error) {
    var raw map[string]interface{}
    json.Unmarshal(data, &raw)
    
    event := NormalizedEvent{
        Timestamp: raw["time"].(string),
        User:      raw["user_id"].(string),
        Action:    raw["event_name"].(string),
        Source:    raw["ip_address"].(string),
    }
    return event, nil
}

The primary challenge here isn't the code; it's the lack of documentation and the difficulty of handling failure states. If your Lambda function fails to push to the SIEM, you need a robust retry mechanism, or you will lose the data entirely. Furthermore, you must be meticulous about date conversions. I once spent an entire day debugging a detection rule because my custom parser was outputting timestamps in a local timezone that the SIEM didn't expect, effectively rendering the logs useless for correlation.

Why Pentesters Should Care

For those of us on the offensive side, these custom integrations are a goldmine. When you are performing a red team engagement or a penetration test, look for the "unsupported" systems. If an organization has a custom-built internal tool or a niche SaaS product that isn't integrated into their SIEM, that is your playground. Because the blue team has no visibility into those logs, you can often perform reconnaissance, credential harvesting, or data exfiltration with near-zero risk of triggering an alert.

When you find these gaps, document them. A report that says "I bypassed your detection by using an unmonitored SaaS platform" is far more valuable to a client than a report that just lists a few missing patches. It forces them to think about their detection coverage as a holistic system rather than just a collection of tools.

The Defensive Reality

Defenders need to stop treating SIEM/XDR as a "set it and forget it" solution. You must perform regular threat modeling to identify what is missing. If you don't have visibility into your identity provider, you don't have security. If you don't have visibility into your cloud storage access patterns, you don't have security.

Start by identifying your most critical assets and working backward. What logs do you need to detect an unauthorized access attempt on those assets? If your current SIEM doesn't ingest those logs, build the pipeline. It is better to have a custom, slightly brittle ingestion script that gives you visibility than to have a "supported" connector that gives you nothing but noise. Stop relying on the vendor's roadmap to secure your environment. Take control of your data, normalize it, and start building detections that actually reflect the reality of how your infrastructure is being attacked.