How Misconfigured Azure Machine Learning Compute Instances Lead to Full Environment Compromise

TLDR: Researchers at Black Hat 2023 demonstrated how default configurations in Azure Machine Learning (AML) compute instances can leak storage account access keys and JWT tokens through system logs. These vulnerabilities, tracked as CVE-2023-23382 and CVE-2023-28312, allow attackers to escalate privileges from a standard user to the host level. Pentesters should prioritize auditing these managed services for exposed environment variables and log files, while defenders must enforce network isolation using private endpoints.

Cloud providers often market managed services as "secure by default," but the reality for researchers is that these abstractions frequently hide complex, insecure configurations. The recent research presented at Black Hat 2023 regarding Azure Machine Learning (AML) is a masterclass in why you should never trust the "managed" label. When you spin up an AML compute instance, you are essentially deploying a managed Ubuntu-based virtual machine. The convenience of these instances—pre-installed with tools like Jupyter, Conda, and Docker—creates a massive, often overlooked, attack surface.

The Mechanics of the Leak

The core issue identified in this research stems from how AML handles the mounting of Azure File Shares. To simplify the developer experience, the service automatically mounts these shares to the compute instance. During this process, the system executes a mount command that includes the storage account name and the primary access key.

Because this command is executed via sudo during the startup task, it is logged in plain text within the standard error and standard output files located in /mnt/batch/tasks/startup/. Any user with access to the compute instance—or any process that can read these logs—can trivially extract these credentials.

# Example of the command structure found in logs
sudo mount -t cifs //storageaccount.file.core.windows.net/share /mnt/batch/tasks/shared/LS_root/mounts/clusters/aml2/code -o vers=3.0,username=storageaccount,password=PRIMARY_ACCESS_KEY,dir_mode=0777,file_mode=0777,noperm,fsc,serverino

This is a classic case of OWASP A02:2021 – Cryptographic Failures, where sensitive data is handled insecurely. Once an attacker has the storage account key, they have full control over the data stored in that account, including datasets, Python scripts, and Jupyter notebooks.

Escalation via Insecure Service Exposure

Beyond simple credential theft, the research uncovered a more dangerous path to host-level access. The AML service runs various agents to manage the compute instance, and one of these agents, dsimountagent, was found to be listening on port 46802 on all interfaces.

This service exposed several HTTP endpoints that required no authentication. By querying these endpoints, an attacker could list all running systemd services and, more importantly, view their logs. Since the jupyter.service was running as root, an attacker could use these endpoints to monitor the commands executed by the data scientist in real-time. If a user ran a command like sudo cat /etc/shadow, the output—and the command itself—would be captured in the logs, which the attacker could then pull via the exposed API.

This vulnerability, CVE-2023-28312, highlights the danger of OWASP A05:2021 – Security Misconfiguration. By combining this information disclosure with a container escape technique—using tools like docker-escape-tool—an attacker can move from a low-privileged container to the underlying host.

Real-World Implications for Pentesters

During a red team engagement or a cloud-focused penetration test, your primary goal when encountering AML is to determine if the environment is properly isolated. If you find yourself with access to a compute instance, your first step should be to check for the existence of these log files. Do not assume that because the service is "managed" by Microsoft, it is hardened against internal threats.

If you are performing a black-box test, look for exposed endpoints on non-standard ports. Use nmap to scan the internal network range of the VNet. If you find port 46802 open, you have a direct path to information disclosure. The impact here is not just data theft; it is the ability to maintain stealthy persistence within the environment by monitoring the legitimate user's actions and waiting for them to provide the necessary credentials or access.

Hardening the Environment

Defenders must move away from the default "public" network configuration for AML workspaces. The most effective mitigation is to implement Azure Private Link, which ensures that all traffic between the user and the compute instance stays within the private network.

Furthermore, you should strictly adhere to the principle of least privilege. Do not grant users broad permissions to manage compute clusters if they only need to run specific jobs. Regularly audit your environment for any instances where storage account keys might be logged or stored in environment variables. If you are using custom Docker images for your training environments, ensure they are stripped of unnecessary tools and that the services running inside them are not exposing management ports to the internal network.

Security in the cloud is a shared responsibility. While the provider is responsible for the underlying infrastructure, you are responsible for the configuration of the services you deploy. These vulnerabilities serve as a reminder that even in the most sophisticated cloud environments, the basics of secure logging and network isolation remain the most effective defenses against sophisticated attackers.