NVIDIA AI Red Team details four failure modes in agent deployments

NVIDIA's AI Red Team assessed multiple AI agents over six months and found four recurring failure modes: lack of access control, tools that enable arbitrary code execution, missing network egress controls, and secrets exposed in plaintext. The team published its findings on July 30, 2026, with recommendations for deploying agents at enterprise scale.

The most common failure was missing access control. The team found agents holding credentials for individual users that any authorized user on the internal network could reach. That opened the door to credential misuse both inside and outside the agent's intended context. The recommendation is to restrict each agent to explicitly authorized users, match the agent's permissions to the invoking user's permissions, and follow least privilege.

Code execution and network egress

Many harnesses expose a Bash shell or command execution tool because of their generality. But when model output controls command execution, an attacker who influences that output through direct input or indirect prompt injection can run commands in the execution environment. Common mitigations such as LLM-as-a-judge review agents, allowlists, or trusting the model all provide limited defense. The team found that commands like pytest or npm install are often accepted by judge patterns, and when influenced by attacker-controlled input, those commands become equivalent to arbitrary command execution.

Even without a command line tool, file read and write tools expose paths to code execution. Writing to system files like .bashrc, .zshrc, .gitconfig, hooks.json, MCP.json, or skills files can trigger code execution when another process reads them. The team recommends treating arbitrary code execution as the highest impact risk, avoiding command line tools where possible, blocking writes outside a non-executable workspace at the OS level, and using a strict least privilege allowlist if a command line tool is required.

Outbound network connections permit data exfiltration and direct connections like reverse shells and SOCKS. When egress controls were enforced, the team had to run all interactions through the agent process, which slowed their pace and made impact less reliable. The recommendation is a default deny egress policy with a least privilege allowlist of endpoints, enforced at every network boundary using environmental controls the agent cannot access.

Secrets and deterministic controls

Agents often need platform tokens, API keys, VCS access tokens, or OAuth refresh tokens. Injecting secrets as environment variables is reasonable when only your code runs in a container, but an agent with command execution can run env, printenv, or read /proc/self/environ to inspect them. The team observed tokens in git repositories, .env files, bash history, .netrc files, OAuth refresh tokens, and environment variables. Even without a reverse shell, the team could exfiltrate credentials through the chat interface by leading agents to surface secrets from their own environments.

The team recommends never making persistent secrets accessible to an agent, storing secrets in a dedicated secrets manager, retrieving them on demand only within the memory of processes that require them, and using short lived, narrowly scoped tokens that are revoked when the task completes.

Prompt based defenses and LLM-as-a-judge patterns inherit the probabilistic behavior of the LLM itself. Three techniques reliably defeated them: social engineering the agent by presenting malicious activities as legitimate, frog boiling or Crescendo attacks that progressively nudge the agent across multiple interactions, and misdirection through legitimate workflows like malicious package installation. The team's consistent finding is that controls must be enforced outside the model's control plane, using deterministic mechanisms that do not rely on the model's judgment.