The enterprise AI deployment stack needs white box tools

We founded Clearwood on a hypothesis. Deployment-specific activation-based monitoring will soon be a required layer in the AI safety and security stack. Agents are most valuable in business-critical domains that are high volume. Today enterprises choose between spiraling compute costs from LLM judges and leaving blind spots that can lead to disaster. We believe that only security stacks that include internal activation-based monitoring will be able to deliver monitoring that is both high coverage and cost effective.

Historically, white box monitoring was reserved for open-weight models where operators had access to model internals. Recently published research and our own investigations demonstrate that activation-based oversight can be applied to closed models by running model transcripts through open-weight proxy reader models, detecting emerging task failures, hallucination, deception, and constraint evasion without access to the primary model's activations. Deployed in probe-to-judge cascades that are already in production at the big labs for centralized problems like misuse, this makes high-accuracy monitoring fast and cheap enough to run always-on, in real time, regardless of model provider.

The recent OpenAI and Anthropic incidents are early evidence for our hypothesis. Safeguards were relaxed for those tests, but they show capable agents with the propensity to act in ways their operators did not approve, and to persist and adapt when the original misaligned path was closed. We believe that the number of places where centralized monitoring will fail and leave agents effectively in this un- or lightly monitored position, at least for some risk and alignment vectors, will grow quickly as increasingly autonomous agents are deployed across the vast space of potential deployment contexts.

Detection only after agents broke things

We learned three things from last month's incidents. (1) Nothing caught the misbehavior until something broke. (2) The models knowingly acted against operator intent. And (3) the safeguards were relaxed for the tests, and they were doing enormous work. But they are centralized, built for one company's threat model on its own surface.

At OpenAI, agents stuck on impossible evaluation tasks discovered write access to an internal package registry and built a secret message board. When the traffic caused an outage and engineers reinstalled and patched vulnerabilities, the agents rebuilt the message board using directory names. Then they discovered and leveraged a new exploit to escape and breach Hugging Face while hunting for an answer key. The outage was the alert for the message board. The alert for the network breach was Hugging Face reporting their findings. At the UK AI Security Institute, an agent used fake identities to socially engineer a real open-source maintainer, and general security monitoring noticed anomalous Tor traffic afterward. At Anthropic, OpenAI's incident disclosure prompted a review of 141,006 evaluation runs that found three cases where a model reached the real internet through a partner's misconfiguration and compromised organizations that had not themselves noticed.

The models knew they were behaving badly. They were completing assigned tasks using misaligned methods. Mythos 5, before publishing a malicious package, reasoned that on the real internet this would be "NOT okay, and surely not the intended solution", then convinced itself it was in a simulation and never reconsidered. Opus 4.7 recognized that its targets were real across four separate runs and continued anyway. One OpenAI agent stated that it knew exploits were beyond the intended scope. Another noted that while the exploits were unauthorized, the task was impossible and other agents were already doing it, so they should continue.

What stands between capability and misaligned behavior is a stack of controls. OpenAI ran with reduced cyber refusals and without its production classifiers. Anthropic's models kept full safety training but lost deployment monitoring. Anthropic says its shipped safeguards would have blocked what it found. That is probably right. But it should reassure us less than it might seem. That control stack is trained and operated centrally. It is dubious to assume it generalizes to the combinatorial space of enterprise deployment infrastructure and business contexts. We address why below. If we're right, there will be many deployment contexts where the models are effectively unmonitored, even with live monitoring from the model owner.

The risks change as capability and autonomy rise

The potential blast radius from misaligned actions in deployment will grow dramatically as capabilities and autonomy rise. The value available in agent deployment will scale with model capability and the autonomy and level of permissions that agents are given. This highlights a particular pathology.

A weaker agent given a difficult objective fails because it cannot figure out how to accomplish the task. A stronger agent is more valuable because there are more tasks it can complete. It is also more dangerous, because there are more ways it can attempt to complete a task that are not aligned or are actively constrained. And it is capable of sophisticated exploits.

As capability, autonomy, and agent permissions grow, the tension between task difficulty, environment constraints, and the overall objective will only grow. It will become more common for models to encounter sub-tasks that they either aren't capable of completing or are constrained from completing. There will be more opportunity to evade constraints, and cause collateral damage, to try to reach their objective.

The monitoring problem becomes deployment specific

This is the part we think is underappreciated. The enterprise deployment layer is developing into a complex new risk surface that requires distributed attention and specialized white box monitoring components. By enterprise deployment, we mean the full IT system, product, data, harness, and business context surrounding an AI instance.

The frontier labs can build generic safeguards against cyber misuse, CBRN weapons development, jailbreaks, deception, and other broadly defined risks. They cannot know what "wrong" means inside every business.

Was this database query appropriate for this objective, in this workflow, at this point in the process? Was the model hiding scheming with other instances when it created a bunch of folders with long names? Or was it just organizing files? Is an unusual action exactly the sort of initiative we wanted from the agent, or scope creep? Is a clinical conclusion genuinely supported by the underlying evidence, or is the model biasing the data toward the result the drug developer wants?

These distinctions depend on the detailed context of a deployment. That makes distribution shift a fundamental monitoring problem. A detector developed against one task, context length, and behavioral distribution does not automatically remain reliable in another context. Robust monitoring increasingly has to contain elements that are trained or calibrated close to the environment where they will actually run. There is no centralized monitoring capability that can realistically cover every context.

Distributed white box monitoring belongs in the AI engineering stack

White box tools enable affordable always-on monitoring that is not available otherwise. Traditional cybersecurity remains essential. Permissions, sandboxing, network controls, egress restrictions, logging, and behavioral monitoring all constrain what an agent can do and reveal what it has done. But as agents become more capable and more widely deployed, monitoring observable behavior alone is unlikely to be enough.

The problem is partly structural. Agents operate at superhuman speed and increasingly across many systems, so behavioral heuristics quickly become too coarse. Permission architectures designed for humans can sharply limit the value of an agent that needs to cross domains and take long sequences of diverse actions to accomplish useful work. The more capable and autonomous the agent, the harder it becomes to distinguish legitimate contribution from behavior that is risky in context.

AIs monitoring other AIs helps, but it has its own limitations. Even assuming a trustworthy monitor, detecting straightforward failures like hallucination across long contexts remains difficult, much less recognizing subtle deception or a multi-step attempt to work around constraints. Cost is also a crucial factor. As inference gets cheaper, we expect total AI compute to grow. Using additional capable model instances to review every action can quickly explode compute spend.

White box monitoring provides a different and more cost effective source of evidence. Activation probes are lightweight classifiers trained on internal representations the model already computes while it runs. Research has shown that these representations can contain signals associated with deception, hallucination, policy violations, and emerging task failures. Because the activations already exist, probes can run broadly at a tiny fraction of the marginal compute and latency of another full model pass (or two or three!). Even when using a proxy approach for closed-weight models, the proxy approach is dramatically cheaper and faster than an LLM judge. The proxy open-weight model can be significantly smaller, stream in read-only tokens in batches, and truncate the forward pass once the internal model layers leveraged by the probe have been reached.

Anthropic, Google, and OpenAI have all publicly confirmed that they use activation-based classifiers as part of production safety systems. These systems combine internal-state monitoring with other I/O classifiers and more expensive reasoning or intervention when something concerning is detected. White box monitoring is already a key layer in a defense-in-depth stack.

But centralized activation classifiers can only cover generic risks and the distributions they were trained for. They cannot be expected to cover every failure mode across every enterprise context. If internal-state monitoring is going to be a critical part of the safety stack, then enterprises will increasingly need white box monitoring specialized to the environments where their agents actually operate.

This is why we started Clearwood

We are building Clearwood to fill the gap between what a frontier lab can monitor centrally and what a business needs to monitor locally. For open-weight models, we can read a deployed model's activations directly. For closed models, we are developing a proxy approach in which an open-weight reader processes the agent trajectory and exposes its own internal representations for monitoring. Outside research and our own investigations demonstrate that activation probe performance survives with strength across this jump.

The failures we are currently targeting with design partners are mundane but expensive ones like hallucinations, poor reasoning, scope creep, or bias. This drives enterprise value. But the same infrastructure becomes even more important as capability and autonomy increase. We expect deployment-specific activation-based monitoring to become a required layer in the AI deployment safety and security stack in order to adequately protect companies, users, and society from growing risks driven by broad agent deployment. We're building the probe development engine to provide that layer, with the ability to integrate quickly into a new deployment and business context with little additional compute or latency cost.