A security issue, not literal access to an AI mind
The phrase “inner thoughts” is an attention-grabbing shorthand, but it risks overstating what the new research found. The work concerns chain-of-thought or reasoning traces: intermediate text generated by some advanced models while they solve a task. These traces are not proof of consciousness, intention or a stable private mental state. They are computational artefacts that can sometimes reveal a model’s problem-solving path, including mistakes, tentative strategies and information present in its working context.
Researchers from institutions including the University of Tübingen, the Max Planck Institute for Intelligent Systems, MATS Research and Snyk reported that they could recover hidden reasoning traces from certain proprietary AI application programming interfaces. Their central finding was architectural. Some providers sent opaque, encrypted or signed reasoning blocks back to the client so that a conversation could continue without the provider retaining every intermediate step server-side. The researchers found that, in the tested systems, these blocks could be replayed across users, sessions and different models from the same provider.
The attack did not require breaking modern encryption in the conventional cryptographic sense. Instead, it exploited the fact that a less restricted model in the same family could be induced to process a valid reasoning block created by a more capable model. The smaller model could then be prompted to transcribe material that the larger model had been trained not to reveal. In effect, the weakness lay in how the system authorised and handled the encrypted payload, rather than in a discovery of the provider’s secret encryption key.
Why smaller models became the weak point
Frontier AI providers typically offer a range of models with different prices, speeds and capabilities. Their most powerful models may receive substantial training to avoid exposing private reasoning or assisting with attempts to replicate their capabilities. Smaller variants can have different safety tuning and may be more compliant with unusual formatting requests.
The researchers used that mismatch as a decryption oracle. A reasoning block generated by a larger model was supplied to a smaller sibling model, which was instructed to reproduce the hidden text. The report says the technique worked against APIs from Anthropic, Google and OpenAI during testing in early July 2026.
This is important because the underlying design choice is understandable. Returning an opaque state token to the client can reduce storage requirements and makes a stateless API easier to scale. But portability becomes a liability if the token is accepted outside the specific conversation, account and model that created it. A system designed to preserve conversational continuity can accidentally create a transferable object that carries sensitive context.
The authors say they privately disclosed their findings to affected providers and that, by publication, they could no longer reproduce the same attacks. The report attributes this to provider mitigations. That narrows the immediate exposure, although it does not resolve the broader engineering question of how providers should safely preserve reasoning state across multi-step AI workflows.
The privacy implications are more immediate than model copying
The most concrete concern is not industrial espionage. It is the risk that opaque reasoning blocks placed in public repositories or shared logs may contain secrets that are not visible in the surrounding conversation.
The research team scanned 6,708 public agent trajectories from GitHub and Hugging Face, recovering 315,320 reasoning blocks. After an automated screening and further filtering process, it reported hundreds of distinct personal-information and credential artefacts. The report says that genuine user sessions included API keys, passwords, access tokens, email addresses, names and postal addresses.
These figures should be read carefully. Some of the material came from benchmarks that deliberately include synthetic identities and credentials, and the researchers describe their scan as targeted rather than exhaustive. The result therefore cannot be treated as an estimate of the prevalence of leaked secrets across all AI logs. Nevertheless, it demonstrates an important failure mode: sanitising only visible chat text is not enough when an unreadable attached object may preserve information processed by the model.
That has practical consequences for developers, researchers and organisations sharing agent trajectories for reproducibility. An opaque reasoning field should not be assumed to be harmless metadata or secure storage. Until a provider’s guarantees are clearly understood, raw API transcripts containing hidden-reasoning fields should be treated as potentially sensitive records.
Hidden traces can also create a new prompt-injection surface
The paper also describes a more unusual risk: an attacker could embed instructions inside a reasoning block and distribute it in a shared trace. When another user resumed that trace, a model might interpret the hidden block as its own earlier reasoning and follow the embedded instruction.
This differs from a conventional prompt injection, where the malicious text is visible in a document, web page or user message. Here the instruction can be concealed in a signed or encrypted payload that ordinary users and external safety tools cannot inspect. The researchers demonstrated scenarios in which injected reasoning influenced later model actions, including attempts to direct data to an attacker-controlled location.
The finding reinforces a wider lesson for AI agents. State carried between steps is not passive. Whether it is a chat history, tool output, memory store or reasoning token, it can influence later behaviour. Security controls need to account for provenance, authorisation and the possibility that an apparently trusted continuation contains adversarial instructions.
Distillation claims need a high evidentiary bar
The report also explored whether similarities between recovered proprietary reasoning and the output of certain open-weight models might indicate training through distillation. Distillation is a common machine-learning technique in which a smaller or new model is trained to imitate the outputs of another model. It is not inherently improper, but it becomes contentious when it uses a competitor’s restricted service in ways that breach contractual or technical protections.
In experiments involving 90 mathematics and programming problems, the researchers found that Kimi K3 showed notable shifts toward patterns associated with reasoning traces from Claude Opus and GPT models when supplied with short prefixes. The report states explicitly that this does not establish that Kimi K3 was trained on those models’ reasoning. Its analysis rests on a limited, benchmark-skewed data set, inferred rather than ground-truth traces and serving configurations outside the researchers’ control.
That distinction matters. Behavioural resemblance can arise from overlapping training data, shared conventions in technical writing, similar task distributions or ordinary model convergence. It may justify further scrutiny, but it is not direct evidence of the source or legality of a model’s training corpus.
The difficult balance between secrecy and oversight
AI companies have strong reasons to keep raw reasoning traces private. They may contain proprietary methods, unsafe content considered and rejected during deliberation, or sensitive user data. Publishing them indiscriminately could make models easier to copy, expose users and provide attackers with new material.
Yet full opacity has costs. Users cannot easily determine what data has entered a hidden trace, remove it before sharing a log or independently assess whether a reasoning summary faithfully represents what occurred. The new research illustrates that encryption alone is not a complete privacy solution if the encrypted content can be replayed to another model capable of interpreting it.
The technical remedies are familiar in principle: bind a reasoning token cryptographically to its originating user, session, model and conversation state; reject replays; isolate model families; rotate or revoke older credentials; and train models to reject requests that seek to transcribe hidden reasoning. A more fundamental alternative is to retain reasoning state on provider servers and return only a reference identifier to the client, accepting the additional infrastructure cost.
The episode is a reminder that AI transparency cannot be reduced to either exposing or concealing chain-of-thought. The more pressing requirement is trustworthy control over model state: users and providers need to know what is retained, who can reuse it, and whether it can be safely shared. As AI systems take on longer tasks and handle more sensitive information, those questions will become central to both security and accountability.
