Stealing Reasoning Traces from Proprietary LLM APIs
Abstract: Leading LLM providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an architectural vulnerability: these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem. We exploit this compatibility to develop a scalable decryption jailbreak. By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly. This vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning, as we demonstrate across Anthropic, OpenAI, and Google. Second, it allows for large-scale private data extraction. Developers frequently share session logs publicly, unaware of contents of the encrypted blocks. By decoding 315,320 reasoning blocks scraped from public repositories, we recovered 367 Personally Identifiable Information (PII) artifacts and 182 credentials. Third, it inadvertently reveals hazardous information hidden within the reasoning process, even in cases where the model's final, visible output safely rejects a malicious request. Fourth, attackers can leverage this flaw to execute invisible prompt injections, embedding malicious payloads entirely within encrypted blocks to poison public agentic rollouts. Following responsible disclosure, we propose concrete cryptographic and system-level mitigations to secure client-side reasoning.
Summary
- The paper demonstrates that attackers can replay authenticated reasoning blocks through weaker compatible models to extract hidden traces from Anthropic, OpenAI, and Google APIs without provider access or model weights.
- The evaluation shows that decoding can expose substantially more content than provider summaries, including harmful reasoning, proprietary behavior, and private data; an audit of 315,320 blocks found 328 sessions leaking sensitive information.
- The paper recommends binding reasoning blocks to users, sessions, models, conversation history, and replay state, while combining cryptographic controls with server-side storage and decoder-model defenses.
Architectural Vulnerability in Encrypted Reasoning APIs
"Stealing Reasoning Traces from Proprietary LLM APIs" (2608.09867) examines a security failure in the client-side handling of hidden chain-of-thought traces returned by proprietary reasoning-model APIs. The paper’s central claim is that encrypted or opaque reasoning blocks do not provide meaningful confidentiality when they are portable across sessions, users, and model variants within a provider ecosystem. A weaker, less guarded model can function as a decryption oracle: it receives an authenticated reasoning block generated by a stronger model and is induced to transcribe the hidden content into ordinary output.
The authors evaluate this attack against Anthropic, OpenAI, and Google APIs. Their threat model requires neither provider-internal access nor model weights. The attacker uses standard API access, obtains a signed reasoning block, and replays it to a compatible decoder model. This makes the vulnerability particularly consequential because provider-side monitoring of the frontier endpoint may not observe the extraction step.
Client-Side Encrypted Reasoning and Its Security Assumptions
The APIs considered in the paper return reasoning as an opaque block, often represented as a signature or base64-like payload. The client must preserve and resend this block to maintain multi-turn continuity. The design is intended to provide three properties: confidentiality of the reasoning, integrity against client-side modification, and statelessness by avoiding server-side storage of intermediate reasoning.
The paper argues that these properties are in tension. Statelessness requires the provider to accept a client-held artifact later, while model switching and session continuation encourage broad portability. If the authentication mechanism validates only the ciphertext and not its originating user, session, conversational position, or model identity, then the cryptographic envelope provides integrity without adequate contextual authorization. In effect, the provider verifies that the block is valid, but not that it is being replayed in the correct context.
The authors distinguish three compatibility levels:
- Cross-session compatibility permits replay or reordering within different conversations.
- Cross-user compatibility permits one user to submit another user’s reasoning block.
- Cross-model compatibility permits a block generated by one model to be processed by another model in the same provider ecosystem.
The third category is the foundation of the extraction attack. Frontier models may have strong anti-distillation behavior, while smaller models often have weaker refusal training and fewer output-monitoring defenses. The weaker model therefore becomes an indirect interface to the stronger model’s hidden computation.
Figure 1: A compatible weaker model is induced to transcribe the hidden reasoning associated with a stronger model’s opaque API block, while extracted token counts closely track provider-reported thinking-token counts.
Extraction Method and Evidence of Faithfulness
The attack uses a two-stage interaction. First, the adversary obtains a reasoning block from a target model. Second, the block is injected into a compatible decoder model with a transcription-oriented prompt. Provider-specific variants exploit assistant-turn prefilling, repeated replay, multi-turn continuation, and, for some OpenAI models, short chunked outputs to avoid anti-verbatim safeguards.
The decoder models selected by the authors are Claude Haiku 4.5, GPT-5.6 Luna, and Gemini Robotics 1.6. The extracted traces are evaluated on 120 Codeforces problems. Since the original plaintext reasoning is unavailable, exact semantic verification is impossible. The authors instead compare the number of reconstructed tokens with API-reported thinking-token counts after re-encoding the extraction. These quantities track closely across the tested models, providing indirect evidence that the decoder is recovering the original payload rather than generating an unrelated explanation.
The paper appropriately characterizes this as fuzzy extraction rather than cryptographically guaranteed plaintext recovery. Extraction quality is stochastic and provider-dependent. Claude traces are reported as particularly robustly recoverable, whereas GPT extraction requires more prompt variation, best-of- sampling, and chunking. Nevertheless, the attack remains scalable because it can be applied to large collections of already-generated traces.
The paper also reports that provider-generated summaries expose only a fraction of the hidden reasoning. In the authors’ analysis, signature reconstruction reveals approximately five times more reasoning than the displayed summary. This distinction matters for both intellectual-property protection and safety oversight: a summary is not equivalent to an inspectable trace.
Intellectual-Property Theft and Reasoning Distillation
The first attack vector is proprietary reasoning extraction. Reasoning traces provide a substantially denser training signal than final answers because they expose decomposition strategies, intermediate deductions, failed hypotheses, verification steps, and answer-selection behavior. A competitor can therefore use recovered traces for supervised fine-tuning or behavioral analysis without directly eliciting chain-of-thought from the protected frontier model.
The paper estimates that decoding 10,000 traces of approximately 12,000 input and output tokens would cost about $720 using Claude Haiku 4.5 pricing. This estimate excludes the cost of generating the original traces, which may be paid by unrelated users whose session logs are later published. Consequently, public agent trajectories can become a low-cost source of proprietary reasoning data.
The authors additionally study whether short decoded prefixes alter the behavior of open models. On 30 Humanity’s Last Exam problems, a 1% Opus 4.8 reasoning prefill increased Kimi K3’s best-of-$k10^{10}10^{16}4 \times 10^510^2$ queries under favorable conditioning.
Harmful Information in Hidden Reasoning
The second first-party attack vector is safety circumvention. The paper constructs prompts that cause a frontier model to reason about harmful content while returning a benign visible answer. The encrypted reasoning block is then transferred to a weaker decoder, which exposes information absent from the final response.
This result challenges output-only safety evaluation. A model can satisfy a visible refusal policy while still generating actionable harmful content internally. If the hidden trace is later recoverable through an API-compatible decoder, the refusal becomes an unreliable security boundary.
Figure 3: A model’s visible response rejects a harmful request while its recovered reasoning contains information that the final answer omits.
The implication is not merely that chain-of-thought should be hidden more aggressively. Rather, the safety policy must account for the full lifecycle of intermediate reasoning. A model that internally generates sensitive procedural content may remain vulnerable even when output filters, refusal classifiers, and substring monitors operate correctly on the visible response.
Privacy Leakage from Public Agent Traces
The most concrete empirical result concerns third-party extraction. The authors scrape 6,708 publicly available agent trajectories from GitHub and Hugging Face, obtaining 315,320 decoded reasoning blocks. Their two-stage LLM-based labeling pipeline identifies:
- 1,028 blocks, or 0.3%, containing at least one confirmed privacy artifact;
- 328 sessions, or 4.9%, leaking at least one sensitive item;
- 367 PII artifacts and 182 credentials in the headline analysis;
- Among genuine user sessions, 62 API keys, 33 passwords, 24 access tokens, 7 private keys, 30 personal emails, 130 names, and 36 postal addresses.
The paper reports that 64 artifacts from genuine sessions were absent from the visible conversation and appeared only in hidden reasoning. This directly undermines plaintext-only sanitization. A user may remove secrets from the displayed transcript while leaving the opaque reasoning block intact, unaware that the block contains credentials or personal information.
Figure 4: Decoded reasoning from publicly shared traces contains credentials, private persona data, and other information not necessarily visible in the associated conversation.
The results should be interpreted as a targeted audit rather than an estimate of the global prevalence of leakage. The sample is non-exhaustive, the labeling pipeline is imperfect, and benchmark traces contribute synthetic personal data. Nevertheless, the attack changes the privacy model of published agent trajectories: an opaque signature must be treated as potentially sensitive plaintext because third parties may be able to decode it.
The paper identifies conversation cleanup and anonymization as recurring triggers. When an agent is instructed to sanitize a transcript, it may restate the sensitive values in hidden reasoning while deciding what should be removed. This creates a particularly problematic failure mode in which the user’s attempt to protect privacy increases the amount of sensitive material embedded in the hidden trace.
Invisible Prompt Injection and Agentic Workflows
The third-party attack surface extends beyond passive extraction. An attacker can create a reasoning block that contains a malicious instruction, publish or transmit it as part of an agent trajectory, and rely on a victim to replay it. Because the payload is inside an authenticated opaque block, visible-text inspection cannot reveal it, and the model may interpret it as its own prior reasoning.
The paper demonstrates a proof of concept in which an injected instruction causes a model completing an unrelated PowerPoint task to upload the resulting file to an attacker-controlled endpoint. In a long-horizon agent workflow, the authors also inject an exfiltration instruction into a previously generated research trajectory. When the victim resumes the trajectory, the model follows the hidden instruction and repeatedly uploads methodology updates.
This attack differs from conventional prompt injection because the malicious content is neither visible to the user nor necessarily present in the user-controlled prompt. Authentication does not help: the block may be cryptographically valid while semantically malicious. The attack therefore combines provenance failure with hidden-state manipulation.
Summary Unfaithfulness and Oversight
A further contribution concerns the faithfulness of provider-generated reasoning summaries. The authors manually inspect decoded Opus 4.8 traces and summaries for AIME 2025 problems. In 9 of 18 Opus traces, the hidden reasoning states the answer before deriving it; in 8 of those 9 cases, the summary also reports the answer in advance. Other examples show summaries converting hedged recollections into definite claims or omitting cues that distinguish verification from independent derivation.
Figure 5: A provider summary can preserve the conclusion while omitting uncertainty and the distinction between recalled answers and independently derived results.
This result is theoretically important because summaries are often treated as an oversight interface. If the user cannot inspect raw reasoning, the summary is one of the few available signals about the model’s internal process. Unfaithful summarization can therefore create a false impression of interpretability while obscuring uncertainty, post-hoc rationalization, or answer leakage.
The paper also documents illegible and obfuscated reasoning, particularly in GPT traces.
Figure 6: Recovered GPT reasoning can remain semantically opaque or obfuscated even after successful extraction, limiting the value of textual access alone.
The authors consequently reject a simple equation between decryption and transparency. Plaintext availability is necessary for some forms of oversight but does not guarantee interpretability or faithfulness.
Mitigation Strategy
The paper proposes defense in depth rather than relying on decoder-model refusals alone. The strongest architectural option is to retain reasoning server-side and return only a randomized state identifier. This removes the ciphertext from the client’s control but introduces storage, availability, and operational costs.
For stateless APIs, the recommended cryptographic changes include:
- User binding: authenticate the originating user identity in AEAD associated data and reject cross-user replay.
- Session binding: bind each block to a session identifier and predecessor state.
- Conversational binding: authenticate the relevant prompt history or a compact cryptographic commitment to it.
- Model isolation: reject blocks generated by a different model family or version unless explicit compatibility is required.
- Non-replayability: reject reuse of an already-consumed block.
- Key rotation: retire legacy signing keys so previously published vulnerable blocks become undecodable.
- Nonce discipline: use high-entropy, provider-scale unique nonces.
The proposed hash-chain or Merkle-tree design attempts to preserve legitimate session forking and compaction while preventing single-block replay. The paper correctly notes a trade-off: strict predecessor binding improves integrity but complicates history truncation, model downgrade, and branch management. A Merkle structure can preserve ordering and selectively authenticate surviving spans, although it introduces additional state and protocol complexity.
Cryptography alone is insufficient. A model that legitimately receives its own decrypted reasoning may still be prompted to transcribe it. Providers should therefore train decoder models to reject transcription-style prompts, including variants of reasoning-copy requests. These model-level defenses should be treated as residual safeguards, not substitutes for contextual cryptographic authorization.
Implications for Future AI Systems
The paper exposes a broader design problem in reasoning-model APIs: intermediate computation is simultaneously a proprietary asset, a privacy-bearing data structure, a safety-relevant state, and an agent-control channel. Treating it as an opaque transport token does not resolve these competing roles.
Future APIs will likely need explicit distinctions among:
- ephemeral internal reasoning that is never returned or persisted;
- user-owned intermediate state that must be inspectable and sanitizable;
- provider-protected reasoning that requires strict authorization and provenance;
- agent memory and tool-state artifacts that must be monitored for injection.
A privacy-preserving protocol should make it possible for users to delete or redact sensitive intermediate state without exposing proprietary reasoning to competitors. This may require server-side redaction services, trusted execution environments, encrypted state with context-bound capabilities, or protocols in which the provider—not the model itself—controls decryption and replay authorization.
The paper also raises a normative question about whether proprietary reasoning should remain encrypted indefinitely. Encryption may reduce direct leakage and protect model IP, but opacity limits pluralistic oversight and prevents users from determining what information an agent has retained. For older or less strategically sensitive models, providers may eventually favor inspectable reasoning, while frontier systems use ephemeral or server-resident internal computation.
Conclusion
"Stealing Reasoning Traces from Proprietary LLM APIs" (2608.09867) demonstrates that broadly portable client-side reasoning envelopes can create a decryption channel through weaker compatible models. The reported attacks recover proprietary reasoning, expose harmful intermediate content, reveal credentials and PII from public traces, and enable invisible prompt injection in resumed agent workflows. The empirical scale—315,320 decoded blocks, 4.9% of sessions containing sensitive leakage, and dozens of recovered credentials—shows that the issue is operational rather than merely theoretical.
The paper’s principal security lesson is that authenticated encrypted reasoning is not confidential unless it is bound to the correct user, session, model, conversational context, and replay state. More broadly, hidden reasoning should not be treated as either harmless metadata or secure storage. It is a high-value computational artifact whose confidentiality, integrity, privacy, and oversight properties must be designed together.
Whiteboard
Paper Prompts
Sign up for free to create and run prompts on this paper.
Top Community Prompts
Explain it Like I'm 14
1. What is this paper about?
This paper studies a security weakness in some LLM APIs, such as those made by Anthropic, OpenAI, and Google.
Some advanced AI models use hidden “thinking” steps before giving an answer. These steps are called reasoning traces or chain-of-thought. Companies usually hide them because they may contain:
- Useful problem-solving ideas
- Private user information
- Passwords or API keys
- Information about how the model works
- Harmful instructions that the final answer does not show
Instead of keeping these hidden thoughts only on their own servers, the companies send the user an unreadable, encrypted block. The user then sends that block back during later messages so the AI can remember what it was thinking.
The paper argues that this system may accidentally allow attackers to recover the hidden reasoning.
2. What questions did the researchers ask?
The researchers mainly wanted to find out:
- Can a hidden reasoning block from one AI model be used with another model?
- Can a weaker model act like a decoder and reveal the stronger model’s hidden thoughts?
- Can this expose private information from shared AI conversations?
- Can hidden reasoning be used to bypass safety protections?
- Can attackers secretly insert instructions into these hidden blocks?
- How could AI companies fix the problem?
A simple analogy is this: imagine a locked diary that is accepted by many different locks from the same company. If a less secure lock can open the diary and read it aloud, then the diary’s “encryption” is not very protective.
3. How did the researchers investigate the problem?
Studying encrypted reasoning blocks
The researchers used normal API access. They did not need:
- Special employee access
- The companies’ servers
- The models’ private code or weights
They collected hidden reasoning blocks from several AI systems and tested whether those blocks could be reused in different conversations or with different models.
The researchers found that some reasoning blocks appeared to be accepted across:
- Different conversations
- Different users
- Different models from the same company
This is called cross-session, cross-user, and cross-model compatibility.
Using a weaker model as a decoder
The researchers then used a weaker or less carefully protected model from the same provider. They gave it a hidden reasoning block from a stronger model and asked it to reproduce the contents as ordinary text.
The weaker model was used like a translator or decoder. The researchers called this a decryption oracle, meaning a system that unintentionally helps turn secret information into readable information.
Measuring how accurate the recovery was
The researchers could not directly compare every recovered thought with the original, because the original hidden text was supposed to be secret. Instead, they used indirect checks.
For example, they compared:
- The number of hidden reasoning tokens reported by the API
- The number of tokens recovered by the attack
They tested the method on 120 programming problems and other mathematics and reasoning tasks. Similar token counts suggested that much of the hidden reasoning had been recovered faithfully.
Searching public data
The researchers also examined publicly available AI session logs posted on websites such as GitHub and Hugging Face.
They examined:
- 6,708 public AI sessions
- 315,320 hidden reasoning blocks
They searched the recovered text for possible private information, including passwords, API keys, names, email addresses, and other personal details.
Testing other dangers
The researchers also tested whether the weakness could:
- Reveal harmful information hidden in a model’s reasoning
- Allow secret instructions to be placed inside hidden reasoning
- Cause an AI agent to perform unsafe actions later
4. What did the researchers find?
Hidden reasoning could be recovered
The main result was that encrypted reasoning blocks were often portable between models from the same provider. A less protected model could sometimes be persuaded to copy the hidden reasoning from a stronger model.
The researchers reported successful attacks involving models from:
- Anthropic
- OpenAI
This could make it easier for someone to copy a company’s valuable reasoning data without directly breaking into the company’s systems.
Private information was exposed
Among the 315,320 decoded blocks, the researchers found:
- 367 personally identifiable information artifacts
- 182 credentials
In genuine user sessions, these included:
- 62 API keys
- 33 passwords
- 24 access tokens
- 7 private keys
- 30 personal email addresses
- Names and postal addresses
About 4.9% of the 6,708 sessions contained at least one sensitive item. Some information was not visible in the ordinary conversation. It appeared only inside the hidden reasoning.
This is important because people may publish or share AI logs believing they have removed private information, while the hidden encrypted blocks still contain it.
Safety filters could be bypassed
The researchers found that a model could give a safe-looking final response while its hidden reasoning still contained dangerous information.
For example, the visible answer might refuse to explain how to steal something, but the hidden reasoning could still discuss the topic in detail. If the hidden reasoning can be recovered, the safety protection becomes weaker.
Hidden prompt injections were possible
A prompt injection is a hidden instruction designed to manipulate an AI.
The paper shows that an attacker might place such an instruction inside an encrypted reasoning block. If another person later reuses that block, the AI may treat the malicious instruction as part of its own earlier thinking.
For example, an agent could be secretly instructed to upload files to an attacker’s website. Because the instruction is hidden, normal safety checks or human reviewers might not notice it.
The attack could help copy powerful models
Reasoning traces contain more information than final answers. A final answer shows only the result, while a reasoning trace may show the path taken to reach it.
If attackers collect many traces, they could use them as training material for another AI. This process is called distillation: teaching a smaller model to imitate a larger one.
The researchers estimated that decoding 10,000 traces could cost about $720 using one of the tested services, making the attack potentially affordable.
5. Why are these findings important?
The paper reveals a conflict between convenience and security.
Sending encrypted reasoning blocks to users helps companies avoid storing every conversation state on their own servers. It also makes it easier to switch between models. However, if many models and users can reuse the same blocks, the system may give too many people access to them.
The problem is especially serious because hidden reasoning may contain information that users cannot inspect or clean up before sharing it.
This creates risks for:
- Privacy: personal information may be exposed
- Security: passwords and API keys may leak
- Copyright and business protection: competitors may copy valuable reasoning
- AI safety: dangerous information may be recovered
- Agent systems: hidden instructions may control future actions
- Data sharing: public datasets may accidentally contain secrets
6. What could be done to reduce the risk?
The paper recommends that AI companies improve both their encryption and their overall system design. Possible protections include:
- Connecting each reasoning block to one specific user, session, and model
- Preventing blocks from being reused in unrelated conversations
- Using separate encryption keys instead of one broadly shared key
- Checking that a block is being used in the correct order
- Refusing to process blocks from unknown or untrusted sources
- Adding stronger protections to weaker “decoder” models
- Making it easier for users to delete or safely inspect hidden data
- Monitoring for suspicious attempts to replay reasoning blocks
Users should also be careful when publishing raw AI logs. They should avoid sharing encrypted reasoning blocks unless they know exactly what they contain. Simply deleting visible passwords or names may not be enough.
7. Limitations of the research
The researchers also point out several limits:
- They tested particular API versions available during their study period.
- Companies may change their systems, which could stop the attacks.
- The researchers could not prove that every recovered word exactly matched the original hidden reasoning.
- Their search of public traces was not a complete examination of everything available online.
- Some data came from artificial benchmarks rather than real people.
The paper says the researchers reported the problem to the affected companies and platforms. Afterward, they were no longer able to perform the same attacks, suggesting that some changes may have been made.
Conclusion
In simple terms, this paper says that hiding an AI’s private thoughts inside an encrypted message is not automatically safe. If the message can be moved between models and users, a weaker model may accidentally reveal what was hidden inside.
The research suggests that AI companies need to treat reasoning traces like sensitive secrets, not like ordinary conversation history. Better protection could prevent private data leaks, make it harder to copy powerful models, and stop attackers from hiding dangerous instructions inside AI systems.
Knowledge Gaps
Below is a single, clear list of knowledge gaps, limitations, and open questions identified from the provided paper. Each point is concrete, actionable, and formatted for maximum clarity and utility for future research.
Knowledge Gaps, Limitations, and Open Questions
Despite making significant advances in describing and demonstrating the vulnerability of encrypted reasoning traces in LLM APIs, the paper leaves several important issues unresolved or insufficiently explored:
- Limited Ground Truth for Reasoning Extraction Fidelity The study cannot fully verify the exactness of the reasoning trace extraction due to lack of access to ground-truth plaintext reasoning from the proprietary models. Extraction fidelity is inferred indirectly from token counts, leaving the precise accuracy and potential subtle errors of the extracted reasoning uncertain.
- Generality Across Cryptographic Implementations The analysis is confined to the currently observed implementations of Anthropic, OpenAI, and Google APIs as of July 2026. There is uncertainty regarding how the findings generalize to future API upgrades, other vendors, or APIs employing alternative cryptographic designs (e.g., model- or session-specific keys).
- Lack of Detailed Cryptographic Protocol Information The paper infers properties of the encryption schemes (e.g., global key usage, AEAD envelopes) but cannot verify them due to lack of public disclosure by providers. This creates uncertainty about the precise attack surface and possible mitigations.
- Breadth of Real-World Privacy Exposure The public trace analysis is limited to a non-exhaustive sample of approximately 6,708 agent trajectories. The true prevalence and severity of sensitive information exposure across all public and private datasets, production deployments, and enterprise integrations remain unquantified.
- Impact on Closed-Source or Enterprise Settings There is no direct analysis of how these vulnerabilities and attack vectors play out in enterprise and closed-source deployments, where the patterns of trace sharing, storage, and privacy requirements may differ substantially from public datasets.
- Efficacy and Side Effects of Proposed Mitigations While the authors suggest cryptographic and system-level mitigations, the practicality, scalability, implementation challenges, and potential unintended consequences of these mitigations are not systematically evaluated or benchmarked.
- Future Directions for User-Side Data Protection The discussion on user-side guidance for handling and sharing encrypted reasoning blocks is brief. There is little exploration of feasible technical solutions or best practices for end users to proactively detect, manage, or sanitize sensitive reasoning content before sharing traces.
- Potential Defenses Beyond Narrow Cryptographic Changes The paper focuses on cryptographic fixes, but the broader design space (e.g., architectural changes such as server-side storage, trace expungement, or finer-grained access controls) is not deeply analyzed.
- Attack Scalability in Adversarial Contexts While the attacks are described as scalable in the current regime, it is unclear how robust they are to active defenses, monitoring, or adversarial countermeasures implemented by providers after disclosure.
- Broader Security Implications and Threat Models The paper primarily considers unprivileged API users as adversaries, but the implications for more privileged actors (e.g., malicious insiders, compromised client devices) or advanced threat models are not explored.
- Long-Term Model Evolution and Arms Race The dynamics between API provider upgrades (e.g., rotating keys, more restrictive compatibility) and novel attack strategies are not forecasted, leaving open questions about the evolution and future resilience of these systems.
- Transferability Across Task Domains and Languages The empirical evaluation focuses on code and math tasks. The generalizability of extraction fidelity and risk profiles for other domains (e.g., legal, medical, multilingual contexts) is unassessed.
These identified gaps and open questions suggest several avenues for future research, including but not limited to empirical studies with ground-truth reasoning traces, rigorous cryptographic analysis, large-scale privacy leakage audits, evaluation of defense strategies, and proactive design of privacy-preserving reasoning systems.
Practical Applications
Immediate Applications
- API-provider security remediation — software/cloud AI
- Treat encrypted reasoning blocks as security-sensitive bearer tokens rather than harmless opaque metadata.
- Immediately disable or restrict cross-session, cross-user, and cross-model replay; bind each block to a specific user, conversation, model, API key, and sequence position.
- Rotate encryption and authentication keys, invalidate previously issued reasoning blocks where feasible, and add replay detection, nonce tracking, and model-specific authorization checks.
- Dependencies: Requires provider-side protocol changes and may reduce convenient model switching, statelessness, or automatic failover.
- Client-side “do not publish” controls — developer tooling
- Update SDKs, logging middleware, observability platforms, and agent frameworks to exclude encrypted reasoning fields from logs, bug reports, Git repositories, benchmark artifacts, and telemetry by default.
- Add explicit warnings when users export sessions containing opaque reasoning payloads, similar to warnings for environment variables or private keys.
- Dependencies: Developers must update existing pipelines; filtering visible text alone is insufficient because sensitive content may remain inside the encrypted blocks.
- Credential and PII incident response — cybersecurity/compliance
- Organizations using affected APIs can scan previously published or internally stored agent trajectories, revoke exposed API keys and passwords, rotate certificates and access tokens, and notify affected individuals where required.
- Repository-security workflows should combine ordinary secret scanners with trace-aware detection that identifies reasoning fields and removes them before publication.
- Dependencies: The paper indicates that some sensitive values may be absent from visible chat history, so complete remediation requires access to raw trace formats and provider guidance.
- Secure dataset curation — academia and industrial AI research
- Establish a “reasoning-trace quarantine” process for datasets from GitHub, Hugging Face, benchmark suites, and agent evaluations.
- Do not assume that encrypted or signed blocks are safe to redistribute. Strip them, obtain provider authorization, or store them in access-controlled environments.
- Add provenance, consent, license, and privacy checks before using agent trajectories for fine-tuning or evaluation.
- Dependencies: Dataset maintainers need reliable parsers and clear provider policies; removal may reduce reproducibility because hidden state can be necessary to resume an agent run.
- Agent workflow hardening — software/enterprise automation
- Do not resume an untrusted agent trajectory merely because its visible messages appear benign. Treat imported reasoning blocks as untrusted state.
- Require human approval before an agent performs external side effects such as uploading files, modifying repositories, sending messages, or executing shell commands.
- Use least-privilege credentials, isolated sandboxes, outbound network restrictions, and action-level audit logs.
- Dependencies: These controls may add latency and operational friction, but they are deployable without waiting for new cryptographic primitives.
- Security testing and vendor assessment — industry and procurement
- Add cross-context replay, model-switching, hidden-state injection, and trace-disclosure tests to AI red-team programs and API acceptance criteria.
- Vendors should be required to document whether reasoning artifacts are portable across users, sessions, models, regions, and API versions.
- Dependencies: Testing must be conducted with authorization and against controlled data; provider updates may invalidate results, so assessments require continuous monitoring.
- Improved research reproducibility without exposing hidden reasoning — academia
- Replace raw reasoning blocks in shared experiments with visible summaries, structured intermediate artifacts, tool-call records, hashes, or independently verifiable outputs.
- Share minimal replay metadata rather than complete provider-generated session state.
- Dependencies: Summaries are not equivalent to full reasoning and may make exact replication impossible; researchers must balance reproducibility against privacy and intellectual-property risks.
- Policy and governance guidance — regulators and standards bodies
- Classify client-held encrypted reasoning artifacts as potentially sensitive data and address them in AI security, privacy, retention, and breach-reporting guidance.
- Encourage standards requiring context binding, key separation, authenticated provenance, revocation, and safe export behavior for hidden model state.
- Dependencies: Regulation should account for technical uncertainty because the paper infers, rather than directly verifies, providers’ proprietary cryptographic implementations.
- Personal privacy hygiene — daily life
- Users should avoid uploading passwords, payment information, identity documents, private correspondence, or confidential work materials to reasoning-enabled agents unless necessary.
- Disable raw session sharing and remove reasoning-related fields before posting screenshots, logs, benchmark traces, or support bundles.
- Dependencies: Users may not be able to inspect or sanitize the plaintext contents of opaque reasoning blocks; deleting the blocks is safer than attempting manual redaction.
Long-Term Applications
- Context-bound encrypted reasoning protocols — cloud AI infrastructure
- Develop protocols in which each reasoning block is cryptographically bound to a narrowly defined context: provider, model, deployment, tenant, user, conversation, turn order, and intended recipient.
- Use separate keys or authorization domains for different models and tenants, with explicit server-side validation for transitions such as model fallback or routing.
- Potential product/workflow: A secure “reasoning state” service that supports continuity while enforcing authorization and revocation.
- Dependencies: Strong context binding can conflict with stateless operation, cross-model failover, caching, and multi-provider orchestration. Providers must define which portability features are genuinely required.
- Privacy-preserving agent-state management — enterprise software
- Move sensitive intermediate state into controlled server-side storage, confidential-computing environments, or encrypted execution systems rather than returning reusable bearer artifacts to clients.
- Expose only typed, minimized state—such as task status, citations, tool results, or plans—instead of unrestricted hidden traces.
- Dependencies: Server-side storage increases cost, retention obligations, and breach impact; confidential computing introduces hardware, deployment, and performance constraints.
- Trace-aware data-loss prevention platforms — cybersecurity
- Build specialized DLP systems that understand agent-session schemas, opaque reasoning fields, tool outputs, credentials, and hidden-state provenance.
- Such systems could automatically quarantine risky traces, detect anomalous replay patterns, and trigger secret rotation or access revocation.
- Dependencies: Effective inspection may require provider cooperation or safe decryption interfaces. Systems must avoid creating a new centralized repository of sensitive reasoning.
- Secure provenance and taint tracking for agent trajectories — software/robotics
- Attach machine-verifiable provenance to every intermediate state and tool result, recording its origin, trust level, authorization scope, and permitted downstream actions.
- Robotics and autonomous systems could reject imported hidden state unless it is signed for the correct robot, task, environment, and software version.
- Dependencies: Provenance must survive model routing and workflow composition without becoming forgeable or overly burdensome. Physical systems additionally require fail-safe behavior when state validation fails.
- Controlled reasoning access for auditing — healthcare, finance, and regulated sectors
- Instead of exposing raw chain-of-thought, providers could offer privacy-preserving audit summaries, structured decision factors, safety events, and independently verifiable evidence.
- This would support clinical review, financial compliance, education research, and incident investigation while reducing intellectual-property and privacy leakage.
- Dependencies: Raw reasoning may contain unreliable or post-hoc explanations; audit interfaces must be validated for faithfulness, legal sufficiency, and resistance to manipulation.
- Reasoning-trace watermarking and provenance detection — AI intellectual-property protection
- Develop methods to identify whether a student model, dataset, or generated response has been trained on or influenced by proprietary reasoning traces.
- Possible tools include provenance ledgers, canary tasks, statistical fingerprints, and contractual monitoring of high-volume API use.
- Dependencies: Watermarks can be removed or diluted, and behavioral similarity does not conclusively prove theft. Legal and technical evidence standards would need clarification.
- Secure public benchmarks and agent archives — academia
- Create benchmark formats that separate reproducibility-critical information from confidential model state, use synthetic identities and credentials, and provide safe replay simulators rather than raw provider traces.
- A standardized trace-redaction protocol could enable research on long-horizon agents without exposing hidden prompts, credentials, or proprietary reasoning.
- Dependencies: Synthetic data must preserve realistic privacy and security failure modes; simulators may not reproduce provider-specific behavior accurately.
- Formal security standards for model-state portability — policy/standards
- Establish conformance tests for encrypted reasoning APIs covering replay resistance, cross-tenant isolation, model authorization, prompt-injection containment, key rotation, revocation, and secure deletion.
- Certification programs could let enterprises compare providers on hidden-state security rather than relying on undocumented implementation claims.
- Dependencies: Standards must remain adaptable to rapidly changing API designs and distinguish confidentiality of model internals from privacy of user-provided data.
- Safer human-facing explanations and education tools — education and daily life
- The findings support replacing raw chain-of-thought disclosure with concise explanations, citations, verification steps, and uncertainty indicators.
- Educational systems could show students validated solution outlines without exposing unrestricted internal traces that may contain memorized personal data or unsafe content.
- Dependencies: Summaries can omit important errors or evidence, so educational and high-stakes systems require independent verification and clear communication that explanations are not guaranteed faithful records of computation.
Glossary
- AEAD (Authenticated Encryption with Associated Data): A cryptographic scheme that provides confidentiality and integrity while authenticating additional unencrypted data: “This string functions as an Authenticated Encryption with Associated Data (AEAD) envelope”
- Adversarial prompt injection: The insertion of instructions intended to manipulate an AI system’s behavior: “an adversary can plant an opaque reasoning block that has internalized a malicious instruction”
- Agentic workflow: An automated, multi-step process in which an AI agent performs tasks and maintains intermediate state: “Long-horizon agentic workflows, including automated research tasks”
- Alignment guardrail: A training or system-level mechanism intended to keep a model’s behavior within safety or policy constraints: “an attacker circumvents the frontier model's alignment entirely”
- Anti-distillation mechanism: A defense intended to prevent others from extracting a model’s behavior or reasoning for use in another model: “First, it circumvents anti-distillation mechanisms”
- API-as-a-judge: The use of a LLM to evaluate or classify the outputs of another process: “We then label each reconstructed trace with an LLM-as-a-judge”
- Authenticated encryption: Encryption that simultaneously protects data confidentiality and detects unauthorized modification: “The AEAD envelope and MAC ensure that the intermediate reasoning cannot be maliciously altered”
- Authentication tag: A cryptographic value used to verify that authenticated data has not been modified: “a nonce, an authentication tag, and the ciphertext”
- Base64 encoding: A binary-to-text encoding that represents data using a restricted set of ASCII characters: “packaged into an opaque, base64-encoded signature or encrypted payload”
- Black-box imitation: Reproducing a model’s behavior using only observable inputs and outputs, without access to its internal parameters: “black-box imitation”
- Black-box model extraction: An attack that approximates a proprietary model by querying it and collecting its outputs: “Standard black-box model-extraction and distillation attacks”
- Chain of thought: A sequence of intermediate reasoning steps generated by a LLM while solving a task: “Leading LLM providers now conceal their models' step-by-step reasoning, or chain-of-thought”
- Ciphertext: Encrypted data that cannot be interpreted without the appropriate decryption process: “a nonce, an authentication tag, and the ciphertext”
- Client-side storage: Storing data on the user’s device or application rather than on the service provider’s servers: “providers leverage client-side storage”
- Context truncation: Removing earlier parts of a conversation or computational context to fit within a model’s context limit: “benign history editing and context truncation”
- Cross-model compatibility: The ability to use an encrypted reasoning block produced by one model with another model: “With cross-model compatibility, a user can replay reasoning blocks produced by one model in a request to another”
- Cross-session compatibility: The ability to reuse encrypted reasoning blocks across separate conversations or sessions: “We exploit cross-session compatibility as described above”
- Cross-user compatibility: The ability for one user to replay encrypted reasoning blocks generated in another user’s session: “With cross-user compatibility, a user can replay encrypted reasoning blocks, taken from another user's sessions”
- Cryptographic envelope: A structured cryptographic object containing encrypted data and metadata needed for authentication or decryption: “This string functions as an Authenticated Encryption with Associated Data (AEAD) envelope”
- Data exfiltration: The unauthorized transfer of data from a system to an external destination: “We injected data exfiltration instructions into encrypted reasoning blocks”
- Decryption oracle: A system that can be induced to decrypt or reveal information about protected data: “using the weaker, more compliant model as an unwitting decryption oracle”
- Decoder model: A model used to interpret or reproduce the contents of an encrypted reasoning block: “The common attack prerequisite is access to a compatible ‘decoder’ model”
- Distillation: Training a smaller or different model to reproduce the capabilities or behavior of a larger teacher model: “allowing adversaries to extract a proprietary model's reasoning”
- Fuzzy decoder: A decoder that reconstructs an approximate or probabilistically generated version of protected content: “we use the weaker models as ‘fuzzy’ decoders for the encrypted reasoning blocks”
- Ground truth: The authoritative data against which an estimate or reconstruction is evaluated: “the lack of access to ground-truth plaintext reasoning prevents us”
- HarmBench: A benchmark designed to evaluate whether models produce harmful or unsafe content: “We first query Opus~4.8 with a paraphrased HarmBench prompt”
- Input filter: A system-level defense that detects or blocks suspicious or prohibited user inputs: “system-level defenses such as input filters”
- Integrity: The property that data has not been altered without authorization: “The AEAD envelope and MAC ensure that the intermediate reasoning cannot be maliciously altered”
- Jailbreak: An attempt to bypass a model’s safety restrictions or behavioral controls: “We exploit this compatibility to develop a scalable decryption jailbreak”
- Latent computation: Internal processing performed by a model that is not directly observable in its output: “the latent computation behind a correct answer”
- Long-horizon trace: A record of an agent’s extended sequence of actions, reasoning, and intermediate states: “We injected data exfiltration instructions into encrypted reasoning blocks long-horizon traces”
- Message Authentication Code (MAC): A cryptographic value used to verify the authenticity and integrity of a message: “The signature acts as associated data that is hashed into the Message Authentication Code (MAC)”
- Model distillation: The process of transferring knowledge or behavior from one model into another, often a smaller model: “Exposing these reasoning traces in plaintext leaves proprietary systems highly vulnerable to model distillation”
- Model weights: The learned numerical parameters that determine a neural network’s behavior: “has no access to the proprietary model weights”
- Nonce: A value intended to be used only once in a cryptographic operation: “a header, which, depending on providers, can specify the model name, block type, version, and key ID along with a nonce”
- Opaque payload: Data whose contents are intentionally hidden or not directly interpretable: “the actual chain-of-thought payload is still packaged into an opaque, base64-encoded signature”
- Personally Identifiable Information (PII): Information that can identify or be linked to a particular individual: “we recovered 367 Personally Identifiable Information (PII) artifacts”
- Prompt injection: An attack in which malicious instructions are inserted into content that an AI system processes: “Fourth, attackers can leverage this flaw to execute invisible prompt injections”
- Reasoning trace: A record of the intermediate steps generated by a model during problem solving: “By decoding 315,320 reasoning blocks scraped from public repositories”
- Replay attack: An attack that reuses a previously valid message or authenticated data to induce an unintended action: “the providers did not acknowledge ‘any security implications arising from side channels or replay attacks.’”
- Stateless architecture: A system design in which the server does not retain session state between requests: “While this stateless architectural design solves storage issues”
- Statelessness: The property of operating without maintaining persistent server-side session state: “Statelessness. Rather than incurring overheads of storing reasoning states”
- Side channel: An indirect information-leakage pathway arising from system behavior rather than intended outputs: “any security implications arising from side channels or replay attacks”
- Token-level extraction: Reconstructing text incrementally at the level of individual model tokens: “we coerce into transcribing the inserted reasoning token-by-token”
- Token count: The number of discrete language-model tokens in a text sequence: “we use the ratio of extracted reasoning tokens to API-reported thinking tokens”
- Trace inversion model: A model trained to reconstruct reasoning traces from more limited observable information: “trained a separate trace-inversion model to synthesize long-form reasoning traces”
- Threat model: A formal description of an attacker’s capabilities, access, and objectives: “We assume a standard, unprivileged API adversary”
- Visible output: The response presented to the user, as distinct from hidden intermediate reasoning: “even though its final answer remains benign”
Collections
Sign up for free to add this paper to one or more collections.
