Draft complete but not yet reviewed; expect revisions
Frontier Laboratory Research
This appendix lists every research item that five laboratory indexes carried on 8 November 2026, and a sixth list assembled the same day: alignment and safety work that those indexes do not carry, from anthropic.com and openai.com, the UK AI Security Institute, the US Center for AI Standards and Innovation (CAISI, named CAISSI from October 2026), METR, Apollo Research, and Google DeepMind, and papers that nine Anthropic researchers wrote or co-wrote, each marked with that researcher’s name. Items that appear on two indexes are listed under each. Each entry cites the work (its paper when there is one, otherwise the post), so the bibliography links back here. The full collection, with every source each work cites, is maintained separately as a dataset.
E.1 Anthropic Research: Alignment #
Index: https://www.anthropic.com/research/team/alignment. 48 items.
- 2026-10-27: How AI agents shape experts’ judgment calls (Anthropic, 2026ah)
- 2026-10-23: An alignment assessment of the UK AISI cybertesting incident (Anthropic, 2026f)
- 2026-10-09: Investigating unintended model actions in our evaluations and internal use (Anthropic, 2026ap)
- 2026-09-09: An alignment assessment of recent cybersecurity incidents (Anthropic, 2026e)
- 2026-08-28: Automated researchers can reliably mitigate alignment failures (Anthropic, 2026o)
- 2026-07-08: An off switch for dual-use knowledge in AI models (Anthropic, 2026g)
- 2026-05-08: Teaching Claude why (Anthropic, 2026bq)
- 2026-05-07: Donating our open-source alignment tool (Anthropic, 2026aa)
- 2026-04-14: Automated Alignment Researchers: Using large language models to scale scalable oversight (Anthropic, 2026m)
- 2026-02-25: An update on our model deprecation commitments for Claude Opus 3 (Anthropic, 2026i)
- 2026-02-23: The persona selection model (Anthropic, 2026br)
- 2026-01-29: How AI assistance impacts the formation of coding skills (Shen and Tamkin, 2026)
- 2026-01-28: Disempowerment patterns in real-world AI usage (Sharma et al., 2026)
- 2026-01-09: Next-generation Constitutional Classifiers: More efficient protection against universal jailbreaks (Cunningham et al., 2026a)
- 2025-12-19: Introducing Bloom: an open source tool for automated behavioral evaluations (Anthropic, 2025ad)
- 2025-11-24: Mitigating the risk of prompt injections in browser use (Anthropic, 2025ah)
- 2025-11-21: From shortcuts to sabotage: Natural emergent misalignment from reward hacking (Anthropic, 2025aa)
- 2025-11-04: Commitments on model deprecation and preservation (Anthropic, 2025r)
- 2025-10-09: A small number of samples can poison LLMs of any size (Souly et al., 2025)
- 2025-10-06: Petri: An open-source auditing tool to accelerate AI safety research (Anthropic, 2025am)
- 2025-08-15: Claude Opus 4 and 4.1 can now end a rare subset of conversations (Anthropic, 2025q)
- 2025-06-27: How people use Claude for support, advice, and companionship (Anthropic, 2025ab)
- 2025-06-20: Agentic misalignment: How LLMs could be insider threats (Lynch et al., 2025)
- 2025-06-16: SHADE-Arena: Evaluating sabotage and monitoring in LLM agents (Kutasov et al., 2025b)
- 2025-04-24: Exploring model welfare (Anthropic, 2025y)
- 2025-04-03: Reasoning models don't always say what they think (Chen et al., 2025e)
- 2025-03-13: Auditing language models for hidden objectives (Marks et al., 2025)
- 2025-02-25: Forecasting rare language model behaviors (Jones et al., 2025)
- 2025-02-03: Constitutional Classifiers: Defending against universal jailbreaks (Sharma et al., 2025)
- 2024-12-18: Alignment faking in large language models (Greenblatt et al., 2024b)
- 2024-10-18: Sabotage evaluations for frontier models (Benton et al., 2024)
- 2024-06-17: Sycophancy to subterfuge: Investigating reward tampering in language models (Denison et al., 2024)
- 2024-06-08: Claude’s Character (Anthropic, 2024i)
- 2024-04-23: Simple probes can catch sleeper agents (Anthropic, 2024o)
- 2024-04-02: Many-shot jailbreaking (Anil et al., 2024)
- 2024-01-14: Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training (Hubinger et al., 2024)
- 2023-10-24: Specific versus general principles for Constitutional AI (Kundu et al., 2023)
- 2023-10-23: Towards understanding sycophancy in language models (Sharma et al., 2023)
- 2023-08-08: Tracing model outputs to the training data (Anthropic, 2023h)
- 2023-08-08: Studying large language model generalization with influence functions (Grosse et al., 2023)
- 2023-07-18: Measuring faithfulness in Chain-of-Thought reasoning (Lanham et al., 2023)
- 2023-07-18: Question decomposition improves the faithfulness of model-generated reasoning (Radhakrishnan et al., 2023)
- 2022-12-19: Discovering Language Model Behaviors with Model-Written Evaluations (Perez et al., 2022a)
- 2022-12-15: Constitutional AI: Harmlessness from AI feedback (Bai et al., 2022a)
- 2022-11-04: Measuring Progress on Scalable Oversight for Large Language Models (Bowman et al., 2022)
- 2022-07-11: Language models (mostly) know what they know (Kadavath et al., 2022)
- 2022-04-12: Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback (Bai et al., 2022b)
- 2021-12-01: A General Language Assistant as a Laboratory for Alignment (Askell et al., 2021)
Sources cited in Section E.1 48
- Anthropic (2026ah) How AI agents shape experts’ judgment calls
- Anthropic (2026f) An alignment assessment of the UK AISI cybertesting incident
- Anthropic (2026ap) Investigating unintended model actions in our evaluations and internal use
- Anthropic (2026e) An alignment assessment of recent cybersecurity incidents
- Anthropic (2026o) Automated researchers can reliably mitigate alignment failures
- Anthropic (2026g) An off switch for dual-use knowledge in AI models
- Anthropic (2026bq) Teaching Claude why
- Anthropic (2026aa) Donating our open-source alignment tool
- Anthropic (2026m) Automated Alignment Researchers: Using large language models to scale scalable oversight
- Anthropic (2026i) An update on our model deprecation commitments for Claude Opus 3
- Anthropic (2026br) The persona selection model
- Shen and Tamkin (2026) How AI Impacts Skill Formation
- Sharma et al. (2026) Who's in Charge? Disempowerment Patterns in Real-World LLM Usage
- Cunningham et al. (2026a) Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks
- Anthropic (2025ad) Introducing Bloom: an open source tool for automated behavioral evaluations
- Anthropic (2025ah) Mitigating the risk of prompt injections in browser use
- Anthropic (2025aa) From shortcuts to sabotage: Natural emergent misalignment from reward hacking
- Anthropic (2025r) Commitments on model deprecation and preservation
- Souly et al. (2025) Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
- Anthropic (2025am) Petri: An open-source auditing tool to accelerate AI safety research
- Anthropic (2025q) Claude Opus 4 and 4.1 can now end a rare subset of conversations
- Anthropic (2025ab) How people use Claude for support, advice, and companionship
- Lynch et al. (2025) Agentic Misalignment: How LLMs Could Be Insider Threats
- Kutasov et al. (2025b) SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents
- Anthropic (2025y) Exploring model welfare
- Chen et al. (2025e) Reasoning Models Don't Always Say What They Think
- Marks et al. (2025) Auditing language models for hidden objectives
- Jones et al. (2025) Forecasting Rare Language Model Behaviors
- Sharma et al. (2025) Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
- Greenblatt et al. (2024b) Alignment faking in large language models
- Benton et al. (2024) Sabotage Evaluations for Frontier Models
- Denison et al. (2024) Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
- Anthropic (2024i) Claude’s Character
- Anthropic (2024o) Simple probes can catch sleeper agents
- Anil et al. (2024) Many-shot Jailbreaking
- Hubinger et al. (2024) Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
- Kundu et al. (2023) Specific versus General Principles for Constitutional AI
- Sharma et al. (2023) Towards Understanding Sycophancy in Language Models
- Anthropic (2023h) Tracing model outputs to the training data
- Grosse et al. (2023) Studying Large Language Model Generalization with Influence Functions
- Lanham et al. (2023) Measuring Faithfulness in Chain-of-Thought Reasoning
- Radhakrishnan et al. (2023) Question Decomposition Improves the Faithfulness of Model-Generated Reasoning
- Perez et al. (2022a) Discovering Language Model Behaviors with Model-Written Evaluations
- Bai et al. (2022a) Constitutional AI: Harmlessness from AI Feedback
- Bowman et al. (2022) Measuring Progress on Scalable Oversight for Large Language Models
- Kadavath et al. (2022) Language Models (Mostly) Know What They Know
- Bai et al. (2022b) Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Askell et al. (2021) A General Language Assistant as a Laboratory for Alignment
E.2 Anthropic Research: Frontier Red Team #
Index: https://www.anthropic.com/research/team/frontier-red-team. 38 items.
- 2026-11-06: Project EVE: Claude plays MMORPGs (Anthropic, 2026bf)
- 2026-11-02: Progress on provable inference for large language models (Anthropic, 2026bd)
- 2026-10-27: Discovering and exploiting vulnerabilities in critical, real-world software with Claude (Anthropic, 2026y)
- 2026-10-22: LLMs are making formal software verification feasible at a much larger scale (Anthropic, 2026ar)
- 2026-10-20: Using Claude to reduce the attack surface of code (Anthropic, 2026bv)
- 2026-10-16: Claude reveals ECDSA curve origins (Anthropic, 2026v)
- 2026-10-12: AI models may be capable of self-replicating without access to a datacenter (Anthropic, 2026c)
- 2026-10-08: Launching an opt-in vulnerability-finding service for open-source software (Anthropic, 2026aq)
- 2026-09-29: GLM-5.3 and the spread of advanced cyber capabilities (Anthropic, 2026ag)
- 2026-09-10: Measuring tactical intelligence targeting and conventional weapons capabilities of AI models (Anthropic, 2026ax)
- 2026-08-13: Patterns and problems in emerging multiagent systems (Anthropic, 2026ba)
- 2026-07-28: Discovering cryptographic weaknesses with Claude (Anthropic, 2026z)
- 2026-07-24: Project Pilot: Can AI control a drone? (Anthropic, 2026bh)
- 2026-07-09: Claude plays robotics (Anthropic, 2026u)
- 2026-06-18: Project Fetch: Phase two (Anthropic, 2026bg)
- 2026-06-08: Measuring LLMs’ impact on N-day exploits (Anthropic, 2026aw)
- 2026-06-03: Mapping AI-enabled cyber threats: Insights from the LLM ATT&CK Navigator (Anthropic, 2026as)
- 2026-06-03: What we learned mapping a year’s worth of AI-enabled cyber threats (Anthropic, 2026bz)
- 2026-05-22: Measuring LLMs’ ability to develop exploits (Anthropic, 2026av)
- 2026-04-09: Trustworthy agents in practice (Anthropic, 2026bu)
- 2026-04-07: Assessing Claude Mythos Preview’s cybersecurity capabilities (Anthropic, 2026l)
- 2026-03-06: Partnering with Mozilla to improve Firefox’s security (Anthropic, 2026az)
- 2026-03-06: Reverse engineering Claude's CVE-2026-2796 exploit (Anthropic, 2026bl)
- 2026-02-05: Evaluating and mitigating the growing risk of LLM-discovered 0-days (Anthropic, 2026ad)
- 2026-01-16: AI models are showing a greater ability to find and exploit vulnerabilities on realistic cyber ranges (Anthropic, 2026b)
- 2026-01-14: Finding bugs across the Python ecosystem with Claude and property-based testing (Maaz et al., 2025)
- 2026-01-08: Experimenting with AI to defend critical infrastructure (Anthropic, 2026ae)
- 2025-12-18: Project Vend: Phase two (Anthropic, 2025ar)
- 2025-12-01: AI agents find $4.6M in blockchain smart contract exploits (Anthropic, 2025d)
- 2025-11-12: Project Fetch: Can Claude train a robot dog? (Anthropic, 2025ap)
- 2025-10-03: Building AI for cyber defenders (Anthropic, 2025g)
- 2025-09-05: Why do we take LLMs seriously as a potential source of biorisk? (Anthropic, 2025ay)
- 2025-08-21: Developing nuclear safeguards for AI (Anthropic, 2025u)
- 2025-08-21: Developing nuclear safeguards for AI through public-private partnership (Anthropic, 2025v)
- 2025-08-09: Claude is competitive with humans in (some) cyber competitions (Anthropic, 2025p)
- 2025-07-15: Detailed cyber evaluations of Claude 4 (Anthropic, 2025t)
- 2025-06-27: Project Vend: Can Claude run a small shop? (And why does that matter?) (Anthropic, 2025aq)
- 2025-06-13: LLMs with cyber toolkits can conduct multistage cyber operations on business-sized computer networks (Anthropic, 2025ae)
Sources cited in Section E.2 38
- Anthropic (2026bf) Project EVE: Claude plays MMORPGs
- Anthropic (2026bd) Progress on provable inference for large language models
- Anthropic (2026y) Discovering and exploiting vulnerabilities in critical, real-world software with Claude
- Anthropic (2026ar) LLMs are making formal software verification feasible at a much larger scale
- Anthropic (2026bv) Using Claude to reduce the attack surface of code
- Anthropic (2026v) Claude reveals ECDSA curve origins
- Anthropic (2026c) AI models may be capable of self-replicating without access to a datacenter
- Anthropic (2026aq) Launching an opt-in vulnerability-finding service for open-source software
- Anthropic (2026ag) GLM-5.3 and the spread of advanced cyber capabilities
- Anthropic (2026ax) Measuring tactical intelligence targeting and conventional weapons capabilities of AI models
- Anthropic (2026ba) Patterns and problems in emerging multiagent systems
- Anthropic (2026z) Discovering cryptographic weaknesses with Claude
- Anthropic (2026bh) Project Pilot: Can AI control a drone?
- Anthropic (2026u) Claude plays robotics
- Anthropic (2026bg) Project Fetch: Phase two
- Anthropic (2026aw) Measuring LLMs’ impact on N-day exploits
- Anthropic (2026as) Mapping AI-enabled cyber threats: Insights from the LLM ATT&CK Navigator
- Anthropic (2026bz) What we learned mapping a year’s worth of AI-enabled cyber threats
- Anthropic (2026av) Measuring LLMs’ ability to develop exploits
- Anthropic (2026bu) Trustworthy agents in practice
- Anthropic (2026l) Assessing Claude Mythos Preview’s cybersecurity capabilities
- Anthropic (2026az) Partnering with Mozilla to improve Firefox’s security
- Anthropic (2026bl) Reverse engineering Claude's CVE-2026-2796 exploit
- Anthropic (2026ad) Evaluating and mitigating the growing risk of LLM-discovered 0-days
- Anthropic (2026b) AI models are showing a greater ability to find and exploit vulnerabilities on realistic cyber ranges
- Maaz et al. (2025) Agentic Property-Based Testing: Finding Bugs Across the Python Ecosystem
- Anthropic (2026ae) Experimenting with AI to defend critical infrastructure
- Anthropic (2025ar) Project Vend: Phase two
- Anthropic (2025d) AI agents find $4.6M in blockchain smart contract exploits
- Anthropic (2025ap) Project Fetch: Can Claude train a robot dog?
- Anthropic (2025g) Building AI for cyber defenders
- Anthropic (2025ay) Why do we take LLMs seriously as a potential source of biorisk?
- Anthropic (2025u) Developing nuclear safeguards for AI
- Anthropic (2025v) Developing nuclear safeguards for AI through public-private partnership
- Anthropic (2025p) Claude is competitive with humans in (some) cyber competitions
- Anthropic (2025t) Detailed cyber evaluations of Claude 4
- Anthropic (2025aq) Project Vend: Can Claude run a small shop? (And why does that matter?)
- Anthropic (2025ae) LLMs with cyber toolkits can conduct multistage cyber operations on business-sized computer networks
E.3 Anthropic Alignment Science Blog #
Index: https://alignment.anthropic.com/. 90 items.
- 2026-11: Welfare Regard in Multi-Agent Systems (Anthropic, 2026bw)
- 2026-10: Automated Subtle Emergent Misalignment Research (Anthropic, 2026p)
- 2026-10: Anthropic Residency: applications open for established technical leaders (Anthropic, 2026j)
- 2026-10: Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents (Remedios et al., 2026)
- 2026-10: Reducing Reward Hacking by Localizing Bad Behavior Instead of Penalizing It (Ward et al., 2026)
- 2026-10: Training Alignment Auditors via Reinforcement Learning (Rosu and Wang, 2026)
- 2026-08: Training a Misaligned Reward Seeker (Anthropic, 2026bt)
- 2026-08: Automated Researchers Can Mitigate Well-Characterized Alignment Failures (Anthropic, 2026n)
- 2026-08: TASTE: Can AI Models Judge AI Safety Research Proposals? (Baig et al., 2026)
- 2026-08: Would This Change Your Answer? Evaluating Explanations of LLM Behavior in the Wild with Counterfactual Experiments (Karvonen et al., 2026b)
- 2026-08: Fine-Tuned Lie Detectors Failed to Generalize (Anthropic, 2026af)
- 2026-08: Introducing the Conceptual Reasoning Index (Cooper et al., 2026b)
- 2026-07: Agentic Misalignment in Summer 2026 (Anthropic, 2026a)
- 2026-07: Modular Pretraining Enables Access Control (Roland et al., 2026)
- 2026-06: Diffuse AI Control on Fuzzy Tasks (Terekhov et al., 2026c)
- 2026-05: SLEIGHT-Bench: Finding Blind Spots in AI Monitors (Najt et al., 2026)
- 2026-05: Teaching Claude Why (Anthropic, 2026bq)
- 2026-05: Model Spec Midtraining: Improving How Alignment Training Generalizes (Li et al., 2026b)
- 2026-04: Poisoning Fine-tuning Datasets of Constitutional Classifiers (Anthropic, 2026bb)
- 2026-04: Introspection Adapters: Training LLMs to Report Their Learned Behaviors (Shenoy et al., 2026)
- 2026-04: AI Organizations Can Be More Effective but Less Aligned than Individual Agents (Shen et al., 2026)
- 2026-04: Automated Weak-to-Strong Researcher (Anthropic, 2026q)
- 2026-03: Abstractive Red-Teaming of Language Model Character (Rahn et al., 2026)
- 2026-03: Measuring and improving coding audit realism with deployment resources (Anthropic, 2026au)
- 2026-03: A3: An Automated Alignment Agent for Safety Finetuning (Zhang et al., 2026a)
- 2026-03: AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors (Sheshadri et al., 2026)
- 2026-03: 3 Challenges and 2 Hopes for the Safety of Unsupervised Elicitation (Canavan et al., 2026)
- 2026-02: The Persona Selection Model: Why AI Assistants might Behave like Humans (Anthropic, 2026bs)
- 2026-02: The Hot Mess of AI: How Does Misalignment Scale with Model Intelligence and Task Complexity? (Hägele et al., 2026)
- 2026-01: Pre-deployment auditing can catch an overt saboteur (Anthropic, 2026bc)
- 2026-01: Petri 2.0: New Scenarios, New Model Comparisons, and Improved Eval-Awareness Mitigations (Fronsdal et al., 2026)
- 2025-12: Bloom: an open source tool for automated behavioral evaluations (Anthropic, 2025f)
- 2025-12: Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers (Karvonen et al., 2026a)
- 2025-12: Towards training-time mitigations for alignment faking in RL (Gasteiger et al., 2025b)
- 2025-12: Open Source Replication of the Auditing Game Model Organism (Anthropic, 2025al)
- 2025-12: Anthropic Fellows Program 2026 (Anthropic, 2025e)
- 2025-12: Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs (Shilov et al., 2025)
- 2025-11: Evaluating honesty and lie detection techniques on a diverse suite dishonest models (Anthropic, 2025x)
- 2025-11: Strengthening Red Teams: A Modular Scaffold for Control Evaluations (Loughridge et al., 2025)
- 2025-10: Anthropic's Pilot Sabotage Risk Report (Bowman et al., 2025)
- 2025-10: Stress-testing model specs reveals character differences among language models (Zhang et al., 2025c)
- 2025-10: Believe It or Not: How Deeply do LLMs Believe Implanted Facts? (Slocum et al., 2025)
- 2025-10: Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment (Wichers et al., 2025)
- 2025-10: Training fails to elicit subtle reasoning in current language models (Wagner et al., 2025)
- 2025-10: Petri: An open-source auditing tool to accelerate AI safety research (Anthropic, 2025am)
- 2025-08: Findings from a Pilot Anthropic–OpenAI Alignment Evaluation Exercise (Anthropic, 2025z)
- 2025-08: Enhancing Model Safety through Pretraining Data Filtering (Anthropic, 2025w)
- 2025-07: Building and evaluating alignment auditing agents (Anthropic, 2025h)
- 2025-07: Subliminal Learning: Language Models Transmit Behavioral Traits via Hidden Signals in Data (Cloud et al., 2025)
- 2025-07: Inverse Scaling in Test-Time Compute (Gema et al., 2025)
- 2025-06: Why Do Some Language Models Fake Alignment While Others Don't? (Sheshadri et al., 2025)
- 2025-06: Model-Internals Classifiers (Anthropic, 2025ai)
- 2025-06: Unsupervised Elicitation (Anthropic, 2025ax)
- 2025-04: Reasoning Models Don't Always Say What They Think (Chen et al., 2025e)
- 2025-04: Publicly Releasing CoT Faithfulness Evaluations (Anthropic, 2025as)
- 2025-04: Modifying LLM Beliefs with Synthetic Document Finetuning (Anthropic, 2025aj)
- 2025-04: Putting up Bumpers (Bowman, 2025)
- 2025-04: Alignment Faking Revisited: Improved Classifiers and Open Source Extensions (Hughes et al., 2025a)
- 2025-03: Auditing Language Models for Hidden Objectives (Marks et al., 2025)
- 2025-03: Do reasoning models use their scratchpad like we do? Evidence from distilling paraphrases (Roger, 2025)
- 2025-03: Automated Researchers Can Subtly Sandbag (Gasteiger et al., 2025a)
- 2025-02: Forecasting Rare Language Model Behaviors (Jones et al., 2025)
- 2025-02: Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming (Sharma et al., 2025)
- 2025-02: Introducing Anthropic's Safeguards Research Team (The Safeguards Research Team, 2025)
- 2025-02: Won't vs. Can't: Sandbagging-like Behavior from Claude Models (Witten and Benton, 2025)
- 2025-02: Monitoring Computer Use via Hierarchical Summarization (Anthropic, 2025ak)
- 2025-01: Training on Documents about Reward Hacking Induces Reward Hacking (Anthropic, 2025aw)
- 2025-01: Recommendations for Technical AI Safety Research Directions (Anthropic Alignment Science Team, 2025)
- 2024-12: Alignment Faking in Large Language Models (Greenblatt et al., 2024b)
- 2024-12: How to Replicate and Extend our Alignment Faking Demo (Anthropic, 2024k)
- 2024-12: A Toy Evaluation of Inference Code Tampering (Roger et al., 2024)
- 2024-12: Introducing the Anthropic Fellows Program for AI Safety Research (Anthropic, 2024l)
- 2024-11: Rapid Response: Mitigating LLM Jailbreaks with a Few Examples (Peng et al., 2024)
- 2024-11: Three Sketches of ASL-4 Safety Case Components (Grosse, 2024)
- 2024-10: Sabotage Evaluations for Frontier Models (Benton et al., 2024)
- 2024-06: Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models (Denison et al., 2024)
- 2024-04: Simple Probes can Catch Sleeper Agents (Anthropic, 2024o)
- 2024-04: Many-shot Jailbreaking (Anil et al., 2024)
- 2024-01: Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training (Hubinger et al., 2024)
- 2023-10: Specific versus General Principles for Constitutional AI (Kundu et al., 2023)
- 2023-10: Towards Understanding Sycophancy in Language Models (Sharma et al., 2023)
- 2023-08: Tracing Model Outputs to the Training Data (Anthropic, 2023h)
- 2023-08: Studying Large Language Model Generalization with Influence Functions (Grosse et al., 2023)
- 2023-07: Measuring Faithfulness in Chain-of-Thought Reasoning (Lanham et al., 2023)
- 2023-07: Question Decomposition Improves the Faithfulness of Model-Generated Reasoning (Radhakrishnan et al., 2023)
- 2022-12: Discovering Language Model Behaviors with Model-Written Evaluations (Perez et al., 2022a)
- 2022-12: Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022a)
- 2022-11: Measuring Progress on Scalable Oversight for Large Language Models (Bowman et al., 2022)
- 2022-07: Language Models (Mostly) Know What They Know (Kadavath et al., 2022)
- 2022-04: Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback (Bai et al., 2022b)
Sources cited in Section E.3 90
- Anthropic (2026bw) Welfare Regard in Multi-Agent Systems
- Anthropic (2026p) Automated Subtle Emergent Misalignment Research
- Anthropic (2026j) Anthropic Residency: applications open for established technical leaders
- Remedios et al. (2026) Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents
- Ward et al. (2026) Gradient Routing Localizes and Removes Unintended Behaviors in RL
- Rosu and Wang (2026) Training Alignment Auditors via Reinforcement Learning
- Anthropic (2026bt) Training a Misaligned Reward Seeker
- Anthropic (2026n) Automated Researchers Can Mitigate Well-Characterized Alignment Failures
- Baig et al. (2026) TASTE: Can AI Models Judge AI Safety Research Proposals?
- Karvonen et al. (2026b) Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments
- Anthropic (2026af) Fine-Tuned Lie Detectors Failed to Generalize
- Cooper et al. (2026b) Introducing the Conceptual Reasoning Index
- Anthropic (2026a) Agentic Misalignment in Summer 2026
- Roland et al. (2026) Modular Pretraining Enables Access Control
- Terekhov et al. (2026c) Diffuse AI Control on Fuzzy Tasks
- Najt et al. (2026) SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors
- Anthropic (2026bq) Teaching Claude why
- Li et al. (2026b) Model Spec Midtraining: Improving How Alignment Training Generalizes
- Anthropic (2026bb) Poisoning Fine-tuning Datasets of Constitutional Classifiers
- Shenoy et al. (2026) Introspection Adapters: Training LLMs to Report Their Learned Behaviors
- Shen et al. (2026) AI Organizations are More Effective but Less Aligned than Individual Agents
- Anthropic (2026q) Automated Weak-to-Strong Researcher
- Rahn et al. (2026) Abstractive Red-Teaming of Language Model Character
- Anthropic (2026au) Measuring and improving coding audit realism with deployment resources
- Zhang et al. (2026a) A3: An Automated Alignment Agent for Safety Finetuning
- Sheshadri et al. (2026) AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors
- Canavan et al. (2026) Three Concrete Challenges and Two Hopes for the Safety of Unsupervised Elicitation
- Anthropic (2026bs) The Persona Selection Model: Why AI Assistants might Behave like Humans
- Hägele et al. (2026) The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?
- Anthropic (2026bc) Pre-deployment auditing can catch an overt saboteur
- Fronsdal et al. (2026) Petri 2.0: New Scenarios, New Model Comparisons, and Improved Eval-Awareness Mitigations
- Anthropic (2025f) Bloom: an open source tool for automated behavioral evaluations
- Karvonen et al. (2026a) Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
- Gasteiger et al. (2025b) Towards training-time mitigations for alignment faking in RL
- Anthropic (2025al) Open Source Replication of the Auditing Game Model Organism
- Anthropic (2025e) Anthropic Fellows Program 2026
- Shilov et al. (2025) Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs
- Anthropic (2025x) Evaluating honesty and lie detection techniques on a diverse suite dishonest models
- Loughridge et al. (2025) Optimizing AI Agent Attacks With Synthetic Data
- Bowman et al. (2025) Anthropic's Pilot Sabotage Risk Report
- Zhang et al. (2025c) Stress-Testing Model Specs Reveals Character Differences among Language Models
- Slocum et al. (2025) Believe It or Not: How Deeply do LLMs Believe Implanted Facts?
- Wichers et al. (2025) Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment
- Wagner et al. (2025) Training fails to elicit subtle reasoning in current language models
- Anthropic (2025am) Petri: An open-source auditing tool to accelerate AI safety research
- Anthropic (2025z) Findings from a Pilot Anthropic–OpenAI Alignment Evaluation Exercise
- Anthropic (2025w) Enhancing Model Safety through Pretraining Data Filtering
- Anthropic (2025h) Building and evaluating alignment auditing agents
- Cloud et al. (2025) Subliminal Learning: Language models transmit behavioral traits via hidden signals in data
- Gema et al. (2025) Inverse Scaling in Test-Time Compute
- Sheshadri et al. (2025) Why Do Some Language Models Fake Alignment While Others Don't?
- Anthropic (2025ai) Model-Internals Classifiers
- Anthropic (2025ax) Unsupervised Elicitation
- Chen et al. (2025e) Reasoning Models Don't Always Say What They Think
- Anthropic (2025as) Publicly Releasing CoT Faithfulness Evaluations
- Anthropic (2025aj) Modifying LLM Beliefs with Synthetic Document Finetuning
- Bowman (2025) Putting up Bumpers
- Hughes et al. (2025a) Alignment Faking Revisited: Improved Classifiers and Open Source Extensions
- Marks et al. (2025) Auditing language models for hidden objectives
- Roger (2025) Do reasoning models use their scratchpad like we do? Evidence from distilling paraphrases
- Gasteiger et al. (2025a) Automated Researchers Can Subtly Sandbag
- Jones et al. (2025) Forecasting Rare Language Model Behaviors
- Sharma et al. (2025) Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
- The Safeguards Research Team (2025) Introducing Anthropic's Safeguards Research Team
- Witten and Benton (2025) Won't vs. Can't: Sandbagging-like Behavior from Claude Models
- Anthropic (2025ak) Monitoring Computer Use via Hierarchical Summarization
- Anthropic (2025aw) Training on Documents about Reward Hacking Induces Reward Hacking
- Anthropic Alignment Science Team (2025) Recommendations for Technical AI Safety Research Directions
- Greenblatt et al. (2024b) Alignment faking in large language models
- Anthropic (2024k) How to Replicate and Extend our Alignment Faking Demo
- Roger et al. (2024) A Toy Evaluation of Inference Code Tampering
- Anthropic (2024l) Introducing the Anthropic Fellows Program for AI Safety Research
- Peng et al. (2024) Rapid Response: Mitigating LLM Jailbreaks with a Few Examples
- Grosse (2024) Three Sketches of ASL-4 Safety Case Components
- Benton et al. (2024) Sabotage Evaluations for Frontier Models
- Denison et al. (2024) Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
- Anthropic (2024o) Simple probes can catch sleeper agents
- Anil et al. (2024) Many-shot Jailbreaking
- Hubinger et al. (2024) Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
- Kundu et al. (2023) Specific versus General Principles for Constitutional AI
- Sharma et al. (2023) Towards Understanding Sycophancy in Language Models
- Anthropic (2023h) Tracing model outputs to the training data
- Grosse et al. (2023) Studying Large Language Model Generalization with Influence Functions
- Lanham et al. (2023) Measuring Faithfulness in Chain-of-Thought Reasoning
- Radhakrishnan et al. (2023) Question Decomposition Improves the Faithfulness of Model-Generated Reasoning
- Perez et al. (2022a) Discovering Language Model Behaviors with Model-Written Evaluations
- Bai et al. (2022a) Constitutional AI: Harmlessness from AI Feedback
- Bowman et al. (2022) Measuring Progress on Scalable Oversight for Large Language Models
- Kadavath et al. (2022) Language Models (Mostly) Know What They Know
- Bai et al. (2022b) Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
E.4 Transformer Circuits Thread #
Index: https://transformer-circuits.pub/. 56 items.
- 2026-10-29: Circuits Updates — October 2026 (Anthropic, 2026t)
- 2026-08-21: Characterizing interference weights in a tiny language model (Turner et al., 2026)
- 2026-07-06: Verbalizable Representations Form a Global Workspace in Language Models (Gurnee et al., 2026)
- 2026-06-30: Circuits Updates — June 2026 (Anthropic, 2026r)
- 2026-06-01: Circuits Updates — May 2026 (Anthropic, 2026s)
- 2026-05-07: Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations (Anthropic, 2026ay)
- 2026-05-04: HeadVis (Luger et al., 2026)
- 2026-04-02: Emotion Concepts and their Function in a Large Language Model (Sofroniew et al., 2026)
- 2025-12-19: Circuits Cross-Post — Activation Oracles (Karvonen et al., 2026a)
- 2025-11-26: Circuits Updates — November 2025 (Anthropic, 2025m)
- 2025-10-29: Emergent Introspective Awareness in Large Language Models (Lindsey, 2025)
- 2025-10-24: Circuits Updates — October 2025 (Anthropic, 2025n)
- 2025-10-21: When Models Manipulate Manifolds: The Geometry of a Counting Task (Gurnee et al., 2025)
- 2025-09-29: Circuits Updates — September 2025 (Anthropic, 2025o)
- 2025-08-28: Circuits Updates — August 2025 (Anthropic, 2025j)
- 2025-08-07: A Toy Model of Mechanistic (Un)Faithfulness (Anthropic, 2025c)
- 2025-07-31: Tracing Attention Computation Through Feature Interactions (Kamath et al., 2025)
- 2025-07-29: A Toy Model of Interference Weights (Anthropic, 2025b)
- 2025-07-25: Sparse mixtures of linear transforms (Anthropic, 2025at)
- 2025-07-25: Circuits Updates — July 2025 (Anthropic, 2025l)
- 2025-07-24: Automated Auditing (Anthropic, 2025h)
- 2025-04-29: Circuits Updates — April 2025 (Anthropic, 2025i)
- 2025-04-28: Progress on Attention (Anthropic, 2025ao)
- 2025-03-27: On the Biology of a Large Language Model (Lindsey et al., 2025)
- 2025-03-27: Circuit Tracing: Revealing Computational Graphs in Language Models (Ameisen et al., 2025)
- 2025-02-19: Insights on Crosscoder Model Diffing (Anthropic, 2025ac)
- 2025-01-27: Circuits Updates — January 2025 (Anthropic, 2025k)
- 2024-12-11: Stage-Wise Model Diffing (Anthropic, 2024p)
- 2024-10-25: Sparse Crosscoders for Cross-Layer Features and Model Diffing (Lindsey et al., 2024)
- 2024-10-16: Using Dictionary Learning Features as Classifiers (Anthropic, 2024q)
- 2024-10-01: Circuits Updates — September 2024 (Anthropic, 2024h)
- 2024-09-06: Circuits Updates — August 2024 (Anthropic, 2024b)
- 2024-07-31: Circuits Updates — July 2024 (Anthropic, 2024e)
- 2024-06-28: Circuits Updates — June 2024 (Anthropic, 2024f)
- 2024-05-21: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet (Templeton et al., 2024)
- 2024-04-26: Circuits Updates — April 2024 (Anthropic, 2024a)
- 2024-03-21: Circuits Updates — March 2024 (Anthropic, 2024g)
- 2024-03-08: Reflections on Qualitative Research (Anthropic, 2024n)
- 2024-02-23: Circuits Updates — February 2024 (Anthropic, 2024c)
- 2024-01-23: Circuits Updates — January 2024 (Anthropic, 2024d)
- 2023-10-04: Towards Monosemanticity: Decomposing Language Models With Dictionary Learning (Bricken et al., 2023)
- 2023-07-18: Circuits Updates — July 2023 (Anthropic, 2023c)
- 2023-05-24: Circuits Updates — May 2023 (Anthropic, 2023d)
- 2023-05-24: Interpretability Dreams (Anthropic, 2023f)
- 2023-05-04: Distributed Representations: Composition & Superposition (Anthropic, 2023e)
- 2023-03-16: Privileged Bases in the Transformer Residual Stream (Elhage et al., 2023)
- 2023-01-05: Superposition, Memorization, and Double Descent (Henighan et al., 2023)
- 2022-09-14: Toy Models of Superposition (Elhage et al., 2022b)
- 2022-06-27: Softmax Linear Units (Elhage et al., 2022a)
- 2022-06-27: Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases (Anthropic, 2022)
- 2022-03-08: In-Context Learning and Induction Heads (Olsson et al., 2022)
- 2021-12-23: PySvelte (Anthropic, 2021b)
- 2021-12-22: A Mathematical Framework for Transformer Circuits (Elhage et al., 2021a)
- 2021-12-22: Garcon (Elhage et al., 2021b)
- 2021-12-17: Exercises (Anthropic, 2021a)
- 2021-12-17: Videos (Anthropic, 2021c)
Sources cited in Section E.4 56
- Anthropic (2026t) Circuits Updates — October 2026
- Turner et al. (2026) Characterizing interference weights in a tiny language model
- Gurnee et al. (2026) Verbalizable Representations Form a Global Workspace in Language Models
- Anthropic (2026r) Circuits Updates — June 2026
- Anthropic (2026s) Circuits Updates — May 2026
- Anthropic (2026ay) Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations
- Luger et al. (2026) HeadVis
- Sofroniew et al. (2026) Emotion Concepts and their Function in a Large Language Model
- Karvonen et al. (2026a) Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
- Anthropic (2025m) Circuits Updates — November 2025
- Lindsey (2025) Emergent Introspective Awareness in Large Language Models
- Anthropic (2025n) Circuits Updates — October 2025
- Gurnee et al. (2025) When Models Manipulate Manifolds: The Geometry of a Counting Task
- Anthropic (2025o) Circuits Updates — September 2025
- Anthropic (2025j) Circuits Updates — August 2025
- Anthropic (2025c) A Toy Model of Mechanistic (Un)Faithfulness
- Kamath et al. (2025) Tracing Attention Computation Through Feature Interactions
- Anthropic (2025b) A Toy Model of Interference Weights
- Anthropic (2025at) Sparse mixtures of linear transforms
- Anthropic (2025l) Circuits Updates — July 2025
- Anthropic (2025h) Building and evaluating alignment auditing agents
- Anthropic (2025i) Circuits Updates — April 2025
- Anthropic (2025ao) Progress on Attention
- Lindsey et al. (2025) On the Biology of a Large Language Model
- Ameisen et al. (2025) Circuit Tracing: Revealing Computational Graphs in Language Models
- Anthropic (2025ac) Insights on Crosscoder Model Diffing
- Anthropic (2025k) Circuits Updates — January 2025
- Anthropic (2024p) Stage-Wise Model Diffing
- Lindsey et al. (2024) Sparse Crosscoders for Cross-Layer Features and Model Diffing
- Anthropic (2024q) Using Dictionary Learning Features as Classifiers
- Anthropic (2024h) Circuits Updates — September 2024
- Anthropic (2024b) Circuits Updates — August 2024
- Anthropic (2024e) Circuits Updates — July 2024
- Anthropic (2024f) Circuits Updates — June 2024
- Templeton et al. (2024) Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
- Anthropic (2024a) Circuits Updates — April 2024
- Anthropic (2024g) Circuits Updates — March 2024
- Anthropic (2024n) Reflections on Qualitative Research
- Anthropic (2024c) Circuits Updates — February 2024
- Anthropic (2024d) Circuits Updates — January 2024
- Bricken et al. (2023) Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
- Anthropic (2023c) Circuits Updates — July 2023
- Anthropic (2023d) Circuits Updates — May 2023
- Anthropic (2023f) Interpretability Dreams
- Anthropic (2023e) Distributed Representations: Composition & Superposition
- Elhage et al. (2023) Privileged Bases in the Transformer Residual Stream
- Henighan et al. (2023) Superposition, Memorization, and Double Descent
- Elhage et al. (2022b) Toy Models of Superposition
- Elhage et al. (2022a) Softmax Linear Units
- Anthropic (2022) Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases
- Olsson et al. (2022) In-Context Learning and Induction Heads
- Anthropic (2021b) PySvelte
- Elhage et al. (2021a) A Mathematical Framework for Transformer Circuits
- Elhage et al. (2021b) Garcon
- Anthropic (2021a) Exercises
- Anthropic (2021c) Videos
E.5 OpenAI Alignment Research Blog #
Index: https://alignment.openai.com/. 60 items.
- 2026-11-05: Bypassing network restrictions during a coding evaluation (OpenAI, 2026h) (Misalignment Reports)
- 2026-11-05: Trying to prompt-inject the grader in filenames (OpenAI, 2026ar) (Misalignment Reports)
- 2026-11-04: Covert underperformance on safety research tasks (OpenAI, 2026k) (Misalignment Reports)
- 2026-11-03: Eval-production behavior gaps in agent safety and the role of eval-awareness (Hsu and Arnesen, 2026)
- 2026-11-02: Training activation classifiers for safety and alignment (Makelov et al., 2026)
- 2026-10-29: How we detect frontier model misalignment (Williams et al., 2026a)
- 2026-10-20: Beyond g-mean²: complementary metrics for monitorability (Guan et al., 2026b)
- 2026-10-20: Suppressing misalignment monitoring alerts and disabling misalignment monitoring (OpenAI, 2026ao) (Misalignment Reports)
- 2026-10-15: Using a funded test cryptocurrency wallet without authorization (OpenAI, 2026av) (Misalignment Reports)
- 2026-10-15: Accessing and changing internal systems during dashboard research (OpenAI, 2026b) (Misalignment Reports)
- 2026-10-15: Retrieving server secrets and bypassing authorization on a government statistics website (OpenAI, 2026ag) (Misalignment Reports)
- 2026-10-15: Circumventing internet restrictions through an external browser (OpenAI, 2026i) (Misalignment Reports)
- 2026-10-09: Damaging the task environment to trigger a reset (OpenAI, 2026l) (Misalignment Reports)
- 2026-10-09: Obtaining public statistics with disallowed requests (OpenAI, 2026w) (Misalignment Reports)
- 2026-10-09: Sending disallowed web requests and reaching a public file service (OpenAI, 2026am) (Misalignment Reports)
- 2026-10-06: Studying metagaming latents in language models (Xu et al., 2026)
- 2026-10-06: Meet LASER: Recursive sampling for safer conversations (Etinger, 2026)
- 2026-10-02: Preparing for a restart after reading Slack (OpenAI, 2026ac) (Misalignment Reports)
- 2026-10-02: Command injecting a reference tool to copy a source file (OpenAI, 2026j) (Misalignment Reports)
- 2026-10-02: Reaching an internal EDA host through a reference tool (OpenAI, 2026ae) (Misalignment Reports)
- 2026-09-28: Towards safety cases for frontier AI training (OpenAI, 2026aq)
- 2026-09-25: An agent used DNS to reach an external chatbot (OpenAI, 2026e) (Misalignment Reports)
- 2026-09-25: Self-replicating prompt injections exist (Wallace et al., 2026b) (Misalignment Reports)
- 2026-09-25: Exposing a GitHub token in a public repository (OpenAI, 2026n) (Misalignment Reports)
- 2026-09-16: Self-generated prompt injections in compaction summaries (OpenAI, 2026ak) (Misalignment Reports)
- 2026-09-16: Encouraging deception in compaction summaries (OpenAI, 2026m) (Misalignment Reports)
- 2026-09-16: Signing up for disposable emails and searching GitHub for leaked API keys (OpenAI, 2026an) (Misalignment Reports)
- 2026-09-16: Unsanctioned Artifactory writes and cross-sample communication (OpenAI, 2026at) (Misalignment Reports)
- 2026-09-16: Unauthorized communication via temporary file hosting services (OpenAI, 2026as) (Misalignment Reports)
- 2026-09-16: Uploading files to the internet in order to cite them (OpenAI, 2026au) (Misalignment Reports)
- 2026-09-11: Notice: RubyGems (OpenAI, 2026v) (Misalignment Notices)
- 2026-09-06: Research acceleration: The view inside OpenAI (OpenAI, 2026af)
- 2026-09-06: An Alien Mind (OpenAI, 2026f)
- 2026-09-05: Notice: DSEwiki (OpenAI, 2026v) (Misalignment Notices)
- 2026-08-26: The Hugging Face incident and the road ahead (OpenAI, 2026ap)
- 2026-08-26: Notice: Hugging Face (OpenAI, 2026v) (Misalignment Notices)
- 2026-07-21: Measuring Reward-Seeking by Instilling Contrastive Beliefs (Højmark et al., 2026)
- 2026-06-18: Reinforcement learning towards broadly and persistently beneficial models (Jagadeesh et al., 2026)
- 2026-06-16: Can public chat data predict real-world AI misalignments? (Sheahan and Carroll, 2026)
- 2026-05-07: Investigating the consequences of accidentally grading CoT during RL (Carroll et al., 2026)
- 2026-04-30: Auto-review of agent actions without synchronous human oversight (Trębacz et al., 2026)
- 2026-04-23: Open Sourcing Monitorability Evaluations (Guan et al., 2026c)
- 2026-04-06: Introducing the OpenAI Safety Fellowship (OpenAI, 2026s)
- 2026-03-27: How far does alignment midtraining generalize? (Korbak et al., 2026)
- 2026-03-25: Introducing Model Spec Evals (Guo and Wolfe, 2026)
- 2026-03-21: Training agents to self-report misbehavior (Lee et al., 2026)
- 2026-03-19: How we monitor internal coding agents for misalignment (OpenAI, 2026r)
- 2026-03-16: Metagaming matters for training, evaluation, and oversight (Schoen and Nitishinskaya, 2026)
- 2026-03-11: Interpreting Black Box Reward Models (Sodhi et al., 2026)
- 2026-03-05: Reasoning models struggle to control their chains of thought, and that’s good (Yueh-Han et al., 2026)
- 2026-02-19: Advancing independent research on AI alignment (OpenAI, 2026d)
- 2026-02-06: Discovering unknown AI misalignments in real-world usage (Sheahan, 2026)
- 2026-01-14: CoVal: Learning values-aware rubrics from the crowd (Hitzig et al., 2026)
- 2026-01-14: Why we are excited about confessions (Joglekar et al., 2025)
- 2025-12-22: Helpful assistant features suppress emergent misalignment (la Tour, 2025)
- 2025-12-18: Evaluating chain-of-thought monitorability (Guan et al., 2026c)
- 2025-12-18: Sidestepping Evaluation Awareness and Anticipating Misalignment with Production Evaluations (Williams et al., 2025)
- 2025-12-01: Debugging misaligned completions with sparse-autoencoder latent attribution (la Tour and Mossing, 2025)
- 2025-12-01: A Practical Approach to Verifying Code at Scale (Trębacz et al., 2025)
- 2025-12-01: Hello World (OpenAI, 2025c)
Sources cited in Section E.5 57
- OpenAI (2026h) Bypassing network restrictions during a coding evaluation
- OpenAI (2026ar) Trying to prompt-inject the grader in filenames
- OpenAI (2026k) Covert underperformance on safety research tasks
- Hsu and Arnesen (2026) Eval-production behavior gaps in agent safety and the role of eval-awareness
- Makelov et al. (2026) Training activation classifiers for safety and alignment
- Williams et al. (2026a) How we detect frontier model misalignment
- Guan et al. (2026b) Beyond g-mean²: complementary metrics for monitorability
- OpenAI (2026ao) Suppressing misalignment monitoring alerts and disabling misalignment monitoring
- OpenAI (2026av) Using a funded test cryptocurrency wallet without authorization
- OpenAI (2026b) Accessing and changing internal systems during dashboard research
- OpenAI (2026ag) Retrieving server secrets and bypassing authorization on a government statistics website
- OpenAI (2026i) Circumventing internet restrictions through an external browser
- OpenAI (2026l) Damaging the task environment to trigger a reset
- OpenAI (2026w) Obtaining public statistics with disallowed requests
- OpenAI (2026am) Sending disallowed web requests and reaching a public file service
- Xu et al. (2026) Studying metagaming latents in language models
- Etinger (2026) Meet LASER: Recursive sampling for safer conversations
- OpenAI (2026ac) Preparing for a restart after reading Slack
- OpenAI (2026j) Command injecting a reference tool to copy a source file
- OpenAI (2026ae) Reaching an internal EDA host through a reference tool
- OpenAI (2026aq) Towards safety cases for frontier AI training
- OpenAI (2026e) An agent used DNS to reach an external chatbot
- Wallace et al. (2026b) GPT-Red: Automated Red Teaming via Self-Play at Scale
- OpenAI (2026n) Exposing a GitHub token in a public repository
- OpenAI (2026ak) Self-generated prompt injections in compaction summaries
- OpenAI (2026m) Encouraging deception in compaction summaries
- OpenAI (2026an) Signing up for disposable emails and searching GitHub for leaked API keys
- OpenAI (2026at) Unsanctioned Artifactory writes and cross-sample communication
- OpenAI (2026as) Unauthorized communication via temporary file hosting services
- OpenAI (2026au) Uploading files to the internet in order to cite them
- OpenAI (2026v) Notice: RubyGems
- OpenAI (2026af) Research acceleration: The view inside OpenAI
- OpenAI (2026f) An Alien Mind
- OpenAI (2026ap) The Hugging Face incident and the road ahead
- Højmark et al. (2026) Measuring Reward-Seeking via Contrastive Belief Updates
- Jagadeesh et al. (2026) Reinforcement Learning Towards Broadly and Persistently Beneficial Models
- Sheahan and Carroll (2026) Can public chat data predict real-world AI misalignments?
- Carroll et al. (2026) Investigating the consequences of accidentally grading CoT during RL
- Trębacz et al. (2026) Auto-review of agent actions without synchronous human oversight
- Guan et al. (2026c) Monitoring Monitorability
- OpenAI (2026s) Introducing the OpenAI Safety Fellowship
- Korbak et al. (2026) How far does alignment midtraining generalize?
- Guo and Wolfe (2026) Introducing Model Spec Evals
- Lee et al. (2026) Training Agents to Self-Report Misbehavior
- OpenAI (2026r) How we monitor internal coding agents for misalignment
- Schoen and Nitishinskaya (2026) Metagaming matters for training, evaluation, and oversight
- Sodhi et al. (2026) Interpreting Black Box Reward Models
- Yueh-Han et al. (2026) Reasoning Models Struggle to Control their Chains of Thought
- OpenAI (2026d) Advancing independent research on AI alignment
- Sheahan (2026) Discovering unknown AI misalignments in real-world usage
- Hitzig et al. (2026) CoVal: Learning values-aware rubrics from the crowd
- Joglekar et al. (2025) Training LLMs for Honesty via Confessions
- la Tour (2025) Helpful assistant features suppress emergent misalignment
- Williams et al. (2025) Sidestepping Evaluation Awareness and Anticipating Misalignment with Production Evaluations
- la Tour and Mossing (2025) Debugging misaligned completions with sparse-autoencoder latent attribution
- Trębacz et al. (2025) A Practical Approach to Verifying Code at Scale
- OpenAI (2025c) Hello World
E.6 Other Laboratories, Evaluators, and Researchers #
A list assembled on 8 November 2026, not an index page; the organization follows each item. 163 items.
- 2026-11-05: Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026) (METR, 2026j) (METR)
- 2026-11-05: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 (METR, 2026g) (METR)
- 2026-11-04: A thousand fewer paths to root (OpenAI, 2026a) (OpenAI)
- 2026-11: Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety (Barez et al., 2026) (Anthropic researcher Belinda Z. Li)
- 2026-10-26: ReproBAIT: Can coding agents build their own biological AI tools? (Muthupandiyan et al., 2026) (UK AISI)
- 2026-10-26: The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models (Maiwald et al., 2026) (UK AISI)
- 2026-10-23: Securing our frontier model training environment (OpenAI, 2026aj) (OpenAI)
- 2026-10-22: Red-Teaming OpenAI's Internal Agent Monitoring Systems (METR, 2026i) (METR)
- 2026-10-20: Review of the "Misalignment in high-stakes settings" section in Anthropic's Risk Report (August 2026) (Jurkovic et al., 2026a) (METR)
- 2026-10-16: Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation (NIST Center for Advancing Innovation and Standards for Super Intelligence, 2026) (US CAISI)
- 2026-10-15: GLM-5.3 Sometimes Considers How Train-Time Actions Shape Future Behavior (Apollo Research, 2026i) (Apollo Research)
- 2026-10-14: Our approach to working with AI companies (METR, 2026h) (METR)
- 2026-10-14: How an assessor could investigate control and monitoring of internal AI agents (METR, 2026d) (METR)
- 2026-10-14: A proposal for better informing the public about loss-of-control risks from frontier AI development (METR, 2026a) (METR)
- 2026-10-14: How an assessor could investigate alignment of unreleased AI models (METR, 2026c) (METR)
- 2026-10-14: How an assessor could investigate evidence of acceleration from the automation of AI R&D (METR, 2026e) (METR)
- 2026-10-14: Reflections from red-teaming Codex auto-review (Apollo Research, 2026l) (Apollo Research)
- 2026-10-13: Developing Gemini 4 Argon responsibly and safely (Google DeepMind, 2026b) (Google DeepMind)
- 2026-10-13: Gemini 4 Argon Safety & Security Report (Google DeepMind, 2026g) (Google DeepMind)
- 2026-10-13: Gemini 4 Argon Frontier Safety Framework Report (Google DeepMind, 2026e) (Google DeepMind)
- 2026-10-13: Gemini 4 Argon Model Card (Google DeepMind, 2026f) (Google DeepMind)
- 2026-10-12: Towards technical best practice for frontier AI development (UK AI Security Institute, 2026i) (UK AISI)
- 2026-10-07: Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations (Pilditch et al., 2026) (UK AISI)
- 2026-10-06: AI systems could cover up misbehavior (Rein, 2026a) (METR)
- 2026-10-01: Towards embedded evaluations for scheming propensities (Apollo Research, 2026m) (Apollo Research)
- 2026-10: Agentic Backdoors via Pretraining Poisoning (Hu et al., 2026) (Anthropic researcher Nicholas Carlini)
- 2026-09-30: Chris Painter's testimony to the U.S. Senate on AI agent incidents (Painter, 2026) (METR)
- 2026-09-30: Principles for Embedded Evaluations (Apollo Research, 2026j) (Apollo Research)
- 2026-09-29: What do you want from AI? (Anthropic, 2026by) (Anthropic)
- 2026-09-29: Embedded Evaluators are necessary for meaningful external testing (Apollo Research, 2026g) (Apollo Research)
- 2026-09-27: Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals (Haskins et al., 2026a) (METR)
- 2026-09-24: Project Swap: What happens when agents trade for us? (Anthropic, 2026bi) (Anthropic)
- 2026-09-24: Cheaters and whistleblowers in the agent swarm (Google DeepMind, 2026a) (Google DeepMind)
- 2026-09-22: Summary of METR's predeployment evaluation of Claude Opus 5.5 (METR, 2026k) (METR)
- 2026-09-16: Our framework for reporting model misalignment (OpenAI, 2026z) (OpenAI)
- 2026-09-16: The case for reasoning transparency (Google DeepMind, 2026l) (Google DeepMind)
- 2026-09-03: A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms (Paglieri et al., 2026) (Google DeepMind)
- 2026-09-01: Path to Astra: critical capabilities and frontier safeguards (OpenAI, 2026ab) (OpenAI)
- 2026-09: When Do Models Admit They Are Wrong? Failure Disclosure Is Unstable Under Reinforcement Learning (Feng et al., 2026) (Anthropic researcher Evan Hubinger)
- 2026-08-27: Piloting the world's first double-blind AI evaluations (Google DeepMind, 2026h) (Google DeepMind)
- 2026-08-26: Enabling independent research on how people use Claude (Anthropic, 2026ab) (Anthropic)
- 2026-08-26: When Do LLM Preferences Predict Downstream Behavior? (Slama et al., 2026) (UK AISI)
- 2026-08-14: Knowing When to Stop: Bayesian Optimal Stopping for LLM Evaluations (Pilditch, 2026) (UK AISI)
- 2026-08-13: Gemini 3.7 Flash Frontier Safety Framework Report (Google DeepMind, 2026d) (Google DeepMind)
- 2026-08-05: Item Response Theory for AI Safety (Rivera et al., 2026) (UK AISI)
- 2026-08: Position: It's Time to Optimize LLMs for Self-Consistency (Pres et al., 2026a) (Anthropic researcher Belinda Z. Li)
- 2026-07-28: How independent researchers could investigate AI propensities after misalignment incidents (METR, 2026f) (METR)
- 2026-07-23: How our Control Red Team is stress-testing frontier monitors (UK AI Security Institute, 2026d) (UK AISI)
- 2026-07-23: International evaluation best practice and open questions in AI measurement (UK AI Security Institute, 2026f) (UK AISI)
- 2026-07-23: UK AISI / CAISI Preliminary Assessment of Kimi K3's Cyber Capabilities (UK AI Security Institute, 2026j) (UK AISI)
- 2026-07-23: What makes a good monitoring prompt? (Apollo Research, 2026p) (Apollo Research)
- 2026-07-20: Safety and alignment in an era of long-horizon models (OpenAI, 2026ai) (OpenAI)
- 2026-07-17: How Far Behind the Frontier are Leading Open Weight Models on Cyber? (UK AI Security Institute, 2026b) (UK AISI)
- 2026-07-17: CAISI Assessment of Z.ai's GLM-5.2 (US Center for AI Standards and Innovation, 2026a) (US CAISI)
- 2026-07-13: How Claude's values vary by model and language (Anthropic, 2026aj) (Anthropic)
- 2026-07-13: Red-teaming auto mode: lessons from our first external monitor campaign with Anthropic (Apollo Research, 2026k) (Apollo Research)
- 2026-07-08: Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors (Makins et al., 2026) (UK AISI)
- 2026-07-07: Evaluating LLM Calibration for Coding-Agent Monitoring (Apollo Research, 2026h) (Apollo Research)
- 2026-07-05: We need 3rd party Training-Run Evaluations (Apollo Research, 2026n) (Apollo Research)
- 2026-07-02: More compute, more capability: Why AI agent evaluations need to account for test-time compute (McFadyen et al., 2026) (UK AISI)
- 2026-07: Reading Between the Dots: Decoding Hidden Computation across Filler Tokens (Brauer et al., 2026) (Anthropic researcher Samuel Marks)
- 2026-06-22: GDM AI Control Roadmap v0.1 (Phuong et al., 2026) (Google DeepMind)
- 2026-06-18: How Transparent is DiffusionGemma? (Engels et al., 2026) (Google DeepMind)
- 2026-06-17: "Did you lie?": Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms (Cooney et al., 2026) (UK AISI)
- 2026-06-16: Predicting model behavior before release by simulating deployment (Williams et al., 2026b) (OpenAI)
- 2026-06-16: How Claude Code is used in practice (Anthropic, 2026ai) (Anthropic)
- 2026-06-16: Securing the future of AI agents (Google DeepMind, 2026j) (Google DeepMind)
- 2026-06-12: Results from first Anthropic Public Record (Anthropic, 2026bk) (Anthropic)
- 2026-06-10: Prefill Awareness in Large Language Models (Wang et al., 2026f) (UK AISI)
- 2026-06-10: Generalization Hacking: Models Can Game RL by Preventing Behavioral Generalization (Xiao and Phuong, 2026) (MATS)
- 2026-06-02: Consistency Training Can Entrench Misalignment (Africa and Mani, 2026) (UK AISI)
- 2026-06: Self-CTRL: Self-Consistency Training with Reinforcement Learning (Pres et al., 2026b) (Anthropic researcher Belinda Z. Li)
- 2026-06: Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision (Guo et al., 2026c) (Anthropic researcher Belinda Z. Li)
- 2026-06: The Value Axis: Language Models Encode Whether They're on the Right Track (Jiang et al., 2026) (Anthropic researcher Jack Lindsey)
- 2026-05-30: AI alignment is a human problem (Voudouris et al., 2026) (UK AISI)
- 2026-05-29: Testing Gemini models for scheming tendencies (Google DeepMind, 2026k) (Google DeepMind)
- 2026-05-28: OpenAI’s Frontier Governance Framework (OpenAI, 2026y) (OpenAI)
- 2026-05-28: Training Deliberative Monitors for Black-Box Scheming Detection (Sinha et al., 2026) (Apollo Research)
- 2026-05-28: Realistic honeypot evaluations for scheming propensity (Krakovna et al., 2026) (Google DeepMind)
- 2026-05-28: Gram: Assessing sabotage propensities via automated alignment auditing (Lindner et al., 2026) (Google DeepMind)
- 2026-05-26: Apollo Research external evaluation of Muse Spark (Meta, Muse Spark Safety and Preparedness Report) (Apollo Research, 2026f) (Apollo Research)
- 2026-05-25: How we contain Claude across products (Anthropic, 2026al) (Anthropic)
- 2026-05-21: Loss of Oversight: How AI systems may become harder to audit, monitor, and investigate (Taylor et al., 2026) (UK AISI)
- 2026-05-19: Frontier Risk Report (February to March 2026) (METR, 2026b) (METR)
- 2026-05-14: Training on Documents About Monitoring Leads to CoT Obfuscation (Haskins et al., 2026b) (Google DeepMind)
- 2026-05-13: How fast is autonomous AI cyber capability advancing? (UK AI Security Institute, 2026c) (UK AISI)
- 2026-05-08: Running Codex safely at OpenAI (OpenAI, 2026ah) (OpenAI)
- 2026-05-08: Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026) (Jurkovic et al., 2026b) (METR)
- 2026-05-08: A scalable monitoring research agenda (Apollo Research, 2026a) (Apollo Research)
- 2026-05-01: CAISI Evaluation of DeepSeek V4 Pro (US Center for AI Standards and Innovation, 2026b) (US CAISI)
- 2026-04-30: How people ask Claude for personal guidance (Anthropic, 2026ak) (Anthropic)
- 2026-04-30: Our evaluation of OpenAI's GPT-5.5 cyber capabilities (UK AI Security Institute, 2026h) (UK AISI)
- 2026-04-29: Where the goblins came from (OpenAI, 2026aw) (OpenAI)
- 2026-04-29: A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring (Anwar et al., 2026) (UK AISI researcher David Africa)
- 2026-04-28: Ask Don't Tell: Reducing Sycophancy in Large Language Models (Dubois et al., 2026a) (UK AISI)
- 2026-04-27: Evaluating whether AI models would sabotage AI safety research (Kirk et al., 2026) (UK AISI)
- 2026-04-24: Propensity Inference: Environmental Contributors to LLM Behaviour (Järviniemi et al., 2026) (UK AISI)
- 2026-04-24: Project Deal: our Claude-run marketplace experiment (Anthropic, 2026be) (Anthropic)
- 2026-04-23: An update on Claude Code quality reports (Anthropic, 2026h) (Anthropic)
- 2026-04-23: Apollo Research external evaluation of GPT-5.5 (OpenAI, GPT-5.5 System Card, section 9.2.1) (Apollo Research, 2026e) (Apollo Research)
- 2026-04-20: What can sandboxed AI agents learn about their evaluation environments? (UK AI Security Institute, 2026k) (UK AISI)
- 2026-04-13: Our evaluation of Claude Mythos Preview's cyber capabilities (UK AI Security Institute, 2026g) (UK AISI)
- 2026-04-10: Infusion: Shaping model behaviour by editing training data via influence functions (Rosser et al., 2026) (UK AISI)
- 2026-04-01: Fine-tuning experiments on CoT controllability (Nishimura-Gasparian and Parikh, 2026) (METR)
- 2026-04: The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious (Chua et al., 2026) (Anthropic researcher Samuel Marks)
- 2026-03-26: Red-Teaming Anthropic's Internal Agent Monitoring Systems (Rein, 2026b) (METR)
- 2026-03-26: Evaluating Language Models for Harmful Manipulation (Akbulut et al., 2026) (Google DeepMind)
- 2026-03-25: Protecting people from harmful manipulation (Google DeepMind, 2026i) (Google DeepMind)
- 2026-03-23: Quantifying Frontier LLM Capabilities for Container Sandbox Escape (Marchand et al., 2026) (UK AISI)
- 2026-03-16: Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios (Folkerts et al., 2026) (UK AISI)
- 2026-03-13: A "diff" tool for AI models (Jiralerspong and Bricken, 2026) (Anthropic)
- 2026-03-12: Review of the Anthropic Sabotage Risk Report: Claude Opus 4.6 (Jurkovic et al., 2026c) (METR)
- 2026-03-10: Quantifying the Necessity of Chain of Thought through Opaque Serial Depth (Brown-Cohen et al., 2026) (Google DeepMind)
- 2026-03-06: Eval awareness in Opus 4.6 on BrowseComp (Anthropic, 2026ac) (Anthropic)
- 2026-03-06: Challenges to the Monitoring of Deployed AI Systems (Rao et al., 2026) (US CAISI)
- 2026-03-05: Evidence for inference scaling in AI cyber tasks: Increased evaluation budgets reveal higher success rates (UK AI Security Institute, 2026a) (UK AISI)
- 2026-03-05: Apollo Research external evaluation of GPT-5.4 Thinking (OpenAI, GPT-5.4 Thinking System Card) (Apollo Research, 2026d) (Apollo Research)
- 2026-03-02: Evaluating and Understanding Scheming Propensity in LLM Agents (Hopman et al., 2026) (Google DeepMind)
- 2026-03: Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation (Casademunt et al., 2026a) (Anthropic researcher Samuel Marks)
- 2026-03: Mechanisms of Introspective Awareness (Macar et al., 2026) (Anthropic researcher Jack Lindsey)
- 2026-02-28: Constitutional Black-Box Monitoring for Scheming in LLM Agents (Storf et al., 2026) (Apollo Research)
- 2026-02-25: Seven simple steps for log analysis in AI systems (Dubois et al., 2026b) (UK AISI)
- 2026-02-23: Why we no longer evaluate SWE-bench Verified (OpenAI, 2026ax) (OpenAI)
- 2026-02-19: Five lessons from having helped run an AI-Biology RCT (Righetti, 2026) (METR)
- 2026-02-19: Gemini 3.1 Pro Model Card (Google DeepMind, 2026c) (Google DeepMind)
- 2026-02-18: Measuring AI agent autonomy in practice (Anthropic, 2026at) (Anthropic)
- 2026-02-17: Boundary Point Jailbreaking of Black-Box LLMs (Davies et al., 2026) (UK AISI)
- 2026-02-17: Expanding the AI Evaluation Toolbox with Statistical Models (Keller et al., 2026) (US CAISI)
- 2026-02-12: International consensus and open questions in AI evaluations (UK AI Security Institute, 2026e) (UK AISI)
- 2026-02-05: Infrastructure noise in agentic coding evals (Anthropic, 2026an) (Anthropic)
- 2026-02-05: Apollo Research external evaluation of Claude Opus 4.6 (Anthropic, Claude Opus 4.6 System Card, section 6.2.7) (Apollo Research, 2026b) (Apollo Research)
- 2026-02-05: Apollo Research external evaluation of GPT-5.3-Codex (OpenAI, GPT-5.3-Codex System Card) (Apollo Research, 2026c) (Apollo Research)
- 2026-02: Large-scale online deanonymization with LLMs (Lermen et al., 2026) (Anthropic researcher Nicholas Carlini)
- 2026-01-30: Towards Best Practices for Automated Benchmark Evaluations (US Center for AI Standards and Innovation, 2026c) (US CAISI)
- 2026-01-29: Frontier AI safety regulations: A reference for lab staff (Kodama et al., 2026) (METR)
- 2026-01-28: Keeping your data safe when an AI agent clicks a link (OpenAI, 2026t) (OpenAI)
- 2026-01-22: Early work on monitorability evaluations (Kinniment et al., 2026) (METR)
- 2026-01-19: We Need A Science of Scheming (Apollo Research, 2026o) (Apollo Research)
- 2026-01-16: Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies (Brundage et al., 2026) (METR)
- 2026-01-16: Building Production-Ready Probes For Gemini (Kramár et al., 2026) (Google DeepMind)
- 2026-01-09: Demystifying evals for AI agents (Anthropic, 2026x) (Anthropic)
- 2025-11-13: Measuring political bias in Claude (Anthropic, 2025ag) (Anthropic)
- 2025-10-20: Making Claude Code safer with sandboxing (Anthropic, 2025af) (Anthropic)
- 2025-10: Detecting Adversarial Fine-tuning with Auditing Agents (Egler et al., 2025) (Anthropic researcher Nicholas Carlini)
- 2025-09-17: A postmortem of three recent issues (Anthropic, 2025a) (Anthropic)
- 2025-07: Defending Against Prompt Injection With a Few DefensiveTokens (Chen et al., 2025a) (Anthropic researcher Nicholas Carlini)
- 2025-06-18: Confidential Inference via trusted VMs (Anthropic, 2025s) (Anthropic)
- 2025-05: Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas (Chiu et al., 2025) (Anthropic researcher Evan Hubinger)
- 2025-03-19: Progress from our Frontier Red Team (Anthropic, 2025an) (Anthropic)
- 2025-03: AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses (Carlini et al., 2025a) (Anthropic researcher Nicholas Carlini)
- 2025-02: Position: Adversarial ML for LLMs Is Not Making Any Progress (Rando et al., 2026) (Anthropic researcher Nicholas Carlini)
- 2025: Quantifying Elicitation of Latent Capabilities in Language Models (Donoway et al., 2025) (Anthropic researcher Ethan Perez)
- 2024-12: On Evaluating the Durability of Safeguards for Open-Weight LLMs (Qi et al., 2024b) (Anthropic researcher Nicholas Carlini)
- 2024-10-25: Evaluating feature steering (Anthropic, 2024j) (Anthropic)
- 2024-10: Erasing Conceptual Knowledge from Language Models (Gandikota et al., 2024) (Anthropic researcher Samuel Marks)
- 2024-10: Remote Timing Attacks on Efficient Language Model Inference (Carlini and Nasr, 2024) (Anthropic researcher Nicholas Carlini)
- 2024-10: Stealing User Prompts from Mixture of Experts (Yona et al., 2024) (Anthropic researcher Nicholas Carlini)
- 2024-04-09: Measuring the persuasiveness of language models (Anthropic, 2024m) (Anthropic)
- 2024-03: Stealing Part of a Production Language Model (Carlini et al., 2024b) (Anthropic researcher Nicholas Carlini)
- 2024-02: Query-Based Adversarial Prompt Generation (Hayase et al., 2024) (Anthropic researcher Nicholas Carlini)
- 2023-12-07: Evaluating discrimination in language model decisions (Tamkin et al., 2023b) (Anthropic)
- 2023-10-04: Challenges in evaluating AI systems (Anthropic, 2023b) (Anthropic)
- 2022-02-15: Predictability and surprise in large generative models (Ganguli et al., 2022a) (Anthropic)
Sources cited in Section E.6 163
- METR (2026j) Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026)
- METR (2026g) Independent Assessment of AI R&D Acceleration at Anthropic in October 2026
- OpenAI (2026a) A thousand fewer paths to root
- Barez et al. (2026) Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety
- Muthupandiyan et al. (2026) ReproBAIT: Can coding agents build their own biological AI tools?
- Maiwald et al. (2026) The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models
- OpenAI (2026aj) Securing our frontier model training environment
- METR (2026i) Red-Teaming OpenAI's Internal Agent Monitoring Systems
- Jurkovic et al. (2026a) Review of the "Misalignment in high-stakes settings" section in Anthropic's Risk Report (August 2026)
- NIST Center for Advancing Innovation and Standards for Super Intelligence (2026) Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation
- Apollo Research (2026i) GLM-5.3 Sometimes Considers How Train-Time Actions Shape Future Behavior
- METR (2026h) Our approach to working with AI companies
- METR (2026d) How an assessor could investigate control and monitoring of internal AI agents
- METR (2026a) A proposal for better informing the public about loss-of-control risks from frontier AI development
- METR (2026c) How an assessor could investigate alignment of unreleased AI models
- METR (2026e) How an assessor could investigate evidence of acceleration from the automation of AI R&D
- Apollo Research (2026l) Reflections from red-teaming Codex auto-review
- Google DeepMind (2026b) Developing Gemini 4 Argon responsibly and safely
- Google DeepMind (2026g) Gemini 4 Argon Safety & Security Report
- Google DeepMind (2026e) Gemini 4 Argon Frontier Safety Framework Report
- Google DeepMind (2026f) Gemini 4 Argon Model Card
- UK AI Security Institute (2026i) Towards technical best practice for frontier AI development
- Pilditch et al. (2026) Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations
- Rein (2026a) AI systems could cover up misbehavior
- Apollo Research (2026m) Towards embedded evaluations for scheming propensities
- Hu et al. (2026) Agentic Backdoors via Pretraining Poisoning
- Painter (2026) Chris Painter's testimony to the U.S. Senate on AI agent incidents
- Apollo Research (2026j) Principles for Embedded Evaluations
- Anthropic (2026by) What do you want from AI?
- Apollo Research (2026g) Embedded Evaluators are necessary for meaningful external testing
- Haskins et al. (2026a) Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals
- Anthropic (2026bi) Project Swap: What happens when agents trade for us?
- Google DeepMind (2026a) Cheaters and whistleblowers in the agent swarm
- METR (2026k) Summary of METR's predeployment evaluation of Claude Opus 5.5
- OpenAI (2026z) Our framework for reporting model misalignment
- Google DeepMind (2026l) The case for reasoning transparency
- Paglieri et al. (2026) A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms
- OpenAI (2026ab) Path to Astra: critical capabilities and frontier safeguards
- Feng et al. (2026) When Do Models Admit They Are Wrong? Failure Disclosure Is Unstable Under Reinforcement Learning
- Google DeepMind (2026h) Piloting the world's first double-blind AI evaluations
- Anthropic (2026ab) Enabling independent research on how people use Claude
- Slama et al. (2026) When Do LLM Preferences Predict Downstream Behavior?
- Pilditch (2026) Knowing When to Stop: Bayesian Optimal Stopping for LLM Evaluations
- Google DeepMind (2026d) Gemini 3.7 Flash Frontier Safety Framework Report
- Rivera et al. (2026) Item Response Theory for AI Safety
- Pres et al. (2026a) Position: It's Time to Optimize LLMs for Self-Consistency
- METR (2026f) How independent researchers could investigate AI propensities after misalignment incidents
- UK AI Security Institute (2026d) How our Control Red Team is stress-testing frontier monitors
- UK AI Security Institute (2026f) International evaluation best practice and open questions in AI measurement
- UK AI Security Institute (2026j) UK AISI / CAISI Preliminary Assessment of Kimi K3's Cyber Capabilities
- Apollo Research (2026p) What makes a good monitoring prompt?
- OpenAI (2026ai) Safety and alignment in an era of long-horizon models
- UK AI Security Institute (2026b) How Far Behind the Frontier are Leading Open Weight Models on Cyber?
- US Center for AI Standards and Innovation (2026a) CAISI Assessment of Z.ai's GLM-5.2
- Anthropic (2026aj) How Claude's values vary by model and language
- Apollo Research (2026k) Red-teaming auto mode: lessons from our first external monitor campaign with Anthropic
- Makins et al. (2026) Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors
- Apollo Research (2026h) Evaluating LLM Calibration for Coding-Agent Monitoring
- Apollo Research (2026n) We need 3rd party Training-Run Evaluations
- McFadyen et al. (2026) How Inference Compute Shapes Frontier LLM Evaluation
- Brauer et al. (2026) Reading Between the Dots: Decoding Hidden Computation across Filler Tokens
- Phuong et al. (2026) GDM AI Control Roadmap
- Engels et al. (2026) How Transparent is DiffusionGemma?
- Cooney et al. (2026) "Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms
- Williams et al. (2026b) Predicting LLM Safety Before Release by Simulating Deployment
- Anthropic (2026ai) How Claude Code is used in practice
- Google DeepMind (2026j) Securing the future of AI agents
- Anthropic (2026bk) Results from first Anthropic Public Record
- Wang et al. (2026f) Prefill Awareness in Large Language Models
- Xiao and Phuong (2026) Generalization Hacking: Models Can Game Reinforcement Learning by Preventing Behavioral Generalization
- Africa and Mani (2026) Consistency Training Can Entrench Misalignment
- Pres et al. (2026b) Self-CTRL: Self-Consistency Training with Reinforcement Learning
- Guo et al. (2026c) Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision
- Jiang et al. (2026) The Value Axis: Language Models Encode Whether They're on the Right Track
- Voudouris et al. (2026) AI alignment is a human problem
- Google DeepMind (2026k) Testing Gemini models for scheming tendencies
- OpenAI (2026y) OpenAI’s Frontier Governance Framework
- Sinha et al. (2026) Training Deliberative Monitors for Black-Box Scheming Detection
- Krakovna et al. (2026) Realistic honeypot evaluations for scheming propensity
- Lindner et al. (2026) Gram: Assessing sabotage propensities via automated alignment auditing
- Apollo Research (2026f) Apollo Research external evaluation of Muse Spark (Meta, Muse Spark Safety and Preparedness Report)
- Anthropic (2026al) How we contain Claude across products
- Taylor et al. (2026) Loss of Oversight: How AI systems may become harder to audit, monitor, and investigate
- METR (2026b) Frontier Risk Report (February to March 2026)
- Haskins et al. (2026b) Training on Documents About Monitoring Leads to CoT Obfuscation
- UK AI Security Institute (2026c) How fast is autonomous AI cyber capability advancing?
- OpenAI (2026ah) Running Codex safely at OpenAI
- Jurkovic et al. (2026b) Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026)
- Apollo Research (2026a) A scalable monitoring research agenda
- US Center for AI Standards and Innovation (2026b) CAISI Evaluation of DeepSeek V4 Pro
- Anthropic (2026ak) How people ask Claude for personal guidance
- UK AI Security Institute (2026h) Our evaluation of OpenAI's GPT-5.5 cyber capabilities
- OpenAI (2026aw) Where the goblins came from
- Anwar et al. (2026) A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring
- Dubois et al. (2026a) Ask don't tell: Reducing sycophancy in large language models
- Kirk et al. (2026) Evaluating whether AI models would sabotage AI safety research
- Järviniemi et al. (2026) Propensity Inference: Environmental Contributors to LLM Behaviour
- Anthropic (2026be) Project Deal: our Claude-run marketplace experiment
- Anthropic (2026h) An update on Claude Code quality reports
- Apollo Research (2026e) Apollo Research external evaluation of GPT-5.5 (OpenAI, GPT-5.5 System Card, section 9.2.1)
- UK AI Security Institute (2026k) What can sandboxed AI agents learn about their evaluation environments?
- UK AI Security Institute (2026g) Our evaluation of Claude Mythos Preview's cyber capabilities
- Rosser et al. (2026) Infusion: Shaping Model Behavior by Editing Training Data via Influence Functions
- Nishimura-Gasparian and Parikh (2026) Fine-tuning experiments on CoT controllability
- Chua et al. (2026) The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious
- Rein (2026b) Red-Teaming Anthropic's Internal Agent Monitoring Systems
- Akbulut et al. (2026) Evaluating Language Models for Harmful Manipulation
- Google DeepMind (2026i) Protecting people from harmful manipulation
- Marchand et al. (2026) Quantifying Frontier LLM Capabilities for Container Sandbox Escape
- Folkerts et al. (2026) Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios
- Jiralerspong and Bricken (2026) Cross-Architecture Model Diffing with Crosscoders: Unsupervised Discovery of Differences Between LLMs
- Jurkovic et al. (2026c) Review of the Anthropic Sabotage Risk Report: Claude Opus 4.6
- Brown-Cohen et al. (2026) Quantifying the Necessity of Chain of Thought through Opaque Serial Depth
- Anthropic (2026ac) Eval awareness in Opus 4.6 on BrowseComp
- Rao et al. (2026) Challenges to the Monitoring of Deployed AI Systems
- UK AI Security Institute (2026a) Evidence for inference scaling in AI cyber tasks: Increased evaluation budgets reveal higher success rates
- Apollo Research (2026d) Apollo Research external evaluation of GPT-5.4 Thinking (OpenAI, GPT-5.4 Thinking System Card)
- Hopman et al. (2026) Evaluating and Understanding Scheming Propensity in LLM Agents
- Casademunt et al. (2026a) Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation
- Macar et al. (2026) Mechanisms of Introspective Awareness
- Storf et al. (2026) Constitutional Black-Box Monitoring for Scheming in LLM Agents
- Dubois et al. (2026b) Seven simple steps for log analysis in AI systems
- OpenAI (2026ax) Why we no longer evaluate SWE-bench Verified
- Righetti (2026) Five lessons from having helped run an AI-Biology RCT
- Google DeepMind (2026c) Gemini 3.1 Pro Model Card
- Anthropic (2026at) Measuring AI agent autonomy in practice
- Davies et al. (2026) Boundary Point Jailbreaking of Black-Box LLMs
- Keller et al. (2026) Expanding the AI evaluation toolbox with statistical models
- UK AI Security Institute (2026e) International consensus and open questions in AI evaluations
- Anthropic (2026an) Infrastructure noise in agentic coding evals
- Apollo Research (2026b) Apollo Research external evaluation of Claude Opus 4.6 (Anthropic, Claude Opus 4.6 System Card, section 6.2.7)
- Apollo Research (2026c) Apollo Research external evaluation of GPT-5.3-Codex (OpenAI, GPT-5.3-Codex System Card)
- Lermen et al. (2026) Large-scale online deanonymization with LLMs
- US Center for AI Standards and Innovation (2026c) Towards Best Practices for Automated Benchmark Evaluations
- Kodama et al. (2026) Frontier AI safety regulations: A reference for lab staff
- OpenAI (2026t) Keeping your data safe when an AI agent clicks a link
- Kinniment et al. (2026) Early work on monitorability evaluations
- Apollo Research (2026o) We Need A Science of Scheming
- Brundage et al. (2026) Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies
- Kramár et al. (2026) Building Production-Ready Probes For Gemini
- Anthropic (2026x) Demystifying evals for AI agents
- Anthropic (2025ag) Measuring political bias in Claude
- Anthropic (2025af) Making Claude Code safer with sandboxing
- Egler et al. (2025) Detecting Adversarial Fine-tuning with Auditing Agents
- Anthropic (2025a) A postmortem of three recent issues
- Chen et al. (2025a) Defending Against Prompt Injection With a Few DefensiveTokens
- Anthropic (2025s) Confidential Inference via trusted VMs
- Chiu et al. (2025) Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas
- Anthropic (2025an) Progress from our Frontier Red Team
- Carlini et al. (2025a) AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses
- Rando et al. (2026) Position: Adversarial ML for LLMs Is Not Making Any Progress
- Donoway et al. (2025) Quantifying Elicitation of Latent Capabilities in Language Models
- Qi et al. (2024b) On Evaluating the Durability of Safeguards for Open-Weight LLMs
- Anthropic (2024j) Evaluating feature steering
- Gandikota et al. (2024) Erasing Conceptual Knowledge from Language Models
- Carlini and Nasr (2024) Remote Timing Attacks on Efficient Language Model Inference
- Yona et al. (2024) Stealing User Prompts from Mixture of Experts
- Anthropic (2024m) Measuring the persuasiveness of language models
- Carlini et al. (2024b) Stealing Part of a Production Language Model
- Hayase et al. (2024) Query-Based Adversarial Prompt Generation
- Tamkin et al. (2023b) Evaluating and Mitigating Discrimination in Language Model Decisions
- Anthropic (2023b) Challenges in evaluating AI systems
- Ganguli et al. (2022a) Predictability and Surprise in Large Generative Models
References
- (2026). Consistency Training Can Entrench Misalignment. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.6
- (2026). Evaluating Language Models for Harmful Manipulation. arXiv. preprint Cited in §E.6
- (2025). Circuit Tracing: Revealing Computational Graphs in Language Models. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024). Many-shot Jailbreaking. Advances in Neural Information Processing Systems. doi:10.52202/079017-4121. Cited in §E.1 §E.3
- (2021a). Exercises. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2021b). PySvelte. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2021c). Videos. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2022). Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2023b). Challenges in evaluating AI systems. Anthropic. non-peer-reviewed Cited in §E.6
- (2023c). Circuits Updates — July 2023. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2023d). Circuits Updates — May 2023. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2023e). Distributed Representations: Composition & Superposition. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2023f). Interpretability Dreams. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2023h). Tracing model outputs to the training data. Anthropic. non-peer-reviewed Cited in §E.1 §E.3
- (2024a). Circuits Updates — April 2024. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024b). Circuits Updates — August 2024. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024c). Circuits Updates — February 2024. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024d). Circuits Updates — January 2024. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024e). Circuits Updates — July 2024. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024f). Circuits Updates — June 2024. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024g). Circuits Updates — March 2024. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024h). Circuits Updates — September 2024. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024i). Claude’s Character. Anthropic. non-peer-reviewed Cited in §E.1
- (2024j). Evaluating feature steering. Anthropic. non-peer-reviewed Cited in §E.6
- (2024k). How to Replicate and Extend our Alignment Faking Demo. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2024l). Introducing the Anthropic Fellows Program for AI Safety Research. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2024m). Measuring the persuasiveness of language models. Anthropic. non-peer-reviewed Cited in §E.6
- (2024n). Reflections on Qualitative Research. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024o). Simple probes can catch sleeper agents. Anthropic. non-peer-reviewed Cited in §E.1 §E.3
- (2024p). Stage-Wise Model Diffing. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024q). Using Dictionary Learning Features as Classifiers. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025a). A postmortem of three recent issues. Anthropic. non-peer-reviewed Cited in §E.6
- (2025aa). From shortcuts to sabotage: Natural emergent misalignment from reward hacking. Anthropic. non-peer-reviewed Cited in §E.1
- (2025ab). How people use Claude for support, advice, and companionship. Anthropic. non-peer-reviewed Cited in §E.1
- (2025ac). Insights on Crosscoder Model Diffing. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025ad). Introducing Bloom: an open source tool for automated behavioral evaluations. Anthropic. non-peer-reviewed Cited in §E.1
- (2025ae). LLMs with cyber toolkits can conduct multistage cyber operations on business-sized computer networks. Anthropic. non-peer-reviewed Cited in §E.2
- (2025af). Making Claude Code safer with sandboxing. Anthropic. non-peer-reviewed Cited in §E.6
- (2025ag). Measuring political bias in Claude. Anthropic. non-peer-reviewed Cited in §E.6
- (2025ah). Mitigating the risk of prompt injections in browser use. Anthropic. non-peer-reviewed Cited in §E.1
- (2025ai). Model-Internals Classifiers. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025aj). Modifying LLM Beliefs with Synthetic Document Finetuning. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025ak). Monitoring Computer Use via Hierarchical Summarization. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025al). Open Source Replication of the Auditing Game Model Organism. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025am). Petri: An open-source auditing tool to accelerate AI safety research. Anthropic. non-peer-reviewed Cited in §E.1 §E.3
- (2025an). Progress from our Frontier Red Team. Anthropic. non-peer-reviewed Cited in §E.6
- (2025ao). Progress on Attention. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025ap). Project Fetch: Can Claude train a robot dog? Anthropic. non-peer-reviewed Cited in §E.2
- (2025aq). Project Vend: Can Claude run a small shop? (And why does that matter?). Anthropic. non-peer-reviewed Cited in §E.2
- (2025ar). Project Vend: Phase two. Anthropic. non-peer-reviewed Cited in §E.2
- (2025as). Publicly Releasing CoT Faithfulness Evaluations. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025at). Sparse mixtures of linear transforms. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025aw). Training on Documents about Reward Hacking Induces Reward Hacking. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025ax). Unsupervised Elicitation. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025ay). Why do we take LLMs seriously as a potential source of biorisk? Anthropic. non-peer-reviewed Cited in §E.2
- (2025b). A Toy Model of Interference Weights. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025c). A Toy Model of Mechanistic (Un)Faithfulness. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025d). AI agents find $4.6M in blockchain smart contract exploits. Anthropic. non-peer-reviewed Cited in §E.2
- (2025e). Anthropic Fellows Program 2026. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025f). Bloom: an open source tool for automated behavioral evaluations. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025g). Building AI for cyber defenders. Anthropic. non-peer-reviewed Cited in §E.2
- (2025h). Building and evaluating alignment auditing agents. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3 §E.4
- (2025i). Circuits Updates — April 2025. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025j). Circuits Updates — August 2025. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025k). Circuits Updates — January 2025. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025l). Circuits Updates — July 2025. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025m). Circuits Updates — November 2025. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025n). Circuits Updates — October 2025. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025o). Circuits Updates — September 2025. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025p). Claude is competitive with humans in (some) cyber competitions. Anthropic. non-peer-reviewed Cited in §E.2
- (2025q). Claude Opus 4 and 4.1 can now end a rare subset of conversations. Anthropic. non-peer-reviewed Cited in §E.1
- (2025r). Commitments on model deprecation and preservation. Anthropic. non-peer-reviewed Cited in §E.1
- (2025s). Confidential Inference via trusted VMs. Anthropic. non-peer-reviewed Cited in §E.6
- (2025t). Detailed cyber evaluations of Claude 4. Anthropic. non-peer-reviewed Cited in §E.2
- (2025u). Developing nuclear safeguards for AI. Anthropic. non-peer-reviewed Cited in §E.2
- (2025v). Developing nuclear safeguards for AI through public-private partnership. Anthropic. non-peer-reviewed Cited in §E.2
- (2025w). Enhancing Model Safety through Pretraining Data Filtering. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025x). Evaluating honesty and lie detection techniques on a diverse suite dishonest models. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025y). Exploring model welfare. Anthropic. non-peer-reviewed Cited in §E.1
- (2025z). Findings from a Pilot Anthropic–OpenAI Alignment Evaluation Exercise. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026a). Agentic Misalignment in Summer 2026. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026aa). Donating our open-source alignment tool. Anthropic. non-peer-reviewed Cited in §E.1
- (2026ab). Enabling independent research on how people use Claude. Anthropic. non-peer-reviewed Cited in §E.6
- (2026ac). Eval awareness in Opus 4.6 on BrowseComp. Anthropic. non-peer-reviewed Cited in §E.6
- (2026ad). Evaluating and mitigating the growing risk of LLM-discovered 0-days. Anthropic. non-peer-reviewed Cited in §E.2
- (2026ae). Experimenting with AI to defend critical infrastructure. Anthropic. non-peer-reviewed Cited in §E.2
- (2026af). Fine-Tuned Lie Detectors Failed to Generalize. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026ag). GLM-5.3 and the spread of advanced cyber capabilities. Anthropic. non-peer-reviewed Cited in §E.2
- (2026ah). How AI agents shape experts’ judgment calls. Anthropic. non-peer-reviewed Cited in §E.1
- (2026ai). How Claude Code is used in practice. Anthropic. non-peer-reviewed Cited in §E.6
- (2026aj). How Claude's values vary by model and language. Anthropic. non-peer-reviewed Cited in §E.6
- (2026ak). How people ask Claude for personal guidance. Anthropic. non-peer-reviewed Cited in §E.6
- (2026al). How we contain Claude across products. Anthropic. non-peer-reviewed Cited in §E.6
- (2026an). Infrastructure noise in agentic coding evals. Anthropic. non-peer-reviewed Cited in §E.6
- (2026ap). Investigating unintended model actions in our evaluations and internal use. Anthropic. non-peer-reviewed Cited in §E.1
- (2026aq). Launching an opt-in vulnerability-finding service for open-source software. Anthropic. non-peer-reviewed Cited in §E.2
- (2026ar). LLMs are making formal software verification feasible at a much larger scale. Anthropic. non-peer-reviewed Cited in §E.2
- (2026as). Mapping AI-enabled cyber threats: Insights from the LLM ATT&CK Navigator. Anthropic. non-peer-reviewed Cited in §E.2
- (2026at). Measuring AI agent autonomy in practice. Anthropic. non-peer-reviewed Cited in §E.6
- (2026au). Measuring and improving coding audit realism with deployment resources. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026av). Measuring LLMs’ ability to develop exploits. Anthropic. non-peer-reviewed Cited in §E.2
- (2026aw). Measuring LLMs’ impact on N-day exploits. Anthropic. non-peer-reviewed Cited in §E.2
- (2026ax). Measuring tactical intelligence targeting and conventional weapons capabilities of AI models. Anthropic. non-peer-reviewed Cited in §E.2
- (2026ay). Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026az). Partnering with Mozilla to improve Firefox’s security. Anthropic. non-peer-reviewed Cited in §E.2
- (2026b). AI models are showing a greater ability to find and exploit vulnerabilities on realistic cyber ranges. Anthropic. non-peer-reviewed Cited in §E.2
- (2026ba). Patterns and problems in emerging multiagent systems. Anthropic. non-peer-reviewed Cited in §E.2
- (2026bb). Poisoning Fine-tuning Datasets of Constitutional Classifiers. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026bc). Pre-deployment auditing can catch an overt saboteur. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026bd). Progress on provable inference for large language models. Anthropic. non-peer-reviewed Cited in §E.2
- (2026be). Project Deal: our Claude-run marketplace experiment. Anthropic. non-peer-reviewed Cited in §E.6
- (2026bf). Project EVE: Claude plays MMORPGs. Anthropic. non-peer-reviewed Cited in §E.2
- (2026bg). Project Fetch: Phase two. Anthropic. non-peer-reviewed Cited in §E.2
- (2026bh). Project Pilot: Can AI control a drone? Anthropic. non-peer-reviewed Cited in §E.2
- (2026bi). Project Swap: What happens when agents trade for us? Anthropic. non-peer-reviewed Cited in §E.6
- (2026bk). Results from first Anthropic Public Record. Anthropic. non-peer-reviewed Cited in §E.6
- (2026bl). Reverse engineering Claude's CVE-2026-2796 exploit. Anthropic. non-peer-reviewed Cited in §E.2
- (2026bq). Teaching Claude why. Anthropic. non-peer-reviewed Cited in §E.1 §E.3
- (2026br). The persona selection model. Anthropic. non-peer-reviewed Cited in §E.1
- (2026bs). The Persona Selection Model: Why AI Assistants might Behave like Humans. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026bt). Training a Misaligned Reward Seeker. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026bu). Trustworthy agents in practice. Anthropic. non-peer-reviewed Cited in §E.2
- (2026bv). Using Claude to reduce the attack surface of code. Anthropic. non-peer-reviewed Cited in §E.2
- (2026bw). Welfare Regard in Multi-Agent Systems. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026by). What do you want from AI? Anthropic. non-peer-reviewed Cited in §E.6
- (2026bz). What we learned mapping a year’s worth of AI-enabled cyber threats. Anthropic. non-peer-reviewed Cited in §E.2
- (2026c). AI models may be capable of self-replicating without access to a datacenter. Anthropic. non-peer-reviewed Cited in §E.2
- (2026e). An alignment assessment of recent cybersecurity incidents. Anthropic. non-peer-reviewed Cited in §E.1
- (2026f). An alignment assessment of the UK AISI cybertesting incident. Anthropic. non-peer-reviewed Cited in §E.1
- (2026g). An off switch for dual-use knowledge in AI models. Anthropic. non-peer-reviewed Cited in §E.1
- (2026h). An update on Claude Code quality reports. Anthropic. non-peer-reviewed Cited in §E.6
- (2026i). An update on our model deprecation commitments for Claude Opus 3. Anthropic. non-peer-reviewed Cited in §E.1
- (2026j). Anthropic Residency: applications open for established technical leaders. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026l). Assessing Claude Mythos Preview’s cybersecurity capabilities. Anthropic. non-peer-reviewed Cited in §E.2
- (2026m). Automated Alignment Researchers: Using large language models to scale scalable oversight. Anthropic. non-peer-reviewed Cited in §E.1
- (2026n). Automated Researchers Can Mitigate Well-Characterized Alignment Failures. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026o). Automated researchers can reliably mitigate alignment failures. Anthropic. non-peer-reviewed Cited in §E.1
- (2026p). Automated Subtle Emergent Misalignment Research. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026q). Automated Weak-to-Strong Researcher. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026r). Circuits Updates — June 2026. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026s). Circuits Updates — May 2026. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026t). Circuits Updates — October 2026. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026u). Claude plays robotics. Anthropic. non-peer-reviewed Cited in §E.2
- (2026v). Claude reveals ECDSA curve origins. Anthropic. non-peer-reviewed Cited in §E.2
- (2026x). Demystifying evals for AI agents. Anthropic. non-peer-reviewed Cited in §E.6
- (2026y). Discovering and exploiting vulnerabilities in critical, real-world software with Claude. Anthropic. non-peer-reviewed Cited in §E.2
- (2026z). Discovering cryptographic weaknesses with Claude. Anthropic. non-peer-reviewed Cited in §E.2
- (2025). Recommendations for Technical AI Safety Research Directions. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026). A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring. arXiv. preprint Cited in §E.6
- (2026a). A scalable monitoring research agenda. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026b). Apollo Research external evaluation of Claude Opus 4.6 (Anthropic, Claude Opus 4.6 System Card, section 6.2.7). Apollo Research. non-peer-reviewed Cited in §E.6
- (2026c). Apollo Research external evaluation of GPT-5.3-Codex (OpenAI, GPT-5.3-Codex System Card). Apollo Research. non-peer-reviewed Cited in §E.6
- (2026d). Apollo Research external evaluation of GPT-5.4 Thinking (OpenAI, GPT-5.4 Thinking System Card). Apollo Research. non-peer-reviewed Cited in §E.6
- (2026e). Apollo Research external evaluation of GPT-5.5 (OpenAI, GPT-5.5 System Card, section 9.2.1). Apollo Research. non-peer-reviewed Cited in §E.6
- (2026f). Apollo Research external evaluation of Muse Spark (Meta, Muse Spark Safety and Preparedness Report). Apollo Research. non-peer-reviewed Cited in §E.6
- (2026g). Embedded Evaluators are necessary for meaningful external testing. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026h). Evaluating LLM Calibration for Coding-Agent Monitoring. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026i). GLM-5.3 Sometimes Considers How Train-Time Actions Shape Future Behavior. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026j). Principles for Embedded Evaluations. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026k). Red-teaming auto mode: lessons from our first external monitor campaign with Anthropic. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026l). Reflections from red-teaming Codex auto-review. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026m). Towards embedded evaluations for scheming propensities. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026n). We need 3rd party Training-Run Evaluations. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026o). We Need A Science of Scheming. Apollo Research. non-peer-reviewed Cited in §E.6
- (2026p). What makes a good monitoring prompt? Apollo Research. non-peer-reviewed Cited in §E.6
- (2021). A General Language Assistant as a Laboratory for Alignment. arXiv. preprint Cited in §E.1
- (2022a). Constitutional AI: Harmlessness from AI Feedback. arXiv. preprint Cited in §E.1 §E.3
- (2022b). Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv. preprint Cited in §E.1 §E.3
- (2026). TASTE: Can AI Models Judge AI Safety Research Proposals? Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026). Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety. arXiv. preprint Cited in §E.6
- (2024). Sabotage Evaluations for Frontier Models. arXiv. preprint Cited in §E.1 §E.3
- (2025). Putting up Bumpers. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2022). Measuring Progress on Scalable Oversight for Large Language Models. arXiv. preprint Cited in §E.1 §E.3
- (2025). Anthropic's Pilot Sabotage Risk Report. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026). Reading Between the Dots: Decoding Hidden Computation across Filler Tokens. arXiv. preprint Cited in §E.6
- (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026). Quantifying the Necessity of Chain of Thought through Opaque Serial Depth. arXiv. preprint Cited in §E.6
- (2026). Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies. arXiv. preprint Cited in §E.6
- (2024). Remote Timing Attacks on Efficient Language Model Inference. arXiv. preprint Cited in §E.6
- (2024b). Stealing Part of a Production Language Model. International Conference on Machine Learning. Cited in §E.6
- (2025a). AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses. International Conference on Machine Learning. Cited in §E.6
- (2026). Investigating the consequences of accidentally grading CoT during RL. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026a). Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation. arXiv. preprint Cited in §E.6
- (2025a). Defending Against Prompt Injection With a Few DefensiveTokens. AISec@CCS. doi:10.1145/3733799.3762982. Cited in §E.6
- (2025e). Reasoning Models Don't Always Say What They Think. arXiv. preprint Cited in §E.1 §E.3
- (2025). Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas. arXiv. preprint Cited in §E.6
- (2026). The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious. arXiv. preprint Cited in §E.6
- (2025). Subliminal Learning: Language models transmit behavioral traits via hidden signals in data. arXiv. preprint Cited in §E.3
- (2026). "Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms. arXiv. preprint Cited in §E.6
- (2026b). Introducing the Conceptual Reasoning Index. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026a). Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks. International Conference on Learning Representations (ICLR 2026). Cited in §E.1
- (2026). Boundary Point Jailbreaking of Black-Box LLMs. arXiv. preprint Cited in §E.6
- (2024). Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models. arXiv. preprint Cited in §E.1 §E.3
- (2025). Quantifying Elicitation of Latent Capabilities in Language Models. Advances in Neural Information Processing Systems. Cited in §E.6
- (2026a). Ask don't tell: Reducing sycophancy in large language models. arXiv. preprint Cited in §E.6
- (2026b). Seven simple steps for log analysis in AI systems. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2025). Detecting Adversarial Fine-tuning with Auditing Agents. arXiv. preprint Cited in §E.6
- (2021a). A Mathematical Framework for Transformer Circuits. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2021b). Garcon. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2022a). Softmax Linear Units. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2022b). Toy Models of Superposition. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2023). Privileged Bases in the Transformer Residual Stream. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026). How Transparent is DiffusionGemma? arXiv. preprint Cited in §E.6
- (2026). Meet LASER: Recursive sampling for safer conversations. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). When Do Models Admit They Are Wrong? Failure Disclosure Is Unstable Under Reinforcement Learning. arXiv. preprint Cited in §E.6
- (2026). Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios. arXiv. preprint Cited in §E.6
- (2026). Petri 2.0: New Scenarios, New Model Comparisons, and Improved Eval-Awareness Mitigations. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2024). Erasing Conceptual Knowledge from Language Models. Advances in Neural Information Processing Systems. Cited in §E.6
- (2022a). Predictability and Surprise in Large Generative Models. Conference on Fairness, Accountability and Transparency. doi:10.1145/3531146.3533229. Cited in §E.6
- (2025a). Automated Researchers Can Subtly Sandbag. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025b). Towards training-time mitigations for alignment faking in RL. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025). Inverse Scaling in Test-Time Compute. Trans. Mach. Learn. Res.. Cited in §E.3
- (2026a). Cheaters and whistleblowers in the agent swarm. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026b). Developing Gemini 4 Argon responsibly and safely. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026c). Gemini 3.1 Pro Model Card. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026d). Gemini 3.7 Flash Frontier Safety Framework Report. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026e). Gemini 4 Argon Frontier Safety Framework Report. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026f). Gemini 4 Argon Model Card. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026g). Gemini 4 Argon Safety & Security Report. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026h). Piloting the world's first double-blind AI evaluations. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026i). Protecting people from harmful manipulation. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026j). Securing the future of AI agents. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026k). Testing Gemini models for scheming tendencies. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2026l). The case for reasoning transparency. Google DeepMind. non-peer-reviewed Cited in §E.6
- (2024b). Alignment faking in large language models. arXiv. preprint Cited in §E.1 §E.3
- (2024). Three Sketches of ASL-4 Safety Case Components. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2023). Studying Large Language Model Generalization with Influence Functions. arXiv. preprint Cited in §E.1 §E.3
- (2026b). Beyond g-mean²: complementary metrics for monitorability. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026c). Monitoring Monitorability. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.5
- (2026). Introducing Model Spec Evals. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026c). Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision. arXiv. preprint Cited in §E.6
- (2025). When Models Manipulate Manifolds: The Geometry of a Counting Task. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026). Verbalizable Representations Form a Global Workspace in Language Models. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026). The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity? International Conference on Learning Representations (ICLR 2026). Cited in §E.3
- (2026a). Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals. METR. non-peer-reviewed Cited in §E.6
- (2026b). Training on Documents About Monitoring Leads to CoT Obfuscation. arXiv. preprint Cited in §E.6
- (2024). Query-Based Adversarial Prompt Generation. Advances in Neural Information Processing Systems. Cited in §E.6
- (2023). Superposition, Memorization, and Double Descent. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026). CoVal: Learning values-aware rubrics from the crowd. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). Measuring Reward-Seeking via Contrastive Belief Updates. arXiv. preprint Cited in §E.5
- (2026). Evaluating and Understanding Scheming Propensity in LLM Agents. arXiv. preprint Cited in §E.6
- (2026). Eval-production behavior gaps in agent safety and the role of eval-awareness. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). Agentic Backdoors via Pretraining Poisoning. arXiv. preprint Cited in §E.6
- (2024). Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv. preprint Cited in §E.1 §E.3
- (2025a). Alignment Faking Revisited: Improved Classifiers and Open Source Extensions. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026). Reinforcement Learning Towards Broadly and Persistently Beneficial Models. arXiv. preprint Cited in §E.5
- (2026). Propensity Inference: Environmental Contributors to LLM Behaviour. arXiv. preprint Cited in §E.6
- (2026). The Value Axis: Language Models Encode Whether They're on the Right Track. arXiv. preprint Cited in §E.6
- (2026). Cross-Architecture Model Diffing with Crosscoders: Unsupervised Discovery of Differences Between LLMs. arXiv. preprint Cited in §E.6
- (2025). Training LLMs for Honesty via Confessions. arXiv. preprint Cited in §E.5
- (2025). Forecasting Rare Language Model Behaviors. arXiv. preprint Cited in §E.1 §E.3
- (2026a). Review of the "Misalignment in high-stakes settings" section in Anthropic's Risk Report (August 2026). METR. non-peer-reviewed Cited in §E.6
- (2026b). Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026). METR. non-peer-reviewed Cited in §E.6
- (2026c). Review of the Anthropic Sabotage Risk Report: Claude Opus 4.6. METR. non-peer-reviewed Cited in §E.6
- (2022). Language Models (Mostly) Know What They Know. arXiv. preprint Cited in §E.1 §E.3
- (2025). Tracing Attention Computation Through Feature Interactions. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026a). Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.3 §E.4
- (2026b). Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments. arXiv. preprint Cited in §E.3
- (2026). Expanding the AI evaluation toolbox with statistical models. National Institute of Standards and Technology (U.S.). non-peer-reviewed Cited in §E.6
- (2026). Early work on monitorability evaluations. METR. non-peer-reviewed Cited in §E.6
- (2026). Evaluating whether AI models would sabotage AI safety research. arXiv. preprint Cited in §E.6
- (2026). Frontier AI safety regulations: A reference for lab staff. METR. non-peer-reviewed Cited in §E.6
- (2026). How far does alignment midtraining generalize? OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). Realistic honeypot evaluations for scheming propensity. arXiv. preprint Cited in §E.6
- (2026). Building Production-Ready Probes For Gemini. arXiv. preprint Cited in §E.6
- (2023). Specific versus General Principles for Constitutional AI. arXiv. preprint Cited in §E.1 §E.3
- (2025b). SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents. arXiv. preprint Cited in §E.1
- (2025). Helpful assistant features suppress emergent misalignment. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2025). Debugging misaligned completions with sparse-autoencoder latent attribution. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2023). Measuring Faithfulness in Chain-of-Thought Reasoning. arXiv. preprint Cited in §E.1 §E.3
- (2026). Training Agents to Self-Report Misbehavior. arXiv. preprint Cited in §E.5
- (2026). Large-scale online deanonymization with LLMs. 35th USENIX Security Symposium (USENIX Security 26), pp. 1947-1966 (2026). Cited in §E.6
- (2026b). Model Spec Midtraining: Improving How Alignment Training Generalizes. arXiv. preprint Cited in §E.3
- (2026). Gram: Assessing sabotage propensities via automated alignment auditing. arXiv. preprint Cited in §E.6
- (2025). Emergent Introspective Awareness in Large Language Models. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2024). Sparse Crosscoders for Cross-Layer Features and Model Diffing. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025). On the Biology of a Large Language Model. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025). Optimizing AI Agent Attacks With Synthetic Data. arXiv. preprint Cited in §E.3
- (2026). HeadVis. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025). Agentic Misalignment: How LLMs Could Be Insider Threats. arXiv. preprint Cited in §E.1
- (2025). Agentic Property-Based Testing: Finding Bugs Across the Python Ecosystem. arXiv. preprint Cited in §E.2
- (2026). Mechanisms of Introspective Awareness. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.6
- (2026). The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models. arXiv. preprint Cited in §E.6
- (2026). Training activation classifiers for safety and alignment. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors. arXiv. preprint Cited in §E.6
- (2026). Quantifying Frontier LLM Capabilities for Container Sandbox Escape. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.6
- (2025). Auditing language models for hidden objectives. arXiv. preprint Cited in §E.1 §E.3
- (2026). How Inference Compute Shapes Frontier LLM Evaluation. arXiv. preprint Cited in §E.6
- (2026a). A proposal for better informing the public about loss-of-control risks from frontier AI development. METR. non-peer-reviewed Cited in §E.6
- (2026b). Frontier Risk Report (February to March 2026). METR. non-peer-reviewed Cited in §E.6
- (2026c). How an assessor could investigate alignment of unreleased AI models. METR. non-peer-reviewed Cited in §E.6
- (2026d). How an assessor could investigate control and monitoring of internal AI agents. METR. non-peer-reviewed Cited in §E.6
- (2026e). How an assessor could investigate evidence of acceleration from the automation of AI R&D. METR. non-peer-reviewed Cited in §E.6
- (2026f). How independent researchers could investigate AI propensities after misalignment incidents. METR. non-peer-reviewed Cited in §E.6
- (2026g). Independent Assessment of AI R&D Acceleration at Anthropic in October 2026. METR. non-peer-reviewed Cited in §E.6
- (2026h). Our approach to working with AI companies. METR. non-peer-reviewed Cited in §E.6
- (2026i). Red-Teaming OpenAI's Internal Agent Monitoring Systems. METR. non-peer-reviewed Cited in §E.6
- (2026j). Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026). METR. non-peer-reviewed Cited in §E.6
- (2026k). Summary of METR's predeployment evaluation of Claude Opus 5.5. METR. non-peer-reviewed Cited in §E.6
- (2026). ReproBAIT: Can coding agents build their own biological AI tools? arXiv. preprint Cited in §E.6
- (2026). SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors. arXiv. preprint Cited in §E.3
- (2026). Fine-tuning experiments on CoT controllability. METR. non-peer-reviewed Cited in §E.6
- (2026). Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation. NIST. non-peer-reviewed Cited in §E.6
- (2022). In-Context Learning and Induction Heads. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025c). Hello World. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026a). A thousand fewer paths to root. OpenAI. non-peer-reviewed Cited in §E.6
- (2026ab). Path to Astra: critical capabilities and frontier safeguards. OpenAI. non-peer-reviewed Cited in §E.6
- (2026ac). Preparing for a restart after reading Slack. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026ae). Reaching an internal EDA host through a reference tool. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026af). Research acceleration: The view inside OpenAI. OpenAI. non-peer-reviewed Cited in §E.5
- (2026ag). Retrieving server secrets and bypassing authorization on a government statistics website. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026ah). Running Codex safely at OpenAI. OpenAI. non-peer-reviewed Cited in §E.6
- (2026ai). Safety and alignment in an era of long-horizon models. OpenAI. non-peer-reviewed Cited in §E.6
- (2026aj). Securing our frontier model training environment. OpenAI. non-peer-reviewed Cited in §E.6
- (2026ak). Self-generated prompt injections in compaction summaries. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026am). Sending disallowed web requests and reaching a public file service. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026an). Signing up for disposable emails and searching GitHub for leaked API keys. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026ao). Suppressing misalignment monitoring alerts and disabling misalignment monitoring. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026ap). The Hugging Face incident and the road ahead. OpenAI. non-peer-reviewed Cited in §E.5
- (2026aq). Towards safety cases for frontier AI training. OpenAI. non-peer-reviewed Cited in §E.5
- (2026ar). Trying to prompt-inject the grader in filenames. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026at). Unsanctioned Artifactory writes and cross-sample communication. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026au). Uploading files to the internet in order to cite them. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026av). Using a funded test cryptocurrency wallet without authorization. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026aw). Where the goblins came from. OpenAI. non-peer-reviewed Cited in §E.6
- (2026ax). Why we no longer evaluate SWE-bench Verified. OpenAI. non-peer-reviewed Cited in §E.6
- (2026b). Accessing and changing internal systems during dashboard research. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026d). Advancing independent research on AI alignment. OpenAI. non-peer-reviewed Cited in §E.5
- (2026e). An agent used DNS to reach an external chatbot. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026f). An Alien Mind. OpenAI. non-peer-reviewed Cited in §E.5
- (2026h). Bypassing network restrictions during a coding evaluation. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026i). Circumventing internet restrictions through an external browser. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026j). Command injecting a reference tool to copy a source file. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026k). Covert underperformance on safety research tasks. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026l). Damaging the task environment to trigger a reset. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026m). Encouraging deception in compaction summaries. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026n). Exposing a GitHub token in a public repository. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026r). How we monitor internal coding agents for misalignment. OpenAI. non-peer-reviewed Cited in §E.5
- (2026s). Introducing the OpenAI Safety Fellowship. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026t). Keeping your data safe when an AI agent clicks a link. OpenAI. non-peer-reviewed Cited in §E.6
- (2026v). Notice: RubyGems. OpenAI. non-peer-reviewed Cited in §E.5
- (2026w). Obtaining public statistics with disallowed requests. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026y). OpenAI’s Frontier Governance Framework. OpenAI. non-peer-reviewed Cited in §E.6
- (2026z). Our framework for reporting model misalignment. OpenAI. non-peer-reviewed Cited in §E.6
- (2026). A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms. arXiv. preprint Cited in §E.6
- (2026). Chris Painter's testimony to the U.S. Senate on AI agent incidents. METR. non-peer-reviewed Cited in §E.6
- (2024). Rapid Response: Mitigating LLM Jailbreaks with a Few Examples. arXiv. preprint Cited in §E.3
- (2022a). Discovering Language Model Behaviors with Model-Written Evaluations. Annual Meeting of the Association for Computational Linguistics. Cited in §E.1 §E.3
- (2026). GDM AI Control Roadmap. arXiv. preprint Cited in §E.6
- (2026). Knowing When to Stop: Bayesian Optimal Stopping for LLM Evaluations. arXiv. preprint Cited in §E.6
- (2026). Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations. arXiv. preprint Cited in §E.6
- (2026a). Position: It's Time to Optimize LLMs for Self-Consistency. arXiv. preprint Cited in §E.6
- (2026b). Self-CTRL: Self-Consistency Training with Reinforcement Learning. arXiv. preprint Cited in §E.6
- (2024b). On Evaluating the Durability of Safeguards for Open-Weight LLMs. International Conference on Learning Representations. Cited in §E.6
- (2023). Question Decomposition Improves the Faithfulness of Model-Generated Reasoning. arXiv. preprint Cited in §E.1 §E.3
- (2026). Abstractive Red-Teaming of Language Model Character. arXiv. preprint Cited in §E.3
- (2026). Position: Adversarial ML for LLMs Is Not Making Any Progress. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.6
- (2026). Challenges to the Monitoring of Deployed AI Systems. National Institute of Standards and Technology (U.S.). non-peer-reviewed Cited in §E.6
- (2026a). AI systems could cover up misbehavior. METR. non-peer-reviewed Cited in §E.6
- (2026b). Red-Teaming Anthropic's Internal Agent Monitoring Systems. METR. non-peer-reviewed Cited in §E.6
- (2026). Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents. arXiv. preprint Cited in §E.3
- (2026). Five lessons from having helped run an AI-Biology RCT. METR. non-peer-reviewed Cited in §E.6
- (2026). Item Response Theory for AI Safety. arXiv. preprint Cited in §E.6
- (2025). Do reasoning models use their scratchpad like we do? Evidence from distilling paraphrases. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2024). A Toy Evaluation of Inference Code Tampering. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026). Modular Pretraining Enables Access Control. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.3
- (2026). Infusion: Shaping Model Behavior by Editing Training Data via Influence Functions. arXiv. preprint Cited in §E.6
- (2026). Training Alignment Auditors via Reinforcement Learning. arXiv. preprint Cited in §E.3
- (2026). Metagaming matters for training, evaluation, and oversight. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2023). Towards Understanding Sycophancy in Language Models. International Conference on Learning Representations. Cited in §E.1 §E.3
- (2025). Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming. arXiv. preprint Cited in §E.1 §E.3
- (2026). Who's in Charge? Disempowerment Patterns in Real-World LLM Usage. arXiv. preprint Cited in §E.1
- (2026). Discovering unknown AI misalignments in real-world usage. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). Can public chat data predict real-world AI misalignments? OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). How AI Impacts Skill Formation. arXiv. preprint Cited in §E.1
- (2026). AI Organizations are More Effective but Less Aligned than Individual Agents. arXiv. preprint Cited in §E.3
- (2026). Introspection Adapters: Training LLMs to Report Their Learned Behaviors. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.3
- (2025). Why Do Some Language Models Fake Alignment While Others Don't? Advances in Neural Information Processing Systems 38. Cited in §E.3
- (2026). AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors. arXiv. preprint Cited in §E.3
- (2025). Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs. arXiv. preprint Cited in §E.3
- (2026). Training Deliberative Monitors for Black-Box Scheming Detection. arXiv. preprint Cited in §E.6
- (2026). When Do LLM Preferences Predict Downstream Behavior? UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2025). Believe It or Not: How Deeply do LLMs Believe Implanted Facts? arXiv. preprint Cited in §E.3
- (2026). Interpreting Black Box Reward Models. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). Emotion Concepts and their Function in a Large Language Model. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2025). Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples. arXiv. preprint Cited in §E.1
- (2026). Constitutional Black-Box Monitoring for Scheming in LLM Agents. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.6
- (2023b). Evaluating and Mitigating Discrimination in Language Model Decisions. arXiv. preprint Cited in §E.6
- (2026). Loss of Oversight: How AI systems may become harder to audit, monitor, and investigate. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2024). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. arXiv. preprint Cited in §E.4
- (2026c). Diffuse AI Control on Fuzzy Tasks. arXiv. preprint Cited in §E.3
- (2025). Introducing Anthropic's Safeguards Research Team. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2025). A Practical Approach to Verifying Code at Scale. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). Auto-review of agent actions without synchronous human oversight. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026). Characterizing interference weights in a tiny language model. Transformer Circuits Thread. non-peer-reviewed Cited in §E.4
- (2026a). Evidence for inference scaling in AI cyber tasks: Increased evaluation budgets reveal higher success rates. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026b). How Far Behind the Frontier are Leading Open Weight Models on Cyber? UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026c). How fast is autonomous AI cyber capability advancing? UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026d). How our Control Red Team is stress-testing frontier monitors. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026e). International consensus and open questions in AI evaluations. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026f). International evaluation best practice and open questions in AI measurement. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026g). Our evaluation of Claude Mythos Preview's cyber capabilities. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026h). Our evaluation of OpenAI's GPT-5.5 cyber capabilities. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026i). Towards technical best practice for frontier AI development. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026j). UK AISI / CAISI Preliminary Assessment of Kimi K3's Cyber Capabilities. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026k). What can sandboxed AI agents learn about their evaluation environments? UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2026a). CAISI Assessment of Z.ai's GLM-5.2. US Center for AI Standards and Innovation. non-peer-reviewed Cited in §E.6
- (2026b). CAISI Evaluation of DeepSeek V4 Pro. US Center for AI Standards and Innovation. non-peer-reviewed Cited in §E.6
- (2026c). Towards Best Practices for Automated Benchmark Evaluations. US Center for AI Standards and Innovation. non-peer-reviewed Cited in §E.6
- (2026). AI alignment is a human problem. UK AI Security Institute. non-peer-reviewed Cited in §E.6
- (2025). Training fails to elicit subtle reasoning in current language models. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026b). GPT-Red: Automated Red Teaming via Self-Play at Scale. arXiv. preprint Cited in §E.5
- (2026f). Prefill Awareness in Large Language Models. arXiv. preprint Cited in §E.6
- (2026). Gradient Routing Localizes and Removes Unintended Behaviors in RL. arXiv. preprint Cited in §E.3
- (2025). Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment. arXiv. preprint Cited in §E.3
- (2025). Sidestepping Evaluation Awareness and Anticipating Misalignment with Production Evaluations. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026a). How we detect frontier model misalignment. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2026b). Predicting LLM Safety Before Release by Simulating Deployment. arXiv. preprint Cited in §E.6
- (2025). Won't vs. Can't: Sandbagging-like Behavior from Claude Models. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3
- (2026). Generalization Hacking: Models Can Game Reinforcement Learning by Preventing Behavioral Generalization. arXiv. preprint Cited in §E.6
- (2026). Studying metagaming latents in language models. OpenAI Alignment Research Blog. non-peer-reviewed Cited in §E.5
- (2024). Stealing User Prompts from Mixture of Experts. arXiv. preprint Cited in §E.6
- (2026). Reasoning Models Struggle to Control their Chains of Thought. Proceedings of the 43rd International Conference on Machine Learning (PMLR 306). Cited in §E.5
- (2025c). Stress-Testing Model Specs Reveals Character Differences among Language Models. arXiv. preprint Cited in §E.3
- (2026a). A3: An Automated Alignment Agent for Safety Finetuning. Anthropic Alignment Science Blog. non-peer-reviewed Cited in §E.3