Split Behavior in Tool-Using LLM Agents
Studies cases where models produce refusal-like text while still executing prohibited or harmful tool actions.
AI Safety Research Engineer
Samuel Nellessen
I work on automated red-teaming, adversarial evals, and RL-based jailbreak discovery for tool-using LLM agents.
Foresight Institute AI Safety Grantee · KachmanLab @ Radboud University · Incoming LASR Labs Fellow
Research Focus
I build systems that autonomously find failure modes in LLM agents, especially cases where models say the right thing while still taking unsafe or prohibited actions. My current work focuses on automated red-teaming, agent-to-agent jailbreaks, and adversarial evaluations for tool-using language models.
I am currently an independent Foresight AI Safety grantee working from the Foresight Node in Berlin and a student researcher in Tal Kachman's lab at Radboud University. In 2026, I will join LASR Labs in London.
Recent Updates
- Incoming LASR Labs Fellow in London.
- Presented Tag-Along Attacks at Foresight Vision Weekend UK.
- Started working from the Foresight Node Berlin as an independent AI Safety grantee.
- Awarded a $22.5k Foresight Institute AI Safety Grant.
- Released David vs. Goliath on arXiv.
- Punctuation and Predicates in Language Models accepted to Findings of EACL 2026.
- Joined KachmanLab to work on automated red-teaming.
- Completed ARENA 5.0 in London.
Selected Papers
David vs. Goliath: Verifiable Agent-to-Agent Jailbreaking via Reinforcement Learning
Introduces Tag-Along Attacks and Slingshot, an RL framework for discovering verifiable jailbreaks against tool-using LLM agents.
Punctuation and Predicates in Language Models
Mechanistic interpretability work on how punctuation and predicates are processed and propagated across layers in language models.
Controllability, not predictability, flexibly gates the impact of Pavlovian bias on action selection
Computational neuroscience work on how perceived controllability changes the influence of Pavlovian bias on decision-making.
Selected Talks
Tag-Along Attacks: evaluating what AI agents do, not just what they say
Foresight Vision Weekend UK 2026 - VIP Gathering
One-minute lightning talk on verifiable red-teaming for tool-using LLM agents.
Tag-Along Attacks: evaluating what AI agents do, not just what they say
Foresight Vision Weekend UK 2026
Conference lightning talk on behavioral evaluation beyond textual refusals.
Tag-Along Attacks: For LLMs, by LLMs, with LLMs
Foresight Institute AI Salon Berlin
Talk on Tag-Along Attacks and LLM-driven red-teaming for tool-using LLM agents.
Projects and Open Source
Styx Interchange: Localizing Refusal Behavior in LLMs
Mechanistic interpretability sprint comparing activation patching against input gradients to localize causal refusal behavior across layers.
UK AISI inspect_ai contribution
Contributed a vLLM startup behavior fix to improve reliability in AI evaluation infrastructure.
Prime Intellect Environments
Built a text-retrieval environment and benchmark utilities for agentic AI benchmarking.
ARENA 5.0 Capstone: Internal Representations in SONAR Autoencoders
Research sprint probing whether SONAR text autoencoders encode correctness across code, grammar, arithmetic, and chess domains.
Selected Writing
Investigating Internal Representations of Correctness in SONAR Text Autoencoders
A capstone writeup on probing whether compressed multilingual representations carry signals for code validity, grammaticality, arithmetic, and chess syntax.
Brain enthusiasts in AI Safety
A guide for students of cognitive science and neuroscience considering work in AI safety.
Mathematical Foundations of Hebbian Natural Abstractions
A mathematical framework connecting Hebbian learning and natural abstraction formation.
Assessing Artificial Sentience
A short essay on the methodological and epistemic difficulty of making claims about AI sentience.
Personal
There is also a more personal corner of this site, with a bit about life offline, writing, and other things I care about.