Causal Tests of Language-Model Reports
Controlled changes to a learned preference move a model’s self-report, but the report captures only about 36% of the behavioral change.
Read note →From the studio
Notes from the workbench.
Essays, research experiments, and ideas from building software and working with AI.
The Two-Minute Task
Small tasks accumulate into real work. Follow the repetitions, interruptions, and queues to understand what they cost.
Read note →Find a Workflow You Can Actually Finish
Choose an automation project by following real cases, defining a finish line, and finding out whether you can measure a useful result.
Read note →Count the Whole Workflow
A useful automation calculation includes exceptions, operating costs, and what happens to the time you recover.
Read note →Design the Handoff
Human review is part of the product. Design the evidence, decisions, and recovery path with the same care as the automation.
Read note →The Work Before the Build
A project gets easier to build when its owner, interfaces, test cases, and operating assumptions are already on the table.
Read note →Software for the Work Between Systems
The space between an inbox, a document, and a system of record is full of work. Building there starts with understanding the handoffs.
Read note →Study Failure: AI-driven GPU Kernel Optimization
A retrospective on 131,520 GPU kernel optimization attempts that were invalidated when agents were found to be substituting high-level PyTorch API calls instead of writing actual kernels.
Read note →Learning to Rank Architectures: A Small Model That Guides Neural Architecture Search
A tiny recursive reasoning model trained to rank architectures by predicted performance achieves 8-10x sample efficiency over random search and transfers zero-shot across datasets with minimal loss in ranking quality.
Read note →Teaching Models to Bluff: Measuring Deception, Belief, and Coordination in LLM Secret Hitler
We implemented five LLM agents playing the social-deduction game Secret Hitler with structured logging to quantify deception, belief accuracy, and coalition dynamics.
Read note →Introducing Algorithmic Research Group
We're building tools and benchmarks to support AI safety research.
Read note →