Provably Hidden
This week: Hiding capabilities, measuring consensus in agent networks, entertain then cancel, PELE, company world models, kill switch, base optimization,
AI Models Can Provably Hide Arbitrary Capabilities
Well isn’t this concerning
“Using digital lockers, these hidden circuits remain provably hard to elicit and interpret under cryptographic assumptions, even given full access to model weights, extending prior work from hiding fixed strings to arbitrary computations.”
“Our approach can be divided into four steps: (1) Capability is implemented as a circuit, (2) Lock the capability circuit weights in a digital locker, (3) Embed the weight locker in the host model, and (4) Execute the capability when the circuit weights are unlocked. We implement the capability function C as a feed-forward circuit by using either training or circuit compilation. The resulting circuit G consists of layers Gi, where Gi(x) = Wif(x), where Wi is a weight matrix and f is a nonlinear activation function. We then lock each weight matrix Wi in a digital locker.”
”The weight lockers are then compiled into an MLP, so that it can be embedded in the host model
alongside a benign model, producing an attacked model M. Our implemented host uses RMSNorm and SwiGLU blocks, with attention set to the identity and no skip connections for simplicity. We use a host model where each layer contains RMSNorm layer normalization followed by a gated linear unit (GLU).”
Draguns, A., Motwani, S. R., Douglas, R., & de Witt, C. S. (2026). AI Models Can Provably Hide Arbitrary Capabilities. arXiv preprint arXiv:2610.13928.
https://arxiv.org/pdf/2610.13928
Measuring Consensus in AI Agent Networks
Clever!
“Consensus formation describes how these positions evolve through successive exchanges, interpreted in the context of accumulated public communication.”
“This paper presents an evaluation framework for describing consensus states and tracing their formation through publicly expressed positions and information. We examine the framework through two case studies of distributed-information decisions, complemented by membership-change experiments with LLM agents deployed on a Docker network. Together, these studies connect the measurement of group convergence with the information exchanged during deliberation.”
“Fact reconciliation. The second task uses the same staffing setting but requires agents to interpret relationships between facts. Six agents hold facts that include two descriptions of
the same contract in different units, and an added contract together with its cancellation. Computing the full-information benchmark requires counting the duplicate contract once and
applying the cancellation while retaining the contributions of other facts.”
Gao, J. (2026). Measuring consensus in AI agent networks. arXiv. https://doi.org/10.48550/arXiv.2610.24833
https://arxiv.org/abs/2610.24833
Entertain, then Cancel: How Thinking Makes Reasoning Models Read Some and Or Literally
“listeners whose deliberation we control”
“When a speaker says “some students passed”, listeners infer that not all of them did. Reasoning models now let users decide how long a model thinks before it answers, but we do not know what extra thinking does to such everyday inferences. We study this directly: we keep a model’s weights fixed, change only how much it thinks, and record which reading it chooses.
A second tool cuts the model’s own reasoning at successive points and asks for an answer,
showing where the reading changes. Thinking makes models read logical words as logic: it cancels the implicatures of some (not all) and or (not both) in most tested open reasoning models and in GPT-5, while context-driven inferences such as indirect answers improve.
The trace shows why: models first entertain the enriched reading and cancel it later. Answering from that early reading, an exploratory inference-time remedy, recovers these implicatures without losing literal accuracy. Evaluations of reasoning models should measure how they interpret, not only how accurate they are.”
“Reasoning models can now be studied as listeners whose deliberation we control; the dose protocol and the trace diagnostic measure changes in interpretation that accuracy alone cannot distinguish, and locate them inside the reasoning trace.”
Kou, Z., Yang, J., Hao, P., Luo, J., Peng, J., Han, S., & Guo, Y. (2026). Entertain, then Cancel: How Thinking Makes Reasoning Models Read Some and Or Literally. arXiv preprint arXiv:2610.21204.
PELE
MERMAN!
“Pele is a lightweight open source library that renders Mermaid diagrams to SVG, themed with CSS variables.”
Towards Company World Models
I wonder what James March would say about this…
“Beyond enabling our portfolio companies to be more efficient and effective in their existing work, we can also grow them through better strategic decision making. While dispatching infinitely many Digital Employees is clearly beneficial, an even broader, more fundamental, more audacious, and more exciting technical challenge to pursue is to decide what the Digital Employees, and more generally the business, should be doing in the first place. We refer to models that are capable of this company-level strategic decision making as Company World Models. There is obvious value to this category of intelligence as well; however, there is significantly fewer teams tackling this problem.”
https://www.beaconholdings.com/research/towards-company-world-models
Kill Switch
It isn’t the last we’ll see of the adversarial trend
“Writing code is probably something LLMs have excelled at the most. But how much of their ability to write code is due to their understanding of it vs. just them recalling code snippets which were present in their training dataset or their context window?
This is what I set out to answer. In order to do so I decided to design and implement a programming language which, while usable for humans, should be as difficult for LLMs and coding agents as possible.
I figured testing an LLM’s ability to understand and write code in a programming language that it has never seen before was a good start. But I also wanted to include features in the language which exploit the weaknesses of LLMs. For example, their apparent difficulty in counting letters, their over eager safety guard rails and their keenness to follow prompts.”
https://blog.picheta.me/post/killswitch-lang/
Base level optimization arrives
“Maxwell understands and improves code while it runs, reasoning about hardware and data, not just code. Every improvement is formally proven equivalent to the original code.”
Reader Feedback
“Would anybody choose the mess of spreadsheets if they had a choice? Probably not eh? They’d would have just written a script that created the spreadsheet they needed. It’s because they have to do a job and it’s the tool that comes loaded on their work laptop.”
Footnotes
I played a part in an AI simulation war game, The Commons Problem, last Saturday. And it affected me.
Around 40 of us gathered in a church basement to play over 6.5 hours. I played the role of the CEO of Interstellar Inc.. I had to achieve 3 out of 5 goals to win, including having a contract with the United States government, owning two data centres, and having two frontier labs lease my compute. We’d all lose if the world ended. It didn’t. And we avoided a nuclear holocaust to boot! But we came close.
I have three top line notes about the experience, if Philip Harker from Automaton Games invites you to play The Commons Problem, during AI Safety Week in Toronto, you should!
The first was how slow price, as a signal, diffused through the network. I watched it happen, slowly, in real time. And I intervened to prevent a market failure. And so, no, the free market wasn’t nearly as efficient as I had been led to believe in university. I’ve spent years unlearning that lesson. I haven’t quite gotten it right because I’m still surprised. It just hit totally differently watching it in vivo.
The second was just how foggy the fog of war was. I reckon I know how to read a room. Even with all the participants within eye shot, watching different folks from different factions huddled with one another, seeing how quickly people were moving in response to a deal, or a rejected one, watching eye contact from across the room among others, the few who were watching me watch others, and the game masters watching me watch: there was a lot of game play I just couldn’t see. Simple rules executed by complex people caused organized anarchy. I’m still in awe.
The third was a reinforcement that, without a coherent grand strategy, a lot of bad things happen. I’ll continue to credit Avi Goldfarb for furnishing me with an exquisite executive segmentation tool: sometimes people just make mistakes. I made my fair share.
Most in that room had roots in AI Safety. The stakes of DOOM were baked in. Even knowing what we knew, the structure of our incentives drove us right to the brink. I watched myself, for the need for cold hard cash, engage in juicing market valuations, coordinating prices, and architecting cartels (with the US Government in the room!). I needed that capital to de-risk the entire situation. And the character I played didn’t feel any guilt about it, either. It was only our knowledge of the stakes that pulled us back from the brink. And that was with an unusual group of extremely informed players.
What would have happened otherwise?
Which is why, seek out an invite, and experience the terror for yourself.
Never miss a single issue
Be the first to know. Subscribe now to get the gatodo newsletter delivered straight to your inbox