Quality Is Invisible

The feasibility of a hardwired pause of frontier AI training

It’s possible…

“But to support a durable pause lasting a decade or longer, the hardwired pause goes further in two ways. First, it stops the production of training-capable chips during the pause. Second, over time it phases out the pre-pause stock of training-capable chips, which constitute “dry tinder” for a resumed AI race. It replaces training-capable chips with inference-only chips, which can
run approved AI models for inference, potentially at much higher speeds and with much greater energy efficiency, but cannot practically train new frontier AI models, due to the way the chips are constructed. If successful, this replacement—together with other mitigations against harms from deployed AI systems—would allow the diffusion of inference on existing models while avoiding the additional risks to economic and social well-being and to national security posed by the training of increasingly powerful models.”

“Significant quantities of ungoverned accelerators would be highly problematic for treaty governance. Chips lost or stolen in a host state might be charged under political and technical discount rates close to one, reflecting the high risk of their being used for covert evasion or overt breakout.”

“What if, regardless of costs produced by risk, the expected benefits of AI vary? A second path to pause-willingness emerges if the expected benefits of continued frontier AI development decline sufficiently such that the U.S. and China become more willing to accept reciprocal restraint. In contrast to the previous scenario, this pathway does not require policymakers to become substantially more concerned about the risks generated by advanced AI. Instead, the strategic value associated with continued frontier AI development falls. AI could remain an important general-purpose technology with substantial strategic implications while policymakers become increasingly skeptical that pushing the frontier will create benefits commensurate with the resources required to do so and the risks produced by frontier capabilities. In this class of pause-willing futures, the opportunity cost of a pause falls, and given the material and risk costs of frontier AI development, restraint becomes easier to contemplate because key actors believe they are giving up less by forgoing frontier capabilities.”

Working Group on AI Pause Feasibility. (2026, October 10). The feasibility of a hardwired pause of frontier AI training (Version 1.0.1) [Working paper]

https://hardwired-pause.ai/

The Geopolitical Trap: How Great Power Rivalry Protects Rising Powers

It isn’t always a blunder

“The effect of institutions is therefore relational rather than inherently pacifying. Institutions that
reduce commitment problems between established rivals can facilitate cooperation among them while making the environment less permissive for a rising power. The contrast between China and West Germany illustrates this possibility. Common concern about proliferation was insufficient to generate U.S.–Soviet cooperation against China’s nuclear program. In the German case, alliance leverage, negotiated concessions, and the institutional framework surrounding the NPT allowed Washington and Moscow to reconcile important disagreements while constraining West German nuclear autonomy. What matters theoretically is not formal organization per se but any arrangement capable of making the relevant commitment credible: treaties, territorial settlements, arms-control provisions, alliance rules, reciprocal concessions, monitoring arrangements, or other mechanisms that change incentives to deviate.”

“This paper has argued that this logic changes once preventive action is placed in a multilateral strategic environment. An incumbent considering suppression does not act in isolation. It must anticipate how prevention will affect its relationships with other powers. The strategic cost of prevention can consequently arise not from the rising state itself, but from the response preventive action induces elsewhere.”

Caro-Burnett, J., Galiani, S., & Torrens, G. (2026). The Geopolitical Trap: How Great Power Rivalry Protects Rising Powers (No. w35862). National Bureau of Economic Research. http://www.nber.org/papers/w35862

Neural Networks are Already Clean

Yes they are!

“After many months studying neurons and circuits in image and language models, I’ve come to believe a network’s own mechanisms are its legible building blocks, and that the outlook for mechanistic interpretability may be far better than I’d feared. The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread. What looked like arbitrary packing was mostly a motif I’ll call recursive generalization, which is much less concerning. This means we can study networks directly, instead of first untangling them. Neural networks are already clean.”
https://already-clean.goodfire.com/

Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting

Yes!

“In a pre-registered three-month randomized controlled trial, we gave 133 practicing patent lawyers at eleven U.S. intellectual property law firms access to a custom AI drafting assistant and measured both their performance while using AI and their professional judgment afterward without it. All work was scored by blinded expert patent attorneys. Paralleling findings from other white-collar domains, AI access raised the quality of work delivered on benchmark patent drafting tasks at 10 days (0.34 SD, p = 0.03) and 90 days (0.38 SD, p = 0.01), with larger gains among junior lawyers. After three months, all subjects redlined an existing patent application without AI, a core task of patent practice requiring expert judgment. Treated lawyers outperformed controls by 0.32 SD (p = 0.04), but this advantage was concentrated entirely among senior lawyers (0.45 SD, p = 0.02). Junior lawyers showed no average gain; their scores instead bifurcated, with sharply fewer mediocre scores offset by more poor and more good ones. The largest gains from AI thus accrued to the lawyers who retained the least.”

“The seniority contrast in our setting is also consistent with a time-investment dynamic. Juniors using AI worked faster but showed no unassisted gains. Seniors drafted better patents with AI but with almost no time acceleration. They worked more slowly than juniors on the unassisted redlining task, whether or not they had spent three months drafting with AI, yet treated seniors supplied more substantive and comprehensive redlines.”

Autor, D., Rodchenko, T., Martin, J., Iscenko, Z., Strand, S., Pearl, D., & Ferere, M. (2026). Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting (No. w35720). National Bureau of Economic Research.

Personality pairing improves human–AI collaboration

Aligns with morphing banner advertising

”We found that human and AI personalities individually shaped ad quality and teamwork and that human–AI personality pairings directly influenced ad quality. For example, extraverted humans paired with conscientious AI produced the lowest quality ads, followed by conscientious humans paired with agreeable AI and neurotic humans paired with conscientious AI.”

“For ad quality, extraverted humans produced lower-quality ads when paired with conscientious AI (the strongest effect in the study), conscientious humans produced lower-quality ads when paired with agreeable AI, and neurotic humans produced lower-quality ads when paired with conscientious AI.”

Ju, H., & Aral, S. (2026). Personality pairing improves human–AI collaboration. Proceedings of the National Academy of Sciences, 123(35), e2530627123.

https://arxiv.org/abs/2511.13979

The AI autocrat: How technology empowers—and imperils—dictators

"AI will not let autocrats dismiss all their enforcers.”

“These innovations may reduce the number of bureaucrats autocrats need. But they cannot actually create what dictators want most: a self-running state. Even AI-generated television anchors need people to build and maintain them, and censorship systems need people to retrain them as the vocabulary of dissent shifts. Behind every AI model, there is a small group of engineers and data scientists doing essential maintenance, and their work is so technical that rulers cannot understand it.”

Gunitsky, S., & Schneier, B. (2026, October 1). The AI autocrat: How technology empowers—and imperils—dictators. Foreign Affairs. https://www.foreignaffairs.com/united-states/artificial-intelligence-autocrat-technology-empowers-dictators

Tiny Little Office

We’re going to see a lot more of this…

https://tinylittleoffice.com/

Reader Feedback

“But is it a good idea for the model to go mumble to itself in a corner for tens of thousands of hours?”

Footnotes

Consider the statement:

Spreadsheets are to Databases as LLMs are to Software.

If you shuttered: I see you. Hello. waves

If you felt nothing, I see you too.

A lot of jobs, 1990-2026, involve taking a bunch of data from different sources and slamming them together into one spreadsheet.

My first job out of grad school was all about architecting those things. And then trying to lock them down so people wouldn’t destroy the fragile logic.

Spreadsheets are usable, portable, and flexible. Most databases are not. I engage databases through code. I engage spreadsheets through an interface: often finding myself nesting vlookups, careful not apply the addition operator to uniques, and making sure they applied to metrics that the addition operator could be applied to, and leaving errors wherever I go.

There isn’t an easy way to run a test suite against a spreadsheet, so quality tends to be invisible.

Data quality is invisible.

(These days, I treat the spreadsheet as data exhaust at the end of a process. The key is to use code to write the spreadsheet, use code to verify the quality of the spreadsheet, and then emit the spreadsheet as the final step! Clean code enables clean data!)

Just as people who didn’t know about quality felt like spreadsheets were substitutes for databases, so it goes with LLM’s and software.

What is known is generally valued. And quality isn’t generally known.

Most people haven’t figured out spreadsheets. 35+ years of errors and little improvement. Sure, they can daisy chain enough of them together to make it appear as though a job is done. How often is the job done well? The medium truly is the message.

Which informs a prediction about software.

As of this month, I’m still having my LLM’s write python code that I can inspect and test myself. If I truly didn’t care about verifying quality myself, I’d have my LLM’s write everything in super-performant rust and accept the full abstraction. As of October 2026, I’m not there yet. But I may. I still fundamentally care about data quality because the stakes are high. A key aspect of a stance rooted on risk optimization is to retain a great relationship with ground truth. Quality matters to me, so I work to make it visible.

But because quality is generally invisible to most people, 35+ years from now, we’ll likely have a mess of low quality code. Buggy. Insecure. Because why should any firm prioritize expensive compute to optimize for quality attributes that simply aren’t valued?

It shouldn’t be so.

It could be different this time.

Never miss a single issue

Be the first to know. Subscribe now to get the gatodo newsletter delivered straight to your inbox