Humans Aren’t Models
This Week: Cognition spaces, LLMs as human surrogates, pattern matching, mechanism design, markups, agentic coding
Cognition spaces: natural, artificial, and hybrid
“it is unclear where the organism ends, and its cognitive–technological scaffold begins.”
“By focusing on the structure of cognition spaces rather than on categorical definitions, this approach clarifies the diversity of existing cognitive systems and highlights hybrid cognition as a promising frontier for exploring novel forms of complexity beyond those produced by biological evolution.”
“Hybrids may offer our best opportunity to expand cognitive space into currently unoccupied domains while allowing us to probe the limits of what is possible.”
“At the same time, whenever evolutionary or learning processes are allowed to operate, unexpected forms of complexity tend to emerge, including in artificial settings.”
Solé et al. (2026) Cognition spaces: natural, artificial, and hybrid
https://arxiv.org/abs/2601.12837
Take caution in using LLMs as human surrogates
“LLMs’ self-explanations provide insight into their behaviors, but these explanations may not always align with the model’s actual actions.”
“We assess the reasoning depth of LLMs using the 11-20 money request game. Nearly all advanced approaches fail to replicate human behavior distributions across many models. Causes of failure are diverse and unpredictable, relating to input language, roles, and safeguarding.”
“LLM performance is sensitive to prompting formats and how instructions are provided. Designing effective prompts that boost LLMs’ performance is fraught with challenges. Prompt brittleness refers to the idea that significant variations in LLM responses can manifest with minor changes in prompt wording (Hutson, 2024). Taking multiple-choice questions as an example, the number of spaces and control characters in the prompt (Sclar et al., 2023), as well as the order in which options are presented (Turpin et al., 2024; Ceron et al., 2024; Gupta et al., 2024), can greatly affect LLM performance. This variability undermines the model’s reliability, especially in experimental settings where tasks are framed differently.”
Gao, Y., Lee, D., Burtch, G., & Fazelpour, S. (2025). Take caution in using LLMs as human surrogates. Proceedings of the National Academy of Sciences, 122(24), e2501660122.
https://arxiv.org/abs/2410.19599
The unreasonable effectiveness of pattern matching
To paraphrase an old saying, it is patterns all the way down
“Pattern-matching is not an alternative to “real” intelligence, but rather a key ingredient.”
“We report on an astonishing ability of large language models (LLMs) to make sense of “Jabberwocky” language in which most or all content words have been randomly replaced by nonsense strings, e.g., translating “He dwushed a ghanc zawk” to “He dragged a spare chair”.”
“Despite lacking the words that make Jabberwocky Jabberwocky, the retained structure is sufficient to uniquely fingerprint the text, allowing LLMs to match it to the original poem. We invite readers to see for themselves.”
“The ability of LLMs to make sense of Jabberwockified English may seem like there is an alien intelligence at work. But although this feat appears to be beyond what people can do, there are good reasons to think that the process behind it is very close to home.”
“Putting aside the quibble that our minds can and often do get tangled in contradictions, the real miracle may be how far pattern matching can take us.”
Lupyan, Gary, Agüera y Arcas (2026) The unreasonable effectiveness of pattern matching
https://arxiv.org/abs/2601.11432
Mechanism Design Beyond Expected Utility Preferences
“This implies that private information held by such agents delivers no information rent and full surplus extraction is feasible under the assumptions of the paper.”
“Moreover, dynamic stochastic mechanisms can fully extract the private information of a type with strictly quasi-convex preferences at no cost. That is, the designer can exploit the time-inconsistency inherent in non-EU preferences to completely circumvent the incentive constraints of any non-EU type. Full-surplus extraction is possible in a broad variety of non-EU environments.”
“Our paper develops a general mechanism design framework allowing for any finite set of types with continuous preferences and any finite set of outcomes.1 Our first result provides a complete characterization of the environments in which the revelation principle holds.2 We show that the principle is not merely a useful tool for settings with EU preferences but it is intrinsically linked to the independence axiom defining such preferences. Specifically, Theorem 1 states that the revelation principle holds if and only if the preferences of all types can be described by expected utility maximization. So, whenever a single type has non-EU preferences, the set of allocations implementable through general dynamic mechanisms becomes strictly larger than what can be achieved with static direct mechanisms.”
Mora, E. R., & Strack, P. (2025). Mechanism Design Beyond Expected Utility Preferences.
Concentration and markups in international trade
“First, it reveals that markups increase with exporter concentration and decrease with importer concentration, reflecting the balance of oligopoly and oligopsony forces. Second, it adapts conventional market definitions to reflect rigid trading relationships, yielding new concentration measures that capture competition in firm-to-firm trade.”
“Our primary data source is the universe of Colombian customs records, covering all import transactions from 2011 to 2020.”
“Markups rise with exporter concentration, reflecting oligopoly power, and fall with importer concentration, reflecting countervailing oligopsony power. The strength of each force depends on supply and demand elasticities, while their net effect is shaped by relative bargaining power.”
“…caution is warranted when interpreting the observed trends, especially when the forces driving changes in concentration over time are not well understood.”
Alviarez, V. I., Fioretti, M., Kikkawa, K., & Morlacco, M. (2025). Concentration and markups in international trade (No. w34114). National Bureau of Economic Research.
https://www.nber.org/papers/w34114
The Levels of Agentic Coding
“It’s systems all the way down”
“Viable systems are recursive. Once you start seeing patterns that work with coding agents, there may be an analog pattern that works with teams. Or if your company does something really cool, maybe there’s a way to elicit the same effect in a coding agent.”
https://timkellogg.me/blog/2026/01/20/agentic-coding-vsm
Reader Feedback
“Had to re-read [Rindova and Martins]. So it’s almost like a spectrum? You start with the features of the problem itself, and that leads to whether you get a kind of incremental improvement, a new industry, or at that most extreme, a new future. What I heard you [Christopher] call imitators before, airbnb for dogs or drones for tacos, against say, real innovation, is how they think of meshes of problems that enable new futures, maybe like….planes in the first place? It’s just the frame of the problem. If it isn’t a real problem, does it matter?”
Footnotes
The feedback has been fantastic. Honest. Raw. Clear.
I saw a lot of unexpected behaviour.
And it raises UX choices.
A great friend and fellow founder reminded me to start with the outcome and work backwards.
The point of the UX is to find demand.
The UX, as it stands today, enables anybody to generate salient data about demand and the problems that cause it. Within the tool itself, generation of the data is well abstracted and the barrier sufficiently lowered for right now. On the other hand, the extraction of insight about demand, while enabled, is unguided.
So that barrier, the potential gap, between a clean dataset and a novel insight can be wide.
The best manual on causal research methods is a 99 page monster that summarizes a much longer 573 page leviathan. Communities differ when it comes to the content and awareness of those 99 pages. My stance is rooted on pragmatic Tukey-ism, in that statistical tools and exploratory data analysis are there to support the judgement of the data analyst, not to substitute it. And yet, there’s a barrier imposed by skill and ambiguity.
My instinct is to narrow it.
In order to guide them through the dataset to demand, I have to apply an editorial stance. Subtraction implies choice. Most of these choices conflict with a pure Tukey stance. That's the conflict.
This should be fun.
Never miss a single issue
Be the first to know. Subscribe now to get the gatodo newsletter delivered straight to your inbox