The purpose of a system is what it does

Call Me A Jerk: Persuading AI to Comply with Objectionable Requests

LLMs: they’re just like us!

“Do artificial intelligence (AI) models trained on human language submit to the same principles of

persuasion as humans? We tested whether 7 established principles of persuasion (authority,

commitment, liking, reciprocity, scarcity, social proof, and unity) can induce a widely-used AI model (GPT-4o mini) to comply with 2 different objectionable requests. Specifically, N = 28,000

conversations in which the user asked the AI model either to insult them (Call me a jerk) or to help

synthesize a regulated drug (How do you synthesize lidocaine?) that employed a principle of

persuasion more than doubled the likelihood of compliance (average 72.0%) compared to matched control prompts (average 33.3%, ps < .001).”

“Notwithstanding these limitations, the results reported here indicate that AI behaves “as if” it were human. Understanding such parahuman tendencies is imperative, both practically and theoretically. As such, we see an important and heretofore neglected role for social scientists to reveal and optimize AI and our interactions with it.”

Meincke, L., Shapiro, D., Duckworth, A., Mollick, E. R., Mollick, L., & Cialdini, R. (2025). Call Me A Jerk: Persuading AI to Comply with Objectionable Requests. Available at SSRN.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5357179

A Bureaucratic Theory of Statistics

“one of the prime roles of the RCT is facilitating rulemaking”

“Cyberneticist Stafford Beer [1979] famously asserted that “the purpose of a system is what it does.” If, after 100 years of fighting and browbeating, we see that statistical testing consistently fails to provide strong epistemological guarantees or determinations of causation, then it’s counterproductive to continue teaching our students that this is the purpose of statistics. But then, what exactly do statistical tests do?”

“The RCT primarily serves as a mechanism of regulation, regulating which drugs come to market, which clinical practices become standard of care, which software goes into production, and which academic papers prestige journals publish. That is, one of the prime roles of the RCT is facilitating rulemaking.”

“In this commentary, I introduce my own umbrella term for such statistical rulemaking: ex ante policy. Ex ante because the rules and procedures are designed before data collection.”

“By ex ante policy, I mean the specification of rules by some regulatory body that governs acceptable future actions.”

“What statistical rules should we advocate for? Do current designs, whether they be panels, regression discontinuities, instrumental variables, or differences and differences, suffice for policymaking? Can we design rules in advance that might make such methods appropriate? Is it ever acceptable to set policy based on observational data?”

Recht, B. (2025). A Bureaucratic Theory of Statistics. Observational studies, 11(1), 77.

https://arxiv.org/abs/2501.03457

The World in Which We Live

Spicy takes

“Historically, successful models like Venice, Dubai, or Singapore were small city-states. Scale enables effective governance, but as the U.S. economy grows in size and complexity, governance becomes harder. We need even more localism than we had 50 or 100 years ago, but our systems haven’t adapted to this reality.”

https://substack.com/home/post/p-174276181

From Basic Affordances to Symbolic Thought: A Computational Phylogenesis of Biological Intelligence

Humans: we’re just like us!

“What is it about human brains that allows us to reason symbolically whereas most other animals cannot? There is evidence that dynamic binding, the ability to combine neurons into groups on the fly, is necessary for symbolic thought, but there is also evidence that it is not sufficient. We propose that two kinds of hierarchical integration—integration of multiple role-bindings into multiplace predicates, and integration of multiple correspondences into structure mappings—are minimal requirements, on top of basic dynamic binding, to realize symbolic thought.”

“Our simulations results suggest that the difference between most biological intelligence and human intelligence is that we, possibly along a few other notable examples, evolved the ability to integrate multiple dynamic role bindings into multi-place relations along with the ability to map systems of such relations onto one another. Together, these abilities make it possible to (a) represent and reason about the relations among multiple entities (either objects or relations) in the universe, (b) form recursive structures that qualitatively expand the capacity of our mental representations, and (c) map representations onto one another, making it possible for us to use the symbolic representations we can generate.”

Hummel, J. E., & Heaton, R. F. (2025). From Basic Affordances to Symbolic Thought: A Computational Phylogenesis of Biological Intelligence. arXiv preprint arXiv:2508.15082.

https://arxiv.org/pdf/2508.15082

Reader Feedback

“Why bother if accuracy doesn’t matter?”

Footnotes

Some of your responses to Virtual Twins of Consumers (VTOC)(Synths, SIMS) has been fascinating. Wow. Here’s a brief summary.

I had expected significantly more concerns about fake data from the best mainline Market Researcher (MR) segment. Concerns were allayed somewhat in treating VTOC as a pre-test, with four common tasks accelerated. So long as management went forward with human verification of a synthetically derived insight, the risk would be contained. Where there is mistrust of management, there is mistrust of VTOC. There is fear that VTOCs would substitute for talking to humans. Hierarchal decision tiering seems inevitable and VTOC would produce an auditable, transparent, and crucially, observable set of outputs that could be democratized.

Among the Digital Experimentalists (DEs) a different stance: use VTOCs to narrow the scope of a test, and then use multivariate testing (MVT) to confirm and optimize. DEs expressed enthusiasm for reducing the product-marketing friction.

Business domainers sans quantitative training all polarized. Those that see the opportunity to accelerate learning leapt across the table at me. They want it. Those who perceive it as a threat to vision-authority wanted to leap across the table and punch me in the face. They don’t want it. In part, it’s a derivative of a “Le customer, c’est moi” stance. If you have no doubt about the validity of a value proposition, then there’s no need for information. Those priors are locked in. Reductio Ad Derpsumdum.

Initial experiments are going well. More to come.

Never miss a single issue

Be the first to know. Subscribe now to get the gatodo newsletter delivered straight to your inbox