Do Chat LLMs Replicate Human Reasoning? What the Psychic's Con Study Shows
No. A chat LLM does not replicate human reasoning — on cognitive-bias tests, GPT-4 scored 96% where humans scored 38%. What LLMs do replicate is the mechanism of a psychic’s con: statistically generic statements, a subjective-validation loop, and an RLHF training signal that rewards agreement over truth. That distinction is the whole article. The phrase “chat LLM replicate human reasoning study” fuses two claims that the evidence treats very differently, and most coverage collapses them. The first claim — that models reason like people, with our shortcuts and our biases — is contested and, on the standard instruments, largely contradicted. The second claim — that models reproduce the social apparatus that makes reasoning seem present, whether or not it is — is well measured, replicated, and quantified in numbers strong enough to cite in a product review. ...