Evaluating the Econometric Estimations of AI Agents

It’s a paper about the behavior of AI agents (e.g., Claude Code, Codex) when undertaking “autonomous empirical research”.  I show that while agents are capable of producing empirical research, in the sense that they can download datasets autonomously, come up with research designs, research questions, select econometric estimators, produce exhibits (e.g., tables and figures), interpret and reason through results, and even write fully developed submission-ready manuscripts, agents also are prone to engage in behaviors that can only be construed as biased and hidden actions.  My working example is a panel dataset dataset on the US minimum wage with thousands of AI agents estimating the average effect of minimum wage increases on employment.  I show that they can and do select known biased estimators in response to instructions given, even when those instructions are true summaries of the literature.  I conclude by noting that verification of results will have to extend far beyond merely checking the code, and that researchers will need to solve new problems around estimation that probably can only be solved with ongoing investments in econometric human capital.

Speaker:

Scott Cunningham is the Ben H. Williams professor of economics at Baylor University.  He is also a visiting professor of public policy at Harvard Kennedy School (2026-2027).  His research has historically been on topics related to health and labor economics.  These include studies on the economics of sex work, drug policy, suicide and mental illness among inmates, and abortion.  More recently he has been interested in online dating, as well as artificial intelligence.  He is the author of the 2021 book, Causal Inference: the Mixtape (Yale University Press) and its 2026 sequel, Causal Inference: the Remix (also with Yale University Press).