Scientific production in the era of large language models: Outcome-triggered treatment timing and spurious event-study dynamics

Summary

Econometric demolition of the identification behind the Science productivity result: defining treatment as the first paper DETECTED as LLM-assisted mechanically produces positive event-study dynamics even with zero causal effect, because high-output months are more likely to yield a detected paper. Reconstructed arXiv data show random assignments, neutral-keyword triggers, inverted treatment and pre-ChatGPT placebos all reproduce the same pattern, as do zero-effect simulations. Essential for any IS/econ author who wants to cite AI-productivity evidence responsibly.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-08-29 literature sweep as a high-priority candidate for the RISE knowledge base (angle: research productivity of AI). Verified against: Semantic Scholar publisher record with full abstract opened 2026-08-29 (3 authors, PNAS vol. 123, published 2026-08-11); PNAS site returns HTTP 403

Critique / open questions

Queued — not yet assessed.

Key quotes

Large language models (LLMs) are increasingly used in scientific writing, but their effect on individual productivity is difficult to identify because adoption is rarely directly observed. [K. (abstract)

Kusumegi et al. , Science 390 , 1240–1243 (2025)] infer adoption from the first paper detected as LLM-assisted and report large productivity gains after adoption. (abstract)