Agentic AI and Managers' Analytics Capabilities: An Exploration

Summary

A field deployment of a no-code agentic-AI tool — described as a "virtual data scientist" — in a core analytics course at a top-30 global MBA program. Students used natural language to perform segmentation, classification, prediction, and text analytics. Using detailed agent-interaction logs and closed-book final exam scores, the authors estimate the effect of usage intensity on learning outcomes and study heterogeneity across student backgrounds.

Contribution

They find that "doubling the intensity of agentic-AI usage increases a student's final exam score by an average of 0.624 points (or about 1.56 percentage points)." Lower-performing students use the tool less but experience greater marginal returns when they do. Gains concentrate in the applied case portion of the exam, not the concepts section, and are largest under an active "going-to-the-gym" usage mode.

Method

Quasi-experimental field study in an MBA course; combines student-agent interaction logs with closed-book exam scores; estimates dose-response of usage intensity on outcomes and decomposes gains by exam section and usage style.

Relevance to RISE

Empirical study (MBA analytics course) of agentic AI as a virtual data scientist. Mostly analytics-skills focused, but the methodology (LLM-as-collaborator with measurable productivity outcomes) generalizes to RISE evaluation.

Critique / open questions

Single course at a single top-30 MBA program — generalizability to research workflows (versus pedagogical exercises) is untested. "Doubling usage" is a within-sample effect; the causal interpretation relies on the matched/log-based identification described later in the paper (not visible in the excerpt).

Key quotes

"We investigate this question through a field deployment of a no-code, agentic-AI tool — functioning as a virtual data scientist — in a core analytics course at a top-30 global MBA program."

"Students benefit most when they adopt a 'going-to-the-gym' usage mode — actively working through MBA-level case exercises with the tool — rather than relying on it for exam preparation or passively observing teammates."