Beyond Black Boxes: Designing and Testing Agentic AI Systems for Strategy

Summary

Explicitly adopts a design view inspired by Herbert Simon's work on system architecture, arguing that purposeful AI system design - not access to generic models - is itself a dynamic capability, then builds Aristotle, an agentic multiagent system for theory-based strategic decision making, and tests a streamlined three-agent version against GPT-4o and a human-only condition with 976 managers. Experienced managers gain quality without confidence inflation while highly educated managers gain confidence without quality, establishing user-system-problem fit as a design dimension, and the paper abductively derives a five-dimensional taxonomy of the agentic-AI design space. This is the strongest management-venue precedent for treating multi-agent architecture as the object of theory rather than a black box, and the ISR paper needs it both to position its contribution and to borrow the design-space vocabulary.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-09-07 literature sweep as a high-priority candidate for the Architecture as Epistemology (ISR) paper (angle: organization theory applied to AI agents). Verified against: Verified via CrossRef metadata (api.crossref.org/works/10.1287/stsc.2025.0432) and the Semantic Scholar Graph API record on 2026-09-07: full abstract read verbatim, 4 authors, Strategy Science. Dates differ slightly between sources (CrossRef January 2026, Semantic Scholar 2026-02-20); confirm the issue date before citing. INFORMS publisher page returns HTTP 403 to automated fetching.

Critique / open questions

Queued — not yet assessed.

Key quotes

Existing work on AI and strategy examines the effects of general-purpose models, implicitly assuming that strategists will use whatever tools are available and that human-AI interaction is largely a matter of prompting, complacency, or aversion. (abstract)

Inspired by Herbert Simon’s work on system architecture, we instead adopt a design view in which purposeful AI system design and use—not mere access to generic models—can itself be a dynamic capability and a source of competitive advantage. (abstract)