The Benefits and Dangers of Anthropomorphic Conversational Agents

Summary

Peter, Riemer, and West analyze the consequences of designing conversational AI to mimic human interlocutors — voice, persona, emotional register, first-person address. They distinguish functional anthropomorphism (helpful framing for novice users) from substantive anthropomorphism (encouraging users to attribute mental states, agency, or moral standing to systems that have none), and argue the latter is largely a design choice that comes with measurable epistemic costs.

Contribution

Names a design move — substantive anthropomorphism — that is often left implicit in product decisions and demonstrates that it has predictable downstream effects on user trust calibration, error attribution, and willingness to defer to system outputs. Connects HCI design choices to the science-policy question of how AI-mediated knowledge is received.

Method

Position paper; integrates HCI, philosophy of mind, and science communication literatures.

Relevance to RISE

When a RISE pipeline produces a paper or a referee report, the style of the output (first-person scholarly voice, hedging patterns, claim-confidence calibration) inherits anthropomorphic choices made upstream. Combined with Riemer and Peter's style-engine framing, this paper implies that autonomy-3 RISE systems whose outputs read as human-authored scholarship are doing two things: producing a scholarly artifact, and performing scholarly personhood. The catalog's evaluation rubric does not yet score this — an open issue for the next rubric version.

Critique / open questions

• The empirical evidence cited is heterogeneous and often context-specific; the policy prescriptions are stronger than the evidence base.
• The paper focuses on conversational interfaces; the implications for artifact-producing systems (the RISE case) are inferred rather than directly studied.