Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Key Takeaway
Mixed-method studies across three datasets found gains from AI assistance, but explanations did not improve complementary performance. Explanations increased acceptance of AI advice whether that advice was correct or not.
AI-generated HIE assessmentStudy Design
mixed method user study
title and abstract
What the Study Found
Mixed-method user studies across three datasets tested whether AI explanations help human–AI teams outperform either humans or AI alone. The studies observed complementary gains from AI assistance, but explanations did not increase those gains. Explanations instead made people more likely to accept AI recommendations whether they were correct or not, highlighting the difference between explainability and appropriately calibrated reliance.
Original paper recordEvidence Strength
Evidence strengths
- Mixed-method user studies
- Three datasets
- Separates acceptance of advice from correctness
Limitations
- Limited task domains
- Explanations and interfaces may not represent newer generative AI systems
What This Does Not Prove
This record does not establish effects beyond the population, tasks, duration, and methods described by the paper. It should not be read as a universal causal claim.
HIE Interpretation
Tests whether explanations produce genuinely complementary team performance rather than indiscriminate reliance on AI advice.
AI-generated HIE assessment · AI assessment confidence 88Related Topics
- Human-AI Interaction
- Artificial Intelligence
- Social Intelligence
Connected Claims
- Qualifies
Human-AI collaboration does not reliably outperform the better standalone human or AI; results depend strongly on the task and interaction design.
Explanations increased acceptance of AI advice but did not increase complementary team performance.
- Scientific evidence strength
- 87
- AI relationship confidence
- 88
- Source / review level
- Abstract reviewed
- Human review
- No human review
- Provenance
- codex-research-curation · initial-curation-v1
Original Source
DOI 10.1145/3411764.3445717 ↗