Nature Human Behaviour
When combinations of humans and AI are useful: A systematic review and meta-analysis
Key Takeaway
This preregistered synthesis of 106 experiments and 370 effect sizes found that human-AI combinations underperformed the better standalone agent on average. Outcomes varied materially by task, with losses in decisions and gains in content creation.
AI-generated HIE assessmentStudy Design
systematic review and meta analysis
title and abstract
What the Study Found
Inspired by the increasing use of artificial intelligence (AI) to augment humans, researchers have studied human–AI systems involving different tasks, systems and populations. This preregistered systematic review and meta-analysis covers 106 experimental studies reporting 370 effect sizes. On average, human–AI combinations performed worse than the best of humans or AI alone, with losses in decision tasks and greater gains in content-creation tasks. The authors note possible publication bias and variation among study designs.
Original paper recordEvidence Strength
Evidence strengths
- Preregistered systematic review
- 106 experimental studies and 370 effect sizes
- Explicit comparison of human, AI, and combined performance
Limitations
- Possible publication bias
- Substantial variation among included study designs
What This Does Not Prove
This record does not establish effects beyond the population, tasks, duration, and methods described by the paper. It should not be read as a universal causal claim.
HIE Interpretation
Directly synthesizes evidence about when human-AI combinations outperform or underperform their components.
AI-generated HIE assessment · AI assessment confidence 95Related Topics
- Human-AI Interaction
- Artificial Intelligence
- Social Intelligence
Connected Claims
- Qualifies
Human-AI collaboration does not reliably outperform the better standalone human or AI; results depend strongly on the task and interaction design.
Across 106 experiments, combinations were worse than the better standalone agent on average, with task type explaining important variation.
- Scientific evidence strength
- 95
- AI relationship confidence
- 95
- Source / review level
- Abstract reviewed
- Human review
- No human review
- Provenance
- codex-research-curation · initial-curation-v1
Original Source
DOI 10.1038/s41562-024-02024-1 ↗