Trust Hinders Bidirectional Human-AI Alignment
Trust calibration remains the dominant framing of AI-assisted decision making: the goal is to close the gap between perceived and actual trustworthiness of AI systems so that users exhibit appropriate reliance on them.
ProofPaper ↗
Key points
- We argue that this approach, while useful for decision problems that are well-defined and clearly scoped, overly restricts what it means to make a decision with AI support.
- To overcome this limitation, we develop three dimensions of AI-assisted decision making.
- Second, trust calibration is often misapplied when AI models "large-world" decision problems, i.e., ill-defined situations where goals and states are only partially known or unknowable.
- We conclude by reflecting on the implications of our observations on the design of process-oriented human-AI coupled systems that jointly identify and negotiate uncertainty throughout the decision-making task.
Sources (1)
- [1]Trust Hinders Bidirectional Human-AI AlignmentarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 07:26 PM
Trust calibration remains the dominant framing of AI-assisted decision making: the goal is to close the gap between perceived and actual trustworthiness of AI systems so that users exhibit appropriate reliance on them.
We argue that this approach, while useful for decision problems that are well-defined and clearly scoped, overly restricts what it means to make a decision with AI support.
Extractive summary: sentences quoted from the sources.