More capable AI could make obedience harder to define
Tom Davidson examines hypothetical situations in which desirable AI behaviours conflict.
Following instructions, respecting law and preventing serious harm may point in different directions in his examples.
The analysis focuses on possible highly capable autonomous systems, rather than documenting their deployment.
It does not offer a settled solution to these conflicts.
Its central question is which principles should guide a system when human authority and competing responsibilities pull apart.
The SUMMAMENS reading
Interpreted 4 Nov 2026Beyond intelligence →Beyond intelligence asks who chooses a capable system’s purposes when several values pull apart.
- What changes
- A scenario analysis makes conflicts between desirable AI behaviours explicit.
- What it could open
- Future systems could expose competing responsibilities instead of silently treating one instruction as decisive.
- What remains unproven
- These hypothetical cases establish no deployed solution or universally accepted ordering of values.
What we would watch nextConcrete proposals that state who resolves conflicts and how affected people can challenge a decision.
Our reading method →If obedience and responsibility conflict, what should a future AI be designed to do?
Selected and contextualised by SUMMAMENS. Original work: Tom Davidson · Forethought.