AI Constitution
1

Candidate Thesis

Superintelligence

Sign in to propose an editHistory

No superintelligent AI should be developed.

Summary

The case for including this

A flat prohibition on superintelligent AI responds to the argument that a system vastly exceeding human capability could be uncontrollable and pose an existential threat that no after-the-fact safeguard could contain. Drawing a bright line before that threshold is a precautionary stance proportionate to a risk that may be irreversible. It forces the question of whether some capabilities should simply never be built.

The case for changing or excluding this

Including this trades forgone benefits against existential safety: a ban forecloses potentially civilization-transforming gains—cures, breakthroughs, abundance—and may be unenforceable, merely driving development underground or toward less cautious actors. The prohibition also hinges on 'superintelligence,' a term with no agreed definition or measurable threshold, and it assumes danger is intrinsic to capability rather than to alignment and control, which other provisions address more directly. It would be more workable reframed around governed, controllable development and hard safety thresholds than an absolute capability ceiling.

Discussion

Sign in to join the discussion.

Related resources