Definition
Artificial Superintelligence (ASI) is a hypothetical AI that greatly exceeds the best human minds across nearly all areas, including science, strategy and social skill. The philosopher Nick Bostrom's 2014 book brought wide attention to the idea, which remains speculative.
In simple words
A calculator beats you at arithmetic and nothing else. ASI would beat everyone at nearly everything, including research into building better AI. That's the source of both the excitement and the worry.
Why it matters
It frames some of the biggest debates in AI policy. Even if you doubt it will ever happen, you'll meet these arguments constantly, so it helps to know what they claim and how they're challenged.
How it works
The main arguments
| Idea | Claim | Common doubt |
|---|---|---|
| Intelligence explosion | An AI that can improve AI could speed up its own progress (I. J. Good, 1965). | Hardware, data and experiments may set limits. |
| Orthogonality | Any level of intelligence can pair with almost any goal. | Goals may not be so independent in practice. |
| Instrumental convergence | Many goals lead to the same sub-goals, like gaining resources. | Real systems may not behave like idealised agents. |
Common mistakes
- Treating science fiction as a forecast, or dismissing every concern because it sounds like science fiction.
- Picturing ASI as an evil robot. The usual concern is a capable system pursuing the wrong goal, with no malice involved.
Check yourself
Summary
- ASI would far exceed the best people at almost everything.
- It is hypothetical, and the arguments about it are contested.
- The usual concern is wrong goals, rather than malice.
Finished reading?