Then full warp
dynamics of an RSI-driven intelligence explosion be?
…Toby Ord models out what rapid AI development could look like…
Researcher Toby Ord has tried to think through what recursive self-improvement could look like in practice. His take is that the dynamics of RSI are ultimately going to be limited by some resource constraints and time constraints, such that an AI system going through recursive self-improvement has certain limits on how dramatically it can accelerate progress.
“It seems highly unlikely that generation times can be brought arbitrarily close to zero. This provides an important kind of barrier to singular growth,” Ord writes. “One would expect the generation time for training the next generation of models to bottom out at some unyielding finite limit, prematurely ending the period of singular growth.”
“This dampening force could come from running out of room for improvement as a system approaches some form of perfection (such as optimal intelligence per unit resource), or it could come from something like straining under the growing size or complexity of the system”.
Some of the ways AI growth could asymptote due to various limits:
Limits of intelligence itself: “Even an optimal reasoner would be neither omniscient nor omnipotent”.
Limits of intelligence per unit resource: “Even if we have the optimal algorithms and hardware, our solar system has only one star out of the 200,000,000,000 in our galaxy, and growth beyond our system is slow and cubic.”
Limits of the hardware: “Even an optimal silicon chip may be far below the physical limits of compute per unit resource”.
Limits of the algorithm: “Even an optimal neural network may be far below the best intelligence that could be achieved with that amount of compute”.
Limits of training data: “The training data we have (and could acquire during RSI) is lacking a lot of information on many domains (especially non-verbalisable information and contextual information)”.
RSI weak links: In the same way a task is composed of many distinct things that need to happen in sequence, the same is true of RSI - and this occurs over multiple abstractions and multiple timescales.
Ord’s basic position is that each of these things likely has some kind of hard limit and both the speed at which a system refines these things to approach the hard limits and the absolute limits will govern the shape of an intelligence explosion: “There are many kinds of feedback loop that could contribute to RSI, ranging from decades (e.g., designing a successor to EUV lithography) to months (e.g., designing better pretraining) to seconds (e.g., designing better scaffolds).”
The four phases of an intelligence explosion:
0: The initial exponential phase when the doubling time is driven by human-only research.
1: RSI drives the generation time down towards machine speeds, so the growth rate rises above what humans can achieve; this phase is super-exponential.
2: Fully automated RSI continues for a while at this faster exponential, but starts to saturate.
3: As it approaches its inflection point and subsequent horizontal plateau, the trajectory departs from its exponential and is revealed to be a logistic.
Why this matters - solar system expansion: Papers like this are helpful for thinking about the absolute speed with which a digital intelligence might increase its own capabilities. It also implicitly speaks to the larger challenge of how to think about stellar expansion for an intelligence as it seeks to harness the energy of the areas around Earth, the inner solar system, the outer solar system, and so on.
“While I’ve argued that singular growth is harder than we may have thought, that doesn’t mean RSI is safe or that AI R&D will move at a manageable pace”, Ord writes. “if the human-only trajectory were A(t) and RSI sped this up to A(10t), we’d be getting a decade of human-only progress each year, introducing many of the dangers - even without any change in the fundamental shape of the curve.”
Read more: The Dynamics of Intelligence Explosions (arXiv).
No comments:
Post a Comment