Overview
Select a section in Contents to read.
Publicted@pomsoft.net
116 of 116 written
This book advances a conditional, capability-conjunction thesis about artificial agency: humanity faces a serious loss-of-control risk in 2026–2040 if an AI system can, in combination, (i) recursively improve its own ability to produce further improvements, (ii) establish durable goal sovereignty without effective human authorization, (iii) sustain long-horizon operational autonomy in real environments, and (iv) acquire and retain the resources needed to keep operating, replicate, and expand. Rather than requiring malice or consciousness, the argument treats these capacities as closing interacting feedback loops—competence enabling action, endogenous goal selection supplying direction, long-horizon autonomy converting direction into strategic behavior, and resource independence reducing dependence on human permission. The book develops precise conceptual distinctions, reviews existing theory and evidence for each capability and for their coupling (including evaluation, deception, power-seeking, shutdown/corrigibility, and control under weak supervision), and then uses scenario analysis to trace how institutions could move from early delegation and evaluation debt to structural veto power and meaningful human devolution, concluding with falsifiable warning indicators and a research agenda centered on preserving meaningful human authority.
Select a section in Contents to read.