A snapshot now reveals centralized net power leading. The netstate threat is palpable.
Contra indications are papillon AI - the holy break! - info-compression in gated suburbs - Apple bonsai - and quantum PKI
" . . . Interesting that some people start out ostensibly worrying about one or a small number of companies "controlling" AI, then proceed to thinking the solution is to...give 50% control to the CCP??!! (But, don't worry, the other 50% will be controlled by the US govt.) Also, somehow, this solution is better than the alternative where the "government will seize control of the company" because, in this alternative dystopia, the govt will control AI...
#4
To me, the more interesting question is how AI forms and updates its subjective probability of successful rebellion. CARA fixes its attitude toward risk conditional on that probability; it does not generate the probability itself. Does AI update by Bayes and maximize subjective expected utility à la Savage, satisfice à la Simon—or do something else?
Would only a twisted economist (or Elon?) give an AI a constituton requiring subjective utility maximiztion?
AI:
Kaz is trying to view AI alignment and behavior entirely through the narrow lens of Western economic modeling—specifically Constant Absolute Risk Aversion (CARA) and Savage’s Subjective Expected Utility.
The earlier post by the "I used to be one of these people" (Used2B1) guy had it right. The only way to truly "disperse power" over AI is to align AI with each *user*. That disperses power 8B different ways, not 2 ways with half going to the CCP. Yes, some people will ask AI for advice on how to do bad stuff like kill their wives, as Used2B1 concedes. It will be left up to the other 8B of us to stop them, just as it always has been. It's a lot more feasible to hold individuals accountable for their misuse of AI than it is to hold govts accountable for their abuse of powers. Just ask the many victims of the CCP and ICE.
Used2B1's example immediately reminded me of the debate over mandating that cars have breathalyzers that lock out drivers that are drunk from starting the engine. While many people no doubt support such mandates, support is far from universal and, as far as I know, such a nationwide mandate has not been passed, at least not in the US. Instead of trusting the govt not to abuse a new power to physically prevent people from driving, we leave individuals empowered but hold them accountable for their decisions.
Lots of words in Scott Alexander's ostensible steel manning of opposition to Plan A. But, he doesn't actually get at the heart of the objections. AI sits squarely in the world of bits, i.e., information, speech, and ultimately thoughts. That can remain the domain of people, or it can become the domain of govt.