Drone factories in Russia a problem?
AI to supply plausible deniability. No problem
Some questions/areas I want to look into more and refine my views about:
- Harnesses, scaffolds, tool use, separate environments, and boundaries generally remain useful regardless of capabilities. What principles determine where they arise and where scale and a degree of generality suffices?
- It's clear that automating domains with verifiable rewards is very solvable. To what degree does technological and societal change depend on these? (We need a more systematic study of bottlenecks.)
- What would a Fields medal type challenge look like in non-STEM domains? How should normative elements be evaluated? What would make me say "ok we have superinteligence in economics."
- If you assume reasoning and information provision are 'solved', the physical world is one clear bottleneck. If you're trying to mitigate large scale risks, which resources, places, factories, land etc will need stronger oversight, redesign, hardening etc?
- Assume "verification" technologies are now mature and deployable (whether with hardware or software), what concrete existing policy problem/conflict could actually be addressed? Ie to what extent is mistrust a core blocker to progress?
- If we assume AGI empowers the executive and the military, reducing the size of the state whilst boosting it's effectiveness; what exactly would it mean for constitutional protections to be adequately bolstered? Where should we look today, as opposed to imagining how it pans out in the future?
- Should there in fact be some sort of "true creativity" deficit with models, would this imply a world where humans and AIs are complements for a long time, with each new training run merely absorbing the fruits of this collaboration?
- How exactly does RLVR mess up or affect safety fine tuning and the model's persona? To what extent does it override and affect the desirable propensities trained through constitutional AI like methods? What does a good RL environment look like?
