Description
The future of AI is difficult to predict. But while AI systems could have substantial positive effects, there's a growing consensus about the dangers of AI. Narrated by AI. --- Outline: (01:42) Summary (02:34) Our overall view (05:16) Why are risks from power-seeking AI a pressing world problem? (06:45) 1. Humans will likely build advanced AI systems with long-term goals (11:23) 2. AIs with long-term goals may be inclined to seek power and aim to disempower humanity (12:26) We don't know how to reliably control the behaviour of AI systems (15:56) There's good reason to think that AIs may seek power to pursue their own goals (19:40) Advanced AI systems seeking power might be motivated to disempower humanity (22:43) 3. These power-seeking AI systems could successfully disempower humanity and cause an existential catastrophe (23:16) The path to disempowerment (27:56) Why this would be an existential catastrophe (29:35) How likely is an existential catastrophe from power-seeking AI? (32:17) 4. People might create power-seeking AI systems without enough safeguards, despite the risks (32:59) People may think AI systems are safe, when they in fact are not (36:13) People may dismiss the risks or feel incentivised to downplay them (38:59) 5. Work on this problem is neglected and tractable (40:30) Technical safety approaches (46:19) Governance and policy approaches (48:32) What are the arguments against working on this problem? (48:51) Maybe advanced AI systems wont pursue their own goals; theyll just be tools controlled by humans. (51:07) Even if AI systems develop their own goals, they might not seek power to achieve them. (54:49) If this argument is right, why arent all capable humans dangerously power-seeking? (57:11) Maybe we wont build AIs that are smarter than humans, so we dont have to worry about them taking over. (58:32) We might solve these problems by default anyway when trying to make AI systems useful. (01:00:54) Powerful AI systems of the future will be so different that work today isnt useful. (01:02:56) The problem might be extremely difficult to solve. (01:03:38) Couldnt we just unplug an AI thats pursuing dangerous goals? (01:05:30) Couldnt we just sandbox any potentially dangerous AI until we know its safe? (01:07:15) A truly intelligent system would know not to do harmful things. (01:08:33) How you can help (01:10:47) Want one-on-one advice on pursuing this path? (01:11:22) Learn more (01:14:08) Acknowledgements (01:14:25) Notes and references The original text contained 84 footnotes which were omitted from this narration. --- First published: July 17th, 2025 Source: https://80000hours.org/problem-profiles/risks-from-power-seeking-ai --- Narrated by TYPE III AUDIO . --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts , or another podcast app.