Summary

About

Phase 1 Invitation

The creation of trusted global governance of AGI will require the participation of not only AGI experts, but also politicians, international lawyers, diplomats, futurists, and ethicists (including philosophers, social scientists). You have been recommended due to your experience in one of more these areas.

AGI experts/thought leaders, but also international lawyers with some UN experience, Ethicist/philosopher/Social Scientists, Futurists, and Politicians —  especially those who will be writing their national regulations are invited to answer to the questionnaire.

This questionnaire is based on personal interviews and statements of 55 AGI experts and thought leaders. A report on that earlier work, Phase 1, is now available here.

The items in this questionnaire are the kind of items that will have to be considered in your national legislation and UN governance on AI and future forms of AGI. You are not asked to answer all the questions, just those you feel comfortable with answering. The first two set of questions are more technical for the AI experienced and the third and fourth set of question relate more to international governance structures.

All those who respond to this questionnaire will be listed in the Phase 2 report (unless you ask not to be) and will receive a copy of the results. No attributions will be made; your name will not be connected to any answer, unless you put your name with the text of your answers. The results will be published and used as input to creating alternative AGI governance scenarios.

Unlike other questionnaires, this Real-Time Delphi allows you to return as often as you like to read other participants’ views and to edit your own comments. You can return as many times as you like until the deadline of December 31, 2023. Please enter your email address here, then use code BNW1, click go and answer a few demographic questions. Please return as often as you like.

Source: Millennium Project

Contact

Email: Information

Locations

4421 Garrison Street, N.W. Washington, D.C. 20016-4055
4421 Garrison Street, N.W. Washington, D.C. 20016-4055
Phone: (202) 685-5179

Web Links

• Webpage

Videos

Transition from Narrow to General Artificial Intelligence

(00:00)
By: The Millennium Project

Jerome C. Glenn, Founder and CEO of The Millennium Project, explains why we need to study the transition from NARROW to GENERAL Artificial Intelligence now in order to get the initial conditions right in this short video made by The Millennium Project. For information and support to the study, please contact info@millennium-project.org or visit www.millennium-project.org

Phase 1 Survey

Overview

Phase 1 Questions:

Origin or Self-Emergence

1. How do you envision the possible trajectories ahead, from today’s AI, to much more capable AGI in the future?
2. What are the most important serious outcomes if these trajectories are not governed, or are governed badly?
3. What are some key initial conditions for AGI so that an artificial super intelligence does not emerge later that is not to humanity’s liking?

Value alignment, morality, values

4. Drawing on the work of the Global Partnership on Artificial Intelligence (GPAI) and others that have already identified norms, principles, and values, what additional or unique values should be considered for AGI?
5. If a hierarchy of values becomes necessary for international treaties and a governance system, what should be the top priorities?
6. How can alignment be achieved? If you think it is not possible, then what is the best way to manage this situation?

Governance and Regulations

7. How to manage the international cooperation necessary to build international agreements and a global governance system while nations and corporations are in an intellectual “arms race” for global leadership?
8. What options or models are there for global governance of AGI?
9. What risks arise from attempts to govern the emergence of AGI? (Might some measures be counterproductive?)
10. Should future AGIs be assigned rights?
11. How can governance be flexible enough to respond to new issues previously unknown at the time of creating that governance system?
12. What international governance trials, tests, or experiments can be constructed to inform the text of an international AGI treaty?
13. How can international treaties and a governance system prevent increased centralization of power crowding out others?
14. Where is the most important or insightful work today being conducted on global governance of AGI?

Control

15. What enforcement powers will be needed to make an international AGI treaty effective?
16. How can the use of AGI by organized crime and terrorism be reduced or prevented? (Please consider new types of crimes and terrorism which might be enabled by AGI.)
17. Assuming AGI audits would have to be continuous rather than one-time certifications, how would audit values be addressed?
18. What disruptions could complicate the task of enforcing AGI governance?
19. How can a governance model correct undesirable action unanticipated in utility functions?
20. How will quantum computing affect AGI control?
21. How can international agreements and a governance system prevent an AGI “arms race” and escalation from going faster than expected, getting out of control and leading to war, be it kinetic, algorithmic, cyber, or information warfare?

And last:  22. What additional issues and/or questions need to be addressed to have a positive AGI outcome?

Initial sample of potential governance models for AGI*

1. IAEA-like model or WTO-like with enforcement powers. These are the easiest to understand, but likely to be too static to manage AGI.
2. IPCC-like model in concert with international treaties.  This approach has not led to a governance system for climate change.
3. Online real-time global collective intelligence system with audit and licensing status, governance by information power. This would be useful to help select and use an AGI system, but no proof that information power would be sufficient to govern the evolution of AGI.
4. GGCC (Global Governance Coordinating Committees) would be flexible and enforced by national sanctions, ad hoc legal rulings in different countries, and insurance premiums. This has too many ways for AGI developers to avoid meeting standards.
5. UN, ISO and/or IEEE standards used for auditing and licensing. Licensing would affect purchases and would have impact, but requires international agreement or treaty with all countries ratifying.
6. Put different parts of AGI governance under different bodies like ITU, WTO, WIPO. Some of this is likely to happen but would not be sufficient to govern all instances of AGI systems.
7. Decentralized Semi-Autonomous TransInstitution. This could be the most effective, but the most difficult to establish since both Decentralized Semi-Autonomous Organizations and TransInstitutions are new concepts.

*Drawn from “Artificial General Intelligence Issues and Opportunities,” by Jerome C. Glenn contracted by the EC for input to Horizons 2024-27 planning.

AGI Experts and Thought Leaders in Phase 1

1. Sam Altman, via YouTube and OpenAI Blog, CEO OpenAI
2. Anonymous, AGI Existential Risk OECD (ret.)
3. Yoshua Bengio. AI pioneer, Quebec AI Institute and the University of Montréal
4. Irakli Beridze, UN Interregional Crime and Justice Res. Ins. Ct. for AI and Robotics
5. Nick Bostrom, Future of Humanity Institute at Oxford University
6. Gregg Brockman, OpenAI co-founder
7. Vint Cerf, Internet Evangelist, V.P. Google.
8. Shaoqun CHEN, CEO of Shenzhen Zhongnong Net Company
9. Anonymous, at Jing Dong AI Research Institute, China
10. Pedro Domingos, University of Washington
11. Dan Faggella, Emerj Artificial Intelligence Research
12. Lex Fridman, MIT and Podcast host
13. Bill Gates
14. Ben Goertzel, CEO SingularityNET
15. Yuval Noah Harari, Hebrew University, Israel
16. Tristan Harris, Center for Humane Technology
17. Demis Hassabis, CEO and co-founder of DeepMind
18. Geoffrey Hinton, AI pioneer, Google (ret)
19. Lambert Hogenhout, Chief Data, Analytics and Emerging Technologies, UN Secretariat
20. Erik Horvitz, Chief Scientific Officer, Microsoft
21. Anonymous, Information Technology Hundred People Association, China
22. Anonymous, China Institute of Contemporary International Relations
23. Andrej Karpathy, Open AI, former AI S Researcher Tesla
24. David Kelley, AGI Lab
25. Dafne Koller, Stanford University, Coursera
26. Ray Kurzweil, Director of Engineering Machine Learning, Google
27. Connor Leahy, CEO Conjecture
28. Yann LeCun, Professor New York University, Chief Scientist for Meta
29. Shane Legg, co-founder of DeepMind
30. Fei Fei Li, Stanford University, Human Centered AI
31. Erwu Liu, Tongji University AI and Blockchain Intelligence Laboratory
32. Gary Marcus, NYU professor emeritus
33. Dale Moore, US Dept of Defense AI consultant
34. Emad Mostaque, CEO of Stability.ai
35. Elon Musk
36. Gabriel Mukobi, PhD student Stanford University
37. Anonymous, National Research University Higher School of Economics
38. Judea Pearl, Professor UCLA
39. Sundar Pichai, Google CEO
40. Francesca Rossi, Pres. of AAAI, IBM Fellow and IBM’s AI Ethics Global Leader
41. Anonymous, Russian Academy of Science
42. Stuart Russell, UC Berkeley
43. Karl Schroeder, Science Fiction Author
44. Bart Selman, Cornel University
45. Javier Del Ser, Tecnalia, Spain
46. David Shapiro, AGI Alignment Consultant
47. Yesha Sivan, Founder and CEO of i8 Ventures
48. Ilya Sutstkever, Open AI co-founder
49. Jaan Tallinn, Ct. Study of Existential Risk at Cambridge Univ., and Future of Life Institute
50. Max Tegmark, Future of Life Institute and MIT
51. Peter Voss, CEO and Chief Scientist at Aigo.ai
52. Paul Werbos, National Science Foundation (ret.)
53. Stephen Wolfram, Wolfram Alpha, Wolfram Language
54. Yudong Yang, Alibaba’s DAMO Research Institute
55. Eliezer Yudkowsky Machine Intelligence Research Institute

There are many excellent centers studying values for and the ethical issues of ANI, but not potential global governance models for the transition to AGI. The distinctions among ANI, AGI, and ASI are usually missing in these studies; even the most comprehensive and detailed US National Security Commission on Artificial Intelligence Report has little mention.[iv] Current work on AI governance is trying to catch up with the artificial narrow intelligence that is proliferating worldwide today; we also need to jump ahead to anticipate governance needs of what AGI could become.

It is argued that creating rules for governance of AGI too soon will stifle its development. Expert judgments vary about when AGI will be possible; however, some working to develop AGI believe it is possible to have AGI as soon as ten years.[v] Since it is likely to take ten years to: 1) develop ANI to AGI international or global agreements; 2) design the governance system; and 3) begin implementation, then it would be wise to begin exploring potential governance approaches and their potential effectiveness now.

Order Phase 1 Results

Order the Phase 1 download here (original English, Spanish or Chinese translations)

Phase 2 Survey

Contact the The Millennium Project for the survey results