Look Who it Was Random video

Ted Goertzel - Minimizing Risks in Developing Artificial General Intelligence

943 views · Published 12 May 2013 · 34:00 · Indexed 28 October 2026

Channel: Science, Technology & the Future · 2013 · Science & Technology

Watch on YouTube

Winter Intelligence Oxford - FHI - http://www.winterintelligence.org  Abstract: This paper posits that elements of human-level or superhuman artificial general intelligence will emerge gradually and unevenly over a period of years rather than appearing full blown in one dramatic event or singularity. This is likely because funding for AI research will go mostly to projects that promise short-term payoffs. Although these projects will focus on narrow goals, achieving them will require that their developers incorporate more general AI capabilities. They will create a new class of AGI experts significantly more advanced than current narrow-AI systems, but without the full range of human-level capabilities. These AGI experts are likely to be in use for a period of years before the much feared development of a superhuman AGI with the capability of modifying itself into new forms and presenting an existential risk to the human species. When this superhuman AGI develops, it will incorporate the capabilities of earlier AGI expert systems, just as the human brain evolved from the brains of more primitive ancestors.

We cannot wait until superhuman AGI is here to begin work on risk control strategies. What we must do, this paper argues, is advance risk protection in step with the development of narrow-AI into AGI expert systems. Doing so will help to minimize the risks of emerging AGI expert systems, risks which may be very substantial. We can also try to ensure that the superhuman AGIs of the future build on this foundation and incorporate risk management strategies used by AGI expert systems.

This paper examines risk prevention strategies being developed today in areas where artificial intelligence is relatively advanced. Areas to be highlighted will include: financial trading, medical devices, security monitoring, rumor management, and response to disasters such as massive electrical blackouts.

Financial trading is a good place to begin because artificial intelligence has already surpassed human intelligence in some ways in that field. It is certainly much quicker, executing very large numbers of trades based on complex analyses in small fractions of a second. This enables investors to profit simply by reacting to what large institutional investors have done before human traders have time to react. Trades caused by bugs in the software can also be executed very quickly, and repetitively, leading to sometimes dramatic instability in markets and massive losses by trading firms or other investors. Risk management strategies include circuit breakers, which cut off trading when prices go beyond reasonable bounds, and transaction taxes which discourage high volume trading. More generally, the problems with computer trading highlight the issue of speed. AGIs that interact with human systems, or that require human oversight in their operations, may need to be slowed down with artificially imposed constraints.

Heart pacemakers have been regulating human heartbeats for some years, and are programmed to respond to emotional and physical processes in the host. Failure could have life-threatening consequences, yet safeguards have made them safe enough for very widespread use. Diseases such as Parkinson's disease, thyroid disease and diabetes can be regulated with automated systems that provide a variable flow of chemicals responding to the body's needs. These technologies may be important steps toward the development of cyborgs. This paper will examine the way in which the risks of these technologies are being minimized, and the ways in which new technologies are projected to improve on them in the future.

... http://www.winterintelligence.org/agi-impacts-extended-abstracts/

Facebook group on xRisk: https://www.facebook.com/groups/ExistentialRisk/

More from this channel