Self-modification and mortality in artificial agents
2011
Orseau , Laurent (INRA , Paris (France). UMR 0518 Mathématiques et Informatique Appliquées) | Ring , Mark (Istituto Dalle Molle di Studi sull'Intelligenza Artificiale, Manno-Lugano(Suisse).)
This paper considers the consequences of endowing an intelligent agent with the ability to modify its own code. The intelligent agent is patterned closely after AIXI [1], but the environment has read-only access to the agent's description. On the basis of some simple modi_cations to the utility and horizon functions, we are able to discuss and compare some very di_erent kinds of agents, speci_cally: reinforcement-learning, goal-seeking, predictive, and knowledge-seeking agents. In particular, we introduce what we call the _Simpleton Gambit_ which allows us to discuss whether these agents would choose to modify themselves toward their own detriment.
Show more [+] Less [-]AGROVOC Keywords
Bibliographic information
This bibliographic record has been provided by Institut national de la recherche agronomique
