Universal knowledge-seeking agents
2014
Orseau, Laurent | Mathématiques et Informatique Appliquées (MIA-Paris) ; Institut National de la Recherche Agronomique (INRA)-AgroParisTech
Reinforcement learning (RL) agents like Hutter's universal, Pareto optimal, incomputable AIXI heavily rely on the definition of the rewards, which are necessarily given by some "teacher" to define the tasks to solve. Therefore, as is, AIXI. cannot be said to be a fully autonomous agent. From the point of view of artificial general intelligence (AGI), this can be argued to be an incomplete definition of a generally intelligent agent. Furthermore, it has recently been shown that AIXI can converge to a suboptimal behavior in certain situations, hence showing the intrinsic difficulty of RL, with its non-obvious pitfalls. We propose a new model of intelligence, the knowledge-seeking agent (KSA), halfway between Solomonoff induction and AIXI, that defines a completely autonomous agent that does not require a teacher. The goal of this agent is not to maximize arbitrary rewards, but to entirely explore its world in an optimal way. A proof of strong asymptotic optimality for a class of horizon functions shows that this agent behaves according to expectation. Some implications of such an unusual agent are proposed.
显示更多 [+] 显示较少 [-]