This paper presents an imitation learning system capable of learning tasks in a complex dynamic real-time environment. In this paper, we argue that social learning should be thought of as a special case of general skill learning, and that the biases it presents to the skill learning problem radically simplify learning for species with sufficient innate predisposition to harness this power. We decompose skill learning into four subproblems, then show how a modification of Roy's CELL system can address all these problems simultaneously. Our system is demonstrated working in the domain of a real-time virtual-reality game, Unreal Tournament. Read more...

In most modern video games, character behavior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. Yet, if game characters could learn through interacting with the player, behavior could improve as the game is played, keeping it interesting. This paper introduces the real-time Neuroevolution of Augmenting Topologies (rtNEAT) method for evolving increasingly complex artificial neural networks in real time, as a game is being played. The rtNEAT method allows agents to change and improve during the game. In fact, rtNEAT makes possible an entirely new genre of video games in which the player trains a team of agents through a series of customized exercises. To demonstrate this concept, the Neuroevolving Robotic Operatives (NERO) game was built based on rtNEAT. In NERO, the player trains a team of virtual robots for combat against other players' teams. This paper describes results from this novel application of machine learning, and demonstrates that rtNEAT makes possible video games like NERO where agents evolve and adapt in real time. In the future, rtNEAT may allow new kinds of educational and training applications through interactive and adapting games. Read more...