Reinforcement LearningRichard S. Sutton Reinforcement learning is the learning of a mapping from situations to actions so as to maximize a scalar reward or reinforcement signal. The learner is not told which action to take, as in most forms of machine learning, but instead must discover which actions yield the highest reward by trying them. In the most interesting and challenging cases, actions may affect not only the immediate reward, but also the next situation, and through that all subsequent rewards. These two characteristics -- trial-and-error search and delayed reward -- are the most important distinguishing features of reinforcement learning. Reinforcement learning is both a new and a very old topic in AI. The term appears to have been coined by Minsk (1961), and independently in control theory by Walz and Fu (1965). The earliest machine learning research now viewed as directly relevant was Samuel's (1959) checker player, which used temporal-difference learning to manage delayed reward much as it is used today. Of course learning and reinforcement have been studied in psychology for almost a century, and that work has had a very strong impact on the AI/engineering work. One could in fact consider all of reinforcement learning to be simply the reverse engineering of certain psychological learning processes (e.g. operant conditioning and secondary reinforcement). Reinforcement Learning is an edited volume of original research, comprising seven invited contributions by leading researchers. |
Contents
Learning R J Williams | 10 |
Practical Issues in Temporal Difference Learning G Tesauro | 33 |
QLearning C J C H Watkins and P Dayan | 55 |
SelfImproving Reactive Agents Based on Reinforcement Learning Planning | 68 |
Transfer of Learning by Composing Solutions of Elemental Sequential Tasks | 99 |
The Convergence of TDX for General X P Dayan | 117 |
A Reinforcement Connectionist Approach to Robot Path Finding in NonMaze | 139 |
Other editions - View all
Common terms and phrases
action model action representation AHCON AHCON-M approach architecture awij backgammon backpropagation Barto behavior characteristic eligibility composite task computed configuration connectionist connectionist networks convergence CQ-L defined deterministic dynamic programming elemental tasks encode environmental reinforcement episode equation error estimate evaluation function expected value experience replay Əln Figure frameworks gating module goal gradient heuristic hidden units learning agent learning algorithm learning rate learning system Lemma lookup table Machine Learning Markov chain move number of hidden obstacles optimal policy output units parameter path finding problem path-finder payoff performance prediction probability proof Q-learning Q-module QCON QCON-M qo-path random real process REINFORCE algorithm reinforcement baseline reinforcement learning reinforcement signal relaxation planning reward robot path finding sequence simulation step strategy subtask supervised learning Sutton TD learning TD(X temporal difference temporal difference learning terminal value theorem tion transfer of learning vector Watkins weights