Reinforcement Learning: Tutorial and Survey

This is a tutorial and survey paper on reinforcement learning, from fundamental reinforcement learning to deep reinforcement learning. It starts with introducing the elements of reinforcement learning. Then, Markov decision process and policy are explained. Bellman equation is introduced. Then, value iteration, policy iteration, and modified policy iteration are introduced for solving Markov decision process. Then, difference of reinforcement learning and Markov decision process is mentioned followed by temporal difference evaluation. Then, Q function, Q-learning, epsilon-greedy policy, gradient Q-learning, experience replay, and deep Q network are covered. Afterwards, policy gradient and the REINFORCE algorithm are explained. Finally, the details of AlphaGo -- as one of the successful applications of reinforcement learning -- are introduced.