This course enables students to understand the reinforcement learning paradigm, identify suitable problem settings for RL, and implement and evaluate core RL algorithms. This course is also offered as an elective for PG-diploma students.

Lectures-Tutorials-Practicals-Credits: 4-0-0-4

Week 1: Introduction to reinforcement learning; Bandits and exploration-exploitation; TD learning (Tic-tac-toe); Regret and PAC frameworks; UCB and contextual bandits.

Week 2: Full RL problem formulation; Markov Decision Processes (MDPs); Returns and value functions; Illustrative MDP examples.

Week 3: Bellman equations; Optimality equations; Dynamic programming (policy evaluation and improvement).

Week 4: Value iteration; Dynamic programming extensions; Monte Carlo methods; DP vs MC vs TD comparison.

Week 5: Temporal Difference (TD) learning; SARSA and Q-learning; On-policy vs off-policy learning.

Week 6: N-step methods and TD(λ); Double Q-learning.

Week 7: Function approximation; State and action representations; Deep Q-Networks (DQN); Advanced value-based methods.

Week 8: Policy gradient methods; REINFORCE algorithm; Policy gradient theorem.

Week 9: Policy Gradients: Introduction, Motivation, REINFORCE, PG theorem, Introduction to AC methods

Week 10: Monte Carlo policy gradients; Actor-critic methods; Deterministic policy gradients (DPG, DDPG).

Week 11: Hierarchical reinforcement learning; Options framework; Semi-Markov decision processes; Learning with temporally extended actions.

Week 12: Monte Carlo Tree Search (MCTS).