ispaik06

Notes

Study notes compiled from my research wiki. Series are read in order, course notes follow the lecturer, and a paper gets a prerequisites note before the paper itself.

Map of the notes

Lecture series

Courses

David Silver's Reinforcement Learning Course

9 lectures·EN / KO·updated 2026-09-13

UCL, 2015. Lecture notes that say only what the slides don't, question lists from my own reading, and a Korean reading companion for lecture 6. Slides are shown under CC BY-NC 4.0.

L2Markov Decision ProcessesL3Planning by Dynamic ProgrammingL4Model-Free PredictionL5Model-Free ControlL6Value Function ApproximationL7Policy GradientL8Integrating Learning and PlanningL9Exploration and ExploitationL10Classic Games

Question Lists: L2, L3, L4, L5·Reading Companion: L6

Papers

Notes

Map

What each note assumes and what it leads to. Follow the arrows; dashed nodes are lectures still to be written.

PROBABILITY AND ESTIMATION CONTROL REINFORCEMENT LEARNING PAPERS AND SYSTEMS L1 Joint, marginal,conditionalBAYES AND PF L2 Bayes theoremBAYES AND PF L3 Hidden statemodelsBAYES AND PF L4 The BayesfilterBAYES AND PF L5 Monte Carlo,particlesBAYES AND PF L6 ImportancesamplingBAYES AND PF L7 Resampling andSIRBAYES AND PF Kalman filterKALMAN FILTER IMU Kalman filter(2023)WORK LQR, Riccati,BellmanNOTE MPC lecturePLANNED Convex MPC for abipedARTICLE L2 Markov decisionprocessesSILVER RL L3 DynamicprogrammingSILVER RL L4 Model-freepredictionSILVER RL L5 Model-freecontrolSILVER RL L6 FunctionapproximationSILVER RL L7 PolicygradientSILVER RL L8 Learning andplanningSILVER RL L9 ExplorationSILVER RL L10 Classic gamesSILVER RL Lee 2020prerequisitesPAPER Lee 2020 papernotePLANNED

PublishedPlannedElsewhere on the web