Reading list

Textbooks

Reinforcement learning: An Introduction

• Chapters 1-6, including Q-learning
• free online

Machine learning

• Machine learning A Probabilistic Perspective, e.g., Chapters 5-8, (free online)
• Pattern Recognition and Machine Learning (free online)

Artificial intelligence: A Modern Approach

• Part I-III, including game tree search methods
• printed version

Deep learning

• free online

Game Theory

• Part I and II, suggested for whom interested in imperfect information games
• free online

Game AI

• free online

Related Conferences

• IEEE Computational Intelligence and Games (CIG)
• Advances in Computer Games
• Computers and Games

Research Topics

MCTS (Monte-Carlo Tree Search)

• Bandit Based Monte-Carlo Planning (original)
• A Survey of Monte Carlo Tree Search Methods (survey, though a bit old), Multi-armed bandits with episode context (adding empirical knowledge)

Go

• Mastering the game of Go with deep neural networks and tree search (AlphaGo)
• Mastering the game of Go without human knowledge (AlphaGo Zero)
• (for introduction to cnn in go) Training Deep Convolutional Neural Networks to Play Go

Shogi, chess

• A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play (AlphaZero)
• Large-Scale Optimization for Evaluation Functions with Minimax Search (learning from human game records)

DQN and others

• Human-level control through deep reinforcement learning
• promising extension, e.g., Self-Imitation Learning, I2A, or a suggested papers by OpenAI

StarCraft2

• StarCraft II: A New Challenge for Reinforcement Learning (domain)
• Grandmaster level in StarCraft II using multi-agent reinforcement learning (AlphaStar), Relational Deep Reinforcement Learning

Settlers of Catan

• Strategic Dialogue Management via Deep Reinforcement Learning

Card games

• Famous projects: Superhuman AI for multiplayer poker, DeepStack: Expert-level artificial intelligence in heads-up no-limit poker, Superhuman AI for heads-up no-limit poker: Libratus beats top professionals, Heads-up Limit Hold'em Poker is Solved
• CFR: An Introduction to Counterfactual Regret Minimization, Regret Minimization in Games with Incomplete Information, Monte Carlo Sampling for Regret Minimization in Extensive Games

Onto communication between AI and human

• Understanding RL agents: Visualizing and Understanding Atari Agents, Graying the black box: Understanding DQNs
• Interpretable Explanations of Black Boxes by Meaningful Perturbation, LIME
• Opponent Modeling in Deep Reinforcement Learning, Learning of Evaluation Functions to Realize Playing Styles in Shogi

Miscellaneous Skills

• Tools: Ubuntu, C++ and/or python3, OpenAI Gym, TensorFlow or PyTorch, latex+bibtex/biber
• We welcome students who participated in ACM-ICPC.