Book recommendations

I often get asked how I learned the AI stuff I'm showing in my videos, so I thought a post about that might interest some people here :)

In short, I started coding with the Python programming language during my studies. I then took some courses on Machine-Learning (ML) and decided to explore the topic further on my own. That's when I first started making videos about training an AI in Trackmania. Over time, I became particularly interested in a subfield of ML called Reinforcement Learning (RL), which is especially well suited for training AI agents to play games.

There are many great free online ressources to learn all this, and I'll probably make another post to share some of the ones I found most useful. But personnaly, I like having everything in one place when I learn about a topic, and I think books are great for that. So here are two books that I found particularly useful to learn about ML and RL! (And of course, this isn't a sponsored post ahah)

1) Hands-on Machine Learning with Scikit-Learn, Keras & Tensorflow, by Aurélien Géron
This book is super useful for learning ML, whether you're a beginner or already have some experience. It covers all the key concepts (data processing and visualisation, metrics, overfitting, ...), and dives into different ML approaches (Supervised Learning, Unsupervised Learning, Reinforcement Learning).
It starts with basic techniques (regression, decision trees, etc.) and progressively moves to more advanced topics (neural networks, deep learning). It's particulalrly useful if you're coding with Python (which is what most people in ML use), as it provides code examples with Python libraries Scikit-Learn, Keras & Tensorflow. That said, since the book was published several years ago and ML evolves quickly, it might not cover some of the latest developments (like Large Language Models). Also, its RL section is quite basic, which is why I later got the second book on this list. (On a side note, from my experience, most RL projects today use PyTorch instead of TensorFlow, but that doesn’t take away from the book’s value.)

2) Reinforcement Learning: An Introduction (second edtion) - by Andrew Barto & Richard Sutton
This book was originally published in 1992 and covers all the fundamental RL concepts (Markov decision processes, dynamic programming, Monte Carlo methods, temporal-difference learning, off-policy vs on-policy learning, Q-learning vs policy-gradient vs actor-critic methods, etc.). It was later re-edited in a second edition which introduces deep RL (i.e. RL combined with deep neural networks). Personnaly, I learned more about RL with online ressources and personnal projects. From my experience, RL is harder to grasp than other ML fields like Supervised Learning—it’s very empirical, and many parts of this book felt quite theoretical when applied to real projects. This is especially true today since most RL implementations focus on deep RL. Still, this book does an excellent job explaining the key concepts that form the foundation of modern deep RL methods.

Bonus) The magazine in the middle of the picture
Good luck to find this ahah, it's an obscure french Trackmania magazine I got in 2008, the year Trackmania Nations Forever was released! It didn’t teach me anything about ML or RL, but it’s one of the many things that fueled my passion for the game. And these Trackmania AI videos would probably not have happend without that passion :)

I hope this post will be helpful to some of you, let me know if you'd like another post with some recommandations of online resources (videos, blog posts, research articles) etc. as well!