Mathematics for Machine Learning by Marc Deisenroth and Deep Learning by Ian Goodfellow. Two great books for theory but practice always helps too.
I also recommend Deep Learning by Goodfellow. Long chapter about the mathematical foundations. I found it very insightful as side lecture during my studies. Also, the online version is free.
Yes! This is an amazing book! It has all the neural models in great detail, considering the detail it goes into it's also very readable.
Pattern Recognition and Machine Learning from Bishop is really good, imho. Its relatively math-heavy, so depending on your skill, reading Mathmatics for ML or Linear algebra and optimization for ML by c. aggarwal might be a good idea.
How is that Aggarwal book? I love Bishop and MML. Thinking about picking it up too.
I enjoyed it. Linear algebra and optimization are treated much more in depth compared to MML. IIRC he then goes to linear regression and derives most other models from there, which is an interesting perspective.