LessWrong also has a textbook repository which contain recommendation on various subjects. Ctrl+F "decision theory" turns up 6-7 books or so.
There's actually only one comment on decision theory textbooks, which recommends the Martin Peterson's one. I skimmed through it at our library and it seems to be more directed at philosophy students and those with poor mathematical background. There's also Resnik or Luce & Raiffa mentioned but they are claimed to be out of date.
While it's always important to grasp the intuition, reading books for the "common people" will give you much slower progress than if you read a book meant for someone who already has solid mathematical foundations.
An Introduction to Decision Theory by Martin Peterson is a good first introduction to the fundamental principles of decision theory as well as the strengths and weaknesses of causal vs evidential decision theory (it doesn't get into the more exotic theories, although it does contain a chapter on social choice).
It also contains exercises to ensure basic mastery of the concepts, but only assumes a background of algebra and probability.
https://www.reddit.com/r/DecisionTheory/search?q=flair%3ATextbook&restrict_sr=on is a starting point.
Try looking around this post (ctrl+f) on The Best Textbooks on Every Subject - and see if there's anything you'll find useful :)
I noticed that textbooks on mathematical statistics sometimes contain chapters about decision theory, e.g. Theory of Statistics by Schervisch contains about 60 pages on Bayesian and classical decision theory.
Eliezer talks a lot about decision theory in his sequences, e.g. the Aumann agreement theorem or the von Neumann-Morgenstern utility theorem. From what I've seen so far, decision theory looks extremely interesting.
Which textbook in decision theory would you recommend to start with? I'd appreciate if the book contained not only theory, but also some exercise/problem section - I have noticed that usually a lecture is not enough to fully grasp a topic. I want a book which will not shy away from mathematical side of the theory.
I have a strong background in mathematics and computer science, but I only know a little about game theory.