Numbering Code	G-INF05 63536 LJ10 G-INF05 63536 LJ54	Year/Term	2022 ・ First semester
Number of Credits	2	Course Type	Lecture
Target Year		Target Student
Language	Japanese	Day/Period	Wed.1
Instructor name	Shimodaira, Hidetoshi (Graduate School of Informatics Professor) HONDA JUNYA (Graduate School of Informatics Associate Professor)
Outline and Purpose of the Course	We will explain the statistical methods for making inferences, predictions, and decisions from data through probabilistic models, as well as their mathematical aspects. In particular, the first half (Shimodaira) deals with model selection and resampling methods based on the information criterion, and the second half (Honda) deals with dynamic decision-making methods based on the Bandit algorithm.
Course Goals	・ You will learn how to apply statistical scientific methods to new applied problems. ・ You will acquire the basic ability to develop new statistical science methods.
Schedule and Contents	1. Linear regression model, least squares method, probability model and maximum likelihood method, likelihood principle, model inclusion relationship 2. Likelihood ratio test, Akaike's information criterion AIC, entropy, Kullback-Leibler information 3. Geometric image, optimal parameters and projection, expansion of KL information, Pythagorean theorem, MLE and projection, consistency 4. Derivation of maximum likelihood estimator asymptotic normality, Fisher information matrix, prediction distribution, loss, risk 5. Derivation of Information Criterion TIC, Derivation of AIC 6. Cross-validation, Bayesian Information Criterion 7. AIC variability, bootstrap, model selection test, multiple comparisons, model selection simulation and bootstrap probabilities, multiscale bootstrap 8, 9. Large deviation principle. Cramer's theorem, Sanov's theorem, interpretation of KL information 10, 11. Bandit problem. UCB algorithm and Thompson sampling, lower and upper bounds of regret 12. Best-arm identification. Fixed confidence and fixed budgeting, sample complexity 13, 14. Bayesian optimization. Gaussian process, GP-UCB / Thompson sampling regret upper bound 15. Discussion
Evaluation Methods and Policy	Mainly year-end reports. Attendance and homework may be taken into account.
Course Requirements	None in particular
Study outside of Class (preparation and review)	In addition to merely learning through lectures, we will ask students to attempt actual data analysis.
Textbooks	Textbooks/References	No textbook will be used. We will distribute materials as necessary.
References, etc.	Hidetoshi Shimodaira "モデル選択 予測・検定・推定の交差点 (統計科学のフロンティア 3)" (Iwanami Shoten) ISBN: 4000068431: Based on this lecture Yasutaka Shimizu, "統計学への確率論、その先へ―ゼロからの測度論的理解と漸近理論への架け橋" (Otsuru Uchida) ISBN: 4753601250: Slightly advanced. The order notation required in the statistical asymptote theory is also explained. Hidetoshi Matsui, Kazuyuki Koizumi "Statistical Model and Guess (Introduction to Data Science Series)" (Kodansha) ISBN: 4065178029: Good for summarizing and confirming basic matters Konishi / Kitagawa "統計モデルと推測 (データサイエンス入門シリーズ)" (Asakura Shoten) ISBN: 4254127820: This is a good book, but be careful when writing your reports because the derivation flow and symbols used here are different from those used in this class. Akaike, Amari, Kitagawa, Kabashima, & Shimodaira "赤池情報量規準AIC―モデリング・予測・知識発見" (Kyoritsu Shuppan) ISBN: 4320121902: May be helpful for understanding how to think about these concepts Junya Honda, Atsushi Nakamura "バンディット問題の理論とアルゴリズム" (Kodansha Scientific) ISBN: 9784061529175: Reference text for the second half of this class