Statistical Learning Theory

Numbering Code G-LAS12 80010 LE10 Year/Term 2021 ・ First semester
Number of Credits 2 Course Type Lecture
Target Year Graduate students Target Student For science students
Language English Day/Period Mon.1
Instructor name KASHIMA HISASHI (Graduate School of Informatics Professor)
Outline and Purpose of the Course This course will cover in a broad sense the fundamental theoretical aspects and applicative possibilities of statistical machine learning, which is now a fundamental block of statistical data analysis and data mining. This course will focus first on the supervised and unsupervised learning problems, including a survey of probably approximately correct learning, Bayesian learning as well as other learning theory frameworks. Following this introduction, several
probabilistic models and prediction algorithms, such as the logistic regression, perceptron, and support vector machine will be introduced.
Advanced topic such as online learning, structured prediction, and sparse modeling will be also introduced.
Course Goals Understanding basic concepts, problems, and techniques of statistical learning and some of the recent topics.
Schedule and Contents 1. Statistical Learning Theory
1-1. Introduction to classification & regression: historical perspective, separating hyperplanes and major algorithms
1-2. Probabilistic framework of classification and statistical learning theory: Learning Bounds, Vapnik-Chervonenkis theory

2. Supervised Learning
2-1 Models for Classification: Logistic Regression, Perceptron, Support Vector Machines
2-2 Regularization: Sparse Models (L1 regularization), Bayesian Modeling
2-3 Model Selection: Performance Measures, Cross-Validation, and Other Information Criterion

3. Advanced topics
3-1 Structured Prediction: Conditional Random Fields, Structured SVM
3-2 Online learning
3-3 Semi-supervised, Active, and Transfer Learning
Evaluation Methods and Policy Reports and final exam.
Course Requirements None
Study outside of Class (preparation and review) Basic knowledge about probability and statistics
References, etc. The Elements of Statistical Learning, Hastie, Friedman, Tibshirani, (Springer)
Understanding Machine Learning: From Theory to Algorithms, Shai Shalev-Shwartz and Shai Ben-David, (Cambridge University Press)