京都大学 京都大学

Mathematical theory of information and communications

Numbering Code U-ENG29 39142 LJ55
U-ENG29 39142 LJ72
U-ENG29 39142 LJ10 		Year/Term 2022 ・ Second semester
Number of Credits 2 Course Type Lecture
Target Year Target Student
Language Japanese Day/Period Tue.3
Instructor name OBUCHI 　TOMOYUKI (Graduate School of Informatics Associate Professor)
HONDA JUNYA (Graduate School of Informatics Associate Professor)
Outline and Purpose of the Course Lectures discuss information theory, a basic theory related to storing and transmission of information. While referring to contents of the course “Information and Coding Theory,” lectures take up topics such as entropy of continuous-valued random variables, Gaussian communication channels, rate-distortion theory, universal coding, etc. More advanced topics are also introduced, including network information theory and more.
Course Goals Our goal is to gain an understanding that enables appropriate responses to questions and issues regarding examples introduced during lectures, topics set for written reports, etc.
Schedule and Contents Introduction (1 class)
Confirmation of basic concepts, including information entropy, mutual information, source coding, channel coding, etc.

Information theory of continuous-valued random variables (3 classes)
When considering wireless communications and measurements, a theory is needed for random variables which take continuous values. The argument will proceed by introducing differential entropy for continuous random variables, and by taking up concrete examples from Gaussian communication channels, with discussion of the information transmission capabilities of such channels.

Rate-distortion theory (4 classes)
Toleration of a certain extent of information degradation enables more efficient data compression than when no degradation is permitted. Lectures focus on rate-distortion theory, the theory underpinning information compression with degradation toleration.

Mid-term review of learning achievement (1 class)
In order to check the level of students' learning achievement, we will ask them to write some reports and give them feedback through questions and explanations about the reports.

Information theory and statistics (4 classes)
Type theory is introduced so as to discuss universal information compression, large-deviations theory, hypothesis testing, and other applications.

Network information theory (1 class)
Thanks to the development and spread of information and communications technologies, one-to-one information exchanges have been superseded by many-to-many information exchanges. There is a growing need, then, for discussions regarding these changes. Lectures will focus on fundamental network information theory, necessary for proceeding with such discussions.

Final review of learning achievement (1 class)
In order to check the level of students' learning achievement, we will ask them to solve problems/questions etc., related to the course, and further advice will be provided.
Evaluation Methods and Policy Grading is performed both on the basis of reports submitted when necessary during the term and the final exam.
Course Requirements Prerequisites are knowledge of basic probability theory, and knowledge regarding the course “Information and Coding Theory.” Knowledge of statistics and Markov chains is also desirable.
Study outside of Class (preparation and review) Since a prerequisite of this class is the course ``Information and Coding Theory,'' an appropriate review of that course's contents is recommended prior to attendance. Assigned pages in the course textbook should be read before each lecture. A good way to review each class is to do the problems at the end of assigned chapters.
Textbooks Textbooks/References Elements of Information Theory, 2nd ed., T. M. Cover and J. A. Thomas, (Wiley-Interscience), ISBN:9780471241959, The e-book version can be accessed from within the university. A Japanese translation is also available from Kyoritsu Shuppan Publishing Co.
PAGE TOP