Home | Outline | Resources | Handouts and Notes | Assessment
Communication and Information Engineering Department
Zewail City of Science and Technology

CIE425: Information Theory and Coding

Lecturer: Ziad A. El-Sahn

Zewail City



This course covers fundamentals of information theory and coding, including entropy, average mutual information, channel capacity, block codes and convolutional codes. Topics also include asymptotic equipartition property, entropy rates of a stochastic process, data compression, channel capacity, differential entropy, and the Gaussian channel.

- Course File Updated as of February 14, 2017 [PDF]

Teaching Load (Spring 2017)

- Lectures (2 hours), Tutorials (2 hours)
- Office Hours: Tuesday from 10:40 am to 11:40 am, and 2:10 pm to 3:30 pm - Room 373
- Teaching Assistants: Eng. Aya Taha and Eng. Nourhan Abdel-Ghaffar


1- Basics and Fundamentals of Information Theory

- Introduction to Information Theory
Entropy, Relative Entropy, Mutual Information
- Asymptotic Equipartition Property
- Entropy Rates of a Stochastic Process

2- Source Coding and Data Compression

3- Channel Capacity Theorems

4- Channel Coding Theorems and Algorithms

5- Related Topics and Applications

- Rate Distortion Theory
- Basics of Network Information Theory
- Information Theory-Related Applications


- Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, 2nd Edition, John Wiley & Sons, Inc., 2006.
- Richard E. Blahut, Principles and Practice of Information Theory, Addison-Wesley, 1987.
- Robert G. Gallager, Information Theory and Reliable Communication, John Wiley & Sons, Inc., 1968.
- Robert M. Gray, Entropy and Information Theory, 2nd Edition, Springer, 2011.

Useful Links:

- Claude Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, 1948. [PDF]
- IEEE Transactions on Information Theory [URL]


Lecture 1: Introduction to Information Theory [PDF]
Lecture 2: Entropy, Relative Entropy, Mutual Information [PDF]
Lecture 3: Asymptotic Equipartition Property [PDF]
Lecture 4: Entropy Rates of a Stochastic Process [PDF]
Lectures 5, 6: Principles of Source Coding and Data Compression [PDF]
Lecture 7: Channel Capacity Theorems [PDF]
Lecture 8: Continuous Sources and Capacity of AWGN Channels [PDF]
Lectures 9, 10: Channel Coding Theorem and Examples of Channel Codes [PDF]
Lecture 11: Rate Distortion Theory [PDF]
Lecture 12: Advanced Topics and Applications of Information Theory [PDF]
Lecture 13: Projects - MIMO Channel Capacity [PDF], Molecular Biology [PDF]

* Thanks to Dr. Bassem Mokhtar for providing part of the course materials

Research Project: Click here for detailed information and important deadlines...


Distribution of a total mark of 100:

- Mid-Term Exams (30 marks): 60 minutes, closed book (2 equal weight exams)
- Class Work and Projects (40 marks)
- Final Exam (30 marks): 2 hours, closed book exam


© Copyright 2005-2017, Ziad A. El-Sahn  |  Last updated: May 16, 2017