An Introduction to Information Theory
An Introduction to Information Theory. Instructor: Prof. Adrish Banerjee, Department of Electrical Engineering, IIT Kanpur. Information Theory answers two fundamental questions: what is the maximum data rate at which we can transmit over a communication link, and what is the fundamental limit of data compression. In this course we will explore answers to these two questions. We will study some practical source compression algorithms. We will also study how to compute channel capacity of simple channels. (from nptel.ac.in)
| Lecture 14 - Coding of Sources with Memory |
In this lecture we first describe a discrete stationary source (DSS) with memory. Then we define uncertainty per letter of output sequences in two different ways and prove some of their properties. We also define the entropy rate of a DSS. Then we prove lower bounds on the output codeword length for block to variable length coding of DSS. We describe Elias coding for positive numbers and use them to design block to variable length codes (Elias Willems source coding) for DSS with memory.
Go to the Course Home or watch other lectures: