InfoCoBuild

Information Theory and Coding

Information Theory and Coding. Instructor: Prof. S. N. Merchant, Department of Electrical Engineering, IIT Bombay. This course covers lessons on information theory and coding, entropy, block code and its properties, Shannon's theorem, huffman coding, Shannon-Fano-Elias coding, arithmetic coding, information channels, rate-distortion theory, Lloyd-Max Quantizer, vector quantization, and transform coding. (from nptel.ac.in)

Lecture 21 - Equivocation and Mutual Information

Go to the Course Home or watch other lectures:

Lecture 01 - Introduction to Information Theory and Coding

Lecture 02 - Definition of Information Measure and Entropy

Lecture 03 - Extension of an Information Source and Markov Source

Lecture 04 - Adjoint of an Information Source, Joint and Conditional Information Measure

Lecture 05 - Properties of Joint and Conditional Information Measures and a Markov Source

Lecture 06 - Asymptotic Properties of Entropy and Problem Solving in Entropy

Lecture 07 - Block Code and its Properties

Lecture 08 - Instantaneous Code and its Properties

Lecture 09 - Kraft-Mcmillan Equality and Compact Codes

Lecture 10 - Shannon's First Theorem

Lecture 11 - Coding Strategies and Introduction to Huffman Coding

Lecture 12 - Huffman Coding and Proof of its Optimality

Lecture 13 - Competitive Optimality of the Shannon Code

Lecture 14 - Non-Binary Huffman Code and Other Codes

Lecture 15 - Adaptive Huffman Coding, Part I

Lecture 16 - Adaptive Huffman Coding, Part II

Lecture 17 - Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding

Lecture 18 - Arithmetic Coding, Part I

Lecture 19 - Arithmetic Coding, Part II

Lecture 20 - Introduction to Information Channels

Lecture 21 - Equivocation and Mutual Information

Lecture 22 - Properties of Different Information Channels

Lecture 23 - Reduction of Information Channels

Lecture 24 - Properties of Mutual Information and Introduction to Channel Capacity

Lecture 25 - Calculation of Channel Capacity for Different Information Channels

Lecture 26 - Shannon's Second Theorem

Lecture 27 - Discussion on Error Free Communication over Noisy Channel

Lecture 28 - Error Free Communication over a Binary Symmetric Channel and Introduction to Continuous Sources and Channels

Lecture 29 - Differential Entropy and Evaluation of Mutual Information for Continuous Sources and Channels

Lecture 30 - Channel Capacity of a Bandlimited Continuous Channel

Lecture 31 - Introduction to Rate-Distortion Theory

Lecture 32 - Definition and Properties of Rate-Distortion Functions

Lecture 33 - Calculation of Rate-Distortion Functions

Lecture 34 - Computational Approach for Calculation of Rate-Distortion Functions

Lecture 35 - Introduction to Quantization

Lecture 36 - Lloyd-Max Quantizer

Lecture 37 - Companded Quantization

Lecture 38 - Variable Length Coding and Problem Solving in Quantizer Design

Lecture 39 - Vector Quantization

Lecture 40 - Transform Coding, Part I

Lecture 41 - Transform Coding, Part II