Entropy

Given a random variable with probability distribution , its entropy is given by

Due to the Mutual Information definition, one can say that the entropy is a measure of self-information of .

---

References