Entropy

Given a random variable $X$ with probability distribution $P$, its entropy is given by

$$H(X)\equiv -{\text{sumlim}}_{x}P(x)\log P(x)$$

Due to the Mutual Information definition, one can say that the entropy is a measure of self-information of $X$.

---

References