Informação Mútua I(X;Y)

Mutual Information, denoted as I(X;Y), is a fundamental measure in information theory that quantifies the amount of information one random variable contains about another. It is calculated as the difference between the entropy of variable X, denoted as H(X), and the conditional entropy of X given Y, denoted as H(X|Y). This provides a way to understand how knowledge of one variable reduces the uncertainty about the other.

The formula for Mutual Information is I(X;Y) = H(X) − H(X|Y). This means that if knowing Y significantly reduces the uncertainty about X, then I(X;Y) will be large. Conversely, if knowing Y does not significantly alter the uncertainty about X, then I(X;Y) will be close to zero. This measure is symmetric, i.e., I(X;Y) = I(Y;X), reflecting the idea that the information X contains about Y is the same as Y contains about X.

Mutual Information is used in various applications, such as analyzing dependency between variables, signal processing, pattern recognition, and machine learning. It helps identify complex relationships between variables, going beyond simple linear correlations. However, caution is needed in interpretation when variables have non-linear relationships or when there are variables with complex probability distributions.

It's crucial to note that Mutual Information does not establish causality between variables; it only quantifies informational association. Therefore, it should be used judiciously, understanding the limits of its interpretation in the specific context of application.