21
Information Gain
The
information gain of an attribute A is the
expected
reduction
in entropy caused by partitioning on this attribute.
Gain(S,A) =
Entropy(S) - å
----- Entropy(Sv)
v
Î Values(A)
|Sv|
|S|
where
Sv is the subset of S for which attribute A has
value
v.
Choose the attribute A that gives the maximum
information gain.