Skip to content

Information Gain

Updated 02 Oct 2026

•
What it is
- •
  the reduction in target entropy produced by a split
•
Formula
- •
  $IG(Y,X)=H(Y)-\sum_vP(X=v)H(Y\mid X=v)$ .
•
Gain ratio
- •
  $\mathrm{GainRatio}=IG/\mathrm{SplitInfo}$ , when the denominator is nonzero.
- •
  $\mathrm{SplitInfo}=-\sum_vP(X=v)\log_2P(X=v)$ .
- •
  Gain ratio addresses information gain’s preference for attributes that create many branches.
•
Example
- •
  Using the supplied fourteen-row dataset: Outlook $\approx0.258521$ , Temperature $\approx0.183851$ , Humidity $\approx0.370507$ , Windy $\approx0.005978$ .
- •
  Humidity has the highest information gain for that dataset.
•
Related
- •
  Decision Tree
- •
  Unavailable reference

Linked references 2

◌ Explore connections in Graph view