Optimal Step Size

Updated 15 Sep 2026

•
What it is
- •
  the step size $h$ that minimizes the total error in numerical differentiation, balancing two competing effects: truncation error (which grows as $h$ increases) against rounding error (which grows as $h$ shrinks, since $\epsilon/h$ blows up)
•
Forward difference
- •
  $h_{opt} = 2\sqrt{\epsilon/M}$
•
Worked example: $f(x)=\sin(x)$ , forward difference, double precision ( $\epsilon_{mach}\approx1.11\times10^{-16}$ )
- •
  $M = |\sin(1.0)| \approx 0.8415$
- •
  $h_{opt} \approx 2.30\times10^{-8}$ , giving a minimum achievable error of about $1.93\times10^{-8}$
•
Key insight
- •
  central difference's better $O(h^2)$ truncation order means its achievable minimum error is dramatically smaller than forward difference's, even though its optimal $h$ formula looks more complex
•
Related
- •
  Numerical Differentiation
- •
  Machine Epsilon
- •

Linked references 2