Theorem. (Taylor) Every functional in has the representation
for some between and , where is the gradientvector evaluated at , and is the Hessian matrix of at , i.e.,
Proof. See Goldstein [Goldstein 1967] p. 119. The Taylor infinite series is treated in Williamson and Crowell [Williamson et al. 1972]. The present form is typically more useful for computing bounds on the error incurred by neglecting higher order terms in the Taylor expansion.