1  Differentiability

f is differentiable at x in the interior of the domain if all partial derivatives exist:

D_v f(x) := \lim_{t \to 0} \frac{f(x+ tv) - f(x)}{t}

and this limit is a linear function of v

Also, if f is differentiable at x and an inner product is defined then the gradient is uniquely defined by:

\forall v, \langle \nabla f(x), v \rangle = \lim_{t\to 0} \frac{f(x+tv) - f(x)}{t}= : D_v f(x)

For standard dot product, we can recover (\nabla f(x))_i = \partial_{x_i} f(x)

Similarly if f is twice differentiable at x, the Hessian is uniquely defined by

\forall u, v \in \mathbb R^n : \langle u, \nabla^2 f(x) v \rangle = D_u D_v f(x)

which recovers (\nabla^2 f(x))_{ij} = \partial_{x_i}\partial_{x_j} f(x)

The first and second order approximations are:

l_x(y) := f(x) + \langle \nabla f(x), y-x \rangle

q_x(y) := f(x) + \langle \nabla f(x), y-x \rangle+ \frac{1}{2} \langle (y-x), \nabla^2 f(x) (y-x) \rangle