gradient

Given a multivariate function , the gradient is a vector field that assigns a vector at each point:

The operation is call “Del” and can be written as \nabla symbol in Latex.

Intuitively, gradient measures the direction of “fastest increase” through a vector field.

Gradient in Coordinates

The most familiar definition is a list of partial derivative:

We can also understand it as a list of directional derivative across the coordinate axis.

This definition has two potential problems:

• Role of inner product is not clear
• No way to differentiate functions of functions F(f) since we don’t have a finite list of coordinates

Gradient As Fastest Increase

For an function , the gradient at a point represents the direction of steepest increase of the function at that point. It also plays a crucial role in the linear approximation of the function:

This interpretation of gradient leads naturally to the gradient descent algorithm, a fundamental optimization technique.

Gradient For Implicit Surface

For an implicitly defined surfaces , the gradient will always be perpendicular (normals) to the level set and is directed towards the direction of maximum increase in value.

Relation with Multi-dimensional Derivative and Gradient

See: Multi-dimensional derivative, gradient, and directional Derivative