Lagrange Multiplier

Lagrange multipliers is a way to solve constrained optimization problems with a form like this:

Minimize (or maximize) constrained by

It does that by finding critical point at for some and (the variable is called the multiplier because it multiple gradients)

The Lagrange multipliers doesn’t tell whether a solution is a maximum, minimum, or a saddle point, but we can use the second derivative test.

Intuition

Lagrange multiplier is behind the observation that at maximum or minimum, the level set of function is tangent to the constraint . In other words, their gradients are parallel: , and thus .

Proof

Suppose that has a maximum at on the constraint surface.

Let be an arbitrary parametrized curve which lies on the constraint surface and has . Also let . This setup guarantees that has a maximum at .

Taking a derivative using the chain rule gives

Since is a local maximum, we have

Thus, is perpendicular to any curve on the constraint surface through , which implies that is perpendicular to the constraint surface.

However, since is also perpendicular to the surface, and must be parallel to each other.