Overview
One of the earliest analytic methods to find an maximum or minimum was developed with the advent of calculus. Calculus provides the toolset necessary to calculate the tangent to a function at any point. (see derivative)
For smooth differentiable functions, the max or min must occur at a point where the derivative is zero. (a derivative equal to zero is a necssary condition, but is not sufficient.)
{% \frac{df(x)}{dx} = 0 %}
To determine if the points found using this method represent a minimum or a maximum generally requires looking at the second
derivative of the function. If the second derivative is positive, it is a local mimimum. If the second derivative is
negative, it is a maximum. If the second derivative is zero, then the test is inconclusive, and the point may not even
be a local optimum.
Minimum and maximum found with this method are local in nature, although they could also be the global optimum as well.
Demo
The graph shows the tangent line to the parabola. The slider moves the point of tangency. The minimum point is where the tangent is flat, in other words, the derivative is equal to zero.
Multivariable
The case where the function is question is a function of multiple inputs is a simple extension of the single variable case. For a multivariable function, each partial derivative (with respect to each input) must equal zero, whcih can be stated succintly as
{% \nabla f(x_1,...,x_n) = \vec{0} %}
(see gradient)