Module 4 of 5 · Lesson 1 of 2
Partial Derivatives, the Gradient and Critical Points
Partial derivatives, the gradient, and the saddle that one variable cannot produce.
What you will be able to do
Given a function of two variables, the learner can compute its partial derivatives, assemble the gradient, evaluate a directional derivative along a normalised direction, identify the direction of steepest increase and the level-curve direction, and locate and classify critical points.
Orientation
A surface has no single slope
Every derivative so far answered one question: how fast does
Standing on a hillside, walking east might climb steeply, walking north gently, walking north-east somewhere between, and walking along the contour not at all. All four are rates of change of the same function at the same point. Asking for "the" slope has no answer.
The resolution is to fix a direction first. The partial derivatives answer for the two coordinate directions, differentiate in
What is new is that those two numbers determine all the others. For
The gradient also answers the geometric questions at once: it points the steepest way up, its length
One genuinely new phenomenon appears. In one variable a vanishing derivative leaves a maximum, a minimum or an inflection. In two, a point can rise along one axis and fall along another: a saddle, with no single-variable counterpart, and the reason a matrix rather than a single number decides the classification.
Definition
Why each definition is shaped as it is
The canonical definition states the partials, the gradient and the directional derivative. What follows is why each carries its particular conditions.
Why
Why the directional derivative needs a unit vector.
Why the gradient points steepest uphill. From
| meaning | ||
|---|---|---|
| steepest increase | ||
| along a level curve | ||
| steepest decrease |
None of this is extra geometry. It is the cosine, read off. The gradient's length being the maximum rate is the same statement.
Why Clairaut's theorem needs continuity. The proof compares two ways of taking a second difference over a small rectangle, and shows both tend to the same limit. Continuity of the mixed partials is what makes those limits agree. Without it the two orders can genuinely differ at a point, which is why the theorem is stated with a hypothesis rather than as an identity.
Why partial derivatives existing is weaker than differentiability. In one variable,
Why the Hessian rather than one second derivative. A critical point needs classifying against every direction. The second-order behaviour along direction
Intuition
Two numbers, and what they do not determine
The gradient's claim is that two numbers settle every directional rate at a point. Worth testing how far that goes.
What it does determine. At
What it does not determine: whether the function is differentiable at all. Both partials probe exactly two lines through the point. A function can be well-behaved along both axes and discontinuous along every other direction, so
What it does not determine: the classification of a critical point.
The pattern. Each step upward in dimension costs a condition that was free before. One variable: the derivative existing gives continuity, and one second derivative classifies. Two: neither holds, and both repairs, differentiability as a genuine linear approximation, definiteness as a statement over all directions, are demands quantified over directions rather than checks at a point.
Representation
A function as a surface and as a gradient field
Take
Which space each object lives in. For
| Question | Surface answers | Gradient answers |
|---|---|---|
| rate along | measure a slope on the drawing | |
| steepest direction | eyeball the contours | along |
| that steepest rate | estimate | |
| zero-change direction | tangent to the contour | perpendicular: |
| is this a critical point? | is the surface level? | is the vector zero? |
Every row above reads at a regular point, where
Why the steepest direction is the gradient. For a unit vector
What each form is for. The gradient computes: every directional rate is a dot product, and optimisation steps along
That last point is why both are kept.
Theorem
The gradient's direction, and the second-derivative test
Theorem (what the gradient maximises). Let
- its maximum
when points along ; - its minimum
when points along ; - the value 0 exactly when
.
Proof. For a unit
The proof is one line because the geometry is entirely in the dot product. Note where
Verification. For
Corollary (level curves). The gradient is perpendicular to the level curve through the point. Moving along a contour,
Theorem (second-derivative test). Let
| Verdict | |
|---|---|
| local minimum | |
| local maximum | |
| saddle point | |
| test is inconclusive |
Why
Verification on three cases.
| critical point | verdict | |||
|---|---|---|---|---|
| minimum, value | ||||
| maximum, value | ||||
| saddle |
For the first, 20,000 random points within
Why
Worked example
Partial derivatives, a directional derivative, and two critical points
1. Partial derivatives of
For
For
Check numerically: symmetric difference quotients at
So
2. A directional derivative along
The vector
Check:
Skipping the normalisation would give
3. Steepest increase and the level direction.
The steepest increase is along
Normalising, the direction is
Perpendicular to that,
4. Classifying critical points of
Set both partials to zero:
The only critical point is
Second derivatives:
and
Check: 20,000 random points within
5. A saddle:
which makes it a saddle. The behaviour is visible directly: along the
Procedure
Working with partial derivatives and the gradient
To compute a partial derivative. Decide which variable is live, treat every other as a constant, and differentiate by the ordinary single-variable rules. A term containing none of the live variable differentiates to 0. Nothing from the derivative unit changes. The only new discipline is keeping track of which symbol is currently a number.
To compute a directional derivative.
- Compute
at the point. - Normalise the direction:
. Skip this and the answer is scaled by . - Take the dot product
.
Check: the result must lie in
To answer the geometric questions. With
| Question | Answer |
|---|---|
| steepest increase | along |
| steepest decrease | along |
| no change | perpendicular to |
To find and classify critical points.
- Solve
and simultaneously. Both must vanish at once; a point where only one does is not critical. - Compute
, , at each solution. - Form
. - Read the verdict:
with is a minimum, with a maximum, a saddle, and undecided, use another argument.
Check: evaluate
Confirming any partial numerically. A symmetric difference quotient in one variable, holding the other fixed, at
Example
Gradients and critical points worth recognising
A plane,
A paraboloid,
A shifted bowl,
An inverted bowl,
The saddle,
A product,
A double integral,
The gradient vanishes at three of these and never at the other two, and where it vanishes the determinant
Non-example
Errors the extra dimension invites
Dotting with an unnormalised direction. For
The check that catches it: a directional derivative can never exceed
Treating one vanishing partial as a critical point. For
Reading a vanishing gradient as an extremum. At the origin,
Classifying with a single second derivative. For that same saddle,
Forgetting that
Assuming partials existing makes
Expecting a partial derivative to lose the other variable. For
Application
Following the gradient
Gradient descent. Every model fitted by minimising a loss follows
The algorithm halts where
Least squares. Fitting
Maximum likelihood. The log-likelihood of a model with several parameters is a surface, and the estimator is where its gradient vanishes. The Hessian at that point becomes the observed Fisher information, whose inverse estimates the estimator's covariance, so the second derivatives that classify the critical point also quantify the uncertainty.
Thermodynamics and physics. Heat flows along
Contour maps in the field. A hiker reading a map uses the perpendicularity directly: the steepest path crosses contours at right angles, and a path along a contour neither climbs nor descends. Tightly packed contours warn of steep ground, which is
The through-line. In one variable a derivative answers "how fast". In several it must first answer "which way", and the gradient packages both: a direction to move and a rate for moving that way. Every application above is one of those two readings put to work.