Optimization and Lagrange Multipliers

Every engineering design hides an optimization: the least material for a fixed volume, the largest area for a fixed length of wall, the cheapest column that still carries the load. The CELE turns these into short calculus problems, and this lesson gives you the two machines that solve them. First the unconstrained machine, where a smooth function is pushed to a peak or a valley by setting its derivative to zero and reading the second derivative to tell peak from valley. Then the constrained machine, Lagrange multipliers, for the far more common case where a side condition ties the variables together. Every rule here is carried all the way to a number with units so you can check your own habits against it.

Unconstrained maxima and minima

For a smooth function of one variable, a local maximum is a peak and a local minimum is a valley, and at either one the tangent line lies flat. That flat tangent is the whole test: an interior extremum can only occur where the first derivative is zero (or fails to exist). Such a point is called a critical number. Solving f'(x) = 0 hands you the candidates; it does not yet tell you which candidate is a peak, which is a valley, and which is neither.

The first-derivative test classifies each critical number from the sign of f' on either side. If f' switches from positive to negative as x increases through the point, the curve stops rising and starts falling, so the point is a local maximum. If f' switches from negative to positive, it is a local minimum. If f' keeps the same sign on both sides, the point is neither, just a momentary flat spot on a still-climbing or still-falling curve.

The second-derivative test

Reading the sign of f' on both sides works but is slow. The second-derivative test settles most problems with a single evaluation. At a critical number c where f'(c) = 0, look at the concavity: if f''(c) > 0 the curve holds water, so c is a local minimum, and if f''(c) < 0 the curve sheds water, so c is a local maximum. Only when f''(c) = 0 does the test go silent, and you fall back on the first-derivative test.

Condition at critical number c	Concavity	Conclusion
f'(c) = 0 and f''(c) > 0	concave up	local minimum
f'(c) = 0 and f''(c) < 0	concave down	local maximum
f'(c) = 0 and f''(c) = 0	undetermined	inconclusive, use the first-derivative test

Worked example: Classify the critical points of f(x) = x^3 - 3x^2 - 9x + 5. The first derivative is f'(x) = 3x^2 - 6x - 9 = 3(x^2 - 2x - 3) = 3(x - 3)(x + 1), which is zero at x = -1 and x = 3. The second derivative is f''(x) = 6x - 6. At x = -1, f''(-1) = 6(-1) - 6 = -12 < 0, so it is a local maximum, and the value there is f(-1) = (-1)^3 - 3(-1)^2 - 9(-1) + 5 = -1 - 3 + 9 + 5 = 10. At x = 3, f''(3) = 6(3) - 6 = 12 > 0, so it is a local minimum, and f(3) = 27 - 27 - 27 + 5 = -22. One cubic, one peak of height 10 and one valley of depth -22, each named by a single second-derivative sign.

Applied maximum and minimum problems

Board problems rarely hand you the function; they hand you a picture and a budget. The reliable routine is always the same. Draw the figure and name the variables. Write the quantity to be optimized (the objective) as a formula. Use the constraint that ties the variables together to eliminate all but one variable, so the objective becomes a function of a single variable. Differentiate, set the derivative to zero, and solve. Confirm it is the maximum or minimum you want (second derivative, or plain reasoning about the physical extremes), then carry the answer to a number with units.

Worked example (maximize an area): A rectangular settling lot is fenced on three sides, with a straight river forming the fourth side, using 120 m of fence. Let x be the side parallel to the river and y each of the two perpendicular sides. The budget is x + 2y = 120, so x = 120 - 2y, and the area is A = x y = (120 - 2y) y = 120y - 2y^2. Then dA/dy = 120 - 4y = 0 gives y = 30 m, and x = 120 - 2(30) = 60 m. Since d^2A/dy^2 = -4 < 0 the critical point is a maximum, and the largest enclosed area is A = (60)(30) = 1800 m^2. Note the optimal shape puts half the fence into the single long side, x = 2y, a signature of the three-sided problem.

Worked example (an open box from a sheet): An open-top box is folded from a 12 cm by 12 cm square sheet by cutting equal squares of side x from the four corners and turning up the sides. The base is then (12 - 2x) by (12 - 2x) and the height is x, so the volume is V = x(12 - 2x)^2. Differentiate with the product rule: V'(x) = (12 - 2x)^2 + x(2)(12 - 2x)(-2) = (12 - 2x)[(12 - 2x) - 4x] = (12 - 2x)(12 - 6x). Setting V' = 0 gives x = 6 (which collapses the base to zero) or x = 2. The meaningful root is x = 2 cm, and the maximum volume is V = 2(12 - 4)^2 = 2(8)^2 = 2(64) = 128 cm^3.

Constrained optimization with Lagrange multipliers

Most real problems keep both variables in play under a side condition, and eliminating one variable can be ugly. Lagrange multipliers avoid the elimination. You optimize f(x, y) subject to a constraint g(x, y) = 0. At the constrained optimum the level curve of f just grazes the constraint curve; the two are tangent, so they share a common normal, and since a gradient always points along the normal to its own curve, grad f and grad g must be parallel. Writing that parallelism with a scalar lambda gives the working system:

grad f = lambda grad g, that is f_x = lambda g_x and f_y = lambda g_y, together with g(x, y) = 0.

That is three equations in three unknowns x, y, and lambda. The fastest route is usually to divide the first equation by the second, which cancels lambda and leaves a clean relation between x and y, then substitute into the constraint.

Worked example (maximize an area under a constraint): Find the largest area A = x y of a rectangle in the first quadrant whose upper-right corner rides on the line x/8 + y/6 = 1. Take g = x/8 + y/6 - 1, so grad A = (y, x) and grad g = (1/8, 1/6). The Lagrange equations are y = lambda(1/8) and x = lambda(1/6). Dividing the second by the first cancels lambda: x/y = (1/6)/(1/8) = 8/6 = 4/3, so x = (4/3) y. Substituting into the constraint, x/8 + y/6 = (4y/3)/8 + y/6 = y/6 + y/6 = y/3 = 1, which gives y = 3 and x = 4. The maximum area is A = (4)(3) = 12 square units. Compare it against the whole triangle the line cuts from the axes, of area (1/2)(8)(6) = 24 square units: the best inscribed rectangle captures exactly half.

The multiplier lambda is not just scaffolding. If the constraint is written g(x, y) = c, then lambda equals the rate of change of the optimal value of f as c is relaxed by one unit, what engineers call the shadow price of the constraint: how much more area one more meter of wall would buy, for instance.

A Lagrange box problem in three variables

The method extends unchanged to three variables: grad f = lambda grad g becomes f_x = lambda g_x, f_y = lambda g_y, f_z = lambda g_z, plus the constraint. The classic case is a box, where you minimize surface area for a fixed volume (or the reverse).

Worked example (least metal for an open tank): An open-top rectangular tank must hold 32 m^3. Using base dimensions x by y and height z, the volume constraint is g = x y z - 32 = 0, and the sheet metal is the base plus four walls, S = x y + 2 x z + 2 y z. The gradients are grad S = (y + 2z, x + 2z, 2x + 2y) and grad g = (y z, x z, x y). The first two Lagrange equations, y + 2z = lambda y z and x + 2z = lambda x z, are symmetric in x and y, which forces x = y (a square base). With x = y the third equation 2x + 2y = lambda x y becomes 4x = lambda x^2, so lambda = 4/x, and the first equation x + 2z = lambda x z = 4z gives x = 2z, that is z = x/2. Now the constraint x^2 z = 32 becomes x^2 (x/2) = x^3/2 = 32, so x^3 = 64 and x = 4 m. Hence y = 4 m and z = 2 m, and the minimum metal is S = (4)(4) + 2(4)(2) + 2(4)(2) = 16 + 16 + 16 = 48 m^2. The optimal open tank is twice as wide as it is tall, and the closed box of least surface for a given volume, by the same symmetry, is a perfect cube.

	Unconstrained problem	Constrained problem (side condition g = 0)
What you solve	f'(x) = 0, or f_x = 0 and f_y = 0	f_x = lambda g_x, f_y = lambda g_y, g = 0
Unknowns	the design variables only	the design variables plus lambda
Fast move	second-derivative test to classify	divide two equations to cancel lambda
Geometry at the optimum	horizontal tangent, f' = 0	level curve of f tangent to g = 0

Exam-day strategy

• Find critical numbers by setting the first derivative to zero (and noting where it fails to exist), then classify each with the sign of f'': positive for a valley (minimum), negative for a peak (maximum); only f'' = 0 sends you back to the slower first-derivative test.
• For an applied max-min problem, always draw the figure first, write the objective, then spend the constraint to reduce the objective to one variable before you differentiate; skipping the drawing is where most sign errors are born.
• Confirm the answer is the extreme you were asked for, not the opposite: check the second derivative or reason from the physical endpoints, since a horizontal tangent alone never proves a maximum.
• Set up Lagrange as grad f = lambda grad g plus the constraint, then divide the first two equations to cancel lambda right away; that usually collapses to a simple relation such as x = y or x = (4/3) y before the constraint is ever touched.
• Exploit symmetry in box problems: if two variables enter the equations identically, they are equal at the optimum, which turns a three-variable system into a one-variable one (square base, cube, and so on).
• Read the multiplier lambda as a shadow price when a problem asks how the best value moves with the budget: with g = c, lambda is the change in the optimal f per unit change in c.
• On a closed, bounded interval or region, the absolute extreme may hide on the boundary, so compare the interior critical values against both endpoints (or the whole edge) before you commit to an answer.