THE MATHEMATICS OF

OPTIMIZATION

1
The Mathematics of Optimization
• Many economic theories begin with the
assumption that an economic agent is
seeking to find the optimal value of some
function
– consumers seek to maximize utility
– firms seek to maximize profit
• This chapter introduces the mathematics
common to these problems
2
Maximization of a Function of
One Variable
• Simple example: Manager of a firm
wishes to maximize profits
  f (q)

Maximum profits of
* * occur at q*
 = f(q)

Quantity
q*
3
 Maximization of a Function of
One Variable
• The manager will likely try to vary q to see
where the maximum profit occurs
– an increase from q1 to q2 leads to a rise in 


0
*

q
2  = f(q)

1

Quantity
q1 q2 q*
4
 Maximization of a Function of
One Variable
• If output is increased beyond q*, profit will
decline
– an increase from q* to q3 leads to a drop in 



0
*

 = f(q)
q
3

Quantity
q* q3
5
 Derivatives
• The derivative of  = f(q) is the limit of
/q for very small changes in q
d df f (q1  h)  f (q1 )
  lim
dq dq h  0 h

• The value of this ratio depends on the

value of q1

6
Value of a Derivative at a Point
• The evaluation of the derivative at the
point q = q1 can be denoted
d
dq q  q1

• In our previous example,

d d d
0 0 0
dq q q
1
dq q q
3
dq q  q *
7
 First Order Condition for a
Maximum
• For a function of one variable to attain
its maximum value at some point, the
derivative at that point must be zero

df
0
dq q q*

8
 Second Order Conditions
• The first order condition (d/dq) is a
necessary condition for a maximum, but
it is not a sufficient condition

If the profit function was u-shaped,
the first order condition would result
in q* being chosen and  would
be minimized

*

Quantity
q*
9
 Second Order Conditions
• This must mean that, in order for q* to
be the optimum,
d d
 0 for q  q * and  0 for q  q *
dq dq

• Therefore, at q*, d/dq must be

decreasing

10
 Second Derivatives
• The derivative of a derivative is called a
second derivative
• The second derivative can be denoted
by
d 2 d 2f
2
or 2
or f " (q )
dq dq

11
 Second Order Condition
• The second order condition to represent
a (local) maximum is

d 
2

2
 f " (q ) q  q *  0
dq q q *

12
Rules for Finding Derivatives
db
1. If b is a constant, then 0
dx
d [bf ( x )]
2. If b is a constant, then  bf ' ( x )
dx

dx b
3. If b is constant, then  bx b 1
dx
d ln x 1
4. 
dx x 13
Rules for Finding Derivatives

da x
5.  a ln a for any constant a
x

dx
– a special case of this rule is dex/dx = ex

14
Rules for Finding Derivatives
• Suppose that f(x) and g(x) are two
functions of x and f’(x) and g’(x) exist
• Then
d [f ( x )  g ( x )]
6.  f '(x)  g'(x)
dx

d [f ( x )  g ( x )]
7.  f ( x )g ' ( x )  f ' ( x )g ( x )
dx
15
Rules for Finding Derivatives
 f (x) 
d  
 g ( x )  f ' ( x )g ( x )  f ( x )g ' ( x )
8.  2
dx [g ( x )]
provided that g ( x )  0

16
Rules for Finding Derivatives
• If y = f(x) and x = g(z) and if both f’(x)
and g’(x) exist, then:
dy dy dx df dg
9.    
dz dx dz dx dz
• This is called the chain rule. The chain
rule allows us to study how one variable
(z) affects another variable (y) through
its influence on some intermediate
variable (x) 17
 Rules for Finding Derivatives
• Some examples of the chain rule
include
deax deax d (ax )
10.    eax  a  aeax
dx d (ax ) dx
d [ln( ax )] d [ln( ax )] d (ax )
11.    ln( ax )  a  a ln( ax )
dx d (ax ) dx
d [ln( x 2 )] d [ln( x 2 )] d ( x 2 ) 1 2
12.  2
  2  2x 
dx d(x ) dx x x18
Example of Profit Maximization
• Suppose that the relationship between
profit and output is
 = 1,000q - 5q2
• The first order condition for a maximum is
d/dq = 1,000 - 10q = 0
q* = 100
• Since the second derivative is always
-10, q = 100 is a global maximum
19
Functions of Several Variables
• Most goals of economic agents depend
on several variables
– trade-offs must be made
• The dependence of one variable (y) on
a series of other variables (x1,x2,…,xn) is
denoted by

y  f (x1, x2 ,..., xn )
20
 Partial Derivatives
• The partial derivative of y with respect
to x1 is denoted by
y f
or or fx or f1
x1 x1 1

• It is understood that in calculating the

partial derivative, all of the other x’s are
held constant
21
 Partial Derivatives
• A more formal definition of the partial
derivative is

f f ( x1  h, x 2 ,..., x n )  f ( x1, x 2 ,..., x n )

 lim
x1 x 2 ,...,x n
h 0 h

22
Calculating Partial Derivatives
1. If y  f ( x1, x 2 )  ax12  bx1 x 2  cx 22 , then
f
 f1  2ax1  bx 2 and
x1
f
 f2  bx1  2cx 2
x 2
ax1  bx 2
2. If y  f (x1, x2 )  e , then
f ax  bx f ax  bx
 f1  ae 1
and
2
 f2  be 1 2

x1 x2
23
Calculating Partial Derivatives
3. If y  f (x1, x2 )  a ln x1  b ln x2 , then
f a f b
 f1  and  f2 
x1 x1 x2 x2

24
 Partial Derivatives
• Partial derivatives are the mathematical
expression of the ceteris paribus
assumption
– show how changes in one variable affect
some outcome when other influences are
held constant

25
 Partial Derivatives
• We must be concerned with how
variables are measured
– if q represents the quantity of gasoline
demanded (measured in billions of gallons)
and p represents the price in dollars per
gallon, then q/p will measure the change
in demand (in billiions of gallons per year)
for a dollar per gallon change in price

26
 Elasticity
• Elasticities measure the proportional
effect of a change in one variable on
another
– unit free
• The elasticity of y with respect to x is
y
y y x y x
ey , x     
x x y x y
x 27
Elasticity and Functional Form
• Suppose that
y = a + bx + other terms
• In this case,
y x x x
ey,x    b  b
x y y a  bx    
• ey,x is not constant
– it is important to note the point at which the
elasticity is to be computed
28
Elasticity and Functional Form
• Suppose that
y = axb
• In this case,

y x b 1 x
ey , x    abx  b  b
x y ax

29
Elasticity and Functional Form
• Suppose that
ln y = ln a + b ln x
• In this case,
y x  ln y
ey , x    b
x y  ln x
• Elasticities can be calculated through
logarithmic differentiation
30
Second-Order Partial Derivatives
• The partial derivative of a partial
derivative is called a second-order
partial derivative

(f / xi )  2f
  fij
x j x j xi

31
 Young’s Theorem
• Under general conditions, the order in
which partial differentiation is conducted
to evaluate second-order partial
derivatives does not matter

fij  f ji

32
Use of Second-Order Partials
• Second-order partials play an important
role in many economic theories
• One of the most important is a
variable’s own second-order partial, fii
– shows how the marginal influence of xi on
y(y/xi) changes as the value of xi
increases
– a value of fii < 0 indicates diminishing
marginal effectiveness
33
 Total Differential
• Suppose that y = f(x1,x2,…,xn)
• If all x’s are varied by a small amount,
the total effect on y will be
f f f
dy  dx1  dx 2  ...  dx n
x1 x2 xn

dy  f1dx1  f2dx 2  ...  fndx n

34
 First-Order Condition for a
Maximum (or Minimum)
• A necessary condition for a maximum (or
minimum) of the function f(x1,x2,…,xn) is
that dy = 0 for any combination of small
changes in the x’s
• The only way for this to be true is if
f1  f2  ...  fn  0
• A point where this condition holds is
called a critical point
35
 Finding a Maximum
• Suppose that y is a function of x1 and x2
y = - (x1 - 1)2 - (x2 - 2)2 + 10
y = - x12 + 2x1 - x22 + 4x2 + 5
• First-order conditions imply that

y
 2 x1  2  0 x1*  1
x1 OR
y x2  2
*

 2 x 2  4  0
x 2
36
Production Possibility Frontier

• Earlier example: 2x2 + y2 = 225

• Can be rewritten: f(x,y) = 2x2 + y2 - 225 = 0
• Because fx = 4x and fy = 2y, the opportunity
cost trade-off between x and y is

dy  fx  4x  2x
  
dx fy 2y y

37
 Implicit Function Theorem
• It may not always be possible to solve
implicit functions of the form g(x,y)=0 for
unique explicit functions of the form y = f(x)
– mathematicians have derived the necessary
conditions
– in many economic applications, these
conditions are the same as the second-order
conditions for a maximum (or minimum)

38
 The Envelope Theorem
• The envelope theorem concerns how the
optimal value for a particular function
changes when a parameter of the function
changes
• This is easiest to see by using an example

39
 The Envelope Theorem
• Suppose that y is a function of x
y = -x2 + ax
• For different values of a, this function
represents a family of parabolas
• If a is assigned a specific value, then y
becomes a function of x only and the value
of x that maximizes y can be calculated

40
 The Envelope Theorem
Optimal Values of x and y for alternative values of a

Value of a Value of x* Value of y*

0 0 0
1 1/2 1/4
2 1 1
3 3/2 9/4
4 2 4
5 5/2 25/4
6 3 9
41
 The Envelope Theorem
y*
10

9
As a increases,
8 the maximal value
7 for y (y*) increases
6

4 The relationship
3
between a and y
2

1
is quadratic
0 a
0 1 2 3 4 5 6 7

42
 The Envelope Theorem
• Suppose we are interested in how y*
changes as a changes
• There are two ways we can do this
– calculate the slope of y directly
– hold x constant at its optimal value and
calculate y/a directly

43
 The Envelope Theorem
• To calculate the slope of the function, we
must solve for the optimal value of x for
any value of a
dy/dx = -2x + a = 0
x* = a/2
• Substituting, we get
y* = -(x*)2 + a(x*) = -(a/2)2 + a(a/2)
y* = -a2/4 + a2/2 = a2/4
44
 The Envelope Theorem
• Therefore,
dy*/da = 2a/4 = a/2 = x*

• But, we can save time by using the

envelope theorem
– for small changes in a, dy*/da can be
computed by holding x at x* and calculating
y/ a directly from y

45
 The Envelope Theorem
y/ a = x
• Holding x = x*
y/ a = x* = a/2

• This is the same result found earlier

46
 The Envelope Theorem
• The envelope theorem states that the
change in the optimal value of a function
with respect to a parameter of that function
can be found by partially differentiating the
objective function while holding x (or
several x’s) at its optimal value

dy * y
 {x  x * (a)}
da a
47
 Constrained Maximization
• What if all values for the x’s are not
feasible?
– the values of x may all have to be positive
– a consumer’s choices are limited by the
amount of purchasing power available
• One method used to solve constrained
maximization problems is the Lagrangian
multiplier method
48
Lagrangian Multiplier Method
• Suppose that we wish to find the values
of x1, x2,…, xn that maximize
y = f(x1, x2,…, xn)
subject to a constraint that permits only
certain values of the x’s to be used
g(x1, x2,…, xn) = 0

49
Lagrangian Multiplier Method
• The Lagrangian multiplier method starts
with setting up the expression
L = f(x1, x2,…, xn ) + g(x1, x2,…, xn)
where  is an additional variable called
a Lagrangian multiplier
• When the constraint holds, L = f
because g(x1, x2,…, xn) = 0

50
Lagrangian Multiplier Method
• First-Order Conditions
L/x1 = f1 + g1 = 0
L/x2 = f2 + g2 = 0
.
.
.
L/xn = fn + gn = 0
L/ = g(x1, x2,…, xn) = 0
51
Lagrangian Multiplier Method
• The first-order conditions can generally
be solved for x1, x2,…, xn and 

• The solution will have two properties:

– the x’s will obey the constraint
– these x’s will make the value of L (and
therefore f) as large as possible

52
Lagrangian Multiplier Method
• The Lagrangian multiplier () has an
important economic interpretation
• The first-order conditions imply that
f1/-g1 = f2/-g2 =…= fn/-gn = 
– the numerators above measure the
marginal benefit that one more unit of xi will
have for the function f
– the denominators reflect the added burden
on the constraint of using more xi
53
Lagrangian Multiplier Method
• At the optimal choices for the x’s, the
ratio of the marginal benefit of increasing
xi to the marginal cost of increasing xi
should be the same for every x
•  is the common cost-benefit ratio for all
of the x’s
marginal benefit of xi

marginal cost of xi
54
Lagrangian Multiplier Method
• If the constraint was relaxed slightly, it
would not matter which x is changed
• The Lagrangian multiplier provides a
measure of how the relaxation in the
constraint will affect the value of y
•  provides a “shadow price” to the
constraint

55
Lagrangian Multiplier Method
• A high value of  indicates that y could
be increased substantially by relaxing
the constraint
– each x has a high cost-benefit ratio
• A low value of  indicates that there is
not much to be gained by relaxing the
constraint
• =0 implies that the constraint is not
binding
56
 Duality
• Any constrained maximization problem
has associated with it a dual problem in
constrained minimization that focuses
attention on the constraints in the
original problem

57
 Duality
• Individuals maximize utility subject to a
budget constraint
– dual problem: individuals minimize the
expenditure needed to achieve a given level
of utility
• Firms minimize the cost of inputs to
produce a given level of output
– dual problem: firms maximize output for a
given cost of inputs purchased
58
 Second Order Conditions -
Functions of One Variable
• Let y = f(x)
• A necessary condition for a maximum is
that
dy/dx = f ’(x) = 0
• To ensure that the point is a maximum, y
must be decreasing for movements away
from it
59
 Second Order Conditions -
Functions of One Variable
• The total differential measures the change
in y
dy = f ’(x) dx
• To be at a maximum, dy must be
decreasing for small increases in x
• To see the changes in dy, we must use
the second derivative of y
60
 Second Order Conditions -
Functions of One Variable
d [f ' ( x )dx ]
d y
2
 dx  f " ( x )dx  dx  f " ( x )dx 2

dx

• Note that d 2y < 0 implies that f ’’(x)dx2 < 0

• Since dx2 must be positive, f ’’(x) < 0

• This means that the function f must have a

concave shape at the critical point
61
 Second Order Conditions -
Functions of Two Variables
• Suppose that y = f(x1, x2)
• First order conditions for a maximum are
y/x1 = f1 = 0
y/x2 = f2 = 0
• To ensure that the point is a maximum, y
must diminish for movements in any
direction away from the critical point
62
 Second Order Conditions -
Functions of Two Variables
• The slope in the x1 direction (f1) must be
diminishing at the critical point
• The slope in the x2 direction (f2) must be
diminishing at the critical point
• But, conditions must also be placed on the
cross-partial derivative (f12 = f21) to ensure
that dy is decreasing for all movements
through the critical point
63
 Second Order Conditions -
Functions of Two Variables
• The total differential of y is given by
dy = f1 dx1 + f2 dx2
• The differential of that function is
d 2y = (f11dx1 + f12dx2)dx1 + (f21dx1 + f22dx2)dx2
d 2y = f11dx12 + f12dx2dx1 + f21dx1 dx2 + f22dx22
• By Young’s theorem, f12 = f21 and
d 2y = f11dx12 + 2f12dx1dx2 + f22dx22
64
 Second Order Conditions -
Functions of Two Variables
d 2y = f11dx12 + 2f12dx1dx2 + f22dx22
• For this equation to be unambiguously
negative for any change in the x’s, f11 and
f22 must be negative
• If dx2 = 0, then d 2y = f11 dx12
– for d 2y < 0, f11 < 0
• If dx1 = 0, then d 2y = f22 dx22
– for d 2y < 0, f22 < 0 65
 Second Order Conditions -
Functions of Two Variables
d 2y = f11dx12 + 2f12dx1dx2 + f22dx22
• If neither dx1 nor dx2 is zero, then d 2y will
be unambiguously negative only if
f11 f22 - f122 > 0
– the second partial derivatives (f11 and f22) must
be sufficiently negative so that they outweigh
any possible perverse effects from the cross-
partial derivatives (f12 = f21)
66
 Constrained Maximization
• Suppose we want to choose x1 and x2
to maximize
y = f(x1, x2)
• subject to the linear constraint
c - b 1x 1 - b 2x 2 = 0
• We can set up the Lagrangian
L = f(x1, x2) + (c - b1x1 - b2x2)

67
 Constrained Maximization
• The first-order conditions are
f1 - b1 = 0
f2 - b2 = 0
c - b 1x 1 - b 2x 2 = 0
• To ensure we have a maximum, we
must use the “second” total differential
d 2y = f11dx12 + 2f12dx1dx2 + f22dx22

68
 Constrained Maximization
• Only the values of x1 and x2 that satisfy
the constraint can be considered valid
alternatives to the critical point
• Thus, we must calculate the total
differential of the constraint
-b1 dx1 - b2 dx2 = 0
dx2 = -(b1/b2)dx1
• These are the allowable relative changes
in x1 and x2 69
 Constrained Maximization
• Because the first-order conditions imply
that f1/f2 = b1/b2, we can substitute and
get
dx2 = -(f1/f2) dx1
• Since
d 2y = f11dx12 + 2f12dx1dx2 + f22dx22
we can substitute for dx2 and get
d 2y = f11dx12 - 2f12(f1/f2)dx12 + f22(f12/f22)dx12
70
 Constrained Maximization
• Combining terms and rearranging
d 2y = f11 f22 - 2f12f1f2 + f22f12 [dx12/ f22]
• Therefore, for d 2y < 0, it must be true
that
f11 f22 - 2f12f1f2 + f22f12 < 0
• This equation characterizes a set of
functions termed quasi-concave functions
– any two points within the set can be joined
by a line contained completely in the set
71
 Concave and Quasi-
Concave Functions
• The differences between concave and
quasi-concave functions can be
illustrated with the function
y = f(x1,x2) = (x1x2)k
where the x’s take on only positive
values and k can take on a variety of
positive values

72
 Concave and Quasi-
Concave Functions
• No matter what value k takes, this
function is quasi-concave
• Whether or not the function is concave
depends on the value of k
– if k < 0.5, the function is concave
– if k > 0.5, the function is convex

73
 Homogeneous Functions
• A function f(x1,x2,…xn) is said to be
homogeneous of degree k if
f(tx1,tx2,…txn) = tk f(x1,x2,…xn)
– when a function is homogeneous of degree
one, a doubling of all of its arguments
doubles the value of the function itself
– when a function is homogeneous of degree
zero, a doubling of all of its arguments
leaves the value of the function unchanged
74
 Homogeneous Functions
• If a function is homogeneous of degree
k, the partial derivatives of the function
will be homogeneous of degree k-1

75
 Euler’s Theorem
• If we differentiate the definition for
homogeneity with respect to the
proportionality factor t, we get
ktk-1f(x1,…,xn) = x1f1(tx1,…,txn) + … + xnfn(x1,…,xn)

• This relationship is called Euler’s theorem

76
 Euler’s Theorem
• Euler’s theorem shows that, for
homogeneous functions, there is a
definite relationship between the
values of the function and the values of
its partial derivatives

77
 Homothetic Functions
• A homothetic function is one that is
formed by taking a monotonic
transformation of a homogeneous
function
– they do not possess the homogeneity
properties of their underlying functions

78
 Homothetic Functions
• For both homogeneous and homothetic
functions, the implicit trade-offs among
the variables in the function depend
only on the ratios of those variables, not
on their absolute values

79
 Homothetic Functions
• Suppose we are examining the simple,
two variable implicit function f(x,y) = 0
• The implicit trade-off between x and y
for a two-variable function is
dy/dx = -fx/fy
• If we assume f is homogeneous of
degree k, its partial derivatives will be
homogeneous of degree k-1
80
 Homothetic Functions
• The implicit trade-off between x and y is
dy t k 1fx (tx, ty ) fx (tx, ty )
  k 1 
dx t fy (tx, ty ) fy (tx, ty )
• If t = 1/y,
x  x 
F ' fx  ,1 fx  ,1
dy
  y 
  y 
dx x  x 
F ' fy  ,1 fy  ,1
y  y 
81
 Homothetic Functions
• The trade-off is unaffected by the
monotonic transformation and remains
a function only of the ratio x to y

82
 Important Points to Note:
• Using mathematics provides a
convenient, short-hand way for
economists to develop their models
– implications of various economic
assumptions can be studied in a
simplified setting through the use of such
mathematical tools

83
 Important Points to Note:
• Derivatives are often used in economics
because economists are interested in
how marginal changes in one variable
affect another
– partial derivatives incorporate the ceteris
paribus assumption used in most economic
models

84
 Important Points to Note:
• The mathematics of optimization is an
important tool for the development of
models that assume that economic
agents rationally pursue some goal
– the first-order condition for a maximum
requires that all partial derivatives equal
zero

85
 Important Points to Note:
• Most economic optimization
problems involve constraints on the
choices that agents can make
– the first-order conditions for a
maximum suggest that each activity be
operated at a level at which the ratio of
the marginal benefit of the activity to its
marginal cost

86
 Important Points to Note:
• The Lagrangian multiplier is used to
help solve constrained maximization
problems
– the Lagrangian multiplier can be
interpreted as the implicit value (shadow
price) of the constraint

87
 Important Points to Note:
• The implicit function theorem illustrates
the dependence of the choices that
result from an optimization problem on
the parameters of that problem

88
Important Points to Note:
• The envelope theorem examines
how optimal choices will change as
the problem’s parameters change
• Some optimization problems may
involve constraints that are
inequalities rather than equalities

89
Important Points to Note:
• First-order conditions are necessary
but not sufficient for ensuring a
maximum or minimum
– second-order conditions that describe
the curvature of the function must be
checked

90
Important Points to Note:
• Certain types of functions occur in
many economic problems
– quasi-concave functions obey the
second-order conditions of constrained
maximum or minimum problems when
the constraints are linear
– homothetic functions have the property
that implicit trade-offs among the
variables depend only on the ratios of
these variables
91