MIT OpenCourseWare

http://ocw.mit.edu

16.323 Principles of Optimal Control

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
16.323 Handout #3
Prof. J. P. How March 6, 2008
Due: March 18, 2008

16.323 Homework Assignment #3

1. Find the curve x� (t) that minimizes the functional

2. One important calculus of variations problem that we did not discuss in class has the
same basic form, but with constraints that are given by an integral - called isoperimetric
constraints:
� tf
min J = g[x, ẋ, t] dt
t0
� tf
such that : e[x, ẋ, t] dt = C
t0

where we will assume that tf is free but x(tf ) is fixed.

(a) Use the same approach followed in the notes to find the necessary and bound­
ary conditions for this optimal control problem. In particular, augment the
constraint to the cost using a constant Lagrange multiplier vector ν, and show
that these conditions can be written in the form:
� �
∂ga d ∂ga
− = 0
∂x dt ∂ẋ
∂ga
ga (tf ) − (tf )ẋ(tf ) = 0
∂ẋ
� tf
e[x, ẋ, t] dt = C
t0

where ga = g + ν T e
(b) Use the results of part (a) to clearly state the differential equations and corre­
sponding boundary conditions that must be solved to find the curve y(x) of a
specified length L with endpoints on the x-axis (i.e., at x = 0 and x = xf ) that
encloses the maximum area, so that
� xf � xf �
J= ydx and 1 + ẏ 2 dx = L
0 0

with xf free.

1
3. Consider the unstable second order system
ẋ1 = x2
ẋ2 = x2 + u
and performance index � ∞
J= (Rxx x21 + Ruu u2 ) dt
0

(a) For Rxx /Ruu = 1 show analytically (i.e. not using Matlab) that the steady-state
LQR gains are:
√
K=[ 1 3 + 1 ]

√
and that the closed-loop poles are at s = −( 3 ± j )/2.
(b) Using the steady-state regulator gains from Matlab in each case, plot (on one
graph) the closed-loop locations for a range of possible values of Rxx /Ruu . Show
the pole locations for the expensive control problem. Do you see any trends here?
4. Find the Hamiltonian and then solve the necessary conditions to compute the optimal
control and state trajectory that minimize
� 1
J= u2 dt
0
for the system ẋ = −2x + u with initial state x(0) = 2 and terminal state x(1) = 0.
Plot the optimal control and state response using Matlab.
5. Consider a disturbance rejection problem that minimizes:
1 1 � tf T
J = x(tf )T Hx(tf ) + x (t)Rxx (t)x(t) + u(t)T Ruu (t)u(t) dt (1)
2 2 t0
subject to
ẋ(t) = A(t)x(t) + B(t)u(t) + w(t). (2)
To handle the disturbance term, the optimal control should consist of both a feedback
term and a feedforward term (assume w(t) is known).
u� (t) = −K(t)x(t) + uf w (t), (3)
Using the Hamilton-Jacobi-Bellman equation, show that a possible optimal value func­
tion is of the form
1 1
J � (x(t), t) = xT (t)P (t)x(t) + bT (t)x(t) + c(t), (4)
2 2
where
−1 −1
K(t) = Ruu (t)B T (t)P (t), uf w = −Ruu (t)B T (t)b(t) (5)
In the process demonstrate that the conditions that must be satisfied are:

−1
−Ṗ (t) = AT (t)P (t) + P (t)A(t) + Rxx (t) − P (t)B(t)Ruu (t)B T (t)P (t)

� �T
−1
ḃ(t) = − A(t) − B(t)Ruu (t)B T (t)P (t) b(t) − P (t)w(t)
−1
ċ(t) = bT (t)B(t)Ruu (t)B T (t)b(t) − 2bT (t)w(t).
with boundary conditions: P (tf ) = H, b(tf ) = 0, c(tf ) = 0.