Systems & Control: Foundations & Applications
Series Editor
Tamer Başar, University of Illinois at Urbana-Champaign
Editorial Board
Karl Johan Åström, Lund Institute of Technology, Lund
Han-Fu Chen, Academia Sinica, Bejing
William Helton, University of California, San Diego
Alberto Isidori, University of Rome (Italy) and
Washington University, St. Louis
Petar V. Kokotović, University of California, Santa Barbara
Alexander Kurzhanski, Russian Academy of Sciences, Moscow and
University of California, Berkeley
H. Vincent Poor, Princeton University
Mete Soner, Koç University, Istanbul
Andrew J. Kurdila
Michael Zabarankin
Convex
Functional Analysis
Birkhauser Verlag
Basel • Boston • Berlin
Authors:
Andrew J. Kurdila Michael Zabarankin
Department of Mechanical and Department of Mathematical Sciences
Aerospace Engineering Stevens Institute of Technology
University of Florida Castle Point on Hudson
Gainesville, FL 32611-6250 Hoboken, NJ 07030
USA USA
ajk@mae.ufl.edu mzabaran@stevens.edu
2000 Mathematics Subject Classification 46N10, 49J15, 49J20, 49J27,
49J40, 49J50, 49K15, 49K20, 49K27, 49N90, 65K10, 90C25, 93C20, 93C25
A CIP catalogue record for this book is available from the Library of
Congress, Washington D.C., USA
Bibliographic information published by Die Deutsche Bibliothek
Die Deutsche Bibliothek lists this publication in the Deutsche
Nationalbibliografie; detailed bibliographic data is available in the Internet
at <http://dnb.ddb.de>.
ISBN 3-7643-2198-9 Birkhäuser Verlag, Basel – Boston – Berlin
This work is subject to copyright. All rights are reserved, whether the whole
or part of the material is concerned, specifically the rights of translation,
reprinting, re-use of illustrations, recitation, broadcasting, reproduction on
microfilms or in other ways, and storage in data banks. For any kind of use
permission of the copyright owner must be obtained.
© 2005 Birkhäuser Verlag, P.O. Box 133, CH-4010 Basel, Switzerland
Part of Springer Science+Business Media
Printed on acid-free paper produced of chlorine-free pulp. TCF f
Printed in Germany
ISBN-10: 3-7643-2198-9
ISBN-13: 978-3-7643-2198-7
987654321
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
1 Classical Abstract Spaces in Functional Analysis
1.1 Introduction and Notation . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Topological Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 Convergence in Topological Spaces . . . . . . . . . . . . . . 13
1.2.2 Continuity of Functions on Topological Spaces . . . . . . . 15
1.2.3 Weak Topology . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2.4 Compactness of Sets in Topological Spaces . . . . . . . . . 19
1.3 Metric Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.3.1 Convergence and Continuity in Metric Spaces . . . . . . . . 21
1.3.2 Closed and Dense Sets in Metric Spaces . . . . . . . . . . . 23
1.3.3 Complete Metric Spaces . . . . . . . . . . . . . . . . . . . . 23
1.3.4 The Baire Category Theorem . . . . . . . . . . . . . . . . . 25
1.3.5 Compactness of Sets in Metric Spaces . . . . . . . . . . . . 27
1.3.6 Equicontinuous Functions on Metric Spaces . . . . . . . . . 30
1.3.7 The Arzela-Ascoli Theorem . . . . . . . . . . . . . . . . . . 33
1.3.8 Hölder’s and Minkowski’s Inequalities . . . . . . . . . . . . 35
1.4 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
1.5 Normed Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . 45
1.5.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . 45
1.5.2 Examples of Normed Vector Spaces . . . . . . . . . . . . . 46
1.6 Space of Lebesgue Measurable Functions . . . . . . . . . . . . . . . 52
1.6.1 Introduction to Measure Theory . . . . . . . . . . . . . . . 52
1.6.2 Lebesgue Integral . . . . . . . . . . . . . . . . . . . . . . . . 54
1.6.3 Measurable Functions . . . . . . . . . . . . . . . . . . . . . 57
1.7 Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2 Linear Functionals and Linear Operators
2.1 Fundamental Theorems of Analysis . . . . . . . . . . . . . . . . . . 65
2.1.1 Hahn-Banach Theorem . . . . . . . . . . . . . . . . . . . . 65
2.1.2 Uniform Boundedness Theorem . . . . . . . . . . . . . . . . 69
2.1.3 The Open Mapping Theorem . . . . . . . . . . . . . . . . . 71
2.2 Dual Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
vi Contents
2.3 The Weak Topology . . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.4 The Weak∗ Topology . . . . . . . . . . . . . . . . . . . . . . . . . . 80
2.5 Signed Measures and Topology . . . . . . . . . . . . . . . . . . . . 88
2.6 Riesz’s Representation Theorem . . . . . . . . . . . . . . . . . . . . 91
2.6.1 Space of Lebesgue Measurable Functions . . . . . . . . . . . 91
2.6.2 Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . . . . 94
2.7 Closed Operators on Hilbert Spaces . . . . . . . . . . . . . . . . . 95
2.8 Adjoint Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
2.9 Gelfand Triples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
2.10 Bilinear Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3 Common Function Spaces in Applications
3.1 The Lp Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.2 Sobolev Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.2.1 Distributional Derivatives . . . . . . . . . . . . . . . . . . . 114
3.2.2 Sobolev Spaces, Integer Order . . . . . . . . . . . . . . . . . 117
3.2.3 Sobolev Spaces, Fractional Order . . . . . . . . . . . . . . . 118
3.2.4 Trace Theorems . . . . . . . . . . . . . . . . . . . . . . . . 122
3.2.5 The Poincaré Inequality . . . . . . . . . . . . . . . . . . . . 123
3.3 Banach Space Valued Functions . . . . . . . . . . . . . . . . . . . . 126
3.3.1 Bochner Integrals
. . . . . . . . . . . . . . . . . . . . . . . . 126
3.3.2 The Space Lp (0, ), X . . . . .
T . . . . . . . . . . . . . . . 131
p,q
3.3.3 The Space W (0, T ), X . . . . . . . . . . . . . . . . . . 133
4 Differential Calculus in Normed Vector Spaces
4.1 Differentiability of Functionals . . . . . . . . . . . . . . . . . . . . 137
4.1.1 Gateaux Differentiability . . . . . . . . . . . . . . . . . . . 137
4.1.2 Fréchet Differentiability . . . . . . . . . . . . . . . . . . . . 139
4.2 Classical Examples of Differentiable Operators . . . . . . . . . . . 143
5 Minimization of Functionals
5.1 The Weierstrass Theorem . . . . . . . . . . . . . . . . . . . . . . . 161
5.2 Elementary Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . 163
5.3 Minimization of Differentiable Functionals . . . . . . . . . . . . . . 165
5.4 Equality Constrained Smooth Functionals . . . . . . . . . . . . . . 166
5.5 Fréchet Differentiable Implicit Functionals . . . . . . . . . . . . . . 171
6 Convex Functionals
6.1 Characterization of Convexity . . . . . . . . . . . . . . . . . . . . . 177
6.2 Gateaux Differentiable Convex Functionals . . . . . . . . . . . . . 180
6.3 Convex Programming in Rn . . . . . . . . . . . . . . . . . . . . . . 183
6.4 Ordered Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . 188
6.4.1 Positive Cones, Negative Cones, and Orderings . . . . . . . 189
6.4.2 Orderings on Sobolev Spaces . . . . . . . . . . . . . . . . . 191
Contents vii
6.5 Convex Programming in Ordered Vector Spaces . . . . . . . . . . . 193
6.6 Gateaux Differentiable Functionals on Ordered Vector Spaces . . . 199
7 Lower Semicontinuous Functionals
7.1 Characterization of Lower Semicontinuity . . . . . . . . . . . . . . 205
7.2 Lower Semicontinuous Functionals and Convexity . . . . . . . . . . 208
7.2.1 Banach Theorem for Lower Semicontinuous Functionals . . 208
7.2.2 Gateaux Differentiability . . . . . . . . . . . . . . . . . . . 210
7.2.3 Lower Semicontinuity in Weak Topologies . . . . . . . . . . 210
7.3 The Generalized Weierstrass Theorem . . . . . . . . . . . . . . . . 212
7.3.1 Compactness in Weak Topologies . . . . . . . . . . . . . . . 213
7.3.2 Bounded Constraint Sets . . . . . . . . . . . . . . . . . . . 215
7.3.3 Unbounded Constraint Sets . . . . . . . . . . . . . . . . . . 215
7.3.4 Constraint Sets on Ordered Vector Spaces . . . . . . . . . . 217
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
List of Figures
1.1 Illustrations of notions of lim sup xn and lim inf xn . . . . . . . . . 4
n→∞ n→∞
1.2 Blurred Sierpinski Gasket, Various N . . . . . . . . . . . . . . . . . 7
1.3 Refinements of a Topological Space . . . . . . . . . . . . . . . . . . 8
1.4 Simple Function and Sequence of Fourier Approximations . . . . . 10
1.5 Nested Refinements . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.6 Path of Triangular Refinements . . . . . . . . . . . . . . . . . . . . 14
1.7 Convergent Sequence, Schematic Representation . . . . . . . . . . 22
1.8 Cauchy Sequence, Schematic Representation . . . . . . . . . . . . . 24
1.9 Embedding of Incomplete Metric Space X in Complete Metric Space
Y via Isometry i . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1.10 Young’s Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.1 Schematic Diagram of Dilation of Open Ball . . . . . . . . . . . . . 72
2.2 C(X), its Dual and Bidual . . . . . . . . . . . . . . . . . . . . . . . 91
2.3 Difference adjacent basis functions . . . . . . . . . . . . . . . . . . 91
2.4 Basis function for Theorem 2.6.1 . . . . . . . . . . . . . . . . . . . 92
3.1 L1 (Ω), its Dual and Bidual . . . . . . . . . . . . . . . . . . . . . . 113
3.2 φx Diffeomorphic from Ox to B1 (0) . . . . . . . . . . . . . . . . . . 120
6.1 Convex and Nonconvex Sets . . . . . . . . . . . . . . . . . . . . . . 175
6.2 Graph of a Function . . . . . . . . . . . . . . . . . . . . . . . . . . 178
6.3 Lower Sections of the Graph . . . . . . . . . . . . . . . . . . . . . . 178
6.4 Convex and Nonconvex Functionals . . . . . . . . . . . . . . . . . . 179
6.5 Epigraph of a Functional . . . . . . . . . . . . . . . . . . . . . . . . 179
6.6 Separation of Sets via a Hyperplane . . . . . . . . . . . . . . . . . 180
6.7 Examples of Cones in R2 . . . . . . . . . . . . . . . . . . . . . . . . 189
6.8 Open and Closed Cones . . . . . . . . . . . . . . . . . . . . . . . . 190
6.9 Ordering Associated with a Convex Cone . . . . . . . . . . . . . . 191
7.1 A function that is lower semicontinuous at x0 but not lower semi-
continuous at x1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Preface
Overview of Book
This book evolved over a period of years as the authors taught classes in varia-
tional calculus and applied functional analysis to graduate students in engineering
and mathematics. The book has likewise been influenced by the authors’ research
programs that have relied on the application of functional analytic principles to
problems in variational calculus, mechanics and control theory.
One of the most difficult tasks in preparing to utilize functional, convex, and
set-valued analysis in practical problems in engineering and physics is the intimi-
dating number of definitions, lemmas, theorems and propositions that constitute
the foundations of functional analysis. It cannot be overemphasized that functional
analysis can be a powerful tool for analyzing practical problems in mechanics and
physics. However, many academicians and researchers spend their lifetime study-
ing abstract mathematics. It is a demanding field that requires discipline and
devotion. It is a trite analogy that mathematics can be viewed as a pyramid of
knowledge, that builds layer upon layer as more mathematical structure is put in
place. The difficulty lies in the fact that an engineer or scientist typically would
like to start somewhere “above the base” of the pyramid. Engineers and scientists
are not as concerned, generally speaking, with the subtleties of deriving theorems
axiomatically. Rather, they are interested in gaining a working knowledge of the
applicability of the theory to their field of interest.
The content and structure of the book reflects the sometimes conflicting
requirements of researchers or students who have formal training in either engi-
neering or applied mathematics. Typically, before taking this course, those trained
within an engineering discipline might have a working knowledge of fundamental
topics in mechanics or control theory. Engineering students may be perfectly com-
fortable with the notion of the stress distribution in an elastic continuum, or the
velocity field in an incompressible flow. The formulation of the equations govern-
ing the static equilibrium of elastic bodies, or the structure of the Navier-Stokes
Equations for incompressible flow, are often familiar to them. This is usually not
the case for first year graduate students trained in applied mathematics. Rather,
these students will have some familiarity with real analysis or functional analy-
sis. The fundamental theorems of analysis including the Open Mapping Theorem,
the Hahn-Banach Theorem, and the Closed Graph Theorem will constitute the
foundations of their training in many cases.
xii Preface
Coupled with this essential disparity in the training to which graduate stu-
dents in these two disciplines are exposed, it is a fact that formulations and so-
lutions of modern problems in control and mechanics are couched in functional
analytic terms. This trend is pervasive.
Thus, the goal of the present text is admittedly ambitious. This text seeks
to synthesize topics from abstract analysis with enough recent problems in control
theory and mechanics to provide students from both disciplines with a working
knowledge of functional analysis.
Organization
This work consists of two volumes. The primary thrust of this series is a discussion
of how convex analysis, as a specific subtopic in functional analysis, has served to
unify approaches in numerous problems in mechanics and control theory. Every
attempt has been made to make the series self-contained.
The first book in this series is dedicated to the fundamentals of convex func-
tional analysis. It presents those aspects of functional analysis that are used in
various applications to mechanics and control theory. The purpose of the first vol-
ume is essentially two-fold. On one hand, we wish to provide a bare minimum of the
theory required to understand the principles of functional, convex and set-valued
analysis. We want this presentation to be accessible to those with little advanced
graduate mathematics, which makes it a formidable task indeed. For this reason,
there are numerous examples and diagrams to provide as intuitive an explanation
of the principles as possible. The interested reader is, of course, referred to the
numerous excellent texts that present a complete treatment of the theory. On the
other hand, we would like to provide a concise summary of definitions and theo-
rems, even for those with a background in graduate mathematics, so that the text
is relatively self-contained.
The second book in the series discusses the application of functional analytic
principles to contact problems in mechanics, shape optimization problems, con-
trol of distributed parameter systems, identification problems in mechanics and
control of incompressible flow. While this list of applications is impressive, it is
hardly exhaustive. The second volume also overviews recent problems that can
be addressed with extensions of convex analysis. These applications include the
homogenization of steady state heat conduction equations, approximation theory
in identification problems and nonconvex variational problems in mechanics.
The first volume is organized as follows. Chapter 1 begins with a brief over-
view of topological spaces, and quickly focuses on metric topologies in particular.
Two of the most fundamental theorems included in this chapter are the Arzela-
Ascoli Theorem and Baire Category Theorem. Next, a brief discussion of normed
vector spaces follows. Section 1.4 presents the foundations of measure and integra-
tion theory, while Section 1.7 introduces Hilbert Spaces.
Preface xiii
Chapter 2 includes a tutorial on bounded linear operator and weak topologies.
The chapter presents the Hahn-Banach Theorem, Uniform Boundedness Theorem,
Closed Graph Theorem, Riesz’s Representation Theorem and common construc-
tions such as Gelfand Triples.
Chapter 3 is particularly important to applications: it includes descriptions
of common abstract spaces encountered in practice. A special emphasis is given to
a discussion of Sobolev Spaces.
Chapter 4 discusses the most common notions of differentiability for func-
tionals: Gateaux and Fréchet differentiability. It also presents classical examples
of differentiable functionals.
Chapter 5 introduces Lagrange multiplier techniques to characterize extrema
of optimization problems. The chapter includes a discussion of certain specialized
techniques for studying Fréchet differentiable functionals and equality constrained
problems.
Chapter 6 represents the fundamentals of classical convex analysis in func-
tional analysis. Convex functionals and sets are defined. The relationship between
continuity, convexity and differentiability is outlined. The chapter discusses formal,
“engineering techniques” for solving certain optimal control problems. Multiplier
methods are derived for convex functionals on ordered vector spaces.
Chapter 7 introduces the important notion of lower semicontinuity and intro-
duces the class of lower semicontinuous functionals. The crucial interplay of semi-
continuity and compactness in weak topologies is discussed. The chapter presents
several versions of the Generalized Weierstrass Theorem formulated for lower semi-
continuous functionals.
Acknowledgements
The authors would like to thank Professor William Hager of the Department of
Mathematics, Professor Panos M. Pardalos of the Department of Industrial and
Systems Engineering and Professor R. Tyrrel Rockafellar of the Department of
Mathematics at the University of Washington for their insight, advice and com-
ments on portions of the manuscript. The authors would also like to thank the
numerous project and contract officers who have had the foresight to support vari-
ous research projects by the authors. Many of the examples discussed in this book
are related to or extracted from research carried out under their guidance. Their
support has resulted in some of the examples discussed in the text. In particular,
the authors would like to acknowledge the generous support of Dr. Kam Ng of
the Office of Naval Research, Dr. Walt Silva of NASA Langley Research Center,
Dr. Marty Brenner of the Dryden Flight Research Center, and Dr. Gary Anderson
of the Army Research Office. Dr. Clifford Rhoades, the director of Physical and
Mathematical Sciences of the Air Force Office of Scientific Research, has had a
profound influence on the development of this text. Both authors have been fortu-
nate in that they have prepared substantial tracts of the text while working under
xiv Preface
AFOSR sponsorship. Furthermore, the authors extends their warmest gratitude to
Dr. Pasquale Storza, Director of the University of Florida Graduate Engineering
and Research Center (UF-GERC). This book was assembled in its final form while
the authors were visiting scholars at UF-GERC.
A large portion of this text was typeset by Maryum Ahmed, Rob Eick, Kristin
Fitzpatrick, and Joel Steward, who have our thanks for their careful attention to
detail and patience with the authors during the editorial process.
Both authors express their deepest gratitude to their wives and families. The
preparation of this book was carried out with their sustained encouragement and
support, despite the burden it sometimes put on the families.
Andrew J. Kurdila and Michael Zabarankin
Gainesville, Florida
April 24, 2004
