Mathematics in Chemical Engineering

BRUCE A. FINLAYSON, Department of Chemical Engineering, University of Washington,

Seattle, Washington, United States
LORENZ T. BIEGLER, Carnegie Mellon University, Pittsburgh, Pennsylvania, United
States
IGNACIO E. GROSSMANN, Carnegie Mellon University, Pittsburgh, Pennsylvania, United
States
KARL-HEINZ KU€ FER, Fraunhofer ITWM, Kaiserslautern, Germany
MICHAEL BORTZ, Fraunhofer ITWM, Kaiserslautern, Germany

1. Solution of Equations . . . . . . . 2 6.4. Stiffness. . . . . . . . . . . . . . . . . 59

1.1. Matrix Properties . . . . . . . . . . 3 6.5. Differential–Algebraic Systems . 60
1.2. Linear Algebraic Equations . . . 4 6.6. Computer Software . . . . . . . . . 61
1.3. Nonlinear Algebraic Equations . 7 6.7. Stability, Bifurcations, Limit
1.4. Linear Difference Equations . . . 9 Cycles . . . . . . . . . . . . . . . . . . 62
1.5. Eigenvalues . . . . . . . . . . . . . . 10 6.8. Sensitivity Analysis . . . . . . . . . 64
2. Approximation and Integration . 11 6.9. Molecular Dynamics . . . . . . . . 65
2.1. Introduction . . . . . . . . . . . . . . 11 7. Ordinary Differential Equations
2.2. Global Polynomial as Boundary Value Problems . . 65
Approximation . . . . . . . . . . . . 11 7.1. Solution by Quadrature . . . . . . 66
2.3. Piecewise Approximation . . . . . 13 7.2. Initial Value Methods . . . . . . . 67
2.4. Quadrature . . . . . . . . . . . . . . 17 7.3. Finite Difference Method . . . . . 67
2.5. Least Squares. . . . . . . . . . . . . 19 7.4. Orthogonal Collocation . . . . . . 70
2.6. Fourier Transforms of Discrete 7.5. Orthogonal Collocation on Finite
Data . . . . . . . . . . . . . . . . . . . 20 Elements . . . . . . . . . . . . . . . . 74
2.7. Two-Dimensional Interpolation 7.6. Galerkin Finite Element Method 76
and Quadrature . . . . . . . . . . . 21 7.7. Cubic B-Splines . . . . . . . . . . . 78
3. Complex Variables . . . . . . . . . 21 7.8. Adaptive Mesh Strategies. . . . . 78
3.1. Introduction to the Complex 7.9. Comparison . . . . . . . . . . . . . . 79
Plane. . . . . . . . . . . . . . . . . . . 21 7.10. Singular Problems and Infinite
3.2. Elementary Functions . . . . . . . 23 Domains . . . . . . . . . . . . . . . . 79
3.3. Analytic Functions of a Complex 8. Partial Differential Equations. . 80
Variable. . . . . . . . . . . . . . . . . 24 8.1. Classification of Equations . . . . 81
3.4. Integration in the Complex Plane 26 8.2. Hyperbolic Equations . . . . . . . 83
3.5. Other Results . . . . . . . . . . . . . 28 8.3. Parabolic Equations in One
4. Integral Transforms . . . . . . . . 29 Dimension . . . . . . . . . . . . . . . 85
4.1. Fourier Transforms . . . . . . . . . 29 8.4. Elliptic Equations . . . . . . . . . . 90
4.2. Laplace Transforms . . . . . . . . 34 8.5. Parabolic Equations in Two or
4.3. Solution of Partial Differential Three Dimensions . . . . . . . . . . 93
Equations by Using Transforms 39 8.6. Special Methods for Fluid
5. Vector Analysis . . . . . . . . . . . 42 Mechanics . . . . . . . . . . . . . . . 93
6. Ordinary Differential Equations 8.7. Computer Software . . . . . . . . . 95
as Initial Value Problems . . . . . 52 9. Integral Equations . . . . . . . . . 96
6.1. Solution by Quadrature . . . . . . 52 9.1. Classification . . . . . . . . . . . . . 96
6.2. Explicit Methods . . . . . . . . . . 53 9.2. Numerical Methods for Volterra
6.3. Implicit Methods. . . . . . . . . . . 57 Equations of the Second Kind . . 98

# 2015 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim

10.1002/14356007.b01_01.pub3
2 Mathematics in Chemical Engineering

9.3. Numerical Methods for 10.8.3. Solution Finding . . . . . . . . . . 135

Fredholm, Urysohn, and 10.8.3.1. Scalarization . . . . . . . . . . . . . 135
Hammerstein Equations of the 10.8.3.2. Approximation. . . . . . . . . . . . 137
Second Kind . . . . . . . . . . . . . . 98 10.8.3.3. Other Approaches . . . . . . . . . 138
9.4. Numerical Methods for 10.8.4. Presentation. . . . . . . . . . . . . . 138
Eigenvalue Problems . . . . . . . . 100 10.9. Optimization under Uncertainty 138
9.5. Green’s Functions . . . . . . . . . . 100 10.9.1. Introduction . . . . . . . . . . . . . 138
9.6. Boundary Integral Equations and 10.9.2. Sensitivity Analysis. . . . . . . . . 139
Boundary Element Method. . . . 101 10.9.3. Robust and Stochastic
10. Optimization . . . . . . . . . . . . . 103 Optimization . . . . . . . . . . . . . 140
10.1. Introduction . . . . . . . . . . . . . . 103 10.9.4. Optimization under
10.2. Gradient-Based Nonlinear Uncertainty by Multicriteria
Programming . . . . . . . . . . . . . 105 Optimization . . . . . . . . . . . . . 143
10.3. Optimization Methods without 11. Probability and Statistics . . . . . 144
Derivatives . . . . . . . . . . . . . . . 113 11.1. Concepts . . . . . . . . . . . . . . . . 144
10.4. Global Optimization . . . . . . . . 115 11.2. Sampling and Statistical
10.5. Mixed Integer Programming . . 119 Decisions . . . . . . . . . . . . . . . . 147
10.6. Dynamic Optimization. . . . . . . 129 11.3. Error Analysis in Experiments . 151
10.7. Development of Optimization 11.4. Factorial Design of Experiments
Models . . . . . . . . . . . . . . . . . 133 and Analysis of Variance . . . . . 151
10.8. Multicriteria Optimization . . . . 134 References . . . . . . . . . . . . . . . 154
10.8.1. Motivation . . . . . . . . . . . . . . 134
10.8.2. Considerations . . . . . . . . . . . . 134

1. Solution of Equations the algorithm is the solution of sets of algebraic

equations. The solution of sets of algebraic
Mathematical models of chemical engineering equations is the focus of Chapter 1.
systems can take many forms: They can be sets A single linear equation is easy to solve for
of algebraic equations, differential equations, either x or y:
and/or integral equations. Mass and energy
balances of chemical processes typically lead y ¼ ax þ b
to large sets of algebraic equations:
a11 x1 þ a12 x2 ¼ b1 If the equation is nonlinear,
a21 x1 þ a22 x2 ¼ b2

f ðxÞ ¼ 0
Mass balances of stirred tank reactors may lead
to ordinary differential equations:
it may be more difficult to find the x satisfying
dy
dt
¼ f ½yðtÞ this equation. These problems are compounded
when there are more unknowns, leading to
Radiative heat transfer may lead to integral simultaneous equations. If the unknowns
equations: appear in a linear fashion, then an important
Z1 consideration is the structure of the matrix
yðxÞ ¼ gðxÞ þ l Kðx; sÞf ðsÞd s representing the equations; special methods
0
are presented here for special structures.
Even when the model is a differential equa- They are useful because they increase the speed
tion or integral equation, the most basic step in of solution. If the unknowns appear in a
 Mathematics in Chemical Engineering 3

nonlinear fashion, the problem is much more determinant are all zero, the value of the deter-
difficult. Iterative techniques must be used (i.e., minant is zero. If the elements of one row or
make a guess of the solution and try to improve column are multiplied by the same constant, the
the guess). An important question then is determinant is the previous value times that
whether such an iterative scheme converges. constant. If two adjacent rows (or columns)
Other important types of equations are linear are interchanged, the value of the new determi-
difference equations and eigenvalue problems, nant is the negative of the value of the original
which are also discussed. determinant. If two rows (or columns) are
identical, the determinant is zero. The value
of a determinant is not changed if one row (or
1.1. Matrix Properties column) is multiplied by a constant and added
to another row (or column).
A matrix is a set of real or complex numbers A matrix is symmetric if
arranged in a rectangular array.
2 3 aij ¼ aji
a11 a12 ... a1n
6 a21 a22 ... a2n 7
6 7
A ¼ 6 .. .. .. .. 7 and it is positive definite if
4 . . . . 5
am1 am2 . . . amn
n X
X n
xT Ax ¼ aij xi xj  0
The numbers aij are the elements of the matrix i¼1 j¼1

A, or (A)ij ¼ aij. The transpose of A is (AT) ¼ aji.

The determinant of a square matrix A is
for all x and the equality holds only if x ¼ 0.

 a11 a12 ... a1n 
 If the elements of A are complex numbers,
  A denotes the complex conjugate in which
 a21 a22 ... a2n 
A ¼  .. .. .. .. 
 . . . .  (A )ij ¼ a ij. If A ¼ A the matrix is Hermitian.
 
an1 an2 ... ann The inverse of a matrix can also be used to
solve sets of linear equations. The inverse is a
If the i-th row and j-th column are deleted, a matrix such that when A is multiplied by its
new determinant is formed having n–1 rows and inverse the result is the identity matrix, a matrix
columns. This determinant is called the minor with 1.0 along the main diagonal and zero
of aij denoted as Mij. The cofactor A0ij of the elsewhere.
element aij is the signed minor of aij determined
by AA1 ¼ I

A0ij ¼ ð1Þiþj M ij
If AT ¼ A1 the matrix is orthogonal.
The value of jAj is given by Matrices are added and subtracted element
by element.
X
n X
n
jAj ¼ aij A0ij or aij A0ij A þ B is aij þ bij
j¼1 i¼1

Two matrices A and B are equal if aij ¼ bij.

where the elements aij must be taken from a
Special relations are
single row or column of A.
If all determinants formed by striking out ðABÞ1 ¼ B1 A1 ; ðABÞT ¼ BT AT
whole rows or whole columns of order greater ðA1 ÞT ¼ ðAT Þ1 ; ðABCÞ1 ¼ C1 B1 A1
than r are zero, but there is at least one deter-
minant of order r which is not zero, the matrix A diagonal matrix is zero except for elements
has rank r. along the diagonal.
The value of a determinant is not changed if

the rows and columns are interchanged. If the aij ¼
aii ; i ¼ j
elements of one row (or one column) of a 0; i 6¼ j
4 Mathematics in Chemical Engineering

A tridiagonal matrix is zero except for elements may be the size of a disturbance, and the output is
along the diagonal and one element to the right the gain [1]. If the rank is less than n, not all the
and left of the diagonal. variables are independent and they cannot all be
controlled. Furthermore, if the singular values
8
< 0 if j < i  1 are widely separated, the process is sensitive to
aij ¼ a otherwise small changes in the elements of the matrix and
: ij
0 if j > i þ 1
the process will be difficult to control.
Block diagonal and pentadiagonal matrices also
arise, especially when solving partial differen-
tial equations in two- and three-dimensions. 1.2. Linear Algebraic Equations

QR Factorization of a Matrix. If A is an m  n Consider the nn linear system

matrix with m  n, there exists an m  m
a11 x1 þ a12 x2 þ    þ a1n xn ¼ f 1
unitary matrix Q ¼ [q1, q2, . . . qm] and an a21 x1 þ a22 x2 þ    þ a2n xn ¼ f 2
m  n right-triangular matrix R such that ...
A ¼ QR. The QR factorization is frequently an1 x1 þ an2 x2 þ    þ ann xn ¼ f n
used in the actual computations when the other
transformations are unstable. In this equation a11, . . . , ann are known
parameters, f1, . . . , fn are known, and the
Singular Value Decomposition. If A is an unknowns are x1, . . . , xn. The values of all
m  n matrix with m  n and rank k  n, unknowns that satisfy every equation must be
consider the two following matrices. found. This set of equations can be represented
as follows:
 
AA and A A
X
n
aij xj ¼ f j or Ax ¼ f
An m  m unitary matrix U is formed from the j¼1

eigenvectors ui of the first matrix.

The most efficient method for solving a set of
U ¼ ½u1 ; u2 ; . . . um  linear algebraic equations is to perform a
lower–upper (LU) decomposition of the corre-
An n  n unitary matrix V is formed from the sponding matrix A. This decomposition is
eigenvectors vi of the second matrix. essentially a Gaussian elimination, arranged
V ¼ ½v1 ; v2 ; . . . ; vn  for maximum efficiency [2, 3].
The LU decomposition writes the matrix as
Then the matrix A can be decomposed into A ¼ LU


A ¼ USV The U is upper triangular; it has zero elements
below the main diagonal and possibly nonzero
where S is a k  k diagonal matrix with diagonal values along the main diagonal and above it
elements dii ¼ s i > 0 for 1  i  k. The eigen- (see Fig. 1). The L is lower triangular. It has
values of S S are s 2i . The vectors ui for k þ 1  i the value 1 in each element of the main
 m and vi for k þ 1  i  n are eigenvectors diagonal, nonzero values below the diagonal,
associated with the eigenvalue zero; the eigen- and zero values above the diagonal (see
values for 1  i  k are s 2i . The values of s i are Fig. 1). The original problem can be solved
called the singular values of the matrix A. If A is in two steps:
real, then U and Vare real, and hence orthogonal
matrices. The value of the singular value decom- Ly ¼ f; Ux ¼ y solves Ax ¼ LUx ¼ f
position comes when a process is represented
by a linear transformation and the elements of A, Each of these steps is straightforward
aij , are the contribution to an output i for a because the matrices are upper triangular or
particular variable as input variable j. The input lower triangular.
 Mathematics in Chemical Engineering 5

the smallest dii (see above). It can also be

expressed in terms of the norm of the matrix:

kðAÞ ¼k A k k A1 k

where the norm is defined as

k Ax k Xn
k A k supx6¼0 ¼ maxk jajk j
kxk j¼1

If this number is infinite, the set of equations

is singular. If the number is too large, the matrix
is said to be ill-conditioned. Calculation of the
condition number can be lengthy so another
criterion is also useful. Compute the ratio of the
largest to the smallest pivot and make judg-
ments on the ill-conditioning based on that.
When a matrix is ill-conditioned the LU
decomposition must be performed by using
pivoting (or the singular value decomposition
described above). With pivoting, the order of
the elimination is rearranged. At each stage,
one looks for the largest element (in magni-
tude); the next stages if the elimination are on
the row and column containing that largest
element. The largest element can be obtained
from only the diagonal entries (partial pivoting)
or from all the remaining entries. If the matrix is
nonsingular, Gaussian elimination (or LU
decomposition) could fail if a zero value
Figure 1. Structure of L and U matrices were to occur along the diagonal and were to
be a pivot. With full pivoting, however, the
Gaussian elimination (or LU decomposition)
When f is changed, the last steps can be done cannot fail because the matrix is nonsingular.
without recomputing the LU decomposition. The Cholesky decomposition can be used for
Thus, multiple right-hand sides can be com- real, symmetric, positive definite matrices. This
puted efficiently. The number of multiplica- algorithm saves on storage (divide by about 2)
tions and divisions necessary to solve for m and reduces the number of multiplications
right-hand sides is: (divide by 2), but adds n square roots.
The linear equations are solved by
1 1
Operation count ¼ n3  n þ mn2
3 3 x ¼ A1 f

The determinant is given by the product of Generally, the inverse is not used in this way
the diagonal elements of U. This should be because it requires three times more operations
calculated as the LU decomposition is per- than solving with an LU decomposition. How-
formed. If the value of the determinant is a ever, if an inverse is desired, it is calculated
very large or very small number, it can be most efficiently by using the LU decomposition
divided or multiplied by 10 to retain accuracy and then solving
in the computer; the scale factor is then accu-
mulated separately. The condition number k AxðiÞ ¼ bðiÞ
 
can be defined in terms of the singular value ðiÞ
bj ¼
0 j 6¼ i
decomposition as the ratio of the largest dii to 1 j¼i
6 Mathematics in Chemical Engineering

If

jbi j > jai j þ jci j

no pivoting is necessary. For solving two-point

boundary value problems and partial differen-
tial equations this is often the case.
Sparse matrices are ones in which the major-
ity of elements are zero. If the zero entries occur
in special patterns, efficient techniques can be
used to exploit the structure, as was done above
for tridiagonal matrices, block tridiagonal
matrices, arrow matrices, etc. These structures
typically arise from numerical analysis applied
Figure 2. Structure of tridiagonal matrices to solve differential equations. Other problems,
such as modeling chemical processes, lead to
sparse matrices but without such a neatly
defined structure — just a lot of zeros in the
Then set matrix. For matrices such as these, special
techniques must be employed: Efficient codes
 
A1 ¼ xð1Þ jxð2Þ jxð3Þ j    jxðnÞ
are available [4]. These codes usually employ a
symbolic factorization, which must be repeated
only once for each structure of the matrix. Then
Solutions of Special Matrices. Special matri- an LU factorization is performed, followed by a
ces can be handled even more efficiently. A solution step using the triangular matrices. The
tridiagonal matrix is one with nonzero entries symbolic factorization step has significant
along the main diagonal, and one diagonal overhead, but this is rendered small and
above and below the main one (see Fig. 2). insignificant if matrices with exactly the
The corresponding set of equations can then be same structure are to be used over and over [5].
written as The efficiency of a technique for solving sets
of linear equations obviously depends greatly
ai xi1 þ bi xi þ ci xiþ1 ¼ di on the arrangement of the equations and
unknowns because an efficient arrangement
The LU decomposition algorithm for solving can reduce the bandwidth, for example. Tech-
this set is niques for renumbering the equations and
unknowns arising from elliptic partial differen-
b01 ¼ b1 tial equations are available for finite difference
for k ¼ 2; n do methods [6] and for finite element methods [7].
ak ak
a0k ¼ 0 ; b0k ¼ bk  0 ck1
bk1 bk1
enddo Solutions with Iterative Methods. Sets of lin-
d01 ¼ d 1
for k ¼ 2; n do
ear equations can also be solved by using
d0k ¼ d k  a0k d 0k1 iterative methods; these methods have a rich
enddo historical background. Some of them are dis-
xn ¼ d0n =b0n
for k ¼ n  1; 1do
cussed in Chapter 8 and include Jacobi, Gauss–
xk ¼ k
d0  ck xkþ1 Seidel, and overrelaxation methods. As the
b0k speed of computers increases, direct methods
enddo
become preferable for the general case, but for
large three-dimensional problems iterative
The number of multiplications and divisions for
methods are often used.
a problem with n unknowns and m right-hand
The conjugate gradient method is an itera-
sides is
tive method that can solve a set of n linear
Operation count ¼ 2ðn  1Þ þ mð3 n  2Þ equations in n iterations. The method primarily
 Mathematics in Chemical Engineering 7

requires multiplying a matrix by a vector, which dimensionless groups govern that phenomenon.
can be done very efficiently on parallel com- In chemical reaction engineering the chemical
puters: For sparse matrices this is a viable reaction stoichiometry can be written as
method. The original method was devised by
HESTENES and STIEFEL [8]; however, more recent X
n
aij Ci ¼ 0; j ¼ 1; 2; . . . ; m
implementations use a preconditioned conju- i¼1
gate gradient method because it converges
faster, provided a good “preconditioner” can where there are n species and m reactions. Then
be found. The system of n linear equations if a matrix is formed from the coefficients aij,
which is an n  m matrix, and the rank of the
Ax ¼ f
matrix is r, there are r independent chemical
reactions. The other nr reactions can be
where A is symmetric and positive definite, is to deduced from those r reactions.
be solved. A preconditioning matrix M is
defined in such a way that the problem

Mt ¼ r 1.3. Nonlinear Algebraic Equations

is easy to solve exactly (M might be diagonal, Consider a single nonlinear equation in one
for example). Then the preconditioned conju- unknown,
gate gradient method is
f ðxÞ ¼ 0
Guess x0
Calculate r0 ¼ f  Ax0
Solve M t0 ¼ r0 ; and set p0 ¼ t0
In Microsoft Excel, roots are found by using
for k ¼ 1; n ðor until convergenceÞ Goal Seek or Solver. Assign one cell to be x, put
r T tk the equation for f(x) in another cell, and let Goal
ak ¼ Tk
pk Apk
xkþ1 ¼ xk þ ak pk Seek or Solver find the value of x making the
rkþ1 ¼ rk  ak Apk equation cell zero. In MATLAB, the process is
Solve Mtkþ1 ¼ rkþ1 similar except that a function (m-file) is defined
rT tkþ1
bk ¼ kþ1T and the command fzero(‘f’, x0) provides the
rk tk
pkþ1 ¼ tkþ1 þ bk pk solution x, starting from the initial guess x0.
test for convergence Iterative methods applied to single equations
enddo
include the successive substitution method

Note that the entire algorithm involves only xkþ1 ¼ xk þ bf ðxk Þ gðxk Þ
matrix multiplications. The generalized mini-
mal residual method (GMRES) is an iterative and the Newton–Raphson method.
method that can be used for nonsymmetric
systems and is based on a modified Gram– f ðxk Þ
xkþ1 ¼ xk 
Schmidt orthonormalization. Additional infor- df =dxðxk Þ
mation, including software for a variety of
methods, is available [9–13]. The former method converges if the derivative
In dimensional analysis if the dimensions of of g(x) is bounded [3]. The latter method
each physical variable Pj (there are n of them)
 
are expressed in terms of fundamental measure- dg 
 ðxÞ  m for jx  aj < h
dx 
ment units mj (such as time, length, mass; there
are m of them):
is based on a Taylor series of the equation about
a a
½Pj  ¼ m1 1j m2 2j    mammj the k-th iterate:

then a matrix can be formed from the aij. If the df d2 f 1

f ðxkþ1 Þ ¼ f ðxk Þ þ j k ðxkþ1  xk Þ þ 2 jxk ðxkþ1  xk Þ2 þ   
rank of this matrix is r, nr independent dx x dx 2
8 Mathematics in Chemical Engineering

The second and higher-order terms are neglec- that is fast and guaranteed to converge, if the
ted and f (xkþ1) ¼ 0 to obtain the method. root can be bracketed initially [15, p. 251].
     2 
In the method of bisection, if a root lies
df     0   
  > 0; x1  x0  ¼  f ðx Þ   b; and d f   c between x1 and x2 because f(x1) < 0 and f(x2)
dx 0 df =dxðx0 Þ 0 dx2 
x x > 0, then the function is evaluated at the center,
xc ¼ 0.5 (x1 þ x2). If f (xc) > 0, the root lies
Convergence of the Newton–Raphson method between x1 and xc. If f (xc) < 0, the root lies
depends on the properties of the first and second between xc and x2. The process is then repeated.
derivative of f(x) [3, 14]. In practice the method If f (xc) ¼ 0, the root is xc. If f (x1) > 0 and f (x2)
may not converge unless the initial guess is > 0, more than one root may exist between x1
good, or it may converge for some parameters and x2 (or no roots).
and not others. Unfortunately, when the method For systems of equations the Newton–Raph-
is nonconvergent the results look as though a son method is widely used, especially for equa-
mistake occurred in the computer program- tions arising from the solution of differential
ming; distinguishing between these situations equations.
is difficult, so careful programming and testing
are required. If the method converges the dif- f i ðfxj gÞ ¼ 0; 1  i; j  n; where fxj g ¼ ðx1 ; x2 ; . . . ; xn Þ ¼ x
ference between successive iterates is some-
thing like 0.1, 0.01, 0.0001, 108. The error Then, an expansion in several variables
(when it is known) goes the same way; the occurs:
method is said to be quadratically convergent
when it converges. If the derivative is difficult X
n
@f i
to calculate a numerical approximation may be fi ðxkþ1 Þ ¼ f i ðxk Þ þ jxk ðxkþ1  xkj Þ þ   
j¼1
@xj j

used.
 The Jacobian matrix is defined as
df  f ðxk þ eÞ  f ðxk Þ
¼
dx xk e  
@f 
Jkij ¼  i 
@x j xk
In the secant method the same formula is used
as for the Newton–Raphson method, except that
the derivative is approximated by using the and the Newton–Raphson method is
values from the last two iterates: X
n
Jkij ðxkþ1  xk Þ ¼ f i ðxk Þ

df  f ðxk Þ  f ðxk1 Þ j¼1
¼
dx  k
x xk  xk1
For convergence, the norm of the inverse of the
This is equivalent to drawing a straight line Jacobian must be bounded, the norm of
through the last two iterate values on a plot of f the function evaluated at the initial guess
(x) versus x. The Newton–Raphson method is must be bounded, and the second derivative
equivalent to drawing a straight line tangent to must be bounded [14, p. 115], [3, p. 12].
the curve at the last x. In the method of false A review of the usefulness of solution meth-
position (or regula falsi), the secant method is ods for nonlinear equations is available [16].
used to obtain xkþ1, but the previous value is This review concludes that the Newton–Raph-
taken as either xk1 or xk. The choice is made so son method may not be the most efficient.
that the function evaluated for that choice has Broyden’s method approximates the inverse
the opposite sign to f (xkþ1). This method is to the Jacobian and is a good all-purpose
slower than the secant method, but it is more method, but a good initial approximation to
robust and keeps the root between two points at the Jacobian matrix is required. Furthermore,
all times. In all these methods, appropriate the rate of convergence deteriorates for large
strategies are required for bounds on the func- problems, for which the Newton–Raphson
tion or when df/dx ¼ 0. Brent’s method com- method is better. Brown’s method [16] is
bines bracketing, bisection, and an inverse very attractive, whereas Brent’s is not worth
quadratic interpolation to provide a method the extra storage and computation. For large
 Mathematics in Chemical Engineering 9

systems of equations, efficient software is avail- is used as the initial guess and the homotopy
able [11–13]. equation is solved for x1.

Homotopy methods can be used to ensure find- @h 1;kþ1

ðx  x1;k Þ ¼ hðx1;k ; tÞ
ing the solution when the problem is especially @x
complicated. Suppose an attempt is made to
solve f (x) ¼ 0, and it fails; however, g (x) ¼ 0 Then t is increased by Dt and the process is
can be solved easily, where g(x) is some func- repeated.
tion, perhaps a simplification of f(x). Then, the In arc-length parameterization, both x and
two functions can be embedded in a homotopy t are considered parameterized by a parameter
by taking s, which is thought of as the arc length along a
curve. Then the homotopy equation is written
hðx; tÞ ¼ t f ðxÞ þ ð1  tÞgðxÞ along with the arc-length equation.

In this equation, h can be a n  n matrix for @h dx @h dt

þ ¼0
problems involving n variables; then x is a @x ds @t ds
T
 2
vector of length n. Then h (x, t) ¼ 0 can be dx dx dt
þ ¼1
ds ds ds
solved for t ¼ 0 and t gradually changes until at
t ¼ 1, h (x, 1) ¼ f (x). If the Jacobian of h with
respect to x is nonsingular on the homotopy The initial conditions are
path (as t varies), the method is guaranteed to
work. In classical methods, the interval from xð0Þ ¼ x0
t ¼ 0 to t ¼ 1 is broken up into N subdivisions. tð0Þ ¼ 0
Set Dt ¼ 1/N and solve for t ¼ 0, which is easy
by the choice of g (x). Then set t ¼ Dt and use The advantage of this approach is that it
the Newton–Raphson method to solve for x. works even when the Jacobian of h becomes
Since the initial guess is presumably pretty singular because the full matrix is rarely
good, this has a high chance of being success- singular. Illustrations applied to chemical
ful. That solution is then used as the initial engineering are available [17]. Software to per-
guess for t ¼ 2 Dt and the process is repeated by form these computations is available (called
moving stepwise to t ¼ 1. If the Newton– LOCA) [18].
Raphson method does not converge, then Dt
must be reduced and a new solution attempted.
Another way of using homotopy is to create 1.4. Linear Difference Equations
an ordinary differential equation by differenti-
ating the homotopy equation along the path Difference equations arise in chemical engi-
(where h ¼ 0). neering from staged operations, such as distil-
lation or extraction, as well as from differential
dh½xðtÞ; t @h dx @h
dt
¼ þ
@x dt @t
¼0 equations modeling adsorption and chemical
reactors. The value of a variable in the n-th
This can be expressed as an ordinary differen- stage is noted by a subscript n. For example, if
tial equation for x(t): yn,i denotes the mole fraction of the i-th species
in the vapor phase on the n-th stage of a
@h dx @h distillation column, xn,i is the corresponding
¼
@x dt @t liquid mole fraction, R the reflux ratio (ratio
of liquid returned to the column to product
If Euler’s method is used to solve this equation, removed from the condenser), and Kn,i the
a value x0 is used, and dx/dt from the above equilibrium constant, then the mass balances
equation is solved for. Then about the top of the column give

dx R 1
x1;0 ¼ x0 þ Dt ynþ1;i ¼ xn;i þ x0;i
dt Rþ1 Rþ1
10 Mathematics in Chemical Engineering

and the equilibrium equation gives where A and B are constants that must be
specified by boundary conditions of some kind.
yn;i ¼ K n;i xn;i When the equation is nonhomogeneous, the
solution is represented by the sum of a particu-
If these are combined, lar solution and a general solution to the homo-
geneous equation.
R 1
K nþ1;i xnþ1;i ¼ xn;i þ x0;i
Rþ1 Rþ1
xn ¼ xn;P þ xn;H

is obtained, which is a linear difference equa- The general solution is the one found
tion. This particular problem is quite compli- for the homogeneous equation, and the partic-
cated, and the interested reader is referred to ular solution is any solution to the non-
[19, Chap. 6]. However, the form of the differ- homogeneous difference equation. This can
ence equation is clear. Several examples are be found by methods analogous to those used
given here for solving difference equations. to solve differential equations: The method of
More complete information is available in [20]. undetermined coefficients and the method of
An equation in the form variation of parameters. The last method
applies to equations with variable coefficients,
xnþ1  xn ¼ f nþ1 too. For a problem such as

can be solved by xnþ1  f n xn ¼ 0

x0 ¼ c
X
n
xn ¼ fi
i¼1
the general solution is

Usually, difference equations are solved ana- Y

n
xn ¼ c f i1
lytically only for linear problems. When the i¼1

coefficients are constant and the equation is

linear and homogeneous, a trial solution of the This can then be used in the method of variation
form of parameters to solve the equation

xn ¼ f n xnþ1  f n xn ¼ gn

is attempted; f is raised to the power n. For

example, the difference equation 1.5. Eigenvalues
cxn1 þ bxn þ axnþ1 ¼ 0
The nn matrix A has n eigenvalues li,
i ¼ 1, . . . , n, which satisfy
coupled with the trial solution would lead to the
equation detðA  li IÞ ¼ 0

af2 þ bf þ c ¼ 0 If this equation is expanded, it can be repre-

sented as
This gives
Pn ðlÞ ¼ ðlÞn þ a1 ðlÞn1 þ a2 ðlÞn2
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi þ    þ an1 ðlÞ þ an ¼ 0
b
b2  4ac
f1;2 ¼
2a
If the matrix A has real entries then ai are real
numbers, and the eigenvalues either are real
and the solution to the difference equation is numbers or occur in pairs as complex numbers
with their complex conjugates (for definition
xn ¼ Afn1 þ Bfn2 of complex numbers, see Chap. 3). The
 Mathematics in Chemical Engineering 11

Hamilton–Cayley theorem [19, p. 127] states 2. Experimental data must be fit with a math-
that the matrix A satisfies its own characteristic ematical model. The data have experimental
equation. error, so some uncertainty exists. The
parameters in the model as well as the
Pn ðAÞ ¼ ðAÞn þ a1 ðAÞn1 þ a2 ðAÞn2 uncertainty in the determination of those
þ    þ an1 ðAÞ þ an I ¼ 0 parameters is desired.

A laborious way to find the eigenvalues of a These problems are addressed in this chap-
matrix is to solve the n-th order polynomial for ter. Section 2.2 gives the properties of polyno-
the li — far too time consuming. Instead the mials defined over the whole domain and
matrix is transformed into another form whose Section 2.3 of polynomials defined on segments
eigenvalues are easier to find. In the Givens of the domain. In Section 2.4, quadrature meth-
method and the Housholder method the matrix ods are given for evaluating an integral. Least-
is transformed into the tridiagonal form; then, squares methods for parameter estimation for
in a fixed number of calculations the eigenval- both linear and nonlinear models are given in
ues can be found [15]. The Givens method Sections 2.5. Fourier transforms to represent
requires 4 n3/3 operations to transform a real discrete data are described in Section 2.7. The
symmetric matrix to tridiagonal form, whereas chapter closes with extensions to two-dimen-
the Householder method requires half that num- sional representations.
ber [14]. Once the tridiagonal form is found, a
Sturm sequence is applied to determine the
eigenvalues. These methods are especially use-
ful when only a few eigenvalues of the matrix 2.2. Global Polynomial Approximation
are desired.
If all the eigenvalues are needed, the QR A global polynomial Pm (x) is defined over the
algorithm is preferred [21]. entire region of space
The eigenvalues of a certain tridiagonal
X
m
matrix can be found analytically. If A is a Pm ðxÞ ¼ cj xj
tridiagonal matrix with j¼0

aii ¼ p; ai;iþ1 ¼ q; aiþ1;i ¼ r; qr > 0 This polynomial is of degree m (highest power

is xm) and order m þ 1 (m þ 1 parameters {cj}).
then the eigenvalues of A are [22] If a set of m þ 1 points is given,

ip y1 ¼ f ðx1 Þ; y2 ¼ f ðx2 Þ; . . . ; ymþ1 ¼ f ðxmþ1 Þ

li ¼ p þ 2ðqrÞ1=2 cos i ¼ 1; 2; . . . ; n
nþ1

then Lagrange’s formula yields a polynomial of

This result is useful when finite difference degree m that goes through the m þ 1 points:
methods are applied to the diffusion equation.
ðx  x2 Þðx  x3 Þ    ðx  xmþ1 Þ
Pm ðxÞ ¼ y þ
ðx1  x2 Þðx1  x3 Þ    ðx1  xmþ1 Þ 1
2. Approximation and Integration
ðx  x1 Þðx  x3 Þ    ðx  xmþ1 Þ
y þ   þ
2.1. Introduction ðx2  x1 Þðx2  x3 Þ    ðx2  xmþ1 Þ 2
ðx  x1 Þðx  x2 Þ    ðx  xm Þ
y
Two types of problems arise frequently: ðxmþ1  x1 Þðxmþ1  x2 Þ    ðxmþ1  xm Þ mþ1

1. A function is known exactly at a set of points Note that each coefficient of yj is a polynomial
and an interpolating function is desired. The of degree m that vanishes at the points {xj}
interpolant may be exact at the set of points, (except for one value of j) and takes the value of
or it may be a “best fit” in some sense. 1.0 at that point, i.e.,
Alternatively it may be desired to represent
a function in some other way. Pm ðxj Þ ¼ yj j ¼ 1; 2; . . . ; m þ 1
12 Mathematics in Chemical Engineering

If the function f (x) is known, the error in the to within multiplicative constants, which can be
approximation is [23] set either by requiring the leading coefficient to
be one or by requiring the norm to be one.
jxmþ1  x1 jmþ1
jerrorðxÞj 
ðm þ 1Þ! Zb
ðmþ1Þ WðxÞP2m ðxÞdx ¼ 1
maxx1 xxmþ1 jf ðxÞj
a

The evaluation of Pm (x) at a point other than The polynomial Pm (x) has m roots in the closed
the defining points can be made with Neville’s interval a to b.
algorithm [15]. Let P1 be the value at x of The polynomial
the unique function passing through the point
(x1, y1); i.e., P1¼ y1. Let P12 be the value at x of pðxÞ ¼ c0 P0 ðxÞ þ c1 P1 ðxÞ þ    cm Pm ðxÞ
the unique polynomial passing through the
points x1 and x2. Likewise, Pijk . . . r is the minimizes
unique polynomial passing through the points
xi, xj, xk, . . . , xr. The following scheme is used: Zb
I¼ WðxÞ½f ðxÞ  pðxÞ2 dx
a

when

Zb
WðxÞ f ðxÞPj ðxÞdx
These entries are defined by using cj ¼ a
;
Wj

Piðiþ1ÞðiþmÞ ¼
Zb
ðx  xiþm ÞPiðiþ1Þðiþm1Þ þ ðxi  xÞPðiþ1Þðiþ2ÞðiþmÞ Wj ¼ WðxÞP2j ðxÞdx
xi  xiþm a

Consider P1234: the terms on the right-hand side Note that each cj is independent of m, the
of the equation involve P123 and P234. The number of terms retained in the series. The
“parents,” P123 and P234, already agree at points minimum value of I is
2 and 3. Here i ¼ 1, m ¼ 3; thus, the parents
agree at xiþ1, . . . , xiþm1 already. The formula Zb X
m
makes Pi (iþ1) . . . (iþm) agree with the function I min ¼ WðxÞ f 2 ðxÞdx  W j c2j
j¼0
at the additional points xiþm and xi. Thus, Pi a

(iþ1) . . . (iþm) agrees with the function at all the

points {xi, xiþ1, . . . , xiþm}. Such functions are useful for continuous data,
i.e., when f (x) is known for all x.
Orthogonal Polynomials. Another form of the
Typical orthogonal polynomials are given
polynomials is obtained by defining them so
in Table 1. Chebyshev polynomials are used
that they are orthogonal. It is required that Pm
in spectral methods (see Chap. 8). The last
(x) be orthogonal to Pk (x) for all k ¼ 0, . . . ,
two rows of Table 1 are widely used in the
m 1.
orthogonal collocation method in chemical
Zb engineering. The last entry (the shifted Legen-
WðxÞPk ðxÞPm ðxÞdx ¼ 0 dre polynomial as a function of x2) is defined
a by
k ¼ 0; 1; 2; . . . ; m  1

Z1
The orthogonality includes a nonnegative Wðx2 ÞPk ðx2 ÞPm ðx2 Þxa1 dx ¼ 0
weight function, W (x)  0 for all a  x  b. 0
This procedure specifies the set of polynomials k ¼ 0; 1; . . .; m  1
 Mathematics in Chemical Engineering 13

Table 1. Orthogonal polynomials [15, 23]

a b W (x) Name Recursion

relation

1 1 1 Legendre (iþ1) Piþ1¼(2 iþ1)xPi i Pi1

1 1 pffiffiffiffiffiffiffi
1 ffi
Chebyshev Tiþ1¼2xTiTi1
1x2

q1 pq
0 1 x (1x) Jacobi (p, q)
2
1 1 ex Hermite Hiþ1¼2xHi2 i Hi–1
c x
0 1 xe Laguerre (c) (iþ1) Liþ1c¼(xþ2 iþcþ1) L ci (iþc) Li1c
0 1 1 shifted Legendre
0 1 1 shifted Legendre, function of x2

where a ¼ 1 is for planar, a ¼ 2 for cylindrical, Rational polynomials are useful for approxi-
and a ¼ 3 for spherical geometry. These func- mating functions with poles and singularities,
tions are useful if the solution can be proved to which occur in Laplace transforms (see
be an even function of x. Section 4.2).
Fourier series are discussed in Section 4.1.
Rational Polynomials. Rational polynomials Representation by sums of exponentials is also
are ratios of polynomials. A rational polynomial possible [24].
Ri(iþ1) . . . (iþm) passing through m þ 1 points In summary, for discrete data, Legendre
polynomials and rational polynomials are
used. For continuous data a variety of orthogo-
yi ¼ f ðxi Þ; i ¼ 1; . . . ; m þ 1
nal polynomials and rational polynomials are
used. When the number of conditions (discrete
is
data points) exceeds the number of parameters,
Pm ðxÞ p0 þ p1 x þ    þ pm xm
then see Section 2.5.
Riðiþ1ÞðiþmÞ ¼ ¼ ;
Qn ðxÞ q 0 þ q 1 x þ    þ q n xn
mþ1¼ mþnþ1 2.3. Piecewise Approximation
An alternative condition is to make the rational Piecewise approximations can be developed
polynomial agree with the first m þ 1 terms in from difference formulas [3]. Consider a case
the power series, giving a Pade approximation, i. in which the data points are equally spaced
e.,
xnþ1  xn ¼ Dx

dk Riðiþ1ÞðiþmÞ dk f ðxÞ yn ¼ yðxn Þ

¼ k ¼ 0; . . . ; m
dxk dxk
forward differences are defined by
The Bulirsch–Stoer recursion algorithm can be Dyn ¼ ynþ1  yn

used to evaluate the polynomial: D2 yn ¼ Dynþ1  Dyn ¼ ynþ2  2ynþ1 þ yn

Then, a new variable is defined

Riðiþ1ÞðiþmÞ ¼ Rðiþ1ÞðiþmÞ
xa  x0
Rðiþ1ÞðiþmÞ  Riðiþ1Þðiþm1Þ a¼
þ Dx
Den
and the finite interpolation formula through the
 
points y0, y1, . . . , yn is written as follows:
x  xi
Den ¼ aða  1Þ 2
x  xiþm ya ¼ y0 þ aDy0 þ D y0 þ   þ
2!
  ð1Þ
Rðiþ1ÞðiþmÞ Riðiþ1Þðiþm1Þ
1 1 aða  1Þ    ða  n þ 1Þ n
Rðiþ1ÞðiþmÞ  Rðiþ1Þðiþm1Þ D y0
n!
14 Mathematics in Chemical Engineering

Keeping only the first two terms gives a straight Thus, the approximation is
line through (x0, y0)  (x1, y1); keeping the first
three terms gives a quadratic function of posi- X
NT X
NT
yðxÞ ¼ ci N i ðxÞ ¼ yðxi ÞN i ðxÞ
tion going through those points plus (x2, y2). i¼1 i¼1
The value a ¼ 0 gives x ¼ x0; a ¼ 1 gives
x ¼ x1, etc. where ci ¼ y (xi). For convenience, the trial
Backward differences are defined by functions are defined within an element by
using new coordinates:
ryn ¼ yn  yn1
x  xi
r2 yn ¼ ryn  ryn1 ¼ yn  2yn1 þ yn2 u¼
Dxi

The interpolation polynomial of order n The Dxi need not be the same from element to
through the points y0, y1, y2, . . . is element. The trial functions are defined as Ni (x)
(Fig. 3 A) in the global coordinate system and
aða þ 1Þ 2
ya ¼ y0 þ ary0 þ
2!
r y0 þ   þ NI (u) (Fig. 3 B) in the local coordinate system
aða þ 1Þ    ða þ n  1Þ n
(which also requires specification of the
n!
r y0 element). For xi < x < xiþ1

The value a ¼ 0 gives x ¼ x0; a ¼  1 gives X

NT
yðxÞ ¼ ci N i ðxÞ ¼ ci N i ðxÞ þ ciþ1 N iþ1 ðxÞ
x ¼ x1. Alternatively, the interpolation poly- i¼1

nomial of order n through the points y1, y0,

y1, . . . is because all the other trial functions are zero
there. Thus
aða  1Þ 2
ya ¼ y1 þ ða  1Þry1 þ r y1 þ   þ
2!
yðxÞ ¼ ci N I¼1 ðuÞ þ ciþ1 N I¼2 ðuÞ;
ða  1Þaða þ 1Þ    ða þ n  2Þ n
r y1 xi < x < xiþ1 ; 0 < u < 1
n!

Now a ¼ 1 gives x ¼ x1; a ¼ 0 gives x ¼ x0.

finite element method can be used for piece- Then
wise approximations [3]. In the finite element
N I¼1 ¼ 1  u; N I¼2 ¼ u
method the domain a  x  b is divided into
elements as shown in Figure 3. Each function Ni
(x) is zero at all nodes except xi; Ni (xi) ¼ 1. and the expansion is rewritten as

X
2
yðxÞ ¼ ceI N I ðuÞ ð2Þ
I¼1

x in e-th element and ci ¼ ceI within the

element e. Thus, given a set of points (xi, yi),
a finite element approximation can be made
to go through them.
Quadratic approximations can also be used
within the element (see Fig. 4). Now the trial
functions are
 
1
N I¼1 ¼ 2ðu  1Þ u 
2
N I¼2 ¼ 4uð1  uÞ ð3Þ
 
Figure 3. Galerkin finite element method–linear functions 1
N I¼3 ¼ 2u u 
A) Global numbering system; B) Local numbering system 2
 Mathematics in Chemical Engineering 15

Figure 5. Finite elements for cubic splines

A) Notation for spline knots. B) Notation for one element

The interpolating function takes on specified

values at the knots.
Figure 4. Finite elements approximation–quadratic elements
C i1 ðxi Þ ¼ C i ðxi Þ ¼ f ðxi Þ
A) Global numbering system; B) Local numbering system

Given the set of values {xi, f (xi)}, the objective

is to pass a smooth curve through those points,
and the curve should have continuous first and
The approximation going through an odd num- second derivatives at the knots.
ber of points (xi, yi) is then
C0i1 ðxi Þ ¼ C 0i ðxi Þ
X
3
C00i1 ðxi Þ ¼ C 00i ðxi Þ
yðxÞ ¼ ceI N I ðuÞ x in e  th element
I¼1

with ceI ¼ yðxi Þ; i ¼ ðe  1Þ2 þ I The formulas for the cubic spline are derived
as follows for one region. Since the function is a
in the e  th element
cubic function the third derivative is constant
and the second derivative is linear in x. This is
written as
Hermite cubic polynomials can also be used;
these are continuous and have continuous first x  xi
C 00i ðxÞ ¼ C00i ðxi Þ þ ½C 00i ðxiþ1 Þ  C 00i ðxi Þ
derivatives at the element boundaries [3]. Dxi

Splines. Splines are functions that match and integrated once to give
given values at the points x1, . . . , xNT, shown
in Figure 5, and have continuous derivatives up C0i ðxÞ ¼ C 0i ðxi Þ þ C 00i ðxi Þðx  xi Þþ
to some order at the knots, or the points ðx  xi Þ2
½C 00i ðxiþ1 Þ  C00i ðxi Þ
x2, . . . , xNT1. Cubic splines are most com- 2Dxi
mon. In this case the function is represented by
a cubic polynomial within each interval and has and once more to give
continuous first and second derivatives at the
knots. Ci ðxÞ ¼ C i ðxi Þ þ C 0i ðxi Þðx  xi Þ þ C 00i ðxi Þ
Consider the points shown in Figure 5A. The ðx  xi Þ2 ðx  xi Þ3
þ ½C 00i ðxiþ1 Þ  C 00i ðxi Þ
notation for each interval is shown in Figure 5B. 2 6Dxi
Within each interval the function is represented
as a cubic polynomial. Now

C i ðxÞ ¼ a0i þ a1i x þ a2i x2 þ a3i x3 yi ¼ Ci ðxi Þ; y0i ¼ C 0i ðxi Þ; y00i ¼ C00i ðxi Þ
16 Mathematics in Chemical Engineering

is defined so that are known at each of the knots, the first

Ci ðxÞ ¼ yi þ y0i ðx
1
 xi Þ þ y00i ðx  xi Þ2
derivatives are given by
2
1 yiþ1  yi Dxi Dxi
þ ðy00  y00i Þðx  xi Þ3 y0i ¼  y00i  y00iþ1
6Dxi iþ1 Dxi 3 6

A number of algebraic steps make the interpo-

The function itself is then known within each
lation easy. These formulas are written for the i-
element.
th element as well as the i  1-th element. Then
the continuity conditions are applied for the
first and second derivatives, and the values y0 i Orthogonal Collocation on Finite Elements. In
and y0 i1 are eliminated [15]. The result is the method of orthogonal collocation on finite
elements the solution is expanded in a polyno-
y00i1 Dxi1 þ y00i 2ðDxi1 þ Dxi Þ þ y00iþ1 Dxi mial of order NP ¼ NCOL þ 2 within each
  element [3]. The choice NCOL ¼ 1 corresponds
y  yi1 yiþ1  yi
¼ 6 i  to using quadratic polynomials, whereas
Dxi1 Dxi
NCOL ¼ 2 gives cubic polynomials. The nota-
This is a tridiagonal system for the set of {y00 i} tion is shown in Figure 6. Set the function to a
in terms of the set of {yi}. Since the continuity known value at the two endpoints
conditions apply only for i ¼ 2, . . . , NT  1,
only NT  2 conditions exist for the NT values y1 ¼ yðx1 Þ
of y0 i. Two additional conditions are needed, yNT ¼ yðxNT Þ
and these are usually taken as the value of the
second derivative at each end of the domain, and then at the NCOL interior points to each
y00 1, y00 NT. If these values are zero, the natural element
cubic splines are obtained; they can also be set
to achieve some other purpose, such as making yei ¼ yi ¼ yðxi Þ; i ¼ ðNCOL þ 1Þe þ I
the first derivative match some desired condi-
tion at the two ends. With these values taken The actual points xi are taken as the roots of the
as zero, in the natural cubic spline, an NT  2 orthogonal polynomial.
system of tridiagonal equations exists, which
is easily solved. Once the second derivatives PNCOL ðuÞ ¼ 0 gives u1 ; u2 ; . . .; uNCOL

Figure 6. Notation for orthogonal collocation on finite elements

 Residual condition; & Boundary conditions; | Element boundary, continuity
NE ¼total no. of elements.
NT ¼ (NCOL þ 1) NE þ 1
 Mathematics in Chemical Engineering 17

and then This corresponds to passing a quadratic func-

tion through three points and integrating. For an
xi ¼ xðeÞ þ Dxe uI xeI even number of intervals and an odd number of
points, 2 N þ 1, with a ¼ x0, a þ Dx ¼ x1, a þ 2
The first derivatives must be continuous at the Dx ¼ x2, . . . , a þ 2 N Dx ¼ b, Simpson’s rule
element boundaries: is obtained.
dy dy
j ¼ jx¼x
dx x¼xð2Þ dx ð2Þþ Simpson’s Rule.

Within each element the interpolation is a Zb

h
yðxÞdx ¼ ðy0 þ 4y1 þ 2y2 þ 4y3 þ 2y4
polynomial of degree NCOL þ 1. Overall the 3
a
function is continuous with continuous first
derivatives. With the choice NCOL ¼ 2, the þ    þ2y2N1 þ 4y2N þ y2Nþ1 Þ þ Oðh5 Þ

same approximation is achieved as with Her-

mite cubic polynomials. Within each pair of intervals the interpolant is
continuous with continuous derivatives, but
only the function is continuous from one pair
2.4. Quadrature to another.
If the finite element representation is used
To calculate the value of an integral, the func- (Eq. 2), the integral is
tion can be approximated by using each of the
xZiþ1 Z1 X
methods described in Section 2.3. Using the 2
yðxÞdx ¼ ceI N I ðuÞðxiþ1  xi Þdu
first three terms in Equation (1) gives xi 0
I¼1

xZ
0 þh Z1 X2 Z1  
1 1
yðxÞdx ¼ ya hda ¼ Dxi ceI N I ðuÞdu ¼ Dxi ce1 þ ce2
I¼1
2 2
x0 0 0

h 1 Dxi
¼ ðy0 þ y1 Þ  h3 y000 ðjÞ; x0  j  x0 þ h ¼ ðy þ yiþ1 Þ
2 12 2 i

This corresponds to passing a straight line Since ce1 ¼ yi and ce2 ¼ yiþ1 , the result is the
through the points (x0, y0), (x1, y1) and integrat- same as the trapezoid rule. These formulas can
ing under the interpolant. For equally spaced be added together to give linear elements:
points at a ¼ x0, a þ Dx ¼ x1, a þ 2 Dx ¼
x2, . . . , a þ N Dx ¼ xN, a þ (N þ 1) Dx ¼ b ¼ Zb X Dxe
xnþ1, the trapezoid rule is obtained. yðxÞdx ¼ ðye1 þ ye2 Þ
e
2
a

Trapezoid Rule.
If the quadratic expansion is used (Eq. 3), the
Zb endpoints of the element are xi and xiþ2, and
h

yðxÞdx ¼ y þ 2y1 þ 2y2 þ    þ 2yN þ yNþ1 Þ þ Oðh3 Þ xiþ1 is the midpoint, here assumed to be equally
2 0
a
spaced between the ends of the element:

The first five terms in Equation (1) are retained xZiþ2 Z1 X

3
and integrated over two intervals. yðxÞdx ¼ ceI N I ðuÞðxiþ2  xi Þdu
I¼1
xi 0

x0Zþ2h Z2 X
3 Z1
h
yðxÞdx ¼ ya hda ¼ ðy0 þ 4y1 þ y2 Þ ¼ Dxi ceI N I ðuÞdu
3 I¼1
x0 0 0
 
5
h ðIVÞ 1 2 1
 y ðjÞ; x0  j  x0 þ 2h ¼ Dxe ce1 þ ce2 þ ce3
90 0 6 3 6
18 Mathematics in Chemical Engineering

For many elements, with different Dxe, qua- Table 2. Gaussian quadrature points and weights*
dratic elements: N xi Wi

Zb 1 0.5000000000 0.6666666667
X Dxe
yðxÞ ¼ ðye1 þ 4ye2 þ ye3 Þ 2 0.2113248654 0.5000000000
e
6 0.7886751346 0.5000000000
a
3 0.1127016654 0.2777777778
0.5000000000 0.4444444445
If the element sizes are all the same this gives 0.8872983346 0.2777777778
Simpson’s rule. 4 0.0694318442 0.1739274226
For cubic splines the quadrature rule within 0.3300094783 0.3260725774
0.6699905218 0.3260725774
one element is 0.9305681558 0.1739274226
5 0.0469100771 0.1184634425
xZiþ1 0.2307653450 0.2393143353
1
Ci ðxÞdx ¼ Dxi ðyi þ yiþ1 Þ 0.5000000000 0.2844444444
2
xi 0.7692346551 0.2393143353
0.9530899230 0.1184634425
1
 Dx3 ðy00 þ y00iþ1 Þ
24 i i *
For a given N the quadrature points x2, x3, . . . , xNP1 are given
above. x1 ¼ 0, xNP ¼ 1. For N ¼ 1, W1 ¼ W3 ¼ 1/6 and for N  2,
For the entire interval the quadrature formula W1 ¼ WNP ¼ 0.
is
xZNT
X
1 NT1
yðxÞdx ¼ Dxi ðyi þ yiþ1 Þ polynomials give the quadrature formula
2 i¼1
x1

X
1 NT1 Z1
 Dx3 ðy00 þ y00iþ1 Þ X
n
24 i¼1 i i ex yðxÞdx ¼ W i yðxi Þ
i¼1
0

with y00 1¼ 0, y00 NT ¼ 0 for natural cubic splines.

When orthogonal polynomials are used, as (points and weights are available in mathemat-
in Equation (1), the m roots to Pm (x) ¼ 0 are ical tables) [23].
chosen as quadrature points and called points For Gauss–Hermite polynomials the quad-
{xj}. Then the quadrature is Gaussian: rature formula is
Z1 X Z1
m
2
X
n
yðxÞdx ¼ W j yðxj Þ ex yðxÞdx ¼ W i yðxi Þ
j¼1 i¼1
0 1

The quadrature is exact when y is a polynomial (points and weights are available in mathemat-
of degree 2 m  1 in x. The m weights and m ical tables) [23].
Gauss points result in 2 m parameters, chosen
to exactly represent a polynomial of degree Romberg’s method uses extrapolation tech-
2 m  1, which has 2 m parameters. The Gauss niques to improve the answer [15]. If I1 is
points and weights are given in Table 2. The the value of the integral obtained by using
weights can be defined with W (x) in the inte- interval size h ¼ Dx, I2 the value of I obtained
grand as well. by using interval size h/2, and I0 the true value
For orthogonal collocation on finite ele- of I, then the error in a method is approximately
ments the quadrature formula is hm, or
Z1 X X
NP
yðxÞdx ¼ Dxe W j yðxeJ Þ I 1  I 0 þ chm
e j¼1
0

 m
Each special polynomial has its own quad- I2  I0 þ c
h
rature formula. For example, Gauss–Legendre 2
 Mathematics in Chemical Engineering 19

Replacing the  by an equality (an approxima- sum of squares of the deviation between the
tion) and solving for c and I0 give experimental data and the theoretical equation

2m I 2  I 1 XN  
I0 ¼ yi  yðxi ; a1 ; a2 ; . . . ; aM Þ 2
2m  1 x2 ¼
i¼1
s i

This process can also be used to obtain

where yi is the i-th experimental data point for
I1, I2, . . . , by halving h each time, calculating
the value xi, yðxi ; a1 ; a2 ; . . . ; aM Þ the theoretical
new estimates from each pair, and calling them
equation at xi, s i the standard deviation of
J1, J2, . . . (i.e., in the formula above, I0 is
the i-th measurement, and the parameters
replaced with J1). The formulas are reapplied
fa1 ; a2 ; . . .; aM g are to be determined to mini-
for each pair of J’s to obtain K1, K2, . . . . The
mize x2 . The simplification is made here that
process continues until the required tolerance is
the standard deviations are all the same. Thus,
obtained.
we minimize the variance of the curve fit.

I1 I2 I3 I4 XN
½yi  yðxi ; a1 ; a2 ; . . . ; aM Þ2
J1 J2 J3 s2 ¼
i¼1
N
K1 K2
L1

Linear Least Squares. When the model is a

Romberg’s method is most useful for a low- straight line, one is minimizing
order method (small m) because significant
improvement is then possible. X
N
x2 ¼ ½yi  a  bxi 2
When the integrand has singularities, a vari- i¼1

ety of techniques can be tried. The integral may

The linear correlation coefficient r is defined by
be divided into one part that can be integrated
analytically near the singularity and another P
N
part that is integrated numerically. Sometimes ðxi  x Þðyi  y Þ
a change of argument allows analytical integra- r ¼ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
i¼1
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
PN PN
tion. Series expansion might be helpful, too. ðxi  x Þ2 ðyi  y Þ2
i¼1 i¼1
When the domain is infinite, Gauss–Legendre
or Gauss–Hermite quadrature can be used. Also and
a transformation can be made [15]. For exam-
ple, let u ¼ 1/x and then X
N
x2 ¼ ð1  r2 Þ ½yi  y 2
i¼1
Zb Z
1=a  
1 1
f ðxÞdx ¼ f du a; b > 0 where y is the average of yi values. Values of r
u2 u
a 1=b
near 1 indicate a positive correlation; r near 1
means a negative correlation, and r near zero
means no correlation. These parameters are
2.5. Least Squares easily found by using standard programs,
such as Microsoft Excel.
When fitting experimental data to a mathemat-
ical model, it is necessary to recognize that the Polynomial Regression. In polynomial regres-
experimental measurements contain error; the sion, one expands the function in a polynomial
goal is to find the set of parameters in the model in x.
that best represents the experimental data. Ref-
X
M
erence [23] gives a complete treatment relating yðxÞ ¼ aj xj1
the least-squares procedure to maximum j¼1

likelihood.
In a least-squares parameter estimation, it is The parameters are easily determined using
desired to find parameters that minimize the computer software. In Microsoft Excel, the
20 Mathematics in Chemical Engineering

data is put into columns A and B and the graph frequency is

is created as for a linear curve fit. Then add a
trendline and choose the degree of polynomial fc ¼
1
; vc ¼
p
desired. 2D D

Multiple Regression. In multiple regression, If a function y (t) is bandwidth limited to fre-

any set of functions can be used, not just quencies smaller than fc, i.e.,
polynomials.
YðvÞ ¼ 0 for v > vc
X
M
yðxÞ ¼ aj f j ðxÞ
j¼1
then the function is completely determined by
where the set of functions ff j ðxÞg is known and its samples yn. Thus, the entire information
specified. Note that the unknown parameters content of a signal can be recorded by sampling
faj g enter the equation linearly. In this case, the at a rate D1¼ 2 fc. If the function is not
spreadsheet can be expanded to have a column bandwidth limited, then aliasing occurs. Once
for x, and then successive columns for f j ðxÞ. In a sample rate D is chosen, information corre-
Microsoft Excel, choose Regression under sponding to frequencies greater than fc is simply
Tools/Data Analysis, and complete the form. aliased into that range. The way to detect this in
In addition to the actual correlation, one gets a Fourier transform is to see if the transform
the expected variance of the unknowns, which approaches zero at
fc; if not, aliasing has
allows one to assess how accurately they were occurred and a higher sampling rate is needed.
determined. Next, for N samples, where N is even

yk ¼ yðtk Þ; tk ¼ kD; k ¼ 0; 1; 2; . . . ; N  1
Nonlinear Regression. In nonlinear regres-
sion, the same procedure is followed except
that an optimization routine must be used to and the sampling rate is D; with only N values
find the minimum x2 (see Chap. 10). {yk} the complete Fourier transform Y (v)
cannot be determined. Calculate the value Y
(vn) at the discrete points
2.6. Fourier Transforms of Discrete
Data [15] vn ¼
2pn N
; n ¼  ; . . . ; 0; . . . ;
N
ND 2 2
Suppose a signal y (t) is sampled at equal X
N1
Yn ¼ yk e2pikn=N
intervals k¼0

Yðvn Þ ¼ DY n
yn ¼ yðnDÞ; n ¼ . . . ;  2;  1; 0; 1; 2; . . .

D ¼ sampling rate
The discrete inverse Fourier transform is
ðe:g:; number of samples per secondÞ

The Fourier transform and inverse transform 1XN 1

yk ¼ Y n e2pikn=N
N k¼0
are

Z1
YðvÞ ¼ yðtÞeivt dt
The fast Fourier transform (FFT) is used to
1
calculate the Fourier transform as well as the
Z1
inverse Fourier transform. A discrete Fourier
1 transform of length N can be written as the sum
yðtÞ ¼ YðvÞeivt dv
2p
1 of two discrete Fourier transforms, each of
length N/2, and each of these transforms is
(For definition of i, see Chap. 3.) The separated into two halves, each half as long.
Nyquist critical frequency or critical angular This continues until only one component is
 Mathematics in Chemical Engineering 21

left. For this reason, N is taken as a power of each of the grid points:
2, N ¼ 2p.
The vector {yj} is filled with zeroes, if need
dY 1 X N
be, to make N ¼ 2p for some p. For the dx
jn ¼ y0 e2ikpxn =L
L k¼N k
computer program, see [15, p. 381]. The stan-
dard Fourier transform takes N2 operations to
calculate, whereas the fast Fourier transform Any nonlinear term can be treated in the same
takes only N log2 N. For large N the difference way: Evaluate it in real space at N points and
is significant; at N ¼ 100 it is a factor of 15, take the Fourier transform. After processing
but for N ¼ 1000 it is a factor of 100. using this transform to get the transform of a
The discrete Fourier transform can also be new function, take the inverse transform to
used for differentiating a function; this is used obtain the new function at N points. This is
in the spectral method for solving differential what is done in direct numerical simulation of
equations. Consider a grid of equidistant turbulence (DNS).
points:

L
xn ¼ nDx; n ¼ 0; 1; 2; . . . ; 2 N  1; Dx ¼ 2.7. Two-Dimensional Interpolation
2N
and Quadrature
the solution is known at each of these grid
points {Y(xn)}. First, the Fourier transform is Bicubic splines can be used to interpolate a set
taken: of values on a regular array, f (xi, yj). Suppose
NX points occur in the x direction and NY points
1 2XN1 occur in the y direction. PRESS et al. [15] suggest
yk ¼ Yðxn Þe2ikpxn =L
2 N n¼0 computing NY different cubic splines of size NX
along lines of constant y, for example, and
The inverse transformation is storing the derivative information. To obtain
the value of f at some point x, y, evaluate each
1 X N of these splines for that x. Then do one spline of
YðxÞ ¼ y e2ikpx=L
L k¼N k size NY in the y direction, doing both the deter-
mination and the evaluation.
which is differentiated to obtain Multidimensional integrals can also be bro-
ken down into one-dimensional integrals. For
dY 1 X N
2pik 2ikpx=L
example,
¼ y e
dx L k¼N k L
Zb f Z2 ðxÞ Zb
zðx; yÞdxdy ¼ GðxÞdx;
Thus, at the grid points a f 1 ðxÞ a

fZ
2 ðxÞ
dY 1 X N
2pik 2ikpxn =L
jn ¼ y e GðxÞ ¼ zðx; yÞdx
dx L k¼N k L
f 1 ðxÞ

The process works as follows. From the solu-

tion at all grid points the Fourier transform is
obtained by using FFT {yk}. This is multiplied
by 2 pi k/L to obtain the Fourier transform of 3. Complex Variables [25–31]
the derivative:
3.1. Introduction to the Complex Plane
2pik
y0k ¼ yk A complex number is an ordered pair of real
L
numbers, x and y, that is written as
The inverse Fourier transform is then taken by
using FFT, to give the value of the derivative at z ¼ x þ iy
22 Mathematics in Chemical Engineering

The variable i is the imaginary unit which has Since the arctangent repeats itself in multiples
the property of p rather than 2 p, the argument must be
defined carefully. For example, the u given
i2 ¼ 1 above could also be the argument of  (x þi y).
The function z ¼ cos u þ i sin u obeys |z| ¼ |cos u
The real and imaginary parts of a complex þ i sin u| ¼ 1.
number are often referred to: The rules of equality, addition, and multipli-
cation are
Re ðzÞ ¼ x; Im ðzÞ ¼ y
z1 ¼ x1 þ i y1 ; z2 ¼ x2 þ i y2
A complex number can also be represented Equality:
graphically in the complex plane, where the
z1 ¼ z2 if and only if x1 ¼ x2 and y1 ¼ y2
real part of the complex number is the abscissa
and the imaginary part of the complex number Addition:
is the ordinate (see Fig. 7). z1 þ z2 ¼ ðx1 þ x2 Þ þ iðy1 þ y2 Þ
Another representation of a complex number Multiplication:
is the polar form, where r is the magnitude and
z1 z2 ¼ ðx1 x2  y1 y2 Þ þ iðx1 y2 þ x2 y1 Þ
u is the argument.
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi The last rule can be remembered by
r ¼ jx þ i yj ¼ x2 þ y2 ; u ¼ argðx þ i yÞ
using the standard rules for multiplication,
Write keeping the imaginary parts separate, and
using i2 ¼  1. In the complex plane, addition
z ¼ x þ i y ¼ rðcosu þ i sinuÞ
is illustrated in Figure 8. In polar form, multi-
plication is
so that
z1 z2 ¼ r1 r2 ½cosðu1 þ u2 Þ þ i sinðu1 þ u2 Þ

x ¼ r cos u; y ¼ r sin u
The magnitude of z1 þ z2 is bounded by
and
jz1
z2 j  jz1 j þ jz2 j and jz1 j  jz2 j  jz1
z2 j
y
u ¼ arctan
x
as can be seen in Figure 8. The magnitude and
arguments in multiplication obey

jz1 z2 j ¼ jz1 jjz2 j; argðz1 z2 Þ ¼ arg z1 þ arg z2

The complex conjugate is z ¼ x i y when z ¼

x þi y and |z| ¼ |z|, arg z ¼  arg z

Figure 7. The complex plane Figure 8. Addition in the complex plane

 Mathematics in Chemical Engineering 23

For complex conjugates then The exponential function is


z z ¼ jzj2 ez ¼ ex ðcos y þ i sin yÞ

The reciprocal is Thus,

  
1 z 1 1 z ¼ rðcos u þ i sin uÞ
¼ ¼ ðcos u  i sin uÞ; arg ¼ arg z
z jzj2 r z
can be written
Then
z ¼ reiu
z1 x1 þ i y1 x2  i y2
¼ ¼ ðx1 þ i y1 Þ 2
z2 x2 þ i y2 x2 þ y22 and
x1 x2 þ y 1 y2 x2 y1  x1 y2
¼ þi
x22 þ y22 x22 þ y22 jez j ¼ ex ; arg ez ¼ yðmod 2pÞ

and The exponential obeys

z1 r1
¼ ½cosðu1  u2 Þ þ i sinðu1  u2 Þ ez 6¼ 0 for every finite z
z2 r2

and is periodic with period 2 p:

3.2. Elementary Functions ezþ2p i ¼ ez

Properties of elementary functions of complex Trigonometric functions can be defined by

variables are discussed here [32]. When the using
polar form is used, the argument must be
specified because the same physical angle eiy ¼ cos y þ i sin y; and eiy ¼ cos y  i sin y
can be achieved with arguments differing by
2 p. A complex number taken to a real power Thus,
obeys ei y þ ei y
cos y ¼ ¼ cosh i y
2
u ¼ zn ; jzn j ¼ jzjn ; arg ðzn Þ ¼ n arg zðmod2pÞ
ei y  ei y
sin y ¼ ¼ i sinh i y
u ¼ zn ¼ rn ðcos nu þ i sin nuÞ 2i

Roots of a complex number are complicated by The second equation follows from the defini-
careful accounting of the argument tions
ez þ ez ez  ez
cosh z ; sinh z
z ¼ w1=n with w ¼ Rðcos Q þ i sin QÞ; 0  Q  2p 2 2

then The remaining hyperbolic functions are

(   sinh z 1
Q 2p tanh z ; coth z
zk ¼ R1=n cos þ ðk  1Þ cosh z tanh z
n n
  1 1
sech z ; csch z
Q 2p cosh z sinh z
þi sin þ ðk  1Þ g
n n
The circular functions with complex argu-
such that ments are defined

ðzk Þn ¼ w for every k ei z þ ei z ei z  ei z

cos z ¼ ; sin z ¼ ;
2 2
z ¼ rðcos u þ i sin uÞ
sin z
tan z ¼
rn ¼ R; nu ¼ Q ðmod 2pÞ cos z
24 Mathematics in Chemical Engineering

and satisfy is always true, but

sinðzÞ ¼ sin z; cosðzÞ ¼ cos z lnðz1 z2 Þ ¼ ln z1 þ ln z2

sinði zÞ ¼ i sinh z; cosði zÞ ¼ cosh z

holds only for some determinations of the
logarithms. The principal determination of
All trigonometric identities for real, circular the argument can be defined as  p < arg  p.
functions with real arguments can be extended
without change to complex functions of com-
plex arguments. For example, 3.3. Analytic Functions of a Complex
Variable
sin2 z þ cos2 z ¼ 1;

sinðz1 þ z2 Þ ¼ sin z1 cos z2 þ cos z1 sin z2 Let f (z) be a single-valued continuous function
of z in a domain D. The function f (z) is
differentiable at the point z0 in D if
The same is true of hyperbolic functions. The
absolute boundaries of sin z and cos z are not f ðz0 þ hÞ  f ðz0 Þ
lim
bounded for all z. h!0 h
Trigonometric identities can be defined by
using exists as a finite (complex) number and is
independent of the direction in which h tends
eiu ¼ cos u þ i sin u to zero. The limit is called the derivative, f 0 (z0).
The derivative now can be calculated with h
approaching zero in the complex plane, i.e.,
For example, anywhere in a circular region about z0. The
function f(z) is differentiable in D if it is dif-
eiðaþbÞ ¼ cosða þ bÞ þ i sinða þ bÞ ferentiable at all points of D; then f(z) is said to
¼ eia eib ¼ ðcos a þ i sin aÞ be an analytic function of z in D. Also, f(z) is
ðcos b þ i sin bÞ
analytic at z0 if it is analytic in some e neigh-
borhood of z0. The word analytic is sometimes
¼ cos a cos b  sin a sin b replaced by holomorphic or regular.
þ i ðcos a sin b þ cos b sin aÞ The Cauchy–Riemann equations can be used
to decide if a function is analytic. Set
Equating real and imaginary parts gives
f ðzÞ ¼ f ðx þ i yÞ ¼ uðx; yÞ þ i vðx; yÞ

cosða þ bÞ ¼ cos a cos b  sin a sin b

sinða þ bÞ ¼ cos a sin b þ cos b sin a Theorem [30, p. 51]. Suppose that f (z) is
defined and continuous in some neighborhood
The logarithm is defined as of z ¼ z0. A necessary condition for the exis-
tence of f 0 (z0) is that u (x, y) and v (x, y) have
ln z ¼ lnjzj þ i arg z first-order partials and that the Cauchy–Rie-
mann conditions (see below) hold.
and the various determinations differ by multi- @u @v @u @v
¼ and ¼ at z0
ples of 2 pi. Then, @x @y @y @x

eln z ¼ z
Theorem [30, p. 61]. The function f (z) is
lnðez Þ  z 0ðmod 2 piÞ analytic in a domain D if and only if u and v
are continuously differentiable and satisfy the
Also, Cauchy–Riemann conditions there.
If f1(z) and f2(z) are analytic in domain D,
lnðz1 z2 Þ  ln z1  ln z2 0ðmod 2 piÞ then a1f1(z) þ a2f2(z) is analytic in D for any
 Mathematics in Chemical Engineering 25

(complex) constants a1, a2. Define za ¼ ea ln z for complex constant a. If the

determination is p < arg z  p, then za is
f 1 ðzÞ þ f 2 ðzÞ is analytic in D
analytic on the complex plane with a cut on the
f 1 ðzÞ=f 2 ðzÞ is analytic in D except where f 2 ðzÞ ¼ 0 negative real axis. If a is an integer n, then e2pin
¼ 1 and zn has the same limits approaching the
An analytic function of an analytic function is cut from either side. The function can be made
analytic. If f (z) is analytic, f0 (z) 6¼ 0 in D, f (z1) continuous across the cut and the function is
6¼ f (z2) for z16¼ z2, then the inverse function g analytic there, too. If a ¼ 1/n where n is an
(w) is also analytic and integer, then
1
g0 ðwÞ ¼ where w ¼ f ðzÞ;
f 0 ðzÞ z1=n ¼ eðln zÞ=n ¼ jzjl=n eiðarg zÞ=n
g ðwÞ ¼ g ½f ðzÞ ¼ z
So w ¼ z1/n has n values, depending on the
Analyticity implies continuity but the con-
choice of argument.
verse is not true: z ¼ x i y is continuous but,
because the Cauchy–Riemann conditions are
not satisfied, it is not analytic. An entire func- Laplace Equation. If f (z) is analytic, where
tion is one that is analytic for all finite values of
f ðzÞ ¼ uðx; yÞ þ i vðx; yÞ
z. Every polynomial is an entire function.
Because the polynomials are analytic, a ratio
of polynomials is analytic except when the the Cauchy–Riemann equations are satisfied.
denominator vanishes. The function f (z) ¼ | Differentiating the Cauchy–Riemann equations
z2| is continuous for all z but satisfies the gives the Laplace equation:
Cauchy–Riemann conditions only at z ¼ 0.
@2 u @2 v @2 v @2 u
Hence, f 0 (z) exists only at the origin, and |z|2 ¼ ¼ ¼  2 or
@x2 @x@y @y@x @y
is nowhere analytic. The function f (z) ¼ 1/z is
analytic except at z ¼ 0. Its derivative is  1/z2, @2 u @2 u
þ ¼0
@x2 @y2
where z 6¼ 0. If ln z ¼ ln |z| þ i arg z in the cut
domain  p < arg z  p, then f (z) ¼ 1/ln z
Similarly,
is analytic in the same cut domain, except at
z ¼ 1, where log z ¼ 0. Because ez is analytic
@2 v @2 v
and
i z are analytic, e
iz is analytic and linear þ
@x2 @y2
¼0
combinations are analytic. Thus, the sine and
cosine and hyperbolic sine and cosine are ana- Thus, general solutions to the Laplace equation
lytic. The other functions are analytic except can be obtained from analytic functions [30, p.
when the denominator vanishes. 60]. For example,
The derivatives of the elementary functions
are 1
ln
jz  z0 j
d z d n
e ¼ ez ; z ¼ n zn1
dz dz
d 1 d is analytic so that a solution to the Laplace
ðln zÞ ¼ ; sin z ¼ cos z;
dz z dz equation is
d
cos z ¼ sin z
dz ln ½ðx  aÞ2 þ ðy  bÞ2 1=2

In addition, A solution to the Laplace equation is called a

d dg df harmonic function. A function is harmonic if,
ðf gÞ ¼ f þg
dz dz dz and only if, it is the real part of an analytic
d d f dg function. The imaginary part is also harmonic.
f ½g ðzÞ ¼
dz dg dz Given any harmonic function u, a conjugate
d dw d dw harmonic function v can be constructed such
sin w ¼ cos w ; cos w ¼ sin w
dz dz dz dz that f ¼ u þi v is locally analytic [30, p. 290].
26 Mathematics in Chemical Engineering

Maximum Principle. If f (z) is analytic in a Also if s (t) is the arc length on C and l (C) is
domain D and continuous in the set consisting the length of C
of D and its boundary C, and if | f (z)|  M on C,
 
then | f (z)| < M in D unless f (z) is a constant Z



 f ðzÞdz  max jf ðzÞjlðCÞ
[30, p. 134].  
  z2C
C

3.4. Integration in the Complex Plane and

 
Let C be a rectifiable curve in the complex Z  Z Z1
 
 f ðzÞdz  jf ðzÞjjdzj ¼ jf ½z ðtÞjdsð tÞ
plane 



C C 0

C : z ¼ zðtÞ; 0  t  1
Cauchy’s Theorem [25, 30, p. 111]. Suppose f
where z (t) is a continuous function of bounded (z) is an analytic function in a domain D and C
variation; C is oriented such that z1 ¼ z (t1) is a simple, closed, rectifiable curve in D such
precedes the point z2 ¼ z (t2) on C if and only if that f (z) is analytic inside and on C. Then
t1 < t2. Define I
f ðzÞdz ¼ 0 ð4Þ
C
Z Z1
f ðzÞdz ¼ f ½z ðtÞ dz ðtÞ
C 0 If D is simply connected, then Equation (4)
holds for every simple, closed, rectifiable curve
C in D. If D is simply connected and if a and b
The integral is linear with respect to the inte- are any two points in D, then
grand:
Zb
R
C ½a1 f 1 ðzÞ þ a2 f 2 ðzÞdz f ðzÞdz
R R a
¼ a1 C f 1 ðzÞdz þ a2 C f 2 ðzÞdz

is independent of the rectifiable path joining a

The integral is additive with respect to the and b in D.
path. Let curve C2 begin where curve C1 ends
and C1 þ C2 be the path of C1 followed by C2. Cauchy’s Integral. If C is a closed contour
Then, such that f (z) is analytic inside and on C, z0 is a
Z Z Z point inside C, and z traverses C in the coun-
f ðzÞdz ¼ f ðzÞdz þ f ðzÞdz terclockwise direction,
C 1 þC 2 C1 C2
I
1 f ðzÞ
f ðz0 Þ ¼ dz
2pi z  z0
Reversing the orientation of the path replaces C

the integral by its negative: I

1 f ðzÞ
f 0 ðz0 Þ ¼ dz
Z Z
2pi ðz  z0 Þ2
C
f ðzÞdz ¼  f ðzÞdz
C C
Under further restriction on the domain [30,
p. 127],
If the path of integration consists of a finite
number of arcs along which z (t) has a contin- I
m! f ðzÞ
uous derivative, then f ðmÞ ðz0 Þ ¼ dz
2pi ðz  z0 Þmþ1
C

Z Z1
f ðzÞdz ¼ f ½zðtÞz0 ðtÞdt Power Series. If f (z) is analytic interior to a
C 0 circle |z  z0| < r0, then at each point inside the
 Mathematics in Chemical Engineering 27

circle the series integral in the positive direction around a path

containing no other singular points.
X
1 ðnÞ
f ðz0 Þ If f (z) is defined and analytic in the exterior
f ðzÞ ¼ f ðz0 Þ þ ðz  z0 Þn
n¼1
n! |z  z0| > R of a circle, and if
 
1 1
converges to f (z). This result follows from vðzÞ ¼ f z0 þ obtained by z ¼
z z  z0
Cauchy’s integral. As an example, ez is an
entire function (analytic everywhere) so that has a removable singularity at z ¼ 0, f (z) is
the MacLaurin series analytic at infinity. It can then be represented
by a Laurent series with nonpositive powers of
X
1 n
z
z  z 0.
ez ¼ 1 þ If C is a closed curve within which and on
n¼1
n!
which f (z) is analytic except for a finite number
represents the function for all z. of singular points z1, z2, . . . , zn interior to the
Another result of Cauchy’s integral formula region bounded by C, then the residue theorem
is that if f (z) is analytic in an annulus R, r1 < |z states
 z0| < r2, it is represented in R by the Laurent I
series f ðzÞdz ¼ 2pið%1 þ %2 þ   %n Þ
C
X
1
f ðzÞ ¼ An ðz  z0 Þn ; r1 < jz  z0 j  r 2
n¼1 where %n denotes the residue of f (z) at zn.
The series of negative powers in
where Equation (6) is called the principal part of
f (z). If the principal part has an infinite number
Z
An ¼
1 f ðzÞ
dz;
of nonvanishing terms, the point z0 is an
2pi ðz  z0 Þnþ1 essential singularity. If Am 6¼ 0, An ¼ 0 for
C

n ¼ 0;
1;
2; . . . ;
all m < n, then z0 is called a pole of order m. It is
a simple pole if m ¼ 1. In such a case,
and C is a closed curve counterclockwise in R.
A1 X
1
f ðzÞ ¼ þ An ðz  z0 Þn
z  z0 n¼0
Singular Points and Residues [33, p. 159,
30, p. 180]. If a function in analytic in every
If a function is not analytic at z0 but can be made
neighborhood of z0, but not at z0 itself, then z0 is
so by assigning a suitable value, then z0 is a
called an isolated singular point of the function.
removable singular point.
About an isolated singular point, the function
When f (z) has a pole of order m at z0,
can be represented by a Laurent series.

A2 A1 fðzÞ ¼ ðz  z0 Þm f ðzÞ; 0 < jz  z0 j < r0

f ðzÞ ¼    þ 2
þ þ A0
ðz  z0 Þ z  z0 ð5Þ
þA1 ðz  z0 Þ þ    0 < jz  z0 j  r0 has a removable singularity at z0. If f (z0) ¼
Am, then f (z) is analytic at z0. For a simple
In particular, pole,
I
1 A1 ¼ fðz0 Þ ¼ lim ðz  z0 Þ f ðzÞ
A1 ¼ f ðzÞdz z!z0
2pi
C

Also | f (z)| ! 1 as z ! z0 when z0 is a pole. Let

where the curve C is a closed, counterclockwise the function p (z) and q (z) be analytic at z0,
curve containing z0 and is within the neighbor- where p (z0) 6¼ 0. Then
hood where f (z) is analytic. The complex
number A1 is the residue of f (z) at the isolated pðzÞ
f ðzÞ ¼
singular point z0; 2 pi A1 is the value of the qðzÞ
28 Mathematics in Chemical Engineering

has a simple pole at z0 if, and only if, q (z0) ¼ 0

and q0 (z0) 6¼ 0. The residue of f (z) at the simple
pole is

pðz0 Þ
A1 ¼
q0 ðz0 Þ

If q(i1)(z0) ¼ 0, i ¼ 1, . . . , m, then z0 is a pole

of f(z) of order m.

Branch [33, p. 163]. A branch of a multiple-

valued function f(z) is a single-valued function
that is analytic in some region and whose value
at each point there coincides with the value of
f (z) at the point. A branch cut is a boundary that
is needed to define the branch in the greatest
possible region. The function f(z) is singular
along a branch cut, and the singularity is not
Figure 9. Integration in the complex plane
isolated. For example,
 
pffiffi u u
z1=2 ¼ f 1 ðzÞ ¼ r cos þ i sin
2 2
An extension of the Cauchy integral for-
 p < u < p; r > 0
mula is useful with Laplace transforms. Let
the curve C be a straight line parallel to the
is double valued alongptheffiffi negative real axis. imaginary axis and z0 be any point to the right
pffiffifunction tends to r i when u ! p and to
The of that (see Fig. 9). A function f (z) is of order
 r i when u !  p; the function has no limit zk as |z| ! 1 if positive numbers M and r0
as z !  r (r > 0). The ray u ¼ p is a branch cut. exist such that
Analytic Continuation [33, p. 165]. If f1(z) is jzk f ðzÞj < M when jzj > r0; i:e:;
analytic in a domain D1 and domain D contains
D1, then an analytic function f (z) may exist that jf ðzÞj < Mjzjk for jzjsufficiently large

equals f1(z) in D1. This function is the analytic

continuation of f1(z) onto D, and it is unique.
For example, Theorem [33, p. 167]. Let f (z) be analytic
X
1
when R (z)  g and O (zk) as |z| ! 1 in
f 1 ðzÞ ¼ zn ; jzj < 1 that half-plane, where g and k are real constants
n¼0
and k > 0. Then for any z0 such that R (z0)
>g
is analytic in the domain D1 : |z| < 1. The series
diverges for other z. Yet the function is the Z
gþib
MacLaurin series in the domain 1 f ðzÞ
f ðz0 Þ ¼  lim dz;
2 p i b!1 z  z0
gib
1
f 1 ðzÞ ¼ ; jzj < 1
1z
i.e., integration takes place along the line
x ¼ g.
Thus,

1
f 1 ðzÞ ¼
1z 3.5. Other Results
is the analytic continuation onto the entire z Theorem [32, p. 84]. Let P (z) be a polynomial
plane except for z ¼ 1. of degree n having the zeroes z1, z2, . . . , zn
 Mathematics in Chemical Engineering 29

and let p be the least convex polygon contain- 4. Integral Transforms [34–39]
ing the zeroes. Then P0 (z) cannot vanish any-
where in the exterior of p. 4.1. Fourier Transforms
If a polynomial has real coefficients, the
roots are either real or form pairs of complex Fourier Series [40]. Let f (x) be a function that
conjugate numbers. is periodic on  p < x < p. It can be expanded
The radius of convergence R of the Taylor in a Fourier series
series of f (z) about z0 is equal to the distance
from z0 to the nearest singularity of f (z). a0 X 1
f ðxÞ ¼ þ ðan cos n x þ bn sin n xÞ
2 n¼1

Conformal Mapping. Let u (x, y) be a har-

monic function. Introduce the coordinate trans-
formation where

x ¼ ^xðj; hÞ; y ¼ ^yðj; hÞ Zp Zp

1 1
a0 ¼ f ðxÞdx; an ¼ f ðxÞcos n x dx;
p p
p p

It is desired that Zp
1
bn ¼ f ðxÞsin n x dx
p
U ðj; hÞ ¼ u½^xðj; hÞ; ^yðj; hÞ p

be a harmonic function of j and h. The values {an} and {bn} are called the finite
cosine and sine transform of f, respectively.
Theorem [30, p. 284]. The transformation Because

z ¼ f ðzÞ ð6Þ 1
cos n x ¼ ðeinx þ einx Þ
2
1 inx
takes all harmonic functions of x and y into and sin n x ¼
2i
ðe  einx Þ
harmonic functions of j and h if and only if
either f (z) or f  (z) is an analytic function of
z ¼ j þi h. the Fourier series can be written as
Equation (6) is a restriction on the transfor-
mation which ensures that X
1
f ðxÞ ¼ cn einx
n¼1

@2 u @2 u @2 U @2 U
if þ ¼ 0 then þ 2 ¼0
@x2 @y2 @z2 @h where
8
Such a mapping with f (z) analytic and f 0 (z) 6¼ 0 >
>
1
< 2 ðan þ i bn Þ for n  0
is a conformal mapping. cn ¼
>
>
If Laplace’s equation is to be solved in the : 1 ðan  i bn Þ for n < 0
2
region exterior to a closed curve, then the point
at infinity is in the domain D. For flow in a long
channel (governed by the Laplace equation) the and
inlet and outlet are at infinity. In both cases the
transformation Zp
1
cn ¼ f ðxÞeinx dx
2p
azþb p
z¼
z  z0
If f is real
takes z0 into infinity and hence maps D into a 
bounded domain D . cn ¼ cn :
30 Mathematics in Chemical Engineering

If f is continuous and piecewise continuously transform is defined as

differentiable
Z1
X
1 F½f  ^f ðvÞ ¼ f ðxÞeivx dx
f 0 ðxÞ ¼ ði nÞcn einx 1
1

This integral converges if

If f is twice continuously differentiable
Z1
X
1
jf ðxÞjdx
00
f ðxÞ ¼ ðn2 Þcn einx
1 1

does. The inverse transformation is

Inversion. The Fourier series can be used
to solve linear partial differential equations 1
Z1
f ðxÞ ¼ ^f ðvÞeivx dv
with constant coefficients. For example, in 2p
the problem 1

@T @ 2 T
If f (x) is continuous and piecewise continu-
¼ 2 ously differentiable,
@t @x
T ðx; 0Þ ¼ f ðxÞ
Z1
T ðp; tÞ ¼ T ðp; tÞ f ðxÞeivx dx
1

Let
converges for each v, and
X
1
T¼ cn ðtÞeinx lim f ðxÞ ¼ 0
1 x!
1

Then, then
 
X
1 X
1 df
dcn inx 2 inx F ¼  ivF ½f 
e ¼ cn ðtÞðn Þe dx
1
dt 1

If f is real F [ f ( v)] ¼ F [ f (v) ]. The real part

Thus, cn (t) satisfies is an even function of v and the imaginary part
is an odd function of v.
dcn 2
¼ n2 cn ; or cn ¼ cn ð0Þen t A function f (x) is absolutely integrable if
dt
the improper integral
Let cn(0) be the Fourier coefficients of the
Z1
initial conditions: jf ðxÞjdx
1
X
1
f ðxÞ ¼ cn ð0Þeinx
1 has a finite value. Then the improper integral

The formal solution to the problem is Z1

f ðxÞdx
1
X
1
2
T¼ cn ð0Þen t einx
1 converges. The function is square integrable if

Z1
Fourier Transform [40]. When the function jf ðxÞj2 dx
f(x) is defined on the entire real line, the Fourier 1
 Mathematics in Chemical Engineering 31

has a finite value. If f (x) and g (x) are square This is also the total power in a signal, which
integrable, the product f (x) g (x) is absolutely can be computed in either the time or the
integrable and satisfies the Schwarz inequality: frequency domain. Also

1 2 Z1 Z1
Z  ^f ðvÞ^g ðvÞdv ¼ 2 p
  f ðxÞg ðxÞdx
 f ðxÞ g ðxÞdx
 
  1 1
1

Z1 Z1 Fourier transforms can be used to solve

 jf ðxÞj2 dx jg ðxÞj2 dx differential equations too. Then it is necessary
1 1
to find the inverse transformation. If f (x) is
square integrable, the Fourier transform of its
The triangle inequality is also satisfied: Fourier transform is 2 p f ( x), or
Z1
1
f ðxÞ ¼ F½^f ðvÞ ¼ ^f ðvÞeivx dv
81 )1=2 81 )1=2 8 Z1 )1=2 2p
<Z <Z < 1
jf þ gj2 dx  jf j2 dx jgj2 dx
: : : Z1 Z1
1
1 1 1
¼ f ðxÞeivx dxeivx dv
2p
11

A sequence of square integrable functions 1 1

f ðxÞ ¼ F½Ff ðxÞ or f ðxÞ ¼ F½F½f 
2p 2p
fn(x) converges in the mean to a square integra-
ble function f(x) if
Properties of Fourier Transforms [[15], 4, p. 324].
Z1
 
lim jf ðxÞ  f n ðxÞj2 dx ¼ 0 df
n!1 F ¼ i v F½f  ¼ i v^f
1 dx
d d ^
F½i x f ðxÞ ¼ F ½f  ¼ f
The sequence also satisfies the Cauchy criterion dv dv
1 v 
F½f ða x  bÞ ¼ eivb=a ^f
Z1 jaj a
lim
n!1
jf n  f m j2 dx ¼ 0 F½eicx f ðxÞ ¼ ^f ðv þ cÞ
m!1
1
1
F½cos v0 x f ðxÞ ¼ ½^f ðv þ v0 Þ þ ^f ðv  v0 Þ
2
Theorem [40, p. 307]. If a sequence of square F½sin v0 x f ðxÞ ¼
1 ^
½f ðv þ v0 Þ  ^f ðv  v0 Þ
integrable functions fn (x) converges to a func- 2i

tion f (x) uniformly on every finite interval F½eiv0 x f ðxÞ ¼ ^f ðv  v0 Þ

a  x  b, and if it satisfies Cauchy’s criterion,
then f (x) is square integrable and fn (x) con- If f (x) is real, then f ( v) ¼ ^f  (v). If f (x) is
verges to f (x) in the mean. imaginary, then ^f ( v) ¼  ^f  (v). If f (x) is
even, then ^f (v) is even. If f (x) is odd, then
Theorem (Riesz–Fischer) [40, p. 308]. To every ^f (v) is odd. If f (x) is real and even, then (v) is
sequence of square integrable functions fn (x) real and even. If f (x) is real and odd, then ^f (v)
that satisfy Cauchy’s criterion, there corre- is imaginary and odd. If f (x) is imaginary and
sponds a square integrable function f (x) such even, then ^f (v) is imaginary and even. If f (x) is
that fn (x) converges to f (x) in the mean. Thus, imaginary and odd, then ^f (v) is real and odd.
the limit in the mean of a sequence of functions
is defined to within a null function. Convolution [40, p. 326].
Square integrable functions satisfy the Par-
Z1
seval equation. f  hðx0 Þ f ðx0  xÞ h ðxÞ dx
1

Z1 Z1 Z1
1
j^f ðvÞj2 dv ¼ 2 p jf ðxÞj2 dx ¼ eivx0 ^f ðvÞ^h ðvÞdv
2p
1 1 1
32 Mathematics in Chemical Engineering

Theorem. The product The finite Fourier sine and cosine transforms
are
^f ðvÞ^h ðvÞ
Zp
2
f s ðnÞ ¼ Fns ½f  ¼ f ðxÞsin n xdx;
p
0
is the Fourier transform of the convolution
product f  h. The convolution permits finding n ¼ 1; 2; . . . ;

inverse transformations when solving differen- Zp

2
f c ðnÞ ¼ Fnc ½f  ¼ f ðxÞcos n xdx
tial equations. To solve p
0

n ¼ 0; 1; 2; . . .
@T @ 2 T
¼ 2 X
1
@t @x f ðxÞ ¼ f s ðnÞ sin n x;
T ðx; 0Þ ¼ f ðxÞ;  1 < x < 1 n¼1

1 X1
T bounded f ðxÞ ¼ f ð0Þ þ f c ðnÞ cos n x
2 c n¼1

take the Fourier transform

They obey the operational properties
dT^  
þ v2 T^ ¼ 0 d2 f
dt Fns ¼ n2 Fns ½f 
dx 2
^
Tðv; 0Þ ¼ ^f ðvÞ
2n
þ ½f ð0Þ  ð1Þn f ðpÞ
p

The solution is
f, f 0 are continuous, f 00 is piecewise continuous
^ tÞ ¼ ^f ðvÞev t on 0  x  p.
2
Tðv;

The inverse transformation is

Z1
1
eivx ^f ðvÞev t dv
2
T ðx; tÞ ¼
2p
1

Because and f1 is the extension of f, k is a constant

 
1
ev t ¼ F pffiffiffiffiffiffiffiffiffi ex =4t
2 2

4pt

the convolution integral can be used to write the

solution as
Z1
1 2
=4t
T ðx; tÞ ¼ pffiffiffiffiffiffiffiffiffi f ðyÞeðxyÞ dy Also,
4pt
1
 
d2 f 2df
Fnc 2
¼ n2 Fnc ½f   ð0Þ
dx p dx
Finite Fourier Sine and Cosine Transform
[41]. In analogy with finite Fourier transforms 2df
þ ð1Þn ðpÞ
p dx
(on  p to p) and Fourier transforms (on  1  
to þ 1), finite Fourier sine and cosine trans- 1 1
f c ðnÞ cos n k ¼ Fnc f 2 ðx  kÞ þ f 2 ðx þ kÞ
2 2
forms (0 to p) and Fourier sine and cosine
transforms (on 0 to þ 1) can be defined. f c ðnÞ ð1Þn ¼ Fnc ½f ðp  xÞ
 Mathematics in Chemical Engineering 33

and f2 is the extension of f. Table 3. Finite sine transforms

Rp
f s ðnÞ ¼ F ðxÞsin n xdx ðn ¼ 1; 2 ; . . .Þ F (x) (0<x<p)
0

(1)nþ1 fs (n) F (px)

1 ðpxÞ
n p

ð1Þnþ1 x
n p

Also, 1ð1Þn
1
n

  ðp  cÞ x ðx  cÞ
df
p
n2 sin n cð0 < c < pÞ
Fns ¼ nFnc ½f  c ðp  xÞ ðx  cÞ
dx 
x ðx < cÞ
  p
n cos n cð0 < c < pÞ
df 2 2 px ðxÞ
Fnc ¼ nFns ½f   f ð0Þ þ ð1Þn f ðpÞ
dx p p p2 ð1Þn1 n

n 2½1ð1Þ
n3 x2
 2

pð1Þn n63  pn x3
When two functions F (x) and G (x) are defined
on the interval  2 p < x < 2 p, the function ½1  ð1Þn ec p  ecx
n
n2 þc2

n sinh cðpxÞ
n2 þc2
Zp sinh c p

F ðxÞ G ðxÞ ¼ f ðx  yÞ g ðyÞdy n
ðjkj 6¼ 0; 1; 2; . . .Þ sin kðpxÞ
n2 k2 sin k p
p

0 ðn 6¼ mÞ; f s ðmÞ ¼ p2 sin m x (m ¼ 1,2, . . . )

is the convolution on  p < x < p. If F and G n
½1  ð1Þnþm ; cos k x (|k| 6¼ 1,2, . . . )
are both even or both odd, the convolution is n2  m2
even; it is odd if one function is even and the n
n2 k2
n
½1  ð1Þ cos k x
other odd. If F and G are piecewise continuous cos m x (m ¼ 1,2, . . . )
ðn 6¼ mÞ; f s ðmÞ ¼ 0
on 0  x  p, then
The material is reproduced with permission of McGrawHill, Inc.
 
1
f s ðnÞ gs ðnÞ ¼ Fnc  F 1 ðxÞ G1 ðxÞ
2
  Table 4. Finite cosine transforms
1 Rp
f s ðnÞ gc ðnÞ ¼ Fns F 1 ðxÞ  G2 ðxÞ
2 f c ðnÞ ¼ FðxÞcos n xdxðn ¼ 0; 1; . . .Þ F (x) (0<x<p)
  0
1 (1)nfc(n) F (px)
f c ðnÞ gc ðnÞ ¼ Fnc F 2 ðxÞ  G2 ðxÞ
2
0 when n¼1,2, . . . ; fc(0)¼p 1
where F1 and G1 are odd extensions of F and G, 2 1 ð0 < x < cÞ
n sin n c; f c ð0Þ ¼ 2 c  p f
respectively, and F2 and G2 are even extensions 1 ðc < x < pÞ
of F and G, respectively. Finite sine and cosine  1ð1Þ
n2
n
; f c ð0Þ ¼ p2
2
x
transforms are listed in Tables 3 and 4. ð1Þ n 2
x2
; f c ð0Þ ¼ p6
On the semi-infinite domain, 0 < x < 1, the n2 2p

Fourier sine and cosine transforms are 1

; f c ð0Þ ¼ 0 ðpxÞ2
 p6
n2 2p

n cx
ð1Þ e 1
Z1 n2 þc2
1 cx
ce
Fvs ½f  f ðxÞsin v x dx;
1 cosh cðpxÞ
0 n2 þc2 c sinh c p

Z1 ð1Þn cos k p1

n2 k2
ðjkj 6¼ 0; 1; . . .Þ 1
k sin k x
Fvc ½f  f ðxÞcos v x dx and
ð1Þnþm 1
n2 m2 ; f c ðmÞ ¼ 0 ðm ¼ 1; . . .Þ
0 1
m sin m x
2 2
f ðxÞ ¼ Fv0 ½Fvs ½f ; f ðxÞ ¼ Fv0 ½Fv ½f 
p s p c c
1
n2 k2
ðjkj 6¼ 0; 1; . . .Þ  cosk sin
kðpxÞ
kx

0 ðn 6¼ mÞ; f c ðmÞ ¼ p2 ðm ¼ 1; 2; . . .Þ cos m x

The sine transform is an odd function of v,
whereas the cosine function is an even The material is reproduced with permission of McGrawHill, Inc.
34 Mathematics in Chemical Engineering

function of v. Also, The Laplace transforms of the first and second

derivatives of F(t) are
 
d2 f
Fvs 2
¼ f ð0Þv  v2 Fvs ½f   
dx dF
L ¼ s f ðsÞ  Fð0Þ
 2  dt
d f df
Fvc ¼  ð0Þ  v2 Fvc ½f   2 
dx2 dx d F dF
L ¼ s2 f ðsÞ  s F ð0Þ  ð0Þ
dt2 dt

provided f (x) and f 0 (x) ! 0 as x ! 1. Thus,

the sine transform is useful when f (0) is known More generally,
and the cosine transform is useful when f 0 (0) is
 
known. dn F
L ¼ sn f ðsÞ  sn1 Fð0Þ
HSU and DRANOFF [42] solved a chemical dtn
engineering problem by applying finite Fourier dF dn1 F
 sn2 ð0Þ      ð0Þ
transforms and then using the fast Fourier dt dtn1
transform (see Chap. 2).
The inverse Laplace transformation is
4.2. Laplace Transforms
FðtÞ ¼ L1 ½f ðsÞ where f ðsÞ ¼ L½F
Consider a function F (t) defined for t > 0. The
Laplace transform of F (t) is [41]
The inverse Laplace transformation is not
Z1 unique because functions that are identical
L½F ¼ f ðsÞ ¼ est FðtÞdt except for isolated points have the same
0
Laplace transform. They are unique to within
a null function. Thus, if
The Laplace transformation is linear, that is,
L½F 1  ¼ f ðsÞ and L½F 2  ¼ f ðsÞ
L½F þ G ¼ L½F þ L½G

it must be that
Thus, the techniques described herein can be
F 2 ¼ F 1 þ NðtÞ
applied only to linear problems. Generally, the
assumptions made below are that F (t) is at least ZT  
where N ðtÞ dt ¼ 0 for every T
piecewise continuous, that it is continuous in
each finite interval within 0 < t < 1, and that it
0

may take a jump between intervals. It is also Laplace transforms can be inverted by using
of exponential order, meaning eat|F (t)| is Table 5, but knowledge of several rules is
bounded for all t > T, for some finite T. helpful.
The unit step function is
Substitution.
0 0t<k
Sk ðtÞ ¼ f
1 t>k f ðs  aÞ ¼ L½eat FðtÞ

and its Laplace transform is This can be used with polynomials. Suppose

eks 1 1 2sþ3
L½Sk ðtÞ ¼ f ðsÞ ¼ þ ¼
s s s þ 3 s ðs þ 3Þ

In particular, if k ¼ 0 then Because

1 1
L½1 ¼ L½1 ¼
s s
 Mathematics in Chemical Engineering 35

Table 5. Laplace transforms (see [204] for a more complete list)

L [F] F (t)
1
s 1

1
s2 t

1 tn1
sn ðn1Þ!

p1ffi p1ffiffiffiffiffi
s pt
pffiffiffiffiffiffiffi
s3/2 2 t=p
Figure 10. Square wave function
G ðkÞ
sk ðk > 0Þ tk1

1
sa ea t
has the Laplace transform
1
ðsaÞn
ðn ¼ 1; 2; . . .Þ 1
ðn1Þ! t
n1 a t
e
1 1
G ðkÞ L½S ðtÞ ¼
ðsaÞk
ðk > 0Þ tk1ea t s 1  ehs

1 1 at
ab ðe  eb t Þ
ðsaÞðsbÞ
The Dirac delta function d (t  t0) has the
s
ðsaÞðsbÞ
1
ab ðae
at
 beb t Þ property
1 1
s2 þa2 a sin a t Z1
dðt  t0 Þ F ðtÞ dt ¼ Fðt0 Þ
s
s2 þa2 cos a t
0

1 1
s2 a2 a sinh a t
Its Laplace transform is
s
s2 a2 cosh a t
L½d ðt  t0 Þ ¼ est0 ; t0  0; s > 0
s t
ðs2 þa2 Þ2 2 a sin a t

s2 a2
t cos a t The square wave function illustrated in
ðs2 þa2 Þ2
Figure 10 has Laplace transform
1 1 at
ðsaÞ2 þb2 be sin b t
1 cs
sa
ea t cos b t L½F c ðtÞ ¼ tanh
ðsaÞ2 þb2 s 2

The triangular wave function illustrated in

Figure 11 has Laplace transform
Then
1 cs
FðtÞ ¼ 1 þ e3t ; t  0 L½T c ðtÞ ¼ tanh
s2 2

More generally, translation gives the following. Other Laplace transforms are listed in Table 5.
Translation.
     t 
b 1
f ða s  bÞ ¼ f a s  ¼ L ebt=a F ;a>0
a a a

The step function

8 1
>
> 0t<
>0
>
>
> h
>
<
1 2
S ðtÞ ¼ 1 t<
>
> h h
>
>
>
>
>
:2 2 3
t<
h h Figure 11. Triangular wave function
36 Mathematics in Chemical Engineering

Convolution properties are also satisfied: If the factor (s  a) is repeated m times, then
Zt p ðsÞ Am Am1
F ðtÞ  G ðtÞ ¼ F ðtÞ G ðt  tÞdt f ðsÞ ¼ ¼ þ þ
q ðsÞ ðs  aÞm ðs  aÞm1
0
A1
þ þ h ðsÞ
sa
and
where
f ðsÞgðsÞ ¼ L½FðtÞ  GðtÞ

ðs  aÞm p ðsÞ
f ðsÞ
q ðsÞ
Derivatives of Laplace Transforms.The Laplace
integrals L [F (t)], L [t F (t)], L [t2 F (t)], . . . are 1 dmk f ðsÞ
Am ¼ f ðaÞ; Ak ¼ ja ;
ðm  kÞ! dsmk
uniformly convergent for s1  a and
k ¼ 1; . . . ; m  1

lim f ðsÞ ¼ 0; lim L½tn F ðtÞ ¼ 0; n ¼ 1; 2; . . .

s!1 s!1
The term h (s) denotes the sum of partial frac-
tions not under consideration. The inverse trans-
and formation is then
dn f  
¼ L½ðtÞn F ðtÞ tm1 tm2 t
dsn F ðtÞ ¼ eat Am þ Am1 þ    þ A2 þ A1
ðm  1Þ! ðm  2Þ! 1!
þ H ðtÞ

Integration of Laplace Transforms.

The term in F (t) corresponding to
Z1 

F ðtÞ
f ðjÞdj ¼ L s  a in q ðsÞ is f ðaÞ ea t
t
s
ðs  aÞ2 in q ðsÞ is ½f0 ðaÞ þ f ðaÞ tea t
1 00
ðs  aÞ3 in q ðsÞ is ½f ðaÞ þ 2f0 ðaÞ t þ f ðaÞ t2 ea t
If F (t) is a periodic function, F (t) ¼ F (t þ a), 2
then
For example, let
Za
1
f ðsÞ ¼ est F ðtÞdt; 1
1  eas f ðsÞ ¼
0
ðs  2Þ ðs  1Þ2
where F ðtÞ ¼ F ðt þ aÞ

For the factor s2,

1
Partial Fractions [43]. Suppose q (s) has m f ðsÞ ¼ ; f ð2Þ ¼ 1
ðs  1Þ2
factors

q ðsÞ ¼ ðs  a1 Þðs  a2 Þ    ðs  am Þ For the factor (s1)2,

1 1
All the factors are linear, none are repeated, f ðsÞ ¼ ; f0 ðsÞ ¼ 
s2 ðs  2Þ2
and the an are all distinct. If p (s) has a smaller
f ð1Þ ¼ 1; f0 ð1Þ ¼ 1
degree than q (s), the Heaviside expansion can
be used to evaluate the inverse transformation:
The inverse Laplace transform is then
  X m
1 p ðsÞ p ðai Þ ai t
L ¼ e
q ðsÞ i¼1
q0 ðai Þ FðtÞ ¼ e2t þ ½1  t et
 Mathematics in Chemical Engineering 37

Quadratic Factors. Let p (s) and q (s) have real To solve an integral equation:
coefficients, and q (s) have the factor
Zt
Y ðtÞ ¼ a þ 2 Y ðtÞcos ðt  tÞdt
ðs  aÞ þ b ; b > 0
2 2
0

where a and b are real numbers. Then define it is written as

f (s) and h (s) and real constants A and B such
that YðtÞ ¼ a þ YðtÞ  cos t

p ðsÞ f ðsÞ
f ðsÞ ¼ ¼
q ðsÞ ðs  aÞ2 þ b2 Then the Laplace transform is used to obtain
AsþB
¼ þ h ðsÞ a s
ðs  aÞ2 þ b2 y ðsÞ ¼ þ 2 y ðsÞ 2
s s þ1
a ðs2 þ 1Þ
Let f1 and f2 be the real and imaginary parts of or y ðsÞ ¼
s ðs  1Þ2
the complex number f (a þi b).

fða þ i bÞ f1 þ i f2 Taking the inverse transformation gives

YðtÞ ¼ s½1 þ 2tet 

Then
Next, let the variable s in the Laplace trans-
f ðsÞ ¼
1 ðs  aÞf2 þ bf1
þ h ðsÞ form be complex. F (t) is still a real-valued
b ðs  aÞ2 þ b2 function of the positive real variable t. The
F ðtÞ ¼
1 at
e ðf2 cos b t þ f1 sin b tÞ þ H ðtÞ
properties given above are still valid for
b s complex, but additional techniques are avail-
able for evaluating the integrals. The real-val-
To solve ordinary differential equations by
ued function is O [exp (x0t)]:
using these results:
jF ðtÞj < Mex0 t ; z0 ¼ x0 þ i y0
Y 00 ðtÞ  2Y 0 ðtÞ þ YðtÞ ¼ e2t

Yð0Þ ¼ 0; Y 0 ð0Þ ¼ 0 The Laplace transform

Taking Laplace transforms Z1

f ðsÞ ¼ est F ðtÞdt
1 0
L½Y 00 ðtÞ  2L½Y 0 ðtÞ þ L½Y ðtÞ ¼
s2
is an analytic function of s in the half-plane
using the rules x > x0 and is absolutely convergent there; it is
uniformly convergent on x  x1 > x0.
1
s2 y ðsÞ  s Y ð0Þ  Y 0 ð0Þ  2½s y ðsÞ  Y ð0Þ þ y ðsÞ ¼
s2
dn f
¼ L½ðtÞn F ðtÞ n ¼ 1; 2; . . . ; x > x0
dsn
and combining terms 
and f ðsÞ ¼ f ðs Þ


1
ðs2  2 s þ 1Þ y ðsÞ ¼ The functions | f(s)| and |x f(s)| are bounded
s2
in the half-plane x  x1 > x0 and f (s) ! 0 as
1
y ðsÞ ¼ |y| ! 1 for each fixed x. Thus,
ðs  2Þðs  1Þ2
jf ðx þ i yÞj < M; jx f ðx þ i yÞj < M;
lead to x  x1 > x 0

lim f ðx þ i yÞ ¼ 0; x > x0
YðtÞ ¼ e2t  ð1 þ tÞ et y!
1
38 Mathematics in Chemical Engineering

If F (t) is continuous, F0 (t) is piecewise contin- Also F (t) and its n derivatives are continuous
uous, and both functions are O [exp (x0t)], then functions of t of order O [exp (x0t)] and they
| f (s)| is O (1/s) in each half-plane x  x1 > x0. vanish at t ¼ 0.

js f ðsÞj < M Fð0Þ ¼ F 0 ð0Þ ¼   F ðmÞ ð0Þ ¼ 0

If F (t) and F0 (t) are continuous, F00 (t) is piece- Series of Residues [41]. Let f (s) be an analytic
wise continuous, and all three functions are O function except for a set of isolated singular
[exp (x0t)], then points. An isolated singular point is one for
which f (z) is analytic for 0 < |z  z0| < % but z0
js2 f ðsÞ  s Fð0Þj < M; x  x1 > x0 is a singularity of f (z). An isolated singular
point is either a pole, a removable singularity, or
an essential singularity. If f (z) is not defined in
The additional constraint F (0) ¼ 0 is necessary the neighborhood of z0 but can be made analytic
and sufficient for | f (s)| to be O (1/s2). at z0 simply by defining it at some additional
points, then z0 is a removable singularity. The
Inversion Integral [41]. Cauchy’s integral for- function f (z) has a pole of order k  1 at z0 if
mula for f (s) analytic and O (sk) in a half-plane (z  z0)k f (z) has a removable singularity at z0
x  y, k > 0, is whereas (z  z0)k1f (z) has an unremovable
isolated singularity at z0. Any isolated singu-
Z
gþib larity that is not a pole or a removable singu-
1 f ðzÞ
f ðsÞ ¼ lim dz; Re ðsÞ > g larity is an essential singularity.
2 pi b!1 sz
gib
Let the function f (z) be analytic except for
the isolated singular point s1, s2, . . . , sn. Let
%n (t) be the residue of ezt f (z) at z ¼ sn (for
Applying the inverse Laplace transformation on
definition of residue, see Section 3.4). Then
either side of this equation gives
X
1
Z
gþib
F ðtÞ ¼ %n ðtÞ
1 zt
F ðtÞ ¼ lim e f ðzÞdz n¼1
2 pi b!1
gib

When sn is a simple pole

If F (t) is of order O [exp (x0t)] and F (t) and F 0
(t) are piecewise continuous, the inversion inte- %n ðtÞ ¼ limz!sn ðz  sn Þezt f ðzÞ
gral exists. At any point t0, where F (t) is ¼ esn t limz!sn ðz  sn Þ f ðzÞ
discontinuous, the inversion integral represents
the mean value
When
1
F ðt0 Þ ¼ lim ½F ðt0 þ eÞ þ Fðt0  eÞ p ðzÞ
e!1 2
f ðzÞ ¼
q ðzÞ
When t ¼ 0 the inversion integral represents
0.5 F (O þ) and when t < 0, it has the value zero. where p (z) and q (z) are analytic at z ¼ sn, p (sn)
If f (s) is a function of the complex variable s 6¼ 0, then
that is analytic and of order O (skm) on R (s)
 x0, where k > 1 and m is a positive integer, p ðsn Þ sn t
%n ðtÞ ¼ e
then the inversion integral converges to F (t) and q0 ðsn Þ

Z
gþib If sn is a removable pole of f (s), of order m,
dn F 1
¼ lim ezt zn f ðzÞdz; then
dtn 2 pi b!1
gib

n ¼ 1; 2; . . . ; m fn ðzÞ ¼ ðz  sn Þm f ðzÞ
 Mathematics in Chemical Engineering 39

is analytic at sn and the residue is Solution is via Fourier transforms

Fn ðsn Þ @ m1 Z1
%n ðtÞ ¼ where Fn ðzÞ ¼ m1 ½fn ðzÞezt  ^
ðm  1Þ! @z Tðv; tÞ ¼ T ðx; tÞeivx dx
1

An important inversion integral is when

Applied to the differential equation
1
f ðsÞ ¼ expðs1=2 Þ
s  
@2 T
F ¼  v2 aF ½T
@x 2
The inverse transform is
@ T^
    þ v2 aT^ ¼ 0; Tðv;
^ 0Þ ¼ ^f ðvÞ
1 1 @t
F ðtÞ ¼ 1  erf pffi ¼ erfc pffi
2 t 2 t
By solving
where erf is the error function and erfc the ^ tÞ ¼ ^f ðvÞ ev at
2
Tðv;
complementary error function.
the inverse transformation gives [40, p. 328],
4.3. Solution of Partial Differential [44, p. 58]
Equations by Using Transforms
ZL
1
eivx ^f ðvÞ ev
2
at
T ðx; tÞ ¼ lim dv
A common problem facing chemical engineers 2 p L!1
L
is to solve the heat conduction equation or
diffusion equation Another solution is via Laplace transforms;
take the Laplace transform of the original dif-
@T @ T 2
@c @ c 2
% Cp ¼ k 2 or ¼D 2 ferential equation.
@t @x @t @x
@2t
s tðs; xÞ  f ðxÞ ¼ a
The equations can be solved on an infinite @x2

domain  1 < x < 1, a semi-infinite domain

0  x < 1, or a finite domain 0  x  L. At a This equation can be solved with Fourier trans-
boundary, the conditions can be a fixed tem- forms [40, p. 355]
perature T (0, t) ¼ T0 (boundary condition of the  rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
Z1
first kind, or Dirichlet condition), or a fixed flux 1
t ðs; xÞ ¼ pffiffiffiffiffi e 
s
jx  yj f ðyÞ dy
k @T@x ð0; tÞ ¼ q0 (boundary condition of the
2 sa
1
a

second kind, or Neumann condition), or a

combination k @T @x ð0; tÞ ¼ h½T ð0;tÞ  T 0  The inverse transformation is [40, p. 357], [44,
(boundary condition of the third kind, or Robin p. 53]
condition).
Z1
The functions T0 and q0 can be functions of 1
T ðx; tÞ ¼ pffiffiffiffiffiffiffiffiffiffi eðxyÞ
2
=4at
f ðyÞdy
time. All properties are constant (%, Cp, k, D, h), 2 pat
1
so that the problem remains linear. Solutions
are presented on all domains with various Problem 2. Semi-infinite domain, boundary
boundary conditions for the heat conduction condition of the first kind, on 0  x  1
problem.
Tðx; 0Þ ¼ T 0 ¼ constant
@T @2T k
¼a 2; a¼ Tð0; tÞ ¼ T 1 ¼ constant
@t @x %C p

The solution is
Problem 1. Infinite domain, on  1 < x < 1.
pffiffiffiffiffiffiffiffi
Tðx; 0Þ ¼ f ðxÞ; initial conditions Tðx ; tÞ ¼ T 0 þ ½T 1  T 0 ½1  erf ðx= 4a tÞ
pffiffiffiffiffiffiffiffi
Tðx; tÞ bounded or Tðx; tÞ ¼ T 0 þ ðT 1  T 0 Þ erfc ðx= 4a tÞ
40 Mathematics in Chemical Engineering

Problem 3. Semi-infinite domain, boundary The solution for T1 can also be obtained by
condition of the first kind, on 0  x < 1 Laplace transforms.

Tðx; 0Þ ¼ f ðxÞ
t1 ¼ L½T 1 
Tð0; tÞ ¼ g ðtÞ

Applying this to the differential equation

The solution is written as

Tðx; tÞ ¼ T 1 ðx; tÞ þ T 2 ðx; tÞ @ 2 t1

st1  f ðxÞ ¼ a ; t1 ð0; sÞ ¼ 0
@x2

where
and solving gives
T 1 ðx; 0Þ ¼ f ðxÞ; T 2 ðx; 0Þ ¼ 0

T 1 ð0; tÞ ¼ 0; T 2 ð0; tÞ ¼ g ðtÞ Zx pffiffiffiffiffi 0

1
t1 ¼ pffiffiffiffiffiffi e s=aðx xÞ f ðx0 Þ dx0
sa
0
Then T1 is solved by taking the sine transform

U 1 ¼ F vs ½T 1  and the inverse transformation is [40, p. 437],

[44, p. 59]
@U 1
¼  v2 a U 1
@t Z1
1 2 2
T 1 ðx; tÞ ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffi ½eðxjÞ =4at  eðxþjÞ =4at  f ðjÞ dj
4pat
U 1 ðv; 0Þ ¼ F vs ½f  0

Thus,
Problem 4. Semi-infinite domain, boun-
2 dary conditions of the second kind, on
U 1 ðv; tÞ ¼ F vs ½f  ev at
0  x < 1.
and [40, p. 322] Tðx; 0Þ ¼ 0

Z1
2 2
@T
T 1 ðx; tÞ ¼ F vs ½f ev at
sin vx dv k ð0; tÞ ¼ q0 ¼ constant
p @x
0

Solve for T2 by taking the sine transform Take the Laplace transform

U 2 ¼ F vs ½T 2  tðx; sÞ ¼ L½Tðx; tÞ

@U 2 @2 t
¼  v2 a U 2 þ a gðtÞ v st ¼ a
@t @x2

U 2 ðv; 0Þ ¼ 0 @t q0
k ¼
@x s

Thus,
The solution is
Zt pffiffiffi
2 q0 a xpffiffiffiffiffi
U 2 ðv; tÞ ¼ ev aðttÞ
av g ðtÞdt t ðx; sÞ ¼ e s=a
k s3=2
0

and [40, p. 435] The inverse transformation is [41, p. 131], [44,

p. 75]
Z1 Zt
2a 2  rffiffiffiffiffiffi  
T 2 ðx; tÞ ¼ v sin vx ev aðttÞ
gðtÞdtdv q0 a t x2 =4at x
p T ðx; tÞ ¼ 2 e  x erfc pffiffiffiffiffiffiffiffiffi
0 0 k p 4at
 Mathematics in Chemical Engineering 41

Problem 5. Semi-infinite domain, boundary Take the Laplace transform

conditions of the third kind, on 0  x < 1
@2 t
s tðx; sÞ  T 0 ¼ a
@x2
Tðx; 0Þ ¼ f ðxÞ

tð0; sÞ ¼ tðL; sÞ ¼ 0
@T
k ð0; tÞ ¼ h T ð0; tÞ
@x
The solution is
Take the Laplace transform pffiffiffiffi pffiffi
T 0 sinh as x T 0 sinh as ðL  xÞ T 0
t ðx; sÞ ¼  pffiffis  pffiffi þ
s sinh a L s sinh as L s
@2 t
s t  f ðxÞ ¼ a
@x2
@t The inverse transformation is [41, p. 220], [44,
k ð0; sÞ ¼ h t ð0; sÞ p. 96]
@x

The solution is 2 X 2 2 2 2 npx

T ðx; tÞ ¼ T0 en p at=L sin
p n¼1;3;5;...
n L
Z1
t ðx; sÞ ¼ f ðjÞ g ðx; j; sÞdj
0
or (depending on the inversion technique) [40,
pp. 362]
where [41, p. 227]
ZL X
1
T0 2
=4at 2
=4at
pffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffi T ðx; tÞ ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffi ½e½ðxjÞþ2nL  e½ðxþjÞþ2nL dj
2 s=a g ðx; j; sÞ ¼ exp ðjx  jj s=aÞ 4pat n¼1
0
pffiffi pffiffiffi pffiffiffiffiffiffiffiffi
s  h a=k
þ pffiffi pffiffiffi exp ½ðx þ jÞ s=a
s þ h a=k Problem 7. Finite domain, boundary condi-
tion of the first kind
One form of the inverse transformation is
[41, p. 228] Tðx; 0Þ ¼ 0

Tð0; tÞ ¼ 0
Z1 Z1 TðL; 0Þ ¼ T 0 ¼ constant
2 2
T ðx; tÞ ¼ eb at
cos ½b x  m ðbÞ f ðjÞ
p
0 0
Take the Laplace transform
cos ½b j  m ðbÞ djdb

m ðbÞ ¼ arg ðb þ i h=kÞ @2t

st ðx; sÞ ¼ a
@x2
T0
Another form of the inverse transformation t ð0; sÞ ¼ 0; t ðL; sÞ ¼
s
when f ¼ T0 ¼ constant is [41, p. 231], [44,
p. 71] The solution is
    pffiffiffiffiffi  pffiffiffiffiffiffiffiffi
x 2 2 h at x T 0 sinh Lx s=a
T ðx; tÞ ¼ T 0 erf pffiffiffiffiffiffiffiffiffi þ ehx=k eh at=k erfc þ pffiffiffiffiffiffiffiffiffi t ðx; sÞ ¼ pffiffiffiffiffiffiffiffi
4at k 4at s sinh s=a

Problem 6. Finite domain, boundary condi- and the inverse transformation is [41, p. 201],
tion of the first kind [44, p. 313]
" #
Tðx; 0Þ ¼ T 0 ¼ constant x 2X 1
ð1Þn n2 p2 at=L2 n p x
T ðx; tÞ ¼ T 0 þ e sin
L p n¼1 n L
Tð0; tÞ ¼ T ðL; tÞ ¼ 0
42 Mathematics in Chemical Engineering

An alternate transformation is [41, p. 139],

[44, p. 310]
1 
X    
ð2 n þ 1Þ L þ x ð2 n þ 1Þ L  x
T ðx; tÞ ¼ T 0 erf pffiffiffiffiffiffiffiffiffi  erf pffiffiffiffiffiffiffiffiffi
n¼0 4at 4at

Figure 12. Addition of vectors

Problem 8. Finite domain, boundary condi-
tion of the second kind

Tðx; 0Þ ¼ T 0 including their directions, and some of the

identities for dyadics are more easily proved
@T by using tensor analysis, which is not presented
ð0; tÞ ¼ 0; T ðL; tÞ ¼ 0
@x here (see also, ! Transport Phenomena). Vec-
tors are also first-order dyadics.
Take the Laplace transform
Vectors. Two vectors u and v are equal if they
@2 t have the same magnitude and direction. If they
s t ðx; sÞ  T 0 ¼ a 2
@x have the same magnitude but the opposite
@t
ð0; sÞ ¼ 0; t ðL; sÞ ¼ 0
direction, then u ¼  v. The sum of two vectors
@x is identified geometrically by placing the vector
v at the end of the vector u, as shown in
The solution is Figure 12. The product of a scalar m and a
" pffiffiffiffiffiffiffiffi # vector u is a vector m u in the same direction as
T0 cosh x s=a u but with a magnitude that equals the magni-
t ðx; sÞ ¼ 1 pffiffiffiffiffiffiffiffi
s cosh L s=a tude of u times the scalar m. Vectors obey
commutative and associative laws.
Its inverse is [41, p. 138]
uþv¼vþu Commutative law for addition
X
1
u þ (v þ w) ¼ (u þ v) þ Associative law for addition
T ðx; tÞ ¼ T 0  T 0 ð1Þn
w
n¼0
     mu¼um Commutative law for scalar
ð2n þ 1Þ L  x ð2n þ 1Þ L þ x
erfc pffiffiffiffiffiffiffiffiffi þ erfc pffiffiffiffiffiffiffiffiffi multiplication
4at 4at
m (n u) ¼ (m n) u Associative law for scalar
multiplication
(m þ n) u ¼ m u þ n u Distributive law
m (u þ v) ¼ m u þ m v Distributive law
5. Vector Analysis
The same laws are obeyed by dyadics, as well.
Notation. A scalar is a quantity having mag-
A unit vector is a vector with magnitude 1.0
nitude but no direction (e.g., mass, length,
and some direction. If a vector has some mag-
time, temperature, and concentration). A vector
nitude (i.e., not zero magnitude), a unit vector
is a quantity having both magnitude and direc-
eu can be formed by
tion (e.g., displacement, velocity, acceleration,
force). A second-order dyadic has magnitude u
and two directions associated with it, as defined eu ¼
juj
precisely below. The most common examples
are the stress dyadic (or tensor) and the velocity The original vector can be represented by the
gradient (in fluid flow). Vectors are printed in product of the magnitude and the unit vector.
boldface type and identified in this chapter by
lower-case Latin letters. Second-order dyadics u ¼ jujeu
are printed in boldface type and are identified in
this chapter by capital or Greek letters. Higher In a cartesian coordinate system the three prin-
order dyadics are not discussed here. Dyadics ciple, orthogonal directions are customarily
can be formed from the components of tensors represented by unit vectors, such as {ex, ey, ez}
 Mathematics in Chemical Engineering 43

Dyadics. The dyadic A is written in component

form in cartesian coordinates as
A ¼ Axx ex ex þ Axy ex ey þ Axz ex ez

þAyx ey ex þ Ayy ey ey þ Ayz ey ez

þAzx ez ex þ Azy ez ey þ Azz ez ez

Quantities such as exey are called unit dyadics.

They are second-order dyadics and have two
Figure 13. Cartesian coordinate system directions associated with them, ex and ey; the
order of the pair is important. The components
Axx, . . . , Azz are the components of the tensor
Aij which here is a 33 matrix of numbers
or {i, j, k}. Here, the first notation is used (see that are transformed in a certain way when
Fig. 13). The coordinate system is right handed; variables undergo a linear transformation.
that is, if a right-threaded screw rotated from the The y x momentum flux can be defined as
x to the y direction, it would advance in the z the flux of x momentum across an area with
direction. Avector can be represented in terms of unit normal in the y direction (! Fluid Mechan-
its components in these directions, as illustrated ics; introduction to ! Transport Phenomena).
in Figure 14. The vector is then written as Since two directions are involved, a second-
order dyadic (or tensor) is needed to represent
u ¼ ux ex þ uy ey þ uz ez
it, and because the y momentum across an area
with unit normal in the x direction may not be
the same thing, the order of the indices must be
The magnitude is kept straight. The dyadic A is said to be sym-
metric if
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
juj ¼ u2x þ u2y þ u2z Aij ¼ Aji

Here, the indices i and j can take the values x, y,

The position vector is or z; sometimes (x, 1), (y, 2), (x, 3) are identified
and the indices take the values 1, 2, or 3. The
r ¼ xex þ yey þ zez dyadic A is said to be antisymmetric if
Aij ¼ Aji

with magnitude
The transpose of A is
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
jrj ¼ x2 þ y 2 þ z 2 ATij ¼ Aji

Any dyadic can be represented as the sum

of a symmetric portion and an antisymmetric
portion.
1
Aij ¼ Bij þ Cij ; Bij ðAij þ Aji Þ;
2
1
Cij ðAij  Aji Þ
2

An ordered pair of vectors is a second-order

dyadic.
XX
uv ¼ ei ej ui vj
i j
Figure 14. Vector components
44 Mathematics in Chemical Engineering

The transpose of this is The double dot product of two dyadics is

XX
ðu vÞT ¼ v u A:B¼ Aij Bji
i j

but
Because the dyadics may not be symmetric, the
u v 6¼ v u order of indices and which indices are summed
are important. The order is made clearer when
The Kronecker delta is defined as the dyadics are made from vectors.

1 if i ¼ j ðu vÞ  ðw xÞ ¼ uðv  wÞx ¼ u xðv  wÞ

dij ¼ f g
0 if i 6¼ j
ðu vÞ : ðw xÞ ¼ ðu  xÞðv  wÞ

and the unit dyadic is defined as

The dot product of a dyadic and a vector is
XX
d¼ ei ej dij X X
i j Au¼ ei ð Aij uj Þ
i j

Operations. The dot or scalar product of two

The cross or vector product is defined by
vectors is defined as

u  v ¼ jujjvjcos u; 0  u  p c ¼ u  v ¼ a j u j j v j sin u; 0  u  p

where u is the angle between u and v. The scalar where a is a unit vector in the direction of uv.
product of two vectors is a scalar, not a vector. It The direction of c is perpendicular to the plane
is the magnitude of u multiplied by the projec- of u and v such that u, v, and c form a right-
tion of v on u, or vice versa. The scalar product handed system. If u ¼ v, or u is parallel to v,
of u with itself is just the square of the magni- then u ¼ 0 and uv ¼ 0. The following laws are
tude of u. valid for cross products.
u  u ¼ ju2 j ¼ u2
uv¼vu Commutative law fails for vector
The following laws are valid for scalar products product
u(vw) 6¼ (uv)w Associative law fails for vector
product
u v ¼ v u Commutative law for scalar u(v þ w) ¼ uv þ uw Distributive law for vector
products product
u (v þ w) ¼ u v þ u Distributive law for scalar products ex ex¼ ey ey¼ ez ez ¼ 0
w
ex ey¼ ez, ey ez¼ ex, ez
ex ex ¼ ey ey¼ ez ez ¼ 1
ex¼ ey
ex ey¼ ex ez¼ ey ez ¼ 0
2 3
ex ey ez
If the two vectors u and v are written in com- 6
6
7
7
ponent notation, the scalar product is u  v ¼ det6 ux uy uz 7
4 5
vx vy vz
u  v ¼ u x vx þ u y vy þ u z vz
¼ ex ðuy vz  vy uz Þ þ ey ðuz vz  ux vz Þ
If u  v ¼ 0 and u and v are not null vectors, then
þez ðux vy  uy vx Þ
u and v are perpendicular to each other and u ¼
p/2.
This can also be written as
The single dot product of two dyadics is
XX X XX
AB¼ ei ej ð Aik Bkj Þ uv¼ ekij ui vj ek
i j k i j
 Mathematics in Chemical Engineering 45

where Invariants of two dyadics are available [46].

Because a second-order dyadic has nine com-
8 ponents, the characteristic equation
> 1 if i; j; k is an even permutation of 123
>
<
eijk ¼ 1 if i; j; k is an odd permutation of 123
>
>
:
0 if any two of i; j; k are equal det ðl d  AÞ ¼ 0

Thus e123 ¼ 1, e132 ¼1, e312 ¼ 1, e112 ¼ 0, for can be formed where l is an eigenvalue. This
example. expression is
The magnitude of uv is the same as the
area of a parallelogram with sides u and v. If
l3  I 1 l2 þ I 2 l  I 3 ¼ 0
uv ¼ 0 and u and v are not null vectors, then u
and v are parallel. Certain triple products are
useful. An important theorem of HAMILTON and CAYLEY
[47] is that a second-order dyadic satisfies its
ðu  vÞw 6¼ uðv  wÞ own characteristic equation.
u  ðv  wÞ ¼ v  ðw  uÞ ¼ w  ðu  vÞ

u  ðv  wÞ ¼ ðu  wÞv  ðu  vÞw A3  I 1 A2 þ I 2 A  I 3 d ¼ 0 ð7Þ

ðu  vÞ  w ¼ ðu  wÞv  ðv  wÞu

Thus A3 can be expressed in terms of d, A, and

The cross product of a dyadic and a vector is A2. Similarly, higher powers of A can be
defined as expressed in terms of d, A, and A2. Decompo-
sition of a dyadic into a symmetric and an
XX XX
Au¼ ei ej ð eklj Aik ul Þ antisymmetric part was shown above. The anti-
i j k l
symmetric part has zero trace. The symmetric
part can be decomposed into a part with a trace
The magnitude of a dyadic is (the isotropic part) and a part with zero trace
rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
(the deviatoric part).
1 1 XX 2
jAj ¼ A ¼ ðA : AT Þ ¼ Aij
2 2 i j
A¼ 1/3dd : A þ 1/2[A þ AT  2/3dd : A] þ 1/2[A  AT]
Isotropic Deviatoric Antisymmetric
There are three invariants of a dyadic. They
are called invariants because they take the same
value in any coordinate system and are thus an Differentiation. The derivative of a vector
intrinsic property of the dyadic. They are the is defined in the same way as the derivative
trace of A, A2, A3 [45]. of a scalar. Suppose the vector u depends on t.
X
Then
I ¼ trA ¼ Aii
i
XX
II ¼ trA2 ¼ Aij Aji du uðt þ DtÞ  uðtÞ
¼ lim
i j dt Dt!0 Dt
XXX
III ¼ trA3 ¼ Aij Ajk Aki
i j k

If the vector is the position vector r(t), then the

The invariants can also be expressed as
difference expression is a vector in the direc-
I1 ¼ I tion of Dr (see Fig. 15). The derivative
1 2
I2 ¼ ðI  IIÞ
2
1 3 dr Dr rðt þ DtÞ  rðtÞ
I3 ¼ ðI  3 I  II þ 2IIIÞ ¼ det A ¼ lim ¼ lim
6 dt Dt!0 Dt Dt!0 Dt
46 Mathematics in Chemical Engineering

The arc-length function can also be defined:

Z t rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
dr dr 
s ðtÞ ¼  dt
dt dt
a

This gives
Figure 15. Vector differentiation
 2  2  2  2
ds dr dr dx dy dz
¼  ¼ þ þ
dt dt dt dt dt dt
is the velocity. The derivative operation obeys
the following laws.
Because
d du dv
ðu þ vÞ ¼ þ
dt dt dt dr ¼ dxex þ dyey þ dzez
d du dv
ðu  vÞ ¼ vþu
dt dt dt then
d du dv
ðu  vÞ ¼ vþu
dt dt dt ds2 ¼ dr  dr ¼ dx2 þ dy2 þ dz2
d du du
ðuuÞ ¼ u þ u
dt dt dt
The derivative dr/dt is tangent to the curve in
If the vector u depends on more than one the direction of motion
variable, such as x, y and z, partial derivatives
are defined in the usual way. For example, dr
u ¼  dt 
dr 
if uðx; y; zÞ; then  dt 
@u uðx þ Dx; y; zÞ  uðx; y; zÞ
¼ lim
@x Dt!0 Dx Also,
Rules for differentiation of scalar and vector dr
u¼
products are ds

@ @u @v
ðu  vÞ ¼ vþu Differential Operators. The vector differential
@x @x @x
@ @u @v operator (del operator) b is defined in cartesian
ðu  vÞ ¼ vþu coordinates by
@x @x @x

Differentials of vectors are @ @ @

r ¼ ex þ ey þ ez
@x @y @z
du ¼ dux ex þ duy ey þ duz ez
The gradient of a scalar function is defined
dðu  vÞ ¼ du  v þ u  dv

dðu  vÞ ¼ du  v þ u  dv @f @f @f
rf ¼ ex þ ey þ ez
@u @u @u @x @y @z
du ¼ dx þ dy þ dz
@x @y @z
and is a vector. If f is height or elevation, the
gradient is a vector pointing in the uphill direc-
If a curve is given by r(t), the length of the tion. The steeper the hill, the larger is the
curve is [43] magnitude of the gradient.
The divergence of a vector is defined by
Z b rffiffiffiffiffiffiffiffiffiffiffiffi
dr dr
L¼  dt @ux @uy @uz
dt dt r u¼ þ þ
a @x @y @z
 Mathematics in Chemical Engineering 47

and is a scalar. For a volume element DV, the net Useful formulas are [49]
outflow of a vector u over the surface of the r  ðfuÞ ¼ rf  u þ fr  u
element is
r  ðfuÞ ¼ rf  u þ fr  u
Z
r  ðu  vÞ ¼ v  ðr  uÞ  u  ðr  vÞ
u  ndS
DS r  ðu  vÞ ¼ v  ru  vðr  uÞ  u  rv þ uðr  vÞ

r  ðr  uÞ ¼ rðr  uÞ  r2 u
This is related to the divergence by [48, p. 411]
@2 f @2 f @2 f
r  ðrfÞ ¼ r2 f ¼ þ þ ; where r2
Z @x2 @y2 @z2
1
r  u ¼ lim u  ndS
DV!0 DV is called the Laplacian operator. r(r
r w) ¼ 0.
DS
The curl of the gradient of f is zero.
Thus, the divergence is the net outflow per unit r  ðr  uÞ ¼ 0
volume.
The curl of a vector is defined by The divergence of the curl of u is zero. Formu-
las useful in fluid mechanics are
 
@ @ @ r  ðrvÞT ¼ rðr  vÞ
ru¼ þ ey þ ez
ex  ðex ux þ ey uy þ ez uz Þ
@x @y @z
    r  ðt  vÞ ¼ v  ðr  tÞ þ t : rv
@uz @uy @ux @uz
¼ ex  þ ey  1
@y @z @z @x v  rv ¼ rðv  vÞ  v  ðr  vÞ
  2
@uy @ux
þez  If a coordinate system is transformed by a rota-
@x @y
tion and translation, the coordinates in the new
and is a vector. It is related to the integral system (denoted by primes) are given by
0 1 0 1
x0 l11 l12 l13
Z Z B 0C B C
By C ¼ B l21 l22 l23 C
u  ds ¼ us ds @ A @ A
C C z0 l31 l32 l33
0 1 0 1
x a1
which is called the circulation of u around path B C
ByC
B C
B a2 C
@ A þ @ A
C. This integral depends on the vector and the
contour C, in general. If the circulation does not z a3

depend on the contour C, the vector is said to be Any function that has the same value in all
irrotational; if it does, it is rotational. The coordinate systems is an invariant. The gradient
relationship with the curl is [48, p. 419] of an invariant scalar field is invariant; the same
Z is true for the divergence and curl of invariant
1
n  ðr  uÞ ¼ lim
D!0 DS
u  ds vectors fields.
C The gradient of a vector field is required in
fluid mechanics because the velocity gradient is
Thus, the normal component of the curl equals used. It is defined as
the net circulation per unit area enclosed by the XX @vj
rv ¼ ei ej and
contour C. i j
@xi
The gradient, divergence, and curl obey a XX @vi
distributive law but not a commutative or asso- ðrvÞT ¼ ei ej
i j
@xj
ciative law.
The divergence of dyadics is defined
rðf þ cÞ ¼ rf þ rc !
P X @t ji
rt ¼ i ei and
r  ðu þ vÞ ¼ r  u þ r  v j
@xj
" #
r  ðu þ vÞ ¼ r  u þ r  v X X @
r  ðfu vÞ ¼ ei ðfuj vi Þ
r  f 6¼ fr i j
@xj

r  u 6¼ u  r where t is any second-order dyadic.

48 Mathematics in Chemical Engineering

Useful relations involving dyadics are or a vector

ðf d : rvÞ ¼ fðr  vÞ
rII  v rII  vII ¼ nrII  v ¼ n n  r  v
r  ðf dÞ ¼ rf

r  ðftÞ ¼ rf  t þ fr  t
Vector Integration [48, pp. 206–212]. If u is a
nt : t ¼ ttn ¼ t : nt vector, then its integral is also a vector.

A surface can be represented in the form Z Z Z

uðtÞdt ¼ ex ux ðtÞdt þ ey uy ðtÞdt
f ðx; y; zÞ ¼ c ¼ constant Z
þ ez uz ðtÞdt
The normal to the surface is given by
rf If the vector u is the derivative of another
n¼
jr f j vector, then

provided the gradient is not zero. Operations Z Z

dv dv
u¼ ; uðtÞdt ¼ dt ¼ v þ constant
can be performed entirely within the surface. dt dt
Define
If r (t) is a position vector that defines a curve C,
@
dII d  n n; rII dII  r; nr the line integral is defined by
@n
vII dII  v; vn n  v Z Z
u  dr ¼ ðux dx þ uy dy þ uz dzÞ
Then a vector and del operator can be decom- C C

posed into
Theorems about this line integral can be written
@ in various forms.
v ¼ vII þ nvn ; r ¼ rII þ n
@n
Theorem [43]. If the functions appearing in the
The velocity gradient can be decomposed into line integral are continuous in a domain D, then
the line integral is independent of the path C if
@vII @vn
rv ¼ rII vII þ ðrII nÞvn þ nrII vn þ n þ nn and only if the line integral is zero on every
@n @n
simple closed path in D.
The surface gradient of the normal is the nega-
tive of the curvature dyadic of the surface. Theorem [43]. If u ¼ rf where f is single-
valued and has continuous derivatives in D,
rII n ¼ B then the line integral is independent of the
path C and the line integral is zero for any
The surface divergence is then closed curve in D.

rII  v ¼ dII : rv ¼ rII  vII  2 Hvn Theorem [43]. If f, g, and h are continuous
functions of x, y, and z, and have continuous
where H is the mean curvature. first derivatives in a simply connected domain
D, then the line integral
1 Z
H ¼ dII : B
2 ðf dx þ gdy þ hdzÞ
C

The surface curl can be a scalar

is independent of the path if and only if
rII  v ¼ eII : rv ¼ eII : rII vII ¼ n  ðr  vÞ;
@h @g @ f @h @g @ f
¼ ; ¼ ; ¼
eII ¼ n  e @y @z @z @x @x @y
 Mathematics in Chemical Engineering 49

or if f, g, and h are regarded as the x, y, and z Then the divergence theorem in component
components of a vector v: form is
rv¼0 Z   Z
@ux @uy @uz
þ þ dxdydz ¼ ½ux cos ðx; nÞ
@x @y @z
Consequently, the line integral is independent V S

of the path (and the value is zero for a closed þuy cos ðy; nÞ þ uz cos ðz; nÞdS
contour) if the three components in it are
regarded as the three components of a vector If the divergence theorem is written for an
and the vector is derivable from a potential (or incremental volume
zero curl). The conditions for a vector to be
Z
derivable from a potential are just those in the r  u ¼ lim
1
un dS
DV!0 DV
third theorem. In two dimensions this reduces DS
to the more usual theorem.
the divergence of a vector can be called the
Theorem [48, p. 207]. If M and N are continu-
integral of that quantity over the area of a closed
ous functions of x and y that have continuous
volume, divided by the volume. If the vector
first partial derivatives in a simply connected
represents the flow of energy and the diver-
domain D, then the necessary and sufficient
gence is positive at a point P, then either a
condition for the line integral
source of energy is present at P or energy is
Z
leaving the region around P so that its temper-
ðMdx þ NdyÞ
C
ature is decreasing. If the vector represents the
flow of mass and the divergence is positive at a
to be zero around every closed curve C in D is point P, then either a source of mass exists at P
or the density is decreasing at the point P. For an
@M @N incompressible fluid the divergence is zero and
¼
@y @x
the rate at which fluid is introduced into a
If a vector is integrated over a surface with volume must equal the rate at which it is
incremental area d S and normal to the surface removed.
n, then the surface integral can be written as Various theorems follow from the diver-
gence theorem.
ZZ ZZ
u  dS ¼ u  ndS Theorem. If f is a solution to Laplace’s equa-
S S tion

If u is the velocity then this integral represents r2 f ¼ 0

the flow rate past the surface S.
in a domain D, and the second partial deriva-
Divergence Theorem [48, 49]. If V is a volume
tives of f are continuous in D, then the integral
bounded by a closed surface S and u is a vector
of the normal derivative of f over any piecewise
function of position with continuous deriva-
smooth closed orientable surface S in D is zero.
tives, then
Suppose u ¼ f r c satisfies the conditions of
Z Z Z Z the divergence theorem, then Green’s theorem
r  udV ¼ n  udS ¼ u  ndS ¼ u  dS results from use of the divergence theorem [49].
V S S S
Z Z
@c
ðfr2 c þ rf  rcÞdV ¼ f dS
where n is the normal pointing outward to S. @n
V S
The normal can be written as

n ¼ ex cosðx; nÞ þ ey cosðy; nÞ þ ez cosðz; nÞ

and
Z Z  
@c @f
where, for example, cos(x, n) is the cosine of ðfr2 c  cr2 fÞdV ¼ f
@n
c
@n
dS
the angle between the normal n and the x axis. V S
50 Mathematics in Chemical Engineering

Also if f satisfies the conditions of the theorem Theorem [48, p. 423]. The necessary and suf-
and is zero on S then f is zero throughout D. If ficient condition that the divergence of a vector
two functions f and c both satisfy the Laplace vanish identically is that the vector is the curl of
equation in domain D, and both take the same some other vector.
values on the bounding curve C, then f ¼ c;
i.e., the solution to the Laplace equation is Leibniz Formula. In fluid mechanics and
unique. transport phenomena, an important result is
The divergence theorem for dyadics is the derivative of an integral whose limits of
integration are moving. Suppose the region V
Z Z (t) is moving with velocity vs. Then Leibniz’s
r  tdV ¼ n  t dS
rule holds:
V S

ZZZ ZZZ ZZ
d @f
fdV ¼ dV þ fvs  ndS
Stokes Theorem [48, 49]. Stokes theorem says dt @t
VðtÞ VðtÞ S
that if S is a surface bounded by a closed,
nonintersecting curve C, and if u has continu- Curvilinear Coordinates. Many of the rela-
ous derivatives then tions given above are proved most easily by
I ZZ ZZ using tensor analysis rather than dyadics. Once
u  dr ¼ ðr  uÞ  ndS ¼ ðr  uÞ  dS proven, however, the relations are perfectly
C S S general in any coordinate system. Displayed
here are the specific results for cylindrical and
The integral around the curve is followed in the spherical geometries. Results are available for a
counterclockwise direction. In component few other geometries: Parabolic cylindrical,
notation, this is paraboloidal, elliptic cylindrical, prolate sphe-
roidal, oblate spheroidal, ellipsoidal, and
H
C ½ux cos ðx; sÞ þ uy cos ðy; sÞ þ uz cos ðz; sÞds bipolar coordinates [45, 50].
ZZ     For cylindrical coordinates, the geometry is
@uz @uy @ux @uz
¼  cosðx; nÞ þ  cosðy; nÞ shown in Figure 16. The coordinates are related
@y @z @z @x
S    to cartesian coordinates by
@uy @ux
þ  cosðz; nÞ dS
@x @y

Applied in two dimensions, this results in Green’s

theorem in the plane:
I ZZ  
@N @M
ðMdx þ NdyÞ ¼  dxdy
@x @y
C S

The formula for dyadics is

ZZ I
n  ðr  tÞdS ¼ tT  dr
S C

Representation. Two theorems give informa-

tion about how to represent vectors that obey
certain properties.
Theorem [48, p. 422]. The necessary and suf-
ficient condition that the curl of a vector vanish
identically is that the vector be the gradient of
some function. Figure 16. Cylindrical coordinate system
 Mathematics in Chemical Engineering 51
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
x ¼ r cos u r ¼ x2 þ
y 2
y ¼ r sin u u ¼ arctan yx
z¼z z¼z

The unit vectors are related by

er ¼ cos u ex þ sin u ey ex ¼ cos u er  sin u eu

eu ¼  sin u ex þ cos u ey ey ¼ sin u er þ cos u eu
ez ¼ ez ez ¼ e z

Derivatives of the unit vectors are

deu ¼ er du; der ¼ eu du; dez ¼ 0

Differential operators are given by [45]

@ eu @ @
r ¼ er þ þ ez ;
@r r @u @z Figure 17. Spherical coordinate system

@f eu @f @f
rf ¼ er þ þ ez
@r r @u @z
For spherical coordinates, the geometry is
 
21@ @f 1 @2 f @2 f shown in Figure 17. The coordinates are related
r f¼ r þ 2 2þ 2
r @r @r r @u @z to cartesian coordinates by
@ 2 f 1 @f 1 @ 2 f @ 2 f
¼ þ þ þ
@r2 r @r r2 @u2 @z2 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
x ¼ r sin u cosf r ¼ x2 þ p y2ffiffiffiffiffiffiffiffiffiffiffiffiffiffi
þ z2 ffi
1@ 1 @vu @vz y ¼ r sin u sinf u ¼ arctanð
x2 þ y2 =zÞ
rv¼ ðr vr Þ þ þ z ¼ r cos u f ¼ arctan yx
r @r r @u @z
   
1 @vz @vu @vr @vz
r  v ¼ er  þ eu 
r @u @z @z @r The unit vectors are related by
 
1@ 1 @vr
þ ez ðr vu Þ  er ¼ sin u cos fex þ sin u sin fey þ cos uez
r @r r @u

  eu ¼ cos u cos fex þ cos u sin fey  sin uez

1@ 1 @t ur @tzr t uu
r  t ¼ er ðrtrr Þ þ þ  þ ef ¼ sin fex þ cos fey
r @r r @u @z r
  ex ¼ sin u cos fer þ cos u cos feu  sin fef
1 @ 1 @tuu @t zu tur  t ru
þ eu 2 ðr 2 tru Þ þ þ þ þ
r @r r @u @z r ey ¼ sin u sin fer þ cos u sin feu þ cos fef
 
1@ 1 @t uz @t zz ez ¼ cos uer  sin ueu
þ ez ðrt rz Þ þ þ
r @r r @u @z

@vr @vu @vz Derivatives of the unit vectors are

rv ¼ er er þ er eu þ er ez þ
@r @r @r
    @er @eu
1 @vr vu 1 @vu vr ¼ eu ; ¼ er
þ eu er  þ eu eu þ þ @u @u
r @u r r @u r
@er @eu
¼ ef sin u; ¼ ef cos u;
1 @vz @vr @vu @vz @f @f
þ eu ez þ ez er þ ez eu þ ez ez
r @u @z @z @z
@ef
¼ er sin u  eu cos u
    @f
@ 1@ 1 @ 2 vr @ 2 vr 2 @vu
r2 v ¼ er ðr vr Þ þ 2 2 þ 2  2 þ
@r r @r r @u @z r @u
   
Others 0
@ 1@ 1 @ 2 vu @ 2 vu 2 @vr Differential operators are given by [45]
þ eu ðr vu Þ þ 2 2 þ 2 þ 2 þ
@r r @r r @u @z r @u
    @ 1@ 1 @
1@ @vz 1 @ 2 vz @ 2 vz r ¼ er þ eu þ ef ;
þ ez r þ 2 2 þ 2 @r r @u r sin u @f
r @r @r r @u @z
52 Mathematics in Chemical Engineering

@c 1 @c 1 @c
rc ¼ er
@r
þ eu
r @u
þ ef
r sin u @f
6. Ordinary Differential Equations
as Initial Value Problems
   
1 @ @c 1 @ @c
r2 c ¼ r2 þ 2 sin u þ
r @r
2 @r r sin u @u @u A differential equation for a function that
1 @2c
depends on only one variable (often time) is
þ 2 2 called an ordinary differential equation. The
r sin u @f2
general solution to the differential equation
1 @ 2 1 @ includes many possibilities; the boundary or
rv¼ ðr vr Þ þ ðvu sin uÞþ
r2 @r r sin u @u initial conditions are required to specify which
1 @vf of those are desired. If all conditions are at one
þ
r sin u @f point, the problem is an initial value problem
and can be integrated from that point on. If
 
1 @ 1 @vu some of the conditions are available at one point
r  v ¼ er ðvf sin uÞ  þ
r sin u @u r sin u @f and others at another point, the ordinary
 
1 @vr 1 @ differential equations become two-point bound-
þeu  ðr vf Þ þ
r sin u @f r @r ary value problems, which are treated in
  Chapter 7. Initial value problems as ordinary
1@ 1 @vr
þef ðr vu Þ 
r @r r @u differential equations arise in control of
 lumped-parameter models, transient models
1 @ 2 1 @ of stirred tank reactors, polymerization
r  t ¼ er ðr t rr Þ þ ðt ur sin uÞ
r2 @r r sin u @u
 reactions and plug-flow reactors, and generally
1 @tfr t uu þ t ff
þ
r sin u @f

r
þ in models where no spatial gradients occur in
 the unknowns.
1 @ 1 @ 1 @tfu
þeu 3 ðr3 t ru Þ þ ðt uu sin uÞ þ
r @r r sin u @u r sin u @f

tur  t ru  t ff cot u
þ þ

r 6.1. Solution by Quadrature
1 @ 1 @ 1 @tff
þef 3 ðr 3 t rf Þ þ ðt uf sin uÞ þ
r @r r sin u @u r sin u @f
 When only one equation exists, even if it is
tfr  t rf þ t fu cot u
þ
r
nonlinear, solving it by quadrature may be
possible. For
@vr @vu @vf
rv ¼ er er þ er eu þ er ef þ
@r @r @r dy
    ¼ f ðyÞ
1 @vr vu 1 @vu vr dt
þeu er  þ eu eu þ þ
r @u r r @u r y ð0Þ ¼ y0
 
1 @vf 1 @vr vf
þeu ef þ ef er  þ
r @u r sinu @f r the problem can be separated
 
1 @vu vf
þef eu  cot u þ
r sinu @f r dy
¼ dt
  f ðyÞ
1 @vf vr vu
þef ef þ þ cot u
r sinu @f r r
     and integrated:
@ 1 @ 2 1 @ @vr
r2 v ¼ er ðr vr Þ þ 2 sin u
@r r2 @r r sin u @u @u
 Zy Zt
1 @ 2 vr 2 @ 2 @vf dy0
þ 2 2  ðvu sin uÞ  2 ¼ dt ¼ t
r sin u @f2 r2 sin u @u r sin u @f f ðy0 Þ
y0 0
    
1 @ @vu 1 @ 1 @
þeu r2 þ 2 ðvu sin uÞ
r @r
2 @r r @u sin u @u If the quadrature can be performed analytically

1 @ 2 vu 2 @vr 2 cot u @vf then the exact solution has been found.
þ 2 2 þ  2
r sin u @f 2 r @u r sin u @f
2
     For example, consider the kinetics problem
1 @ @vf 1 @ 1 @
þef 2 r2 þ 2 ðvf sin uÞ with a second-order reaction.
r @r @r r @u sin u @u

1 @ 2 vf 2 @vr 2 cot u @vu
þ 2 2 þ þ 2 dc
r sin u @f 2 r sin u @f r sin u @f
2
¼  kc2 ; cð0Þ ¼ c0
dt
 Mathematics in Chemical Engineering 53

To find the function of the concentration versus by using Frechet differentials [51]. In this case,
time, the variables can be separated and inte-
      
grated. exp
Ft dc F
þ c ¼
d
exp
Ft
c
V dt V dt V
dc
¼  kdt;
c2 Thus, the differential equation can be written as
1
 ¼  kt þ D       
c d Ft Ft F
exp c ¼ exp cin
dt V V V
Application of the initial conditions gives the
solution: This can be integrated once to give

1 1   Zt  0
¼ kt þ Ft F Ft
c c0 exp c ¼ c ð0Þ þ exp cin ðt0 Þ dt0
V V V
0

For other ordinary differential equations an or

integrating factor is useful. Consider the prob-
lem governing a stirred tank with entering fluid  
Ft
having concentration cin and flow rate F, as c ðtÞ ¼ exp  c0
V
shown in Figure 18. The flow rate out is also F Zt  
and the volume of the tank is V. If the tank is F F ðt  t0 Þ
þ exp  cin ðt0 Þdt0
V V
completely mixed, the concentration in the tank 0
is c and the concentration of the fluid leaving
the tank is also c. The differential equation is If the integral on the right-hand side can be
then calculated, the solution can be obtained analyt-
ically. If not, the numerical methods described
dc in the next sections can be used. Laplace trans-
V ¼ F ðcin  cÞ; c ð0Þ ¼ c0
dt
forms can also be attempted. However, an
analytic solution is so useful that quadrature
Upon rearrangement, and an integrating factor should be tried before
resorting to numerical solution.
dc F F
þ c ¼ cin
dt V V

is obtained. An integrating factor is used to 6.2. Explicit Methods

solve this equation. The integrating factor is a
function that can be used to turn the left-hand Consider the ordinary differential equation
side into an exact differential and can be found
dy
¼ f ðyÞ
dt

Multiple equations that are still initial value

problems can be handled by using the same
techniques discussed here. A higher order dif-
ferential equation
 
yðnÞ þ F yðn1Þ ; yðn2Þ ; . . . ; y0 ; y ¼ 0

with initial conditions

 
Gi yðn1Þ ð0Þ; yðn2Þ ð0Þ; . . . ; y0 ð0Þ; y ð0Þ ¼ 0

Figure 18. Stirred tank i ¼ 1; . . . ; n

54 Mathematics in Chemical Engineering

can be converted into a set of first-order equa- Euler’s method is first order
tions. By using
ynþ1 ¼ yn þ Dtf ðyn Þ
ði1Þ
d y d ði2Þ dyi1
yi yði1Þ ¼ ¼ y ¼
dtði1Þ dt dt
Adams–Bashforth Methods. The second-order
Adams–Bashforth method is
the higher order equation can be written as a set
of first-order equations: Dt
ynþ1 ¼ yn þ ½3 f ðyn Þ  f ðyn1 Þ
2
dy1
¼ y2
dt
The fourth-order Adams–Bashforth method is
dy2
¼ y3
dt
Dt
dy3 ynþ1 ¼ yn þ ½55 f ðyn Þ  59 f ðyn1 Þ þ 37 f ðyn2 Þ 9 f ðyn3 Þ
¼ y4 24
dt
...
Notice that the higher order explicit methods
dyn
¼  Fðyn1 ; yn2 ; . . . ; y2; y1 Þ require knowing the solution (or the right-hand
dt
side) evaluated at times in the past. Because
The initial conditions would have to be speci- these were calculated to get to the current time,
fied for variables y1(0), . . . , yn (0), or equiv- this presents no problem except for starting the
alently y (0), . . . , y(n1)(0). The set of evaluation. Then, Euler’s method may have to
equations is then written as be used with a very small step size for several
steps to generate starting values at a succession
dy of time points. The error terms, order of the
¼ fðy; tÞ
dt method, function evaluations per step, and
stability limitations are listed in Table 6.
All the methods in this chapter are described for The advantage of the fourth-order Adams–
a single equation; the methods apply to the Bashforth method is that it uses only one
multiple equations as well. Taking the single function evaluation per step and yet achieves
equation in the form high-order accuracy. The disadvantage is the
necessity of using another method to start.
dy
dt
¼ f ðyÞ Runge–Kutta Methods. Runge–Kutta methods
are explicit methods that use several function
evaluations for each time step. The general
multiplying by dt, and integrating once yields form of the methods is

Ztnþ1   X
v
dy 0 X
tnþ1
dt ¼ f y ðt0 Þ dt0 ynþ1 ¼ yn þ wi k i
dt0 tn i¼1
tn

with
This is
 X
i1 
ki ¼ Dt f tn þ ci Dt; yn þ aij kj
Ztnþ1 j¼1
dy 0
ynþ1 ¼ yn þ dt
dt0
tn
Runge–Kutta methods traditionally have been
writen for f (t, y) and that is done here, too. If
The last substitution gives a basis for the vari- these equations are expanded and compared
ous methods. Different interpolation schemes with a Taylor series, restrictions can be placed
for y (t) provide different integration schemes; on the parameters of the method to make it first
using low-order interpolation gives low-order order, second order, etc. Even so, additional
integration schemes [3]. parameters can be chosen. A second-order
 Mathematics in Chemical Engineering 55

Table 6. Properties of integration methods for ordinary differential equations

Method Error term Order Function evaluations per step Stability limit, l Dt

Explicit methods
h2 00
Euler 2 y 1 1 2.0

5 3 000
Second-order Adams–Bashforth 12 h y 2 1

251 5 ð5Þ
Fourth-order Adams–Bashforth 720 h y 4 1 0.3
Second-order Runge–Kutta
(midpoint) 2 2 2.0
Runge–Kutta–Gill 4 4 2.8
nþ1 nþ1
Runge–Kutta–Feldberg y z 5 6 3.0

Predictor–corrector methods
Second-order Runge–Kutta 2 2 2.0
Adams, fourth-order 2 2 1.3

Implicit methods, stability limit 1

Backward Euler 1 many, iterative 1*
Trapezoid rule 1 3 000
 12 h y 2 many, iterative 2*
Fourth-order Adams–Moulton 4 many, iterative 3*
*
Oscillation limit, l D t.

Runge–Kutta method is [52]; although the method is fourth-order, it

achieves fifth-order accuracy. The popular inte-
Dt n gration package RKF 45 is based on this
ynþ1 ¼ yn þ ½f þ f ðtn þ Dt; yn þ Dt f n Þ
2 method.
The midpoint scheme is another second-order k1 ¼ Dt f ðtn ; yn Þ
Runge–Kutta method:  
Dt k1
k2 ¼ Dt f tn þ ; yn þ
  4 4
Dt Dt n  
ynþ1 ¼ yn þ Dt f t n þ ; yn þ f 3 3 9
2 2 k3 ¼ Dt f tn þ Dt; yn þ k1 þ k2
8 32 32
 
A popular fourth-order method is the Runge– 12
k4 ¼ Dt f tn þ Dt; yn þ
1932
k1 
7200
k2 þ
7296
k3
Kutta–Gill method with the formulas 13 2197 2197 2197
 
439 3680 845
k5 ¼ Dt f tn þ Dt; yn þ k 1  8 k2 þ k3  k4
k1 ¼ Dt f ðtn ; yn Þ 216 513 4104
   
Dt k1 Dt 8 3544 1859 11
k2 ¼ Dt f tn þ ; yn þ k6 ¼ Dt f tn þ ; yn  k1 þ 2k2  k3 þ k4  k5
2 2 2 27 2565 4104 40
 
Dt ynþ1 ¼ yn þ
25
k1 þ
1408
k3 þ
2197 1
k4  k5
k3 ¼ Dt f tn þ ; yn þ a k1 þ b k2 216 2565 4104 5
2
16 6656
k4 ¼ Dt f ðtn þ Dt; yn þ c k2 þ d k3 Þ znþ1 ¼ yn þ k1 þ k3
135 12 825
1 1
ynþ1 ¼ yn þ ðk1 þ k4 Þ þ ðb k2 þ d k3 Þ 28 561
6 3 þ
9 2
pffiffiffi pffiffiffi 56 430 k4  k5 þ k6
21 2 2 50 55
a¼ ; b¼ ;
2 2
pffiffiffi pffiffiffi
2 2 The value of ynþ1  znþ1 is an estimate of the
c¼  ; d ¼1þ
2 2 error in ynþ1 and can be used in step-size
control schemes.
Another fourth-order Runge–Kutta method is Generally, a high-order method should be
given by the Runge–Kutta–Feldberg formulas used to achieve high accuracy. The Runge–
56 Mathematics in Chemical Engineering

Kutta–Gill method is popular because it is high For example, Euler’s method gives
order and does not require a starting method (as
does the fourth-order Adams–Bashforth ynþ1 ¼ yn  l Dtyn or ynþ1 ¼ ð1  l DtÞyn
method). However, it requires four function or rmn ¼ 1  l Dt
evaluations per time step, or four times as
many as the Adams–Bashforth method. For The stability limit is then
problems in which the function evaluations
are a significant portion of the calculation l Dt  2
time this might be important. Given the speed
of computers and the widespread availability of The Euler method will not oscillate provided
desktop computers, the efficiency of a method
is most important only for very large problems l Dt  1
that are going to be solved many times. For
other problems the most important criterion for The stability limits listed in Table 6 are
choosing a method is probably the time the user obtained in this fashion. The limit for the Euler
spends setting up the problem. method is 2.0; for the Runge–Kutta–Gill
The stability of an integration method is best method it is 2.785; for the Runge–Kutta–
estimated by determining the rational polyno- Feldberg method it is 3.020. The rational poly-
mial corresponding to the method. Apply this nomials for the various explicit methods are
method to the equation illustrated in Figure 19. As can be seen, the
methods approximate the exact solution well as
dy
¼  ly; yð0Þ ¼ 1 l Dt approaches zero, and the higher order
dt
methods give a better approximation at high
values of l Dt.
and determine the formula for rmn: In solving sets of equations

dy
ykþ1 ¼ rmn ðl DtÞyk ¼ A y þ f; yð0Þ ¼ y0
dt

The rational polynomial is defined as all the eigenvalues of the matrix A must be
examined. FINLAYSON [3] and AMUNDSON [54,
pn ðzÞ p. 197–199] both show how to transform these
rmn ðzÞ ¼  ez
qm ðzÞ

and is an approximation to exp(z), called a

Pade approximation. The stability limits are the
largest positive z for which

jr mn ðzÞj  1

The method is A acceptable if the inequality

holds for Re z > 0. It is A(0) acceptable if the
inequality holds for z real, z > 0 [53]. The
method will not induce oscillations about the
true solution provided

rmn ðzÞ > 0

A method is L acceptable if it is A acceptable

and
Figure 19. Rational approximations for explicit methods
lim r mn ðzÞ ¼ 0
a) Euler; b) Runge–Kutta–2; c) Runge–Kutta–Gill; d) Exact
z!1 curve; e) Runge–Kutta–Feldberg
 Mathematics in Chemical Engineering 57

equations into an orthogonal form so that each error estimate, then Dt is reduced to meet that
equation becomes one equation in one criterion. Also, Dt is increased to as large a
unknown, for which single equation analysis value as possible, because this shortens com-
applies. For linear problems the eigenvalues do putation time. If the local truncation error has
not change, so the stability and oscillation been achieved (and estimated) by using a step
limits must be satisfied for every eigenvalue size Dt1
of the matrix A. When solving nonlinear prob-
lems the equations are linearized about the LTE ¼ c Dtp1
solution at the local time, and the analysis
applies for small changes in time, after which and the desired error is e, to be achieved using a
a new analysis about the new solution must be step size Dt2
made. Thus, for nonlinear problems the eigen-
values keep changing.
Richardson extrapolation can be used to e ¼ c D tp2

improve the accuracy of a method. Step for-

ward one step Dt with a p-th order method. then the next step size Dt2 is taken from
Then redo the problem, this time stepping
 p
forward from the same initial point but in LTE Dt1
¼
two steps of length Dt/2, thus ending at the e Dt2
same point. Call the solution of the one-step
calculation y1 and the solution of the two-step Generally, things should not be changed too
calculation y2. Then an improved solution at often or too drastically. Thus one may choose
the new time is given by not to increase Dt by more than a factor (such
as 2) or to increase Dt more than once every so
2 p y2  y1 many steps (such as 5) [55]. In the most
y¼
2p  1 sophisticated codes the alternative exists to
change the order of the method as well. In this
This gives a good estimate provided Dt is small case, the truncation error of the orders one
enough that the method is truly convergent with higher and one lower than the current one are
order p. This process can also be repeated in the estimated, and a choice is made depending on
same way Romberg’s method was used for the expected step size and work.
quadrature (see Section 2.4).
The accuracy of a numerical calculation 6.3. Implicit Methods
depends on the step size used, and this is chosen
automatically by efficient codes. For example, By using different interpolation formulas,
in the Euler method the local truncation error involving ynþ1, implicit integration methods
LTE is can be derived. Implicit methods result in a
nonlinear equation to be solved for ynþ1 so that
Dt2 00 iterative methods must be used. The backward
LTE ¼ y
2 n
Euler method is a first-order method:
Yet the second derivative can be evaluated by
ynþ1 ¼ yn þ Dt f ðynþ1 Þ
using the difference formulas as
The trapezoid rule (see Section 2.4) is a second-
y00n ¼ rðDt y0n Þ ¼ Dt ðy0n  y0n  1 Þ ¼ Dt ðf n  f n  1 Þ
order method:

Thus, by monitoring the difference between the Dt

ynþ1 ¼ yn þ ½ f ðyn Þ þ f ðynþ1 Þ
right-hand side from one time step to another, 2
an estimate of the truncation error is obtained.
This error can be reduced by reducing Dt. If the When the trapezoid rule is used with the
user specifies a criterion for the largest local finite difference method for solving partial
58 Mathematics in Chemical Engineering

differential equations it is called the Crank– difference formulas. The formulas of various
Nicolson method. Adams methods exist as orders are [57]:
well, and the fourth-order Adams–Moulton
method is 1 : ynþ1 ¼ yn þ Dt f ðynþ1 Þ

4 n 1 n1 2
2 : ynþ1 ¼ y  y þ Dt f ðynþ1 Þ
Dt 3 3 3
ynþ1 ¼ yn þ ½9 f ðynþ1 Þ þ 19 f ðyn Þ  5 f ðyn1 Þ þ f ðyn2 Þ
24
18 n 9 n1 2 n2
3 : ynþ1 ¼ y  y þ y
11 11 11
The properties of these methods are given in 6
þ Dt f ðynþ1 Þ
Table 6. The implicit methods are stable for any 11
step size but do require the solution of a set of 48 n 36 n1 16 n2 3 n3
4 : ynþ1 ¼ y  y þ y  y
nonlinear equations, which must be solved 25 25 25 25
iteratively. An application to dynamic distilla- 12
tion problems is given in [56]. þ Dt f ðynþ1 Þ
25
All these methods can be written in the
300 n 300 n1 200 n2
form 5 : ynþ1 ¼ y  y þ y
137 137 137

75 n3 12 n4
X
k X
k  y þ y
137 137
y nþ1
¼ ai y nþ1i
þ Dt bi f ðy nþ1i
Þ
i¼1 i¼0
60
þ Dt f ðynþ1 Þ
137

or The stability properties of these methods are

determined in the same way as explicit meth-
ynþ1 ¼ Dt b0 f ðynþ1 Þ þ wn ods. They are always expected to be stable, no
matter what the value of Dt is, and this is
confirmed in Figure 20.
where wn represents known information. This
equation (or set of equations for more than one Predictor–corrector methods can be employed
differential equation) can be solved by using in which an explicit method is used to predict
successive substitution: the value of ynþ1. This value is then used in an
implicit method to evaluate f (ynþ1).
ynþ1; kþ1 ¼ Dt b0 f ðynþ1; k Þ þ wn

Here, the superscript k refers to an iteration

counter. The successive substitution method is
guaranteed to converge, provided the first
derivative of the function is bounded and a
small enough time step is chosen. Thus, if it
has not converged within a few iterations, Dt
can be reduced and the iterations begun again.
The Newton–Raphson method (see Section 1.3)
can also be used.
In many computer codes, iteration is allowed
to proceed only a fixed number of times (e.g.,
three) before Dt is reduced. Because a good
history of the function is available from previ-
ous time steps, a good initial guess is usually
possible.
The best software packages for stiff equa- Figure 20. Rational approximations for implicit methods
tions (see Section 6.4) use Gear’s backward a) Backward Euler; b) Exact curve; c) Trapezoid; d) Euler
 Mathematics in Chemical Engineering 59

6.4. Stiffness where e is the porosity of the catalyst, R is the

catalyst radius, and De is the effective diffusion
Why is it desirable to use implicit methods that coefficient inside the catalyst. The time con-
lead to sets of algebraic equations that must be stant for heat transfer is
solved iteratively whereas explicit methods
lead to a direct calculation? The reason lies tinternal heat transfer ¼
R 2 %s C s R 2
¼
in the stability limits; to understand their a ke
impact, the concept of stiffness is necessary.
When modeling a physical situation, the time where %s is the catalyst density, Cs is the cata-
constants governing different phenomena lyst heat capacity per unit mass, ke is the
should be examined. Consider flow through a effective thermal conductivity of the catalyst,
packed bed, as illustrated in Figure 21. and a is the thermal diffusivity. The time con-
The superficial velocity u is given by stants for diffusion of mass and heat through a
boundary layer surrounding the catalyst are
Q
u¼
Af
R
texternal diffusion ¼
kg
where Q is the volumetric flow rate, A is the %s C s R
cross-sectional area, and f is the void fraction. texternal heat transfer ¼
hp
A time constant for flow through the device is
then where kg and hp are the mass-transfer and heat-
transfer coefficients, respectively. The impor-
L fAL
tflow ¼
u
¼
Q
tance of examining these time constants comes
from realization that their orders of magnitude
where L is the length of the packed bed. If a differ greatly. For example, in the model of an
chemical reaction occurs, with a reaction rate automobile catalytic converter [58] the time
given by constant for internal diffusion was 0.3 s, for
internal heat transfer 21 s, and for flow through
Moles the device 0.003 s. Flow through the device is
¼ k c so fast that it might as well be instantaneous.
Volume time
Thus, the time derivatives could be dropped
where k is the rate constant (time1) and c is the from the mass balance equations for the flow,
concentration (moles/volume), the characteris- leading to a set of differential-algebraic equa-
tic time for the reaction is tions (see below). If the original equations had
to be solved, the eigenvalues would be roughly
trxn ¼
1 proportional to the inverse of the time con-
k stants. The time interval over which to integrate
would be a small number (e.g., five) multiplied
If diffusion occurs inside the catalyst, the time
by the longest time constant. Yet the explicit
constant is
stability limitation applies to all the eigenval-
ues, and the largest eigenvalue would determine
e R2
tinternal diffusion ¼
De
the largest permissible time step. Here, l ¼
1/0.003 s1. Very small time steps would have
to be used, e.g., Dt  2  0.003 s, but a long
integration would be required to reach steady
state. Such problems are termed stiff, and
implicit methods are very useful for them. In
that case the stable time constant is not of any
interest, because any time step is stable. What is
of interest is the largest step for which a solution
can be found. If a time step larger than the
Figure 21. Flow through packed bed smallest time constant is used, then any
60 Mathematics in Chemical Engineering

phenomena represented by that smallest time column) are

constant will be overlooked — at least transi-
ents in it will be smeared over. However, the dxn
M ¼ V nþ1 ynþ1  Ln xn  V n yn þ Ln1 xn1
method will still be stable. Thus, if the very dt
rapid transients of part of the model are not of xn1  xn ¼ En ðxn1  x ;n Þ


interest, they can be ignored and an implicit

method used [59]. X
N
xi ¼ 1
The idea of stiffness is best explained by i¼1
considering a system of linear equations:
where x and y are the mole fractions in the
dy
¼ Ay liquid and vapor, respectively; L and V are
dt
liquid and vapor flow rates, respectively; M
is the holdup; and the superscript n is the stage
Let li be the eigenvalues of the matrix A. This number. The efficiency is E, and the concen-
system can be converted into a system of n tration in equilibrium with the vapor is x . The
equations, each of them having only one first equation is an ordinary differential equa-
unknown; the eigenvalues of the new system tion for the mass of one component on the
are the same as the eigenvalues of the original stage, whereas the third equation represents a
system [3, pp. 39–42], [54, pp. 197–199]. constraint that the mass fractions add to one. As
Then the stiffness ratio SR is defined as a second example, the following kinetics prob-
[53, p. 32] lem can be considered:

maxi j Re ðli Þj
SR ¼ dc1
mini j Re ðli Þj ¼ f ðc1 ; c2 Þ
dt
dc2
SR ¼ 20 is not stiff, SR ¼ 103 is stiff, and SR dt
¼ k1 c1  k2 c22
¼ 106 is very stiff. If the problem is nonlinear,
the solution is expanded about the current
The first equation could be the equation for a
state:
stirred tank reactor, for example. Suppose both
X
k1 and k2 are large. The problem is then stiff,
dyi n
@f i
¼ f i ½yðtn Þ þ ½y  yj ðtn Þ but the second equation could be taken at
dt j¼1
@yj j
equilibrium. If
The question of stiffness then depends on the
c1 @2c2
eigenvalue of the Jacobian at the current time.
Consequently, for nonlinear problems the prob-
lem can be stiff during one time period and not The equilibrium condition is then
stiff during another. Packages have been devel-
oped for problems such as these. Although the c22 k1
¼ K
c1 k 2
chemical engineer may not actually calculate
the eigenvalues, knowing that they determine
the stability and accuracy of the numerical Under these conditions the problem becomes
scheme, as well as the step size employed, is
useful. dc1
¼ f ðc1 ; c2 Þ
dt
0 ¼ kl c1  k2 c22
6.5. Differential–Algebraic Systems
Thus, a differential-algebraic system of equa-
Sometimes models involve ordinary differen- tions is obtained. In this case, the second equa-
tial equations subject to some algebraic con- tion can be solved and substituted into the first
straints. For example, the equations governing to obtain differential equations, but in the gen-
one equilibrium stage (as in a distillation eral case that is not possible.
 Mathematics in Chemical Engineering 61

Differential-algebraic equations can be writ- 6.6. Computer Software

ten in the general notation
Efficient software packages are widely availa-
 
F t; y;
dy
¼0
ble for solving ordinary differential equations
dt as initial value problems. In each of the pack-
ages the user specifies the differential equation
or the variables and equations may be separated to be solved and a desired error criterion. The
according to whether they come primarily from package then integrates in time and adjusts the
differential [y(t)] or algebraic equations [x(t)]: step size to achieve the error criterion, within
the limitations imposed by stability.
dy
¼ f ðt; y; xÞ; g ðt; y; xÞ ¼ 0 A popular explicit Runge–Kutta package is
dt
RKF 45. An estimate of the truncation error at
Another form is not strictly a differential-alge- each step is available. Then the step size can be
braic set of equations, but the same principles reduced until this estimate is below the user-
apply; this form arises frequently when the specified tolerance. The method is thus auto-
Galerkin finite element is applied: matic, and the user is assured of the results.
Note, however, that the tolerance is set on the
dy local truncation error, namely, from one step to
A ¼ f ðyÞ another, whereas the user is generally interested
dt
in the global trunction error, i.e., the error after
The computer program DASSL [60, 61] can several steps. The global error is generally
solve such problems. They can also be solved made smaller by making the tolerance smaller,
by writing the differential equation as but the absolute accuracy is not the same as the
tolerance. If the problem is stiff, then very small
dy step sizes are used and the computation
¼ A1 fðyÞ
dt
becomes very lengthy. The RKF 45 code dis-
covers this and returns control to the user with a
When A is independent of y, the inverse (from message indicating the problem is too hard to
LU decompositions) need be computed only solve with RKF 45.
once. A popular implicit package is LSODE, a
In actuality, higher order backward-differ- version of Gear’s method [57] written by ALAN
ence Gear methods are used in the computer HINDMARSH at Lawrence Livermore Labora-
program DASSL [60, 61]. tory. In this package, the user specifies the
Differential-algebraic systems are more differential equation to be solved and the
complicated than differential systems because tolerance desired. Now the method is implicit
the solution may not always be defined. and, therefore, stable for any step size. The
PONTELIDES et al. [62] introduced the term accuracy may not be acceptable, however, and
“index” to identify possible problems. The sets of nonlinear equations must be solved.
index is defined as the minimum number of Thus, in practice, the step size is limited but
times the equations must be differentiated with not nearly so much as in the Runge–Kutta
respect to time to convert the system to a set methods. In these packages both the step
ofordinary differential equations. These higher size and the order of the method are adjusted
derivatives may not exist, and the process by the package itself. Suppose a k-th order
places limits on which variables can be given method is being used. The truncation error is
initial values. Sometimes the initial values must determined by the (k þ 1)-th order derivative.
be constrained by the algebraic equations [62]. This is estimated by using difference formulas
For a differential-algebraic system modeling a and the values of the right-hand sides at pre-
distillation tower, the index depends on the vious times. An estimate is also made for the
specification of pressure for the column [62]. k-th and (k þ 2)-th derivative. Then, the errors
Several chemical engineering examples of dif- in a (k  1)-th order method, a k-th order
ferential-algebraic systems and a solution for method, and a (k þ 1)-th order method can be
one involving two-phase flow are given in [63]. estimated. Furthermore, the step size required to
62 Mathematics in Chemical Engineering

satisfy the tolerance with each of these methods Netlib web site, http://www.netlib.org/, choose
can be determined. Then the method and step “ode” to find packages which can be
size for the next step that achieves the biggest downloaded. Using Microsoft Excel to solve
step can be chosen, with appropriate adjustments ordinary differential equations is cumbersome,
due to the different work required for each order. except for the simplest problems.
The package generally starts with a very small
step size and a first-order method — the back-
ward Euler method. Then it integrates along, 6.7. Stability, Bifurcations, Limit
adjusting the order up (and later down) depend- Cycles
ing on the error estimates. The user is thus
assured that the local truncation error meets In this section, bifurcation theory is discussed
the tolerance. A further difficulty arises because in a general way. Some aspects of this subject
the set of nonlinear equations must be solved. involve the solution of nonlinear equations;
Usually a good guess of the solution is available, other aspects involve the integration of ordinary
because the solution is evolving in time and differential equations; applications include
past history can be extrapolated. Thus, the chaos and fractals as well as unusual operation
Newton–Raphson method will usually converge. of some chemical engineering equipment. An
The package protects itself, though, by only excellent introduction to the subject and details
doing a few (i.e., three) iterations. If convergence needed to apply the methods are given in [65].
is not reached within these iterations, the step For more details of the algorithms described
size is reduced and the calculation is redone for below and a concise survey with some chemical
that time step. The convergence theorem for the engineering examples, see [66] and [67]. Bifur-
Newton–Raphson method (Chap. 1) indicates cation results are closely connected with stabil-
that the method will converge if the step size ity of the steady states, which is essentially a
is small enough. Thus, the method is guaranteed transient phenomenon.
to work. Further economies are possible. The Consider the problem
Jacobian needed in the Newton–Raphson
method can be fixed over several time steps. @u
¼ F ðu; lÞ
Then if the iteration does not converge, the @t
Jacobian can be reevaluated at the current
time step. If the iteration still does not converge, The variable u can be a vector, which makes F a
then the step size is reduced and a new Jacobian vector, too. Here, F represents a set of equations
is evaluated. The successive substitution method that can be solved for the steady state:
can also be used — which is even faster, except
that it may not converge. However, it too will F ðu; lÞ ¼ 0
converge if the time step is small enough.
The Runge–Kutta methods give extremely If the Newton–Raphson method is applied,
good accuracy, especially when the step size is
F su d us ¼ F ðus ; lÞ
kept small for stability reasons. If the problem
is stiff, though, backward difference implicit usþ1 ¼ us þ dus
methods must be used. Many chemical reactor
problems are stiff, necessitating the use of is obtained, where
implicit methods. In the MATLAB suite of
ODE solvers, ode45 uses a revision of the @F s
F su ¼ ðu Þ
RKF45 program, while the ode15s program @u
uses an improved backward difference method.
Ref. [64] gives details of the programs in MAT- is the Jacobian. Look at some property of the
LAB. Fortunately, many packages are available. solution, perhaps the value at a certain point or
On the NIST web page, http://gams.nist.gov/ the maximum value or an integral of the solu-
choose “problem decision tree”, and then tion. This property is plotted versus the param-
“differential and integral equations” to find eter l; typical plots are shown in Figure 22. At
packages which can be downloaded. On the the point shown in Figure 22 A, the determinant
 Mathematics in Chemical Engineering 63

Figure 22. Limit points and bifurcation–limit points

A) Limit point (or turning point); B) Bifurcation-limit point (or singular turning point or bifurcation point)

of the Jacobian is zero: gives

det F u ¼ 0 sest X ¼ F ss st
ue X

For the limit point, The exponential term can be factored out and

@F ðF ss
u  s dÞX ¼ 0
6¼ 0
@l
A solution exists for X if and only if
whereas for the bifurcation-limit point
det jF ss
u  s dj ¼ 0
@F
¼0
@l
The s are the eigenvalues of the Jacobian.
The stability of the steady solutions is also of Now clearly if Re (s) > 0 then u0 grows with
interest. Suppose a steady solution uss; the time, and the steady solution uss is said to be
function u is written as the sum of the known unstable to small disturbances. If Im (s) ¼ 0 it
steady state and a perturbation u0 : is called stationary instability, and the distur-
bance would grow monotonically, as indicated
u ¼ uss þ u0 in Figure 23A. If Im (s) 6¼ 0 then the distur-
bance grows in an oscillatory fashion, as shown
This expression is substituted into the original in Figure 23B, and is called oscillatory
equation and linearized about the steady-state instability. The case in which Re (s) ¼ 0 is
value: the dividing point between stability and
instability. If Re (s) ¼ 0 and Im (s) ¼ 0 —
@uss @u0
þ ¼ F ðuss þ u0 ; lÞ the point governing the onset of stationary
@t @t
instability — then s ¼ 0. However, this means
@F
 F ðuss ; lÞ þ ju u0 þ    that s ¼ 0 is an eigenvalue of the Jacobian, and
@u ss
the determinant of the Jacobian is zero. Thus,
The result is the points at which the determinant of the
Jacobian is zero (for limit points and bifurca-
@u0 0
tion-limit points) are the points governing the
¼ F ss
u u
@t onset of stationary instability. When Re (s) ¼ 0
but Im (s) 6¼ 0, which is the onset of oscillatory
A solution of the form instability, an even number of eigenvalues pass
from the left-hand complex plane to the right-
u0 ðx; tÞ ¼ est XðxÞ hand complex plane. The eigenvalues are
64 Mathematics in Chemical Engineering

Figure 23. Stationary and oscillatory instability

A) Stationary instability; B) Oscillatory instability

complex conjugates of each other (a result of the and apply Newton–Raphson with this initial
original equations being real, with no complex guess and the new value of l. This will be a
numbers), and this is called a Hopf bifurcation. much better guess of the new solution than just
Numerical methods to study Hopf bifurcation u0 by itself.
are very computationally intensive and are not Even this method has difficulties, however.
discussed here [65]. Near a limit point the determinant of the
To return to the problem of solving for the Jacobian may be zero and the Newton method
steady-state solution: near the limit point or may fail. Perhaps no solutions exist at all for the
bifurcation-limit point two solutions exist that chosen parameter l near a limit point. Also, the
are very close to each other. In solving sets of ability to switch from one solution path to
equations with thousands of unknowns, the another at a bifurcation-limit point is necessary.
difficulties in convergence are obvious. For Thus, other methods are needed as well:
some dependent variables the approximation arc-length continuation and pseudo-arc-length
may be converging to one solution, whereas continuation [66]. These are described in
for another set of dependent variables it may be Chapter 1.
converging to the other solution; or the two
solutions may all be mixed up. Thus, solution is
difficult near a bifurcation point, and special 6.8. Sensitivity Analysis
methods are required. These methods are dis-
cussed in [66]. Often, when solving differential equations, the
The first approach is to use natural contin- solution as well as the sensitivity of the solution
uation (also known as Euler–Newton contin- to the value of a parameter must be known. Such
uation). Suppose a solution exists for some information is useful in doing parameter estima-
parameter l. Call the value of the parameter l0 tion (to find the best set of parameters for a
and the corresponding solution u0. Then model) and in deciding whether a parameter
needs to be measured accurately. The differential
F ðu0 ; l0 Þ ¼ 0 equation for y (t, a) where a is a parameter, is

Also, compute ul as the solution to dy

¼ f ðy; aÞ; y ð0Þ ¼ y0
dt
F ss
u ul ¼ F l
If this equation is differentiated with respect to
at this point [l0, u0]. Then predict the starting a, then because y is a function of t and a
guess for another l using  
@ dy @ f @y @ f
¼ þ
u0 ¼ u0 þ ul ðl  l0 Þ @a dt @y @a @a
 Mathematics in Chemical Engineering 65

Exchanging the order of differentiation in the that depends upon the location of all the parti-
first term leads to the ordinary differential equa- cles (but not their velocities). Since the major
tion part of the calculation is in the evaluation of the
forces, or potentials, a method must be used that
 
d @y @ f @y @ f minimizes the number of times the forces are
¼ þ
dt @a @y @a @a calculated to move from one time to another
time. Rewrite this equation in the form of an
The initial conditions on @y/@a are obtained by acceleration.
differentiating the initial conditions
d 2 ri 1
¼ Fi ðfrgÞ ai
dt2 mi
@ @y
½y ð0; aÞ ¼ y0 ; or ð0Þ ¼ 0
@a @a
In the Verlot method, this equation is written
Next, let using central finite differences (Eq. 12). Note
that the accelerations do not depend upon the
@y velocities.
y1 ¼ y; y2 ¼
@a
ri ðt þ DtÞ ¼ 2ri ðtÞ  ri ðt  DtÞ þ ai ðtÞDt2
and solve the set of ordinary differential equa-
tions The calculations are straightforward, and no
explicit velocity is needed. The storage require-
dy1
¼ f ðy1 ; aÞ y1 ð0Þ ¼ y0 ment is modest, and the precision is modest (it
dt
is a second-order method). Note that one must
dy2 @ f @f
¼ ðy ; aÞ y2 þ y2 ð0Þ ¼ 0 start the calculation with values of {r} at time t
dt @y 1 @a
and tDt.
Thus, the solution y (t, a) and the derivative with In the Velocity Verlot method, an equation is
respect to a are obtained. To project the impact written for the velocity, too.
of a, the solution for a ¼ a1 can be used:
dvi
¼ ai
dt
@y
y ðt; aÞ ¼ y1 ðt; a1 Þ þ ðt; a1 Þða  a1 Þ þ   
@a
The trapezoid rule (see 2.4.) is applied to obtain
¼ y1 ðt; a1 Þ þ y2 ðt; a1 Þða  a1 Þ þ   

1
This is a convenient way to determine the vi ðt þ DtÞ ¼ vi ðtÞ þ ½ai ðtÞ þ ai ðt þ DtÞDt
2
sensitivity of the solution to parameters in the
problem. The position of the particles is expanded in a
Taylor series.

6.9. Molecular Dynamics 1

ri ðt þ DtÞ ¼ ri ðtÞ þ vi Dt þ ai ðtÞDt2
2
Special integration methods have been devel-
oped for molecular dynamics calculations due Beginning with values of {r} and {v} at time
to the structure of the equations. A very large zero, one calculates the new positions and then
number of equations are to be integrated, with the new velocities. This method is second order
the following form based on molecular inter- in Dt, too. For additional details, see [68–72].
actions between molecules

d 2 ri
7. Ordinary Differential Equations
mi ¼ Fi ðfrgÞ; Fi ðfrgÞ ¼ rV
dt2 as Boundary Value Problems
where mi is the mass of the i-th particle, ri is the Diffusion problems in one dimension lead to
position of the i-th particle, Fi is the force acting boundary value problems. The boundary con-
on the i-th particle, and V is the potential energy ditions are applied at two different spatial
66 Mathematics in Chemical Engineering

locations: at one side the concentration may be equation is [73]

fixed and at the other side the flux may be fixed.
Because the conditions are specified at two 1d Dp
ðrtÞ ¼ 
different locations the problems are not initial r dr L
value in character. To begin at one position and
integrate directly is impossible because at least where r is the radial position from the center of
one of the conditions is specified somewhere the pipe, t is the shear stress, Dp is the pressure
else and not enough conditions are available to drop along the pipe, and L is the length over
begin the calculation. Thus, methods have which the pressure drop occurs. The variables
been developed especially for boundary value are separated once
problems. Examples include heat and mass
transfer in a slab, reaction–diffusion problems Dp
in a porous catalyst, reactor with axial disper- dðrtÞ ¼  rdr
L
sion, packed beds, and countercurrent heat
transfer.
and then integrated to give

7.1. Solution by Quadrature rt ¼ 

Dp r2
þ c1
L 2
When only one equation exists, even if it is
nonlinear, it may possibly be solved by quad- Proceeding further requires choosing a consti-
rature. For tutive relation relating the shear stress and the
velocity gradient as well as a condition speci-
dy
¼ f ðyÞ
fying the constant. For a Newtonian fluid
dt
yð0Þ ¼ y0 dv
t ¼ h
dr
the problem can be separated
where v is the velocity and h the viscosity. Then
dy the variables can be separated again and the
¼ dt
f ðyÞ result integrated to give

and integrated Dp r 2
hv ¼  þ c1 lnr þ c2
L 4
Zy Zt
dy0
¼ dt ¼ t Now the two unknowns must be specified from
f ðy0 Þ
y0 0 the boundary conditions. This problem is a
two-point boundary value problem because
If the quadrature can be performed analytically, one of the conditions is usually specified at
the exact solution has been found. r ¼ 0 and the other at r ¼ R, the tube radius.
As an example, consider the flow of a non- However, the technique of separating variables
Newtonian fluid in a pipe, as illustrated and integrating works quite well.
in Figure 24. The governing differential When the fluid is non-Newtonian, it
may not be possible to do the second step
analytically. For example, for the Bird–
Carreau fluid [74, p. 171], stress and velocity
are related by
h0
t¼h
dv 2 ið1nÞ=2
1þl dr

where h0 is the viscosity at v ¼ 0 and l the time

Figure 24. Flow in pipe constant.
 Mathematics in Chemical Engineering 67

Putting this value into the equation for stress The model for a chemical reactor with axial
as a function of r gives diffusion is

h0 Dp r c1 1 dc2 dc
h
dv 2 ið1nÞ=2 ¼  L 2 þ r  ¼ DaRðcÞ
1þl Pe dz2 dz
dr

1 dc dc
 ð0Þ þ cð0Þ ¼ cin ; ð1Þ ¼ 0
This equation cannot be solved analytically for Pe dz dz
dv/dr, except for special values of n. For prob-
lems such as this, numerical methods must be where Pe is the Peclet number and Da the
used. Damk€ohler number.
The boundary conditions are due to
7.2. Initial Value Methods DANCKWERTS [75] and to WEHNER and WILHELM
[76]. This problem can be treated by using
An initial value method is one that utilizes the initial value methods also, but the method is
techniques for initial value problems but allows highly sensitive to the choice of the parameter s,
for an iterative calculation to satisfy all the as outlined above. Starting at z ¼ 0 and making
boundary conditions. Suppose the nonlinear small changes in s will cause large changes in
boundary value problem the solution at the exit, and the boundary con-
dition at the exit may be impossible to satisfy.
  By starting at z ¼ 1, however, and integrating
d2 y dy
¼ f x; y;
dx 2 dx backwards, the process works and an iterative
scheme converges in many cases [77]. How-
with the boundary conditions ever, if the problem is extremely nonlinear the
iterations may not converge. In such cases, the
a0 yð0Þ  a1
dy
ð0Þ ¼ a; ai  0 methods for boundary value problems
dx described below must be used.
dy Packages to solve boundary value problems
b0 yð1Þ  b1 ð1Þ ¼ b; bi  0
dx
are available on the internet. On the NIST web
page, http://gams.nist.gov/ choose “problem
Convert this second-order equation into two
decision tree”, and then “differential and integral
first-order equations along with the boundary
equations”, then “ordinary differential equa-
conditions written to include a parameter s.
tions”, “multipoint boundary value problems”.
du
On the Netlib web site, http://www.netlib.org/,
¼v search on “boundary value problem”. Any
dx
dv spreadsheet that has an iteration capability can
¼ f ðx; u; vÞ
dx be used with the finite difference method. Some
uð0Þ ¼ a1 s  c1 a packages for partial differential equations also
vð0Þ ¼ a0 s  c0 a
have a capability for solving one-dimensional
boundary value problems [e.g., Comsol
The parameters c0 and c1 are specified by the Multiphysics (formerly Femlab)].
analyst such that

a 1 c0  a 0 c1 ¼ 1 7.3. Finite Difference Method

This ensures that the first boundary condition is To apply the finite difference method, we first
satisfied for any value of parameter s. If the spread grid points through the domain. Figure 25
proper value for s is known, u (0) and u0 (0) can shows a uniform mesh of n points (nonuniform
be evaluated and the equation integrated as an meshes are also possible). The unknown, here c
initial value problem. The parameter s should (x), at a grid point xi is assigned the symbol ci ¼ c
be chosen iteratively so that the last boundary (xi). The finite difference method can be derived
condition is satisfied. easily by using a Taylor expansion of the
68 Mathematics in Chemical Engineering

To see how to solve a differential equation,

consider the equation for convection, diffusion,
and reaction in a tubular reactor:

Figure 25. Finite difference mesh; Dx uniform 1 d2 c dc

 ¼ Da RðcÞ
Pe dx2 dx

To evaluate the differential equation at the i-th

grid point, the finite difference representations
solution about this point. of the first and second derivatives can be used to
  give
dc  d2 c  Dx2
ciþ1 ¼ ci þ Dx þ þ ...
dx i dx i 2
2
1 ciþ1  2ci þ ci1 ciþ1  ci1
  ð8Þ  ¼ Da R ð13Þ
dc  d2 c  Dx2 Pe Dx2 2Dx
ci1 ¼ ci  Dx þ  ...
dx i dx2 i 2
This equation is written for i ¼ 2 to n  1 (i.e.,
These formulas can be rearranged and divided the internal points). The equations would then
by Dx to give be coupled but would involve the values of c1
and cn, as well. These are determined from the
  boundary conditions.
dc  ciþ1  ci d2 c  Dx
¼  2 þ ... ð9Þ
dx i Dx dx i 2 If the boundary condition involves a deriva-
tive, the finite difference representation of it
must be carefully selected; here, three possibil-
  ities can be written. Consider a derivative
dc  ci  ci1 d2 c  Dx
¼  þ ... ð10Þ
dx i Dx dx2 i 2 needed at the point i ¼ 1. First, Equation (9)
could be used to write
which are representations of the first derivative. 
dc  c2  c1
¼ ð14Þ
Alternatively the two equations can be sub- dx 1 Dx
tracted from each other, rearranged and divided
by Dx to give Then a second-order expression is obtained that
  is one-sided. The Taylor series for the point ciþ2
dc  ciþ1  ci1 d2 c  Dx2 is written:
¼  3 ð11Þ
dx i 2Dx dx i 3!  
dc  d2 c  4Dx2
ciþ2 ¼ ci þ  2Dx þ 2 
dx i dx i 2!
If the terms multiplied by Dx or Dx2 are neglec- 
d3 c  8Dx3
ted, three representations of the first derivative þ þ ...
dx3 i 3!
are possible. In comparison with the Taylor
series, the truncation error in the first two
Four times Equation (8) minus this equation,
expressions is proportional to Dx, and the meth-
with rearrangement, gives
ods are said to be first order. The truncation error
in the last expression is proportional to Dx2, and 
dc  3ci þ 4ciþ1  ciþ2
the method is said to be second order. Usually, ¼ þ OðDx2 Þ
dx i 2Dx
the last equation is chosen to ensure the best
accuracy. Thus, for the first derivative at point i ¼ 1
The finite difference representation of the
second derivative can be obtained by adding 
dc  3ci þ 4c2  c3
¼ ð15Þ
the two expressions in Equation (9). Rearrange- dx i 2Dx
ment and division by Dx2 give
  This one-sided difference expression uses only
d2 c  ciþ1  2ci þ ci1 d4 c  Dx2
 ¼  þ ... ð12Þ the points already introduced into the domain.
dx2 i Dx2 dx4 i 4!
The third alternative is to add a false point,
The truncation error is proportional to Dx2. outside the domain, as c0¼ c (x ¼ Dx). Then
 Mathematics in Chemical Engineering 69

the centered first derivative, Equation (11), can boundary point

be used:
1 c 2  c0
  þ c1 ¼ cin
dc  c2  c0 Pe 2Dx
¼
dx 1 2Dx cnþ1  cn1
¼0
2Dx
Because this equation introduces a new varia-
ble, another equation is required. This is plus Equation (13) at points i ¼ 1 through n.
obtained by also writing the differential equa- The sets of equations can be solved by using
tion (Eq. 13), for i ¼ 1. the Newton–Raphson method, as outlined in
The same approach can be taken at the other Section 1.2.
end. As a boundary condition, any of three Frequently, the transport coefficients (e.g.,
choices can be used: diffusion coefficient D or thermal conductivity)
depend on the dependent variable (concentra-
 tion or temperature, respectively). Then the
dc  cn  cn1
¼
dx n Dx differential equation might look like

dc  cn2  4cn1 þ 3cn  
¼
dx n
d dc
2Dx DðcÞ ¼0
 dx dx
dc  cnþ1  cn1
¼
dx n 2Dx
This could be written as
The last two are of order Dx2 and the last one
would require writing the differential equation dJ
 ¼0 ð16Þ
dx
(Eq. 13) for i ¼ n, too.
Generally, the first-order expression for the
boundary condition is not used because the in terms of the mass flux J, where the mass flux
error in the solution would decrease only as is given by
Dx, and the higher truncation error of the
dc
differential equation (Dx2) would be lost. For J ¼ DðcÞ
dx
this problem the boundary conditions are
Because the coefficient depends on c the equa-
1 dc
 ð0Þ þ cð0Þ ¼ cin tions are more complicated. A finite difference
Pe dx
method can be written in terms of the fluxes at
dc
dx
ð1Þ ¼ 0 the midpoints, i þ 1/2. Thus,

Thus, the three formulations would give first J iþ1=2  J i1=2

 ¼0
Dx
order in Dx
1 c 2  c1 Then the constitutive equation for the mass flux
 þ c1 ¼ cin
Pe Dx
can be written as
cn  cn1
¼0
Dx
ciþ1  ci
J iþ1=2 ¼ Dðciþ1=2 Þ
Dx
plus Equation (13) at points i ¼ 2 through n  1;
second order in Dx, by using a three-point one- If these are combined,
sided derivative
1 3c1 þ 4c2  c3 Dðciþ1=2 Þðciþ1  ci Þ  Dðci1=2 Þðci  ci1 Þ
 þ c1 ¼ cin ¼0
Pe 2Dx Dx2
cn2  4cn1 þ 3cn
¼0 This represents a set of nonlinear algebraic
2Dx
equations that can be solved with the
plus Equation (13) at points i ¼ 2 through Newton–Raphson method. However, in this
n  1; second order in Dx, by using a false case a viable iterative strategy is to evaluate
70 Mathematics in Chemical Engineering

the transport coefficients at the last value and particularly for chemical reaction engineering.
then solve In the collocation method [3], the dependent
variable is expanded in a series.
Dðckiþ1=2 Þðckþ1 kþ1
iþ1  ci Þ  Dðcki1=2 Þðcikþ1  ckþ1
i1 Þ
¼0
Dx2 X
Nþ2
yðxÞ ¼ ai yi ðxÞ ð18Þ
i¼1
The advantage of this approach is that it is
easier to program than a full Newton–Raphson
method. If the transport coefficients do not vary Suppose the differential equation is
radically, the method converges. If the method
does not converge, use of the full Newton– N½y ¼ 0
Raphson method may be necessary.
Three ways are commonly used to evaluate Then the expansion is put into the differential
the transport coefficient at the midpoint. The equation to form the residual:
first one employs the transport coefficient eval-
uated at the average value of the solutions on X
Nþ2
Residual ¼ N½ ai yi ðxÞ
either side: i¼1

 
1
Dðciþ1=2 Þ  D ðciþ1 þ ci Þ
2 In the collocation method, the residual is set to
zero at a set of points called collocation
The second approach uses the average of the points:
transport coefficients on either side:
XN þ2

1 N½ ai yi ðxj Þ ¼ 0; j ¼ 2; . . . ; N þ 1
Dðciþ1=2 Þ  ½Dðciþ1 Þ þ Dðci Þ ð17Þ i¼1
2

This provides N equations; two more equa-

The truncation error of these approaches is also
tions come from the boundary conditions, giv-
Dx2 [78, Chap. 14], [3, p. 215]. The third
ing N þ 2 equations for N þ 2 unknowns. This
approach employs an “upstream” transport
procedure is especially useful when the expan-
coefficient.
sion is in a series of orthogonal polynomials,
and when the collocation points are the roots to
DðCiþ1=2 Þ  Dðciþ1 Þ; when Dðciþ1 Þ > Dðci Þ
an orthogonal polynomial, as first used by
Dðciþ1=2 Þ  ðci Þ; when Dðciþ1 Þ < Dðci Þ LANCZOS [80, 81]. A major improvement was
the proposal by VILLADSEN and STEWART [82]
This approach is used when the transport coef- that the entire solution process be done in terms
ficients vary over several orders of magnitude of the solution at the collocation points rather
and the “upstream” direction is defined as the than the coefficients in the expansion. Thus,
one in which the transport coefficient is larger. Equation (18) would be evaluated at the collo-
The truncation error of this approach is only Dx cation points
[78, Chap. 14], [3, p. 253], but this approach is
useful if the numerical solutions show X
N þ2

unrealistic oscillations [3, 78]. yðxj Þ ¼ ai yi ðxj Þ; j ¼ 1; . . . ; N þ 2

i¼1
Rigorous error bounds for linear ordinary
differential equations solved with the finite and solved for the coefficients in terms of the
difference method are dicussed by ISAACSON solution at the collocation points:
and KELLER [79, p. 431].
X
N þ2
ai ¼ ½yi ðxj Þ1 yðxj Þ; i ¼ 1; . . . ; N þ 2
7.4. Orthogonal Collocation j¼1

The orthogonal collocation method has found Furthermore, if Equation (18) is differentiated
widespread application in chemical engineering, once and evaluated at all collocation points, the
 Mathematics in Chemical Engineering 71

first derivative can be written in terms of the Note that 1 is the first collocation point (x ¼ 0)
values at the collocation points: and N þ 2 is the last one (x ¼ 1). To apply the
method, the matrices Aij and Bij must be found
dy X
Nþ2
dy and the set of algebraic equations solved, per-
ðxj Þ ¼ ai i ðxj Þ; j ¼ 1; . . . ; N þ 2
dx i¼1
dx haps with the Newton–Raphson method. If
orthogonal polynomials are used and the col-
location points are the roots to one of the
This can be expressed as orthogonal polynomials, the orthogonal collo-
cation method results.
dy XN þ2
dy In the orthogonal collocation method the
ðxj Þ ¼ ½yi ðxk Þ1 yðxk Þ i ðxj Þ; j ¼ 1; . . . ; N þ 2
dx i;k¼1
dx solution is expanded in a series involving
orthogonal polynomials, where the polynomi-
als Pi1(x) are defined in Section 2.2.
or shortened to
X
Nþ2 X
N
dy y ¼ a þ bx þ xð1  xÞ ai Pi1 ðxÞ
ðxj Þ ¼ Ajk yðxk Þ;
dx k¼1 i¼1
ð22Þ
X
N þ2
dyi X
Nþ2
Ajk ¼ ½yi ðxk Þ1 ðxj Þ ¼ bi Pi1 ðxÞ
i¼1
dx i¼1

Similar steps can be applied to the second which is also

derivative to obtain
X
Nþ2

X
N þ2 y¼ di xi1
d2 y
ðxj Þ ¼ Bjk yðxk Þ; i¼1
dx2 k¼1

X
Nþ2
d2 yi
Bjk ¼ ½yi ðxk Þ1 ðxj Þ The collocation points are shown in Figure 26.
i¼1
dx2
There are N interior points plus one at each end,
This method is next applied to the differ- and the domain is always transformed to lie on
ential equation for reaction in a tubular 0 to 1. To define the matrices Aij and Bij this
reactor, after the equation has been made expression is evaluated at the collocation point-
nondimensional so that the dimensionless s; it is also differentiated and the result is
length is 1.0. evaluated at the collocation points.

X
Nþ2
1 d2 c dc yðxj Þ ¼ di xi1
 ¼ Da RðcÞ; j
Pe dx2 dx i¼1
ð19Þ
dc dc
 ð0Þ ¼ Pe½cð0Þ  cin ; ð1Þ ¼ 0 dy X
Nþ2
dx dx ðxj Þ ¼ di ði  1Þxji2
dx i¼1

The differential equation at the collocation

d2 y X
Nþ2
points is ðxj Þ ¼ di ði  1Þði  2Þxji3
dx2 i¼1

1 XNþ2 X
Nþ2
Bjk cðxk Þ  Ajk cðxk Þ ¼ Da Rðcj Þ ð20Þ
Pe k¼1 k¼1

and the two boundary conditions are

X
Nþ2
 Alk cðxk Þ ¼ Peðc1  cin Þ;
k¼1
ð21Þ
X
N þ2
ANþ2;k cðxk Þ ¼ 0
k¼1 Figure 26. Orthogonal collocation points
72 Mathematics in Chemical Engineering

These formulas are put in matrix notation, found; and once d is known, the solution for
where Q, C, and D are N þ 2 by N þ 2 matrices. any x can be found.
To use the orthogonal collocation method,
y ¼ Qd;
dy d2 y
¼ Cd; 2 ¼ Dd
the matrices are required. They can be calcu-
dx dx lated as shown above for small N (N < 8) and by
Qji ¼ xji1 ; Cji ¼ ði  1Þxji2 ; using more rigorous techniques, for higher N
Dji ¼ ði  1Þði  2Þxi3
(see Chap. 7). However, having the matrices
j
listed explicitly for N ¼ 1 and 2 is useful; this is
In solving the first equation for d, the first and shown in Table 7.
For some reaction diffusion problems, the
second derivatives can be written as
solution can be an even function of x. For
dy
example, for the problem
d ¼ Q1 y; ¼ CQ1 y ¼ Ay;
dx
ð23Þ
d2 y d2 c dc
¼ DQ1 y ¼ By ¼ kc; ð0Þ ¼ 0; cð1Þ ¼ 1 ð24Þ
dx2 dx2 dx

Thus the derivative at any collocation point the solution can be proved to involve only even
can be determined in terms of the solution at powers of x. In such cases, an orthogonal
the collocation points. The same property is collocation method, which takes this feature
enjoyed by the finite difference method (and into account, is convenient. This can easily
the finite element method described below), be done by using expansions that only involve
and this property accounts for some of even powers of x. Thus, the expansion
the popularity of the orthogonal collocation
X
N
method. In applying the method to yðx2 Þ ¼ yð1Þ þ ð1  x2 Þ ai Pi1 ðx2 Þ
Equation (19), the same result is obtained; i¼1

Equations (20) and (21), with the matrices

defined in Equation (23). To find the solution is equivalent to
at a point that is not a collocation point,
Equation (22) is used; once the solution is X
Nþ1 X
Nþ1
yðx2 Þ ¼ bi Pi1 ðx2 Þ ¼ d i x2i2
known at all collocation points, d can be i¼1 i¼1

Table 7. Matrices for orthogonal collocation

 Mathematics in Chemical Engineering 73

The polynomials are defined to be orthogonal orthogonal collocation is applied at the interior
with the weighting function W(x2). points

Z1 X
N þ1
Bji ci ¼ f2 Rðcj Þ; j ¼ 1; . . . ; N
Wðx2 ÞPk ðx2 ÞPm ðx2 Þxa1 dx ¼ 0 i¼1
ð25Þ
0

k m1
and the boundary condition solved for is

cNþ1 ¼ 1
where the power on xa1 defines the geometry as
planar or Cartesian (a ¼ 1), cylindrical (a ¼ 2),
and spherical (a ¼ 3). An analogous develop- The boundary condition at x ¼ 0 is satisfied
ment is used to obtain the (N þ 1)(N þ 1) automatically by the trial function. After the
matrices solution has been obtained, the effectiveness
factor h is obtained by calculating
X
Nþ1
yðxj Þ ¼ di x2i2 R1 P
Nþ1
j R½cðxÞxa1 dx W j Rðcj Þ
i¼1 0 i¼1
h ¼
R1 P
Nþ1
dy X
N þ1 R½cð1Þxa1 dx W j Rð1Þ
ðxj Þ ¼ di ð2i  2Þx2i3
j 0 i¼1
dx i¼1

X
Nþ1 Note that the effectiveness factor is the average
r2 yðxi Þ ¼ d i r2 ðx2i2 Þjxj reaction rate divided by the reaction rate eval-
i¼1
uated at the external conditions. Error bounds
dy have been given for linear problems [83, p.
y ¼ Qd; ¼ Cd; r2 y ¼ Dd
dx
356]. For planar geometry the error is
Qji ¼ x2i2
j ; Cji ¼ ð2i  2Þx2i3
j ;
w2ð2Nþ1Þ
Dji ¼ r2 ðx2i2 Þjxj Error in h ¼
ð2N þ 1Þ!ð2N þ 2Þ!

dy
d ¼ Q1 y; ¼ CQ1 y ¼ Ay; This method is very accurate for small N (and
dx
small f2); note that for finite difference meth-
r2 y ¼ DQ1 y ¼ By
ods the error goes as 1/N2, which does not
decrease as rapidly with N. If the solution is
In addition, the quadrature formula is desired at the center (a frequent situation
because the center concentration can be the
WQ ¼ f; W ¼ fQ1
most extreme one), it is given by

X
N þ1
cð0Þ ¼ d1 ½Q1 1i yi
where i¼1

Z1 X
Nþ1 The collocation points are listed in Table 8.
x2i2 xa1 dx ¼ W j x2i2
j For small N the results are usually more
j¼1
0
accurate when the weighting function in
¼
1
fi Equation (25) is 1  x2. The matrices for
2i  2 þ a
N ¼ 1 and N ¼ 2 are given in Table 9 for
the three geometries. Computer programs to
As an example, for the problem generate matrices and a program to solve
reaction diffusion problems, OCRXN, are
  available [3, p. 325, p. 331].
1 d dc
xa1 ¼ f2 RðcÞ
xa1 dx dx Orthogonal collocation can be applied to
dc distillation problems. STEWART et al. [84, 85]
ð0Þ ¼ 0; cð1Þ ¼ 1
dx developed a method using Hahn polynomials
74 Mathematics in Chemical Engineering

Table 8. Collocation points for orthogonal collocation with The boundary conditions are typically
symmetric polynomials and W ¼ 1
dc dc
Geometry ð0Þ ¼ 0;  ð1Þ ¼ Bim ½cð1Þ  cB 
dx dx
N Planar Cylindrical Spherical

1 0.5773502692 0.7071067812 0.7745966692 where Bim is the Biot number for mass transfer.
2 0.3399810436 0.4597008434 0.5384693101 These become
0.8611363116 0.8880738340 0.9061793459
3 0.2386191861 0.3357106870 0.4058451514 1 dc
0.6612093865 0.7071067812 0.7415311856 ðu ¼ 0Þ ¼ 0;
Dx1 du
0.9324695142 0.9419651451 0.9491079123
4 0.1834346425 0.2634992300 0.3242534234
0.5255324099 0.5744645143 0.6133714327 in the first element;
0.7966664774 0.8185294874 0.8360311073
0.9602898565 0.9646596062 0.9681602395
5 0.1488743390 0.2165873427 0.2695431560 1 dc
 ðu ¼ 1Þ ¼ Bim ½cðu ¼ 1Þ  cB ;
0.4333953941 0.4803804169 0.5190961292 DxNE du
0.6794095683 0.7071067812 0.7301520056
0.8650633667 0.8770602346 0.8870625998
0.9739065285 0.9762632447 0.9782286581 in the last element. The orthogonal collocation
method is applied at each interior collocation
point.

that retains the discrete nature of a plate-to- 1 XNP

a1 1 X NP
B I J cJ þ AI J c J ¼
plate distillation column. Other work treats Dxk J¼1
2 x ðkÞ þ u I Dx k Dx k J¼1

problems with multiple liquid phases [86]. ¼ f2 RðcJ Þ; I ¼ 2; . . . ; N P  1

Some of the applications to chemical engineer-
ing can be found in [87–90].
The local points i ¼ 2, . . . , NP  1 represent
the interior collocation points. Continuity of the
7.5. Orthogonal Collocation on Finite function and the first derivative between ele-
Elements ments is achieved by taking

In the method of orthogonal collocation on 1 X NP

AN P;J cJ jelement k1
finite elements, the domain is first divided Dxk1 J¼1
into elements, and then within each element
1 XNP
orthogonal collocation is applied. Figure 27 ¼
Dxk J¼1
A1;J cJ jelement k
shows the domain being divided into NE ele-
ments, with NCOL interior collocation points
within each element, and NP ¼ NCOL þ 2 total at the points between elements. Naturally, the
points per element, giving NT ¼ NE  (NCOL þ computer code has only one symbol for the
1) þ 1 total number of points. Within each solution at a point shared between elements, but
element a local coordinate is defined the derivative condition must be imposed.
Finally, the boundary conditions at x ¼ 0 and
x  xðkÞ x ¼ 1 are applied:
u¼ ; Dxk ¼ xðkþ1Þ  xðkÞ
Dxk

1 XNP
The reaction–diffusion equation is written as Dxk J¼1
A1;J cJ ¼ 0;

 
1 d dc d2 c a  1 dc
xa1 ¼ 2þ ¼ f2 RðcÞ in the first element;
xa1 dx dx dx x dx

and transformed to give 1 X NP

 AN P;J cJ ¼ Bim ½cNP  cB ;
DxN E J¼1
1 d2 a1 1 dc
þ ¼ f2 RðcÞ
Dx2k du2 xðkÞ þ uDxk Dxk du in the last element.
 Mathematics in Chemical Engineering 75

Table 9. Matrices for orthogonal collocation with symmetric polynomials and W¼1x2
76 Mathematics in Chemical Engineering

Figure 27. Grid for orthogonal collocation on finite elements

These equations can be assembled into an the unknown solution is expanded in a series of
overall matrix problem known functions {bi(x)}, with unknown coef-
ficients {ai}.
A Ac ¼ f
X
NT
cðxÞ ¼ ai bi ðxÞ
The form of these equations is special and is i¼1

discussed by FINLAYSON [3, p. 116], who also

gives the computer code to solve linear equa- The series (the trial solution) is inserted into the
tions arising in such problems. Reaction–diffu- differential equation to obtain the residual:
sion problems are solved by the program X
NT  
1 d dbi
OCFERXN [3, p. 337]. See also the program Residual ¼ ai xa1
i¼1
xa1 dx dx
COLSYS described below. " #
XNT
The error bounds of DEBOOR [91] give the f2 R ai bi ðxÞ
following results for second-order problems i¼1

solved with cubic trial functions on finite ele-

ments with continuous first derivatives. The The residual is then made orthogonal to the set
error at all positions is bounded by of basis functions.
Z1 (   " #)
 i  XNT
1 d a1 dbi X
NT
d  bj ðxÞ ai x f2 R ai bi ðxÞ xa1 dx ¼ 0
 ðy  yexact Þ  constantjDxj2 xa1 dx dx
dxi  0
i¼1 i¼1
1
j ¼ 1; . . . ; N T ð26Þ

The error at the collocation points is more This process makes the method a Galerkin
accurate, giving what is known as superconver- method. The basis for the orthogonality condi-
gence. tion is that a function that is made orthogonal to
 i 
d  each member of a complete set is then zero. The
 ðy  yexact Þ  constantjDxj4
dxi 
collocation points residual is being made orthogonal, and if the
basis functions are complete, and an infinite
number of them are used, then the residual is
7.6. Galerkin Finite Element Method zero. Once the residual is zero the problem is
solved. It is necessary also to allow for the
In the finite element method the domain is boundary conditions. This is done by integrat-
divided into elements and an expansion is ing the first term of Equation (26) by parts and
made for the solution on each finite element. then inserting the boundary conditions:
In the Galerkin finite element method an addi- Z1  
1 d a1 dbi
tional idea is introduced: the Galerkin method bj ðxÞ
xa1 dx
x
dx
xa1 dx ¼
is used to solve the equation. The Galerkin 0

method is explained before the finite element Z1   Z1

d dbi dbj dbi a1
bj ðxÞxa1 dx  x dx
basis set is introduced. dx dx dx dx
0 0
To solve the problem   Z1
ð27Þ
dbi 1 dbj dbi a1
  ¼ bj ðxÞxa1  x dx
1 d dx 0 dx dx
a1 dc
x ¼ f2 RðcÞ 0
xa1 dx dx Z1
dbj dbi a1
dc dc ¼ x dx  Bim bj ð1Þ½bi ð1Þ  cB 
ð0Þ ¼ 0;  ð1Þ ¼ Bim ½cð1Þ  cB  dx dx
dx dx 0
 Mathematics in Chemical Engineering 77

Combining this with Equation (26) gives in the e-th element. Then the Galerkin method
is
NTZ
1
X dbj dbi a1
N PZ
1
 x dxai X 1 X dN J dN I
i¼1
dx dx  ðxe þ uDxe Þa1 duceI
0
e
Dxe I¼1 du du
0
" #
XNT " #
Bim bj ð1Þ ai bi ð1Þ  cB X X
NP
Bim N J ð1Þ ceI N I ð1Þ  c1
i¼1 ð28Þ
e I¼1 ð29Þ
Z1 " # " #
XNT
X Z1 XNP
¼ f2 bj ðxÞ ai bi ðxÞ xa1 dx ¼ f2 Dxe N J ðuÞR ceI N I ðuÞ
i¼1 e I¼1
0 0

j ¼ 1; . . . ; NT ðxe þ uDxe Þa1 du

This equation defines the Galerkin method, The element integrals are defined as
and a solution that satisfies this equation (for
all j ¼ 1, . . . , 1) is called a weak solution. Z1
For an approximate solution the equation is 1 dN J dN I
BeJI ¼  ðxe þ uDxe Þa1 du;
Dxe du du
written once for each member of the trial 0

function, j ¼ 1, . . . , NT. If the boundary Z1 "

XNP
#
condition is F eJ ¼ f2 Dxe N J ðuÞR ceI N I ðuÞ
I¼1
0

a1
cð1Þ ¼ cB ðxe þ uDxe Þ du

then the boundary condition is used (instead of whereas the boundary element integrals are
Eq. 29) for j ¼ NT,
B BeJI ¼ Bim N J ð1ÞN I ð1Þ;
X
NT
F F eJ ¼ Bim N J ð1Þc1
ai bi ð1Þ ¼ cB
i¼1

Then the entire method can be written in the

compact notation
The Galerkin finite element method results
when the Galerkin method is combined with a X X X X
BeJI ceI þ B BeJI ceI ¼ F eJ þ F F eJ
finite element trial function. Both linear and e e e e
quadratic finite element approximations
are described in Chapter 2. The trial functions The matrices for various terms are given in
bi (x) are then generally written as Ni(x). Table 10. This equation can also be written
in the form
X
NT
cðxÞ ¼ ci N i ðxÞ AAc ¼ f
i¼1

where the matrix AA is sparse. If linear ele-

Each Ni(x) takes the value 1 at the point xi and ments are used the matrix is tridiagonal. If
zero at all other grid points (Chap. 2). Thus ci quadratic elements are used the matrix is
are the nodal values, c (xi) ¼ ci. The first pentadiagonal. Naturally the linear algebra
derivative must be transformed to the local is most efficiently carried out if the sparse
coordinate system, u ¼ 0 to 1 when x goes structure is taken into account. Once the solu-
from xi to xi þ Dx. tion is found the solution at any point can be
recovered from
dN j 1 dN J
¼ ; dx ¼ Dxe du
dx Dxe du ce ðuÞ ¼ ceI¼1 ð1  uÞ þ ceI¼2 u
78 Mathematics in Chemical Engineering

Table 10. Element matrices for Galerkin method

for linear elements 7.8. Adaptive Mesh Strategies

  In many two-point boundary value problems,
1
ce ðuÞ ¼ ceI¼1 2ðu  1Þ u  the difficulty in the problem is the formation of
2
  a boundary layer region, or a region in which
1
þceI¼2 4uð1  uÞ þ ceI¼3 2u u  the solution changes very dramatically. In such
2
cases small mesh spacing should be used there,
either with the finite difference method or the
for quadratic elements finite element method. If the region is known a
Because the integrals in Equation (28) may priori, small mesh spacings can be assumed at
be complicated, they are usually formed by the boundary layer. If the region is not known
using Gaussian quadrature. If NG Gauss points though, other techniques must be used. These
are used, a typical term would be techniques are known as adaptive mesh tech-
niques. The general strategy is to estimate the
Z1 " # error, which depends on the grid size and
X
NP
N J ðuÞR ceI N I ðuÞ ðxe þ uDxe Þa1 du derivatives of the solution, and refine the
0
I¼1
mesh where the error is large.
" #
X
NG XNP The adaptive mesh strategy was employed
¼ W k N J ðuk ÞR ceI N I ðuk Þ by ASCHER et al. [93] and by RUSSELL and
k¼1 I¼1

a1
CHRISTIANSEN [94]. For a second-order differen-
ðxe þ uk Dxe Þ tial equation and cubic trial functions on finite
elements, the error in the i-th element is given by
For an application of the finite element method
k Error ki ¼ cDx4i k uð4Þ ki
in fluid mechanics, see ! Fluid Mechanics.
Because cubic elements do not have a nonzero
7.7. Cubic B-Splines fourth derivative, the third derivative in adjacent
elements is used [3, p. 166]:
Cubic B-splines have cubic approximations
1 d 3 ci 1 d3 ciþ1
within each element, but first and second deriv- ai ¼ ; aiþ1 ¼ 3
Dx3i du3 Dxiþ1 du3
atives continuous between elements. The func- 2 3
tions are the same ones discussed in Chapter 2, 1 6 ai  ai1 aiþ1  ai 7
k uð4Þ ki  4 þ 5
and they can be used to solve differential 2 1 1
ðxiþ1  xi1 Þ ðxiþ2  xi Þ
equations [92]. 2 2
 Mathematics in Chemical Engineering 79

Element sizes are then chosen so that the fol-

lowing error bounds are satisfied

CDx4i k uð4Þ ki  e for all i

These features are built into the code COLSYS

(http://www.netlib.org/ode/).
The error expected from a method one order
higher and one order lower can also be defined.
Then a decision about whether to increase or
decrease the order of the method can be made
by taking into account the relative work of the
different orders. This provides a method of
adjusting both the mesh spacing (Dx, some-
times called h) and the degree of polynomial
(p). Such methods are called h–p methods.
Figure 28. Concentration solution for different values of
Thiele modulus
7.9. Comparison

What method should be used for any given

problem? Obviously the error decreases with preferred. It gives a very accurate solution, and
some power of Dx, and the power is higher for N can be quite small so the work is small. If the
the higher order methods, which suggests that problem has a steep front in it, the finite differ-
the error is less. For example, with linear ence method or finite element method is
elements the error is indicated, and adaptive mesh techniques should
probably be employed. Consider the reaction–
yðDxÞ ¼ yexact þ c2 Dx2 diffusion problem: as the Thiele modulus f
increases from a small value with no diffusion
for small enough (and uniform) Dx. A computer limitations to a large value with significant
code should be run for varying Dx to confirm diffusion limitations, the solution changes as
this. For quadratic elements, the error is shown in Figure 28. The orthogonal collocation
method is initially the method of choice. For
yðDxÞ ¼ yexact þ c3 Dx3 intermediate values of f, N ¼ 3–6 must be used,
but orthogonal collocation still works well (for
If orthogonal collocation on finite elements is h down to approximately 0.01). For large f, use
used with cubic polynomials, then of the finite difference method, the finite ele-
ment method, or an asymptotic expansion for
yðDxÞ ¼ yexact þ c4 Dx4 large f is better. The decision depends entirely
on the type of solution that is obtained. For
However, the global methods, not using finite steep fronts the finite difference method and
elements, converge even faster [95], for exam- finite element method with adaptive mesh are
ple, indicated.
 NCOL
1
yðNÞ ¼ yexact þ cN
NCOL 7.10. Singular Problems and Infinite
Domains
Yet the workload of the methods is also differ-
ent. These considerations are discussed in [3]. If the solution being sought has a singularity, a
Here, only sweeping generalizations are given. good numerical solution may be hard to find.
If the problem has a relatively smooth solu- Sometimes even the location of the singularity
tion, then the orthogonal collocation method is may not be known [96, pp. 230–238]. One
80 Mathematics in Chemical Engineering

method of solving such problems is to refine the The Blasius equation becomes
mesh near the singularity, by relying on the
 
better approximation due to a smaller Dx. d3 f d2 f
2 z3 3  3z2 2  z
df
Another approach is to incorporate the singular dz dz dz
trial function into the approximation. Thus, if  
d2 df
þf z2 2 þ z ¼ 0 for 0  z  1:
the solution approaches f(x) as x goes to zero, dz dz
and f(x) becomes infinite, an approximation
may be taken as The differential equation now has variable
coefficients, but these are no more difficult to
X
N
handle than the original nonlinearities.
yðxÞ ¼ f ðxÞ þ ai yi ðxÞ
i¼1 Another approach is to use a variable mesh,
perhaps with the same transformation. For
This function is substituted into the differential example, use z ¼ eh and a constant mesh
equation, which is solved for ai. Essentially, a size in z. Then with 101 points distributed
new differential equation is being solved for a uniformly from z ¼ 0 to z ¼ 1, the following
new variable: are the nodal points:

uðxÞ yðxÞ  f ðxÞ z ¼ 0:; 0:01; 0:02; . . . ; 0:99; 1:0

h ¼ 1; 4:605; 3:912; . . . ; 0:010; 0

The differential equation is more complicated Dh ¼ 1; 0:639; . . . ; 0:01
but has a better solution near the singularity
(see [97, pp. 189–192], [98, p. 611]). Still another approach is to solve on a finite
Sometimes the domain is infinite. Boundary mesh in which the last point is far enough away
layer flow past a flat plate is governed by the that its location does not influence the solution.
Blasius equation for stream function [99, p. A location that is far enough away must be
117]. found by trial and error.

d3 d2
2 þf 2 ¼0
dh3 dh
8. Partial Differential Equations
df
f ¼ ¼ 0 at h ¼ 0
dh
Partial differential equations are differential
df
¼ 1 at h ! 1
equations in which the dependent variable is
dh a function of two or more independent varia-
bles. These can be time and one space dimen-
Because one boundary is at infinity using a sion, or time and two or more space dimensions,
mesh with a constant size is difficult! One or two or more space dimensions alone. Prob-
approach is to transform the domain. For exam- lems involving time are generally either hyper-
ple, let bolic or parabolic, whereas those involving
spatial dimensions only are often elliptic.
z ¼ eh Because the methods applied to each type of
equation are very different, the equation must
first be classified as to its type. Then the special
Then h ¼ 0 becomes z ¼ 1 and h ¼ 1 becomes methods applicable to each type of equation are
z ¼ 0. The derivatives are described. For a discussion of all methods, see
[100–103]; for a discussion oriented more
dz d2 z toward chemical engineering applications, see
¼ eh ¼ z; ¼ eh ¼ z
dh dh2
[104]. Examples of hyperbolic and parabolic
df df dz df equations include chemical reactors with radial
¼ ¼ z
dh dz dh dz
dispersion, pressure-swing adsorption, disper-
2  2
2
d f
¼
d f dz
þ
df d2 z d2 f
¼ z2 2 þ z
df sion of an effluent, and boundary value prob-
dh2 dz2 dh dz dh2 dz dz lems with transient terms added (heat transfer,
 Mathematics in Chemical Engineering 81

mass transfer, chemical reaction). Examples of type. The surface is defined by this equation
elliptic problems include heat transfer and mass plus a normalization condition:
transfer in two and three spatial dimensions and
steady fluid flow. X
n
s 2k ¼ 1
k¼0

8.1. Classification of Equations

The shape of the surface defined by
A set of differential equations may be hyper- Equation (30) is also related to the type: elliptic
bolic, elliptic, or parabolic, or it may be of equations give rise to ellipses; parabolic equa-
mixed type. The type may change for different tions give rise to parabolas; and hyperbolic
parameters or in different regions of the flow. equations give rise to hyperbolas.
This can happen in the case of nonlinear prob-
lems; an example is a compressible flow prob- s 21 s 22
þ ¼ 1; Ellipse
lem with both subsonic and supersonic regions. a2 b2
Characteristic curves are curves along which a s 0 ¼ as 21 ; Parabola
discontinuity can propagate. For a given set of
equations, it is necessary to determine if char- s 20  as 21 ¼ 0; Hyperbola
acteristics exist or not, because that determines
whether the equations are hyperbolic, elliptic,
If Equation (30) has no nontrivial real zeroes
or parabolic.
then the equation is called elliptic. If all the
Linear Problems For linear problems, the the- roots are real and distinct (excluding zero) then
ory summarized by JOSEPH et al. [105] can be the operator is hyperbolic.
used. This formalism is applied to three basic
types of equations. First consider the equation
@ @ @ arising from steady diffusion in two dimen-
; ;...;
@t @xi @xn sions:

is replaced with the Fourier variables @2 c @2 c

þ ¼0
@x2 @y2
ij0 ; ij1 ; . . . ; ijn

If the m-th order differential equation is This gives

X X j21  j22 ¼ ðj12 þ j22 Þ ¼ 0

P¼ aa @ a þ ba @ a
jaj¼m jaj<m

Thus,
where
s 21 þ s 22 ¼ 1 ðnormalizationÞ
X
n
a ¼ ða0 ; a1 ; . . . ; an Þ; jaj ¼ ai s 21 þ s 22 ¼ 0 ðequationÞ
i¼0

@ jaj
@a ¼
@ta0 @xa1 1 . . . @xan n These cannot both be satisfied so the problem is
elliptic. When the equation is
the characteristic equation for P is defined as
X @2 u @2 u
 ¼0
aa s a ¼ 0; s ¼ ðs 0 ; s 1 ; . . . ; s n Þ @t2 @x2
jaj¼m ð30Þ
s a ¼ s a0 0 s a1 1 . . . s an n
then
where s represents coordinates. Thus only the
highest derivatives are used to determine the j20 þ j21 ¼ 0
82 Mathematics in Chemical Engineering

Now real j0 can be solved and the equation is First-order quasi-linear problems are written
hyperbolic in the form

s 20 þ s 21 ¼ 1 ðnormalizationÞ Xn
@u
A1 ¼ f; x ¼ ðt; x1 . . . ; xn Þ
s 20 þ s 21 ¼ 0 ðequationÞ l¼0
@x1 ð31Þ
u ¼ ðu1 ; u2 ; . . . ; uk Þ
When the equation is
 2 
@c @ c @2c The matrix entries A1 is a kk matrix whose
¼D þ 2
@t @x 2 @y entries depend on u but not on derivatives of u.
Equation (31) is hyperbolic if
then
A ¼ Am
s 20 þ s 21 þ s 22 ¼ 1 ðnormalizationÞ

s 21 þ s 22 ¼ 0 ðequationÞ is nonsingular and for any choice of real l1,

l ¼ 0, . . . , n, l 6¼ m the roots ak of
thus we get
n !
Xl¼0
s 20 ¼ 1 ðfor normalizationÞ det aA  l1 A 1 ¼0
l 6¼ m

and the characteristic surfaces are hyperplanes

with t ¼ constant. This is a parabolic case. are real. If the roots are complex the equation is
Consider next the telegrapher’s equation: elliptic; if some roots are real and some are
complex the equation is of mixed type.
@T @2 T @2 T Apply these ideas to the advection equation
þb 2 ¼ 2
@t @t @x
@u @u
þ FðuÞ ¼0
Replacing the derivatives with the Fourier var- @t @x
iables gives
Thus,
ij0  bj20 þ j21 ¼ 0
detðaA0  l1 A1 Þ ¼ 0 or detðaA1  l0 A0 Þ ¼ 0
The equation is thus second order and the type
is determined by In this case,

bs 20 þ s 21 ¼ 0 n ¼ 1; A0 ¼ 1; A1 ¼ FðuÞ

Using the first of the above equations gives

The normalization condition
 
s 20 þ s 21 ¼ 1 det a  l1 FðuÞ ¼ 0; or a ¼ l1 FðuÞ

is required. Combining these gives Thus, the roots are real and the equation is
hyperbolic.
1  ð1 þ bÞs 20 ¼ 0 The final example is the heat conduction
problem written as
The roots are real and the equation is hyper-
bolic. When b ¼ 0 %C p
@T @q
¼  ; q ¼ k
@T
@t @x @x
j21 ¼ 0
In this formulation the constitutive equation for
and the equation is parabolic. heat flux is separated out; the resulting set of
 Mathematics in Chemical Engineering 83

equations is first order and written as on concentration. If diffusive phenomenon are

important, the equation is changed to
@T @q
%Cp þ ¼0
@t @x @c @FðcÞ @2 c
þ ¼D 2 ð32Þ
@T @t @x @x
k ¼ q
@x

where D is a diffusion coefficient. Special cases

In matrix notation this is are convective diffusive equation
2 3 2 3
@T 2 3 @c @c @2c
" # @T þu ¼D 2 ð33Þ
%C p 0 6 7
6 @t 7 0 1 6
6 @x 7
7 6
0
7
@t @x @x
6 7þ½ 6
6
7¼4
7 5
0 0 4 @q 5 k 0 4 @q 5 q
@t @t and Burgers viscosity equation
@u @u @2u
This compares with þu ¼n 2 ð34Þ
@t @x @x

@u @u where u is the velocity and n is the kinematic

A0 þ A1 ¼f
@x0 @x1
viscosity. This is a prototype equation for the
Navier–Stokes equations (! Fluid Mechanics).
In this case A0 is singular whereas A1 is non- For adsorption phenomena [106, p. 202],
singular. Thus,
@c @c df @c
f þ fu þ ð1  fÞ ¼0 ð35Þ
@t @x dc @t
detðaA1  l0 A0 Þ ¼ 0

where f is the void fraction and f (c) gives the

is considered for any real l0. This gives
equilibrium relation between the concentra-
  tions in the fluid and in the solid phase. In
 %C p l0 a 
 these examples, if the diffusion coefficient D or
 ¼0
 ka 0 the kinematic viscosity n is zero, the equations
are hyperbolic. If D and n are small, the phe-
or nomenon may be essentially hyperbolic even
though the equations are parabolic. Thus the
a2 k ¼ 0 numerical methods for hyperbolic equations
may be useful even for parabolic equations.
Thus the a is real, but zero, and the equation is Equations for several methods are given
parabolic. here, as taken from [107]. If the convective
term is treated with a centered difference
expression the solution exhibits oscillations
8.2. Hyperbolic Equations from node to node, and these vanish only if a
very fine grid is used. The simplest way to avoid
The most common situation yielding hyper- the oscillations with a hyperbolic equation is to
bolic equations involves unsteady phenomena use upstream derivatives. If the flow is from left
with convection. A prototype equation is to right, this would give the following for
Equation (33):
@c @FðcÞ
þ ¼0
@t @x dci Fðci Þ  Fðci1 Þ ciþ1  2ci þ ci1
þ ¼D
dt Dx Dx2

Depending on the interpretation of c and F (c), for Equation (34):

this can represent accumulation of mass and
convection. With F (c) ¼ u c, where u is the dui ui  ui1 uiþ1  2ui þ ui1
þ ui ¼n
velocity, the equation represents a mass balance dt Dx Dx2
84 Mathematics in Chemical Engineering

and for Equation (35): The concentration profile is steeper for the
dci ci  ci1 df dci MacCormack method than for the upstream
f þ fui þ ð1  fÞ ji ¼0 derivatives, but oscillations can still be present.
dt Dx dc dt
The flux-corrected transport method can be
If the flow were from right to left, then the added to the MacCormack method. A solution
formula would be is obtained both with the upstream algorithm
dci Fðciþ1 Þ  Fðci Þ ciþ1  2ci þ ci1
and the MacCormack method; then they are
þ ¼D combined to add just enough diffusion to elim-
dt Dx Dx2
inate the oscillations without smoothing the
If the flow could be in either direction, a local solution too much. The algorithm is compli-
determination must be made at each node i and cated and lengthy but well worth the effort
the appropriate formula used. The effect of [107–109].
using upstream derivatives is to add artificial If finite element methods are used, an
or numerical diffusion to the model. This can be explicit Taylor–Galerkin method is appropriate.
ascertained by taking the finite difference form For the convective diffusion equation the
of the convective diffusion equation method is
dci ci  ci1 ciþ1  2ci þ ci1 1 nþ1 2 1
þu ¼D ðc  cniþ1 Þ þ ðcnþ1  cni Þ þ ðcnþ1  cni1 Þ
dt Dx Dx2 6 iþ1 3 i 6 i1
 
uDt n DtD u2 Dt2
and rearranging ¼ ðciþ1  cni1 Þ þ þ ðcniþ1  2cni þ cni1 Þ
2Dx Dx2 2Dx2

dci ciþ1  ci1

dt
þu
2Dx Leaving out the u2Dt2 terms gives the
  Galerkin method. Replacing the left-hand
uDx ciþ1  2ci þ ci1
¼ Dþ side with
2 Dx2

Thus the diffusion coefficient has been changed cnþ1

i  cni
from
gives the Taylor finite difference method,
uDx and dropping the u2 Dt2 terms in that gives
D to D þ
2 the centered finite difference method. This
method might require a small time step if
Expressed in terms of the cell Peclet number, reaction phenomena are important. Then the
PeD ¼ uDx=D, this is D is changed to implicit Galerkin method (without the Taylor
D½1 þ PeD =2: terms) is appropriate
The cell Peclet number should always be A stability diagram for the explicit methods
calculated as a guide to the calculations. Using applied to the convective diffusion equation is
a large cell Peclet number and upstream deriv- shown in Figure 29. Notice that all the methods
atives leads to excessive (and artificial) smooth- require
ing of the solution profiles.
Another method often used for hyperbolic uDt
Co ¼ 1
equations is the MacCormack method. This Dx
method has two steps; it is written here for
Equation (33). where Co is the Courant number. How much
Co should be less than one depends on the
uDt n
method and on r ¼ D Dt/Dx2, as given in

ci nþ1 ¼ cni  ðc  cni Þ
Dx iþ1
DtD n
Figure 29. The MacCormack method with
þ ðc  2cni þ cni1 Þ flux correction requires a smaller time step
Dx2 iþ1
1  uDt  nþ1 
than the MacCormack method alone (curve a),
cnþ1 ¼ ðcni þ ci nþ1 Þ  ðc nþ1
 ci1 Þ
i
2 2Dx i and the implicit Galerkin method (curve e) is
DtD  nþ1   stable for all values of Co and r shown in
þ ðc  2ci nþ1 þ ci1
nþ1
Þ
2Dx2 iþ1 Figure 29 (as well as even larger values).
 Mathematics in Chemical Engineering 85

can be solved when the adsorption phenomenon

is governed by a Langmuir isotherm.

ac
f ðcÞ ¼
1 þ Kc

Similar numerical considerations apply and

similar methods are available [110–112].

8.3. Parabolic Equations in One

Dimension

In this section several methods are applied to

parabolic equations in one dimension: separa-
Figure 29. Stability diagram for convective diffusion equa- tion of variables, combination of variables,
tion (stable below curve) finite difference method, finite element method,
a) MacCormack; b) Centered finite difference; c) Taylor and the orthogonal collocation method. Sepa-
finite difference; d) Upstream; e) Galerkin; f) Taylor–
Galerkin
ration of variables is successful for linear prob-
lems, whereas the other methods work for
linear or nonlinear problems. The finite differ-
ence, the finite element, and the orthogonal
collocation methods are numerical, whereas
Each of these methods tries to avoid oscilla- the separation or combination of variables
tions that would disappear if the mesh were fine can lead to analytical solutions.
enough. For the steady convective diffusion
Analytical Solutions. Consider the diffusion
equation these oscillations do not occur pro-
equation
vided
@c @2 c
uDx PeD ¼D 2
¼ <¼ 1 ð36Þ @t @x
2D 2

with boundary and initial conditions

For large u, Dx must be small to meet
this condition. An alternative is to use a small cðx; 0Þ ¼ 0
Dx in regions where the solution changes
cð0; tÞ ¼ 1; cðL; tÞ ¼ 0
drastically. Because these regions change in
time, the elements or grid points must move.
The criteria to move the grid points can be A solution of the form
quite complicated, and typical methods are
reviewed in [107]. The criteria include moving cðx; tÞ ¼ TðtÞXðxÞ
the mesh in a known way (when the movement
is known a priori), moving the mesh to keep
some property (e.g., first- or second-derivative is attempted and substituted into the equation,
measures) uniform over the domain, using a with the terms separated to give
Galerkin or weighted residual criterion to
move the mesh, and Euler–Lagrange methods 1 dT 1 d2 X
¼
DT dt X dx2
which move part of the solution exactly by
convection and then add on some diffusion
after that. One side of this equation is a function of x
The final illustration is for adsorption in a alone, whereas the other side is a function of t
packed bed, or chromatography. Equation (35) alone. Thus, both sides must be a constant.
86 Mathematics in Chemical Engineering

Otherwise, if x is changed one side changes, The boundary conditions for u are
but the other cannot because it depends on t.
Call the constant  l and write the separate uð0; tÞ ¼ 0
equations uðL; tÞ ¼ 0

dT d2 X The initial conditions for u are found from the

 lDT;  lX
dt dx2
initial condition
x
The first equation is solved easily uðx; 0Þ ¼ cðx; 0Þ  f ðxÞ ¼ 1
L

TðtÞ ¼ Tð0ÞelDt
Separation of variables is now applied to this
equation by writing
and the second equation is written in the form
uðx; tÞ ¼ TðtÞXðxÞ
d2 X
þ lX ¼ 0
dx2
The same equation for T (t) and X (x) is
Next consider the boundary conditions. If obtained, but with X (0) ¼ X (L) ¼ 0.
they are written as
d2 X
þ lX ¼ 0
dx2
cðL; tÞ ¼ 1 ¼ TðtÞXðLÞ
Xð0Þ ¼ XðLÞ ¼ 0
cð0; tÞ ¼ 0 ¼ TðtÞXð0Þ

the boundary conditions are difficult to satisfy Next X (x) is solved for. The equation
because they are not homogeneous, i.e. with a is an eigenvalue problem. The general solution
zero right-hand side. Thus, the problem must is obtained by using emx pffiffiffi and finding that
be transformed to make the boundary condi- m2þ l ¼ 0; thus m ¼
i l. The exponential
tions homogeneous. The solution is written as term
the sum of two functions, one of which pffiffi
e
i lx
satisfies the nonhomogeneous boundary con-
ditions, whereas the other satisfies the homo-
geneous boundary conditions. is written in terms of sines and cosines, so that
the general solution is
cðx; tÞ ¼ f ðxÞ þ uðx; tÞ
pffiffiffi pffiffiffi
uð0; tÞ ¼ 0 X ¼ Bcos lx þ Esin lx
uðL; tÞ ¼ 0
The boundary conditions are
Thus, f (0) ¼ 1 and f (L) ¼ 0 are necessary. Now pffiffiffi pffiffiffi
the combined function satisfies the boundary XðLÞ ¼ Bcos lL þ Esin lL ¼ 0
conditions. In this case the function f (x) can be Xð0Þ ¼ B ¼ 0
taken as
If B ¼ 0, then E 6¼ 0 is required to have any
f ðxÞ ¼ L  x solution at all. Thus, l must satisfy
pffiffiffi
The equation for u is found by substituting for c sin lL ¼ 0
in the original equation and noting that the f (x)
drops out for this case; it need not disappear in This is true for certain values of l, called
the general case: eigenvalues or characteristic values. Here,
they are
@u @2u
¼D 2
@t @x ln ¼ n2 p2 =L2
 Mathematics in Chemical Engineering 87

Each eigenvalue has a corresponding eigen- number of terms are used, some error always
function occurs. For large times a single term is ade-
quate, whereas for small times many terms are
X n ðxÞ ¼ Esin n p x=L needed. For small times the Laplace transform
method is also useful, because it leads to
The composite solution is then solutions that converge with fewer terms. For
small times, the method of combination of
n p x ln Dt
X n ðxÞT n ðtÞ ¼ E A sin e variables may be used as well. For nonlinear
L
problems, the method of separation of variables
This function satisfies the boundary conditions fails and one of the other methods must be
and differential equation but not the initial used.
condition. To make the function satisfy the The method of combination of variables is
initial condition, several of these solutions useful, particularly when the problem is posed
are added up, each with a different eigenfunc- in a semi-infinite domain. Here, only one exam-
tion, and E A is replaced by An. ple is provided; more detail is given in [3, 113,
114]. The method is applied here to the non-
X
1
n p x n2 p2 Dt=L2
linear problem
uðx; tÞ ¼ An sin e
L
n¼1    
@c @ @c @ 2 c d DðcÞ @c 2
¼ DðcÞ ¼ DðcÞ 2 þ
@t @x @x @x dc @x
The constants An are chosen by making u (x, t)
satisfy the initial condition.
with boundary and initial conditions
X
1
npx x
uðx; 0Þ ¼ An sin ¼ 1 cðx; 0Þ ¼ 0
n¼1
L L
cð0; tÞ ¼ 1; cð1; tÞ ¼ 0

The residual R (x) is defined as the error in the

initial condition: The transformation combines two variables into
one
x X1
npx x
RðxÞ ¼ 1 An sin cðx; tÞ ¼ f ðhÞ where h ¼ pffiffiffiffiffiffiffiffiffiffi
L L 4D0 t
n¼1

Next, the Galerkin method is applied, and the The use of the 4 and D0 makes the analysis below
residual is made orthogonal to a complete set of simpler. The equation for c (x, t) is transformed
functions, which are the eigenfunctions. into an equation for f (h)
@c df @h @c df @h
ZL  ¼ ; ¼
x  mpx @t dh @t @x dh @x
 1 sin dx  
L L @ 2 c d2 f @h 2 df @ 2 h
0 ¼ þ
@x2 dh2 @x dh @x2
X1 Z L
mpx npx Am
¼ An sin sin dx ¼ @h x=2 @h 1 @2 h
L L 2 ¼  pffiffiffiffiffiffiffiffiffiffiffiffi ; ¼ pffiffiffiffiffiffiffiffiffiffi ; ¼0
n¼1
0 @t 4D0 t3 @x 4D0 t @x2

The Galerkin criterion for finding An is the The result is

same as the least-squares criterion [3, p. 183].
 
The solution is then d
KðcÞ
df df
þ 2h ¼ 0
dh dh dh
x X1
n p x n2 p2 Dt=L2 KðcÞ ¼ DðcÞ=D0
cðx; tÞ ¼ 1  þ An sin e
L n¼1 L
The boundary conditions must also combine. In
This is an “exact” solution to the linear prob- this case the variable h is infinite when either x is
lem. It can be evaluated to any desired accuracy infinite or t is zero. Note that the boundary
by taking more and more terms, but if a finite conditions on c (x, t) are both zero at those
88 Mathematics in Chemical Engineering

points. Thus, the boundary conditions can be where the matrix B is tridiagonal. The stability
combined to give of the integration of these equations is governed
by the largest eigenvalue of B. If Euler’s
f ð1Þ ¼ 0 method is used for integration,

D 2
The other boundary condition is for x ¼ 0 or Dt 
Dx2 jljmax
h ¼ 0,

f ð0Þ ¼ 1 The largest eigenvalue of B is bounded by the

Gerschgorin theorem [14, p. 135].
Thus, an ordinary differential equation must be X
n
solved rather than a partial differential equation. jljmax  max2<j<n jBji j ¼ 4
i¼2
When the diffusivity is constant the solution is
the well-known complementary error function:
This gives the well-known stability limit
cðx; tÞ ¼ 1  erf h ¼ erfc h
D 1
Zh Dt 
2 Dx2 2
ej dj
erf h ¼ 0Z1
2 If other methods are used to integrate in time,
ej dj
then the stability limit changes according to the
0
method. It is interesting to note that the eigen-
values of Equation (37) range from D p2/L2
This is a tabulated function [23]. (smallest) to 4 D/Dx2 (largest), depending on
Numerical Methods. Numerical methods are the boundary conditions. Thus the problem
applicable to both linear and nonlinear prob- becomes stiff as Dx approaches zero [3, p. 263].
lems on finite and semi-infinite domains. The Implicit methods can also be used. Write a
finite difference method is applied by using the finite difference form for the time derivative
method of lines [115]. In this method the same and average the right-hand sides, evaluated at
equations are used for the spatial variations of the old and new times:
the function, but the function at a grid point can
cnþ1  cni cn  2cni þ cni1
vary with time. Thus the linear diffusion prob- i
¼ Dð1  uÞ iþ1
Dt Dx2
lem is written as
cnþ1 nþ1
iþ1  2ci þ cnþ1
i1
þDu
Dx2
dci ciþ1  2ci þ ci1
¼D ð37Þ
dt Dx2
Now the equations are of the form
This can be written in the general form  
DDtu nþ1 DDtu nþ1 DDtu nþ1
 c þ 1 þ 2 c  c
Dx2 iþ1
Dx2 i Dx2 i1
dc
¼ AAc
dt DDtð1  uÞ n
¼ cni þ ðciþ1  2cni þ cni1 Þ
Dx2

This set of ordinary differential equations can

be solved by using any of the standard methods. and require solving a set of simultaneous equa-
The stability of explicit schemes is deduced tions, which have a tridiagonal structure. Using
from the theory presented in Chapter 6. The u ¼ 0 gives the Euler method (as above); u ¼ 0.5
equations are written as gives the Crank–Nicolson method; u ¼ 1 gives
the backward Euler method. The stability limit
is given by
dci ciþ1  2ci þ ci1 D X
nþ1
¼D ¼ 2 Bij cj DDt 0:5
dt Dx2 Dx j¼1 
Dx2 1  2u
 Mathematics in Chemical Engineering 89

whereas the oscillation limit is given by boundary value problems by letting the solution
DDt 0:25
at the nodes depend on time. For the diffusion
 equation the finite element method gives
Dx2 1  u

If a time step is chosen between the oscillation XX dceI X X e e

C eJI ¼ BJI cI
limit and stability limit, the solution will oscil- e I
dt e I
late around the exact solution, but the oscilla-
tions remain bounded. For further discussion, with the mass matrix defined by
see [3, p. 218].
Finite volume methods are utilized exten- Z1
sively in computational fluid dynamics (see C eJI ¼ Dxe N J ðuÞN I ðuÞdu
also, ! Computational Fluid Dynamics). In 0

this method, a mass balance is made over a

cell accounting for the change in what is in the This set of equations can be written in matrix
cell and the flow in and out. Figure 33 illustrates form
the geometry of the i-th cell. A mass balance
made on this cell (with area A perpendicular to dc
CC ¼ AAc
dt
the paper) is

ADxðcnþ1
i  cni Þ ¼ DtAðJ j1=2  J iþ1=2 Þ Now the matrix C C is not diagonal, so that a set
of equations must be solved for each time step,
where J is the flux due to convection and even when the right-hand side is evaluated
diffusion, positive in the þx direction. explicitly. This is not as time-consuming as it
seems, however. The explicit scheme is written
@c ci  ci1=2
J ¼ uc  D ; J i1=2 ¼ ui1=2 ci1=2  D as
@x Dx

cnþ1  cni
The concentration at the edge of the cell is CC ji i
¼ AAji cni
Dt
taken as
1 and rearranged to give
ci1=2 ¼ ðci þ ci1 Þ
2
CC ji ðcnþ1
i  cni Þ ¼ DtAAji cni or
Rearrangement for the case when the veloc-
CCðcnþ1  cn Þ ¼ DtAAc
ity u is the same for all nodes gives
cnþ1
i  cni uðciþ1  ci1 Þ D This is solved with an L U decomposition (see
þ ¼ 2 ðciþ1  2ci þ ci1 Þ
Dt 2Dx Dx Section 1.2) that retains the structure of the mass
matrix C C. Thus,
This is the same equation as obtained using
the finite difference method. This is not always CC ¼ LU
true, and the finite volume equations are easy to
derive. In two- and three-dimensions, the mesh At each step, calculate
need not be rectangular, as long as it is possible
to compute the velocity normal to an edge of the
cnþ1  cn ¼ DtU1 L1 AAcn
cell. The finite volume method is useful for
applications involving filling, such as injection
molding, when only part of the cell is filled with This is quick and easy to do because the inverse
fluid. Such applications do involve some of L and U are simple. Thus the problem is
approximations, since the interface is not reduced to solving one full matrix problem and
tracked precisely, but they are useful engineer- then evaluating the solution for multiple right-
ing approximations. hand sides. For implicit methods the same
The finite element method is handled in a approach can be used, and the LU decomposi-
similar fashion, as an extension of two-point tion remains fixed until the time step is changed.
90 Mathematics in Chemical Engineering

The method of orthogonal collocation uses and evaluated at each collocation point
a similar extension: the same polynomial of x
n
is used but now the coefficients depend on unþ1  unj @u 
þ f ðunj Þ  ¼ 0
j
time. Dt @x j

 The trial function is taken as

@c  dcðxj ; tÞ dcj
¼ ¼
@t xj dt dt
X
N
ppj
uj ðtÞ ¼ ap ðtÞcos ; unj ¼ uj ðtn Þ ð39Þ
p¼0
N
Thus, for diffusion problems
Assume that the values unj exist at some time.
dcj X
N þ2
Then invert Equation (39) using the fast Fourier
¼ Bji ci ; j ¼ 2; . . . ; N þ 1
dt i¼1 transform to obtain {ap} for p ¼ 0, 1, . . . , N;
then calculate Sp
This can be integrated by using the standard Sp ¼ Spþ2 þ ðp þ 1Þapþ1 ; 0  p  N  1
methods for ordinary differential equations as
SN ¼ 0; SNþ1 ¼ 0
initial value problems. Stability limits for
explicit methods are available [3, p. 204].
The method of orthogonal collocation on and finally
finite elements can also be used, and details
2Sp
are provided elsewhere [3, pp. 228–230]. að1Þ
p ¼
cp
The maximum eigenvalue for all the meth-
ods is given by Thus, the first derivative is given by
LB
jljmax ¼ 2 ð38Þ
Dx @u X
N
ppj
j ¼ að1Þ ðtÞcos
@x j p¼0 p N
where the values of LB are as follows:
This is evaluated at the set of collocation points
Finite difference 4 by using the fast Fourier transform again. Once
Galerkin, linear elements, lumped 4 the function and the derivative are known at
Galerkin, linear elements 12 each collocation point the solution can be
Galerkin, quadratic elements 60
advanced forward to the n þ 1-th time level.
Orthogonal collocation on finite elements, cubic 36
The advantage of the spectral method is that
it is very fast and can be adapted quite well to
Spectral methods employ the discrete Fou- parallel computers. It is, however, restricted in
rier transform (see Section 2.6 and Chebyshev the geometries that can be handled.
polynomials on rectangular domains [116]).
In the Chebyshev collocation method, N þ 1
collocation points are used 8.4. Elliptic Equations

pj Elliptic equations can be solved with both finite

xj ¼ cos ; j ¼ 0; 1; . . . ; N
N difference and finite element methods. One-
dimensional elliptic problems are two-point
As an example, consider the equation boundary value problems and are covered in
@u @u Chapter 7. Two-dimensional elliptic problems
þ f ðuÞ ¼0 are often solved with direct methods, and iter-
@t @x
ative methods are usually used for three-dimen-
An explicit method in time can be used sional problems. Thus, two aspects must be
considered: how the equations are discretized
n
unþ1  un @u to form sets of algebraic equations and how the
þ f ðun Þ  ¼ 0
Dt @x algebraic equations are then solved.
 Mathematics in Chemical Engineering 91

The prototype elliptic problem is steady- And this is converted to the Gauss–Seidel iter-
state heat conduction or diffusion, ative method.
 2   
@ T @2T Dx2
k þ 2 ¼Q 2 1 þ 2 T sþ1 s sþ1
i;j ¼ T iþ1;j þ T i1;j
@x 2 @y Dy

Dx2 s 2 Qi;j
þ ðT i;j1 Þ  Dx
þ T sþ1
Dy2 i;jþ1 k
possibly with a heat generation term per unit
volume, Q. The boundary conditions can be Calculations proceed by setting a low i, com-
puting from low to high j, then increasing i and
Dirichlet or 1st kind: T ¼ T1 on boundary S1 repeating the procedure. The relaxation method
uses
Neumann or 2nd kind: k @T
@n ¼ q2 on boundary S2
 
k @T Dx2  Dx2
Robin, mixed, or 3rd kind: @n ¼ hðT  T 3 Þ on boundary S3 2 1 þ 2 T i;j ¼ T siþ1;j þ T sþ1
i1;j þ
Dy Dy2
Qi;j
i;j1 Þ  Dx
ðT si;jþ1  T sþ1 2
Illustrations are given for constant physical k
properties k, h, while T1, q2, T3 are known
functions on the boundary and Q is a known
function of position. For clarity, only a two- T sþ1 s 
s
i;j ¼ T i;j þ bðT i;j  T i;j Þ
dimensional problem is illustrated. The finite
difference formulation is given by using the If b ¼ 1, this is the Gauss–Seidel method. If
following nomenclature b > 1, it is overrelaxation; if b < 1, it is
underrelaxation. The value of b may be chosen
T i;j ¼ TðiDx; jDyÞ empirically, 0 < b < 2, but it can be selected
theoretically for simple problems like this [117,
The finite difference formulation is then p. 100], [3, p. 282]. In particular, the optimal
value of the iteration parameter is given by
T iþ1;j  2T i;j þ T i1;j T i;jþ1  2T i;j þ T i;j1
þ ¼ Qi;j =k ð40Þ
Dx2 Dy2 lnðbopt  1Þ  R

and the error (in solving the algebraic equation)

T i;j ¼ T 1 for i; j on boundary S1 is decreased by the factor (1  R)N for every N

@T  iterations. For the heat conduction problem and
k ¼ q2 for i; j on boundary S2
@n i;j Dirichlet boundary conditions,

@T 
k ¼ hðT i;j  T 3 Þfor i; j on boundary S3
@n i;j R¼
p2
2n2

If the boundary is parallel to a coordinate axis (when there are n points in both x and y direc-
the boundary slope is evaluated as in Chapter 7, tions). For Neumann boundary conditions, the
by using either a one-sided, centered difference value is
or a false boundary. If the boundary is more
irregular and not parallel to a coordinate line,
p2 1
more complicated expressions are needed and R¼
2n2 1 þ max½Dx2 =Dy2 ; Dy2 =Dx2 
the finite element method may be the better
method.
Equation (40) is rewritten in the form Iterative methods can also be based on lines
(for 2D problems) or planes (for 3D problems).
 
Dx2 Dx2 Preconditioned conjugate gradient methods
2 1 þ 2 T i;j ¼ T iþ1;j þ T i1;j þ 2
Dy Dy have been developed (see Chap. 1). In these
Qi;j methods a series of matrix multiplications are
ðT i;jþ1 þ T i;j1 Þ  Dx2
k done iteration by iteration; and the steps lend
92 Mathematics in Chemical Engineering

themselves to the efficiency available in paral- In these equations I and J refer to the nodes
lel computers. In the multigrid method the of the triangle forming element e and the
problem is solved on several grids, each summation is made over all elements. These
more refined than the previous one. In iterating equations represent a large set of linear equa-
between the solutions on different grids, one tions, which are solved using matrix techniques
converges to the solution of the algebraic equa- (Chap. 1).
tions. A chemical engineering application is If the problem is nonlinear, e.g., with k or Q a
given in [118]. Software for a variety of these function of temperature, the equations must be
methods is available, as described below. solved iteratively. The integrals are given for a
The Galerkin finite element method triangle with nodes I, J, and K in counter-
(FEM) is useful for solving elliptic problems clockwise order. Within an element,
and is particularly effective when the domain
or geometry is irregular [119–125]. As an T ¼ N I ðx; yÞT I þ N J ðx; yÞT J þ N K ðx; yÞT K
example, cover the domain with triangles
aI þ bI x þ cI y
and define a trial function on each triangle. NI ¼
2D
The trial function takes the value 1.0 at one aI ¼ xJ yK  xK yJ
corner and 0.0 at the other corners, and is
bI ¼ yI  yK
linear in between (see Fig. 30). These trial
functions on each triangle are pieced together cI ¼ xK  x J
to give a trial function on the whole domain. plus permutation on I; K; J
For the heat conduction problem the method 2
1 xI yI
3
gives [3] 6 7
2D ¼ det6 7
4 1 xJ yJ 5 ¼ 2ðarea of triangleÞ
XX XX
AeIJ T eJ ¼ F eI ð41Þ 1 xK yK
e J e J
aI þ aJ þ aK ¼ 1

where bI þ bJ þ bK ¼ 0
Z Z cI þ c J þ cK ¼ 0
AeIJ ¼  krN I  rN J dA  h3 N I N J dC
k
C3 AeIJ ¼  ðbI bJ þ cI cJ Þ
4D
Z Z Z
F eI ¼ N I QdA þ N I q2 dC  Q QD
N I h3 T 3 dC F eIJ ¼ ðaI þ bI x þ cI y Þ ¼
2 3
C2 C3
xI þ xJ þ x K y þ y J þ yK
x¼ ; y ¼ I
3 3
Also, a necessary condition is that
2
T i ¼ T 1 on C 1 aI þ bI x þ cI y ¼ D
3

The trial functions in the finite element

method are not limited to linear ones. Quadratic
functions, and even higher order functions, are
frequently used. The same considerations hold
as for boundary value problems: the higher
order trial functions converge faster but require
more work. It is possible to refine both the mesh
(h) and power of polynomial in the trial func-
tion (p) in an hp method. Some problems have
constraints on some of the variables. If singu-
larities exist in the solution, it is possible to
include them in the basis functions and solve
for the difference between the total solution and
the singular function [126–129].
Figure 30. Finite elements trial function: linear polynomi- When applying the Galerkin finite element
als on triangles method, one must choose both the shape of the
 Mathematics in Chemical Engineering 93

element and the basis functions. In two dimen- In the finite difference method an explicit
sions, triangular elements are usually used technique would evaluate the right-hand side at
because it is easy to cover complicated geome- the n-th time level:
tries and refine parts of the mesh. However,
rectangular elements sometimes have advan- T nþ1 n
i;j  T i;j
rC p
tages, particularly when some parts of the Dt
solution do not change very much and the ¼
k
ðT n  2T ni1;j þ T ni1;j Þ
elements can be long. In three dimensions Dx2 iþ1;j
the same considerations apply: tetrahedral ele- k
þ ðT n  2T ni;j þ T ni;j1 Þ  Q
Dy2 i;jþ1
ments are frequently used, but brick elements
are also possible. While linear elements (in both
two and three dimensions) are usually used, When Q ¼ 0 and Dx ¼ Dy, the time step is
higher accuracy can be obtained by using qua- limited by
dratic or cubic basis functions within the ele-
Dx2 %C p Dx2
ment. The reason is that all methods converge Dt  or
4k 4D
according to the mesh size to some power,
and the power is larger when higher order These time steps are smaller than for one-
elements are used. If the solution is dis- dimensional problems. For three dimensions,
continuous, or has discontinuous first deriva- the limit is
tives, then the lowest order basis functions are
used because the convergence is limited by the Dx2
Dt 
properties of the solution, not the finite element 6D
approximation.
One nice feature of the finite element To avoid such small time steps, which must
method is the use of natural boundary condi- be smaller when Dx decreases, an implicit
tions. In this problem the natural boundary method could be used. This leads to large
conditions are the Neumann or Robin condi- sparse matrices, rather than convenient tri-
tions. When using Equation (41), the problem diagonal matrices.
can be solved on a domain that is shorter than
needed to reach some limiting condition (such
as at an outflow boundary). The externally
8.6. Special Methods for Fluid
applied flux is still applied at the shorter Mechanics
domain, and the solution inside the truncated
domain is still valid. Examples are given in The method of operator splitting is also
[107] and [131]. The effect of this is to allow useful when different terms in the equation
solutions in domains that are smaller, thus are best evaluated by using different methods
saving computation time and permitting the or as a technique for reducing a larger problem
solution in semi-infinite domains. to a series of smaller problems. Here the
method is illustrated by using the Navier–
Stokes equations. In vector notation the equa-
8.5. Parabolic Equations in Two or tions are
Three Dimensions
@u
% þ %u  ru ¼ %f  rp þ mr2 u
Computations become much more lengthy with @t

two or more spatial dimensions, for example,

The equation is solved in the following steps
the unsteady heat conduction equation
 2 
@T @ T @2T 
u  un
%C p ¼k þ Q
@t @x2 @y2 % ¼ %un  run þ %f þ mr2 un
Dt
1 
or the unsteady diffusion equation r2 pnþ1 ¼ ru
Dt
 2 
@c @ T @2c unþ1  u

¼D þ  RðcÞ % ¼ rp
@t @x2 @y2 Dt
94 Mathematics in Chemical Engineering

This can be done by using the finite difference here follows [142]. A lattice is defined, and one
[132, p. 162] or the finite element method [133– solves the following equation for f i ðx; tÞ, the
135]. probability of finding a molecule at the point x
In fluid flow problems solved with the with speed ci.
finite element method, the basis functions
for pressure and velocity are often different. @f i f  f eq
þ ci  rf i ¼  i i
This is required by the LBB condition (named @t t
after LADYSHENSKAYA, BREZZI, and BABUSKA)
[134, 135]. Sometimes a discontinuous basis The right-hand side represents a single time
function is used for pressure to meet this relaxation for molecular collisions, and t is
condition. Other times a penalty term is added, related to the kinematic viscosity. By means
or the quadrature is done using a small number of a simple stepping algorithm, the computa-
of quadrature points. Thus, one has to be tional algorithm is
careful how to apply the finite element method
to the Navier–Stokes equations. Fortunately, Dt
f i ðx þ ci Dt; t þ DtÞ ¼ f i ðx; tÞ  ðf  f eq
i Þ
software exists that has taken this into t i
account.
Consider the lattice shown in Figure 34. The
Level Set Methods. Multiphase problems are various velocities are
complicated because the terms in the
equations depend on which phase exists at a c1 ¼ ð1; 0Þ; c3 ¼ ð0; 1Þ; c5 ¼ ð1; 0Þ; c7 ¼ ð0; 1Þ
particular point, and the phase boundary may c2 ¼ ð1; 1Þ; c4 ¼ ð1; 1Þ; c6 ¼ ð1; 1Þ; c8 ¼ ð1; 1Þ
move or be unknown. It is desirable to compute
on a fixed grid, and the level set formulation c0 ¼ ð0; 0Þ

allows this. Consider a curved line in a two-

dimensional problem or a curved surface in a The density, velocity, and shear stress (for
three-dimensional problem. One defines a level some k) are given by
set function f, which is the signed distance
X X X
function giving the distance from some point to r¼ f i ðx; tÞ; ru ¼ ci f i ðx; tÞ; t ¼ k ci ci ½f i ðx; tÞ f eq
i ðx; tÞ
the closest point on the line or surface. It i i i

defined to be negative on one side of the

interface and positive on the other. Then the For this formulation, the kinematic viscosity
curve and speed of sound are given by
 
fðx; y; zÞ ¼ 0 1 1 1
n¼ t  ; cs ¼
3 2 3
represents the location of the interface. For
example, in flow problems the level set func- The equilibrium distribution is
tion is defined as the solution to " #
ci  u ðci  uÞ2 u  u
f eq
i ¼ rwi 1 þ þ  2
@f c2s 2c4s 2cs
þ u  rf ¼ 0
@t

The physics governing the velocity of the where the weighting functions are
interface must be defined, and this equation is 4 1 1
solved along with the other equations repre- w0 ¼ ; w1 ¼ w3 ¼ w5 ¼ w7 ¼ ; w2 ¼ w4 ¼ w6 ¼ w8 ¼
9 9 36
senting the problem [130–137].
With these conditions, the solution for velocity
Lattice Boltzmann Methods. Another way to is a solution of the Navier—Stokes equation.
solve fluid flow problems is based on a These equations lead to a large computational
molecular viewpoint and is called the Lattice problem, but it can be solved by parallel proc-
Boltzmann method [138–141]. The treatment essing on multiple computers.
 Mathematics in Chemical Engineering 95

8.7. Computer Software the computation. This phenomenon introduces

the same complexity that occurs in problems
A variety of general-purpose computer pro- with a large Peclet number, with the added
grams are available commercially. Mathe- difficulty that the free surface moves between
matica (http://www.wolfram.com/), Maple mesh points, and improper representation can
(http://www.maplesoft.com/) and Mathcad lead to unphysical oscillations. The method
(http://www.mathcad.com/) all have the used to solve the equations is important, and
capability of doing symbolic manipulation so both explicit and implicit methods (as described
that algebraic solutions can be obtained. For above) can be used. Implicit methods may
example, Mathematica can solve some ordinary introduce unacceptable extra diffusion, so the
and partial differential equations analytically; engineer needs to examine the solution care-
Maple can make simple graphs and do linear fully. The methods used to smooth unphysical
algebra and simple computations, and Mathcad oscillations from node to node are also impor-
can do simple calculations. In this section, tant, and the engineer needs to verify that the
examples are given for the use of Matlab added diffusion or smoothing does not give
(http://www.mathworks.com/), which is a pack- inaccurate solutions. Since current-day prob-
age of numerical analysis tools, some of which are lems are mostly nonlinear, convergence is
accessed by simple commands, and some of always an issue since the problems are solved
which are accessed by writing programs in C. iteratively. Robust programs provide several
Spreadsheets can also be used to solve simple methods for convergence, each of which is
problems. A popular program used in chemical best in some circumstance or other. It is
engineering education is Polymath (http://www wise to have a program that includes many
.polymath-software.com/), which can numeri- iterative methods. If the iterative solver is not
cally solve sets of linear or nonlinear equations, very robust, the only recourse to solving a
ordinary differential equations as initial value steady-state problem may be to integrate the
problems, and perform data analysis and time-dependent problem to steady state. The
regression. solution time may be long, and the final
The mathematical methods used to solve result may be further from convergence than
partial differential equations are described in would be the case if a robust iterative solver
more detail in [143–148]. Since many computer were used.
programs are available without cost, consider A variety of computer programs is available
the following decision points. The first decision on the internet, some of them free. First
is whether to use an approximate, engineering consider general-purpose programs. On the
flow model, developed from correlations, or to NIST web page, http://gams.nist.gov/ choose
solve the partial differential equations that gov- “problem decision tree”, and then “differential
ern the problem. Correlations are quick and and integral equations”, then “partial differen-
easy to apply, but they may not be appropriate tial equations”. The programs are organized
to your problem, or give the needed detail. by type of problem (elliptic, parabolic, and
When using a computer package to solve partial hyperbolic) and by the number of spatial
differential equations, the first task is always to dimensions (one or more than one). On the
generate a mesh covering the problem domain. Netlib web site, http://www.netlib.org/ , search
This is not a trivial task, and special methods on “partial differential equation”. The website:
have been developed to permit importation of a http://software.sandia.gov has a variety of pro-
geometry from a computer-aided design pro- grams available. LAU [141, 145] provides many
gram. Then, the mesh must be created automat- programs in Cþþ (also see http://www.nr.com/).
ically. If the boundary is irregular, the finite The multiphysics program Comsol Multi- physics
element method is especially well-suited, (formerly FEMLAB) also solves many standard
although special embedding techniques can equations arising in Mathematical Physics.
be used in finite difference methods (which Computational fluid dynamics (CFD) pro-
are designed to be solved on rectangular grams are more specialized, and most of them
meshes). Another capability to consider is the have been designed to solve sets of equations
ability to track free surfaces that move during that are appropriate to specific industries. They can
96 Mathematics in Chemical Engineering

then include approximations and correlations for evolutionary problems, whereas Fredholm
some features that would be difficult to solve for equations are analogous to boundary value
directly. Four widely used major packages are problems. The terms in the integral can be
Fluent (http://www.fluent.com/), CFX (now unbounded, but still yield bounded integrals,
part of ANSYS), Comsol Multiphysics and these equations are said to be weakly
(formerly FEMLAB) (http://www.comsol.com/), singular. A Volterra equation of the second
and ANSYS (http://www.ansys.com/). Of these, kind is
Comsol Multiphysics is particularly useful
because it has a convenient graphical user Zt
interface, permits easy mesh generation and re- yðtÞ ¼ gðtÞ þ l Kðt; sÞyðsÞds ð42Þ
a
finement (including adaptive mesh refinement),
allows the user to add in phenomena and
additional equations easily, permits solution whereas a Volterra equation of the first kind is
by continuation methods (thus enhancing
convergence), and has extensive graphical Zt
output capabilities. Other packages are also yðtÞ ¼ l Kðt; sÞyðsÞds
available (see http://cfd-online.com/), and a

these may contain features and correlations

specific to the engineer’s industry. One Equations of the first kind are very sensitive to
important point to note is that for turbulent solution errors so that they present severe
flow, all the programs contain approximations, numerical problems.
using the k-epsilon models of turbulence, or An example of a problem giving rise to a
large eddy simulations; the direct numerical Volterra equation of the second kind is the
simulation of turbulence is too slow to apply it following heat conduction problem:
to very big problems, although it does give
insight (independent of any approximations) @T @2 T
%Cp ¼ k 2 ; 0  x < 1; t > 0
that is useful for interpreting turbulent pheno- @t @x
mena. Thus, the method used to include those @T
Tðx; 0Þ ¼ 0; ð0; tÞ ¼ gðtÞ;
turbulent correlations is important, and @x
the method also may affect convergence or lim Tðx; tÞ ¼ 0; lim
@T
¼0
accuracy. x!1 x!1 @x

If this is solved by using Fourier transforms the

9. Integral Equations [149–155] solution is

If the dependent variable appears under an Zt

1 1
gðsÞ pffiffiffiffiffiffiffiffiffiffi ex =4ðtsÞ ds
2

integral sign an equation is called an integral Tðx; tÞ ¼ pffiffiffi

p ts
0
equation; if derivatives of the dependent varia-
ble appear elsewhere in the equation it is called
an integrodifferential equation. This chapter Suppose the problem is generalized so that the
describes the various classes of equations, gives boundary condition is one involving the solu-
information concerning Green’s functions, and tion T, which might occur with a radiation
presents numerical methods for solving integral boundary condition or heat-transfer coefficient.
equations. Then the boundary condition is written as

@T
¼ GðT; tÞ; x ¼ 0; t > 0
@x
9.1. Classification
The solution to this problem is
Volterra integral equations have an integral with
a variable limit, whereas Fredholm integral Zt   1
1
equations have a fixed limit. Volterra equations G Tð0; sÞ; s pffiffiffiffiffiffiffiffiffiffi ex =4ðtsÞ ds
2
Tðx; tÞ ¼ pffiffiffi
p ts
are usually associated with initial value or 0
 Mathematics in Chemical Engineering 97

If T (t) is used to represent T (0, t), then problems. An eigenvalue problem is a homoge-
neous equation of the second kind.
Zt   1
 1
T ðtÞ ¼ pffiffiffi G TðsÞ; s pffiffiffiffiffiffiffiffiffiffi ds Zb
p ts
0 yðxÞ ¼ l Kðx; sÞyðsÞds ð44Þ
a
Thus the behavior of the solution at the bound-
ary is governed by an integral equation. NAGEL Solutions to this problem occur only for spe-
and KLUGE [156] use a similar approach to solve cific values of l, the eigenvalues. Usually the
for adsorption in a porous catalyst. Fredholm equation of the second or first kind is
The existence and uniqueness of the solution solved for values of l different from these,
can be proved [151, p. 30, 32]. which are called regular values.
Sometimes the kernel is of the form Nonlinear Volterra equations arise naturally
from initial value problems. For the initial value
Kðt; sÞ ¼ Kðt  sÞ problem
dy  
Equations of this form are called convolution ¼ F t; yðtÞ
dt
equations and can be solved by taking the
Laplace transform. For the integral equation
both sides can be integrated from 0 to t to obtain
Zt
YðtÞ ¼ GðtÞ þ l Kðt  tÞYðtÞdt Zt  
0 yðtÞ ¼ yð0Þ þ F s; yðsÞ ds
Zt 0

KðtÞ YðtÞ Kðt  tÞYðtÞdt

0 which is a nonlinear Volterra equation. The
general nonlinear Volterra equation is
the Laplace transform is
Zt  
yðsÞ ¼ gðsÞ þ kðsÞyðsÞ yðtÞ ¼ gðtÞ þ K t; s; yðsÞ ds ð45Þ
 0
kðsÞyðsÞ ¼ L½KðtÞ YðtÞ

Solving this for y(s) gives Theorem [151, p. 55]. If g (t) is continuous, the
kernel K (t, s, y) is continuous in all variables
yðsÞ ¼
gðsÞ and satisfies a Lipschitz condition
1  kðsÞ

jKðt; s; yÞ  Kðt; s; zÞj  Ljy  zj

If the inverse transform can be found, the
integral equation is solved.
A Fredholm equation of the second kind is then the nonlinear Volterra equation has a
unique continuous solution.
Zb A successive substitution method for its
yðxÞ ¼ gðxÞ þ l Kðx; sÞyðsÞds ð43Þ solution is
a

Zt
whereas a Fredholm equation of the first kind is ynþ1 ðtÞ ¼ gðtÞ þ K½t; s; yn ðsÞds
0

Zb
Kðx; sÞyðsÞds ¼ gðxÞ
Nonlinear Fredholm equations have special
a
names. The equation

Z1
The limits of integration are fixed, and these f ðxÞ ¼ K½x; y; f ðyÞdy
problems are analogous to boundary value 0
98 Mathematics in Chemical Engineering

is called the Urysohn equation [150, p. 208]. solve for y1, y2, . . . in succession. For a single
The special equation integral equation, at each step one must solve a
Z1
single nonlinear algebraic equation for yn. Typ-
f ðxÞ ¼ K½x; yF½y; f ðyÞdy ically, the error in the solution to the integral
0 equation is proportional to Dtm, and the power
m is the same as the power in the quadrature
is called the Hammerstein equation [150, p. error [151, p. 97].
209]. Iterative methods can be used to solve The stability of the method [151, p. 111] can
these equations, and these methods are closely be examined by considering the equation
tied to fixed point problems. A fixed point
problem is Zt
yðtÞ ¼ 1  l yðsÞds
x ¼ FðxÞ 0

and a successive substitution method is whose solution is

xnþ1 ¼ Fðxn Þ yðtÞ ¼ elt

Local convergence theorems prove the process Since the integral equation can be differentiated
convergent if the solution is close enough to the to obtain the initial value problem
answer, whereas global convergence theorems
are valid for any initial guess [150, p. 229–231]. dy
¼ ly; yð0Þ ¼ 1
The successive substitution method for non- dt
linear Fredholm equations is
the stability results are identical to those for
Z1 initial value methods. In particular, using the
ynþ1 ðxÞ ¼ K½x; s; yn ðsÞds trapezoid rule for integral equations is identical
0 to using this rule for initial value problems. The
method is A-stable.
Typical conditions for convergence include that Higher order integration methods can also
the function satisfies a Lipschitz condition. be used [151, p. 114, 124]. When the kernel is
infinite at certain points, i.e., when the problem
has a weak singularity, see [151, p. 71, 151].
9.2. Numerical Methods for Volterra
Equations of the Second Kind

Volterra equations of the second kind are ana- 9.3. Numerical Methods for Fredholm,
logous to initial value problems. An initial Urysohn, and Hammerstein Equations
value problem can be written as a Volterra of the Second Kind
equation of the second kind, although not all
Volterra equations can be written as initial value Whereas Volterra equations could be solved
problems [151, p. 7]. Here the general nonlinear from one position to the next, like initial value
Volterra equation of the second kind is treated differential equations, Fredholm equations
(Eq. 46). The simplest numerical method must be solved over the entire domain, like
involves replacing the integral by a quadrature boundary value differential equations. Thus,
using the trapezoid rule. large sets of equations will be solved and the
notation is designed to emphasize that.
yn yðtn Þ ¼ gðtn Þ
( ) The methods are also based on quadrature
1 X
n1
1
þ Dt Kðtn ; t0 ; y0 Þ þ Kðtn ; ti ; yi Þ þ Kðtn ; tn ; yn Þ formulas. For the integral
2 i¼1
2

Zb
This equation is a nonlinear algebraic equation IðwÞ ¼ fðyÞdy
for yn. Since y0 is known it can be applied to a
 Mathematics in Chemical Engineering 99

a quadrature formula is written: Substitute f into the equation to form the

residual
X
n
IðfÞ ¼ wi fðyi Þ
i¼0 X
n X
n Zb
ai fi ðxÞ  l ai Kðx; yÞfi ðyÞdy ¼ gðxÞ
i¼0 i¼0
a

Then the integral Fredholm equation can be

rewritten as
Evaluate the residual at the collocation
X
n points
f ðxÞ  l wi Kðx; yi Þf ðyi Þ ¼ gðxÞ;
i¼0 ð46Þ
X
n X
n Zb
axb ai fi ðxj Þ  l ai Kðxj ; yÞfi ðyÞdy ¼ gðxj Þ
i¼0 i¼0
a

If this equation is evaluated at the points x ¼ yj,

The expansion can be in piecewise polynomi-
X
n
als, leading to a collocation finite element
f ðyj Þ  l wi Kðyj ; yi Þf ðyi Þ ¼ gðyi Þ
i¼0 method, or global polynomials, leading to a
global approximation. If orthogonal polynomi-
als are used then the quadratures can make use
is obtained, which is a set of linear equations to of the accurate Gaussian quadrature points to
be solved for {f (yj)}. The solution at any point calculate the integrals. Galerkin methods are
is then given by Equation (46). also possible [149, p. 406]. MILLS et al. [157]
A common type of integral equation has a consider reaction–diffusion problems and say
singular kernel along x ¼ y. This can be trans- the choice of technique cannot be made in
formed to a less severe singularity by writing general because it is highly dependent on the
kernel.
Zb Zb
Kðx; yÞf ðyÞdy ¼ Kðx; yÞ½f ðyÞ  f ðxÞdy When the integral equation is nonlinear, iter-
a a ative methods must be used to solve it. Conver-
Zb Zb
gence proofs are available, based on Banach’s
þ Kðx; yÞf ðxÞdy ¼ Kðx; yÞ½f ðyÞ  f ðxÞdy þ f ðxÞHðxÞ contractive mapping principle. Consider the
a a Urysohn equation, with g (x) ¼ 0 without loss
of generality:
where Zb
f ðxÞ ¼ F½x; y; f ðyÞdy
Zb a
HðxÞ ¼ Kðx; yÞf ðxÞdy
a
The kernel satisfies the Lipschitz condition

is a known function. The integral equation is maxax;yb jF½x; y; f ðyÞ  F½x; z; f ðzÞj  Kjy  zj

then replaced by
Theorem [150, p. 214]. If the constant K is < 1
X
n
f ðxÞ ¼ gðxÞ þ wi Kðx; yi Þ½f ðyi Þ  f ðxÞ þ f ðxÞHðxÞ and certain other conditions hold, the succes-
i¼0 sive substitution method

Collocation methods can be applied as well Zb

f nþ1 ðxÞ ¼ F½x; y; f n ðyÞdy; n ¼ 0; 1 . . .
[149, p. 396]. To solve integral Equation (43)
a
expand f in the function

X
n
converges to the solution of the integral
f ¼ ai fi ðxÞ
i¼0 equations.
100 Mathematics in Chemical Engineering

9.4. Numerical Methods for Eigenvalue where Green’s function is [50, p. 891]
Problems
1
Gðr; r0 Þ ¼ in three dimensions
r
Eigenvalue problems are treated similarly to
¼ 2 ln r in two dimensions
Fredholm equations, except that the final equa- qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
tion is a matrix eigenvalue problem instead of a where r ¼ ðx  x0 Þ2 þ ðy  y0 Þ2 þ ðz  z0 Þ2
set of simultaneous equations. For example, in three dimensions
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
X
n and r ¼ ðx  x0 Þ2 þ ðy  x0 Þ2
wi Kðyi ; yi Þf ðyi Þ ¼ lf ðyj Þ;
i¼1 in two dimensions
i ¼ 0; 1; . . . ; n
For the problem
leads to the matrix eigenvalue problem @u
¼ Dr2 u; u ¼ 0 on S;
@t
K D f ¼ lf with a point source at x0 ; y0 ; z0

Green’s function is [44, p. 355]

where D is a diagonal matrix with Dii ¼ wi.
1
u¼
8½pDðt  tÞ3=2
9.5. Green’s Functions [158–162] 2
þðyy0 Þ2 þðzz0 Þ2 =4 DðttÞ
e½ðxx0 Þ

Integral equations can arise from the formula-

tion of a problem by using Green’s function. When the problem is
For example, the equation governing heat con- @c
¼ Dr2 c
duction with a variable heat generation rate is @t
represented in differential forms as c ¼ f ðx; y; zÞin S at t ¼ 0

c ¼ fðx; y; zÞon S; t > 0

d2 T QðxÞ
¼ ; Tð0Þ ¼ Tð1Þ ¼ 0
dx2 k
the solution can be represented as [44, p. 353]
In integral form the same problem is [149, pp. ZZZ
57–60] c¼ ðuÞt¼0 f ðx; y; zÞdxdydz

ZZZ
t
Z1 @u
þD fðx; y; z; tÞ dSdt
TðxÞ ¼
1
Gðx; yÞQðyÞdy @n
k 0
0
8
< xð1  yÞ xy When the problem is two-dimensional,
Gðx; yÞ ¼
: 1 2 2
yð1  xÞ yx u ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi e½ðxx0 Þ þðyy0 Þ =4DðttÞ
4pDðt  tÞ
ZZ
Green’s functions for typical operators are c¼ ðuÞt¼0 f ðx; yÞdxdy
given below.
ZZt
For the Poisson equation with solution þD
@u
fðx; y; tÞ dCdt
decaying to zero at infinity @n
0

2
r c ¼ 4p% For the following differential equation and
boundary conditions
the formulation as an integral equation is  
1 d dc
xa1 ¼ f ½x; cðxÞ;
Z xa1 dx dx
cðrÞ ¼ rðr0 ÞGðr; r0 ÞdV 0 dc 2 dc
ð0Þ ¼ 0; ð1Þ þ cð1Þ ¼ g
V dx Sh dx
 Mathematics in Chemical Engineering 101

where Sh is the Sherwood number, the problem Thus, the problem ends up as one directly
can be written as a Hammerstein integral equa- formulated as a fixed point problem:
tion:
f ¼ Fðf Þ
Z1
cðxÞ ¼ g  Gðx; y; ShÞf ½y; cðyÞya1 dy
When the problem is the diffusion–reaction
0
one, the form is
Green’s function for the differential operators Zx
are [163] cðxÞ ¼ g  ½uðxÞvðyÞ  uðyÞvðxÞ
0

a¼1
( 2
f ½y; cðyÞya1 dy  avðxÞ

1 þ  x; yx Z1
Sh a¼ uðyÞf ½y; cðyÞya1 dy
Gðx; y; ShÞ ¼ a ¼ 2Gðx; y; ShÞ
2
1 þ  y; x<y 0
Sh

(2 DIXIT and TAULARIDIS [164] solved problems

 lnx; yx involving Fischer–Tropsch synthesis reactions
Sh
¼ a ¼ 3Gðx; y; ShÞ in a catalyst pellet using a similar method.
2
 lny; x<y
Sh

(2 1
9.6. Boundary Integral Equations and
þ  1; yx
Sh x Boundary Element Method
¼
2 1
þ  1; x<y
Sh y The boundary element method utilizes Green’s
theorem and integral equations. Here, the
Green’s functions for the reaction diffusion method is described briefly for the following
problem were used to provide computable error boundary value problem in two or three dimen-
bounds by FERGUSON and FINLAYSON [163]. sions
If Green’s function has the form
@f
( r2 f ¼ 0; f ¼ f 1 on S1 ; ¼ f 2 on S2
uðxÞvðyÞ 0yx @n
Kðx; yÞ ¼
uðyÞvðxÞ xy1
Green’s theorem (see chap. 5) says that for any
functions sufficiently smooth
the problem
Z Z  
@c @f
Z1 ðfr2 c  cr2 fÞdV ¼ f c dS
@n @n
f ðxÞ ¼ Kðx; yÞF½y; f ðyÞdy V S

Suppose the function c satisfies the equation

may be written as
r2 c ¼ 0
Zx
f ðxÞ  ½uðxÞvðyÞ  uðyÞvðxÞ
0 In two and three dimensions, such a function is
f ½y; f ðyÞdy ¼ avðxÞ
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
c ¼ ln r; r ¼ ðx  x0 Þ2 þ ðy  y0 Þ2
where
in two dimensions
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
Z1 1
c ¼ ; r ¼ ðx  x0 Þ2 þ ðy  y0 Þ2 þ ðz  z0 Þ2
a¼ uðyÞF½y; f ðyÞdy r
0 in three dimensions
102 Mathematics in Chemical Engineering

If P is on the boundary, the result is [165, p.

464]
Z  
1 @ ln r @f
fðPÞ ¼ f  ln r dS
p @n @n
S

Putting in the boundary conditions gives

 
R @ ln r @f
pfðPÞ ¼ f1  ln r dS
@n
S1 @n
  ð48Þ
Figure 31. Domain with singularity at P R @ ln r
þ S2 f  f 2 ln r dS
@n

where {x0, y0} or {x0, y0, z0} is a point in the This is an integral equation for f on the bound-
domain. The solution f also satisfies ary. Note that the order is one less than the
original differential equation. However, the
r2 f ¼ 0 integral equation leads to matrices that are
dense rather than banded or sparse, so some
so that of the advantage of lower dimension is lost.
Once this integral equation (Eq. 48) is solved to
Z  
@c @f find f on the boundary, it can be substituted in
f c dS ¼ 0
@n @n Equation (47) to find f anywhere in the domain.
S
In the boundary finite element method, both
Consider the two-dimensional case. Since the function and its normal derivative along the
the function c is singular at a point, the inte- boundary are approximated.
grals must be carefully evaluated. For the
X N  
region shown in Figure 31, the domain is N
@f X @f
f¼ fj N j ðjÞ; ¼ N j ðjÞ
S ¼ S1þ S2; a small circle of radius r0 is placed j¼1
@n j¼1
@n j
around the point P at x0, y0. Then the full
integral is One choice of trial functions can be the piece-
 
wise constant functions shown in Figure 32.
R @ ln r @f The integral equation then becomes
f  ln r dS
S @n @n
2 3
Z 
u¼2p  Z  Z
@ ln r0 @f XN
6 @ ln ri @fj 7
þ f  ln r0 r0 du ¼ 0 pfi 4f j dS  ln r i 5ds
@n @n @n @n
u¼0 j¼1
sj sj

As r0 approaches 0, The function fj is of course known along s1,

whereas the derivative @fj/@n is known along s2.
lim r 0 ln r0 ¼ 0
r 0 !0

and

Z
u¼2p
@ ln r0
lim f r0 du ¼ fðPÞ2p
r 0 !0 @n
u¼0

Thus for an internal point,

Z  
1 @ ln r @f
fðPÞ ¼ f  ln r dS ð47Þ Figure 32. Trial function on boundary for boundary finite
2p @n @n
S element method
 Mathematics in Chemical Engineering 103

This set of equations is then solved for fi and mathematical optimization problems involving
@fi/@n along the boundary. This constitutes the continuous and discrete (or integer) variables.
boundary integral method applied to the Laplace This is followed by a review of solution meth-
equation. ods of the major types of optimization problems
If the problem is Poisson’s equation for continuous and discrete variable optimiza-
tion, particularly nonlinear and mixed-integer
r2 f ¼ gðx; yÞ nonlinear programming. In addition, we discuss
direct search methods that do not require deriv-
Green’s theorem gives ative information as well as global optimization
methods, before reviewing extensions of these
Z   Z
@ ln r @f methods for the optimization of systems
f  ln r dS þ g ln r dA ¼ 0
@n @n described by differential and algebraic equa-
S A
tions. These methods, which are designed for
Thus, for an internal point, single-objective optimization problems, form
the basis to tackle multicriteria optimization
 
R @ ln r @f problems and optimization under uncertainty,
2pfðPÞ ¼ f  ln r dS
S @r @n ð49Þ which are reviewed in the two final parts of this
R chapter.
þ A g ln rdA

and for a boundary point,

10.1. Introduction
 
R @ ln r @f
pfðPÞ ¼ f  ln r dS Optimization is a key enabling tool for decision
S @n @n ð50Þ
R support in chemical engineering [169]. It has
þ A g ln rdA
evolved from a methodology of academic inter-
est into a technology that continues to make
If the region is nonhomogeneous this method
significant impact in engineering research and
can be used [165, p. 475], and it has been
practice. Optimization algorithms form the core
applied to heat conduction by HSIEH and SHANG
tools for a) experimental design, parameter
[166]. The finite element method can be applied
estimation, model development, and statistical
in one region and the boundary finite element
analysis; b) process synthesis analysis, design,
method in another region, with appropriate
and retrofit; c) model predictive control and
matching conditions [165, p. 478]. If the prob-
real-time optimization; and d) planning, sched-
lem is nonlinear, then it is more difficult. For
uling, and the integration of process operations
example, consider an equation such as Pois-
into the supply chain [170, 171].
son’s in which the function depends on the
solution as well As shown in Figure 35, optimization prob-
lems that arise in chemical engineering can be
r2 f ¼ gðx; y; fÞ
classified in terms of continuous and discrete
variables. For the former, nonlinear program-
Then the integral appearing in Equation (50) ming (NLP) problems form the most general
must be evaluated over the entire domain, and case, and widely applied specializations
the solution in the interior is given by include linear programming (LP) and quadratic
Equation (49). For further applications, see
[167] and [168].

10. Optimization
We provide a survey of systematic methods for
a broad variety of optimization problems. The
survey begins with a general classification of Figure 33.
104 Mathematics in Chemical Engineering

integer; this gives rise to an integer program-

ming (IP) problem. IPs can be further classified
into many special problems (e.g., assignment,
traveling salesman, etc.), which are not shown
in Figure 35. Similarly, the MINLP problem
also gives rise to special problem classes,
although here the main distinction is whether
its relaxation is convex or nonconvex.
The ingredients of formulating optimization
problems include a mathematical model of the
system, an objective function that quantifies a
criterion to be extremized, variables that can
serve as decisions, and, optionally, inequality
Figure 34. constraints on the system. When represented in
algebraic form, the general formulation of dis-
crete/continuous optimization problems can be
programming (QP). An important distinction written as follows:
for NLP is whether the optimization problem is
convex or nonconvex. The latter NLP problem Min f ð xÞ
may have multiple local optima, and an impor- s:t: x2X
tant question is whether a global solution is
required for the NLP. Another important dis- where, s.t. stand for subject to and X denotes the
tinction is whether the problem is assumed to be feasible set of the optimization problem. It can
differentiable or not. be modeled by equality and unequality con-
Mixed-integer problems also include dis- straints for the continuous as well as for discrete
crete variables. These can be written as variables. This results in the following mixed-
mixed-integer nonlinear programs (MINLP) integer optimization problem:
or as mixed-integer linear programs (MILP)
if all variables appear linearly in the constraint Min f ðx; yÞ
and objective functions. For the latter an impor- s:t: hðx; yÞ ¼ 0
tant case occurs when all the variables are ð51Þ
gðx; yÞ  0

x 2 n ; y 2 f0; 1gt

where f(x, y) is the objective function (e.g., cost,

energy consumption, etc.), h(x, y) ¼ 0 are the
equations that describe the performance of the
system (e.g., material balances, production
rates), the inequality constraints g(x, y)  0
can define process specifications or constraints
for feasible plans and schedules. Note that the
operator Max f(x) is equivalent to Min f(x).
We define the real n-vector x to represent the
continuous variables while the t-vector y repre-
sents the discrete variables, which, without loss
of generality, are often restricted to take 0–1
values to define logical or discrete decisions,
such as assignment of equipment and sequenc-
ing of tasks. (These variables can also be for-
mulated to take on other integer values as well.)
Figure 35. Classes of optimization problems and Problem (51) corresponds to a mixed-integer
algorithms nonlinear program (MINLP) when any of the
 Mathematics in Chemical Engineering 105

functions involved are nonlinear. If all functions Strict convexity requires that the inequalities
are linear it corresponds to a mixed-integer linear (53) and (54) be strict. Convex feasible
program (MILP). If there are no 0–1 variables, regions require g(x) to be a convex function
then Problem (51) reduces to a nonlinear pro- and h(x) to be linear. If Problem (52) is a
gram (52) or linear program (65) depending on convex problem, than any local solution is
whether or not the functions are linear. guaranteed to be a global solution to Problem
We first start with continuous variable opti- (52). Moreover, if the objective function is
mization and consider in the next section the strictly convex, then this solution x is unique.
solution of NLPs with differentiable objective On the other hand, nonconvex problems may
and constraint functions. If only local solutions have multiple local solutions, i.e., feasible
are required for the NLP, then very efficient solutions that minimize the objective function
large-scale methods can be considered. This is within some neighborhood about the solution.
followed by methods that are not based on local We first consider methods that find only
optimality criteria; we consider direct search local solutions to nonconvex problems, as
optimization methods that do not require deriv- more difficult (and expensive) search proce-
atives, as well as deterministic global optimi- dures are required to find a global solution.
zation methods. Following this, we consider the Local methods are currently very efficient and
solution of mixed-integer problems and outline have been developed to deal with very large
the main characteristics of algorithms for their NLPs. Moreover, by considering the structure
solution. Finally, we conclude with a discussion of convex NLPs (including LPs and QPs), even
of optimization modeling software and its more powerful methods can be applied. To
implementation in engineering models. study these methods, we first consider condi-
tions for local optimality.

10.2. Gradient-Based Nonlinear

Programming Local Optimality Conditions — A Kinematic
Interpretation. Local optimality conditions
For continuous variable optimization we con- are generally derived from gradient information
sider Problem (51) without discrete variables y. from the objective and constraint functions. The
The general NLP Problem (52) is presented proof follows by identifying a local minimum
below: point that has no feasible descent direction.
Invoking a theorem of the alternative (e.g.,
Min f ðxÞ Farkas–Lemma) leads to the celebrated Kar-
ush–Kuhn–Tucker (KKT) conditions [172].
s:t: hðxÞ ¼ 0 ð52Þ
Instead of a formal development of these con-
gðxÞ  0 ditions, we present here a more intuitive, kine-
matic illustration. Consider the contour plot of
and we assume that the functions f(x), h(x), and the objective function f(x) given in Figure 36 as
g(x) have continuous first and second deriva- a smooth valley in space of the variables x1 and
tives. A key characteristic of Problem (53) is x2. For the contour plot of this unconstrained
whether the problem is convex or not, i.e., problem, Min f(x), consider a ball rolling in this
whether it has a convex objective function valley to the lowest point of f(x), denoted by x .
and a convex feasible region. A function f(x) This point is at least a local minimum and is
of x in some domain X is convex if and only if defined by a point with zero gradient and at
for all points x1, x2 2 X: least nonnegative curvature in all (nonzero)
directions p. We use the first derivative (gradi-
f½ax1 þ ð1  aÞx2   afðx1 Þ þ ð1  aÞfðx2 Þ ð53Þ ent) vector rf(x) and second derivative (Hes-
sian) matrix rxxf(x) to state the necessary first-
holds for all a 2 (0, 1). If f(x) is differentiable, and second-order conditions for unconstrained
then an equivalent definition is: optimality:

f x1 Þ þ rfðx1 ÞT ðx  x1 Þ  fðxÞ ð54Þ 
rx f ðx Þ ¼ 0

pT rxx f ðx Þp  0 for all p 6¼ 0 ð55Þ
106 Mathematics in Chemical Engineering

 
fences [rgðx Þ] and rails [rhðx Þ] on the
ball are balanced by the force of “gravity”

[rf ðx Þ]. This condition can be stated by
the following KKT necessary conditions for
constrained optimality:
Stationarity Condition: It is convenient to
define the Lagrangian function L(x, l, n) ¼ f(x) þ
g(x)Tl þ h(x)Tn along with “weights” or multi-
pliers l and n for the constraints. These multipli-
ers are also known as “dual variables” and
“shadow prices”. The stationarity condition
(balance of forces acting on the ball) is then given
by:

Figure 36. Unconstrained minimum rLðx; l; nÞ ¼ rf ðxÞ þ rhðxÞl þ rgðxÞn ¼ 0 ð56Þ

Feasibility: Both inequality and equality con-

These necessary conditions for local optimality straints must be satisfied (ball must lie on the rail
can be strengthened to sufficient conditions by and within the fences):
making the inequality in the Relations (56)
strict (i.e., positive curvature in all directions). hðxÞ ¼ 0; gðxÞ  0 ð57Þ
Equivalently, the sufficient (necessary) curva-

ture conditions can be stated as: rxx f ðx Þ has all Complementarity: Inequality constraints are
positive (nonnegative) eigenvalues and is there- either strictly satisfied (active) or inactive,
fore defined as a positive (semi-)definite in which case they are irrelevant to the solu-
matrix. tion. In the latter case the corresponding KKT
Now consider the imposition of inequality multiplier must be zero. This is written as:
[g(x)  0] and equality constraints [h(x) ¼ 0] in
Figure 37. Continuing the kinematic interpre- nT gðxÞ ¼ 0; n0 ð58Þ
tation, the inequality constraints g(x)  0 act as
“fences” in the valley, and equality constraints Constraint Qualification: For a local optimum
h(x) ¼ 0 as “rails”. Consider now a ball, to satisfy the KKT conditions, an additional
constrained on a rail and within fences, to regularity condition or constraint qualification
roll to its lowest point. This stationary point (CQ) is required. The KKT conditions are
occurs when the normal forces exerted by the derived from gradient information, and the
CQ can be viewed as a condition on the relative
influence of constraint curvature. In fact, lin-
early constrained problems with non feasible
regions require no constraint qualification. On
the other hand, as seen in Figure 38, the
problem Min x1, s.t. x2  0, (x1)3  x2 has a
minimum point at the origin but does not
satisfy the KKT conditions because it does
not satisfy a CQ.
CQs be defined in several ways. For
instance, the linear independence constraint
qualification (LICQ) requires that the active
constraints at x be linearly independent, i.e.,
 
the matrix ½rhðx ÞjrgA ðx Þ is full column
rank, where gA is the vector of inequality
constraints with elements that satisfy

Figure 37. Constrained minimum gA;i ðx Þ ¼ 0. With LICQ, the KKT multipliers
 Mathematics in Chemical Engineering 107

for the first-order Conditions (54):

   
2x1  4 þ x2  1=x1  1:3475
rf ðxÞ ¼ ¼0)x ¼ ð61Þ
3x2  7 þ x1  1=x2 2:0470

and f(x ) ¼ 2.8742. Checking the second-

order conditions leads to:
" 
#
2 þ 1=ðx1 Þ2 1
rxx f ðx Þ ¼  2
1 3 þ 1=ðx2 Þ
 
 2:5507 1
) rxx f ðx Þ ¼ ðpositive definiteÞ ð62Þ
1 3:2387
Figure 38. Failure of KKT conditions at constrained mini-
mum (note linear dependence of constraint gradients)
Now consider the constrained NLP:

(l, n) are guaranteed to be unique at the optimal Min ðx1 Þ2  4x1 þ 3=2ðx1 Þ2  7x2 þ x1 x2 þ 9  lnx1  lnx2
solution. The weaker Mangasarian–Fromovitz s:t: 4  x1 x 2  0 ð63Þ
constraint qualification (MFCQ) requires only 2x1  x2 ¼ 0
that rh(x ) have full column rank and that a
direction p exist that satisfies rh(x )Tp ¼ 0 and that corresponds to the plot in Figure 37. The
rgA(x )Tp > 0. With MFCQ, the KKT multi- optimal solution can be found by applying the
pliers (l, n) are guaranteed to be bounded (but first-order KKT Conditions (56–58):
not necessarily unique) at the optimal solution.
Additional discussion can be found in [172]. rLðx; l; nÞ ¼ rf ðxÞ þ rhðxÞl þ rgðxÞn
Second-Order Conditions: As with un-      
2x1  4 þ x2  1=x1 2 x2
constrained optimization, nonnegative (posi- ¼ þ lþ n¼0
3x2  7 þ x1  1=x2 1 x1
tive) curvature is necessary (sufficient) in all gðxÞ ¼ 4  x1 x2  0; hðxÞ ¼ 2x1  x2 ¼ 0 ð64Þ
of the allowable (i.e., constrained) nonzero gðxÞn ¼ ð4  x1 x2 Þn; n  0
  +
directions p. This condition can be stated in 1:4142
several ways. A typical necessary second-order x ¼ ; l ¼ 1:036; n ¼ 1:068
2:8284
condition requires a point x that satisfies LICQ
and first-order Conditions (56–58) with multi- and f(x ) ¼ 1.8421. Checking the second-
pliers (l, n) to satisfy the additional conditions order Conditions (60) leads to:
given by:
rxx Lðx ; l ; n Þ ¼ rxx ½f ðx Þ þ hðx Þl þ gðx Þn 
   
  
pT rxx Lðx ; l; nÞp  0 for all p 6¼ 0; rhðx ÞT p ¼ 0; rgA ðx ÞT p ¼ 0 2
2:5 0:068
ð59Þ ¼ 2 þ 1=ðx1 Þ 1n
2 ¼
1n 3 þ 1=ðx2 Þ 0:068 3:125
 
2 2:8284
The corresponding sufficient conditions require ½rhðx ÞjrgA ðx Þp ¼ p ¼ 0; p 6¼ 0
1 1:4142
that the inequality in Condition (59) be strict.
Note that for the example in Figure 36, the Note that LICQ is satisfied. Moreover, because
allowable directions p span the entire space for  
½rhðx ÞjrgA ðx Þ is square and nonsingular,
x while in Figure 37, there are no allowable there are no nonzero vectors p that satisfy the
directions p. allowable directions. Hence, the sufficient sec-

Example: To illustrate the KKT conditions,  
ond-order conditions (pT rxx Lðx ; l ; n Þ > 0,
consider the following unconstrained NLP: for all allowable p) are vacuously satisfied
for this problem.
Min ðx1 Þ2  4x1 þ 3=2ðx2 Þ2  7x2 þ x1 x2 þ 9  lnx1  lnx2
ð60Þ Convex Cases of NLP. Linear programs and
quadratic programs are special cases of Problem
corresponding to the contour plot in Figure 36. (52) that allow for more efficient solution, based
The optimal solution can be found by solving on application of KKT Conditions (56–59).
108 Mathematics in Chemical Engineering

Because these are convex problems, any locally developed in the late 1940s [173], although,
optimal solution is a global solution. In particu- starting from KARMARKAR’s discovery in 1984,
lar, if the objective and constraint functions in interior point methods have become quite
Problem (52) are linear then the following linear advanced and competitive for large scale prob-
program (LP): lems [174]. The simplex method proceeds by
moving successively from vertex to vertex with
Min cT x improved objective function values. Methods to
s:t: Ax ¼ b ð65Þ solve Problem (66) are well implemented and
widely used, especially in planning and logisti-
Cx  d
cal applications. They also form the basis for
MILP methods (see below). Currently, state-of-
can be solved in a finite number of steps, and the
the-art LP solvers can handle millions of vari-
optimal solution lies at a vertex of the polyhe-
ables and constraints and the application of
dron described by the linear constraints. This is
further decomposition methods leads to the
shown in Figure 39, and in so-called primal
solution of problems that are two or three orders
degenerate cases, multiple vertices can be alter-
of magnitude larger than this [175, 176]. Also,
nate optimal solutions with the same values of
the interior point method is described below
the objective function. The standard method to
from the perspective of more general NLPs.
solve Problem (65) is the simplex method,
Quadratic programs (QP) represent a slight
modification of Problem (65) and can be stated
as:

1
Min cT x þ xT Qx
2
s:t: Ax ¼ b ð66Þ

xd

If the matrix Q is positive semidefinite (positive

definite), when projected into the null space of
the active constraints, then Problem (66) is
(strictly) convex and the QP is a global (and
unique) minimum. Otherwise, local solutions
exist for Problem (66), and more extensive
global optimization methods are needed to
obtain the global solution. Like LPs, convex
QPs can be solved in a finite number of steps.
However, as seen in Figure 40, these optimal
solutions can lie on a vertex, on a constraint
boundary, or in the interior. A number of active
set strategies have been created that solve the
KKT conditions of the QP and incorporate
efficient updates of active constraints. Popular
QP methods include null space algorithms,
range space methods, and Schur complement
methods. As with LPs, QP problems can also be
solved with interior point methods [174].

Solving the General NLP Problem. Solution

techniques for Problem (52) deal with satisfac-
tion of the KKT Conditions (56–59). Many NLP
solvers are based on successive quadratic pro-
Figure 39. Contour plots of linear programs gramming (SQP) as it allows the construction
 Mathematics in Chemical Engineering 109

satisfy Problem 67e] is to formulate, at a point

xk, and solve the following QP at iteration k:

1
Min rf ðxk ÞT p þ pT rxx Lðxk ; lk ; nk Þp
2
ð68Þ
s:t: hðxk Þ þ rhðxk ÞT p ¼ 0

gðxk Þ þ rgðxk ÞT p þ s ¼ 0; s  0

Figure 40. Contour plots of convex quadratic programs The KKT conditions of Equation (68) are given
by:

rf ðxk Þ þ r2 Lðxk ; lk ; nk Þp þ rhðxk Þl þ rgðxk Þn ¼ 0 (69a)

of a number of NLP algorithms based on the
Newton–Raphson method for equation solving.
hðxk Þ þ rhðxk ÞT p ¼ 0 (69b)
SQP solvers have been shown to require the
fewest function evaluations to solve NLPs [177]
and they can be tailored to a broad range of gðxk Þ þ rgðxk ÞT p þ s ¼ 0 (69c)

process engineering problems with different

structure. SVe ¼ 0 (69d)
The SQP strategy applies the equivalent of a
Newton step to the KKT conditions of the ðs; nÞ  0 (69e)
nonlinear programming problem, and this leads
to a fast rate of convergence. By adding slack where the Hessian of the Lagrange function
variables s the first-order KKT conditions can rxx Lðx; l; nÞ ¼ rxx ½f ðxÞ þ hðxÞT l þ gðxÞT n
be rewritten as: is calculated directly or through a quasi-
Newtonian approximation (created by differ-
rf ðxÞ þ rhðxÞl þ rgðxÞn ¼ 0 (67a) ences of gradient vectors). It is easy to show that
Equations (69a–69c) correspond to a Newton–
hðxÞ ¼ 0 (67b) Raphson step for Equations (67a–67c) applied
at iteration k. Also, selection of the active set
gðxÞ þ s ¼ 0 (67c) is now handled at the QP level by satisfying
the Conditions (69d, 69e). To evaluate and
SVe ¼ 0 (67d) change candidate active sets, QP algorithms
apply inexpensive matrix-updating strategies
ðs; nÞ  0 (67e) to the KKT matrix associated with the QP
68. Details of this approach can be found in
where e ¼ [1, 1, . . . , 1]T, S ¼ diag(s) and V ¼ [178, 179].
diag(n). SQP methods find solutions that satisfy As alternatives that avoid the combinatorial
Equations (67a–67e) by generating Newton- problem of selecting the active set, interior
like search directions at iteration k. However, point (or barrier) methods modify the NLP
Equation (67d) and active bounds (67e) are Problem (52) to form Problem (70)
dependent and serve to make the KKT system
P
ill-conditioned near the solution. SQP algo- Min f ðxk Þ  m i ln si
rithms treat these conditions in two ways. In s:t: hðxk Þ ¼ 0 ð70Þ
the active set strategy, discrete decisions are k
gðx Þ þ s ¼ 0
made regarding the active constraint set,

i 2 I ¼ fijgi ðx Þ ¼ 0g, and Equation (67d) is
replaced by si ¼ 0, i 2 I, and vi ¼ 0, i 2 = I. where the solution to this problem has s > 0 for
Determining the active set is a combinatorial the penalty parameter m > 0, and decreasing m
problem, and a straightforward way to deter- to zero leads to solution of Problem (52). The
mine an estimate of the active set [and also KKT conditions for this problem can be written
110 Mathematics in Chemical Engineering

as Equation (71) scale problems. Generally, these methods

require more function evaluations than SQP
rf ðx Þ þ rhðx Þl þ rgðx Þn ¼ 0 methods, but they perform very well when
interfaced to optimization modeling platforms,
hðx Þ ¼ 0
ð71Þ where function evaluations are cheap. All of
gðx Þ þ s ¼ 0 these can be derived from the perspective of
SVe ¼ me applying Newton steps to portions of the KKT
conditions.
LANCELOT [186] is based on the solution
and at iteration k the Newton steps to solve of bound constrained subproblems. Here an
Equation (71) are given by: augmented Lagrangian is formed from Problem
(52) and Subproblem (73) is solved.
2 3
rxx Lðxk ; lk ; nk Þ rhðxk Þ rgðxk Þ
6 S1
k Vk I 7
6 7 ð72Þ   1
4 rhðxk ÞT 5
Min f ðxÞ þ lT hðxÞ þn gðxÞ þ s þ rk gðxÞ; gðxÞ þ s k2
rgðxk ÞT I 2 ð73Þ
2 3 2 3 s:t: s  0
Dx rx Lðxk ; lk ; nk Þ
6 Ds 7 6 1 7
6 7 ¼ 6 nk  Sk me 7
4 Dl 5 4 hðxk Þ 5
The above subproblem can be solved very
Dn gðxk Þ þ sk
efficiently for fixed values of the multipliers
l and v and penalty parameter r. Here a gradi-
A detailed description of this algorithm, called ent-projection, trust-region method is applied.
IPOPT, can be found in [180]. Once Subproblem (73) is solved, the multipliers
Both active set and interior point methods and penalty parameter are updated in an outer
have clear trade-offs. Interior point methods loop and the cycle repeats until the KKT con-
may require more iterations to solve Problem ditions for Problem (52) are satisfied. LANCE-
(70) for various values of m, while active set LOT works best when exact second derivatives
methods require the solution of the more expen- are available. This promotes a fast convergence
sive QP Subproblem (68). Thus, if there are few rate in solving each subproblem and allows a
inequality constraints or an active set is known bound constrained trust-region method to
(say from a good starting guess, or a known QP exploit directions of negative curvature in the
solution from a previous iteration) then solving Hessian matrix.
Problem (68) is not expensive and the active set Reduced gradient methods are active set
method is favored. On the other hand, for strategies that rely on partitioning the variables
problems with many inequality constraints, and solving Equations (67a–67e) in a nested
interior point methods are often faster as they manner. Without loss of generality, Problem
avoid the combinatorial problem of selecting (52) can be rewritten as Problem (74).
the active set. This is especially the case for
large-scale problems and when a large number Min f ðzÞ
of bounds are active. Examples that demon- s:t: cðzÞ ¼ 0 ð74Þ
strate the performance of these approaches azb
include the solution of model predictive control
(MPC) problems [181–183] and the solution of Variables are partitioned as nonbasic variables
large optimal control problems using barrier (those fixed to their bounds), basic variables
NLP solvers. For instance, IPOPT allows the (those that can be solved from the equality
solution of problems with more than 106 var- constraints), and superbasic variables (those
iables and up to 50 000 degrees of freedom remaining variables between bounds that serve
[184, 185]. to drive the optimization); this leads to
zT ¼ ½zTN ; zTS ; zTB . This partition is derived from
Other Gradient-Based NLP Solvers. In addi- local information and may change over the
tion to SQP methods, a number of NLP solvers course of the optimization iterations. The cor-
have been developed and adapted for large- responding KKT conditions can be written as
 Mathematics in Chemical Engineering 111

Equations (75a–75e) from the reduced gradient method. At the

solution of this subproblem, the constraints
rN f ðzÞ þ rN cðzÞg ¼ ba  bb (75a)
75d are relinearized and the cycle repeats until
the KKT conditions of Equations (75a–75e) are
rS f ðzÞ þ rS cðzÞg ¼ 0 (75b)
satisfied. The augmented Lagrangian function
(Eq. 73) is used to penalize movement away
rB f ðzÞ þ rB cðzÞg ¼ 0 (75c) from the feasible region. For problems with few
degrees of freedom, the resulting approach
cðzÞ ¼ 0 (75d) leads to an extremely efficient method even
for very large problems. MINOS has been
zN;j ¼ aj or bj ; ba;j  0; bb;j ¼ 0 or bb;j  0; ba;j ¼ 0 (75e) interfaced to a number of modeling systems
and enjoys widespread use. It performs espe-
where l and b are the KKT multipliers for the cially well on large problems with few non-
equality and bound constraints, respectively, linear constraints. However, on highly
and Equation (75e) replaces the complementar- nonlinear problems it is usually less reliable
ity Conditions (58). Reduced gradient methods than other reduced gradient methods.
work by nesting Equations (75b), within
Equation (75d). At iteration k, for fixed values Algorithmic Details for NLP Methods. All of
of zkN and zkS , we can solve for zB using Equation the above NLP methods incorporate concepts
(75d) and for l using Equation (75b). Moreover, from the Newton–Raphson Method for equa-
linearization of these equations leads to tion solving. Essential features of these meth-
constrained derivatives or reduced gradients ods are a) providing accurate derivative
(Eq. 76) information to solve for the KKT conditions,
df b) stabilization strategies to promote conver-
¼ rf s  rcs ðrcB Þ1 rf B ð76Þ gence of the Newton-like method from poor
dzs
starting points, and c) regularization of the
which indicate how f(z) (and zB) change with Jacobian matrix in Newton’s method (the so-
respect to zS and zN. The algorithm then pro- called KKT matrix) if it becomes singular or ill-
ceeds by updating zS using reduced gradients in conditioned.
a Newton-type iteration to solve equation 75c.
Following this, bound multipliers b are calcu- a. Providing first and second derivatives: The
lated from Equation (75a). Over the course of KKT conditions require first derivatives to
the iterations, if the variables zB or zS exceed define stationary points, so accurate first
their bounds or if some bound multipliers b derivatives are essential to determine
become negative, then the variable partition locally optimal solutions for differentiable
needs to be changed and the Equations NLPs. Moreover, Newton–Raphson meth-
(75a–75d) are reconstructed. These reduced ods that are applied to the KKT conditions,
gradient methods are embodied in the popular as well as the task of checking second-
GRG2, CONOPT, and SOLVER codes [176]. order KKT conditions, necessarily require
The SOLVER code has been incorporated into information on second derivatives. (Note
Microsoft Excel. CONOPT [187] is an efficient that second-order conditions are not
and widely used code in several optimization checked by methods that do not use second
modeling environments. derivatives). With the recent development
MINOS [188] is a well-implemented pack- of automatic differentiation tools, many
age that offers a variation on reduced gradient modeling and simulation platforms can
strategies. At iteration k, Equation (75d) is provide exact first and second derivatives
replaced by its linearization (Eq. 77) for optimization. When second derivatives
are available for the objective or constraint
cðzkN ; zkS ; zkB Þ þ rB cðzk ÞT ðzB  zkB Þ þ rS cðzk ÞT ðzS  zkS Þ ¼ 0
functions, they can be used directly in
ð77Þ
LANCELOT, SQP and, less efficiently,
and Equations (75a, 75c, 75e) are solved with in reduced gradient methods. Otherwise,
Equation (77) as a subproblem using concepts for problems with few superbasic
112 Mathematics in Chemical Engineering

variables, reduced gradient methods and if there is sufficient reduction of some

reduced space variants of SQP can be merit function (e.g., the objective function
applied. Referring to problem 74 with n weighted with some measure of the con-
variables and m equalities, we can write the straint violations). The size of the trust
QP step from Problem (68) as Equation (78). region D is adjusted based on the agree-
ment of the reduction of the actual merit
Min rf ðzk ÞT p þ 12 pT rxx Lðzk ; g k Þp ð78Þ function compared to its predicted reduc-
s:t: cðzk Þ þ rcðzk ÞT p ¼ 0 tion from the subproblem [186]. Such
methods have strong global convergence
a  zk þ p  b properties and are especially appropriate
Defining the search direction as p ¼ ZkpZ þ for ill-conditioned NLPs. This approach
YkpY, where rc(xk)TZk ¼ 0 and [Yk|Zk] is a has been applied in the KNITRO code
nonsingular n  n matrix, allows us to form [189]. Line-search methods can be more
the following reduced QP subproblem (with efficient on problems with reasonably good
nm variables) starting points and well-conditioned sub-
problems, as in real-time optimization.
1
Min ½Z Tk rf ðzk Þ þ wk T pZ þ pzT Bk pZ Typically, once a search direction is calcu-
2 ð79Þ
lated from Problem (68), or other related
s:t: a  zk þ Z k pZ þ Y k pY  b
subproblem, a step size a 2 (0, 1) is chosen
where pY ¼ [rc(zk)TYk]1c(zk). Good so that xk þ ap leads to a sufficient
choices of Yk and Zk, which allow sparsity decrease of a merit function. As a recent
to be exploited in rc(zk), are: Y Tk ¼ ½0jI alternative, a novel filter-stabilization
and Z Tk ¼ ½Ij  rN;S cðzk ÞrB cðzk Þ1 . Here strategy (for both line-search and trust-
we define the reduced Hessian Bk ¼ region approaches) has been developed
Z Tz rzz Lðzk ; g k ÞZ k and wk ¼ Z Tk rzz Lðzk ; based on a bicriterion minimization, with
g k ÞY k pY . In the absence of second-deriva- the objective function and constraint
tive information, Bk can be approximated infeasibility as ompeting objectives
using positive definite quasi-Newton [190]. This method often leads to better
approximations [178]. Also, for the interior performance than those based on merit
point method, a similar reduced space functions.
decomposition can be applied to the New- c. Regularization of the KKT matrix for the
ton step given in Equation (72). NLP subproblem (e.g., in Eq. 72) is essen-
Finally, for problems with least-squares tial for good performance of general pur-
functions, as in data reconciliation, parame- pose algorithms. For instance, to obtain a
ter estimation, and model predictive control, unique solution to Equation (72), active
one can often assume that the values of the constraint gradients must be full rank,
objective function and its gradient at the and the Hessian matrix, when projected
solution are vanishingly small. Under these into the null space of the active constraint
conditions, one can show that the multipliers gradients, must be positive definite. These
(l, n) also vanish and rxxL(x , l, n) can be properties may not hold far from the solu-
substituted by rxxf(x ). This Gauss–Newton tion, and corrections to the Hessian in SQP
approximation has been shown to be very may be necessary [179]. Regularization
efficient for the solution of least-squares methods ensure that subproblems like
problems [178]. Equations (68) and (72) remain well-con-
b. Line-search and trust-region methods are ditioned; they include addition of positive
used to promote convergence from poor constants to the diagonal of the Hessian
starting points. These are commonly used matrix to ensure its positive definiteness,
with the search directions calculated from judicious selection of active constraint gra-
NLP subproblems such as Problem (69). In dients to ensure that they are linearly inde-
a trust-region approach, the constraint, kpk pendent, and scaling the subproblem to
 D is added and the iteration step is taken reduce the propagation of numerical errors.
 Mathematics in Chemical Engineering 113

Table 11. Representative NLP solvers

Method Algorithm type Stabilization Second-order information

CONOPT [187] reduced gradient line search exact and quasi-Newton

GRG2 [176] reduced gradient line search quasi-Newton
IPOPT [180] SQP, barrier line search exact
KNITRO [189] SQP, barrier trust region exact and quasi-Newton
LANCELOT [186] aug mented Lagrangian, bound constrained trust region exact and quasi-Newton
LOQO [191] SQP, barrier line search exact
MINOS [188] reduced gradient, augmented Lagrangian line search quasi-Newton
NPSOL [192] SQP, Active set line search quasi-Newton
SNOPT [193] reduced space SQP, active set line search quasi-Newton
SOCS [194] SQP, active set line search exact
SOLVER [176] reduced gradient line search quasi-Newton
SRQP [195] reduced space SQP, active set line search quasi-Newton

Often these strategies are heuristics built one-at-a-time search and methods based on
into particular NLP codes. While quite experimental designs [196]. At that time, these
effective, most of these heuristics do not direct search methods were the most popular
provide convergence guarantees for gen- optimization methods in chemical engineering.
eral NLPs. Methods that fall into this class include the
pattern search [197], the conjugate direction
Table 11 summarizes the characteristics of a method [198], simplex and complex searches
collection of widely used NLP codes. Much [199], and the adaptive random search methods
more information on widely available codes [200–202]. All of these methods require only
can also be found on the NEOS server objective function values for unconstrained
(www-neos.mcs.anl.gov) and the NEOS minimization. Associated with these methods
Software Guide. are numerous studies on a wide range of process
problems. Moreover, many of these methods
include heuristics that prevent premature ter-
10.3. Optimization Methods without mination (e.g., directional flexibility in the
Derivatives complex search as well as random restarts
and direction generation). To illustrate these
A broad class of optimization strategies does methods, Figure 41 illustrates the performance
not require derivative information. These of a pattern search method as well as a
methods have the advantage of easy imple- random search method on an unconstrained
mentation and little prior knowledge of the problem.
optimization problem. In particular, such
methods are well suited for “quick and dirty” Simulated Annealing. This strategy is related
optimization studies that explore the scope of to random search methods and derives from a
optimization for new problems, prior to class of heuristics with analogies to the motion
investing effort for more sophisticated model- of molecules in the cooling and solidification of
ing and solution strategies. Most of these metals [203]. Here a “temperature” parameter u
methods are derived from heuristics that nat- can be raised or lowered to influence the prob-
urally spawn numerous variations. As a result, ability of accepting points that do not improve
a very broad literature describes these meth- the objective function. The method starts with a
ods. Here we discuss only a few important base point x and objective value f(x). The next
trends in this area. point x’ is chosen at random from a distribution.
If f(x’) < f(x), the move is accepted with x’
Classical Direct Search Methods. Developed as the new point. Otherwise, x’ is accepted
in the 1960s and 1970s, these methods include with probability p(u, x’,x). Options include the
114 Mathematics in Chemical Engineering

of genetic modification, crossover or mutation,

are used and the elements of the optimization
vector x are represented as binary strings.
Crossover deals with random swapping of vec-
tor elements (among parents with highest
objective function values or other rankings of
population) or any linear combinations of two
parents. Mutation deals with the addition of a
random variable to elements of the vector.
Genetic algorithms (GAs) have seen wide-
spread use in process engineering and a number
of codes are available. A related GA algorithm
is described in [176].

Derivative-Free Optimization (DFO). In the

past decade, the availability of parallel comput-
ers and faster computing hardware and the need
to incorporate complex simulation models
within optimization studies have led a number
of optimization researchers to reconsider clas-
sical direct search approaches. In particular,
DENNIS and TORCZON [205] developed a multi-
dimensional search algorithm that extends the
simplex approach [199]. They note that the
Nelder–Mead algorithm fails as the number
of variables increases, even for very simple
problems. To overcome this, their multi-
dimensional pattern-search approach combines
reflection, expansion, and contraction steps that
act as line search algorithms for a number of
linear independent search directions. This
approach is easily adapted to parallel computa-
tion and the method can be tailored to the
Figure 41. Examples of optimization methods without
derivatives number of processors available. Moreover,
A) Pattern search method; B) Random search method this approach converges to locally optimal
Circles: 1st phase; Triangles: 2nd phase; Stars: 3rd phase solutions for unconstrained problems and
exhibits an unexpected performance synergy
when multiple processors are used. The work
of DENNIS and TORCZON [205] has spawned
Metropolis distribution, p(u, x, x’) ¼ exp considerable research on analysis and code
[{f(x’)f(x)}/u], and the Glauber distri- development for DFO methods. Moreover,
bution, p(u, x, x’) ¼ exp [{f(x’)f(x)}/(1 þ exp CONN et al. [206] construct a multivariable
[{f(x’)f(x)}]/u)]. The u parameter is then DFO algorithm that uses a surrogate model
reduced and the method continues until no for the objective function within a trust-region
further progress is made. method. Here points are sampled to obtain a
well-defined quadratic interpolation model, and
Genetic Algorithms. This approach, first pro- descent conditions from trust-region methods
posed in [204], is based on the analogy of enforce convergence properties. A number of
improving a population of solutions through trust-region methods that rely on this approach
modifying their gene pool. It also has similar are reviewed in [206]. Moreover, a number of
performance characteristics as random search DFO codes have been developed that lead to
methods and simulated annealing. Two forms black-box optimization implementations for
 Mathematics in Chemical Engineering 115

large, complex simulation models. These variables and compares lower bound and upper
include the DAKOTA package at Sandia bound for fathoming each subregion. The one
National Lab [207], http://endo.sandia.gov/ that contains the optimal solution is found by
DAKOTA/software.html and FOCUS eliminating subregions that are proved not to
developed at Boeing Corporation [208]. contain the optimal solution.
Direct search methods are easy to apply to a For nonconvex NLP problems, QUESADA and
wide variety of problem types and optimization GROSSMANN [210] proposed a spatial branch-
models. Moreover, because their termination and-bound algorithm for concave separable,
criteria are not based on gradient information linear fractional, and bilinear programs using
and stationary points, they are more likely to linear and nonlinear underestimating functions
favor the search for global rather than locally [211]. For nonconvex MINLP, RYOO and SAHI-
optimal solutions. These methods can also be NIDIS [212] and later TAWARMALANI and SAHINIDIS
adapted easily to include integer variables. [213] developed BARON, which branches on
However, rigorous convergence properties to the continuous and discrete variables with
globally optimal solutions have not yet been bounds reduction method. ADJIMAN et al.
discovered. Also, these methods are best suited [214, 215] proposed the SMIN-aBB and
for unconstrained problems or for problems GMIN-aBB algorithms for twice-differentiable
with simple bounds. Otherwise, they may nonconvex MINLPs. Using a valid convex
have difficulties with constraints, as the only underestimation of general functions as well
options open for handling constraints are equal- as for special functions, ADJIMAN et al. [216]
ity constraint elimination and addition of pen- developed the aBB method, which branches on
alty functions for inequality constraints. Both both the continuous and discrete variables
approaches can be unreliable and may lead to according to specific options. The branch-
failure of the optimization algorithm. Finally, and-contract method [217] has bilinear, linear
the performance of direct search methods scales fractional, and concave separable functions in
poorly (and often exponentially) with the num- the continuous variables and binary variables,
ber of decision variables. While performance uses bound contraction, and applies the outer-
can be improved with the use of parallel com- approximation (OA) algorithm at each node of
puting, these methods are rarely applied to the tree. SMITH and PANTELIDES [218] proposed a
problems with more than a few dozen decision reformulation method combined with a spatial
variables. branch-and-bound algorithm for nonconvex
MINLP and NLP.
Because in global optimization one cannot
10.4. Global Optimization exploit optimality conditions like the KKT
conditions for a local optimum, these methods
Deterministic optimization methods are avail- work by first partitioning the problem domain
able for nonconvex nonlinear programming (i.e., containing the feasible region) into sub-
problems of the form Problem (52) that guar- regions (see Fig. 42). Upper bounds on the
antee convergence to the global optimum. objective function are computed over all sub-
More specifically, one can show under mild regions of the problem. In addition, lower
conditions that they converge to an e distance bounds can be derived from convex underesti-
to the global optimum on a finite number of mators of the objective function and constraints
steps. These methods are generally more for each subregion. The algorithm then pro-
expensive than local NLP methods, and they ceeds to eliminate all subregions that have
require the exploitation of the structure of the lower bounds that are greater than the least
nonlinear program. upper bound. After this, the remaining regions
Global optimization of nonconvex programs are further partitioned to create new subregions
has received increased attention due to their and the cycle continues until the upper and
practical importance. Most of the deterministic lower bounds converge. Below we illustrate
global optimization algorithms are based on the specific steps of the algorithm for nonlinear
spatial branch-and-bound algorithm [209], programs that involve bilinear, linear fractional,
which divides the feasible region of continuous and concave separable terms [210, 217].
116 Mathematics in Chemical Engineering

The feasible region of Problem (80) is denoted

by D. Note that a nonlinear equality constraint
of the form f k ðxÞ ¼ 0 can be accommodated in
problem 51 through the representation by the
inequalities f k ðxÞ  0 and f k ðxÞ  0, pro-
vided hk ðxÞ is separable.
To obtain a lower bound LBðVÞ for the
global minimum of Problem (80) over D \ V,
where V ¼ fx 2 Rn : xL  x  xU g  V0 , the
following problem is proposed:
X X X
Min ^f ðx; y; zÞ ¼ aij yij þ bij zij þ ^gi ðxi Þ þ hðxÞ
ðx;y;zÞ
ði;jÞ2BL0 ði;jÞ2LF 0 i2C 0

subject to
X X X
^f k ðx; y; zÞ ¼ aijk yij þ bijk zij þ ^gi;k ðxi Þ
ði;jÞ2BLk ði;jÞ2LF k i2C k

þ hk ðxÞ  0 k2K
ðx; y; zÞ 2 TðVÞ R  R  Rn2 n n1

Figure 42. Convex underestimator for nonconvex function x 2 S \ V Rn ; y 2 Rnþ1 ; z 2 Rnþ2 ;

ð81Þ

where the functions and sets are defined as

Nonconvex NLP with Bilinear, Linear Frac- follows:
tional, and Concave Separable Terms. Con-
sider the following specific nonconvex NLP a. g^i ðxi Þ and ^gi;k ðxi Þ are the convex envelopes
problem, for the univariate functions over the
domain xi 2 ½xLi ; xU i  [219]:
P P xi X
Minx f ðxÞ ¼ aij xi xj þ bij þ g ðxi Þ þ hðxÞ
ði;jÞ2BL0 ði;jÞ2LF 0
xj i2C0 i  

g ðxU Þ  gi ðxLi Þ
subject to ^gi ðxi Þ ¼ gi xLi þ i i U ðxi  xLi Þ  gi ðxi Þ
xi  xLi
P P xi ð80Þ
f k ðxÞ ¼ ði;jÞ2BLk aijk xi xj þ ði;jÞ2LF k bijk
xj
X
þ gi;k ðxi Þ þ hk ðxÞ  0 k2K  L 

gi;k ðxU
i Þ  gi;k ðxi Þ
i2C k ^
gi;k ðxi Þ ¼ gi;k xLi þ U L ðxi  xLi Þ  gi;k ðxi Þ
xi  xi
x 2 S \ V0 Rn ð81Þ

where aij, aijk, bij, bijk are scalars with i 2 I ¼ where ^gi ðxi Þ ¼ gi ðxi Þ at xi ¼ xLi , and
f1; 2;    ; ng, j 2 J ¼ f1; 2;    ; ng, and k 2 xi ¼ xUi ; likewise, ^ gi;k ðxi Þ ¼ gi;k ðxi Þ at
K ¼ f1; 2;    ; mg. BL0 ; BLk ; LF 0 ; LF k are xi ¼ xLi , and xi ¼ xUi .
(i, j)-index sets, with i 6¼ j, that define the b. y ¼ fyij g is a vector of additional variables
bilinear and linear fractional terms present in for relaxing the bilinear terms in Problem
the problem. The functions hðxÞ; hk ðxÞ are (80), and is used in the following inequali-
convex, and twice continuously differentiable. ties which determine the convex and con-
C0 and Ck are index sets for the univariate twice cave envelopes of bilinear terms:
continuously differentiable concave functions
gi ðxi Þ; gi;k ðxi Þ. The set S Rn is convex, and
v0Rn is an n-dimensional hyperrectangle yij  xLj xi þ xLi xj  xLi xLj ði; jÞ 2 BLþ
defined in terms of the initial variable bounds ð82Þ
yij  xU
j xi þ xU
i xj  xU U
i xj ði; jÞ 2 BLþ
xL,in and xU,in:
yij  xLj xi þ xU U L
i xj  xi x j ði; jÞ 2 BL
V0 ¼ fx 2 Rn : 0  xL;in  x  xU;in ; xL;in
j >0
ð83Þ
ifði; jÞ 2 LF 0 [ LF k ; i 2 I; j 2 J; k 2 Kg yij  xU L L U
j xi þ xi xj  xi x j ði; jÞ 2 BL
 Mathematics in Chemical Engineering 117

where where the overall upper bound (OUB) is the

value of f(x) at the best available feasible
BLþ ¼ fði; jÞ : ði; jÞ 2 BL0 [ BLk ; aij > 0 or aijk > 0; k 2 Kg point x 2 D; if no feasible point is available,
then OUB ¼ 1.
BL ¼ fði; jÞ : ði; jÞ 2 BL0 [ BLk ; aij < 0 or aijk < 0; k 2 Kg

Note that the underestimating Problem (82)

The Inequalities (82) were first derived by is a linear program if LF þ ¼ ? . During the
MCCORMICK [211], and along with the execution of the spatial branch-and-bound
Inequalities (83) theoretically character- algorithm, problem (NBNC) is solved initially
ized by AL-KHAYYAL and FALK [220, 221]. over MðV0 Þ (root node of the branch-and-
c. z ¼ fzij g is a vector of additional variables bound tree). If a better approximation is
for relaxing the linear fractional terms in required, MðV0 Þ is refined by partitioning V0
Problem (80); these variables are used in into two smaller hyperrectangles V01 and V02 ,
the following inequalities: and two children nodes are created with relaxed
feasible regions given by MðV01 Þ and MðV02 Þ.
!
xi 1 1 Problem (82) might be regarded as a basic
zij  þ xU  ði; jÞ 2 LF þ
xLj i
xj xLj underestimating program for the general
! ð84Þ Problem (81). In some cases, however, it is
xi 1 1
zij  U þ xLi  ði; jÞ 2 LF þ possible to develop additional convex
xj x j xU
j
estimators that might strengthen the underesti-
mating problem. Examples are the projections
pffiffiffiffiffiffiffiffiffiffi !2
1 xi þ xLi xU proposed by QUESADA and GROSSMANN [210],
zij  pffiffiffiffiffi p ffiffiffiffiffi
i
ði; jÞ 2 LF þ ð85Þ
xj xi þ xU
L
i
the reformulation–linearization technique by
SHERALI and ALAMEDDINE [223], and the
1 reformulation–convexification approach by
zij  ðxU L L L
j xi  xi xj þ x i xj Þ ði; jÞ 2 LF 
xLj xU
j
SHERALI and TUNCBILEK [224].
ð86Þ
1
zij  ðxLj xi  xU U U
i xj þ xi xj Þ ði; jÞ 2 LF  The Set of Branching Variables. A set of
xLj xU
j
branching variables, characterized by the index
set BVðVÞ defined below, is determined by
where considering the optimal solution ð^x; ^y; ^zÞV of
the underestimating problem:
LF þ ¼ fði; jÞ : ði; jÞ 2 LF 0 [ LF k ; bij > 0 or bijk > 0; k 2 Kg 8 9
>
> i; j : j^yij  ^xi ^xj j ¼ zl or j^zij  ^xi =^xj j ¼ zl >
>
LF  ¼ fði; jÞ : ði; jÞ 2 LF 0 [ LF k ; bij < 0 or bijk < 0; k 2 Kg >
> >
>
< =
BVðVÞ ¼ or g ð^xi Þ  ^g ð^xi Þ ¼ z or g ð^xi Þ  ^g ð^xi Þ ¼ z ;
>
> i i l i;k i;k l>
>
The Inequalities (84–86) are convex under- >
> >
>
: ;
for i 2 I; j 2 J; k 2 K; l 2 L
estimators due to QUESADA and GROSSMANN
ð88Þ
[210, 222] and ZAMORA and GROSSMANN
[217], respectively. where, for a prespecified number ln, L ¼
d. TðVÞ ¼ fðx; y; zÞ 2 Rn  Rn1  Rn2 : Prob- f1; 2; . . . ; ln g and z1 is the magnitude of the
lems (80–85) are satisfied with xL ; xU as in largest approximation error for a nonconvex
Vg. The feasible region, and the solution of term in Problem (80) evaluated at ð^x; ^y; ^zÞV :
Problem (52) are denoted by MðVÞ, and
ð^x; ^y; ^zÞV ,respectively.Wedefinetheapprox- j1 ¼ Max ½j^yij ^xi ^xj j; j^zij  ^xi =^xj j; gi ð^xi Þ  ^gi ð^xi Þ; gi;k ð^xi Þ  ^gi;k ð^xiÞ
imationgapeðVÞatabranch-and-boundnode i 2 I; j 2 J; k 2 K
as
Similarly, we define jl < jl1 with l 2 L f1g as
8 9
>
>
>
1 >
> the l-th largest magnitude for an approximation
> if OUB ¼ 1 > >
>
>
< LBðVÞ
>
>
= error; for instance, j2 < j1 is the second largest
eðVÞ ¼   if OUB ¼ 0 ð87Þ magnitude for an approximation error. Note
>
> >
>
>
> OUB  LBðVÞ >
>
>
>
: otherwise >
>
;
that in some cases it might be convenient to
jOUBj introduce weights in the determination of
118 Mathematics in Chemical Engineering

BVðVÞ in order to scale differences in the

approximation errors or to induce preferential
branching schemes. This might be particularly
useful in applications where specific informa-
tion can be exploited by imposing an order of
precedence on the set of complicating
variables.
This basic concept in spatial branch-and-
bound for global optimization is as follows.
Bounds are related to the calculation of upper
and lower bounds. For the former, any feasible
point or, preferably, a locally optimal point in
the subregion, can be used. For the lower
bound, convex relaxations of the objective
Figure 43. Global optimization example with partitions
and constraint function are derived such as in
Problem (82). The refining step deals with the
construction of partitions in the domain and
further partitioning them during the search 4. Update: Obtain upper bounds, OUBj1 and
process. Finally, the selection step decides on OUBj2 to new partitions if present. Set k ¼ k
the order of exploring the open subregions. þ 1, update partition sets and go to step 1.
Thus, the feasible region and the objective
5. Example: To illustrate the spatial branch-
function are replaced by convex envelopes to
and-bound algorithm, consider the global
form relaxed problems. Solving these convex
solution of:
relaxed problems leads to global solutions that
are lower bounds to the NLP in the particular
Min f ðxÞ ¼ 5=2 x4  20 x3 þ 55 x2  57 x
subregion. Finally, we see that gradient-based ð89Þ
NLP solvers play an important role in global s:t: 0:5  x  2:5
optimization algorithms, as they often yield the
lower and upper bounds for the subregions. The As seen in Figure 43, this problem has local
following spatial branch-and-bound global solutions at x ¼ 2.5 and at x ¼ 0.8749. The
optimization algorithm can therefore be given latter is also the global solution with f(x ) ¼
by the following steps: 19.7. To find the global solution we note that
all but the 20 x3 term in Problem (89) are
0. Initialize algorithm: Calculate upper convex, so we replace this term by a new
bound by obtaining a local solution to variable and a linear underestimator within a
Problem (80). Calculate a lower bound particular subregion, i.e.:
solving Problem (80) over the entire
(relaxed) feasible region V0 . Min f L ðxÞ ¼ 5=2 x4  20 w þ 55 x2  57 x

s:t: x1  x  x u
ð90Þ
For iteration k with a set of partitions Vk;j 3ðxu  xÞ ðx  xl Þ
and bounds in each subregion OLBj and OUBj: w ¼ ðxl Þ þ ðxu Þ3
ðxu  xl Þ ðxu  xl Þ

1. Bound: Define best upper bound: OUB ¼ In Figure 43 we also propose subregions that
Minj OUBj and delete (fathom) all subre- are created by simple bisection partitioning
gions j with lower bounds OLBj  OUB. If rules, and we use a “loose” bounding tolerance
OLBj  OUB  e, stop. of e ¼ 0:2. In each partition the lower bound, fL
2. Refine: Divide the remaining active subre- is determined by Problem (90) and the upper
gions into partitiions Vk;j1 and Vk;j2 . (Several bound fU is determined by the local solution of
branching rules are available for this step.) the original problem in the subregion. Figure 44
3. Select: Solve the convex NLP (80) in the shows the progress of the spatial branch-
new partitions to obtain OLBj1 and OLBj2. and-bound algorithm as the partitions are
Delete partition if no feasible solution. refined and the bounds are updated. In
 Mathematics in Chemical Engineering 119

Figure 44. Spatial branch-and-bound sequence for global optimization example

Figure 43, note the definitions of the partitions Unlike local optimization methods, there are no
for the nodes, and the sequence numbers in each optimality conditions, like the KKT conditions,
node that show the order in which the partitions that can be applied directly.
are processed. The grayed partitions correspond
to the deleted subregions and at termination of Mixed Integer Linear Programming. If the
the algorithm we see that fLj  fU  e (i.e., objective and constraint functions are all linear
19.85  19.7–0.2), with the gray subregions in Problem (52), and we assume 0–1 binary
in Figure 43 still active. Further partitioning in variables for the discrete variables, then this
these subregions will allow the lower and upper gives rise to a mixed integer linear program-
bounds to converge to a tighter tolerance. ming (MILP) problem given by Equation (91).
A number of improvements can be made to
the bounding, refinement, and selection strate- Min Z ¼ aT x þ cT y
gies in the algorithm that accelerate the con- s:t: Ax þ By  b ð91Þ
x  0; y 2 f0; 1gt
vergence of this method. A comprehensive
discussion of all of these options can be found
in [225–227]. Also, a number of efficient global As is well known, the (MILP) problem is NP-
optimization codes have recently been devel- hard. Nevertheless, an interesting theoretical
oped, including aBB, BARON, LGO, and result is that it is possible to transform it into
OQNLP. An interesting numerical comparison an LP with the convexification procedures pro-
of these and other codes can be found in [228]. posed by LOVACZ and SCHRIJVER [229], SHERALI
and ADAMS [230], and BALAS et al. [231]. These
procedures consist of sequentially lifting the
10.5. Mixed Integer Programming original relaxed xy space into higher dimen-
sion and projecting it back to the original space
Mixed integer programming deals with both so as to yield, after a finite number of steps, the
discrete and continuous decision variables. integer convex hull. Since the transformations
For simplicity in the presentation we consider have exponential complexity, they are only of
the most common case where the discrete deci- theoretical interest, although they can be used
sions are binary variables, i.e., yi ¼ 0 or 1, and as a basis for deriving cutting planes (e.g. lift
we consider the mixed integer Problem (52). and project method by [231]).
120 Mathematics in Chemical Engineering

As for the solution of problem (MILP), it

should be noted that this problem becomes an
LP problem when the binary variables are
relaxed as continuous variables 0  y  1.
The most common solution algorithms for
problem (MILP) are LP-based branch-and-
bound methods, which are enumeration meth-
ods that solve LP subproblems at each node of
the search tree. This technique was initially
conceived by LAND and DOIG [232], BALAS
[233], and later formalized by DAKIN, [234].
Figure 45. Branch and bound sequence for MILP example
Cutting-plane techniques, which were initially
proposed by GOMORY [235], and consist of
successively generating valid inequalities that
are added to the relaxed LP, have received The basic idea in the search is as follows.
renewed interest through the works of CROWDER The top, or root node, in the tree is the solution
et al. [236], VAN ROY and WOLSEY [237], and to the linear programming relaxation of
especially the lift-and-project method of BALAS Equation (91). If all the y variables take on
et al. [231]. A recent review of branch-and-cut 0–1 values, the MILP problem is solved, and no
methods can be found in [238]. Finally, Benders further search is required. If at least one of the
decomposition [239] is another technique for binary variables yields a fractional value, the
solving MILPs in which the problem is succes- solution of the LP relaxation yields a lower
sively decomposed into LP subproblems for bound to Problem (91). The search then consists
fixed 0–1 and a master problem for updating of branching on that node by fixing a particular
the binary variables. binary variable to 0 or 1, and the corresponding
restricted LP relaxations are solved that in turn
LP-Based Branch and Bound Method. We yield a lower bound for any of their descendant
briefly outline in this section the basic ideas nodes. In particular, the following properties are
behind the branch-and-bound method for solv- exploited in the branch-and-bound search:
ing MILP problems. Note that if we relax the t
binary variables by the inequalities 0  y  1  Any node (initial, intermediate, leaf node)
then Equation (91) becomes a linear program that leads to feasible LP solution corre-
with a (global) solution that is a lower bound to sponds to a valid upper bound to the
the MILP Problem (91). There are specific solution of the MILP Problem (91).
MILP classes in which the LP relaxation of  Any intermediate node with an infeasible
Equation (91) has the same solution as the
LP solution has infeasible leaf nodes and
MILP. Among these problems is the well-
can be fathomed (i.e., all remaining chil-
known assignment problem. Other MILPs
dren of this node can be eliminated).
that can be solved with efficient special-pur-
pose methods are the knapsack problem, the  If the LP solution at an intermediate node
set-covering and set-partitioning problems, and is not less than an existing integer solu-
the traveling salesman problem. See [240] for a tion, then the node can be fathomed.
detailed treatment of these problems.
The branch-and-bound algorithm for solving These properties lead to pruning of the nodes
MILP problems [234] is similar to the spatial in the search tree. Branching then continues in
branch-and-bound method of the previous sec- the tree until the upper and lower bounds
tion that explores the search space. As seen in converge.
Figure 45, binary variables are successively The basic concepts outlined above lead to a
fixed to define the search tree and a number branch-and-bound algorithm with the following
of bounding properties are exploited in order to features. LP solutions at intermediate nodes are
fathom nodes in to avoid exhaustive enumera- relatively easy to calculate since they can be
tion of all the nodes in the tree. effectively updated with the dual simplex
 Mathematics in Chemical Engineering 121

method. The selection of binary variables for OSL, XPRESS. Recent trends in MILP include
branching, known as the branching rule, is the development of branch-and-price [241]
based on a number of different possible criteria; and branch-and-cut methods such as the lift-
for instance, choosing the fractional variable and-project method [231] in which cutting
closest to 0.5, or the one involving the largest of planes are generated as part of the branch-
the smallest pseudocosts for each fractional and-bound enumeration. See also [238] for a
variable. Branching strategies to navigate the recent review on MILP. A description of sev-
tree take a number of forms. More common eral MILP solvers can also be found in the
depth-first strategies expand the most recent NEOS Software Guide.
node to a leaf node or infeasible node and
then backtrack to other branches in the tree. Mixed Integer Nonlinear Programming The
These strategies are simple to program and most basic form of an MINLP problem when
require little storage of past nodes. On the other represented in algebraic form is as follows:
hand, breadth-first strategies expand all the
min z ¼ f ðx; yÞ
nodes at each level of the tree, select the s:t: gj ðx; yÞ  0 j 2 J ðP1Þ ð93Þ
node with the lowest objective function, and x 2 X; y 2 Y
then proceed until the leaf nodes are reached.
Here, more storage is required, but generally where f(), g() are convex, differentiable func-
fewer nodes are evaluated than in depth-first tions, J is the index set of inequalities, and x and
search. In practice, a combination of both y are the continuous and discrete variables,
strategies is commonly used: branch on the respectively. The set X is commonly assumed
dichotomy 0–1 at each node (i.e., like to be a convex compact set, e.g.,
breadth-first), but expand as in depth-first. X ¼ fxjx 2 Rn ; Dx  d; xL  x  xU g; the
Additional description of these strategies can discrete set Y corresponds to a polyhedral set
be found in [240]. of integer points, Y ¼ fyjx 2 Z m ; Ay  ag,
Example: To illustrate the branch-and-bound which in most applications is restricted to
approach, we consider the MILP: 0–1 values, y 2 {0,1}m. In most applications
of interest the objective and constraint func-
Min Z ¼ x þ y1 þ 2y2 þ 3y3 tions f(), g() are linear in y (e.g., fixed cost
s:t:  x þ 3y1 þ y2 þ 2y3  0 charges and mixed-logic constraints):
ð92Þ
 4y1  8y2  3y3  10
x  0; y1 ; y2 ; y3 ¼ f0; 1g
f ðx; yÞ ¼ cT y þ rðxÞ; gðx; yÞ ¼ By þ hðxÞ.
Methods that have addressed the solution of
Problem (93) include the branch-and-bound
The solution to Problem (92) is given by x ¼ 4, method (BB) [242–246], generalized Benders
y1 ¼ 1, y2 ¼ 1, y3 ¼ 0, and Z ¼ 7. Here we use a decomposition (GBD) [247], outer-approxima-
depth-first strategy and branch on the variables tion (OA) [248–250], LP/NLP based branch-
closest to zero or one. Figure 45 shows the and-bound [251], and extended cutting plane
progress of the branch-and-bound algorithm as method (ECP) [252].
the binary variables are selected and the bounds There are three basic NLP subproblems that
are updated. The sequence numbers for each can be considered for Problem (93):
node in Figure 45 show the order in which they
are processed. The grayed partitions correspond a. NLP relaxation
to the deleted nodes and at termination of the
algorithm we see that Z ¼ 7 and an integer
Min Z kLB ¼ f ðx; yÞ
solution is obtained at an intermediate node s:t: gj ðx:yÞ  0 j 2 J
where coincidentally y3 ¼ 0. x 2 X; y 2 Y R ðNLP1Þ ð94Þ
Mixed integer linear programming (MILP) yi  aki i 2 I kFL
yi  bki i 2 I kFU
methods and codes have been available and
applied to many practical problems for more
than twenty years (e.g., [240]. The LP-based where YR is the continuous relaxation of the
branch-and-bound method [234] has been set Y, and I kFL ; I kFU are index subsets of
implemented in powerful codes such as the integer variables yi, i 2 I which are
122 Mathematics in Chemical Engineering

restricted to lower and upper bounds aki ; bki K previous steps:

at the k-th step of a branch-and-bound
enumeration procedure. Note that Min Z KL ¼ a

i e; l < k; m < k, where

aki ¼ byli c; bki ¼ dym 2 3 9
x  xk >
>
yli ; ym
i are noninteger values at a previous st k T6
a  f ðx ; y Þ þ rf ðx ; y Þ 4
k k k 7
5
>
>
>
>
>
>
step, and bc, de are the floor and ceiling y  yk >
= k ¼ 1; . . . K
functions, respectively. 2 3 >
x  xk >
>
Also note that if I kFU ¼ I kFL ¼ ? ðk ¼ 0Þ, >
>
k k T6 7 k>
gj ðx ; y Þ þ rgj ðx ; y Þ 4
k k
5  0j 2 J >
>
>
Problem (94) corresponds to the continu- ;
y  yk
ous NLP relaxation of Problem (93).
x 2 X; y 2 Y ð97Þ
Except for few and special cases, the solu-
tion to this problem yields in general a
noninteger vector for the discrete variables. where Jk  J. When only a subset of lineariza-
Problem (94) also corresponds to the k-th tions is included, these commonly correspond
step in a branch-and-bound search. The to violated constraints in Problem (93). Alter-
optimal objective function Z oLB provides an natively, it is possible to include all lineariza-
absolute lower bound to Problem (93); for tions in Problem (97). The solution of Problem
m  k, the bound is only valid for (97) yields a valid lower bound Z kL to Problem
(93). This bound is nondecreasing with the
FL ; I FU I FL .
I kFL I m k m
number of linearization points K. Note that
b. NLP subproblem for fixed yk: since the functions f(x, y) and g(x, y) are convex,
the linearizations in problem correspond to outer
Min Z kU ¼ f ðx; yk Þ approximations of the nonlinear feasible region
s:t: gj ðx; yk Þ  0 j 2 J ð95Þ
x2X
in Problem (93). A geometrical interpretation is
shown in Figure 46, where it can be seen that the
convex objective function is being underesti-
which yields an upper bound Z kU to Prob- mated, and the convex feasible region overesti-
lem (96), provided Problem (95) has a mated with these linearizations.
feasible solution. When this is not the
case, we consider the next subproblem: Algorithms. The different methods can be
c. Feasibility subproblem for fixed yk: classified according to their use of the Subpro-
blems (94–96), and the specific specialization
Min u
of the MILP problem , as seen in Figure 47. In
s:t: gj ðx; yk Þ  u j 2 J ð96Þ the GBD and OA methods (case b), as well in
x 2 X; u 2 R1 the LP/NLP-based branch-and-bound mehod
(case d), Problem (96) is solved if infeasible
subproblems are found. Each of the methods is
which can be interpreted as the minimiza- explained next in terms of the basic
tion of the infinity-norm measure of subproblems.
infeasibility of the corresponding NLP sub-
problem. Note that for an infeasible sub- Branch and Bound. While the earlier work in
problem the solution of Problem (96) branch and bound (BB) was aimed at linear
yields a strictly positive value of the scalar problems [234] this method can also be applied
variable u. to nonlinear problems [242–246]. The BB
method starts by solving first the continuous
The convexity of the nonlinear functions is NLP relaxation. If all discrete variables take
exploited by replacing them with supporting integer values the search is stopped. Otherwise,
hyperplanes, which are generally, but not nec- a tree search is performed in the space of the
essarily, derived at the solution of the NLP integer variables yi, i 2I. These are successively
subproblems. In particular, the new values fixed at the corresponding nodes of the tree,
yK (or (xK, yK) are obtained from a cutting- giving rise to relaxed NLP subproblems of the
plane MILP problem that is based on the K form (NLP1) which yield lower bounds for
points (xk, yk), k ¼ 1 . . . K, generated at the the subproblems in the descendant nodes.
 Mathematics in Chemical Engineering 123

Fathoming of nodes occurs when the lower

bound exceeds the current upper bound,
when the subproblem is infeasible, or when
all integer variables yi take on discrete values.
The last-named condition yields an upper
bound to the original problem.
The BB method is generally only attractive
if the NLP subproblems are relatively
inexpensive to solve, or when only few of
them need to be solved. This could be either
because of the low dimensionality of the dis-
crete variables, or because the integrality gap
of the continuous NLP relaxation of Problem
(93) is small.
Outer Approximation [248, 249, 248]. The
OA method arises when NLP Subproblems
(95) and MILP master problems 97 with
Jk ¼ J are solved successively in a cycle of
iterations to generate the points (xk, yk). Since
the master problem requires the solution of all
feasible discrete variables yk, the following
MILP relaxation is considered, assuming that
the solution of K different NLP subproblems
(K ¼ |KFS [ KIS|), where KFS is a set of
solutions from Problem (95), and KIS set of
solutions from Problem (96) is available:
Min Z KL ¼ a
2 3 9
x  xk >
6 7 >
>
Figure 46. Geometrical interpretation of linearizations in k T6 7 >
>
st a  f ðxk ; yk Þ þ rf ðxk ; y Þ 4 5 >
>
>
>
master problem >
>
y  yk >
>
=k ¼ 1; . . . K
2 3 >
>
x  xk >
>
>
>
6 7 >
gj ðxk ; yk Þ þ rgj ðxk ; yk ÞT 6 7  0 j 2 J>
>
>
>
4 5 >
>
k
;
yy

x 2 X; y 2 Y ð98Þ
Given the assumption on convexity of the func-
tions f(x, y) and g(x, y), it can be proved that the
solution of Problem (98) Z kL corresponds to a
lower bound of the solution of Problem (93).
Note that this property can be verified in
Figure 46. Also, since function linearizations
are accumulated as iterations proceed, the mas-
ter Problems (98) yield a nondecreasing
sequence of lower bounds Z 1L . . .  Z kL . . . Z kL
since linearizations are accumulated as itera-
tions k proceed.
The OA algorithm as proposed by DURAN
and GROSSMANN consists of performing a cycle
of major iterations, k ¼ 1,..K, in which Problem
Figure 47. Major steps in the different algorithms (95) is solved for the corresponding yK, and the
124 Mathematics in Chemical Engineering

relaxed MILP master Problem (98) is updated fjjgj ðxk ; yk Þ ¼ 0g and the set x 2 X is disre-
and solved with the corresponding function garded. In particular, consider an outer-approx-
linearizations at the point (xk, yk) for which imation given at a given point (xk, yk)
the corresponding subproblem NLP2 is solved.
If feasible, the solution to that problem is used 2 3
x  xk
to construct the first MILP master problem; k T67
a  f ðx ; y Þ þ rf ðx ; y Þ 4
k k k
5
otherwise a feasibility Problem (96) is solved y  yk
to generate the corresponding continuous point 2 3 ð100Þ
x  xk
[250]. The initial MILP master Problem (98) k k T6
gðx ; y Þ þ rg ðx ; y Þ 4
k k 7
50
then generates a new vector of discrete varia- y  yk
bles. The Subproblems (95) yield an upper
bound that is used to define the best current
solution, UBk ¼ mink fZ kU g. The cycle of itera- where for a fixed yk the point xk corresponds to
tions is continued until this upper bound and the the optimal solution to Problem (95). Making
lower bound of the relaxed master problem Z kL use of the Karush–Kuhn–Tucker conditions and
are within a specified tolerance. One way to eliminating the continuous variables x, the
avoid solving the feasibility Problem (96) in the inequalities in Problem (100) can be reduced
OA algorithm when the discrete variables in as follows [251]:
Problem (93) are 0–1, is to introduce the fol-
lowing integer cut whose objective is to make a  f ðxk ; yk Þ þ ry f ðxk ; yk ÞT ðy  yk Þ þ ðmk ÞT ½gðxk ; yk Þ
infeasible the choice of the previous 0–1 values þ ry gðxk ; yk ÞT ðy  yk Þ ð101Þ
generated at the K previous iterations [248]:
X X
yi  yi  jBk j  1 k ¼ 1; . . . K ð99Þ which is the Lagrangian cut projected in the y-
k
i2B i2N k
space. This can be interpreted as a surrogate
constraint of the equations in Problem (100),
where Bk¼fijyki ¼ 1g; N k ¼ fijyki ¼ 0g; k ¼ because it is obtained as a linear combination
1; . . . K. This cut becomes very weak as the of these.
dimensionality of the 0–1 variables increases. For the case when there is no feasible solu-
However, it has the useful feature of ensuring tion to Problem (95), then if the point xk is
that new 0–1 values are generated at each obtained from the feasibility subproblem
major iteration. In this way the algorithm (NLPF), the following feasibility cut projected
will not return to a previous integer point in y can be obtained using a similar procedure.
when convergence is achieved. Using the
above integer cut the termination takes place ðlk ÞT ½gðxk ; yk Þ þ ry gðxk ; yk ÞT ðy  yk Þ  0 ð102Þ
as soon as Z KL  UBK .
The OA algorithm trivially converges in
In this way, Problem (97) reduces to a problem
one iteration if f(x, y) and g(x, y) are linear.
projected in the y-space:
This property simply follows from the fact that
if f(x, y) and g(x, y) are linear in x and y the Min Z KL ¼ a
MILP master problem is identical to the origi-
st a  f ðxk ; yk Þ þ ry f ðxk ; yk ÞT ðy  yk Þ þ ðmk ÞT
nal Problem (93). It is also important to note
that the MILP master problem need not be ½gðxk ; yk Þ þ ry gðxk ; yk ÞT ðy  yk Þ k 2 KFS

solved to optimality.
ðlk ÞT ½gðxk ; yk Þ þ ry gðxk ; yk ÞT ðy  yk Þ  0 k 2 KIS

Generalized Benders Decomposition (GBD) x 2 X; a 2 R1

[247]. The GBD method [253] is similar to
the outer-approximation method. The differ- where KFS is the set of feasible Subproblems
ence arises in the definition of the MILP (95), and KIS the set of infeasible subproblems
master Problem (97). In the GBD method whose solution is given by Problem (96). Also
only active inequalities are considered J k ¼ |KFS [ KIS| ¼ K. Since master Problem (103)
 Mathematics in Chemical Engineering 125

can be derived from master Problem (98), in the Convergence is achieved when the maximum
context of Problem (93), GBD can be regarded constraint violation lies within the specified
as a particular case of the outer-approximation tolerance. The optimal objective value of
algorithm. In fact one can prove that given the Problem (97) yields a nondecreasing sequence
same set of K subproblems, the lower bound of lower bounds. It is of course also possible to
predicted by the relaxed master problem is either add to problem linearizatons of all the
greater than or equal to that predicted by the violated constraints in the set Jk, or lineariza-
relaxed master Problem (103) [248]. This proof tions of all the nonlinear constraints j 2 J. In
follows from the fact that the Lagrangian and the ECP method the objective must be defined
feasibility cuts (Eqs. 101 and 102) are surro- as a linear function, which can easily be
gates of the outer-approximations (Problems accomplished by introducing a new variable
103). Given the fact that the lower bounds of to transfer nonlinearities in the objective as an
GBD are generally weaker, this method com- inequality.
monly requires a larger number of cycles or Note that since the discrete and continuous
major iterations. As the number of 0–1 variables variables are converged simultaneously, the
increases this difference becomes more pro- ECP method may require a large number of
nounced. This is to be expected since only one iterations. However, this method shares with the
new cut is generated per iteration. Therefore, OA method Property 2 for the limiting case
user-supplied constraints must often be added when all the functions are linear.
to the master problem to strengthen the bounds.
Also, it is sometimes possible to generate LP/NLP-Based Branch and Bound [251]. This
multiple cuts from the solution of an NLP method is similar in spirit to a branch-and-cut
subproblem in order to strengthen the lower method, and avoids the complete solution of the
bound [254]. As for the OA algorithm, the MILP master problem (M-OA) at each major
trade-off is that while it generally predicts iteration. The method starts by solving an initial
stronger lower bounds than GBD, the computa- NLP subproblem, which is linearized as in (M-
tional cost for solving the master problem (M- OA). The basic idea consists then of performing
OA) is greater, since the number of constraints an LP-based branch-and-bound method for (M-
added per iteration is equal to the number of OA) in which NLP Subproblems (95) are
nonlinear constraints plus the nonlinear solved at those nodes in which feasible integer
objective. solutions are found. By updating the represen-
If Problem (93) has zero integrality gap, the tation of the master problem in the current open
GBD algorithm converges in one iteration once nodes of the tree with the addition of the
the optimal (x , y ) is found [255]. This prop- corresponding linearizations, the need to restart
erty implies that the only case in which one can the tree search is avoided.
expect the GBD method to terminate in This method can also be applied to the
one iteration is that in which the initial discrete GBD and ECP methods. The LP/NLP method
vector is the optimum, and when the objective commonly reduces quite significantly the
value of the NLP relaxation of Problem (93) is number of nodes to be enumerated. The
the same as the objective of the optimal mixed- trade-off, however, is that the number of
integer solution. NLP subproblems may increase. Computa-
tional experience has indicated that often
Extended Cutting Plane (ECP) [252]. The
the number of NLP subproblems remains
ECP method, which is an extension of Kelley’s
unchanged. Therefore, this method is better
cutting-plane algorithm for convex NLP [256],
suited for problems in which the bottleneck
does not rely on the use of NLP subproblems
corresponds to the solution of the MILP mas-
and algorithms. It relies only on the iterative
ter problem. LEYFFER [257] has reported sub-
solution of problem by successively adding a
stantial savings with this method.
linearization of the most violated constraint at
Example: Consider the following MINLP
the predicted point (xk, yk):
problem, whose objective function and con-
J k ¼ f^j 2 argfmax gj ðxk ; yk Þgg straints contain nonlinear convex terms
j2J
126 Mathematics in Chemical Engineering

quadratic approximations can help to reduce

Min Z ¼ y1 þ 1:5y2 þ 0:5y3 þ x21 þ x22
the number of major iterations, since an
s:t: ðx1  2Þ2  x2  0 improved representation of the continuous
space is obtained. This, however, comes at
x1  2y1  0
the price of having to solve an MIQP instead
of an MILP at each iteration.
x1  x2  4ð1  y2 Þ  0
The master problem can involve a rather
x2  y2  0
large number of constraints, due to the accu-
ð103Þ
mulation of linearizations. One option is to
x1 þ x2  3y3 keep only the last linearization point, but this
can lead to nonconvergence even in convex
y1 þ y2 þ y 3  1 problems, since then the monotonic increase
of the lower bound is not guaranteed. As shown
0  x1  4; 0  x2  4 [251], linear approximations to the nonlinear
objective and constraints can be aggregated
y1 ; y2 ; y3 ¼ 0; 1
with an MILP master problem that is a hybrid
of the GBD and OA methods.
The optimal solution of this problem is given by For the case when linear equalities of the
y1 ¼ 0, y2 ¼ 1, y3 ¼ 0, x1 ¼ 1, x2 ¼ 1, Z ¼ 3.5. form h(x, y) ¼ 0 are added to Problem (93) there
Figure 48 shows the progress of the iterations is no major difficulty, since these are invariant
with the OA and GBD methods, while the table to the linearization points. If the equations are
lists the number of NLP and MILP subproblems nonlinear, however, there are two difficulties.
that are solved with each of the methods. For First, it is not possible to enforce the linearized
the case of the MILP problems the total number equalities at K points. Second, the nonlinear
of LPs solved at the branch-and-bound nodes equations may generally introduce nonconvex-
are also reported. ities, unless they relax as convex inequalities
Extensions of MINLP Methods. Extensions of [258]. KOCIS and GROSSMANN [259] proposed an
the methods described above include a qua- equality relaxation strategy in which the non-
dratic approximation to (RM-OAF) [250] using linear equalities are replaced by the inequalities
an approximation of the Hessian matrix. The
" #
x  xk
T k rhðxk ; yk ÞT 0 ð104Þ
y  yk

where T k ¼ ftkii g, and tkii ¼ signðlki Þ sign in

which lki is the multiplier associated to the
equation hi(x, y) ¼ 0. Note that if these equa-
tions relax as the inequalities h(x, y)  0 for all y
and h(x, y) is convex, this is a rigorous proce-
dure. Otherwise, nonvalid supports may be
generated. Also, in the master Problem (103)
of GBD, no special provision is required to
handle equations, since these are simply
included in the Lagrangian cuts. However, sim-
ilar difficulties as in OA arise if the equations
do not relax as convex inequalities.
When f(x, y) and g(x, y) are nonconvex in
problem 93, or when nonlinear equalities
h(x, y) ¼ 0 are present, two difficulties arise.
Figure 48. Progress of iterations with OA and GBD meth-
First, the NLP Subproblems (94–96) may not
ods, and number of subproblems for the BB, OA, GBD, and have a unique local optimum solution. Second,
ECP methods. the master problem (M-MIP) and its variants
 Mathematics in Chemical Engineering 127

(e.g., M-MIPF, M-GBD, M-MIQP) do not Since the tree searches are not finite (except for
guarantee a valid lower bound Z KL or a valid e convergence), these methods can be compu-
bounding representation with which the global tationally expensive. However, their major
optimum may be cut off. advantage is that they can rigorously find the
Rigorous global optimization approaches for global optimum. Specific cases of nonconvex
addressing nonconvexities in MINLP problems MINLP problems have been handled. An exam-
can be developed when special structures are ple is the work of PO€ RN and WESTERLUND [264],
assumed in the continuous terms (e.g. bilinear, who addressed the solution of MINLP prob-
linear fractional, concave separable). Specifi- lems with pseudoconvex objective function and
cally, the idea is to use convex envelopes or convex inequalities through an extension of the
underestimators to formulate lower-bounding ECP method.
convex MINLP problems. These are then com- The other option for handling nonconvex-
bined with global optimization techniques for ities is to apply a heuristic strategy to try to
continuous variables [209, 210, 212, 217, 219, reduce as much as possible the effect of non-
225, 260], which usually take the form of convexities. While not being rigorous, this
spatial branch-and-bound methods. The lower requires much less computational effort. We
bounding MINLP problem has the general describe here an approach for reducing the
form, effect of nonconvexities at the level of the
MILP master problem.
Min Z ¼ f ðx; yÞ VISWANATHAN and GROSSMANN [265] proposed
s:t: g j ðx; yÞ  0 j 2 J ð105Þ
x 2 X; y 2 Y
to introduce slacks in the MILP master problem
to reduce the likelihood of cutting off feasible
solutions. This master problem (augmented pen-
where f ; g
 , are valid convex underestimators alty/equality relaxation) has the form:
such that f ðx; yÞ  f ðx; yÞ and the inequalities X
K

g ðx; yÞ  0 are satisfied if gðx; yÞ  0. A typi- min Z K ¼ a þ wkp pk þ wkq qk
cal example of convex underestimators are the k¼1 2 39
x  xk>
>
convex envelopes for bilinear terms [211]. 6 7>
s:t: a  f ðxk ; yk Þ þ rf ðxk ; yk ÞT 4 5>
>
>
>
>
Examples of global optimization methods yy >
k >
>
>
>
>
for MINLP problems include the branch-and- 2 3 >
>
>
x  xk >
=
reduce method [212, 213], the a-BB method k T6
T k rhðxk ; y Þ 4
7
k ¼ 1; . . . K
5  pk
[215], the reformulation/spatial branch-and- >
>
yy k >
>
>
>
bound search method [261], the branch-and- 2 3 >
>
>
>
x  xk >
>
cut method [262], and the disjunctive branch- k T6 7 >
>
>
gðx ; y Þ þ rgðx ; y Þ 4
k k k
5  qk >
>
and-bound method [263]. All these methods >
;
k
yy
rely on a branch-and-bound procedure. The P P
i2Bk yi  i2N k yi  jBk j  1 k ¼ 1; . . . K
difference lies in how to perform the branching
x 2 X; y 2 Y; a 2 R1 ; pk ; qk  0
on the discrete and continuous variables. Some ð106Þ
methods perform the spatial tree enumeration
on both the discrete and continuous variables of where wkp ; wkq
are weights that are chosen
Problem (105). Other methods perform a spa- sufficiently large (e.g., 1000 times the magni-
tial branch and bound on the continuous vari- tude of the Lagrange multiplier). Note that if the
ables and solve the corresponding MINLP functions are convex then the MILP master
Problem (105) at each node using any of the Problem (106) predicts rigorous lower bounds
methods reviewed above. Finally, other meth- to Problem (93) since all the slacks are set to
ods branch on the discrete variables of Problem zero.
(105), and switch to a spatial branch and bound
on nodes where a feasible value for the discrete Computer Codes for MINLP. Computer codes
variables is found. The methods also rely on for solving MINLP problems include the fol-
procedures for tightening the lower and upper lowing. The program DICOPT [265] is an
bounds of the variables, since these have a great MINLP solver that is available in the modeling
effect on the quality of the underestimators. system GAMS [266]. The code is based on the
128 Mathematics in Chemical Engineering

master Problem (106) and the NLP Subpro- review of logic-based optimization can be found
blems (95). This code also uses relaxed Sub- in [272, 273].
problem (94) to generate the first linearization Generalized disjunctive programming in
for the above master problem, with which the Problem (107) [269] is an extension of dis-
user need not specify an initial integer value. junctive programming [274] that provides an
Also, since bounding properties of Problem alternative way of modeling MILP and MINLP
(106) cannot be guaranteed, the search for problems. The general formulation of Problem
nonconvex problems is terminated when there (107) is as follows:
is no further improvement in the feasible NLP P
Min Z ¼ k2K ck þ f ðxÞ
subproblems. This is a heuristic that works
s:t: gðxÞ  0 ð107Þ
reasonably well in many problems. Codes 2 3
Y jk
that implement the branch-and-bound method _
j2J k 6 7
6 hjk ðxÞ  0 7; k 2 K
using Subproblems (94) include the code 4 5
ck ¼ g jk
MINLP_BB, which is based on an SQP algo-
rithm [246] and is available in AMPL, the code VðYÞ ¼ True
BARON [267], which also implements global
x 2 Rn ; c 2 Rm ; Y 2 ftrue; falsegm
optimization capabilities, and the code SBB,
which is available in GAMS [266]. The code a-
ECP implements the extended cutting-plane where Yjk are the Boolean variables that decide
method [252], including the extension by PO€ RN whether a term j in a disjunction k 2 K is true or
and WESTERLUND [264]. Finally, the code MIN- false, and x are continuous variables. The objec-
OPT [268] also implements the OA and GBD tive function involves the term f(x) for the
methods, and applies them to mixed-integer continuous variables and the charges ck that
dynamic optimization problems. It is difficult depend on the discrete choices in each dis-
to derive general conclusions on the efficiency junction k 2 K. The constraints g(x)  0 hold
and reliability of all these codes and their regardless of the discrete choice, and hjk(x)  0
corresponding methods, since no systematic are conditional constraints that hold when Yjk is
comparison has been made. However, one true in the j-th term of the k-th disjunction.
might anticipate that branch-and-bound codes The cost variables ck correspond to the fixed
are likely to perform better if the relaxation of charges, and are equal to gjk if the Boolean
the MINLP is tight. Decomposition methods variable Yjk is true. V(Y) are logical relations for
based on OA are likely to perform better if the the Boolean variables expressed as proposi-
NLP subproblems are relatively expensive to tional logic.
solve, while GBD can perform with some effi- Problem (107) can be reformulated as an
ciency if the MINLP is tight and there are many MINLP problem by replacing the Boolean var-
discrete variables. ECP methods tend to per- iables by binary variables yjk,
form well on mostly linear problems. P P
Min Z ¼ k2K j2J k g jk yjk þ f ðxÞ

s:t: gðxÞ  0
Logic-Based Optimization. Given difficulties
in the modeling and solution of mixed integer hjk ðxÞ  M jk ð1  yjk Þ; j 2 J k ; k 2 K ðBMÞ
P ð108Þ
problems, the following major approaches based j2J k yjk ¼ 1; k 2 K
on logic-based techniques have emerged: gener-
Ay  a
alized disjunctive programming (Problem 107)
[269], mixed-logic linear programming (MLLP) 0  x  xU ; yjk 2 f0; 1g; j 2 J k ; k 2 K
[270], and constraint programming (CP) [271].
The motivations for this logic-based modeling where the disjunctions are replaced by “Big-M”
has been to facilitate the modeling, reduce the constraints which involve a parameter Mjk and
combinatorial search effort, and improve binary variables yjk. The propositional logic
the handling of nonlinearities. In this section statements V(Y) ¼ True are replaced by the
we mostly concentrate on generalized dis- linear constraints Ay  A [275] and [276]. Here
junctive programming and provide a brief refer- we assume that x is a nonnegative variable with
ence to constraint programming. A general finite upper bound xU. An important issue in
 Mathematics in Chemical Engineering 129

P P
Problem (108) is how to specify a valid value Min Z L ¼ k2K j2J k g jk ljk þ f ðxÞ
for the Big-M parameter Mjk. If the value is too s:t: gðxÞ  0
small, then feasible points may be cut off. If Mjk P P
x¼ vjk ; j2J k ljk ¼ 1; k 2 K ðCRPÞ
is too large, then the continuous relaxation j2J k

might be too loose and yield poor lower bounds. 0  x; vjk  xU ; 0  ljk  1; j 2 J k ; k 2 K ð111Þ
Therefore, finding the smallest valid value for ljk hjk ðv =ljk Þ  0; j 2 J k ; k 2 K
jk

Mjk is desired. For linear constraints, one can use

Al  a
the upper and lower bound of the variable x to
calculate the maximum value of each constraint, 0  x; v  x ; 0  ljk  1; j 2 J k ; k 2 K
jk U

which then can be used to calculate a valid value

of Mjk. For nonlinear constraints one can in where xU is a valid upper bound for x and v. Note
principle maximize each constraint over the that the number of constraints and variables
feasible region, which is a nontrivial calculation. increases in Problem (111) compared with Prob-
LEE and GROSSMANN [277] have derived the lem (101). Problem (111) has a unique optimal
convex hull relaxation of Problem (107). The solution and it yields a valid lower bound to the
basic idea is as follows. Consider a disjunction optimal solution of Problem (107) [277]. GROSS-
k 2 K that has convex constraints MANN and LEE [280] proved that Problem (111)
2 3 has the useful property that the lower bound is
Y jk
6 h ðxÞ  0 7 greater than or equal to the lower bound pre-
_j2J k 4 jk 5
ð109Þ
dicted from the relaxation of Problem (131).
c ¼ g jk
Further description of algorithms for dis-
0  x  xU ; c  0 junctive programming can be found in [281].

where hjk(x) are assumed to be convex and Constraint Programming. Constraint pro-
bounded over x. The convex hull relaxation of gramming (CP) [271, 272] is a relatively new
disjunction (Problem 109) [245] is given as modeling and solution paradigm that was orig-
follows: inally developed to solve feasibility problems,
but it has been extended to solve optimization
P P problems as well. Constraint programming is
x¼ j2J k vjk ; c¼ j2J ljk g jk
very expressive, as continuous, integer, and
jk ; Boolean variables are permitted and, moreover,
jk
0v  ljk xU j 2 Jk
P variables can be indexed by other variables.
j2J k ljk ¼ 1; 0  ljk  1; j 2 J k ðCHÞ ð110Þ
Constraints can be expressed in algebraic form
ljk hjk ðvjk =ljk Þ  0; j 2 J k (e.g., h(x)  0), as disjunctions (e.g., [A1x  b1]
_ A2x  b2]), or as conditional logic statements
x; c; vjk  0; j 2 J k
(e.g., If g(x)  0 then r(x)  0). In addition, the
language can support special implicit functions
where vjk are disaggregated variables that are such as the all-different (x1, x2, . . . xn) con-
assigned to each term of the disjunction k 2 K, straint for assigning different values to the
and ljk are the weight factors that determine the integer variables x1, x2, . . . xn. The language
feasibility of the disjunctive term. Note that consists of Cþþ procedures, although the
when ljk is 1, then the j-th term in the k-th recent trend has been to provide higher level
disjunction is enforced and the other terms are languages such as OPL. Other commercial CP
ignored. The constraints ljk hjk ðvjk =ljk Þ are con- software packages include ILOG Solver [282],
vex if hjk(x) is convex [278, p. 160]. A formal CHIP [283], and ECLiPSe [284].
proof can be found in [245]. Note that the
convex hull (Eqs. 110) reduces to the result
by BALAS [279] if the constraints are linear. 10.6. Dynamic Optimization
Based on the convex hull relaxation Equations
(110), LEE and GROSSMANN [277] proposed Interest in dynamic simulation and optimiza-
the following convex relaxation program of tion of chemical processes has increased
Problem (107). significantly since the 1990s. Chemical
130 Mathematics in Chemical Engineering

processes are modeled dynamically using those that discretize the state and control pro-
differential-algebraic equations (DAEs), con- files (full discretization). Basically, the partially
sisting of differential equations that describe discretized problem can be solved either by
the dynamic behavior of the system, such as dynamic programming or by applying a non-
mass and energy balances, and algebraic equa- linear programming (NLP) strategy (direct-
tions that ensure physical and thermodynamic sequential). A basic characteristic of these
relations. Typical applications include control methods is that a feasible solution of the
and scheduling of batch processes; startup, DAE system, for given control values, is
upset, shut-down, and transient analysis; obtained by integration at every iteration of
safety studies; and the evaluation of control the NLP solver. The main advantage of these
schemes. We state a general differential- approaches is that, for the NLP solver, they
algebraic optimization Problem (112) as generate smaller discrete problems than full
follows: discretization methods.
Methods that fully discretize the continuous
  time problem also apply NLP strategies to solve
Min F zðtf Þ; yðtf Þ; uðtf Þ; tf ; p
the discrete system and are known as direct-
 
s:t: F dz=dt; zðtÞ; uðtÞ; t; p ¼ 0; zð0Þ ¼ z0 simultaneous methods. These methods can use
 different NLP and discretization techniques,
Gs ½zðts Þ; yðts Þ; uðts Þ; ts ; p  ¼ 0 but the basic characteristic is that they solve
zL  zðtÞ  xU ð112Þ
the DAE system only once, at the optimum. In
addition, they have better stability properties
yL  yðtÞ  yU than partial discretization methods, especially
uL  uðtÞ  yU in the presence of unstable dynamic modes. On
pL  p  pU the other hand, the discretized optimization
problem is larger and requires large-scale
ttf  tf  tU
f
NLP solvers, such as SOCS, CONOPT, or
IPOPT.
where F is a scalar objective function at final With this classification we take into account
time tf, and F are DAE constraints, Gs addi- the degree of discretization used by the differ-
tional point conditions at times ts, z(t) differ- ent methods. Below we briefly present the
ential state profile vectors, y(t) algebraic state description of the variational methods, followed
profile vectors, u(t) control state profile vec- by methods that partially discretize the
tors, and p is a time-independent parameter dynamic optimization problem, and finally
vector. we consider full discretization methods for
We assume, without loss of generality, that Problem (112).
the index of the DAE system is one, consistent
initial conditions are available, and the objec- Variational Methods. These methods are
tive function is in the above Mayer form. based on the solution of the first-order neces-
Otherwise, it is easy to reformulate problems sary conditions for optimality that are obtained
to this form. Problem (112) can be solved either from Pontryagin’s maximum principle [285,
by the variational approach or by applying 286]. If we consider a version of Problem
some level of discretization that converts the (112) without bounds, the optimality conditions
original continuous time problem into a dis- are formulated as a set of DAEs:
crete problem. Early solution strategies, known
@Fðz; y; u; p; tÞ 0 @H @Fðz; y; u; p; tÞ
as indirect methods, were focused on solving l ¼ ¼ l (113a)
@z0 @z @z
the classical variational conditions for optimal-
ity. On the other hand, methods that discretize
Fðz; y; u; p; tÞ ¼ 0 (113b)
the original continuous time formulation can be
divided into two categories, according to the
Gf ðz; y; u; p; tf Þ ¼ 0 (113c)
level of discretization. Here we distinguish
between the methods that discretize only the
Gs ðz; y; u; p; ts Þ ¼ 0 (113d)
control profiles (partial discretization) and
 Mathematics in Chemical Engineering 131

@H @Fðz; y; u; p; tÞ
¼ l¼0 (113e) control parametrization n. The sequential
@y @y
method is reliable when the system contains
only stable modes. If this is not the case, finding
@H @Fðz; y; u; p; tÞ
@u
¼
@u
l¼0 (113f) a feasible solution for a given set of control
parameters can be very difficult. The time
Ztf
horizon is divided into time stages and at
@Fðz; y; u; p; tÞ each stage the control variables are represented
l dt ¼ 0 (113g)
@p
0 with a piecewise constant, a piecewise linear, or
a polynomial approximation [287, 288]. A
where the Hamiltonian H is a scalar function of common practice is to represent the controls
the form H(t) ¼ F(z, y, u, p, y)Tl(t) and l(t) is a as a set of Lagrange interpolation polynomials.
vector of adjoint variables. Boundary and jump For the NLP solver, gradients of the objec-
conditions for the adjoint variables are given tive and constraint functions with respect to the
by: control parameters can be calculated with the
sensitivity equations of the DAE system, given
@F @F @Gf by:
lð t f Þ þ þ vf ¼ 0
@z0 @z @z
ð114Þ
@F
 @Gs @F
þ @F T @F @F @F T @zð0Þ
l ts þ vs ¼ l t sk 0 þ T sk þ T wk þ ¼ 0; sk ð0Þ ¼ k ¼ 1; . . . N q
@z0 @z @z0 s @z0 @z @y @qk @qk
ð115Þ

where vf, vs are the multipliers associated with

the final time and point constraints, respec- where sk ðtÞ ¼ @zðtÞ @yðtÞ
@qk , wk ðt Þ ¼ @qk , and q
T
¼
tively. The most expensive step lies in obtaining [p , n ]. As can be inferred from
T T

a solution to this boundary value problem. Equation (115), the cost of obtaining these sensi-
Normally, the state variables are given as initial tivities is directly proportional to Nq, the number
conditions, and the adjoint variables as final of decision variables in the NLP. Alternately,
conditions. This formulation leads to boundary gradients can be obtained by integration of the
value problems (BVPs) that can be solved by a adjoint Equations (113a, 113e, 113g) [282, 285,
number of standard methods including single 286] at a cost independent of the number of input
shooting, invariant embedding, multiple shoot- variables and proportional to the number of con-
ing, or some discretization method such as straints in the NLP.
collocation on finite elements or finite differ- Methods that are based on this approach
ences. Also the point conditions lead to an cannot treat directly the bounds on state vari-
additional calculation loop to determine the ables, because the state variables are not
multipliers vf and vs. On the other hand, included in the nonlinear programming prob-
when bound constraints are considered, the lem. Instead, most of the techniques for dealing
above conditions are augmented with addi- with inequality path constraints rely on defining
tional multipliers and associated complemen- a measure of the constraint violation over the
tarity conditions. Solving the resulting system entire horizon, and then penalizing it in the
leads to a combinatorial problem that is pro- objective function, or forcing it directly to zero
hibitively expensive except for small problems. through an end-point constraint [291]. Other
techniques approximate the constraint satisfac-
Partial Discretization. With partial discretiza- tion (constraint aggregation methods) by intro-
tion methods (also called sequential methods or ducing an exact penalty function [290, 292] or a
control vector parametrization), only the con- Kreisselmeier–Steinhauser function [292] into
trol variables are discretized. Given the initial the problem.
conditions and a given set of control parame- Finally, initial value solvers that handle path
ters, the DAE system is solved with a differen- constraints directly have been developed [288].
tial algebraic equation solver at each iteration. The main idea is to use an algorithm for con-
This produces the value of the objective func- strained dynamic simulation, so that any admis-
tion, which is used by a nonlinear programming sible combination of the control parameters
solver to find the optimal parameters in the produces an initial value problem that is
132 Mathematics in Chemical Engineering

feasible with respect to the path constraints. techniques to enforce feasibility, like the ones
The algorithm proceeds by detecting activation used in the sequential methods.
and deactivation of the constraints during the The resulting NLP is solved using SQP-type
solution, and solving the resulting high-index methods, as described above. At each SQP iter-
DAE system and their related sensitivities. ation, the DAEs are integrated in each stage and
objective and constraint gradients with respect to
Full Discretization. Full discretization meth- p, zi, and ni are obtained using sensitivity equa-
ods explicitly discretize all the variables of the tions, as in Problem (115). Compared to sequen-
DAE system and generate a large-scale non- tial methods, the NLP contains many more
linear programming problem that is usually variables, but efficient decompositions have
solved with a successive quadratic program- been proposed [294] and many of these calcula-
ming (SQP) algorithm. These methods follow a tions can be performed in parallel.
simultaneous approach (or infeasible path In collocation methods, the continuous time
approach); that is, the DAE system is not solved problem is transformed into an NLP by approx-
at each iteration; it is only solved at the opti- imating the profiles as a family of polynomials
mum point. Because of the size of the problem, on finite elements. Various polynomial repre-
special decomposition strategies are used to sentations are used in the literature, including
solve the NLP efficiently. Despite this charac- Lagrange interpolation polynomials for the
teristic, the simultaneous approach has advan- differential and algebraic profiles [295]. In
tages for problems with state variable (or path) [194] a Hermite–Simpson collocation form is
constraints and for systems where instabilities used, while CUTHRELL and BIEGLER [296] and
occur for a range of inputs. In addition, the TANARTKIT and BIEGLER [297] use a monomial
simultaneous approach can avoid intermediate basis for the differential profiles. All of these
solutions that may not exist, are difficult to representations stem from implicit Runge–
obtain, or require excessive computational Kutta formulae, and the monomial repre-
effort. There are mainly two different appro- sentation is recommended because of smaller
aches to discretize the state variables explicitly, condition numbers and smaller rounding
multiple shooting [293, 294] and collocation on errors. Control and algebraic profiles, on the
finite elements [184, 194, 295]. other hand, are approximated using Lagrange
With multiple shooting, time is discretized polynomials.
into P stages and control variables are parame- Discretizations of Problem (112) using col-
trized using a finite set of control parameters in location formulations lead to the largest NLP
each stage, as with partial discretization. The problems, but these can be solved efficiently
DAE system is solved on each stage, i ¼ using large-scale NLP solvers such as IPOPT
1, . . . P and the values of the state variables and by exploiting the structure of the colloca-
z(ti) are chosen as additional unknowns. In this tion equations. BIEGLER et al. [184] provide a
way a set of relaxed, decoupled initial value review of dynamic optimization methods using
problems (IVP) is obtained: simultaneous methods. These methods offer a
number of advantages for challenging dynamic
 
F dz=dt; zðtÞ; yðtÞ; ni ; p ¼ 0; optimization problems, which include:
t 2 ½ti1 ; ti ; zðti1 Þ ¼ zi ð116Þ  Control variables can be discretized at the
ziþ1  zðti ; zi ; ni ; pÞ ¼ 0; i ¼ 1; . . . P  1 same level of accuracy as the differential
and algebraic state variables. The KKT
Note that continuity among stages is treated conditions of the discretized problem can
through equality constraints, so that the final be shown to be consistent with the varia-
solution satisfies the DAE system. With this tional conditions of Problem (116). Finite
approach, inequality constraints for states and elements allow for discontinuities in con-
controls can be imposed directly at the grid trol profiles.
points, but path constraints for the states may  Collocation formulations allow problems
not be satisfied between grid points. This prob- with unstable modes to be handled in an
lem can be avoided by applying penalty efficient and well-conditioned manner.
 Mathematics in Chemical Engineering 133

The NLP formulation inherits stability platforms are essential for the formulation task.
properties of boundary value solvers. These are classified into two broad areas: opti-
Moreover, an elementwise decomposition mization modeling platforms and simulation
has been developed that pins down platforms with optimization.
unstable modes in Problem (112).
 Collocation formulations have been pro- Optimization modeling platforms provide gen-
posed with moving finite elements. This eral purpose interfaces for optimization algo-
allows the placement of elements both rithms and remove the need for the user to
for accurate breakpoint locations of con- interface to the solver directly. These plat-
trol profiles as well as accurate DAE forms allow the general formulation for all
solutions. problem classes discussed above with direct
interfaces to state of the art optimization
Dynamic optimization by collocation meth- codes. Three representative platforms are
ods has been used for a wide variety of process GAMS (General Algebraic Modeling Sys-
applications including batch process optimiza- tems), AMPL (A Mathematical Programming
tion, batch distillation, crystallization, dynamic Language), and AIMMS (Advanced Integrated
data reconciliation and parameter estimation, Multidimensional Modeling Software). All
nonlinear model predictive control, polymer three require problem-model input via a declar-
grade transitions and process changeovers, ative modeling language and provide exact
and reactor design and synthesis. A review of gradient and Hessian information through auto-
this approach can be found in [298]. matic differentiation strategies. Although pos-
sible, these platforms were not designed to
handle externally added procedural models.
10.7. Development of Optimization As a result, these platforms are best applied
Models on optimization models that can be developed
entirely within their modeling framework.
The most important aspect of a successful Nevertheless, these platforms are widely used
optimization study is the formulation of the for large-scale research and industrial applica-
optimization model. These models must reflect tions. In addition, the MATLAB platform
the real-world problem so that meaningful allows the flexible formulation of optimization
optimization results are obtained, and they models as well, although it currently has only
also must satisfy the properties of the problem limited capabilities for automatic differentia-
class. For instance, NLPs addressed by gradi- tion and limited optimization solvers. More
ent-based methods require functions that are information on these and other modeling
defined in the variable domain and have platforms can be found on the NEOS server
bounded and continuous first and second deriv- www-neos.mcs.anl.gov.
atives. In mixed integer problems, proper for-
mulations are also needed to yield good lower Simulation platforms with optimization are
bounds for efficient search. With increased often dedicated, application-specific modeling
understanding of optimization methods and tools to which optimization solvers have been
the development of efficient and reliable opti- interfaced. These lead to very useful optimiza-
mization codes, optimization practitioners now tion studies, but because they were not origi-
focus on the formulation of optimization nally designed for optimization models, they
models that are realistic, well-posed, and need to be used with some caution. In particu-
inexpensive to solve. Finally, convergence lar, most of these platforms do not provide exact
properties of NLP, MILP, and MINLP solvers derivatives to the optimization solver; often
require accurate first (and often second) deriv- they are approximated through finite differ-
atives from the optimization model. If these ence. In addition, the models themselves are
contain numerical errors (say, through finite constructed and calculated through numerical
difference approximations) then performance procedures, instead of through an open declar-
of these solvers can deteriorate considerably. ative language. Examples of these include
As a result of these characteristics, modeling widely used process simulators such as
134 Mathematics in Chemical Engineering

Aspen/Plus, PRO/II, and Hysys. More recent 10.8.1. Motivation

platforms such as Aspen Custom Modeler and
gPROMS include declarative models and exact Optimization problems in chemical engineering
derivatives. often aim at minimizing some cost functions
For optimization tools linked to procedural while maximizing suitable quality measures.
models, reliable and efficient automatic differ- Optimizing just one of these will usually not
entiation tools are available that link to models provide satisfactory solutions. There are trade-
written, say, in FORTRAN and C, and calculate offs between the different objectives that have to
exact first (and often second) derivatives. be taken into account before a decision is made.
Examples of these include ADIFOR, ADOL- Multicriteria optimization aims at providing
C, GRESS, Odyssee, and PADRE. When used tools for dealing with this situation and to sup-
with care, these can be applied to existing port the decision-making process.
procedural models and, when linked to modern
NLP and MINLP algorithms, can lead to 10.8.2. Concepts and Definitions
powerful optimization capabilities. More infor-
mation on these and other automatic differenti- Basics. An optimization problem with several
ation tools can be found on: http://www-unix. objectives is given by the generalization of Prob-
mcs.anl.gov/autodiff/AD_Tools/. lem (51) to more than one objective functions:
Finally, the availability of automatic differ-
entiation and related sensitivity tools for differ- minðf 1 ðxÞ; f 2 ðxÞ; . . . ; f k ðxÞÞ s:t: x 2 X

ential equation models allows for considerable

flexibility in the formulation of optimization This kind of problem has usually not one opti-
models. In [299] a seven-level modeling hier- mal solution, since in general there is no solu-
archy is proposed that matches optimization tion which is optimal with respect to all
algorithms with models that range from com- objectices at once. Instead, we look at the
pletely open (fully declarative model) to fully concept of non-dominance: We say a solution
closed (entirely procedural without sensitivi- x dominates a solution y if each f i ðxÞ  f i ðyÞ for
ties). At the lowest, fully procedural level, all components i and f j ðxÞ < f j ðyÞ for at least
only derivative-free optimization methods are one component j. The interesting solutions are
applied, while the highest, declarative level those which are not dominated by any other
allows the application of an efficient large-scale solution. These are the nondominated, Pareto
solver that uses first and second derivatives. optimal or efficient solutions. The set of such
Depending on the modeling level, optimization solutions is called Pareto set, nondominated set,
solver performance can vary by several orders or efficient frontier (Fig. 49). One can consider
of magnitude. these solutions as best compromises: Given a
Pareto optimal solution, it is not possible to
improve one objective without worsening
another or leaving the feasible set X.
10.8. Multicriteria Optimization
Advanced. There is also the notion of weakly
Karl-Heinz K€
ufer, Michael Bortz Pareto optimal solutions. A solution x is weakly

x2 f2
f
X f (X) Solutions dominated
x by f (x)
f(x)

x1 f1

Figure 49. The Pareto set

 Mathematics in Chemical Engineering 135

Pareto optimal if there is no other solution y The convexity property, which is eligible for
such that f i ðxÞ < f i ðyÞ for all components i. optimization problems with one objective, is
Usually, weakly Pareto optimal solutions, also desirable for multicriteria problems. We
which are not Pareto optimal, are undesired, call a multicriteria optimization problem con-
because they are dominated, but some algo- vex if the feasible set X is convex and the
rithms for obtaining Pareto optimal solutions objective functions fi are convex. In that case
only guarantee weak Pareto optimality. But the Pareto set plus R kþ is convex. Otherwise, it
there might also be undesired Pareto optimal needs not to be convex and not even connected.
solutions. Assume two solutions x and y where x The concept of Pareto optimal solutions fits
is significantly worse than y with respect to fj the practical needs often better than breaking all
but at most only slightly better with respect to aspects down to one objective and several
any other objective, the deterioration may be constraints. Solving such a multicriteria opti-
not worth the improvement. So, the tradeoff mization problem is, however, much more
between these two components may be consid- involved than the solution of a single objective
ered undesired. Using a more general definition optimization problem: For an optimization
of dominance and Pareto optimality using so problem with a single objective, it is clear
called “ordering cones” one can exclude those that the problem is solved if an optimal solution
solutions. An equivalent formulation of “x is found. Finding all Pareto optimal solutions
dominates y” is the following: for a multicriteria optimization problem is in
general not possible nor usually necessary. The
f ðyÞ  f ðxÞ 2 R kþ nf0g aim is to support the person or the group of
persons who have eventually to select a solution
The set R kþ :¼ fz 2 R k : z  0g is the usual by making the decision. That person or that
“ordering cone” for defining Pareto opti- group is referred to as decision-maker in the
mality. A point x is Pareto optimal if following. Therefore, two issues remain: 1)
f ðxÞ  R kþ \ f ðxÞ ¼ u. By using a larger order- How to calculate Pareto optimal solutions;
ing cone such as: and 2) how to support the selection of a final
solution, that is, to make a decision.
R ke :¼ fz 2 R k : distðz; R kþ Þ  e k z kg

we define a point x to be e-proper Pareto 10.8.3. Scalarization

optimal if f ðxÞ  R ke \ f ðxÞ ¼ u. These points
are still Pareto optimal but have bounds on A common way to deal with several objectives
the trade-offs leading to a smaller Pareto set is to define a new real-value objective function
(Fig. 50). using one, several or all original objective
functions and obtaining a standard optimization
problem. The new optimization problem is
parametrized and by changing the parameters,
f2
different Pareto optimal solutions can be
obtained.
This reformulation is known as
Solutions dominated “scalarization”. Its most relevant properties are:
by f (x) w.r.t. ℝk∊
 Are all obtained solution Pareto optimal or
just weakly Pareto optimal?
∊-proper
Pareto optimal
 Can all Pareto optimal solutions be
solutions obtained by an adequate adjustment of
the parameters?
Weakly Pareto optimal solutions
f1 The most straighforward approach is to take
the weighted sum of the objectives as new
Figure 50. e-Proper and weakly Pareto optimal points objective:
136 Mathematics in Chemical Engineering

f2 f2

Weight vector
Pareto points Pareto point
obtained by 1-norm
Pareto point
obtained by weighted
Tangent planes Euclidean norm
zref

f1 f1

Figure 51. Scalarization by a weighted sum function

according to Equation (117) for two objectives Figure 53. Scalarization based on norms
The weight vector is normal to the hyperplane, which The function ggoal from Equation (118) and its generaliza-
touches the Pareto set in a Pareto point tion gweighted norm from Equation (119) are sketched for two
objectives

X
gweighted sum ðxÞ :¼ wi f i ðxÞ (117)
for given barrieres ej with ji. Solving this
with nonnegative weights wi, see Figure 51. The problem for any i and any ej one obtains a
solution of weakly Pareto optimal solution and if the solu-
tion is unique it is Pareto optimal. Furthermore,
min gweighted sum ðxÞ s:t: x 2 X all Pareto optimal solutions can be obtained this
way.
is weakly Pareto optimal and Pareto optimal if
Another reformulation is based on norms
all weights are positive or if the solution is
(Fig. 53). Let zref 2k be a reference point, that
unique. If the original multicriteria optimiza-
is, a point in the objective space which is
tion problem is convex, then any Pareto optimal
desirable to be obtained but not necessarily
point is a solution of the weighted sum problem
possible to obtain. For any norm jj  jj we can
for appropriately chosen weights. However,
define a new objective function ggoal ðxÞ :¼
otherwise this need not be the case.
jjf ðxÞ  zref jj and solve:
Another approach is the e-constraint
method, see Figure 52. Here the new optimiza- min ggoal ðxÞ s:t: x 2 X: (118)
tion problem has the following shape

min f i ðxÞ s:t: f j ðxÞ  ej for j 6¼ i and x 2 X

Each component of the reference point can be
thought of as a goal or target value to be
achieved. The “achievement function” ggoal
f2 minimizes deviations from the goals. This
approach is known as goal programming.
Apart from the vector zref we can introduce
a weight vector w 2 R kþ and if w  z :¼
ðw1 z1 ; . . . ; wk zk Þ is the element-wise multipli-
Pareto point
cation of two vectors, we can use the objective
ϵ2
gweighted norm ðxÞ :¼ jjw  ðf ðxÞ  zref Þjj: (119)

Assuming all weights arepffiffiffiffiffiffiffiffiffiffiffiffiffi

positive
P ffi and the norm
is the Lp -norm jjzjj ¼ p jzi jp for p  1, all
f1
obtained solutions are Pareto optimal. If
Figure 52. Scalarization by the e-constraint method for the reference point has the property that zref;i <
two objectives minx2X f i ðxÞ and the norm is the maximum
 Mathematics in Chemical Engineering 137

f2 weighting method with evenly distributed

weights. Yet, this does not necessarily lead
to an even distribution of the Pareto points as
shown in [300]. In many pratical cases it is not
r
clear what good parameters are neither for the
a decision-maker nor for an educated guess.
Pareto point However, there are schemes for approximating
ℝ k+ the Pareto set which determine the next set of
parameters based on the Pareto optimal solu-
tions obtained so far.
f1
10.8.4. Approximation Schemes
Figure 54. Sketch of the Pascoletti-Serafini scalarization
for two objectives according to Equation 120 A rough estimate of the range of the Pareto set
is obtained by the ideal and the nadir points as
norm jjzjj ¼ maxjzi j for each Pareto optimal x follows: One is the ideal point zideal :¼
point there are weights w 2 R kþ such that x is a ðminx2X f 1 ðxÞ; . . . ; minx2X f k ðxÞÞ which states
solution of the corresponding parametrized the best value which can be reached in each
problem. component. Its counterpart is the nadir point
A very generic formulation is the so-called znadir which contains the worst value which can
Pascoletti–Serafini scalarization, see Figure 54. be obtained among the Pareto optimal solu-
For a reference point a 2 R k and a reference tions. It is much harder to calculate. An approx-
direction 2 R k , the following problem is to be imation is to calculate the optima x1 ; . . . ; xk of
solved: the problems minf i ðxÞ s:t: x 2 X and take
min t s:t: ai þ tri  f i ðxÞ  0 and x 2 X: (120) zapprox nadir :¼ ðmaxi f 1 ðxi Þ; . . . ; maxi f k ðxi ÞÞ.
Note that the approximation in the bicriteria
The weighted sum and the e-constrained case, that is, k ¼ 2, is usually easier. In this
method can be seen as a special case of this case, the above approximation of the Nadir
formulation. point is always exact and there are specially
Hence, there are several approaches to tackle crafted algorithms for the approximation of the
such a problem. Methods which do not interact Pareto set, e.g., [301].
with the decision-maker and just present a Now the more general case k  2 is con-
single Pareto optimal solution are called no- sidered. For convex multicriteria optimization
preference method. In the literature, one finds problems, see e.g., [4] we can use the fact that
three types of methods requiring preference there is a tangential hyperplane at each Pareto
information from the decision-maker: a priori, optimal point f ðxÞ such that all other Pareto
a posteriori, and interactive methods. The first optimal points are on the same halfspace
requires information such as, for instance, the H :¼ fz 2 R k : hw; f ðxÞi  hw; zig. If a set of
reference point or the weights before the actual Pareto optimal points is already calculated,
calculation of one or more Pareto optimal the intersection of all corresponding halfspa-
solutions, the second calculates Pareto optimal ces is an approximation of the Pareto set
solutions and then requires precedence infor- containing all Pareto optimal points f ðxÞ.
mation while the third iteratively presents solu- The convex combination of the Pareto optimal
tions and adjusts the parameters based on the points is an approximation from the interior.
user input to obtain further solutions. This is By determining the place where the distance
often based on the assumption that the decision- between both approximation is largest, one can
maker has an implicit utility function which is determine a weight for obtaining the next Pareto
approximated while presenting him solutions optimal solution improving both approxima-
and evaluating his rating of them. tions. Applications of this procedure to chemical
The challenge is to determine appropriate process engineering can be found in [302].
parameters for the above methods. In the If the multicriteria optimization problem can
convex case on may think of using the be decomposed into a convex part and discrete
138 Mathematics in Chemical Engineering

enumeration variable with few values attained, solutions. If this is combined with an intuitive
we can approximate the convex part for each navigation mechanism it gives the decision-
value of the enumeration variable and end up maker the feeling of having the complete Pareto
with a set of approximations. set at hand, even though only some point were
If the multicriteria optimization problem is actually calculated.
simply nonconvex, the Pareto optimal solu-
tions f ðxÞ can be approximated by hyperboxes. 10.9. Optimization under Uncertainty
This approach has the disadvantage that it
becomes very slow if the number of objectives Karl-Heinz K€ufer, Michael Bortz
increases.
10.9.1. Introduction
10.8.5. Other Approaches
In practical applications, models are affected by
A popular approach to obtain a set of Pareto
uncertainties of their parameters. These result
optimal solutions are population-based meta-
from the incomplete knowledge of model
heuristics. Here a set of solutions is modified
parameters, stemming from the model that is
iteratively according to some rules such that
only an approximation of the reality. Thus,
it gets closer to the set of Pareto optimal
model parameters (such as parameters in
solutions. Their main advantage is that they
reaction kinetics or equilibria) are often esti-
do not rely on differentiabity or continuity
mated from experiments. In addition, there are
assumptions, hence the objective evaluation
uncertainties in process control, affecting, for
can be a black box. One of the most well-
example, flow rates or feed compositions, but
known metaheuristics is the evaluationary
also in varying commodity or energy prices and
algorithm variant NSGA-II. In each iteration
in changing environmental conditions such as
the population is modified and extended and
atmospheric temperature and pressure. Since
afterwards a subset of solutions is selected as
these factors often randomly fluctuate, one
population of the next generation, respectively,
cannot simply fix them by one value but has
iteration. Possible criteria for the selection
to consider their “whole behavior”.
are non-dominance and the diversity or spread
The impact of uncertainties on optimal solu-
of the solutions to approximate the complete
tions is of special interest. Not only is it impor-
Pareto set.
tant to know how changes in the input affect the
It is also possible to use methods from
optimal settings, it is also desirable to compute
differential geometry to obtain Pareto optimal
settings that are robust to changes in the input
solutions: In the homotopy approach [303] the
(in this case, a local optimum can be better than
Pareto set is considered as a manifold which is
a global one, Fig. 55) or to compute settings that
iteratively sampled.
take into account the randomness of the input.
10.8.6. Decision Support The first aspect can be studied by using sensi-
tivity analysis, for the second, which is the
Having obtained a set of Pareto optimal solu- actual optimization under uncertainty, tech-
tions, it is still the question what to do with it. niques of robust and stochastic optimization
While it is sometimes enough to just calculate can be used. These tools are discussed in
one or more Pareto optimal solutions, often the more detail in the following.
decision-maker has to make a choice given a set In analogy to the general NLP Problem (52)
of solutions. And it is a challenge to support the we consider a parametric optimization problem:
decision in such a way that the decision-maker PðjÞ : minn f ðx; jÞs:t :gðx; jÞ  0;
x2R
can make use of all information available and (121)
not just select some solution after spending hðx; jÞ ¼ 0:

some time for consideration. How to do it

depends very much on the specific problem where x 2 R n is the vector of optimization
and needs. Sometimes it is also possible to parameters and j 2 R m is the vector of uncertain
interpolate two or more Pareto optimal parameters.
 Mathematics in Chemical Engineering 139

Figure 55. Robustness vs. global optimality

10.9.2. Sensitivity Analysis Sensitivity analysis can be used for optimal

experimental design, for example, by deter-
Sensitivity analysis is concerned with how mining initial conditions, measurement posi-
(much) the variation of the input variables a tions, and sampling time, and to generate
of a model or system z ¼ f ðaÞ affects the output informative data critical to estimation accu-
variables z. The most important applications racy. In simulation, it can be used to identify
are: redundant species and reactions allowing
model reduction.
 Obtaining a better understanding of the The choice of a sensitivity analysis method
relationships between the input and the typically depends on the problem settings:
output variables in complex or black-box-  Computational expense: Sensitivity anal-
models
ysis is almost always performed by run-
 Identification of input variables which
ning the model a (possibly large) number
determine a significant variability of the of times. This can be sticky when a single
model output run of the model takes a significant
 Fixing model inputs that have no or only amount of time (minutes, hours, or longer)
little effect on the output, or identifying or the model has a large number of inputs.
and removing redundant parts of the The latter is problematic because sensitiv-
model structure (model simplification) ity analysis tries to explore the multi-
In the context of optimization, sensitivity dimensional input space, which grows
analysis is particularly useful for: exponentially in size with the number of
inputs.
 Testing the robustness of optimization  Correlated inputs: Most common sensi-
results in presence of uncertainty tivity analysis methods assume indepen-
 Identifying regions in the space of input dence between model inputs, but
variables for which the model output is sometimes inputs can be strongly
good (preoptimization) correlated.
 Nonlinearity: Some sensitivity analysis
In process engineering, for example, kinetic approaches can merely inaccurately mea-
parameters are frequently determined from sure sensitivity when the model response
experimental data via nonlinear regression: is nonlinear with respect to its inputs.
140 Mathematics in Chemical Engineering

 Interactions: Interactions occur when the Screening methods are sampling-based meth-
perturbation of two or more inputs simul- ods. Their objective is to identify input variables
taneously causes variation in the output which are contributing significantly to the output
greater than that of varying each of the variability in high-dimensional models, rather
inputs alone. than exactly quantifying sensitivity. They tend to
 Given data: While in many cases the prac- have a relatively low computational cost com-
titioner has access to the model, in some pared to other approaches, and can be used in a
instances a sensitivity analysis must be preliminary analysis to weed out uninfluential
performed with “given data”, i.e. where variables before applying a more informative
the sample points (the values of the model analysis to the remaining set. One of the most
inputs for each run) cannot be chosen by the commonly used screening methods is the ele-
analyst. mentary effect method, where averaged finite
differences are calculated in order to estimate
There are a large number of approaches to the effect of uncertain model parameters.
perform a sensitivity analysis, many of which Variance-based methods are a class of prob-
have been developed to address one or more of abilistic approaches that quantify the input and
the constraints discussed above: output uncertainties as probability distributions
One of the simplest and most common and decompose the output variance into parts
approaches is that of changing one-factor-at- attributable to input variables and combinations
a-time (OFAT or OAT), to see what effect this of them. The sensitivity of the output to an input
produces on the output?. Thus, any change variable is therefore measured by the amount of
observed in the output will unambiguously be variance in the output caused by that input.
due to the single variable changed. The fixation Further approaches are emulators (also
of all other variables to their central or baseline known as meta- or surrogate models or response
values increases the comparability of the results surfaces) and Fourier amplitude sensitivity test
(all “effects“ are computed with reference to the (FAST).
same central point in space). However, this For a detailed introduction into sensitivity
approach does not fully explore the input space, analysis and its application to chemical engi-
since it does not take into account the simulta- neering amongst others see [304] and [305].
neous variation of input variables. Consequently, Many of the above mentioned methods, but
the OAT approach cannot detect the presence of also others are available in form of software.
interactions between input variables. Some references are:
Local methods involve taking the partial
derivative of the output with respect to an input  SimLab: a free software for global sensi-
factor. Similar to OAT/OFAT, they do not tivity analysis [306]
attempt to fully explore the input space, since  Sensitivity Analysis Excel Add-In: for sim-
they examine only small perturbations, typi- ple sample based sensitivity analysis runs
cally one variable at a time. in Excel [307]
Another simple but useful approach consists  MUCM Project: an extensive resource for
in scatter plots: There the output variables are uncertainty and sensitivity analysis of
plotted against individual input variables, after computationally-demanding models [308]
(randomly) sampling the model over its input  SALib: a sensitivity analysis library in
distributions. This approach can also deal with Python (Numpy), [309]
“given data” and gives a direct visual indication
of sensitivity.
Regression analysis, in the context of sensi- 10.9.3. Robust and Stochastic Optimization
tivity analysis, involves fitting a linear function
to the model response and using standardized Depending on the circumstances, there are
regression coefficients as measures of sensitiv- different approaches to treat optimization prob-
ity, which is only suitable when the model is lems depending on uncertain parameters:
linear. The advantages of regression analysis  It is only known which values (discrete or
are its simplicity and low computational cost. continuous) a parameter can have.
 Mathematics in Chemical Engineering 141

 Moreover, distribution information about Summarizing, the robust formulation or

the parameter values is available. robust counterpart of a parametric optimization
problem PðjÞ is given by
In both cases, the simplest approach is to
replace the parameters by an average value PðjÞrobust : min t s:t: f ðx; jÞ  t  0 for all j 2 X;
ðx;tÞ2R n R
resp. their mean values. However, this approach
gðx; jÞ  0 for all j 2 X; (124)
guarantees the inequality system to be satisfied
on average only and a decision x leading to a hðx; jÞ ¼ 0 for all j 2 X:
failure of a system for about half of the real-
izations of j will be usually considered as
Fortunately, the robust counterpart of a linear
unacceptable.
optimization problem is still a linear, but larger
Robust Optimization. If a constraint function, optimization problem [312]. However, in gen-
e.g. gðx; jÞ, depends on some uncertain param- eral, the robust reformulation of a parametric
eter vector j 2 X  R m and no distribution optimization problem results in a greater and/or
information about the parameter vector values more difficult to solve optimization problem.
is available, the “most cautious” and common Besides, this reformulation can destroy useful
way to deal with this constraint is to use its problem properties such as convexity. General
worst-case reformulation: references to robust optimization are [312–314].
The following software is available for solv-
gðx; jÞ  0 for all j 2 X (122) ing certain classes of robust optimization
problems:
If X is finite, Equation (122) represents finitely
many, possibly more complicated, ordinary  AIMMS Robust Programming Software:
constraints. If X is a set with infinite cardinality, Solvers for robust linear and robust
Equation (122) formally constitutes a (ordi- mixed-integer programming problems
nary) semi-infinite constraint [310]. If X also [315]
depends on x, Equation (122) is formally a  YALMIP: A modelling language for
general semi-infinite constraint [311]. In cer- advanced modeling and solution of con-
tain cases, for example, if g is bilinear and X vex and nonconvex optimization prob-
polyhedral or elliptical, the semi-infinite con- lems; implemented as free toolbox for
straint, Equation (122), can be transferred into a Matlab; supports a wide range of robust
usual finite one. programming problems [316]
Similarly, if the objective function f ðx; jÞ  ROME: An algebraic modeling toolbox
depends on the vector j, in the worst case designed to solve a class of robust optimi-
one has to minimize the maximal objective zation problems in Matlab [317]
value:
min max f ðx; jÞ (123)
Stochastic Optimization. If, in addition to the
x2R n j2X knowledge of the possible parameter values,
distribution information is available, the
Through the epigraph reformulation, Equation uncertain parameters can be modeled as
123 can be rewritten into random variables. Optimizations problems of
this kind are called stochastic optimization/pro-
min t s:t: max f ðx; jÞ  t
ðx;tÞ2R n R j2X gramming problems. General references to sto-
chastic programming are [318–321] and the
or, equivalently, stochastic programming community homepage
http://stoprog.org/. For the application of
min t s:t: f ðx; jÞ  t for all j 2 X
ðx;tÞ2R n R stochastic programming to design and opera-
tion of chemical process systems see, for
Thus, in the worst-case approach, parametric example, [322–323], and the references
objective functions can be brought into the therein. Since equality constraints are harder
same form as parametric constraints. to treat in this framework, we restrict ourselves
142 Mathematics in Chemical Engineering

in this section to inequality constraint functions This formulation can be extended to a multi-
gðx; jÞ. stage setting by modeling the uncertainty as a
The simplest approach to deal with random- filtration process (successive realization of
ness in an optimization problem is to replace parameters). Under discrete distributions, this
the random variables or the whole (objective reduces to a scenario tree of parameter realiza-
or constraint) function by their means: tions. The compensation approach, however,
gðx; E½jÞ  0 or E½gðx; jÞ  0: But this leads requires that compensating actions exist at all
to the same problems as mentioned above. and can be reasonably modeled, which is not
Of the many concepts that have been dis- always possible.
cussed over the years, two concepts have been Two-stage linear stochastic programs can be
prevailed. efficiently solved for a discrete set of scenarios
in many cases. The solution methods build on
Two- and Multistage Stochastic Programs decomposition techniques of linear program-
with Recourse. The basic idea of recourse ming. Multi-stage integer and nonlinear sto-
(or compensation) relies on the possibility to chastic programs are often not or only with
adjust constraints in the system gðx; jÞ  0; great effort solvable for practically relevant
after observation of j by later compensating problem sizes.
actions. Accordingly in a two-stage program,
the set of variables splits into first stage deci-
Stochastic Programs with Chance/Probabilis-
sions x (to be fixed before realization of j) and
tic Constraints. If the emphasis is on the reli-
second stage or recourse decisions y (to be
ability of a system, it can be claimed that the
fixed after realization of j, see Fig. 56).
(system) constraints are not fulfilled for every
The adjustment of the constraint violation is
j 2 X but with high probability. That means
modeled by an inequality system ~ gðx; j; yÞ  0,
gðx; jÞ  0 is replaced by
connecting all three types of variables, and it
causes additional costs vðy; jÞ for the second
P½gðx; jÞ  0  a (126)
stage decisions. Of course, given x and j, y
should be chosen as to minimize second stage
costs among all feasible decisions. where a 2 ð0; 1 is some probability level. A
Summarizing, recourse models replace the constraint of this type is called chance or
original problem of minimizing the costs of first probabilistic constraint. Of course, the higher
stage decisions under stochastic inequalities by a the more reliable is the modeled system. On
a problem where the sum of first stage costs and the other hand, the set of feasible values is more
expected optimal second stage costs is mini- and more shrunken with a%, which increases
mized: the optimal value of the objective function, e.g.
the minimal costs, at the same time. The
min ff ðxÞ þ E½uðx; jÞg (125) extreme case a ¼ 1 is similar to the worst-
x2R n
case approach while the case a ¼ 1=2 relates
to the expected value approach. This makes
where
use of probabilistic constraints a good compro-
uðx; jÞ ¼ minn fvðy; jÞj~
gðx; j; yÞ  0g
mise between these two methods.
y2R

Figure 56. Two-stage stochastic programming with recourse

 Mathematics in Chemical Engineering 143

If there are several inequality constraints  AIMMS Stochastic Programming Soft-

g1 ðx; jÞ  0; . . .; gp ðx; jÞ  0, one has the ware: solvers for stochastic linear and
choice of integrating or separating them with stochastic mixed-integer programming
respect to the probability measure P: problems with recourse [324]
 
 SLP-IOR: an interactive model manage-
P g1 ðx; jÞ  0; . . .; gp ðx; jÞ  0  a (127) ment system for stochastic linear pro-
grams which supports two- and
or multistage programs with recourse, prob-
 
lems with individual and joint chance
P½g1 ðx; jÞ  0  a; . . . ; P gp ðx; jÞ  0  a (128) constraints, and problems with joint
chance constraints and CVaR constraints
where a can depend on the function or objective [325]
gi ; i ¼ 1; . . . ; p. These alternatives are referred  SMI: a modeling interface for stochastic
to as joint and individual probabilistic con- optimization [326]
straints. It is obvious that the first formulation  NEOS solvers for stochastic linear pro-
is more restrictive than the second one. Further, gramming [327]
the numerical treatment of probability func-
 AMPLDev SP Edition: an integrated
tions involving high-dimensional random
vectors is much more difficult than for (one- modeling and solving environment for
dimensional) random variables where a reduc- stochastic programming (multistage as
tion to quantiles of one-dimensional distribu- well as chance constraints) and for robust
tions can be carried out. However, the choice of optimization [328]
formulation also depends on the modeling point
of view. The efficient solvability of this type of
model depends on whether an efficiently solv- 10.9.4. Optimization under Uncertainty by
able equivalent deterministic model can be Multicriteria Optimization
derived. If this is not the case, these models
are very difficult to treat numerically. Another way to handle uncertainties in an
In summary, it has to be said that the solution optimization problem is to add uncertainty
of optimization problems involving probabilis- measures as further objectives to the problem
tic constraints requires at least the ability of and treat the problem as a multicriteria one (see
evaluating the function fðxÞ ¼ P½gðx; jÞ  0. Section 10.8).
If j is a continuous m-dimensional random The simplest, not even obvious multicriteria
vector with density f j the function f is formally approach is to add the variability (e.g., the
defined as the parameter-dependent multi- variance) of the uncertain parameters multi-
variate integral plied by a weight to the expected value of
the objective function:
Z
fðxÞ ¼ f j ðzÞdz Ej ½f ðx; jÞ þ l  Varj ½f ðx; jÞ (129)
gðx;jÞ0

By the parameter l one can punish (x > 0) or

where integration takes place over an m-dimen- support (x < 0) the variability of the uncertain-
sional domain. Even in moderate dimensions an ties. Furthermore, large values of l result into
“exact” evaluation of this integral by numerical solutions that reduce variance while small val-
integration is far from realistic. In this case, ues of l reduce expectations. In the optimiza-
two “inexact” strategies have to be proven tion of financial assets portfolios this approach
powerful in the past: bounding and (Monte is well-known and widely used as mean-vari-
Carlo) simulation. For details see [323] and ance (portfolio) optimization [329]. The dis-
the references therein. advantage of this approach is that l has to be
The following software is available for solv- fixed before optimization. If one adds on the
ing multi-stage stochastic programs and sto- other hand the variance as a second objective
chastic programs with chance constraints: and considers the problem as a multi-criteria
144 Mathematics in Chemical Engineering

one, Equation (129) is nothing more than the The root-mean-square or quadratic mean is
scalarization of the (two) objective functions by
means of a weighted sum. By this, one can vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u N
pffiffiffiffiffiffiffiffiffiffiffi uX
compare solutions for different l. Root  mean  square ¼ Eðy Þ ¼ t
2 y2i =N
In the same way, one can add the variability i¼1

of the uncertain parameters in the restriction

functions to the objective(s). Furthermore, one The range of a set of numbers is the differ-
can include the worst case over a finite or ence between the largest and the smallest mem-
infinite set of scenarios of the uncertainties bers in the set. The mean deviation is the mean
or sensitivity measures like finite differences of the deviation from the mean.
or difference quotients as objective to the prob-
lem. A reference for taking these ideas in P
N
chemical process engineering into account is jyi  EðyÞj
i¼1
[330]. Mean  deviation ¼
N

11. Probability and Statistics The variance is

[15, 331–338] N 
P 2
yi  EðyÞ
varðyÞ ¼ s 2 ¼ i¼1
The models treated thus far have been determi- N
nistic, that is, if the parameters are known the
outcome is determined. In many situations, all and the standard deviation s is the square root
the factors cannot be controlled and the out- of the variance.
come may vary randomly about some average
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
value. Then a range of outcomes has a certain uN 
uP 2
u
probability of occurring, and statistical meth- ti¼1 yi  EðyÞ
s¼
ods must be used. This is especially true in N
quality control of production processes and
experimental measurements. This chapter pres- If the set of numbers {yi} is a small sample
ents standard statistical concepts, sampling the- from a larger set, then the sample average
ory and statistical decisions, and factorial
design of experiments or analysis of variances. P
n
yi
Multivariant linear and nonlinear regression is y ¼ i¼1
n
treated in Chapter 2.
is used in calculating the sample variance
11.1. Concepts
P
n
ðyi  y Þ2
Suppose N values of a variable y, called y1, s2 ¼ i¼1
n1
y2, . . . , yN, might represent N measurements
of the same quantity. The arithmetic mean E and the sample standard deviation
(y) is
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
P uP
N
un
yi u ðyi  y Þ2
ti¼1
EðyÞ ¼ i¼1 s¼
N n1

The median is the middle value (or average of The value n  1 is used in the denominator
the two middle values) when the set of numbers because the deviations from the sample average
is arranged in increasing (or decreasing) order. must total zero:
The geometric mean is
X
n
ðyi  y Þ ¼ 0
y G ¼ ðy1 y2 . . . yN Þ1=N i¼1
 Mathematics in Chemical Engineering 145

Thus, knowing n1 values of yiy and the fact Table 12. Area under normal curve*
that there are n values automatically gives the n- Rz z2 =2
F ðzÞ ¼ p1ffiffiffiffi
2p
e dz
th value. Thus, only n1 degrees of freedom n 0

exist. This occurs because the unknown mean E z F (z)00 z F (z)00

(y) is replaced by the sample mean y derived 0.0 0.0000 1.5 0.4332
from the data. 0.1 0.0398 1.6 0.4452
0.2 0.0793 1.7 0.4554
If data are taken consecutively, running
0.3 0.1179 1.8 0.4641
totals can be kept to permit calculation of the 0.4 0.1554 1.9 0.4713
mean and variance without retaining all the 0.5 0.1915 2.0 0.4772
data: 0.6 0.2257 2.1 0.4821
0.7 0.2580 2.2 0.4861
0.8 0.2881 2.3 0.4893
X
n X
n X
n
ðyi  y Þ2 ¼ y21  2y yi þ ðy Þ2 0.9 0.3159 2.4 0.4918
i¼1 i¼1 i¼1 1.0 0.3413 2.5 0.4938
X
n 1.1 0.3643 2.7 0.4965
y ¼ yi =n 1.2 0.3849 3.0 0.4987
i¼1 1.3 0.4032 4.0 0.499968
1.4 0.4192 5.0 0.4999997
Thus, *
Table gives the probability F that a random variable will fall in the
shaded region of Figure 58. For a more complete table (in slightly
X
n X
n
different form), see [23, Table 26.1]. This table is obtained in
n; y21 ; and yi
i¼1 i¼1
Microsoft Excel with the function NORMDIST(z,0,1,1)-0.5.

are retained, and the mean and variance are

computed when needed.
This is called a normal probability distribution
Repeated observations that differ because of
function. It is important because many results
experimental error often vary about some
are insensitive to deviations from a normal
central value in a roughly symmetrical distri-
distribution. Also, the central limit theorem
bution in which small deviations occur more
says that if an overall error is a linear combina-
frequently than large deviations. In plotting the
tion of component errors, then the distribution
number of times a discrete event occurs, a
of errors tends to be normal as the number of
typical curve is obtained, which is shown in
components increases, almost regardless of the
Figure 57. Then the probability p of an event
distribution of the component errors (i.e., they
(score) occurring can be thought of as the ratio
need not be normally distributed). Naturally,
of the number of times it was observed divided
several sources of error must be present and one
by the total number of events. A continuous
error cannot predominate (unless it is normally
representation of this probability density func-
distributed). The normal distribution function is
tion is given by the normal distribution
calculated easily; of more value are integrals of
1 2
the function, which are given in Table 12; the
pðyÞ ¼ pffiffiffiffiffiffi e½yEðyÞ =2s
2
(130) region of interest is illustrated in Figure 58.
s 2p
For a small sample, the variance can only
be estimated with the sample variance s2.
Thus, the normal distribution cannot be used

Figure 57. Frequency of occurrence of different scores Figure 58. Area under normal curve
146 Mathematics in Chemical Engineering

Table 13. Percentage points of area under Students t-distribution*

n a ¼ 0.10 a ¼ 0.05 a ¼ 0.01 a ¼ 0.001

1 6.314 12.706 63.657 636.619

2 2.920 4.303 9.925 31.598
3 2.353 3.182 5.841 12.941
4 2.132 2.776 4.604 8.610
5 2.015 2.571 4.032 6.859
6 1.943 2.447 3.707 5.959
7 1.895 2.365 3.499 5.408
8 1.860 2.306 3.355 5.041
9 1.833 2.262 3.250 4.781
10 1.812 2.228 3.169 4.587
15 1.753 2.131 2.947 4.073
20 1.725 2.086 2.845 3.850
25 1.708 2.060 2.787 3.725
30 1.697 2.042 2.750 3.646
Figure 59. Student’s t-distribution. 1 1.645 1.960 2.576 3.291
For explanation of n see text *
Table gives t values such that a random variable will fall in the
shaded region of Figure 60 with probability a. For a one-sided test
the confidence limits are obtained for a/2. For a more complet table
(in slightly different form), see [23, Table 26.10]. This table is
because s is not known. In such cases
obtained in Microsoft Excel with the function TINV(a,n).
Student’s t-distribution, shown in Figure 59
[338, p. 70], is used:

y0 y EðyÞ is the probability of x being between x and

pðyÞ ¼
; t ¼ s=pffiffinffi
t2 n=2
1 þ n1 x þ dx. The probability density function satis-
fies
and y0 is chosen such that the area under the
curve is one. The number n ¼ n  1 is the pðxÞ  0
degrees of freedom, and as n increases, Z1
Student’s t-distribution approaches the normal pðxÞdx ¼ 1
distribution. The normal distribution is ade- 1

quate (rather than the t-distribution) when n

The Bernoulli distribution applies when the
> 15, except for the tails of the curve which
outcome can take only two values, such as
require larger n. Integrals of the t-distribution
heads or tails, or 0 or 1. The probability distri-
are given in Table 13, the region of interest is
bution function is
shown in Figure 60.
Other probability distribution functions are
pðx ¼ kÞ ¼ pk ð1  pÞ1k ; k ¼ 0 or 1
useful. Any distribution function must satisfy
the following conditions:

0  FðxÞ  1

Fð1Þ ¼ 0; Fðþ1Þ ¼ 1

FðxÞ  FðyÞ when x  y

The probability density function is

dFðxÞ
pðxÞ ¼
dx

where

dF ¼ pdx Figure 60. Percentage points of area under Student’s t-

distribution
 Mathematics in Chemical Engineering 147

and the mean of a function g(x) depending on x function is

is
8 1
< a<x<b
E½gðxÞ ¼ gð1Þp þ gð0Þð1  pÞ p¼ ba
:
0 x < a; x > b
The binomial distribution function applies
when there are n trials of a Bernoulli event; and the probability distribution function is
it gives the probability of k occurrences of the 8
>
> 0 x<a
event, which occurs with probability p on each >
< xa
trial FðxÞ ¼ a<x<b
>
> ba
>
:
n! 1 b<x
pðx ¼ kÞ ¼ pk ð1  pÞnk
kðn  kÞ!
The mean and variance are
The mean and variance are aþb
EðxÞ ¼
2
EðxÞ ¼ np
ðb  aÞ2
varðxÞ ¼
varðxÞ ¼ npð1  pÞ 12

The hypergeometric distribution function The normal distribution is given by

applies when there are N objects, of which M Equation (130) with variance s 2.
are of one kind and N  M are of another kind. The log normal probability density function
Then the objects are drawn one by one, without is
replacing the last draw. If the last draw had been " #
replaced the distribution would be the binomial 1 ðlogx  mÞ2
pðxÞ ¼ pffiffiffiffiffiffi exp  2
distribution. If x is the number of objects of type xs 2p 2s
M drawn in a sample of size n, then the proba-
bility of x ¼ k is and the mean and variance are [340, p. 89]
 
s2
EðxÞ ¼ exp m þ
pðx ¼ kÞ 2
M!ðN  MÞ!n!ðN  nÞ! varðxÞ ¼ expðs 2  1Þexpð2m þ s 2 Þ
¼
k!ðM  kÞ!ðn  kÞ!ðN  M  n þ kÞ!N!

11.2. Sampling and Statistical Decisions

The mean and variance are
Two variables can be statistically dependent or
EðxÞ ¼
nM independent. For example, the height and diam-
N eter of all distillation towers are statistically
Nn independent, because the distribution of diam-
varðxÞ ¼ npð1  pÞ
N1
eters of all columns 10 m high is different from
that of columns 30 m high. If yB is the diameter
The Poisson distribution is
and yA the height, the distribution is written as
lk
pðx ¼ kÞ ¼ el pðyB jyA ¼ constantÞ; or here
k!
pðyB jyA ¼ 10Þ 6¼ pðyB jyA ¼ 30Þ
with a parameter l. The mean and variance are
A third variable yC, could be the age of the
EðxÞ ¼ l operator on the third shift. This variable is
varðxÞ ¼ l probably unrelated to the diameter of the col-
umn, and for the distribution of ages is
The simplest continuous distribution is the
uniform distribution. The probability density pðyC jyA Þ ¼ pðyC Þ
148 Mathematics in Chemical Engineering

Thus, variables yA and yC are distributed inde- Suppose Y, which is the sum or difference of
pendently. The joint distribution for two vari- two variables, is of interest:
ables is
Y ¼ y A
yB
pðyA ; yB Þ ¼ pðyA ÞpðyB jyA Þ

Then the mean value of Y is

if they are statistically dependent, and
EðYÞ ¼ EðyA Þ
EðyB Þ
pðyA; yB Þ ¼ pðyA ÞpðyB Þ

and the variance of Y is

if they are statistically independent. If a set of
variables yA, yB, . . . is independent and iden-
s 2 ðYÞ ¼ s 2 ðyA Þ þ s 2 ðyB Þ
tically distributed,
More generally, consider the random variables
pðyA ; yB ; . . .Þ ¼ pðyA ÞpðyB Þ . . .
y1, y2, . . . with means E (y1), E (y2), . . . and
variances s 2 (y1), s2(y2), . . . and correlation
Conditional probabilities are used in hazard
coefficients %ij. The variable
analysis of chemical plants.
A measure of the linear dependence between Y ¼ a1 y1 þ a2 y2 þ . . .
variables is given by the covariance

CovðyA ; yB Þ ¼ Ef½yA  EðyA Þ½ðyB  EðyB ÞÞg has a mean

X
N
½yAi  EðyA Þ½yBi  EðyB Þ EðYÞ ¼ a1 Eðy1 Þ þ a2 Eðy2 Þ þ . . .
i¼1
¼
N
and variance [338, p. 87]
The correlation coefficient % is
X
n
s 2 ðYÞ ¼ a2i s 2 ðyi Þ
CovðyA ; yB Þ
%ðyA ; yB Þ ¼ i¼1
sA sB
Xn X
n
þ2 ai aj sðyi Þsðyj Þ%ij
If yA and yB are independent, then Cov (yA, yB) i¼1 j¼iþ1

¼ 0. If yA tends to increase when yB decreases

then Cov (yA, yB) < 0. The sample correlation or
coefficient is [15, p. 484]
X
n

P
n s 2 ðYÞ ¼ a2i s 2 ðyi Þ
ðyAi  y A ÞðyBi  y B Þ i¼1
(131)
rðyA ; yB Þ ¼ i¼1 Xn X
n
ðn  1ÞsA sB þ2 ai aj Covðyi ; yj Þ
i¼1 j¼iþ1
If measurements are for independent, iden-
tically distributed observations, the errors are If the variables are uncorrelated and have the
independent and uncorrelated. Then y varies same variance, then
about E (y) with variance s 2/n, where n is the
!
number of observations in y . Thus if some- X
n
s 2 ðYÞ ¼ a2i s 2
thing is measured several times today and i¼1
every day, and the measurements have the
same distribution, the variance of the means
decreases with the number of samples in each This fact can be used to obtain more accurate
day’s measurement n. Of course, other factors cost estimates for the purchased cost of a
(weather, weekends) may cause the observa- chemical plant than is true for any one piece
tions on different days to be distributed of equipment. Suppose the plant is composed of
nonidentically. a number of heat exchangers, pumps, towers,
 Mathematics in Chemical Engineering 149

etc., and that the cost estimate of each device is

40 % of its cost (the sample standard devia-

tion is 20 % of its cost). In this case the ai are the
numbers of each type of unit. Under special
conditions, such as equal numbers of all types
of units and comparable cost, the standard
deviation of the plant costs is
s
sðYÞ ¼ pffiffiffi Figure 61. Two-sided statistical decision
n

pffiffiffi
and is then
ð40= nÞ%. Thus the standard
deviation of the cost for the entire plant is of significance, 0.95 ¼ 0.5 (for negative z) þ F
the standard deviation of each piece of equip- (for positive z). Thus, F ¼ 0.45 or z ¼ 1.645. In
ment divided by the square root of the number the two-sided test (see Fig. 61), if a single
of units. Under less restrictive conditions the sample is chosen and z < 1.96 or z > 1.96,
actual numbers change according to the above then this could happen with probability 0.05 if
equations, but the principle is the same. the hypothesis were true. This z would be
Suppose modifications are introduced into significantly different from the expected value
the manufacturing process. To determine if the (based on the chosen level of significance) and
modification causes a significant change, the the tendency would be to reject the hypothesis.
mean of some property could be measured If the value of z was between 1.96 and 1.96,
before and after the change; if these differ, the hypothesis would be accepted.
does it mean the process modification caused The same type of decisions can be made for
it, or could the change have happened by other distributions. Consider Student’s t-distri-
chance? This is a statistical decision. A hypoth- bution. At a 95 % level of confidence, with n ¼
esis H0 is defined; if it is true, action A must be 10 degrees of freedom, the t values are
2.228.
taken. The reverse hypothesis is H1; if this is Thus, the sample mean would be expected to be
true, action B must be taken. A correct decision between
is made if action A is taken when H0 is true or s
action B is taken when H1 is true. Taking action y
tc pffiffiffi
n
B when H0 is true is called a type I error,
whereas taking action A when H1 is true is with 95 % confidence. If the mean were outside
called a type II error. this interval, the hypothesis would be rejected.
The following test of hypothesis or test of The chi-square distribution is useful for
significance must be defined to determine if the examining the variance or standard deviation.
hypothesis is true. The level of significance is The statistic is defined as
the maximum probability that an error would be
accepted in the decision (i.e., rejecting the ns2
x2 ¼
hypothesis when it is actually true). Common s2
levels of significance are 0.05 and 0.01, and the ðy1  y Þ2 þ ðy2  y Þ2 þ . . . þ ðyn  y Þ2
¼
test of significance can be either one or two s2
sided. If a sampled distribution is normal, then
the probability that the z score and the chi-square distribution is

y  y pðyÞ ¼ y0 xn2 ex

2
=2
z¼
sy

n ¼ n  1 is the number of degrees of freedom

is in the unshaded region is 0.95. Because a and y0 is chosen so that the integral of p (y) over
two-sided test is desired, F ¼ 0.95/2 ¼ 0.475. all y is 1. The probability of a deviation larger
The value given in Table 12 for F ¼ 0.475 is z ¼ than x2 is given in Table 14; the area in ques-
1.96. If the test was one-sided, at the 5 % level tion, in Figure 62. For example, for 10 degrees
150 Mathematics in Chemical Engineering

Table 14. Percentage points of area under chi-square distribution with n degrees of freedom*

n a ¼ 0.995 a ¼ 0.99 a ¼ 0.975 a ¼ 0.95 a ¼ 0.5 a ¼ 0.05 a ¼ 0.025 a¼0.01 a ¼ 0.005

1 7.88 6.63 5.02 3.84 0.455 0.0039 0.0010 0.0002 0.00004

2 10.6 9.21 7.38 5.99 1.39 0.103 0.0506 0.0201 0.0100
3 12.8 11.3 9.35 7.81 2.37 0.352 0.216 0.115 0.072
4 14.9 13.3 11.1 9.49 3.36 0.711 0.484 0.297 0.207
5 16.7 15.1 12.8 11.1 4.35 1.15 0.831 0.554 0.412
6 18.5 16.8 14.4 12.6 5.35 1.64 1.24 0.872 0.676
7 20.3 18.5 16.0 14.1 6.35 2.17 1.69 1.24 0.989
8 22.0 20.1 17.5 15.5 7.34 2.73 2.18 1.65 1.34
9 23.6 21.7 19.0 16.9 8.34 3.33 2.70 2.09 1.73
10 25.2 23.2 20.5 18.3 9.34 3.94 3.25 2.56 2.16
12 28.3 26.2 23.3 21.0 11.3 5.23 4.40 3.57 3.07
15 32.8 30.6 27.5 25.0 14.3 7.26 6.26 5.23 4.60
17 35.7 33.4 30.2 27.6 16.3 8.67 7.56 6.41 5.70
20 40.0 37.6 34.2 31.4 19.3 10.9 9.59 8.26 7.43
25 46.9 44.3 40.6 37.7 24.3 14.6 13.1 11.5 10.5
30 53.7 50.9 47.0 43.8 29.3 18.5 16.8 15.0 13.8
40 66.8 63.7 59.3 55.8 39.3 26.5 24.4 22.2 20.7
50 79.5 76.2 71.4 67.5 49.3 34.8 32.4 29.7 28.0
60 92.0 88.4 83.3 79.1 59.3 43.2 40.5 37.5 35.5
70 104.2 100.4 95.0 90.5 69.3 51.7 48.8 45.4 43.3
80 116.3 112.3 106.6 101.9 79.3 60.4 57.2 53.5 51.2
90 128.3 124.1 118.1 113.1 89.3 69.1 65.6 61.8 59.2
100 140.2 135.8 129.6 124.3 99.3 77.9 74.2 70.1 67.3
*
Table value is x2a; x2< x2a with probability a. For a more complete table (in slightly different form), see [23, Table 26.8]. The Microsoft Excel
function CHIINV(1-a,n) gives the table value.

of freedom and a 95 % confidence level, the called y, with N2 samples. First, compute the
critical values of x2 are 0.025 and 0.975. Then standard error of the difference of the means:
pffiffiffi pffiffiffi vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
s n s n uP P
<s< u N1 N2
u ðxi  xÞ2 þ ðyi  y Þ2  
x0:975 x0:025 ti¼1 i¼1 1 1
sD ¼ þ
N1 þ N2  2 N1 N2
or
pffiffiffi pffiffiffi
Next, compute the value of t
s n s n
<s<
20:5 3:25 x y
t¼
sD
with 95% confidence.
Tests are available to decide if two distribu- and evaluate the significance of t using
tions that have the same variance have different Student’s t-distribution for N1 þ N2 2 degrees
means [15, p. 465]. Let one distribution be of freedom.
called x, with N1 samples, and the other be If the samples have different variances, the
relevant statistic for the t-test is

x  y
t ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
varðxÞ=N 1 þ varðyÞ=N 2

The number of degrees of freedom is now taken

approximately as
 2
varðxÞ varðyÞ
N 1 þ N2
n¼
½varðxÞ=N 1 2 2
Figure 62. Percentage points of area under chi-squared N 1 1 þ ½varðyÞ=N
N 2 1
2

distribution with n degrees of freedom

 Mathematics in Chemical Engineering 151

There is also an F-test to decide if two If each measured quantity has some variance,
distributions have significantly different vari- what is the variance in the thermal
ances. In this case, the ratio of variances is conductivity?
calculated: Suppose a model for Y depends on various
measurable quantities, y1, y2, . . . Suppose
varðxÞ
F¼
varðyÞ
several measurements are made of y1, y2, . . .
under seemingly identical conditions and sev-
where the variance of x is assumed to be larger. eral different values are obtained, with means E
Then, a table of values is used to determine (y1), E (y2), . . . and variances s 21 , s 22 , . . .
the significance of the ratio. The table Next suppose the errors are small and indepen-
[23, Table 26.9] is derived from the formula dent of one another. Then a change in Y is
[15, p. 169] related to changes in yi by
n 
2 n1
QðFjn1 ; n2 Þ ¼ I n2 =ðn2 þn1 FÞ ; @Y @Y
2 2 dY ¼ dy þ dy þ . . .
@y1 1 @y2 2
where the right-hand side is an incomplete beta
function. The F table is given by the Microsoft If the changes are indeed small, the partial
Excel function FINV(fraction, nx, ny), where derivatives are constant among all the samples.
fraction is the fractional value ( 1) represent- Then the expected value of the change is
ing the upper percentage and nx and ny are the
N 
X 
degrees of freedom of the numerator and @Y
EðdYÞ ¼ Eðdyi Þ
denominator, respectively. i¼1
@yi
Example. For two sample variances with 8
degrees of freedom each, what limits will Naturally E (dyi) ¼ 0 by definition so that E
bracket their ratio with a midarea probability (dY) ¼ 0, too. However, since the errors are
of 90 %? FINV(0.95, 8, 8)  3.44. The 0.95 is independent of each other and the partial deriv-
used to get both sides to total 10%. Then atives are assumed constant because the errors
P½1=3:44  varðxÞ=varðyÞ  3:44 ¼ 0:90:
are small, the variances are given by Equation ()
[331, p. 550]
11.3. Error Analysis in Experiments N 
X 
@Y 2
s 2 ðdYÞ ¼ s 2i (132)
i¼1
@yi
Suppose a measurement of several quantities
is made and a formula or mathematical model Thus, the variance of the desired quantity Y can
is used to deduce some property of interest. be found. This gives an independent estimate of
For example, to measure the thermal conduc- the errors in measuring the quantity Y from the
tivity of a solid k, the heat flux q, the thickness errors in measuring each variable it depends
of the sample d, and the temperature differ- upon.
ence across the sample DT must be measured.
Each measurement has some error. The heat
flux q may be the rate of electrical heat input 11.4. Factorial Design of Experiments
Q_ divided by the area A, and both quantities and Analysis of Variance
are measured to some tolerance. The thickness
of the sample is measured with some accuracy, Statistically designed experiments consider, of
and the temperatures are probably measured course, the effect of primary variables, but
with a thermocouple, to some accuracy. These they also consider the effect of extraneous
measurements are combined, however, to variables, the interactions among variables,
obtain the thermal conductivity, and the error and a measure of the random error. Primary
in the thermal conductivity must be deter- variables are those whose effect must be deter-
mined. The formula is mined. These variables can be quantitative or
d qualitative. Quantitative variables are ones
k¼ Q
ADT that may be fit to a model to determine the
152 Mathematics in Chemical Engineering

model parameters. Curve fitting of this type is Table 15. Estimating the effect of four treatments
discused in Chapter 2. Qualitative variables Treatment 1 2 3 4
are ones whose effect needs to be known; no
attempt is made to quantify that effect other    
   
than to assign possible errors or magnitudes.    
Qualitative variables can be further subdivided   
into type I variables, whose effect is deter-  
mined directly, and type II variables, which 
Treatment average, y t    
contribute to performance variability, and Grand average, y 
whose effect is averaged out. For example,
in studying the effect of several catalysts on
yield in a chemical reactor, each different type
that is, whether their means are different.
of catalyst would be a type I variable, because
The samples are assumed to have the same
its effect should be known. However, each
variance. The hypothesis is that the treatments
time the catalyst is prepared, the results are
are all the same, and the null hypothesis is that
slightly different, because of random varia-
they are different. Deducing the statistical
tions; thus, several batches may exist of what
validity of the hypothesis is done by an analysis
purports to be the same catalyst. The variabil-
of variance.
ity between batches is a type II variable.
The data for k ¼ 4 treatments are arranged
Because the ultimate use will require using
in Table 15. Each treatment has nt experi-
different batches, the overall effect including
ments, and the outcome of the i-th experiment
that variation should be known, because
with treatment t is called yti. The treatment
knowing the results from one batch of
average is
one catalyst precisely might not be represent-
ative of the results obtained from all batches of P
nt
the same catalyst. A randomized block design, i¼1
yti
incomplete block design, or Latin square y t ¼
nt
design, for example, all keep the effect of
experimental error in the blocked variables and the grand average is
from influencing the effect of the primary
variables. Other uncontrolled variables are P
k
nt y t X
k
accounted for by introducing randomization y ¼ t¼1
; N¼ nt
in parts of the experimental design. To study N t¼1

all variables and their interaction requires a

factorial design, involving all possible combi- Next, the sum of squares of deviations is
nations of each variable, or a fractional facto- computed from the average within the t-th
rial design, involving only a selected set. treatment
Statistical techniques are then used to deter-
mine the important variables, the important X
nt
St ¼ ðyti  y t Þ2
interactions and the error in estimating these i¼1

effects. The discussion here is a brief overview

of [338]. Since each treatment has nt experiments, the
If only two methods exist for preparing some number of degrees of freedom is nt1. Then
product, to see which treatment is best, the the sample variances are
sampling analysis discussed in Section 11.2
can be used to deduce if the means of the St
s2t ¼
nt  1
two treatments differ significantly. With more
treatments, the analysis is more detailed. Sup-
pose the experimental results are arranged as The within-treatment sum of squares is
shown in Table 15, i.e., several measurements
X
k
for each treatment. The objective is to see if the SR ¼ St
treatments differ significantly from each other, t¼1
 Mathematics in Chemical Engineering 153

and the within-treatment sample variance is Name Formula Degrees of

freedom
2
SR Average SA ¼ nky 1
s2R ¼ P n
Nk Blocks SB ¼ k ðy t  y Þ2 n-1
i¼1
Pk
Treatments ST ¼ n ðy t  y Þ2 k-1
Now, if no difference exists between treat- t¼1
ments, a second estimate of s 2 could be Residuals SR ¼
P
k P n
2
ðyti  y i  y t þ y Þ (n-1)(k-1)
obtained by calculating the variation of the t¼1 i¼1
P
k Pn
treatment averages about the grand average. Total S¼ y2ti N¼nk
t¼1 i¼1
Thus, the between-treatment mean square
is computed: The key test is again a statistical one, based
on the value of
ST Xk
s2T ¼ ; ST ¼ nt ðy t  y Þ2
k1 ST
t¼1 s2T =s2R ; where s2T ¼
k1
SR
and s2R ¼
Basically the test for whether the hypo- ðn  1Þðk  1Þ
thesis is true or not hinges on a comparison
between the within-treatment estimate s2R and the F distribution for nR and nT degrees of
(with nR ¼ N  k degrees of freedom) and freedom [338, p. 636]. The assumption behind
the between-treatment estimate s2T (with nT ¼ the analysis is that the variations are linear [338,
k1 degrees of freedom). The test is made p. 218]. Ways to test this assumption as well as
based on the F distribution for nR and nT transformations to make if it is not true are
degrees of freedom [23, Table 26.9], [338, provided in [338], where an example is given of
p. 636]. how the observations are broken down into a
Next consider the case in which random- grand average, a block deviation, a treatment
ized blocking is used to eliminate the effect of deviation, and a residual. For two-way factorial
some variable whose effect is of no interest, design, in which the second variable is a real
such as the batch-to-batch variation of the one rather than one you would like to block out,
catalysts in the chemical reactor example. see [338, p. 228].
With k treatments and n experiments in each To measure the effects of variables on a
treatment, the results from n k experiments can single outcome, a factorial design is appropri-
be arranged as shown in Table 16; within each ate. In a two-level factorial design, each varia-
block, various treatments are applied in a ble is considered at two levels only, a high and
random order. The block average, the treat- low value, often designated as a þ and a . The
ment average, and the grand average are com- two-level factorial design is useful for indicat-
puted as before. The following quantities are ing trends and showing interactions; it is also
also computed for the analysis of variance the basis for a fractional factorial design. As an
table: example, consider a 23 factorial design, with 3
variables and 2 levels for each. The experiments
are indicated in Table 17. The main effects are
calculated by determining the difference
Table 16. Block design with four treatments and five blocks
between results from all high values of a varia-
ble and all low values of a variable; the result is
Treatment 1 2 3 4 Block divided by the number of experiments at each
average
level. For example, for the first variable, calcu-
Block 1      late
Block 2     
Block 3     
Effect of variable 1 ¼ ½ðy2 þ y4 þ y6 þ y8 Þ½ðy1 þ y3 þ y5 þ y7 Þ=4
Block 4     
Block 5     
Treatment     grand Note that all observations are being used to
average average supply information on each of the main effects
154 Mathematics in Chemical Engineering

Table 17. Two-level factorial design with three variables chosen. CROPLEY [333] gives an example of
Run Variable 1 Variable 2 Variable 3
how to combine heuristics and statistical argu-
ments in application to kinetics mechanisms in
1    chemical engineering.
2 þ  
3  þ 
4 þ þ 
5   þ References
6 þ  þ
7  þ þ 1 D.E. Seborg, T.F. Edgar, D.A. Mellichamp: Process Dynamics
8 þ þ þ and Control, 2nd ed., John Wiley & Sons, New York 2004.
2 G. Forsyth, C.B. Moler: Computer Solution of Lineart Alge-
braic Systems, Prentice-Hall, Englewood Cliffs 1967.
3 B.A. Finlayson: Nonlinear Analysis in Chemical Engineering,
McGraw-Hill, New York 1980; reprinted, Ravenna Park, Seat-
and each effect is determined with the preci- tle 2003.
4 S.C. Eisenstat, M.H. Schultz, A.H. Sherman: “Algorithms and
sion of a fourfold replicated difference. The Data Structures for Sparse Symmetric Gaussian Elimination,”
advantage of a one-at-a-time experiment is the SIAM J. Sci. Stat. Comput. 2 (1981) 225–237.
gain in precision if the variables are additive 5 I.S. Duff: Direct Methods for Sparse Matrices, Charendon
and the measure of nonadditivity if it occurs Press, Oxford 1986.
6 H.S. Price, K.H. Coats, “Direct Methods in Reservoir Simu-
[338, p. 313]. lation,” Soc. Pet. Eng. J. 14 (1974) 295–308.
Interaction effects between variables 1 and 2 7 A. Bykat: “A Note on an Element Re-Ordering Scheme,” Int. J.
are obtained by comparing the difference Num. Methods Egn. 11 (1977) 194 –198.
between the results obtained with the high 8 M.R. Hesteness, E. Stiefel: “Methods of conjugate gradients
for solving linear systems,” J. Res. Nat. Bur. Stand 29 (1952)
and low value of 1 at the low value of 2 with 409–439.
the difference between the results obtained with 9 Y. Saad: Iterative Methods for Sparse Linear Systems, 2nd ed.,
the high and low value 1 at the high value of 2. Soc. Ind. Appl. Math., Philadelphia 2003.
The 12-interaction is 10 Y. Saad, M. Schultz: “GMRES: A Generalized Minimal Resid-
ual Algorithm for Solving Nonsymmetric Linear Systems.”
12 interaction ¼ ½ðy4  y3 þ y8  y7 Þ  ½ðy2  y1 þ y6  y5 Þ=2 SIAM J. Sci. Statist. Comput. 7 (1986) 856–869.
11 http://mathworld.wolfram.com/
The key step is to determine the errors associ- GeneralizedMinimalResidualMethod.html.
ated with the effect of each variable and each 12 http://www.netlib.org/linalg/html_templates/Templates.html.
13 http://software.sandia.gov/.
interaction so that the significance can be deter- 14 E. Isaacson, H.B. Keller, Analysis of Numerical Methods,
mined. Thus, standard errors need to be J. Wiley and Sons, New York 1966.
assigned. This can be done by repeating the 15 W.H. Press, B.P. Flannery, S.A. Teukolsky, W.T. Vetterling:
experiments, but it can also be done by using Numerical Recipes, Cambridge University Press, Cambridge
1986.
higher order interactions (such as 123 interac- 16 R.W.H. Sargent: “A Review of Methods for Solving Non-linear
tions in a 24 factorial design). These are Algebraic Equations,” in R.S.H. Mah, W.D. Seider (eds.):
assumed negligible in their effect on the Foundations of Computer-Aided Chemical Process Design,
mean but can be used to estimate the standard American Institute of Chemical Engineers, New York 1981.
17 J.D. Seader: “Computer Modeling of Chemical Processes,”
error [338, pp. 319–328]. Then calculated AIChE Monogr. Ser. 81 (1985) no. 15.
effects that are large compared to the standard 18 http://software.sandia.gov/trilinos/packages/nox/loca_user.
error are considered important, whereas those html.
that are small compared to the standard error 19 N.R. Amundson: Mathematical Methods in Chemical Engi-
neering, Prentice-Hall, Englewood Cliffs, N.J. 1966.
are considered due to random variations and are 20 R.H. Perry, D. W Green: Perry’s Chemical Engineers’ Hand-
unimportant. book, 7th ed., McGraw-Hill, New York 1997.
In a fractional factorial design, only part of 21 D.S. Watkins: “Understanding the QR Algorithm,” SIAM Rev.
the possible experiments is performed. With k 24 (1982) 427–440.
22 G.F. Carey, K. Sepehrnoori: “Gershgorin Theory for Stiffness
variables, a factorial design requires 2k experi- and Stability of Evolution Systems and Convection-Diffusion,”
ments. When k is large, the number of experi- Comp. Meth. Appl. Mech. 22 (1980) 23–48.
ments can be large; for k ¼ 5, 25 ¼ 32. For k this 23 M. Abranowitz, I.A. Stegun: Handbook of Mathematical Func-
large, BOX et al. [331, p. 235] do a fractional tions, National Bureau of Standards, Washington, D.C. 1972.
24 J.C. Daubisse: “Some Results about Approximation of Func-
factorial design. In the fractional factorial tions of One or Two Variables by Sums of Exponentials, ” Int. J.
design with k ¼ 5, only 8 experiments are Num. Meth. Eng. 23 (1986) 1959–1967.
 Mathematics in Chemical Engineering 155

25 O.C. McGehee: An Introduction to Complex Analysis, John 53 J.D. Lambert: Computational Methods in Ordinary Differential
Wiley & Sons, New York 2000. Equations, J. Wiley and Sons, New York 1973.
26 H.A. Priestley: Introduction to complex analysis, Oxford Uni- 54 N.R. Amundson: Mathematical Methods in Chemical Engi-
versity Press, New York 2003. neering, Prentice-Hall, Englewood Cliffs, N.J. 1966.
27 Y.K. Kwok: Applied complex variables for scientists 55 J.R. Rice: Numerical Methods, Software, and Analysis,
and engineers, Cambridge University Press, New York McGraw-Hill, New York 1983.
2002. 56 M.B. Bogacki, K. Alejski, J. Szymanowski: “The Fast Method
28 N. Asmar, G.C. Jones: Applied complex analysis with partial of the Solution of Reacting Distillation Problem,” Comput.
differential equations, Prentice Hall, Upper Saddle River, NJ, Chem. Eng. 13 (1989) 1081–1085.
2002. 57 C.W. Gear: Numerical Initial-Value Problems in Ordinary
29 M.J. Ablowitz, A.S. Fokas: Complex variables: Introduction Differential Equations, Prentice-Hall, Englewood Cliffs, N.J.
and applications, Cambridge University Press, New York 1971.
2003. 58 N.B. Ferguson, B.A. Finlayson: “Transient Modeling of a
30 J.W. Brown, R.V. Churchill: Complex variables and applica- Catalytic Converter to Reduce Nitric Oxide in Automobile
tions, 6th ed., McGraw-Hill, New York 1996; 7th ed. 2003. Exhaust,” AIChE J 20 (1974) 539–550.
31 W. Kaplan: Advanced calculus, 5th ed., Addison-Wesley, 59 W.F. Ramirez: Computational Methods for Process Simula-
Redwood City, Calif., 2003. tions, 2nd ed., Butterworth-Heinemann, Boston 1997.
32 E. Hille: Analytic Function Theory, Ginn and Co., Boston 60 U.M. Ascher, L.R. Petzold: Computer methods for ordinary
1959. differential equations and differential-algebraic equations,
33 R.V. Churchill: Operational Mathematics, McGraw-Hill, SIAM, Philadelphia 1998.
New York 1958. 61 K.E. Brenan, S.L. Campbell, L.R. Petzold: Numerical Solution
34 J.W. Brown, R.V. Churchill: Fourier Series and Boundary of Initial-Value Problems in Differential-Algebraic Equations,
Value Problems, 6th ed., McGraw-Hill, New York 2000. Elsevier, Amsterdam 1989.
35 R.V. Churchill: Operational Mathematics, 3rd ed., McGraw- 62 C.C. Pontelides, D. Gritsis, K.R. Morison, R.W.H. Sargent:
Hill, New York 1972. “The Mathematical Modelling of Transient Systems Using
36 B. Davies: Integral Transforms and Their Applications, 3rd ed., Differential-Algebraic Equations,” Comput. Chem. Eng. 12
Springer, Heidelberg 2002. (1988) 449–454.
37 D.G. Duffy: Transform Methods for Solving Partial Differen- 63 G.A. Byrne, P.R. Ponzi: “Differential-Algebraic Systems,
tial Equations, Chapman & Hall/CRC, New York 2004. Their Applications and Solutions,” Comput. Chem. Eng. 12
38 A. Varma, M. Morbidelli: Mathematical Methods in Chemical (1988) 377–382.
Engineering, Oxford, New York 1997. 64 L.F. Shampine, M.W. Reichelt: The MATLAB ODE Suite,
39 H. Bateman, Tables of Integral Transforms, vol. I, McGraw- SIAM J. Sci. Comp. 18 (1997) 122.
Hill, New York 1954. 65 M. Kubicek, M. Marek: Computational Methods in Bifurcation
40 H.F. Weinberger: A First Course in Partial Differential Equa- Theory and Dissipative Structures, Springer Verlag, Berlin
tions, Blaisdell, Waltham, Mass. 1965. 1983.
41 R.V. Churchill: Operational Mathematics, McGraw-Hill, New 66 T.F.C. Chan, H.B. Keller: “Arc-Length Continuation and
York 1958. Multi-Grid Techniques for Nonlinear Elliptic Eigenvalue Prob-
42 J.T. Hsu, J.S. Dranoff: “Numerical Inversion of Certain lems,” SIAM J. Sci. Stat. Comput. 3 (1982) 173–194.
Laplace Transforms by the Direct Application of Fast Fourier 67 M.F. Doherty, J.M. Ottino: “Chaos in Deterministic Systems:
Transform (FFT) Algorithm,” Comput. Chem. Eng. 11 (1987) Strange Attractors, Turbulence and Applications in Chemical
101–110. Engineering,” Chem. Eng. Sci. 43 (1988) 139–183.
43 E. Kreyzig: Advanced Engineering Mathematics, 9th ed., John 68 M.P. Allen, D.J. Tildesley: Computer Simulation of Liquids,
Wiley & Sons, New York 2006. Clarendon Press, Oxford 1989.
44 H.S. Carslaw, J.C. Jaeger: Conduction of Heat in Solids, 2nd 69 D. Frenkel, B. Smit: Understanding Molecular Simulation,
ed., Clarendon Press, Oxford–London 1959. Academic Press, San Diego 2002.
45 R.B. Bird, R.C. Armstrong, O. Hassager: Dynamics of Poly- 70 J.M. Haile: Molecular Dynamics Simulation, John Wiley &
meric Liquids, 2nd ed., Appendix A, Wiley-Interscience, New Sons, New York 1992.
York 1987. 71 A.R. Leach: Molecular Modelling: Principles and Applica-
46 R.S. Rivlin, J. Rat. Mech. Anal. 4 (1955) 681–702. tions, Prentice Hall, Englewood Cliffs, NJ, 2001.
47 N.R. Amundson: Mathematical Methods in Chemical Engi- 72 T. Schlick: Molecular Modeling and Simulations, Springer,
neering; Matrices and Their Application, Prentice-Hall, Engle- New York 2002.
wood Cliffs, N.J. 1966. 73 R.B. Bird, W.E. Stewart, E.N. Lightfoot: Transport Phe-
48 I.S. Sokolnikoff, E.S. Sokolnikoff: Higher Mathematics for nomena, 2nd ed., John Wiley & Sons, New York 2002.
Engineers and Physicists, McGraw-Hill, New York 1941. 74 R.B. Bird, R.C. Armstrong, O. Hassager: Dynamics of Poly-
49 R.C. Wrede, M.R. Spiegel: Schaum’s outline of theory and meric Liquids, 2nd ed., Wiley-Interscience, New York
problems of advanced calculus, 2nd ed, McGraw Hill, New 1987.
York 2006. 75 P.V. Danckwerts: “Continuous Flow Systems,” Chem. Eng. Sci.
50 P. M. Morse, H. Feshbach: Methods of Theoretical Physics, 2 (1953) 1–13.
McGraw-Hill, New York 1953. 76 J.F. Wehner, R. Wilhelm: “Boundary Conditions of Flow
51 B.A. Finlayson: The Method of Weighted Residuals and Vari- Reactor,” Chem. Eng. Sci. 6 (1956) 89–93.
ational Principles, Academic Press, New York 1972. 77 V. Hlavacek, H. Hofmann: “Modeling of Chemical Reactors-
52 G. Forsythe, M. Malcolm, C. Moler: Computer Methods for XVI-Steady-State Axial Heat and Mass Transfer in Tubular
Mathematical Computation, Prentice-Hall, Englewood Cliffs, Reactors. An Analysis of the Uniqueness of Solutions,” Chem.
N.J. 1977. Eng. Sci. 25 (1970) 173–185.
156 Mathematics in Chemical Engineering

78 B.A. Finlayson: Numerical Methods for Problems with Moving 100 R. Aris, N.R. Amundson: Mathematical Methods in Chemi-
Fronts, Ravenna Park Publishing Inc., Seattle 1990. cal Engineering, vol. 2, First-Order Partial Differential
79 E. Isaacson, H.B. Keller: Analysis of Numerical Methods, J. Equations with Applications, Prentice-Hall, Englewood
Wiley and Sons, New York 1966. Cliffs, NJ, 1973.
80 C. Lanczos: “Trigonometric Interpolation of Empirical and 101 R. Courant, D. Hilbert: Methods of Mathematical Physics, vol.
Analytical Functions,” J. Math. Phys. (Cambridge Mass.) 17 I and II, Intersicence, New York 1953, 1962.
(1938) 123–199. 102 P.M. Morse, H. Feshbach: Methods of Theoretical Physics, vol.
81 C. Lanczos: Applied Analysis, Prentice-Hall, Englewood I and II, McGraw-Hill, New York 1953.
Cliffs, N.J. 1956. 103 A.D. Polyanin: Handbook of Linear Partial Differential Equa-
82 J. Villadsen, W.E. Stewart: “Solution of Boundary-Value Prob- tions for Engineers and Scientists, Chapman and Hall/CRC,
lems by Orthogonal Collocation,” Chem. Eng. Sci. 22 (1967) Boca Raton, FL 2002.
1483–1501. 104 D. Ramkrishna, N.R. Amundson: Linear Operator Methods in
83 M.L. Michelsen, J. Villadsen: “Polynomial Solution of Differ- Chemical Engineering with Applications to Transport and
ential Equations” in R.S.H. Mah, W.D. Seider (eds.): Founda- Chemical Reaction Systems, Prentice Hall, Englewood Cliffs,
tions of Computer-Aided Chemical Process Design, NJ, 1985.
Engineering Foundation, New York 1981, pp. 341–368. 105 D.D. Joseph, M. Renardy, J.C. Saut: “Hyperbolicity and
84 W. E. Stewart, K. L. Levien, M. Morari: “Collocation Methods Change of Type in the Flow of Viscoelastic Fluids,” Arch.
in Distillation,” in A. W. Westerberg, H. H. Chien (eds.): Rational Mech. Anal. 87 (1985) 213–251.
Proceedings of the Second Int. Conf. on Foundations of Com- 106 H.K. Rhee, R. Aris, N.R. Amundson: First-Order Partial Differ-
puter-Aided Process Design, Computer Aids for Chemical ential Equations, Prentice-Hall, Englewood Cliffs, N.J. 1986.
Engineering Education (CACHE), Austin Texas, 1984, 107 B.A. Finlayson: Numerical Methods for Problems with Moving
pp. 535–569. Fronts, Ravenna Park Publishing Inc., Seattle 1990.
85 W.E. Stewart, K.L. Levien, M. Morari: “Simulation of Frac- 108 D.L. Book: Finite-Difference Techniques for Vectorized Fluid
tionation by Orthogonal Collocation,” Chem. Eng. Sci. 40 Dynamics Calculations, Springer Verlag, Berlin 1981.
(1985) 409–421. 109 G.A. Sod: Numerical Methods in Fluid Dynamics, Cambridge
86 C.L.E. Swartz, W.E. Stewart: “Finite-Element Steady State University Press, Cambridge 1985.
Simulation of Multiphase Distillation,” AIChE J. 33 (1987) 110 G.H. Xiu, J.L. Soares, P. Li, A.E. Rodriques: “Simulation of
1977–1985. five-step one-bed sorption-endanced reaction process,” AIChE
87 K. Alhumaizi, R. Henda, M. Soliman: “Numerical Analysis of J. 48 (2002) 2817–2832.
a reaction-diffusion-convection system,” Comp. Chem. Eng. 27 111 A. Malek, S. Farooq: “Study of a six-bed pressure swing
(2003) 579–594. adsorption process,” AIChE J. 43 (1997) 2509–2523.
88 E.F. Costa, P.L.C. Lage, E.C. Biscaia, Jr.: “On the numerical 112 R.J. LeVeque: Numerical Methods for Conservation Laws,
solution and optimization of styrene polymerization in tubular Birkh€auser, Basel 1992.
reactors,” Comp. Chem. Eng. 27 (2003) 1591–1604. 113 W.F. Ames: “Recent Developments in the Nonlinear Equations
89 J. Wang, R.G. Anthony, A. Akgerman: “Mathematical simula- of Transport Processes,” Ind. Eng. Chem. Fundam. 8 (1969)
tions of the performance of trickle bed and slurry reactors for 522–536.
methanol synthesis,” Comp. Chem. Eng. 29 (2005) 2474–2484. 114 W.F. Ames: Nonlinear Partial Differential Equations in Engi-
90 V.K.C. Lee, J.F. Porter, G. McKay, A.P. Mathews: “Application neering, Academic Press, New York 1965.
of solid-phase concentration-dependent HSDM to the acid dye 115 W.E. Schiesser: The Numerical Method of Lines, Academic
adsorption system,” AIChE J. 51 (2005) 323–332. Press, San Diego 1991.
91 C. deBoor, B. Swartz: “Collocation at Gaussian Points,” SIAM 116 D. Gottlieb, S.A. Orszag: Numerical Analysis of Spectral
J. Num. Anal. 10 (1973) 582–606. Methods: Theory and Applications, SIAM, Philadelphia, PA
92 R.F. Sincovec: “On the Solution of the Equations Arising From 1977.
Collocation With Cubic B-Splines,” Math. Comp. 26 (1972) 117 D.W. Peaceman: Fundamentals of Numerical Reservoir Simu-
893–895. lation, Elsevier, Amsterdam 1977.
93 U. Ascher, J. Christiansen, R.D. Russell: “A Collocation Solver 118 G. Juncu, R. Mihail: “Multigrid Solution of the Diffusion-
for Mixed-Order Systems of Boundary-Value Problems,” Convection-Reaction Equations which Describe the Mass and/
Math. Comp. 33 (1979) 659–679. or Heat Transfer from or to a Spherical Particle,” Comput.
94 R.D. Russell, J. Christiansen: “Adaptive Mesh Selection Strat- Chem. Eng. 13 (1989) 259–270.
egies for Solving Boundary-Value Problems,” SIAM J. Num. 119 G.R. Buchanan: Schaum’s Outline of Finite Element Analysis,”
Anal. 15 (1978) 59–80. McGraw-Hill, New York 1995.
95 P.G. Ciarlet, M.H. Schultz, R.S. Varga: “Nonlinear Boundary- 120 M.D. Gunzburger: Finite Element Methods for Viscous
Value Problems I. One Dimensional Problem,” Num. Math. 9 Incompressible Flows, Academic Press, San Diego 1989.
(1967) 394–430. 121 H. Kardestuncer, D.H. Norrie: Finite Element Handbook,
96 W.F. Ames: Numerical Methods for Partial Differential Equa- McGraw-Hill, New York 1987.
tions, 2nd ed., Academic Press, New York 1977. 122 J.N. Reddy, D.K. Gartling: The Finite Element Method in Heat
97 J.F. Botha, G.F. Pinder: Fundamental Concepts in The Numer- Transfer and Fluid Dynamics, 2nd ed., CRC Press, Boca Raton,
ical Solution of Differential Equations, Wiley-Interscience, FL 2000.
New York 1983. 123 O.C. Zienkiewicz, R.L. Taylor, J.Z. Zhu: The Finite Element
98 W.H. Press, B.P. Flanner, S.A. Teukolsky, W.T. Vetterling: Method: Its Basis & Fundamentals, vol. 1, 6th ed., Elsevier
Numerical Recipes, Cambridge Univ. Press, Cambridge Butterworth-Heinemann, Burlington, MA 2005.
1986. 124 O.C. Zienkiewicz, R.L. Taylor: The Finite Element Method,
99 H. Schlichting, Boundary Layer Theory, 4th ed. McGraw-Hill, Solid and Structural Mechanics, vol. 2, 5th ed., Butterworth-
New York 1960. Heinemann, Burlington, MA 2000.
 Mathematics in Chemical Engineering 157

125 O.C. Zienkiewicz, R.L. Taylor: The Finite Element Method, 149 C.T.H. Baker: The Numerical Treatment of Integral Equations,
Fluid Dynamics, vol. 3, 5th ed., Butterworth-Heinemann, Clarendon Press, Oxford 1977.
Burlington, MA 2000. 150 L.M. Delves, J. Walsh (eds.): Numerical Solution of Integral
126 Z.C. Li: Combined Methods for Elliptic Equations with Sin- Equations, Clarendon Press, Oxford 1974.
gularities, Interfaces and Infinities, Kluwer Academic Pub- 151 P. Linz: Analytical and Numerical Methods for Volterra Equa-
lishers, Boston, MA, 1998. tions, SIAM Publications, Philadelphia 1985.
127 G.J. Fix, S. Gulati, G.I. Wakoff: “On the use of singular 152 M.A. Golberg (ed.): Numerical Solution of Integral Equations,
functions with finite element approximations,” J. Comp. Plenum Press, New York 1990.
Phys. 13 (1973) 209–238. 153 C. Corduneanu: Integral Equations and Applications, Cam-
128 N.M. Wigley: “On a method to subtract off a singularity at a bridge Univ. Press, Cambridge 1991.
corner for the Dirichlet or Neumann problem,” Math. Comp. 23 154 R. Kress: Linear Integral Equations, 2nd ed. Springer, Heidel-
(1968) 395–401. berg 1999.
129 H.Y. Hu, Z.C. Li, A.H.D. Cheng: “Radial Basis Collocation 155 P.K. Kythe P. Purl: Computational Methods for Linear Integral
Methods for Elliptic Boundary Value Problems,” Comp. Math. Equations, Birkh€auser, Basel 2002.
Applic. 50 (2005) 289–320. 156 G. Nagel, G. Kluge: “Non-Isothermal Multicomponent
130 S.J. Osher, R.P. Fedkiw:, Level Set Methods and Dynamic Adsorption Processes and Their Numerical Treatment by
Implicit Surfaces, Springer, New York 2002. Means of Integro-Differential Equations,” Comput. Chem.
131 M.W. Chang, B.A. Finlayson: “On the Proper Boundary Con- Eng. 13 (1989) 1025–1030.
dition for the Thermal Entry Problem,” Int. J. Num. Methods 157 P.L. Mills, S. Lai, M.P. Dudukovic, P.A. Ramachandran: “A
Eng. 15 (1980) 935–942. Numerical Study of Approximation Methods for Solution of
132 R. Peyret, T.D. Taylor: Computational Methods for Fluid Flow, Linear and Nonlinear Diffusion-Reaction Equations with Dis-
Springer Verlag, Berlin–Heidelberg–New York–Tokyo 1983. continuous Boundary Conditions,” Comput. Chem. Eng. 12
133 P.M. Gresho, S.T. Chan, C. Upson, R.L. Lee: “A Modified (1988) 37–53.
Finite Element Method for Solving the Time-Dependent, 158 D. Duffy: Green’s Functions with Applications, Chapman and
Incompressible Navier–Stokes Equations,” Int. J. Num. Hall/CRC, New York 2001.
Method. Fluids (1984) “Part 1. Theory”: 557–589; “Part 2: 159 I. Statgold: Green’s Functions and Boundary Value Problems,
Applications”: 619–640. 2nd ed., Interscience, New York 1997.
134 P.M. Gresho, R.L. Sani: Incompressible Flow and the Finite 160 B. Davies: Integral Transforms and Their Applications, 3rd ed.,
Element Method, Advection-Diffusion, vol. 1, John Wiley & Springer, New York 2002.
Sons, New York 1998. 161 J.M. Bownds, “Theory and performance of a subroutine for
135 P.M. Gresho, R.L. Sani: Incompressible Flow and the Finite solving Volterra integral equations,” J. Comput. 28 317–332
Element Method , Isothermal Laminar Flow, vol. 2, John Wiley (1982).
& Sons, New York 1998. 162 J.G. Blom, H. Brunner, “Algorithm 689; discretized collo-
136 J.A. Sethian, Level Set Methods, Cambridge University Press, cation and iterated collocation for nonlinear Volterra integral
Cambridge 1996. equations of the second kind,” ACM Trans. Math. Software
137 C.C. Lin, H. Lee, T. Lee, L.J. Weber: “A level set characteristic 17 (1991) 167–177.
Galerkin finite element method for free surface flows,” Int. J. 163 N.B. Ferguson, B.A. Finlayson: “Error Bounds for Approxi-
Num. Methods Fluids 49 (2005) 521–547. mate Solutions to Nonlinear Ordinary Differential Equations,”
138 S. Succi: The Lattice Boltzmann Equation for Fluid Dynamics AIChE J. 18 (1972) 1053–1059.
and Beyond, Oxford University Press, Oxford 2001. 164 R.S. Dixit, L.L. Taularidis: “Integral Method of Analysis of
139 M.C. Sukop, D.T. Thorne, Jr.: Lattice Boltzmann Modeling: An Fischer–Tropsch Synthesis Reactions in a Catalyst Pellet,”
Introduction for Geoscientists and Engineers, Springer, New Chem. Eng. Sci. 37 (1982) 539–544.
York 2006. 165 L. Lapidus, G.F. Pinder: Numerical Solution of Partial Differ-
140 S. Chen, S.G.D. Doolen: “Lattice Boltzmann method for fluid ential Equations in Science and Engineering, Wiley-Inter-
flows,” Annu. Rev. Fluid Mech. 30 (2001) 329–364. science, New York 1982.
141 H.T. Lau: Numerical Library in C for Scientists and Engineers, 166 C.K. Hsieh, H. Shang: “A Boundary Condition Dissection
CRC Press, Boca Raton, FL 1994. Method for the Solution of Boundary-Value Heat Conduction
142 Y.Y. Al-Jaymany, G. Brenner, P.O. Brum: “Comparative study Problems with Position-Dependent Convective Coefficients,”
of lattice-Boltzmann and finite volume methods for the simu- Num. Heat Trans. Part B: Fund. 16 (1989) 245–255.
lation of laminar flow through a 4:1 contraction,” Int. J. Num. 167 C.A. Brebbia, J. Dominguez: Boundary Elements–An Intro-
Methods Fluids 46 (2004) 903–920. ductory Course, 2nd ed., Computational Mechanics Publica-
143 R.L. Burden, R. L J. D. Faires, A.C. Reynolds: Numerical tions, Southhamtpon 1992.
Analysis, 8th ed., Brooks Cole, Pacific Grove, CA 2005. 168 J. Mackerle, C.A. Brebbia (eds.): Boundary Element Reference
144 S.C. Chapra, R.P. Canal: Numerical Methods for Engineers,” Book, Springer Verlag, Berlin 1988.
5th ed., McGraw-Hill, New York 2006. 169 L.T. Biegler, I.E. Grossmann, A.W. Westerberg: Systematic
145 H.T. Lau: Numerical Library in Java for Scientists and Engi- Methods for Chemical Process Design, Prentice-Hall Engle-
neers, CRC Press, Boca Raton, FL 2004. wood Cliffs, NJ 1997.
146 K.W. Morton, D.F. Mayers: Numerical Solution of Partial 170 I.E. Grossmann (ed.), “Global Optimization in Engineering
Differential Equations, Cambridge University Press, New Design”, Kluwer, Dordrecht 1996.
York 1994. 171 J. Kallrath, “Mixed Integer Optimization in the Chemical
147 A. Quarteroni, A. Valli: Numerical Approximation of Partial Process Industry: Experience, Potential and Future,” Trans.
Differential Equations, Springer, Heidelberg 1997. I. Chem. E., 78 (2000) Part A, 809–822.
148 F. Scheid: “Schaum’s Outline of Numerical Analysis,” 2nd ed., 172 O.L. Mangasarian: Nonlinear Programming, McGraw-Hill,
McGraw-Hill, New York 1989. New York 1969.
158 Mathematics in Chemical Engineering

173 G.B. Dantzig: Linear Programming and Extensions, Princeton 195 gPROMS User’s Guide, PSE Ltd., London 2002.
University Press, Princeton, N.J, 1963. 196 G.E.P. Box: “Evolutionary Operation: A Method for Increasing
174 S.J. Wright: Primal-Dual Interior Point Methods, SIAM, Industrial Productivity,” Applied Statistics 6 (1957) 81–101.
Philadelphia 1996. 197 R. Hooke, T.A. Jeeves: “Direct Search Solution of Numerical
175 F. Hillier, G.J. Lieberman: Introduction to Operations and Statistical Problems,” J. ACM 8 (1961) 212.
Research, Holden-Day, San Francisco, 1974. 198 M.J.D. Powell: “An Efficient Method for Finding the Mini-
176 T.F. Edgar, D.M. Himmelblau, L.S. Lasdon: Optimization of mum of a Function of Several Variables without Calculating
Chemical Processes, McGraw-Hill Inc., New York 2002. Derivatives, Comput. J. 7 (1964) 155.
177 K. Schittkowski: More Test Examples for Nonlinear Program- 199 J.A. Nelder, R. Mead: “A Simplex Method for Function
ming Codes, Lecture notes in economics and mathematical Minimization,” Computer Journal 7 (1965) 308.
systems no. 282, Springer-Verlag, Berlin 1987. 200 R. Luus, T.H.I. Jaakola: “Direct Search for Complex Systems,”
178 J. Nocedal, S.J. Wright: Numerical Optimization, Springer, AIChE J 19 (1973) 645–646.
New York 1999. 201 R. Goulcher, J.C. Long: “The Solution of Steady State Chemi-
179 R. Fletcher: Practical Optimization, John Wiley & Sons, Ltd., cal Engineering Optimization Problems using a Random
Chichester, UK 1987 Search Algorithm,” Comp. Chem. Eng. 2 (1978) 23.
180 A. W€achter, L.T. Biegler: “On the Implementation of an 202 J.R. Banga, W.D. Seider: in “Global Optimization of Chemical
Interior Point Filter Line Search Algorithm for Large-Scale Processes using Stochastic Algorithms,” C. Floudas, P. Parda-
Nonlinear Programming,” 106 Mathematical Programming los (eds.): State of the Art in Global Optimization, Kluwer,
(2006) no. 1, 25–57. Dordrecht 1996, p. 563.
181 C.V. Rao, J.B. Rawlings S. Wright: On the Application of 203 P.J.M. van Laarhoven, E.H.L. Aarts: Simulated Annealing:
Interior Point Methods to Model Predictive Control. J. Optim. Theory and Applications, Reidel Publishing, Dordrecht 1987.
Theory Appl. 99 (1998) 723. 204 J.H. Holland: Adaptations in Natural and Artificial Systems,
182 J. Albuquerque, V. Gopal, G. Staus, L.T. Biegler, B.E. Ydstie: University of Michigan Press, Ann Arbor 1975.
“Interior point SQP Strategies for Large-scale Structured Pro- 205 J.E. Dennis, V. Torczon: “Direct Search Methods on Parallel
cess Optimization Problems,” Comp. Chem. Eng. 23 (1997) Machines,” SIAM J. Opt. 1 (1991) 448.
283. 206 A.R. Conn, K. Scheinberg, P. Toint: “Recent Progress in
183 T. Jockenhoevel, L.T. Biegle, A. W€achter: “Dynamic Optimi- Unconstrained Nonlinear Optimization without Deriva-
zation of the Tennessee Eastman Process Using the tives,” Mathemtatical Programming, Series B, 79 (1997)
OptControlCentre,” Comp. Chem. Eng. 27 (2003) no. 11, no. 3, 397.
1513–1531. 207 M. Eldred: “DAKOTA: A Multilevel Parallel Object-Oriented
184 L.T. Biegler, A.M. Cervantes, A. W€achter: “Advances in Framework for Design Optimization, Parameter Estimation,
Simultaneous Strategies for Dynamic Process Optimization,” Uncertainty Quantification, and Sensitivity Analysis,” 2002,
Chemical Engineering Science 57 (2002) no. 4, 575–593. http://endo.sandia.gov/DAKOTA/software.html
185 C.D. Laird, L.T. Biegler, B. van Bloemen Waanders, R.A. 208 A.J. Booker et al.: “A Rigorous Framework for Optimization of
Bartlett: “Time Dependent Contaminant Source Determination Expensive Functions by Surrogates,” CRPC Technical Report
for Municipal Water Networks Using Large Scale Opti- 98739, Rice University, Huston TX, February 1998.
mization,” ASCE Journal of Water Resource Management 209 R. Horst, P.M. Tuy: “Global Optimization: Deterministic
and Planning 131 (2005) no. 2, 125. Approaches,” 3rd ed., Springer Verlag, Berlin 1996.
186 A.R. Conn, N. Gould, P. Toint: Trust Region Methods, SIAM, 210 I. Quesada, I.E. Grossmann: “A Global Optimization Algo-
Philadelphia 2000. rithm for Linear Fractional and Bilinear Programs,” Journal of
187 A. Drud: “CONOPT–A Large Scale GRG Code,” ORSA Global Optimization 6 (1995) no. 1, 39–76.
Journal on Computing 6 (1994) 207–216. 211 G.P. McCormick: “Computability of Global Solutions to Fac-
188 B.A. Murtagh, M.A. Saunders: MINOS 5.1 User’s Guide, torable Nonconvex Programs: Part I–Convex Underestimating
Technical Report SOL 83-20R, Stanford University, Palo Problems,” Mathematical Programming 10 (1976) 147–175.
Alto 1987. 212 H.S. Ryoo, N.V. Sahinidis: “Global Optimization of Noncon-
189 R.H. Byrd, M.E. Hribar, J. Nocedal: An Interior Point Algo- vex NLPs and MINLPs with Applications in Process Design,”
rithm for Large Scale Nonlinear Programming, SIAM J. Opt. 9 Comp. Chem. Eng. 19 (1995) no. 5, 551–566.
(1999) no. 4, 877. 213 M. Tawarmalani, N.V. Sahinidis: “Global Optimization of
190 R. Fletcher, N.I.M. Gould, S. Leyffer, Ph. L. Toint, A. Mixed Integer Nonlinear Programs: A Theoretical and Com-
Waechter: “Global Convergence of a Trust-region (SQP)-filter putational Study,” Mathematical Programming 99 (2004) no.
Algorithms for General Nonlinear Programming,” SIAM J. 3, 563–591.
Opt. 13 (2002) no. 3, 635–659. 214 C.S. Adjiman., I.P. Androulakis, C.A. Floudas: “Global Opti-
191 R.J. Vanderbei, D.F. Shanno: An Interior Point Algorithm for mization of MINLP Problems in Process Synthesis and Design.
Non-convex Nonlinear Programming. Technical Report SOR- Comp. Chem. Eng., 21 (1997) Suppl., S445–S450.
97-21, CEOR, Princeton University, Princeton, NJ, 1997. 215 C.S. Adjiman, I.P. Androulakis, C.A. Floudas: “Global Opti-
192 P.E. Gill, W. Murray, M. Wright: Practical Optimization, mization of Mixed-Integer Nonlinear Problems,” AIChE Jour-
Academic Press, New York 1981. nal, 46 (2000) no. 9, 1769–1797.
193 P.E. Gill, W. Murray, M.A. Saunders: User’s guide for SNOPT: 216 C.S. Adjiman, I.P. Androulakis, C.D. Maranas, C.A. Floudas:
A FORTRAN Package for Large-scale Nonlinear Program- “A Global Optimization Method, aBB, for Process Design,”
ming, Technical report, Department of Mathematics, Univer- Comp. Chem. Eng., 20 (1996) Suppl., S419–S424.
sity of California, San Diego 1998. 217 J.M. Zamora, I.E. Grossmann: “A Branch and Contract Algo-
194 J.T. Betts: Practical Methods for Optimal Control Using rithm for Problems with Concave Univariate, Bilinear and
Nonlinear Programming, Advances in Design and Control 3, Linear Fractional Terms,” Journal of Gobal Optimization 14
SIAM, Philadelphia 2001. (1999) no. 3, 217–249.
 Mathematics in Chemical Engineering 159

218 E.M.B. Smith, C.C. Pantelides: “Global Optimization of 239 J.F. Benders: “Partitioning Procedures for Solving Mixed-
Nonconvex NLPs and MINLPs with Applications in Process variables Programming Problems,” Numeri. Math. 4 (1962)
Design,” Comp. Chem. Eng., 21 (1997) no. 1001, 238–252.
S791–S796. 240 G.L. Nemhauser, L.A. Wolsey: Integer and Combinatorial
219 J.E. Falk, R.M. Soland: “An Algorithm for Separable Non- Optimization, Wiley-Interscience, New York 1988.
convex Programming Problems,” Management Science 15 241 C. Barnhart, E.L. Johnson, G.L. Nemhauser, M.W.P. Savels-
(1969) 550–569. bergh, P.H. Vance: “Branch-and-price: Column Generation for
220 F.A. Al-Khayyal, J.E. Falk: “Jointly Constrained Biconvex Solving Huge Integer Programs,” Operations Research 46
Programming,” Mathematics of Operations Research 8 (1998) 316–329.
(1983) 273–286. 242 O.K. Gupta, V. Ravindran: “Branch and Bound Experiments in
221 F.A. Al-Khayyal: “Jointly Constrained Bilinear Programs and Convex Nonlinear Integer Programming,” Management Sci-
Related Problems: An Overview,” Computers and Mathemat- ence 31 (1985) no. 12, 1533–1546.
ics with Applications, 19 (1990) 53–62. 243 S. Nabar, L. Schrage: “Modeling and Solving Nonlinear
222 I. Quesada, I.E. Grossmann: “A Global Optimization Algo- Integer Programming Problems,” AIChE Meeting, Chicago
rithm for Heat Exchanger Networks,” Ind. Eng. Chem. Res. 32 1991.
(1993) 487–499. 244 B. Borchers, J.E. Mitchell: “An Improved Branch and Bound
223 H.D. Sherali, A. Alameddine: “A New Reformulation-Linear- Algorithm for Mixed Integer Nonlinear Programming,” Com-
ization Technique for Bilinear Programming Problems,” Jour- puters and Operations Research 21 (1994) 359–367.
nal of Global Optimization 2 (1992) 379–410. 245 R. Stubbs, S. Mehrotra: “A Branch-and-Cut Method for 0-1
224 H.D. Sherali, C.H. Tuncbilek: “A Reformulation-Convexi- Mixed Convex Programming,” Mathematical Programming 86
fication Approach for Solving Nonconvex Quadratic Pro- (1999) no. 3, 515–532.
gramming Problems,” Journal of Global Optimization 7 246 S. Leyffer: “Integrating SQP and Branch and Bound for Mixed
(1995) 1–31. Integer Noninear Programming,” Computational Optimization
225 C.A. Floudas: Deterministic Global Optimization: Theory, and Applications 18 (2001) 295–309.
Methods and Applications, Kluwer Academic Publishers, Dor- 247 A.M. Geoffrion: “Generalized Benders Decomposition,” Jour-
drecht, The Netherlands, 2000. nal of Optimization Theory and Applications 10 (1972) no. 4,
226 M. Tawarmalani, N. Sahinidis: Convexification and Global 237–260.
Optimization in Continuous and Mixed-Integer Nonlinear 248 M.A. Duran, I.E. Grossmann: “An Outer-Approximation Algo-
Programming: Theory, Algorithms, Software, and Applica- rithm for a Class of Mixed-integer Nonlinear Programs,” Math
tions, Kluwer Academic Publishers, Dordrecht 2002. Programming 36 (1986) 307.
227 R. Horst, H. Tuy: Global optimization: Deterministic 249 X. Yuan, S. Zhang, L. Piboleau, S. Domenech : “Une Methode
approaches. Springer-Verlag, Berlin 1993. d’optimisation Nonlineare en Variables Mixtes pour la Con-
228 A. Neumaier, O. Shcherbina, W. Huyer, T. Vinko: “A Compar- ception de Procedes,” RAIRO 22 (1988) 331.
ison of Complete Global Optimization Solvers,” Math. Pro- 250 R. Fletcher, S. Leyffer: “Solving Mixed Integer Nonlinear
gramming B 103 (2005) 335–356. Programs by Outer Approximation,” Math Programming 66
229 L. Lovasz A. Schrijver: “Cones of Matrices and Set-functions (1974) 327.
and 0-1 Optimization,” SIAM J. Opt. 1 (1991) 166–190. 251 I. Quesada, I.E. Grossmann: “An LP/NLP Based Branch and
230 H.D. Sherali, W.P. Adams: “A Hierarchy of Relaxations Bound Algorithm for Convex MINLP Optimization Prob-
Between the Continuous and Convex Hull Representations lems,” Comp. Chem. Eng. 16 (1992) 937–947.
for Zero-One Programming Problems,” SIAM J. Discrete 252 T. Westerlund, F. Pettersson: “A Cutting Plane Method for
Math. 3 (1990) no. 3, 411–430. Solving Convex MINLP Problems,” Comp. Chem. Eng. 19
231 E. Balas, S. Ceria, G. Cornuejols: “A Lift-and-Project Cutting (1995) S131–S136.
Plane Algorithm for Mixed 0-1 Programs,” Mathematical 253 O.E. Flippo, A.H.G.R. Kan: “Decomposition in General Math-
Programming 58 (1993) 295–324. ematical Programming,” Mathematical Programming 60
232 A.H. Land, A.G. Doig: “An Automatic Method for Solving (1993) 361–382.
Discrete Programming Problems,” Econometrica 28 (1960) 254 T.L. Magnanti, R.T. Wong: “Acclerated Benders Decomposi-
497–520. tion: Algorithm Enhancement and Model Selection Criteria,”
233 E. Balas: “An Additive Algorithm for Solving Linear Programs Operations Research 29 (1981) 464–484.
with Zero-One Variables,” Operations Research 13 (1965) 255 N.V. Sahinidis, I.E. Grossmann: “Convergence Properties of
517–546. Generalized Benders Decomposition,” Comp. Chem. Eng. 15
234 R.J. Dakin: “A Tree Search Algorithm for Mixed-Integer Pro- (1991) 481.
gramming Problems”, Computer Journal 8 (1965) 250–255. 256 J.E. Kelley Jr.: “The Cutting-Plane Method for Solving Convex
235 R.E. Gomory: “Outline of an Algorithm for Integer Solutions Programs,” J. SIAM 8 (1960) 703–712.
to Linear Programs,” Bulletin of the American Mathematics 257 S. Leyffer: “Deterministic Methods for Mixed-Integer Non-
Society 64 (1958) 275–278. linear Programming,” Ph.D. thesis, Department of Mathemat-
236 H.P. Crowder, E.L. Johnson, M.W. Padberg: “Solving Large- ics and Computer Science, University of Dundee, Dundee
Scale Zero-One Linear Programming Problems,” Operations 1993.
Research 31 (1983) 803–834. 258 M.S. Bazaraa, H.D. Sherali, C.M. Shetty: Nonlinear Program-
237 T.J. Van Roy, L.A. Wolsey: “Valid Inequalities for Mixed 0-1 ming, John Wiley & Sons, Inc., New York 1994.
Programs,” Discrete Applied Mathematics 14 (1986) 259 G.R. Kocis, I.E. Grossmann: “Relaxation Strategy for the
199–213. Structural Optimization of Process Flowsheets,” Ind. Eng.
238 E.L. Johnson, G.L. Nemhauser, M.W.P. Savelsbergh: “Progress Chem. Res. 26 (1987) 1869.
in Linear Programming Based Branch-and-Bound Algorithms: 260 I.E. Grossmann: “Mixed-Integer Optimization Techniques for
Exposition,” INFORMS Journal of Computing 12 (2000) 2–23. Algorithmic Process Synthesis”, Advances in Chemical
160 Mathematics in Chemical Engineering

Engineering, vol. 23, Process Synthesis, Academic Press, 282 ILOG, Gentilly Cedex, France 1999, www.ilog.com,
London 1996, pp. 171–246. 283 M. Dincbas, P. Van Hentenryck, H. Simonis, A. Aggoun, T.
261 E.M.B. Smith, C.C. Pantelides: “A Symbolic Reformulation/ Graf, F. Berthier: The Constraint Logic Programming Lan-
Spatial Branch and Bound Algorithm for the Global Optimi- guage CHIP, FGCS-88: Proceedings of International Con-
zation of Nonconvex MINLPs,” Comp. Chem. Eng. 23 (1999) ference on Fifth Generation Computer Systems, Tokyo,
457–478. 693–702.
262 P. Kesavan P.P.I. Barton: “Generalized Branch-and-cut Frame- 284 M. Wallace, S. Novello, J. Schimpf: “ECLiPSe: a Platform for
work for Mixed-integer Nonlinear Optimization Problems,” Constraint Logic Programming,” ICL Systems Journal 12
Comp. Chem. Eng. 24 (2000) 1361–1366. (1997) no.1, 159–200.
263 S. Lee I.E. Grossmann: “A Global Optimization Algorithm for 285 V. Pontryagin, V. Boltyanskii, R. Gamkrelidge, E. Mishchenko:
Nonconvex Generalized Disjunctive Programming and Appli- The Mathematical Theory of Optimal Processes, Interscience
cations to Process Systems,” Comp. Chem. Eng. 25 (2001) Publishers Inc., New York, NY, 1962.
1675–1697. 286 A.E. Bryson, Y.C. Ho: “Applied Optimal Control: Optimiza-
264 R. P€orn, T. Westerlund: “A Cutting Plane Method for Mini- tion, Estimation, and Control,” Ginn and Company, Waltham,
mizing Pseudo-convex Functions in the Mixed-integer Case,” MA, 1969.
Comp. Chem. Eng. 24 (2000) 2655–2665. 287 V. Vassiliadis, PhD Thesis, Imperial College, University of
265 J. Viswanathan, I.E. Grossmann: “A Combined Penalty Func- London 1993.
tion and Outer-Approximation Method for MINLP Opti- 288 W.F. Feehery, P.I. Barton: “Dynamic Optimization with State
mization,” Comp. Chem. Eng. 14 (1990) 769. Variable Path Constraints,” Comp. Chem. Eng., 22 (1998)
266 A. Brooke, D. Kendrick, A. Meeraus, R. Raman: GAMS–A 1241–1256.
User’s Guide, www.gams.com, 1998. 289 L. Hasdorff:. Gradient Optimization and Nonlinear Control,
267 N.V.A. Sahinidis, A. Baron: “A General Purpose Global Opti- Wiley-Interscience, New York, NY, 1976.
mization Software Package,” Journal of Global Optimization 8 290 R.W.H. Sargent, G.R. Sullivan: “Development of Feed
(1996) no.2, 201–205. Changeover Policies for Refinery Distillation Units,” Ind.
268 C.A. Schweiger, C.A. Floudas: “Process Synthesis, Design and Eng. Chem. Process Des. Dev. 18 (1979) 113–124.
Control: A Mixed Integer Optimal Control Framework,” Pro- 291 V. Vassiliadis, R.W.H. Sargent, C. Pantelides: “Solution of a
ceedings of DYCOPS-5 on Dynamics and Control of Process Class of Multistage Dynamic Optimization Problems,” I & EC
Systems, Corfu, Greece 1998, pp. 189–194. Research 33 (1994) 2123.
269 R. Raman, I.E. Grossmann: “Modelling and Computational 292 K.F. Bloss, L.T. Biegler, W.E. Schiesser: “Dynamic Process
Techniques for Logic Based Integer Programming,” Comp. Optimization through Adjoint Formulations and Constraint
Chem. Eng. 18 (1994) no.7, 563. Aggregation,”. Ind. Eng. Chem. Res. 38 (1999) 421–432.
270 J.N. Hooker, M.A. Osorio: “Mixed Logical.Linear Pro- 293 H.G. Bock, K.J. Plitt: A Multiple Shooting Algorithm for Direct
gramming,” Discrete Applied Mathematics 96-97 (1994) Solution of Optimal Control Problems, 9th IFAC World Con-
395–442. gress, Budapest 1984.
271 P.V. Hentenryck: Constraint Satisfaction in Logic Program- 294 D.B. Leineweber, H.G. Bock, J.P. Schl€oder, J.V. Gallitzend€or-
ming, MIT Press, Cambridge, MA, 1989. fer, A. Sch€afer, P. Jansohn: “A Boundary Value Problem
272 J.N. Hooker: Logic-Based Methods for Optimization: Combin- Approach to the Optimization of Chemical Processes
ing Optimization and Constraint Satisfaction, John Wiley & Described by DAE Models,” Computers & Chemical Engi-
Sons, New York 2000. neering, April 1997. (IWR-Preprint 97-14, Universit€at Heidel-
273 J.N. Hooker: Logic-Based Methods for Optimization, John berg, March 1997.
Wiley & Sons, New York 1999. 295 J.E. Cuthrell, L.T. Biegler: “On the Optimization of Differen-
274 E. Balas: “Disjunctive Programming,” Annals of Discrete tial- algebraic Process Systems,” AIChE Journal 33 (1987)
Mathematics 5 (1979) 3–51. 1257–1270.
275 H.P. Williams: Mathematical Building in Mathematical Pro- 296 A. Cervantes, L.T. Biegler: “Large-scale DAE Optimization
gramming, John Wiley, Chichester 1985. Using Simultaneous Nonlinear Programming Formulations.
276 R. Raman, I.E. Grossmann: “Relation between MILP Model- AIChE Journal 44 (1998) 1038.
ling and Logical Inference for Chemical Process Synthesis,” 297 P. Tanartkit, L.T. Biegler: “Stable Decomposition for Dynamic
Comp. Chem. Eng. 15 (1991) no. 2, 73. Optimization,” Ind. Eng. Chem. Res. 34 (1995) 1253–1266.
277 S. Lee, I.E. Grossmann: “New Algorithms for Nonlinear 298 S. Kameswaran, L.T. Biegler: “Simultaneous Dynamic Opti-
Generalized Disjunctive Programming,” Comp. Chem. Eng. mization Strategies: Recent Advances and Challenges,” Chem-
24 (2000) no. 9-10, 2125–2141. ical Process Control–7, to appear 2006.
278 J. Hiriart-Urruty, C. Lemarechal: Convex Analysis and Mini- 299 B. Bloemen Waanders, R. Bartlett, K. Long, P. Boggs, A.
mization Algorithms, Springer-Verlag, Berlin 1993. Salinger: “Large Scale Non-Linear Programming for PDE
279 E. Balas: “Disjunctive Programming and a Hierarchy of Constrained Optimization,” Sandia Technical Report
Relaxations for Discrete Optimization Problems,” SIAM J. SAND2002-3198, October 2002.
Alg. Disc. Meth. 6 (1985) 466–486. 300 I. Das, J.E. Dennis: “A closer look at drawbacks of minimizing
280 I.E. Grossmann, S. Lee: “Generalized Disjunctive Program- weighted sums of objectives for pareto set generation in multi-
ming: Nonlinear Convex Hull Relaxation and Algorithms,” criteria optimization problems”, Structural optimization 14
Computational Optimization and Applications 26 (2003) (1997) no. 1, 63–69.
83–100. 301 B. Schandl, K. Klamroth, M.M. Wiecek: “Norm-based approx-
281 S. Lee, I.E. Grossmann: “Logic-based Modeling and imation in bicriteria programming”, Computational Optimiza-
Solution of Nonlinear Discrete/Continuous Optimization tion and Applications 20 (2001) no. 1, 23–42.
Problems,” Annals of Operations Research 139 2005 302 M. Bortz, J. Burger, N. Asprion, S. Blagov, R. B€ottcher, U.
267–288. Nowak, A. Scheithauer, R. Welke, H. Hasse, K.-H. K€ufer:
 Mathematics in Chemical Engineering 161

“Multi-criteria optimization in chemical process design and 321 A. Shapiro, D. Dentcheva, A. Ruszczy’nski: Lectures on
decision support by navigation on pareto sets”, Comp. Chem. Stochastic Programming–Modeling and Theory. 2nd ed.,
Eng. 60 (2014) 354–363. MOS-SIAM, Philadelphia 2014.
303 C. Hillermeier: “Nonlinear Multiobjective Optimization: A 322 N.V. Sahinidis: “Optimization under uncertainty: state-of-the-
Generalized Homotopy Approach”, International Series of art and opportunities”, Comp. & Chem. Eng. 28 (2004)
Numerical Mathematics, Springer, New York 2001. 971–983.
304 Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, 323 R. Henrion, P. Li, A. M€oller, M.C. Steinbach, M. Wendt, G.
J., Gatelli, D. Saisana, M., Tarantola, S.: Global Sensitivity Wozny: “Stochastic Optimization for Operating Chemical
Analysis - The Primer. Wiley, New York 2008. Processes under Uncertainty”, Online Optimization of Large
305 A. Saltelli, S. Tarantola, F. Campolongo, M. Ratto: Sensitivity Scale Systems, Springer-Verlag, Berlin Heidelberg 2001,
Analysis in Practice: A Guide to Assessing Scientific Models, pp. 457–478.
Wiley, New York 2004. 324 AIMMS Stochastic Programming Software: http://www
306 SimLab, http://ipsc.jrc.ec.europa.eu/?id=756 (accessed 1st .aimms.com/operations-research/mathematical-programming/
September 2015). stochastic-programming/ (accessed 1st September 2015).
307 Sensitivity Analysis Excel Add-In: http://www.life-cycle- 325 SLP-IOR: http://www.business.uzh.ch/professorships/qba/
costing.de/sensitivity_analysis/ (accessed 1st September 2015). publications/stochOpt.html (accessed 1st September 2015).
308 MUCM Project: http://mucm.ac.uk/index.html (accessed 1st 326 Stochastic Modeling Interface: http://www.coin-or.org/projects/
September 2015). Smi.xml (accessed 1st September 2015).
309 SALib: http://jdherman.github.io/SALib/ (accessed 1st Sep- 327 NEOS solver for stochastic linear programming: http://www
tember 2015). .neos-server.org/neos/solvers/index.html (accessed 1st Sep-
310 R. Hettich, K.O. Kortanek: “Semi-infinite programming– tember 2015).
Theory, methods, and applications”, SIAM Rev. 35 (1993) 328 AMPLDev SP Edition: http://www.optiriskindia.com/products/
380–429. ampl-dev-sp-edition-fort-sp (accessed 1st September 2015).
311 F. Guerra Vazquez, J.-J. R€uckmann, O. Stein, G. Still: 329 S. Zenios, A. Consiglio, S.S. Nielsen: “Mean-variance portfo-
“Generalized semi-infinite programming–A tutorial”, J. lio optimization”, Practical Financial Optimization–A Library
Comp. Appl. Math. 217 (2008) no. 2, 394–419. of GAMS Models, Wiley, New York 2009.
312 D. Bertsimas, D.B. Brown, C. Caramanis: “Theory and appli- 330 M. Suh, T. Lee: “Robust optimization method for the economic
cations of robust optimization”, SIAM Rev. 53 (2011) no. 3, term in chemical process design and planning”, Ind. Eng.
464–501. Chem. Res. 40 (2001) no. 25, 5950–5959.
313 A. Ben-Tal, L. El Ghaoui, A. Nemirovski: Robust Optimiza- 331 G.P. Box, J.S. Hunter, W.G. Hunter: Statistics for Experiment-
tion. Princeton University Press, 2009. ers: Design, Innovation, and Discovery, 2nd ed., John Wiley &
314 A. Ben-Tal, A. Nemirovski: “Robust optimization–method- Sons, New York 2005.
ology and applications”, Math. Prog. 92 (2002) no. 3, 332 D.C. Baird: Experimentation: An Introduction to Measurement
453–480. Theory and Experiment Design, 3rd ed., Prentice Hall, Engel-
315 AIMMS Robust Programming Software: http://www.aimms. wood Cliffs, NJ, 1995.
com/operations-research/mathematical-programming/robust- 333 J.B. Cropley: “Heuristic Approach to Comples Kinetics,”
optimization/ (accessed 1st September 2015). ACS Symp. Ser. 65 (1978) 292–302.
316 Robust Optimization with YALMIP: http://users.isy.liu.se/ 334 S. Lipschutz, J.J. Schiller, Jr: Schaum’s Outline of Theory and
johanl/yalmip/pmwiki.php?n=Tutorials.RobustOptimization Problems of Introduction to Probability and Statistics,
(accessed 1st September 2015). McGraw-Hill, New York 1988.
317 ROME - Robust Optimization Made Easy: http://www.robustopt 335 D.S. Moore, G.P. McCabe: Introduction to the Practice of
.com/ (accessed 1st September 2015). Statistics,” 4th ed., Freeman, New York 2003.
318 J.R. Birge, F. Louveaux: Introduction to Stochastic Program- 336 D.C. Montgomery, G.C. Runger: Applied Statistics and Prob-
ming. 2nd ed., Springer, Heidelberg 2011. ability for Engineers, 3 rd ed., John Wiley & Sons, New York
319 P. Kall, S.W. Wallace: Stochastic Programming–Numerical 2002.
Techniques and Engineering Applications, Springer, Heidel- 337 D.C. Montgomery, G.C. Runger, N.F. Hubele: Engineering
berg 1994. Statistics, 3rd ed., John Wiley & Sons, New York 2004.
320 A. Ruszczy’n ski, A. Shapiro (eds.): “Stochastic Pro- 338 G.E.P. Box, W.G. Hunter, J.S. Hunter: Statistics for Experi-
gramming”, Handbook in Operations Research and Manage- menters, John Wiley & Sons, New York 1978.
ment Science, Vol. 10, Elsevier, Amsterdam 2003.