Verification and Validation in CFD

1. Introduction
2. Numerical error
3. Code Verification
4. Solution Verification
5. Validation
6. Final remarks
Verification and Validation in CFD
In most flows of practical engineering interest, there are no
analytical solutions for the equations that govern the flow
(Continuity and Navier-Stokes equations). At high Reynolds
numbers, turbulent flow, analytical solutions are inexistent
Computational Fluid Dynamics (CFD) is an established
alternative. However, the ability to produce a numerical
solution is (no longer) not enough. The credebility of CFD
as an Engineering tool depends on the quantification of
the error/uncertainty of the results
1. Introduction
Verification and Validation in CFD
Verification and Validation are current research
topics related to the quantification of the
numerical error/uncertainty (Verification) and of
the modelling error (Validation)
The Reynolds averaged Navier-Stokes (RANS) equations
are still the most popular mathematical model in flows
at high Reynolds numbers (10
6
to 10
9
). Therefore,
most of the examples presented are related to RANS
solutions. Nevertheless, most of the problems discussed
in this presentation are common to other areas of
Computational Mechanics
1. Introduction
Verification and Validation in CFD
Round-off error
Iterative error
Discretization error
The three components of the numerical error behave
in a different way with the increase of the number of
degrees of freedom (grid refinement)
2. Numerical error
Verification and Validation in CFD
Round-off error:
> Due to the finite precision of computers
> Double precision reduces its influence (or 64 bits machines)
> In ill-conditioned problems it may become the dominant
contribution of the numerical error
> Tends to grow with the increase of degrees of freedom
(grid refinement)
2. Numerical error
Verification and Validation in CFD
Round-off error, example:
Polynomial interpolation in 2-D (or 3-D)
(n+1)
2
coefficients determined from the (n+1)
2
points where
i
(x
i
,y
i
) is known
[ ]
n
n
n
n
y b y b b
x a
x a
a
y x ..., , ,
...
) , (
2 1
2
1
(
(
(
(
(

=
2. Numerical error
Verification and Validation in CFD
Round-off error, example:
Polynomial interpolation in 2-D (or 3-D)
Coefficients of the polynomial are determined
from the solution of a linear system of equations
First term of the main diagonal : 1
Last term of the main diagonal : x
n
y
n
2. Numerical error
Verification and Validation in CFD
Iterative error:
> Non linearity of the equations to solve
(convection terms of the transport equations, free
surface boundary conditions)
> Decoupling of the equations
(Turbulence quantities determined with the available
mean velocity field and momentum equations
solved with a known eddy-viscosity)
2. Numerical error
Verification and Validation in CFD
Iterative error:
> High-order discretization schemes implemented
as deferred corrections
> Linear systems of equations solved with iterative
methods (Jacobi, Gauss-Seidel, Conjugate Gradient,
GMRES, ...).
2. Numerical error
Verification and Validation in CFD
Iterative Error:
> In principle, it can be reduced to machine precision
(if there are no problems with the round-off error)
> Increase of the number of degrees of freedom
(grid refinement) tends to make the reduction of the
iterative error more troublesome. Multigrid techniques
may avoid problems originated by the increase of the
size of the problem to be solved
2. Numerical error
Verification and Validation in CFD
Iterative error:
> It is important to define (know) what is the meaning of
one iteration (number of cycles involved)
> Estimates of the iterative error based on the differences
between consecutive sweeps or residuals of the last
iteration performed are not reliable
> The L

norm is more adequate than the L

1
and L
2
norms
to obtain conservative estimates of the iterative error
2. Numerical error
Verification and Validation in CFD
Iterative error, example:
> Turbulent flow in a channel. RANS equations
with the one-equation eddy-viscosity model
of Spalart & Allmaras
> Initial guess of the flow field is obtained copying the
inlet boundary conditions (taken from the experimental
results) to all the interior (normal) grid lines
> Solution converged to machine precision (10
-14
).
2. Numerical error
Verification and Validation in CFD
Iterative error, example:
> Convergence criteria based on the maximum
differences, L

, between consecutive iterations, e

t
> Iterative error computed from the difference to the
solution converged to machine accuracy
> Example of next slide corresponds to the horizontal
component of the velocity vector, U
1
2. Numerical error
Verification and Validation in CFD
Iterative error:
Largest iterative
error is 2
orders of
magnitude larger
than e
t
!
Using the L
2
norm
in the convergence
criteria may lead to
largest differences
3 orders of
magnitude higher
than e
t
!
2. Numerical error
Verification and Validation in CFD
Discretization error:
> Consequence of the transformation of the
continuum equation(s) into a system of
algebraic equations
> It may have a geometric component, which
may even be the dominant contribution in domains
bounded by surfaces with high curvature
2. Numerical error
Verification and Validation in CFD
Discretization error:
> Usually, it is the main contribution to the numerical error
> Can only be determined with the knowledge of
the exact solution
> Tends to diminish with the increase of the number
of degrees of freedom (grid refinement)
> Estimate of the discretization error may be obtained
from grid refinement studies
2. Numerical error
Verification and Validation in CFD
Discretization error:
> Assumed behaviour of the error
Local or functional variable

exact
Exact solution
Constant related to the error level
h
i
Typical dimension of the grid
p Order of accuracy
p
i exact
h e = ) (
2. Numerical error
Verification and Validation in CFD
Discretization error:
> Asymptotic region, i.e. all remaining (high-order) terms
of the power series expansion are negligible
> Typical grid size, h
i
, may be troublesome to define
Geometrical similarity of the grids is required
(Multi-block, unstructured grids)
p
i exact
h e = ) (
2. Numerical error
Verification and Validation in CFD
Discretization error:
> Minimum number of grids to obtain and
exact
: 2.
> It is not safe to use only two grids. There is no
guarantee that the data is in the asymptotic range.
Therefore, the order of accuracy, p, is unknown
> In non-linear problems, there is no guarantee that the
theoretical order of accuracy is equal to the lowest
order of all the discretization schemes adopted
p
i exact
h e = ) (
2. Numerical error
Verification and Validation in CFD
Discretization error:
> Minimum number of grids to estimate , p and
exact
: 3.
> In practical applications, the data may exhibit scatter
(h
i
definition, interpolations, integrations,...)
Therefore, three grids do not guarantee a reliable estimate
of , p and
exact
> If more than three grids are available, , p and
exact
may be determined in the least squares sense
p
i exacto
h e = ) (
2. Numerical error
Verification and Validation in CFD
Discretization error, example:
> Determination of the area of a cylindrical surface
with a Gauss quadrature rule using 1 point per
direction
> Two types of grids:
A. Equidistant points along the diameter, Z=constant.
B. Equidistant points along the surface, =constant.
( ) 1 1 =
i i
N h
2. Numerical error
Verification and Validation in CFD
X
Z
Y
X
Z
Y
Discretization error, example:
Grid Z=constant
Z
t
- 3 digits in Z
Grid =constant
2. Numerical error
Verification and Validation in CFD
Discretization error, example:
Grid
Grid Z
Grid Z
t
2. Numerical error
Verification and Validation in CFD
Guarantee that the computer code does not contain
errors (bugs)
Although it could be assumed otherwise, it is not of the
exclusive responsibility of code developers (any user
of a popular computer operating system is aware of
such reality)
It requires error evaluation and so the exact solution
must be known
Mathematical problem
3. Code Verification
Verification and Validation in CFD
In some practical applications, there are no exact
solutions available. For example, the Reynolds-averaged
Navier-Stokes equations supplemented by a
turbulence model do not have any known exact
solutions
The Method of Manufactured Solutions (MMS) is a
viable alternative to perform Code Verification when
analytical solutions are not available. It is important
to include all the equations of the mathematical model
(including transport equations of turbulence quantities)
in the Code Verification process
3. Code Verification
Verification and Validation in CFD
Method of Manufactured Solutions:
1. Choose the calculation domain
2. Choose the analytical solution for the dependent
variables of the problem
3. Substitute the chosen solution in the differential
equations to determine source terms that guarantee
that the selected exact solution satisfies the new
system of differential equations
3. Code Verification
Verification and Validation in CFD
Method of Manufactured Solutions, example:
> Manufactured solution includes the dependent variable
of the turbulence model (Spalart & Allmaras).
> Mean velocity field is similar to a boundary-layer
in zero pressure gradient
> Three different exercises:
1. Solve all the equations
2. No damping function in the turbulence model
3. Solve only the continuity and momentum equations
with the manufactured eddy-viscosity
3. Code Verification
Verification and Validation in CFD
Method of Manufactured Solutions, example:
h
i
/h
1
R
M
S
[
e
(
u
x
)
]

1
0
5
0 1 2 3 4
0
1 0
2 0
3 0
4 0
5 0
M S 2
p= 1 .3
M S 2 , No damping function ( f
v1
= 1 )
p= 2 .1
M S 2 , M anufa ctured
t
p= 2 .0
Root mean squared
(RMS) error of
the horizontal
velocity
component, u
x
1
2
3
3. Code Verification
Verification and Validation in CFD
Exact solution is not available
Numerical error estimation usually assumes that the
discretization error is dominant (this requires an iterative
error at least two orders of magnitude smaller)
Methods based on grid refinement studies are one of
the alternatives for the estimation of the discretization
error/uncertainty
Mathematical problem
4. Solution Verification
Verification and Validation in CFD
Estimate the uncertainty, U, of a numerical solution
of a flow quantity for which the exact solution is
not known
Objective:
) ( ) ( U U
exact
+
with a confidence level of 95%
( ) e F U
S
= ) (

S
F
( ) e
Safety factor
Error estimate
4. Solution Verification
Verification and Validation in CFD

i
Numerical solution of a local or functional variable

o
Estimate of the exact solution

RE
Estimate of the error

j
Constant related to the error level
h
i
Typical cell size
p
j
Observed order of accuracy
p
i RE o i i
h e = = = ) (
4. Solution Verification
Verification and Validation in CFD
X
X
X
h
i

o
At least 3 grids are required to calculate
o
, , p
p
i RE o i i
h e = = = ) (
( )
( )
( )
0
1
1
1
1 2
2 3
1
2
1 2
2 3
1 2
1 2
1
=

|
|

\
|

= =
p
p
p
p
RE o
h h
h h
h
h
h h




RE

4. Solution Verification
Verification and Validation in CFD
Apparent convergence or divergence for a grid
triplet with h
2
/h
1
=h
3
/h
2
Convergence ratio:
0 < R <1 Monotonic Convergence
-1 < R <0 Oscillatory Convergence
R > 1 Monotonic Divergence
R <-1 Oscillatory Divergence
2 3
1 2

= R
4. Solution Verification
Verification and Validation in CFD
Example for the turbulent flow over a flat plate
> Time-averaged RANS equations with eddy-viscosity
k- two equation models
> Convergence of the friction resistance coefficient of
the plate with the grid refinement. Estimate of the
numerical uncertainty for two levels of grid refinement:
h
1
1537193, h
i
/h
1
=2 76997.

4. Solution Verification
Verification and Validation in CFD
h
i
/h
1
C
F

1
0
3
0 1 2 3 4
2.92
2.94
2.96
2.98
3
3.02
k- BSL
1,2
2,3
k- Wilcox
2,3
p= 1.9
Re
L
=10
7
4. Solution Verification
Example for the turbulent flow over a flat plate
Verification and Validation in CFD
h
i
/h
1
C
F

1
0
3
0 1 2 3 4
2.08
2.1
2.12
2.14
2.16
2.18
k- BSL
2,3
2,3
k- Wilcox
2,3
p= 1.8
Re
L
=10
8
4. Solution Verification
Example for the turbulent flow over a flat plate
Verification and Validation in CFD
h
i
/h
1
C
F

1
0
3
0 1 2 3 4
1.54
1.56
1.58
1.6
1.62
1.64
k- BSL
2,3
2,3
k- Wilcox
p= 2.0
p= 1.7
Re
L
=10
9
4. Solution Verification
Example for the turbulent flow over a flat plate
Verification and Validation in CFD
Estimate of the modeling error by comparison to
experimental data
Validation method proposed recently by the ASME:
> Comparison error obtained from the difference
between the numerical solution and the experimental
result, |E|
> Validation uncertainty, U
val
, that combines numerical
uncertainty, experimental uncertainty and input parameters
uncertainty (boundary conditions, Reynolds number...)
5. Validation
Verification and Validation in CFD
Method to evaluate the modeling error recently
proposed by the ASME:
S Numerical result
D Experimental measurement
U
num
Numerical uncertainty
U
D
Experimental uncertainty
U
input
Input parameters uncertainty
(Boundary conditions, Reynolds number...)
D S E =
( ) ( ) ( )
2 2 2
input D num val
U U U U + + =
5. Validation
Verification and Validation in CFD
Estimate with 95% confidence the interval that contains
the modeling error
Modeling error is probably similar to |E|. Therefore, there
is information about the need to improve or not the model
Modeling error smaller than the noise originated by
the numerical, experimental and input parameters
uncertainty.
val
U E <
val
U E >>
[ ]
val val
U E U E + ,
5. Validation
Verification and Validation in CFD
Example: Velocity field at the propeller plane of a tanker
at model scale Reynolds number
> Time-averaged RANS equations supplemented with
with the two-equation k- SST turbulence model
No wall functions
> Grid refinement study including 6 grids (r=h
6
/h
1
=2 )
Finest grid 6,410
6
nodes Coarsest grid 0,810
6
nodes
> Experimental uncertainty obtained from assymetry of
the measurements (estimate by default)
> Input parameters uncertainty assumed to be zero
5. Validation
Verification and Validation in CFD
y/L
PP
z
/
L
P
P
-0.01 0 0.01 0.02
-0.065
-0.06
-0.055
-0.05
-0.045
-0.04
-0.035
-0.03
U
x
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
Experimental
5. Validation
Numerical
Example: Velocity field at the propeller plane of a tanker
at model scale Reynolds number
Verification and Validation in CFD

U
x
0 30 60 90 120 150 180
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Experimental
SST
Usual comparison:
Quality of the results
depends on the size of
the symbols and thickness
of the line...
5. Validation
Example: Velocity field at the propeller plane of a tanker
at model scale Reynolds number
Verification and Validation in CFD

U
x
0 30 60 90 120 150 180
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Experimental
SST
Experimental
uncertainty added
(estimated by default)
5. Validation
Example: Velocity field at the propeller plane of a tanker
at model scale Reynolds number
Verification and Validation in CFD

U
x
0 30 60 90 120 150 180
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Experimental
SST
Estimated numerical
uncertainty added
5. Validation
Example: Velocity field at the propeller plane of a tanker
at model scale Reynolds number
Verification and Validation in CFD

U
x
0 30 60 90 120 150 180
0
0.1
0.2
0.3
E=|S-D|
U
val
=(U
2
num
+U
2
D
)
1/2
Comparison error is
smaller than the validation
uncertainty for most of the
locations with experimental
data available. Evaluation
of the modeling error
requires smaller numerical
(and experimental)
uncertainties
5. Validation
Example: Velocity field at the propeller plane of a tanker
at model scale Reynolds number
Verification and Validation in CFD
The present capabilities of CFD as an Engineering tool
are unquestionable. For example, CFD may have a strong
impact in the study/evaluation of scale effects in ship
hydrodynamics
However, it is essential to demonstrate that CFD produces
correct results for the right reasons
The time is past when we should accept presentations that
use the words verification and validation interchangeably
or that present code validation as a comparison of data
and calculations with no consideration for the uncertainties in
the two.
Hugh Coleman,
3
rd
Workshop on CFD Uncertainty Analysis, October 2008
6. Final Remarks
Verification and Validation in CFD
http://www.maretec.ist.utl.pt/html_files/VV2010.htm