UNIT 12 NUMERICAL DIFFERENTIATION

Structure
12.1 Introduction
Objectives
12.2 Methods Based on Undetermined Coefficients
12.3 Methods Based on Finite Difference Operators
12.4 Methods Based on Interpolation
12.5 Richardson's Extrapolation
12.6 Optimum Choice of Step Length
12.7 Sulnma~y
12.8 Solutions/Answers

12.1 INTRODUCTION

Differentiation of a f u ~ ~ c t i of(x)
n is a fundamental and important concept in calculus.
W h e l ~the function is given explicitly its derivatives f'(x), fl'(x), ... etc. can be easily
found using the methods of calculus. For example, if f(x) = x2, we know that fl(x) =
2x, ftl(x) = 2 and all the higher order derivatives are zero. However, if the function is
not known explicitly but, we are given a table of values of f'(x) corresponding to a set
of values of x, then we cannot find the derivatives by using calculus methods. For
instance if f(x& represents distance travelled by a car in time xk, k = 0, 1, 2, ... seconds,
and we require the velocity and acceleration of the car at any time xk, then the
derivatives f '(x) and f "(x) representing velocity and acceleration respectively, cannot be
found analytically. Hence, the need arises to develop methods of differentiation to
obtain the derivative of a given function f(x), using the data given in the form of a
table which might have been formed as a result of scientific experiments.

Numerical inethods have the advantage that they are easily adaptable on calculators and
computers. These methods make use of the interpolating polynomials, which we
discussed in Block-3. We shall now discuss, in this unit, a few numerical differentiation
methods, namely, the method based on undetennined coefficients, methods based on
finite difference operators and methods based on interpolation.
\

Objectives
After studying this unit you should be able to
explain the iinportance of the numerical inethods over the calculus inethods;
use the method of undetennined coefficients and methods Lased on finit difference
operators to derive differentiation formulas and 'obtain the derivative of a function at
step points;
use the inethods derived from the interpolation formulas to obtain the derivative of a
function at off step points;
use Richandson's extrapolation method for obtaining higher order solutions;
obtain the optimal steplength for the given formula.

12.2 METHODS BASED ON UNDETERMINED

COEFFICIENTS
In Unit 1, we introduced you to the concepts of round-off and truncation errors. In the
derivation of the inethods of numerical differentiation, we shall be referring to these
errors quite often. Let us first quickly recall these concepts here before going further.
Numerical Differentiation Integration
and Solution of Differential Equations Definition : The round-off e m r is the quantity R which must be added to the finite
representatio~iof a computed number in order to make it the true represe~ltatio~l
of that
number. Thus
machine representation) + R = y(true representation).
Definition : The truncation error denoted by TE is the quantity which must be
added to the finite representation of the computed quantity in order that the result be
exactly equal to the quantity we are seeking to generate. Thus
y(true representation) + TE = y(exact)
The total error En is then given by
= I machine representation) - y(exact) I
lEnI = 1 ~(ntachinerepresentation) - y(true representation) I
+ y(true representation) - y(exact) (
s ) machine representation) - y(tme representation) (
+ I y(true representation) - y(exact) 1
s IRI + I n1
Defintion : Let f(h) be the exact analytical value of a given problem obtained by
using an analytical formula and fh be the approximate value obtained by using a
numerical method. If the error f(h) - & = C hP, where C is a constant, then p is known
as the order of the numerical method.
Let us consider a function f(x), whose values are given at a set o f l t a b u l ~points.
~ For
developing numerical differentiation formulas for the derivatives f (x), f (x), ...
at a
point x = x,, we express the derivative fyx), q z 1, as'a linear combination of the
values of f(x) at all arbitrarily chosen set of tabular points. Here, we assume that the
tabular points are equally spaced with the steplength h i.e. various step (nodal) points are
xm = x,, * mh, m = 0, 1, ... etc. Then we write

where yi, i = - s, - s + 1, ......, n are the unknowns to be determined and f, +,

denotes
f(xk + mh). For example, when s = n = 1 and q = 1, Eqn. (1) reduces to
h f l (x3 = Y-If,-, +Y$, +Y,f,+,.
Similarly, when s = 1, n = 2 and q = 2, we have
h2f " ( ~ 3= y-~fk-+ ybfk + ylfk+1 + yzfk+,
Now suppose we wish to determine a numercial differentiation formula for P ( x 3 of
order p using the method of undetermined coefficients. In other words, we want our
formula to give the exact derivative values when f(x) is a polynomial of degree s p,
that is, for f(x) = 1, x, x2, x3, ..., xP. We then get p+l equations for the determination
of the unknowns yi, i = -s, -s + 1 .....,n. You know that if a method is of order p,
then its is of the form chP+l kl) (a),for some constant C. This implies that if f(x)
= xm, m = 0, 1, 2, ....., p then the method gives exact results, since

- (xm)- 0, for m
dP+ 1
1 0, 1, ...,p.
dxP+
Let us now illustrate this idea to find the numerical differentiation formula of 0 @4for f "(x&
Derivation of formula for fW(x)
Without loss of generality let us take xk = 0. We shall take the points symmetrically,
that is, x,= mb; m = 0, * 1, i2.
Let f-2, f-,, f, f,, f, denote the values of f(x) at x = -2h, - h, 0, h, 2h respectively.
In this case the formula given by Eqn. (1) can be written as
h2fl1(0) = Y-2f-2 + Y-lf-, + Y&+ ylfl + y2f2 (2)
Let us now make the formula exact for f(x) = 1, x, x2, x3, x4. Then, we have
f(x) = 1, ftl(0) = 0; f-2 = f-, = q, = f, = f , = 1
f(x) = x, fl'(0) = 0, f-2 = -2h; tl F 41; = 0; fl = b; f2 = 2b;
2 "
f(x) = x , f (0) f-2 = 4h2 = f2; ffl = b2 s f,; (,r 0; (3)
 . .
f(x) = 2,f "(0) = 0. f, = - 8h3; f-, = -a3; 6 = 0; f, h3, 6 8h3
f(x) = x4 , f "(0) = 0; f-2 = 16h4 = f2; f-, = h4 = f,; 6 = 0
Numerical Differentiation

Substituting these values in Eqn. (2), we obtain the followi~~g

set of equations for
determining y,, m = 0, * 1, t 2.
Y-;! + Y-1 + Yo + Y1 + Y2 = 0

-2~-2 -Y-1 + Y1+2 Y2 = 0

4 1 + ~Y - ~~+Y, + 4y2 = 2 (4)
43Y-2-y-1 +Y1 + 8 y 2 = 0
16~-2+ Y-, + Y, + 1 3 , = 0
Thus we have a system of five equations for five unknowns. The solution of this system
of Eqns. (4) is
yq2= y2 = -1112; y-l = y1 = 16/12; yo= 30112;
Hence, the numerical differentiation formula of 0(h> for ft'(0) as given by Eqn. (2) is

- ",
f l1((J) f r --f-,
12h2 [
+ 16fi-Mfo + 16f -f
21 (5)
Now, we know that the TE of the formula (5) is given by the f i t non-zero term in the
Taylor expression of

The Taylor series expansions give

f(x, - 2h) = f(xo)- 2hf1(x0)+ 2h2 f "(x,) --
4b3 f "'(xJ + -
3
2h4 fN(xo)
3
- f4h5
15
v
(x0)+-f
4h6 VI
45
(x0)- ......
h2 h3 h4 N h5
f(x0 - h) 2 f(x0)- hf '(xo) + -f "(xo) - - f "'(xo) + -f (xJ
2 6 24
--
120
f '(x0)

+ fh6 VI(xo)+ ......

720
h2 h3 h4 w h5 v
f(xo + h) = f(xo) +Pf ' ( x ~ +) -f "(x,) + -f "'(xo) + -f (xo) + -f (x0)
2 6 24 120
h6
+ -fW(x&+......
720
2h2 4h3 2h4 4h5
f(xo + 2b) = f(xo) + 2hf '(xo) + -f "(xo) + T f '"(x0) + 7fw(xo)+ -f "(x,)
2 15
+ f4h6 n(xo)+ ......
45
Substituting these expansions in Eqn. (6) and simplifying, we get the f i t non-zero term
or the TE of the formula (5) as

You may now try the following .exercise.

El) A differentiation rule of the form

f i = a&+a,f, +a&
is given. Find a,,,a, and a, so that the rule is exact for polynomials of degree 2.

You must have observed that in the numerical differentiation formula discussed above, we
have to solve a linear system of equations. If the number of nodal points involved is large
or if we have to determine a method of hi@ order, then we have to solve a large system of
linear equations, which becomes tedious. To avoid this, we can use f i t e difference
operators to obtain the differentiation formulas, which we shall illustrate in the next section.
Numerical Differentiation Integration
and Solution of Differential Equations 12.3 METHODS BASED ON FINITE DIFFERENCE
OPERATORS

Recall ttiat in Unit 10 of Block 3, we introduced the finite difference operators E, V, A,

p and 6. There we also gave the relations among Garious operators.

In order to construct the numerical differentiation formulas using these operators, we

shall first derive relations between the differential operator D where Df(x) = f ' (x), and
the various difference operators.

By Taylor series, we have

f(x+h)= f ( x ) + h f t ( x ) + h 2 f f f ( x ) + . .
= [ l + hD + h 2 ~ +' . . . ] f(x)
= ehDf(x)
Since, Ef (x) = f (x + h)

we obtain from Eqn. (7), the identity

E = ehn
which gives the relations

We can relate D with 6 as follows :

We know that 6 = EV2- E-". Using identity (8), we can write

Hence, 6 = Bin h (hDI2)

or hD = Bin h-' (612) (1 I)
Similarly p = wsh (hD12) (12)
We also have p6 = sinh (hD) or hD = sinh-' (pa) (1 3,
ti2
and pZ = cosh2 (hD/2) = 1 + sinh2 (hDn) = 1 + -
4 (14)
Using the.Maclaurin9s expansion of sinh-'x, in relation (ll), we can express hD as an
infinite series in 612.

Thus, we have
hD = 2sinh-' (612)

Notice that this formula involves off-step points when operated on f(x). The formula
involving only the step points can be obtained by using the relation (13), i.e., '
hD = sinh-' (p6)

Using the relation (14) in Eqn. (16), we obbin

Thus, Eqns. (9), (lo), (15) and (16) give us the relations between hD and various
difference operators. Let us see bow we can use these relations to derive numerical
 differentiation formulas for f; , f: etc. Numerical Differentiation
We first derive formulas for ftt. From Eqn. (9), we get

Thus forward difference formulas of qh), 0(h2), 0(h3 and 0(h4) can be obtained by '
retaining respectively 1, 2 3, and 4 terms of the relation (9) as follows :

q h ) method : hf; 5 fk+,- fk (18)

I 1
0(h2) method : hfk a r(-fk+2 + 44+,- 31,) (19)

0(h3 method : hf; - 1

:(2fk+3- 9fk+,+ 1 8 6 , - llfk) (20)
I 1
O(h3 method : hfk a =(-3fk+, +164+3- 36fk+,+ 48fk+,- 25fk) (21)

TE of the formula (18) is

and that of formula (19) is

Similarly the TE of formulas (20) and (21) can be calculated. Backward difference
formulas of qh), 0(h2), 6(h3) and 0(h3 for fkl can be obtained in the same way by .
using the equality (10) and retaining l,2,3 or 4 terms. We are leaving it as an exercise
for you to derive these formulas.

E2) Derive backward difference formulas for E,' of qb), 0(h2),0(h3) and 0(h4).

Central difference formulas for fl, can be obtained by using the relation (17), i.e.,

Note that relation (17) will give us methods of 0(h2) and 0@>,on retaining 1 and 2 tern,

0(h2) method : bfi - ?( 4

1
+ - fk- (24)
1
0(h4)mculod: hf;-i?(-fk-2+8fk-1+8fk+I+fk+2) (25)

We ROW. illustpte these methods &rough an example.

/

Exampk 1 : Given the following h b k of v s b of qx) = ex, f i d f ' (0.2) using

formulas (Is), (19), (24) and (25).

b
S o l u t h : Were h = 0.1 and exact value of ex at x = 0.2 is 1.221402758.

it Using(l8), f '(0.2) =
-
f(0.3) f(0.2)
0.1
Numerical Differentiation Integration
Actual error 1.221402758 - 1.28456 = - 0.063157
- &-
r

-
and Solution of Differential Equations
1
Using (19), f '(0.2) f(0.4) + 4f(0.3) - 3f(0.2)] 1.21701

-
h2
TE -
3
f 111(0.2) -
-eo.2 O.,,,,'$ml;
= O.O1
3
Actual error = 0.004393
1
Using (24), f ' (0.2) = - (w.3) - f (0.1)) = 1.22344
0.2

I-E = - -
h2 f"'(0.2) = - -0.01
ea2= - 0.002035f;
6 6 ,
Actual m r = - 0.002037
1
f ' (0.2) = [-f (0.0) + Sf (0 I) - 8f (0.3) + f (0.4))
_
Ushg (U), = 1.221399167

TE = h4fYI(0.2) 0-0001 ea2 0.4071

I
10-5;
30 30
Actual error = 0.3591 x

Numerical differentiation formulas for f", can be obtained by considering

We can write the forward difference methods of qh), 0(h2), m 3 ) and 0@> for f'; by
using Eqn. (26) and retaining 1, 2, 3 and 4 tenns as follows :

0@3 method : h?C=-fk+3+4$+2-5$+1+2ft 00) (

q h > method: h?~----8fk+5+51~+4-136fk+3+194$+2-144$+l+43$)
:2( (32)l

Backward difference formulas can be written in the same way by using Egn. (27).
Central differenceformulas of qh2) and 0@3 for f[ are obtained by using Eqa. (28)
and retaining 1 or 2 terms in the form :

Let us consitter an example,

Example 2 : For the table of values of qx) = ex, given h -Example 1, frnd f'' (0.2)
using the formuhs (33) and (34).

Actual error P - 0.0009972

Using Eqn.(34), f "(0.2) =
[- f(O.0) + 16f(0.1) - 30f(0.2) + 16f(0.3) - f(0.4)] Numerical Differentiation
= 1.221375
0.12
h4fw 0 2)
TE = = 0.13571 x 10-5

Actual error = 0.27758 x ,

And now the fo!lowing exercises tor you.

E3) From the following table of values find f (6.0) using an O(h) formula and f I' (6.3)
usil~gan 0(h2) formula.

E4) Calculate the first and second derivatives of lnx at x = 500 from the following table.
Use q h 2 ) forward difference method. Compute TE and actual errors.
x : 500 510 520 530
f(x) : 6.2146 6.2344 6.2538 6.2729

In Secs. 12.2 and 12.3, we have derived numerical differentiation formulas to obtain the
derivative values at nodal points or step points, when the function values are given in
the form of a table. However, these methods cannot be used to find the derivative
values at off-step points. In the next section we shall derive methods which can be used
for finding the derivative values at the off-step points as well as at step-points.

12.4 METHODS BASED ON INTERPOLATION

In these methods, given the values of f(x) at a set of points x,,, xl,...%, the general
approach for deriving numerical differentiation formulas is to obtain the unique
interpolating polynomial P,(x) fitting the data. We then differentiate this polynomial q
times (q s n), to get (x). Tbe value 9Jq)(xk)then gives us the approximate value of
fi9)(x,J where X, may be a step point br an off-step point. We would like to point out
here that even when the original data are known to be accurate i.e. Pn(x,J = f ( G ,
k = 0, 1, 2, ..., n, yet the derivative values may differ considerably at these points. The
approximations may further deteriorate while finding the values at off-step points or as
the order of the derivative increases. However, these disadvantages are present in every
numerical differentiation formula, as in general, one does not know whether the function
representing a table of values has a derivative at every point or not.
We shall first derive differentiation formulas for the derivatives using non-uniform nodal
points. That is, when the diffetence between any two consecutive points is not uniform.

Non-uniform nodal points

Let the data (x,, Q, k = O,l, ..., n be given at n + 1 points where the step length xi-x14
may not be uniform.
In Unit 9 you have seen that the Lagrange interpolating polynomial fitting the data
(x,, Q,'k = 0, 1, ..., n is given by .
n
pn (XI= 2 4(x) fk (35)
k=O
&(x) am the fundamental Lagrange polynomials given

Nx) (36)
V X )= ix - xk) x1(xk)

-
and n: (x) = (X x0) (X xl) - (X- x,) (37)
Numerical Differeatiation Integration
and Solution o f Differential Equations xl(xk)= ( x ~ -x0)(xk- x ~ ) . . . ( x ~ Xk-l)(xk-
- X~+~)"-(X~- (38)

The error of interpolation is given by

En (x) = f(x) - P,(X) = f(' '1 (a), xo< a c xn,

(n + 1) !
+

Differentiating Po (x) w.r.t. x, we obtain

and the error is given by

Since in Eqn. (40), the function 4 x ) is not known in the second term on the right
hand side, we cannot evaluate EL (x) directly. However, since at a nodal point xk,
n(x3 = 0, we obtain

If we want to obtain the differentiation formulas for any higher order, say qth
(1 s q r n) order derivative, then we differentiate Pn(x), q times and get

Similarly, the error term is obtained by differeutiating En(x), q times.

Let us consider the following examples.
Example 3 : Find fl(x) and the error of approximation using Lagrange interpolation
for the date (+, 53, k = 0, 1.
Solution : We know that Pl(x) = h(x) & +Ll (x) fl
X- X1 X- X1
where Lo (x) = -and Ll(x) = -
Xo- X l x1- xo
Now,
P; (x) = L; $ + L; (x) f,
1 1
and L; (x) = - , L ' ~ ( X=) -
xo- x1 XI- xo
fo +-=-
Hence, f ' (x) = pfl(x) = -
fl (f1- Q
Xo- Xt X1- Xo (x,- XJ

(xo- xl) (xl- xo)

E' (xJ = - 2
f l1 (a) and E1(x1)= -f f l(a), xo c a xl.
2
Example 4 : Find f (x) and fl'(x) given fo, fl, f2 at ,x x,, x2 respectively, using the
Lagrange interpolation.
Solution : By Lagmnge's interpolation formula

where,
(x- xl) (x- x2) 2x- xl- x,
Lo(x) = (xo- xl) (xo- x2) ; a x ) = (xo- xl) (xo- x,)
Hence, fl(x) = P; (x) = (x) fo + L; (x) fl + (x) f2 Numerical Differentiation

P (x) '= :L (x)

a'nd : fo + L: -(x) fl + :L (x) f2

Exan r;e 5 : Given the following values of f(x) = In x, find the approximate value of
f 1 (2.C nd f"(2.0). Also find the errors of approximations.

Solution : Using the Lagrange's interpolation formula, we have

.'. we get

The exact value of f 1 (20) = 0.5

Error is given by

Similarly,

f" (x,,) = 2 (xo - x,)fo(x,

" I
- x2) + (x, - XJ f 1(xl - x2) + (x2 - XO)( ~ -2xi)

The exact value of f'' (2.0) = - 0.25.

Error ii given by
1 1
E; (xo) = 5 ~ - x2) f '"(2.0) + -(xo- x,) (x,-
( 2 ~ x1-
24
x2) [fN!?3) + fN(2.0)]
= -0.06917
You may now try the following exercise.

E5) Use Lagrange's interpolation to find f '(x) and f "(x) at x = 2.5'5.0 from the
following table

Now let us consider the case of uniform nodal points.

Uniform nodal points

When the difference between any two consecutive points is same; i.e., when we are
given values of f(x) at equally spaced points, we can use Newton's forward or
Numerical Differentiation Integration backward interpolation formulas to find the unique interpolating polynomial P,(x). Wc
and Solution of Differential Equations
can then differentiate this polynomial to fmd the derivative values either at the nodal
points or at off-step poihts.

Let the data (xk, f3, k = 0, 1, ..., n be given at (n + 1) points where the step points xL;
k = 0, 1, ..., n are equispaced with step length h. That is, we have

You know that by Newton's forward interpolation fonnula

with error
~x-xo)(x-xl)...(x-~n)An+l-
En (x) = f(a) xo<a<x,
(n+l)!hn+'

If we put
X
-
- Xo
s or x = xg + sh, then Eqns. (44) and (45) reduce respectively to
h

and
-
S(S 1). . . (s-n) h ( n + ~ ) f ( a + ~ )
n' (1' = (n + 1) ! (a)

Differentiation of Pn(x) w.r.t. x gives us

At x = xg, we have s = 0 and hence

which is same as formula (9) obtained in Sec. 123 by difference operator method. We can
obatin the derivative at any step or off-step point by finding the value of s and substituting
the same in Eqn. (48). The formula mrrespondug to Eqn. (47) in backward differences is,

where x = x,, + sh.

Formulas .for higher order derivaiives can be obtained by differentiating P: (x) further
and the corresponding error can be obtained by differentiating <(x).

Let us illustrate the method through the following examples :

Example 6 : Find the first and second derivatives of f(x) at x = 1.1 from the
following tabulated values.
 Solution : Since we have to find the derivative at x = 1.1, we shall use the forward Numerical Differentiation
difference formula. The forward differences for the given data are given in Table 1.

Table 1

Since, x = xo + s h, xo = 1, h = 0.2 and x = 1.1, we have s = -

lml- 0.5
0.2
-
Substituting the value of s in formula (48), we get

Substituting the values of Afo and A34, in Eqn. (50) from Table 1, we get
f '(1.1) = 0.63
To obtain the second derivative, we differentie formula (48) and obtain

- 1
f "(x) = ~ " ( x ) -
h
k2$+ (s- 1) A3fOI
Thus f"(l.1) = 6.6

Notp : . If you construct a folward difference interpolating polynomial P(x), fitting the
dataIgiven in Table 1, you will find that f(x) = P(x) = x3 - 3x + 2 Also, ff(l.l) = 6.3,
f"(l.1) = 6.6. The values obtained from this equation or directly as done above have to
be same as the interpolating polynomial is unique.

Example 7 : Find ff(x) at x = 0.4 from the following table of values.

Solution : Since we are required to find the derivative at the end pint, we will use the
backward difference formula. The backward differencle table for the given data is given by

I Table 2
Numerical Differentiation Integration
and Solution of Differential Equations
Since x, = 0.4, h = 0.1, x = 0.4, we get s = 0

~hbstitutin~
the value of s in formula (49), we get

= 1.14913
How about trying a few exercises now ?

E6) The position qx) of a particle moving in a line at various times x, is given in the
following table. Estimate the velocity and acceleration of the particle at x = 1.5 and 35

E7) Construct a difference table for the following data

Taking h = 0.2, compute f '(1.5) and the error, if f(x) = ex.

You m z t have by now observed 'that to obtain numerical differentiation methods of
higher order, we require a large number of tabular points and thus a large number of
function evaluations at these tabular points. consequently, there is a possibility that the
round-off errors may increase so much that the numerical results may become useless.
However, it is possible to obtain higher order solutions by combining the computed
values obtained by using the same method with two different step sizes. This technique
is called extrapolation method or Richardson's extrapolation. We shall now discuss this
method in the next section.

12.5 RICHARDSON'S EXTRAPOLATION

The underlying idea in this method is as follows :

Let fi9) (h) denote the approximate value of fiP)(x,J, obtained by using a formula of
order p, with steplength h and p9)(rh) denote the value of Pd(x,J obtained by using the
same method of order p, with steplength rh. Then,
f (40) = f (q)(~,J+ ChP + 0 (hp+') (5 1)

and f(q)(rh) = f(q)(x,J + C ( r w + 0 (hp + I) (52)

Eliminatinn C between Eqns. (51) and (52), we get

Thepew approximation to fid(x& is therefore

The expression on the right hand side of Eqn. (54) for finding the value of the qth
derivative by a certain method of order p has now become a method of order p + 1.
This technique of combining two computed values obtained by using the same method
with two different step sizes, to obtain higher order solutions is called Richardson's Numetkai Differentiation
extrapolation method.

We know that the truncation error of a numerical method of order Q is given by

TE = ClhP + O(hp*').
where C1+ 0.
If, instead of denoting the higher order terms by O(hP I), we write down the actual
+

terms, we have
TE= C ~ ~ ~ + C $ ~ + ' + C ~ ~ ~ + ~ + . . .
By repeated application of Rcihardson's extrapolation technique we can obtain solutions
of higher orders, i.e. O(hP+'), O(hP+a), 0(hP+3etc. by eliminating C1, Cz,C, respectively.
Let us see how this can be done.

Consider the central difference differentiation formula (24) of qh2) given by

Let g(xJ = fl(xJ be the exact value of the derivative, which is to be obtained and

be the value given by the O(hZ) method. The truncation error of this nlethod may be
written as
g(h) = g(xJ + clh2 + c414+ w6 ...
+ (55)
h
Let f; be evaluated with different step sizes - r = 0, 1, 2, . . .
2I'
Then, we have

Eliminating C1 from Eqns. (55) and (56), we get

Eliminating Cl from Eqns. (56) and (57), we obtain

Notke that the methods g(')(h) and g("(hL2) given by Eqns. (58) and (59) are 0(h>
approximations to g(xJ

Eliminating C2 from Eqns. (58) and (59), we get

which gives an O(h6) approximation to g(x&. Gencl&ing, we f i d that the successive

higher order methods can be obtained frarn the
I Numaid Diffd*ion Inlegration This procedure is known as the Richardson's repeated extrapolstion to the limit.
and Solution of Differential Equations
I

I
Tbtse extrapolations can be stopped when

for a given error tolerence E.

Similarly, forward difference method of 0@2) can be obtained by cansidering

and using Richardson's extrapolation technique in the form

Tbis method is of qh2).

You may note that in Richardson's extrapolation, each improvement made for foxward
(or backward) difference formula increases the order of solutions by one, whereas for
central difference formula each improvement increases the order by two.

Let us now solve the following problems.

Example 8 : Tbe following table of values of f(x) = x4, is given :

Using the formula f '(x,) = [s(.3;"]' and Richardson's extrapolation method

find f l(3).

Solutioa : Note that in this example xl = 3.0. The largest step h that can be taken
h
is h = 4. computations can also be done by using step lengths hl = - = 2 and
2
hp = hlR = 1.

Using the ionnula

we get
qh2) method.

qh2) method.

qh2) method.

Therefore, using the f m u l a given by Eqn. (a)we

, have

g(l)(h) - )!(
48 - g@) m624*UW)1108
q h ? method.
3 3

0(h? method.
 Numerical Differentiation

0(h6)method.

Writing- in tabular form, we have

Step Second Fourth Sixth

length order method order metbod order metbod

Thus f'(3) = 108, must be the exact solution as we have

~ x a m p k9 : Let qx) = ex. Using a central difference formula of O(h3 find fl'(l)
Improve this value using Richardson's extrapolation by taking h = 0.1 and h = 0.05.
Solution : Witb h = 0.1 and

fk11 = e+~-~f,+fl-i 9
h2
we get

With h = 0.05 we get fl'(l) =

el.05 - ze
(0.05)~
+ e0.95
- 2.718848

Both the solutions are q h 2 ) approximations. Richardson's approximation using relation

(54) with r = --
0.05
2 and p = 2, gives us

The actual value i s e = 2718282

Your may now e y the following exercises :

E8) Compute f "(0.6) from the following table using w 2 ) central difference formula.
Improve it by Richardson's extrapolation metbod using step lengths h = 0.4.0.2,O.l.

E9) Using central difference formula of 0@') find f "(0.3) Emuthe given table and improve
tbe adracy using Richardson's empolation method using step lengths h = 0.1'0.2
F

x : 0.1 0.2 0.3 0.4 0.5

f(x) : 0.091 0.155 0.182 0.171 0.130

In the numerical differentiation methods, tbe trunation e m r is of tbe form ChP which
tends to zero as h-0. However, the metbod which approximates f (9)(x) contains hq in
Numerical Dimamtiation'ntevarionthe denominator. As h is successively reduced to smaller values, the truficatioil error
and Solution of Differential Equations
decreases but the round-off error in the method may inctease as we are dividing by a
small number. It may happen that after a certain critical value of h, the round-off enor
may become more dominant than the truncation error and the numerical results obtained
may start worsening as h is further reduced. The problem of finding a steplength h
small enough so that the truncation error is small, ye, large enough so that round-off
error does not dominate the actual error is referred to as themstep size dilemma. Such a
step length, if it can be determined is called the optimal steplength for that formula.
We shall now discuss in the next section how t9 determine the optimal steplength.

12.6 OPTIMUM CHOICE 0.F STEPLENGTH

We begin by considering an example.
Consider the numerical differentiation formula

I 1

fk = (fk+ 1- fk)

2
Let f(x) = ex and we want to approximate fl(l) by taking h = - m = l , 2,..., 7.
lom'
We have from the differentiation formula (63),

The exact solution is f '(1) = 2.718282. The actual error is e - fl(l) and the truncation
-
error is ehn. With h = - 2
lorn'
m = 1, 2 ,..., 7, we have the results as given in Table 3.

Table 3
h f'(l) Actual error Approximate Tmllcation error
2 x lo-' 3.009175 - 0.290893 - 0.271828

If you look at Table 3, you will observe that the improved accuracy of the formula,
i.e. fl(l), with decreasing h does not continue indefinitely. Tbe truncation error agrees
with the a&ual error till h = 2 x = 0.002. As h is further reduced, the truncation
error ceases to approximate the actual error. This is because the actual error is
dominated by round-off error rather than the truncation error. This effect gets
worsened as h is reduced further. In such cases we determine the optimal steplength.
When f(x) is given ie tabular form, these values may not be exact These values ccntain
round-off errors. In other words, f(x3 = E, + E,, where f(x3 is the exact value, fk is the
tabulated value and E~ is the round-off error. For the numerical differentiation fnnnllla
(63), we have

If the round-off errors in f, and 4 + ,aw E~ and E~ + , then we have

 Numerical Differentiation

where R is the round-off error and TE is the truncation error.

If we take E = max (IE,~), (IE,+ ,I) and Mi = max Ifu(xjl, we Fmd that

We define the optimum value of h as the one which satisfies either of the following
col;ditions :

(9 IRI- (ii) IR(+ITE(=minimum (64)

By the first condition in (64), we have

The value of the error is

-
If we use the second condition (R( ITE 1 = min, we have

2E hMz
-+-
h 2
- min.

To find the mi~imumis Eqn. (65), we differentiate the left hand side of Eqn. (65) with
respect to

2E
--
hZ
Mz
I- -= 0 or, h2
2
- 4E
-or,
2
h 1 2qKT2..

:. The minimum total error = (R(+ (TE( - w.

Let us now consider an example.

Example 10 : For the method

determine the optimal value of h using the criteria

Using this method and the first criterion, find the value of b and determine the value of
f1(2.0), from the following tabulated values of f(x) = In x. It is g i ~ e nthat the
maximum round-off error in the function evaluation is 5 x lo4

Solution : If E ~el, and E~ are the round-off errors in the given function evaluations of
fo, f,, f2 respectively, then we have

(- 3fo+ 4f1 - 5) (- 3Eo+ 4E1 - E2) hZ

f,=
2h
+ 2h
+-
3
f 11' (a)
Numerical Differeotiation Integration
and Solution of Differential Equations

Let E = max (IE,,~, [ell . and M, = m u 1f1"(x)I

We obtain

I f w e use IRI =]TEI= min, w e g e t

Hence h3 = -or
M,
12E
hTt =
R)"
-

The error is given by

If we use I R( + (TEI = min, we get

Minimum total error = 6m E~ MY

For f (x) = I n x, we have M3 = max ~ f ' " ( x )=~0.25
2 s x s 2.12
Using the criterion (RI + lTEl and E = 5 X lo4, w e get

For h = 0.06, we get

If we take h = 0.01, we get

The *exactvalue is f '(2.0) = 0.5

Clearly for h c h,, the result deteriorate.

You may now tly the following exercise.

E10) For the method

-
fk+ I fk- 1 h2 rrr
2h -6 (a),
xk- 1< a < xk+1
 determine hq, using the criteria. Numerical Diffir,.~tiation

(i) ( R ( = ITE( and

(ii) 1 R ( + ( T E1 = minimum.
Using this method and the second criterion, fmd hqt for f(x) = I n x and determine
the value o f f '(2.03) from the following table of values of f(x), if it is given that the
maximum round-off error in the function evaluation is 5 x lod
-

W e now end this unit by giving a summary of what we have convered in it.

12.7 SUMMARY
In this unit w e have covered the following :
1) .If a function f(x) is not known explicitly but a table of values of €(x) corresponding to
a set of values of x is given then its derivatives can be obtained by numerical
differentiation methods.
2) Numerical differentiation formulas using
(i) the method of undetermined coefficients and
(ii) methods based on finite difference operators can be obtained for the derivatives
of a function at nodal or step points when the function is given in the form of s
table.
3) When it is required to find the derivative of a function at off-step points then the
methods mentioned in (2) above cannot be used. In such cases, the methods derived
from the interpolation formulas are useful.
4) Higher order solutions can be obtained by Richardson's extrapolation method which
uses the lower order solutions. These results are more accurate than the results
obtained directly from higher order differentiation formulas.
5) Round-off errors play a very important role in numerical differentiation. Sometimes,
if the step size is too small, the round-off errors gets magnified unmanageably. In
such cases the optimal step length for the given formula could be used, provided that
it can be detennined.

El) Let f '(x) = a, + a , f, + a,fi. Setting f(x) = 1,x, x2, we obtain

3 2 1
Solving we obtain a, = - a, = 2 s2= - -
2b

E2) O(h) method : hf; - (Ik-fk-

0(h2) method : hf; =

Numerical Differentiation Integration
and Solution of Differential Equations
0(h3) method : hf l-l l f k - 22f,- + 9$-,- 6fk-3
6

E3) Using formula (It!), we have

Ushg formula (33).

E4) Using formula (U)),we have

Using (32), we have

t
Exact value fl(x) = lh = 0.002; f "'(x) = - l/x2 = - 0.4 x lo-'
Actual error in f '(500) is 0, whereas in f "(500) it is 0.1 x lo-'. Truncation error in

f '(x) is
- ,Zf"' - -5.33
- -
x lo-' and in f "(x) it is "h2fn = 8.8 x 10-9
3 12

E5) fn tbe given problem xo = 1,xl = 2.. x2 s 3, x3 = 4 and f, = 1, fl = 16 f, = 81 and f:, = 256.
Constructing the Lagrange fundamental polynomials, we get

x3 - 7x2 + 14x - 8
) (x3 -6
3 )
-6x2+ l l x
6
1
P3(x) = G (x) f, + t;(4 f, + L;(x) f, L;( 4 f3 +

p;(x)=g(x) $+L; +L;(x) f,+L;(x)f,

P;' (x) =g(x) f, +L;I (x) f1 +L;'(x) f,+L;'(x) f3
We obtain after substitution,

The exact values off '(x) and f "(x) are (from f(x) = x4)

E6) We are required to find fl(x) and fl'(x) at x = 1.5 and 3.5 which are off-step paihfi
Using the Newton's f(,.ward difference formula with xo = 0, x = 1.5, s = 1.5, w e p t
f'(1.5) = 8.7915 and f "(1.5) = -4.0834.

Using the backward difference formula with xn = 4, x = 3.5, s = - 0.5, we B C ~

f '(3.5) = 7.393 and f "(3.5) = 1.917.
 E7) The difference table for the given problem is : Numerical Differentiation

x f(x) Af A2 f A3 f A4 f
1.3 3.669
0.8i3
1.5 4.482 0.179
0.992 0.41
1.7 5.574 0.220 0.007
1.212 0.48
1.9 6.686 0.268 0.012
1.480 0.060
2.1 8.166 0.328 0.012
1.808 0.072
2.3 9.974 0.400
2.208
-2.5 12182

5:~ Taking xo = 1.5 we see tbat s = 0 and we obtain from tbe interpolation fonnula
6

Exact value is el.' = 4.4817 and error is t 0.0067

E8) Use the q2) formula (33). With h = 0.1, f1'(0.6) = 1.25%, h = 0.2, f "(0.6) = 1.26545,
h = 0.4, f "(0.6) = 1.289394.

Using Richardson's extrapolation formula,

These two results are of q h 3 . To get qh6) result we repeat the extnpolation
technique and obtain

E9) Using (24) with h = 0.1,0.2, we have

E10) If e-,, em 6, are the round-off errors in the given function evaluations f-,, 6,f,
respectively, and if E = mar ((E- ,1 , leal , 1el 1) and M3 = max ( f "'(x) 1 then

E h2
(R(a band (TEIr 7 M Y
Numerical Differeatiotion integration
and Solution of Differential Equations
-
Ifweuse IRJ JTEJ,weget

and error is given by

Ifweuse -
IRI lTEl =mi& then

and error is ="(&r

3 2

-
For f(x) = In x and using the second aiterion, we get

b=Pt
-(30 x 1 0 ~ ~0.03.
) ~

For h = 0.03, we get

f '(2.03) =
- -
0.72271 0.69315 o,492667.
0.06

If we take b = 0.01, we get

f '(2.03) = 0.4925.
Tbe exact value off ' (2.03) = 0.492611,

The result deteriorate for b < bw