OLES: JOURNAL OF AOAC INTERNATIONAL VOL. 76, No.

3,1993 615

METHOD PERFORMANCE

Fractional Factorial Design Approach for Optimizing Analytical

Methods
PHILIP J. OLES
Lancaster Laboratories, Inc., 2425 New Holland Pike, Lancaster, PA 17601

Downloaded from https://academic.oup.com/jaoac/article-abstract/76/3/615/5689167 by guest on 18 June 2020

Experimental designs based on procedures of is emphasized rather than just the identification of statistically
Taguchi are described for optimizing analytical significant effects.
methods. Methods were efficiently developed, with Taguchi's methods are often taught in the context of process
improvements in precision and accuracy, for the engineering. This approach in developing robust analytical
determination of cholesterol in foods, magnesium methods is logical when methods are considered as processes.
in feed premix, and moisture in mayonnaise. Signif- Each step in an analysis usually involves one or more factors
icant factors and interactions were identified by or variables that can affect the desired output. In method devel-
using 2-level designs, analysis of variance tech- opment, many of these variables are commonly set at a fixed
niques, and commercially available computer soft- value. Thus, a method specifies "heat for 30 min," "centrifuge
ware. The designs were prepared by using Taguchi for 5 min," or "heat at 40°C". However, the outcome of the
linear diagrams. The methods developed as a re- analysis may be changed because of the normal variation of the
sult of these experiments have proven to be rug- factors, and this effect may never be understood. Simple calcu-
ged and reliable. lations dictate that variations in all of these factors cannot be
studied under normal circumstances. To study x factors at Z
levels, Zx experiments would be required. The consequence of
xperimental designs were first introduced by R.A. these numbers is that many factors go unstudied. The analytical

E Fisher as an agricultural research tool in the 1920s (1).

His primary aim was to obtain the most information pos-
sible about a process with the least number of experiments.
method, under normal and expected variations in these factors,
may then fail to deliver the accuracy and precision obtained
during development. However, some underlying assumptions
can be made based on the knowledge of the process; thereby,
Experimental designs and optimization methods for chemists
were reviewed by Bayne and Rubin (2) and Debets (3) and are the number of experiments can be reduced significantly from
also described in Handbook No. 9 of the National Bureau of Zx. When experiments are carried out in an orthogonal array,
Standards (4). Experimental design techniques for optimizing the assumptions may be controlled, calculations simplified,
gas and liquid chromatographic separations and detector re- and experimental set-up facilitated. However, an unbiased per-
sponses are provided in these surveys. spective from the overall design should be maintained by ran-
Genichi Taguchi (5-7), a Japanese engineer, helped develop domizing the order of trials when they are actually performed
these tools so that they now have found much wider accep- in the laboratory.
tance. Taguchi simplified the application of experimental de- The number of trials chosen for an experimental design is
sign by using a standardized library of basic designs called or- based on the resolution desired. In a factorial design, one can
thogonal arrays, along with some simple methods to modify study main effects as well as interactions between factors. This
these layouts to fit individual situations. His methodology was latter characteristic is a major advantage of the technique but a
developed within an industrial environment and favors produc- major disadvantage of one-at-a-time variable testing. Main ef-
tivity and cost effectiveness over statistical rigor. It also departs fects and interactions can be confounded with one another if
from traditional experimental designs by emphasizing an ap- experimental designs of sufficiently low resolution are chosen.
proach in which variability would be minimized first, and then The relationship between the number of experiments, the num-
the appropriate output variable would be adjusted to the de- ber of factors studied, and the resolution obtained is shown in
sired level. Table 1 (8).
Orthogonal arrays are used to assign factors to a series of
experimental combinations whose results can then be analyzed
Degrees of Resolution
by using a common mathematical procedure. The main effects
of these factors and preselected interactions between the fac-
In a full resolution design, all main effects and all interac-
tors are independently extracted. The identification of control- tions are resolved. Full resolution is obtained by performing Zx
ling factors and the quantitation of the magnitude of the effects experiments. In a resolution V design, all main effects and 2-
Received January 24, 1992. Accepted July 13, 1992. way interactions are resolved. In a resolution IV design, 2-way
616 OLES: JOURNAL OF AOAC INTERNATIONAL VOL. 76, No. 3,1993

Table 1. Resolution in experimental designs Table 3. Process for determination of cholesterol in

foods
Trials Factors Resolution

Weigh sample
3 Full
4 IV Add alcohol
5 III
6 III Saponify sample with KOH
7 III
Extract cholesterol with solvent
16 4 Full
Treat combined extracts
5 IV
6 IV

Downloaded from https://academic.oup.com/jaoac/article-abstract/76/3/615/5689167 by guest on 18 June 2020

7 IV
8 IV
9 III
experiments in fractional factorial designs, the assumptions
10 III
11 III
that often must be made during method development can be
12 III
easily tested. A major failing of this technique occurs when
13 III interactions that confound the observed variance with a main
14 III factor are not identified. Therefore, higher degrees of resolu-
15 III tion should be chosen in those cases where interactions are sus-
pected.
32 5 Full
6 V Results and Discussion
7 IV
— IV For the analyses given in Table 2, a description of the pro-
— IV cess, the factors and levels that were studied, the choice of ex-
16 IV
perimental design, and a summary of the analysis of variance
17 III
with interpretation of results will be described.
Determination of Cholesterol in Foods
31
The determination of cholesterol in foods is generally per-
formed by 2 independent processes, sample preparation and
gas chromatographic analysis of the sample preparation. The
interactions are confounded with one another but are separate sample preparation process used by our laboratory was adapted
from main effects, and in a resolution ILT design, main effects from an AOAC procedure (9) and involved direct saponifica-
and 2-way interactions are confounded. tion of the sample. Spike recoveries were good to excellent for
Experimental design and simplex optimization both can some matrixes, such as eggs or meats, but fair or unacceptable
lead to optimum responses for a process. Tlieir approaches are for many high carbohydrate food products, such as muffins or
quite different, however. Simplex optimization continually pretzels. Sample preparation was the process chosen for inves-
drives a process toward an optimum and may only test a given tigation because the chromatographic process was not ma-
factor once at a specific level. Experimental design, on the trix sensitive (Table 3).
other hand, tests a given factor at a given level several times The factors and levels studied for this process are shown in
and lends some statistical credence to the observed result. It Table 4.
does not necessarily, however, direct the experimenter toward We chose a pretzel matrix for this experimental design be-
the optimum result. cause this represented a matrix from which our lowest recov-
In the present paper, 3 applications of experimental design eries of spiked cholesterol were obtained.
based on the methods of Taguchi are described in detail. Be- In addition to the 6 factors chosen, we expected an interac-
cause many factors can be studied with a minimum number of tion between the type of alcohol and extraction solvent. There-

Table 2. Analysis problems identified for experimental designs

Analysis Problem

Determination of cholesterol in foods Low spike recoveries were obtained (<90%)

Determination of magnesium in feed premix Results were 12% low relative
Determination of moisture in mayonnaise Results were 1 % low relative
 OLES: JOURNAL OF AOAC INTERNATIONAL VOL. 76, No. 3,1993 617

Table 4. Factors and levels for cholesterol design Table 6. Process for determination of magnesium
experiment in feed premix
Factor Level 1 Level 2
Weigh sample

A Type of alcohol Ethanol Isopropanol Ash in muffle furnace

B Extraction solvent Toluene Hexane-diethylether
(85 + 15) Dissolve residue in acids
C Use hexadecane No Yes
D Use pyrogallol No Yes Dilute sample
E Hydrolysis conditions 16h,RT 60 min reflux
Analyze by atomic absorption spectrophotometry
F Spiking level 0.025% 0.05%

Downloaded from https://academic.oup.com/jaoac/article-abstract/76/3/615/5689167 by guest on 18 June 2020

fore, a resolution of IV was desired, and reference to Table 1 19.0±0.2 mg/g). Spike recoveries were carried out for each new
shows that a 16-trial design was required. matrix over a period of 18 months. For over 200 matrixes, the
The spike recoveries for the 16 trials were analyzed by using average spike recovery was 97±5%.
analysis of variance (ANOVA) and computer software ob-
Determination of Magnesium in Feed Premix
tained from the American Supplier Institute, Inc., (Dearborn,
MI, 48126) and are tabulated in Table 5. Insignificant effects Magnesium in a feed premix was determined by our labo-
and interactions are pooled with error, and the F ratios of the ratory using AOAC method 977.29, "Calcium, Potassium, and
variances are computed. Taguchi also computed a term called Sodium in Electrolyte Replenishers" (11). Magnesium was de-
the "purified sum of squares" S', which is the sum of squares termined by using atomic absorption spectrophotometry.
minus the variance due to error. Finally, a term used by Taguchi Samples and standards were diluted with a solution of lan-
was calculated called the rho %, defined as the relative contri- thanum as a matrix modifier so that the final concentration was
bution of S' for each factor, interaction, or error to the total S' equivalent to 0.1% La 2 0 3 . The objective of this experimental
for all causes. design was to determine why our Mg results were consistently
These results were previously interpreted (10). As a result 3.0% vs an expected value of 3.4% for a collaboratively studied
of this experimental design, a procedure was developed that check sample (Table 6).
has proven to be reliable and rugged and is matrix independent. The factors and levels that were studied in this process are
Eighteen months of quality control data for cholesterol in an shown in Table 7.
egg powder reference material (SRM No. 1845, National Insti- The feed premix contained 25% calcium, and we decided to
tute of Standards and Technology, Gaithersburg, MD 20899) study calcium as a factor by increasing this level to 37%. Nor-
produced an average value of 19.3±0.7 mg/g (certified at mal digestion consisted of treating the ash residue with nitric
acid. An additional treatment with hydrogen peroxide (30%)
Table 5. Analysis of variance for cholesterol design was also applied to the residue. Two ash temperatures were
experiment, pooled error 3 investigated. The last dilution of the sample is made with a
solution of lanthanum, which is a matrix modifier. The 2 levels
Factor Df S V F S' rho, %
investigated were 0.1%, as specified in the method, and 0.5%.
A 1.2 1.2
With only 4 factors, a resolution of IV was possible in an
1 — — —
B 1 205.2 205.2 14.8 191.4 10.7
8-trial design. The array for this design is shown in Table 8.
AxB 1 0.0 0.0 — — — The %magnesium that resulted from the 8 trials was ana-
C 1 199.5 199.5 14.4 185.7 10.4 lyzed using ANOVA and is tabulated in Table 9. Again, insig-
AxC 1 0.8 0.8 — — — nificant effects are pooled with error, and in this case, it is im-
BxC 1 200.9 200.9 14.5 187.1 10.5 mediately apparent that only one factor, the lanthanum level, is
DxF 1 28.8 28.9 — — — significant in affecting the magnesium response. The ash tem-
D 1 70.1 70.1 — — — perature over the range studied was insignificant in affecting
AxD 1 5.4 5.4 — — —
Error 2 7.8 7.8 — — —
E 1 714.2 714.2 51.6 700.4 39.2 Table 7. Factors and levels for magnesium design
AxE 1 22.3 22.3 — — — experiment
BxE 1 326.7 326.7 23.6 312.9 17.5 Factor Level 1 Level 2
F 1 1.8 1.8 — — —
(e) 10 138.3 13.8 — 207.4 11.6 A Calcium level 25% 37%
Total 15 1784.9 119.0 — — — B Digestion Normal Peroxide
Df, degrees of freedom; S, sum of squares; V, variance; F, F ratio C Ash temperature, °C 500 600
of variances; S', "purified sum of squares"; rho, %, relative D Lanthanum addition 0.1% (normal) 0.5%
contribution of factor, interaction or error to observed variance.
618 OLES: JOURNAL OF AOAC INTERNATIONAL VOL. 76, No. 3,1993

Table 8. Array for magnesium experiment Table 10. Process for determination of moisture
in mayonnaise
Column
Trial AxB Bx D BxC Open sample container
No. 3 5 6
Stir or sample directly
1 25% Acid 500 1x
Withdraw sample with pipet or spatula
2 25% Acid - 600 5x
3 25% H202 - 500 5x Transfer to weighing dish (tared)
4 25% H202 - 600 1x
5 37% Acid 500 5x Record weight
6 37% Acid 600 1x
Dry (optional) in air

Downloaded from https://academic.oup.com/jaoac/article-abstract/76/3/615/5689167 by guest on 18 June 2020

7 37% H202 500 1x
8 37% H202 600 5x Place in oven No. 1 at 100"C
I
Apply vacuum
Table 9. Analysis of variance for magnesium design
experiment, pooled error' After 4 h, remove, cool, and reweigh

Factor Df S V F S' rho, %

The determination of moisture in food is often considered
A 1 5.62E-5 5.62E-5
to be simple and straightforward. However, when a difference
B 1 0.0138 0.0138 — — — of 3 parts in 370 is being considered, every step in the process
AxB 1 0.00681 0.00681 — — — needs to be examined for its significance.
C 1 0.00226 0.00226 — — —
The factors and levels considered for this experiment are
BxD 1 0.00856 0.00856 — — —
shown in Table 11.
BxC 1 3.063E-4 3.063E^ — — —
D 1 0.597 0.597 0.59 94
Temperature set point is usually a factor in moisture deter-
220
14 0.0380 0.00271 — 0.041 6
minations, but over the relatively narrow range of 95-105°C its
(e)
Total 15 0.635 0.0423 — — — significance was uncertain. Oven No. 1 had wire rack shelves,
and Oven No. 2 had solid aluminum shelves; both were cali-
Df, degrees of freedom; S, sum of squares; V, variance; F, F ratio brated. Two weighing techniques were examined: one using a
of variances; S', "purified sum of squares"; rho, %, relative spatula and the other using a plastic disposable pipet. The for-
contribution of factor, interaction or error to observed variance.
mer technique provides greater exposure of the sample to air.
Two sources of vacuum were also examined: house vacuum
the % magnesium. As a result of this experiment, the level of with potentially greater variation and a vacuum pump. Two
lanthanum routinely used was increased to 0.5% to minimize sample weighing dishes were considered because one provided
matrix effects. greater surface area. Temperature variation within an oven is
common, so oven position was chosen as a factor. Sampling
Determination of Moisture in Mayonnaise
was performed either without stirring to minimize exposure of
In this application, a mayonnaise check sample with an ex- the bulk of the sample to air or with stirring to homogenize the
pected level of 37.3±0.5% (w/w) was analyzed by using a vac- bulk of the sample before each aliquot was removed. Samples
uum oven drying method. Our laboratory obtained 37.0±0.6% were air dried after weighing but before placing in the oven to
for several determinations. The difference in means was con- minimize spattering. We wanted to determine if this step was
sidered significant prompting this investigation (Table 10). necessary. Finally, we chose 2 analysts who had different tech-

Analyte: % Moisture
Matrix: Mayonnaise

37.44

37.37

37.31

37.24

37.18

37.11

37.04

H h
A1 A2 B1 B2 G1 G2 A1 A2 A2 B1 B2
Figure 1. Level average graphs for moisture determination design experiment
 OLES: JOURNAL O F AOAC INTERNATIONAL VOL. 76, No. 3,1993 619

Table 11. Factors and levels for moisture determination Table 12. Analysis of variance for moisture design
design experiment experiment, pooled error3
Factor Level 1 Level 2 Factor Df S V F S' rho, %

A Temperature, °C 95 105 A 0.39 0.39 19.5 0.37 15.4

B Oven Oven No. 1 Oven No. 2 B 0.54 0.54 27.0 0.52 21.7
C Weighing technique Spatula Pipet AxB 0.17 0.17 8.5 0.15 6.3
D Vacuum House Vacuum pump C 0.06 0.06 — — —
E Sample vessel USDAdish Disposable D 0.00 0.00 — — —
F Oven position Top Bottom E 0.00 0.00 — — —
G Sampling Stir Unstirred AxE 0.00 0.00 — — —
H Predry Yes No F 0.07 0.07 — — —

Downloaded from https://academic.oup.com/jaoac/article-abstract/76/3/615/5689167 by guest on 18 June 2020

I Analyst No. 1 No. 2 AxF 0.24 0.24 12.0 0.22 9.2
BxF 0 14 0 14 70 0 12 50
G 0.15 0.15 7.5 0.13 5.4
nical skill levels to determine if there might be variations in H 0.01 0.01 — — —
procedure that were overlooked by the other factors. AxH 0.01 0.01 ~
In choosing a design, we took into account 9 factors and the I 0.04 0.04
possibility of at least 4 interactions. Temperature and oven, CxG 0.01 0.01
(e) 41 0.78 0.02 0.90 37.5
temperature and oven position, oven and oven position, weigh-
Total 47 2.40 0.05
ing technique and sampling were all considered to be likely
interactions. With this number of factors and interactions, we Df, degrees of freedom; S, sum of squares; V, variance; F, F ratio
accepted aresolutionof HI, which still required 16 trials. Given of variances; S', "purified sum of squares"; rho, %, relative
contribution of factor, interaction or error to observed variance.
the simplicity of running a moisture test, we also ran each trial
in triplicate to obtain an estimate of error. The moisture results
obtained from the check sample for the 16 trials were analyzed nificant have more credibility because they are studied several
by using ANOVA and are tabulated in Table 12. Insignificant times. Taguchi's approach simplifies both the assignments of
effects are pooled with error. Three factors are significant: the factors or interactions to columns and the calculations. If al-
temperature of the oven, the oven, and the sampling technique. lowances are made for interactions or, better yet, if a process is
Three interactions are also significant and most easily visual- developed that is free of interactions, the variances from main
ized by using level average graphs as shown in Figure 1. From effects are easily calculated.
these, we concluded that highest moisture results are obtained The designs described in this work include only 2 level fac-
at 105°C with oven No. 2, and stirring before samphng. The tors; therefore, linearity is assumed. Quadratic relationships
interactions show that the following combinations yield the can be studied because Taguchi's designs also extend to 3 level
highest results: temperature at 105°C and oven No. 2; temper- factors. However, resolution drops significantly, and usually a
ature at 105°C and use of the bottom of the oven; and oven No. minimum of 27 trials are required even to study as few as 4 or
2 and use of the bottom of the oven. Equally important, we 5 factors at a resolution of in.
eliminated all the other factors as being significant. No visual Limitations to experimental design may seem obvious but
evidence of sample decomposition was observed; therefore, are worth stating here because neglecting them often leads to
the assumption was made that any combination of factors lead- the failing of this approach: (7) The variance observed for a
ing to the highest result is desired. factor or interaction is only valid over the range studied for
On the basis of these findings, a new procedure was devel- that factor. (2) Confounding an interaction with a main effect
oped. New racks for oven No. 1 were fabricated of solid alumi- can negate the benefit of minimizing the number of experi-
num, a temperature of 100°C was used, samples were stirred ments through the use of the fractional factorial design.
before sampling, samples were not air dried, a pipet was used
for sampling, disposable dishes (less expensive) were used, and
either of 2 analysts can run the test. References
A confirming run of 6 determinations was carried out with
(1) Fisher, R.A., & Yates, F. (1953) Statistical Tables for Biologi-
the check sample using these conditions. An average result of
cal, Agricultural, and Medical Research, 4th Ed., Oliver and
37.2+0.08% moisture was obtained demonstrating an improve- Boyd, Edinburgh
ment in precision and accuracy. (2) Bayne, C , & Rubin, I. (1986) in Practical Experimental De-
signs and Optimization Methods for Chemists, VCH
Conclusions Publishers Inc., New York, NY
(3) Debets, H.J.G. (1985) J. Liq. Chromatogr. 8, 2725-2780
The use of experimental design is described for optimizing (4) National Bureau of Standards (1966) Handbook No. 91, Ex-
analytical methods. Many factors can be studied, and interac- perimental Statistics, U.S. Government Printing Office,
tions can be identified. In addition, the factors identified as sig- Washington, DC, p. 12-1
620 VIGGERS ET AL.: JOURNAL OF AOAC INTERNATIONAL VOL. 76, No. 3,1993

(5) Taguchi, G. (1988) System of Experimental Design, Vol. 1-2, (9)

Official Methods of Analysis (1990) 15th Ed., AOAC, Arling-
American Supplier Institute, Inc., Dearborn, MI ton, VA, sec. 954.03 and 970.51
(6) Wu, Y., & Moore, W. (1986) Quality Engineering, Product (10)
Oles, P., Gates, G., Kensinger, S., Patchell, J., Schu-
and Process Design Optimization, American Supplier Insti- macher, D., Showers, T., & Silcox, A. (1990) J. Assoc. Off.
tute, Inc., Dearborn, MI, pp. 11-26 Anal. Chem. 73, 724-728
(7) Montgomery, D.C. (1991) Design and Analysis of Experi- (11)Official Methods ofAnalysis (1990) 15th Ed., AOAC, Arling-
ments, 3rd Ed., John Wiley & Sons, New York, NY, p. 414 ton, VA, sec. 977.29
(8) Montgomery, D.C. (1991) Design and Analysis of Experi-
ments, 3rd Ed., John Wiley & Sons, New York, NY, p. 339

METHOD PERFORMANCE

Downloaded from https://academic.oup.com/jaoac/article-abstract/76/3/615/5689167 by guest on 18 June 2020

Reporting Ongoing Results of Interlaboratory Comparison
Programs
ELIZABETH A. VIGGERS and JUNE ATKINSON
Ministry of Agriculture and Fisheries, Head Office, Applied Statistics Group, PO Box 2526, Wellington, New Zealand
STEVE PURCELL
Dairy Proficiency, PO Box 10222, Te Rap a, New Zealand

International standards have been established for Many reports use inappropriate techniques, such as correla-
reporting individual ring trials, but no established tion, regression analysis, and the coefficient of nondetermina-
general method has been developed! for reporting tion (12, 13, Harrop, B., personal communication, 1988); as-
ongoing Interlaboratory Comparison Programs sumptions made on the distribution of results are rarely tested
(ILCPs) to participants, although some attempts or even explicit; techniques used may label exactly the same
have been made to address the problem. The pres- analytic performance as adequate for one sample but outlying
ent paper describes a system that has been devel- or unacceptable for another sample; and analysts and labora-
oped, tested, and used in New Zealand and which tory managers often find it very difficult to use the results as
has improvements over many current schemes. It given to assess or improve their own performance.
is consistent with international standards and Discussions took place with industry users of various ILCP
meets the needs of laboratory analysts and manag- schemes to establish what they needed in an output, and a
ers. It emphasizes graphic output, and essential scheme was designed and set up with these needs in mind. The
statistics are given but not emphasized. Feedback final outcome was the result of several years development.
is produced within a week of the analysts' report- Minor modifications to suit the needs of individual users are
ing deadline. The system is based on principles easy to carry out.
agreed to among analysts, the statistician, the sys-
tems analyst, and the scheme manager, and it em-
braces a wide variety of chemical and microbiologi- METHOD
cal tests. The present paper covers only the
statistical and reporting aspects of the system. Certain principles were agreed to as important to the report-
ing of schemes:
(7) Reporting must be timely; when historical data is re-
any Interlaboratory Comparison Programs (ILCPs) ported, it should be built up and reported progressively. Feed-

M are ongoing. Much attention has been given to the

preparation and handling of the samples involved and
some of the essential statistics, such as repeatability and repro-
back should be within days, rather than weeks or months, of the
original analyses.
(2) Reports must be statistically valid, and they must also be
ducibility (1-6), but relatively little has teen actually done clear and easily understood. Histograms and scatter diagrams
since Youden's paper (7) to make the results useful to the ana- should be used rather than regression and correlation analysis.
lysts and laboratory managers involved, although some at- Historical data should be included in a form such as control
tempts have been made to address the problem (8-11). charts.
(3) A central value must be established and reported for each
Received April 20,1992. Accepted September 22, 1992. sample. This will be a known 'true' value if one exists but will