Gifted Education Intemational2003 Vol18, pp 164-174

©2003 A B Academic Publishers

Hongwon Kim and Seokhee Cho*

Korean Educational Development Institute, Seoul, Korea
and Doechee Ahn
Chung-Ang University, Seoul, Korea

development of
mathematical creative
problem solving ability test
for identification of the
gifted in math
Abstract

To identify the mathematically gifted, Mathematical Creative Problem Solving Ability Test (MCPSAT)
was developed. The test was developed for 2 years. In the first year, mathematical creative problem
solving ability was conceptualized through literature review. Exemplary items were developed and pilot
tested. In the second year, the actual test was constructed and standardized. National and local norms
were constructed. Statistical analyses verified that the MCPSAT was valid and reliable to be used in
identification of the mathematically gifted. Types of problems, specific criteria for scoring and statistical
analyses for verification on goodness of tests are described in this paper.

Introduction NRCGTE developed MCPSAT rather

than an intelligence test, aptitude test, or
In 1995, Presidential Commission for basic math skills test. It was because
Educational Reform suggested the mathematics was thought to be one of the
strengthening of gifted education in schools basic subjects for the gifted who wish to be
to maximize the development of the creative professionals either in math,
potential of the gifted. Since 1996, the engineering, or technology areas in the
Ministry of Education has been providing future. There were many traditional math
financial support to the National Research tests, but it was difficult to find valid and
Center on Gifted and Talented Education standardized instruments that can be used
(NRCGTE) at Korean Educational for identification of the gifted in
Development Institute (KEDI). According to mathematics in Korea. Creativity is believed
the 5 year plan (1996-2000) for promotion of to be the important component of giftedness.
gifted education system, the NRCGTE However, traditional math tests used for
research team had developed MCPSAT for identification of the mathematically gifted
identification of the mathematically gifted. did not seem to measure creativity in

164, Gifted Education International

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

mathematics. Traditional math tests The process of math creative problem
generally measured either math solving was regarded as composed of four
computational skills, logical thinking skills, stages, namely, understanding of problems,
or simple application of mathematical planning to solve the problems, execution of
formula to solve problems. the plan, and reflection of the answer and the
whole problem solving process. Throughout
the four stages, mathematical thinking
Definition of Mathematically ability, mathematical creativity,
Creative Problem Solving Ability mathematical task commitment, and
knowledge base are utilized for
mathematical creative problem solving.
The mathematically gifted was defined as
Divergent thinking and convergent thinking
those who show superior ability in solving
are concurrently operated during
the math problems in a creative way. In other
mathematical creative problem solving. So
words, they have high potential to be a
MCPSA can be measured best when the
creative mathematician in the future and
tasks require both of the convergent and
show superior Mathematical Creative
divergent thinking.
Problem Solving Ability (MCPSA). MCPSA
is an ability to produce new solutions by
using existing knowledge base, principles, Characteristics of Instruments for
concepts, and various thinking strategies. Mathematical Creative Problem
Solving Ability Test
To formulate the conceptual frame of
MCPSA represented in Figure 1, theories on Target Population
creativity, problem-solving, and giftedness
of many researchers (Balka, 1974; Haylock, The test was developed to be used for
1984, 1985, 1987; Isaksen et al., 1994; Polya, identification of the mathematically gifted.
1957; Renzulli, 1978, 1985; Urban, 1995; Therefore, the target population of this test is
Wallas, 1926) had been reviewed. those in upper 15-20% in each grade level
from Grade 2-11 in terms of either
intelligence or achievement in math.

Mathematical One Correct

Closed Problems Solution
Thinking Ability

/
Mathematical
Task Commitment

Mathematical Various Different

Open Problems
Creativity Solutions
Figure 1: The Conceptual Frame of MCPSA

Volume 18 No 2, 2003, 165

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

Structure of Tests: Part 1 and Part 2 Example of problem (Haylock, 1984)
1. What do you think the numbers on
The MCPSAT is composed of two parts: Part cards A, B, C might be? List as many
1 and Part 2. Test items in Part 1 are different possibilities as you can think of.
characterized as open and requiring various (A+B)x(C)=9
different answers, meanwhile those in Part 2
are characterized as closed and requiring Example of problem (Haylock, 1984)
only one correct answer. Even when a
mathematical task requires one correct • Redefinition refers to ability to give up
response, it can be considered as creative previous, existing interpretation of
problem solving task if it requires new ideas, familiar objects in order to use them in
approaches and principles to be solved. some new ways.

Each of Part 1 and Part 2 is composed of

4 different difficulty levels from grade 2 to
grade ll(grade 2-3, grade 4-6, grade 7-9,
grade 10-11). In each level, two identical tests
(Type A and Type B) are constructed. Write down as many answers as you can
think of;
Part 1: Test on Mathematical Creativity These two figures are common in................ .
These two figures are different in.............. ..
Mathematical creativity is measured in Part
1 of test. Mathematical creativity means an Part 2: Test on Mathematical Thinking Ability
ability that can produce various solutions for
a math problem. Mathematical creativity is Mathematical thinking ability is measured in
composed of four sub-factors as follows: Part 2 of test. It is regarded as composed of
• Fluency refers to generation and creation seven sub-abilities as follows:
of many responses and ideas. The more • Intuitive insight refers to figuring out the
the number of correct answers, the more relationship and structure of given
fluent the person is. information and conditions, find out the
• Flexibility refers to generation of many critical cues of problem solving.
different categories of responses and • Organization of knowledge refers to
ideas overcoming the fixedness. The collecting and manipulating the
more the number of different categories necessary information for solving
of correct answers, the more flexible the problems.
person is. • Space perception and visualization refers
• Originality refers to generation of to imagery ability that transforms the
responses and ideas different from other given information into the space
persons. It means rarity and uniqueness information.
of answers. • Abstraction refers to representing the ill-
• Elaboration refers to extension of a structured mathematical problems into
simple design to a more complex or concept, symbol, formula, and figure.
intricate design. However, elaboration is • Reasoning refers to systematic reasoning
not measured by MCPSAT. in terms of inductive thinking and
• Problem-solving refers to the ability to deductive thinking.
produce many solutions. • Generalization and application refers to

166, Gifted Education International

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

 generalizing and applying the problems, all sub-abilities and contents in
mathematical relationships. the taxonomy matrix could not be included
• Reflective thinking refers to a kind of in the tests. The number of problems in Part
metacognitive processes on his/her own 1 of each level and type is 8 or 9. The number
problem solving process and its of problems in Part 2 is 16 for each of the
relevancy with the problem. level and type. Time needed for
implementation of a test varies from 50 to 80
Problems in Part 2 require one correct minutes depending on the level of the tests.
answer, but it is difficult for students solve
the problems by applying knowledge and
strategies simply and directly that they learn Procedures of Test Development
in school.
The test was developed for 2 years. In the
Taxonomy of Sub-categories of Abilities and first year, conceptualization of MCPSA
Contents of MCPSAT through literature review was carried out,
the types of exemplary items were
Example of sub-thinking abilities and sub- developed, and then pilot test was carried
contents of MCPSAT(Part 1 and Part 2) of out. In the second year, actual test items were
Type A for grade 2-3 are shown in Table 1. developed and standardization was carried
Because of limitation of the number of out and national/local norms were made.
Table 1: An example of taxonomy of abilities and content domains for grades 2-3 in
primary school(Type A)
en
§
·,p
1l
~
nl
s g.
:0
en
p;:j
.....
s:: .....ubO
.s
"0 nl nl >,
"0
s
1-< 1-< .....
s::: {l {l
s:::
nl s::: en
0 nl .8 .e-.
;.:::
~
,..J
~ en
s::: en bO
0 §
"§ <
1-<
~ s::: u
~
QJ :0 "0 0
::s 0 :0
"S;:i
nl
1-<
QJ
CLJ
nl
QJ
:0
nl
Ql ....~ g.
:0
nl
:0
u
s:::
s::: "0
(il ·ca ~
i
.g..... nl
....
en
:0
z 8' :::E ~
QJ
Cll p;:j
;:i
~ u p:; Cll
0
u 0... Cll
.e-
..... Intuitive Insight • • • • •
== • • • • •
~ Organizing of information
bl)
Visualization • • • •
.....s===
~
Abstraction • • • •
E-o Reasoning Induction • • • • •
.,
o:S
Deduction • • •
:E
Generalization • • • • •
Reflection • • • • • •
.e- Fluency • • • • • • •
o:S ·:;:
ns:C Flexibility • • • • • • •
:E ...~
u Originality • • • • • • •
• Grade 2-3 students don't learn probability and statistics in regular school curriculum.
Volume 18 No 2, 2003, 167

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

Pilot Test Subjects. The subjects for verification of the
goodness of the actual test were recruited by
Three types of test A , B, and C were made stratified random sampling techniques from
and the pilot test was carried out. metropolitan cities, mid-sized cities, and
The Subjects were 1,214 primary school rural areas. The number of subjects was
students, 722 middle school students, 672 4,991 primary school students, 3,367 middle
high school students, and 89 science high school students, 2,345 high school students,
school students. and 116 science high school students. Total
number of subjects was 10,819.
Verification of goodness of test and each
test item was carried out through analysis of Scoring. Part 1: Math Creativity. Math
reliability (i.e., Cronbach a coefficients); creativity in one problem was measured by a
analysis of fitness and difficulty level of each sum of scores on fluency, flexibility, and
item based on item response theory; analysis originality in one problem. Fluency was
of correlation coefficients between item score scored according to the number of correct
and total score; analysis of mean differences answers on one problem. Flexibility was
among grade levels, and between high- scored according to the number of categories
achievers and low-achievers; and analysis of of correct answers on one problem.
easiness for making scoring criteria for Originality was scored according to a rarity
measuring math creative problem solving or uniqueness of correct answers on one
ability. Based on the results of statistical problem. An originality score for each
analyses, inappropriate test items were taken subject was calculated by using the
out or revised . categories obtained for the flexibility scores
from all the subjects. Based on the
Development of Actual Test Items and frequencies of the categories expressed by
Standardization the entire sample, scores were awarded
according to the following schedule.
Actual test items were developed based on
the results of the pilot test. The goodness of Scores Number of subjects who
each test item was analyzed and only those answered the same category
good items were selected and actual test 0 5% or more subjects
instruments were constructed. As a result, 1 2% to 4.99% subjects
two identical types of tests, Type A and Type 2 less than 2% subjects
B were constructed.
The example of scoring criteria for math
creativity is presented in Table 2.
Table 2: Example of scoring criteria for math creativity
Grade 2-3 Type A No. of item Patterns of number series

Problem :John arranged numbers in four succeeding parentheses according to a rule as shown
in the example. Place four numbers according to your own rule and write down the rule that
you used.
Example of problem and answers
Problem: ( ) - ( ) - ( ) - ( )
Answer: ( 1 ) - ( 2 ) - ( 3 ) - ( 4 ): increase the number by 1
(N = 999)
168, Gifted Education International

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

Category of Responses Real Responses grade 2 (%) grade 3(%) Total(%)

1. increase by ( ) • increase by 2, 3, 4 380 400 780

(38.0) (40.0) (78.0)

2. decrease by ( ) • decrease by 2, 3, 4 124 88 212

(12.4) (8.8) (21.2)

3. multiply by ( ) • multiply by 3, 6 (2)* - 19 19

(1.9) (1.9)

4. an odd (even) number •2-4-8-10 - 5 5

• 2 - 10 - 16 - 18 (0.5) (0.5)
•1-3-5-7 - 5 5
• 1-5-11-17 (0.5) (0.5)

5. divide by ( ) • 16-8-4-2 - 5* 5*
• 81-27-9-3 (0.3) (0.3)

6. increase by ( ) and •2-3-4-1 (2)* 2* 1* 3*

decrease by ( ) (0.2) (0.1) (0.3)

7. add all number before •1-2-3-6 (2)* - 3* 3*

(0.3) (0.3)

8. increase by 1, 2, 3, 4 • 1 - 3 - 6 - 10 (2)* - 3* 3*
(0.3) (0.3)

9. decrease by ( ), •3-2-1-4 (2)* 2* - 2*

increase by ( ) (0.2) (0.2)

10. increase by ( ) and • 1 - 4 - 2 - 3 (2)* 1* - 1*

decrease by ( ) and (0.1) - (0.1)
increase by ( )

11. multiply by 1, 2, 3 • 1 - 1 - 2 - 6 (2)* - 1* 1*

(0.1) (0.1)

12. multiply by itself and • 2 - 4 - 16 - 256 (2)* - 1* 1*

write the number after (0.1) (0.1)

• ( )* : originality score awarded

Part 2: Math Thinking Ability. Since only one range was very wide in all schools, even in
correct answer was required, the score was the science high school where only those
given depending on whether the answer was who are already identified as the
right or wrong. scientifically gifted are studying. Thus, it
was verified that MCPSAT was good in
Analysis of Goodness. Means, Score Range on discriminating abilities in students.
each test are presented in Table 3. The score

Volume 18 No 2, 2003, 169

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

 Table 3: Score range and Means of MCPSAT

Grade 2-3 Grade 4-6 Grade 7-9 Grade 10-11

~-===~~=---~
General Science

Part 1 Type A Range 0-92 0-134 0-77 0-50 9-62

TypeB

Part2 TypeA

TypeB

Note. * The maximum score of Part 2 of the test is 108.

Reliability. Reliability was analyzed by and bigger than .9 'reliablity is very high.' As
computing Cronbach a coefficients, as shown in the analyses of Cronbach a
presented in Table 4. Cronbach a coefficients coefficients in Table 5, reliability of the
which are bigger than .5 means 'reliability MCPSAT was verified as good in general.
exists.'; bigger than .7 'reliablity is good.';

Table 4: Cronbach a coefficients of MCPSAT Grade level

Part 1 Part 2
Type A TypeB Type A TypeB
Grade 2-3 .62 .68 .73 .74
Grade 4-6 .76 .67 .73 .74
Grade 7-9 .60 .63 .73 .67
Grade 10-11 .55 .60 .72 .74

Internal Validity. Internal validity was the logit coefficients of items between -.73
analyzed based on item response theory. The and .73. The 2nd part of the test showed the
coefficients on item adequateness were logit coefficients of items between -2.51 and
computed and analyzed. The infit and outfit 4.35. The range of item difficulty of the 2nd
coefficients of items were almost 1.0. part of the test was greater than that of 1st
Therefore, it was verified that the internal part of the test.
validity was good.
Discriminality. The item discriminality was
Difficulty. The item difficulty was analyzed analyzed based on item response theory. The
based on item response theory. If the logit point-biserial correlation coefficients were
coefficient is 0.00, the difficulty is average, computed. The point-biserial correlation
and if it is bigger than 0.00, it means the item means correlation between item scores and
is difficult, and if it is smaller than 0.00, the total scores, and if the value is negative, the
item is easy. The 1st part of the test showed item does not discriminate adequately

170, Gifted Education International

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

abilities of students. All items' values were middle, one high school are presented in
positive in the point-biserial correlation Table 5. Table 6 shows that MCPSA scores,
analyses. math academic achievements scores, and IQ
are inter-related and at the same time each of
Correlation Coefficients among MCPSA Scores, them are composed of separated abilities.
Math Academic Achievements Scores, and IQ. The correlation coefficients between Part 1
Correlation coefficients among MCPSA scores and Part 2 scores is .44-.60, and this
scores, math academic achievements scores, shows that two constructs are somewhat
and IQ of students in one primary, one different.

Table 5: Correlation coefficients among MCPSA scores, math academic

achievements scores, and IQ(Type A)

Grade 2-3 (n=73) Grade 4-6 (n=162)

Part 2 IQ MA Part 2 IQ MA
Part 1 .45** .40** .31** Part 1 .60** .56** .58**
Part 2 .42** .43** Part 2 .39** .42**
IQ .30** IQ .59**

Grade 7-9 (n=162) Grade 10-11 (n=180)

Part2 IQ MA Part 2 IQ MA
Part 1 .56** .48** .46** Part 1 .44** .34** .49**
Part 2 .39** .44** Part2 .40** .53**
IQ .60** IQ .52**

Note. ** p<.OOl MA: Math Academic Achievement Score in school

MCPSA and National Math Olympiad.

The Math Olympiad can be one of the
good means to identify math
giftedness. There were eleven high
school students who were subjects of
MCPSAT and participated in
National Math Olympiad held by
Korean Math Society and won
medals. The results of these students
in MCPSAT and Olympiad are
presented in Table 6. At Part 2 of the
test measuring math thinking
abilities, when science high school
students were excluded, eight
students' scores exceeded the scores
of ninety-nine percent of total
subjects. At Part 1 of the test Mathematical problem-solving

Volume 18 No 2, 2003, 171

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

 measuring math creativity, when science Use of MCPSAT. MCPSAT is currently used
high school students were excluded, seven by gifted education centers attached to
students' scores exceeded the scores of universities and school boards and special
ninety-eight percent of total subjects. classes for the gifted in regular schools to
Two bronze-medal winning students' identify the mathematically gifted all over
MCPSAT scores on Part 1 and Part 2 of the the country. These educational institutions
test were very high. These results showed go through at least three different stages of
that MCPSAT had a good validity. identification: Recommendation, MCPSAT,
and oral interview or observation after
Construction of Norms. Norms were placement in enrichment program. Later, the
constructed in accordance with test level tests can be used not only for identification
(grade 2-3, grade 4-6, grade 7-9, grade 10-11), but also for evaluation of the effects of
Parts (Part 1 and Part 2), gender, grade, enrichment programs using two different
fluency, flexibility, and originality. The types of MCPST for pre- and post-tests.
student's raw score, frequency, and
percentile are included in the norm.

Table 6: Comparison of MCPSAT results and Math Olympiad results

Subjects School Grade Gender Type Partl Part2 Olympiad

Score Percentile Score Percentile
1 Science 10 M A 18 72.0 (69.4) 84 - (99.6) EM
2 High School 10 F A 35 98.1 (96.4) 98 - (100.0) within
15% from
the top
3 10 M B 50 99.4 (97.6) 81 - (99.8) Bronze
Medal
4 11 M B 82 100.0 (100.0 88 - (100.0) Bronze
Medal
5 General 11 F A 16 69.9 (62.4) 53 99.5 (97.1) ME
6 High 11 M A 46 99.7 (98.8) 38 96.7 (93.6) ME
7 School 11 F A 46 99.7 (98.8) 44 98.4 (95.3) ME
8 11 M A 24 87.6 (85.0) 31 92.8 (89.1) ME
9 11 F A 47 99.9 (99.3) 50 99.4 (96.9) ME
10 11 M B 25 72.5 (69.2) 48 99.2 (96.8) ME
11 11 M B 47 98.7 (96.6) 65 100.0(99.0) ME

Note. * ME : Medal for Encouragement

* The number in parentheses is a percentile when science high school students are not included and the
number out of parentheses is percentile when science high school students are included.

172, Gifted Education International

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

Implications Finally, it is highly recommended that
Part 1 of the test for mathematical creative
The implications acquired from this research problem solving ability be introduced in
are as follows. schools, since it can influence the modes of
presentation of math problems in the
First, when both tests of Part 1 and Part 2 direction of stimulating students'
are used, the identification of the mathematical divergent thinking and
mathematically gifted can be more valid and enhance their interest in mathematics.
reliable.

Second, the good items to measure math References

creativity are unstructured and ill-defined
problems to which students can respond Balka, D, S. (1974). The development of an
with diversity of solutions. However, it was instrument to measure creative ability in mathematics,
difficult to devise such items. It is, therefore, Unpublished doctoral dissertation, Univ. of
necessary to carry out more in-depth study Missouri, Columbia.
in devising ill-defined or unstructured test
Haylock, D. W. (1984). Aspect of Mathematical
items more to measure math creativity. Creativity in Children Aged 11-12. Unpublished
doctoral dissertation, London University,
Third, in math creativity, the correlation London, Great Britain.
between fluency and flexibility appeared to
be very high, more than r = .95. This fact Haylock, D. W. (1985). Conflicts in the assessment
implies that it may be sufficient to measure and encouragement of mathematical creativity in
only one variable. In addition, by requiring school children. International Journal of
students to produce as many responses as Mathematics Education, Science, and Technology,
16(4), 547-533.
possible within the limited time period to
measure flexibility and fluency, students' Haylock, D. W. (1987). A framework for assessing
ability to produce original solution could not mathematical creativity in school children.
be maximally executed. Therefore, further Educational Studies in Mathematics 18, 59-74.
study about this is needed.
Heller; K. A., Monks, F. Z., & Passow, A. H. (Eds.).
Fourth, elaboration of math creativity (1993), International Handbook of Research and
was not measured in this study considering Development of Giftedness and Talent, Pergamon Press.
the elaboration as rendering simple designs
Kim, H.W., Kim, M.S., & Song, S. H. (1996),
into more complex designs. However,
Development of Mathematical Creative Problem
elaboration in math problem solving can be Solving Ability Test for Identification of the Gifted in
redefined. Elaboration of mathematical Math (1), Seoul: KED!.
thinking can result in an elegant formula for
expressing various solutions or responses. If Kim, H.W., Kim, M. S., Bang, S.-J., & Hwang,
elaboration in math problem solving can be Dong-Jou. (1997). Development of Mathematical
redefined in this way, it may become Creative Problem Solving Ability Test for
possible to measure elaboration and Identification of the Gifted in Math (2), Seoul: KED!.
meaningfully use this in developing math
creativity. Isaksen, S. G., Dorvel, K. B., & Treffinger, D.
(1994). Creative Approaches to Problem Solving.
Iowa: Kendall/Hunt Publishing Company.

Volume 18 No 2, 2003, 173

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015

Poyla, G. (1957). How to solve it, 2nd ed, New York: Urban K. K. (1995). Creativity: A Componential
Doubleday & Company, Inc. Approach. Paper presented at the Post Conference
China Meeting of the 11th World Conference on
Renzulli, J. S. (1978). What makes giftedness? Gifted and Talented Children. Beijing, China,
Reexamining a definition. Phi Delta Kappan, 60 (3), August 5-8.
180-184, 261.
Wallas, G. (1926). The Art of thought. New York:
Renzulli, J. S., & Reis, S. M. (1985). The Schoolwide Harcourt, Brace.
Enrichment Model: A Comprehensive Plan for
Educational Excellence, Creative Learning Press, Inc.

An Ant

An Ant
Pauses
At a twig
On my mattress,
A temporary reprieve.
The stunted grass
Casts a shadow,
Superimposed on a mosaic
Of dappled shade,
Strewn over the abstract continuity
Of a muddy print.
Tiger Asics?

An Ant
Dashes on
Colliding.
Helter-skelter.
The shrivelled cocoon
Of a tortured Acacia leaf falls,
Rolls -
A broken parabola
Bounding in phase
With the chaos
Of avalanching folds.

An Ant
Sprawls with futility-catapulted into the grass.
How vast the scale of life is
Surveyed from a blue mattress
Under a tree.

Scorr BREDIN
(age 14)

174, Gifted Education International

Downloaded from gei.sagepub.com at The University of Iowa Libraries on June 2, 2015