Received: 8 February 2020 Revised: 9 June 2020 Accepted: 20 June 2020

DOI: 10.1111/jcal.12474

ARTICLE

Developing a flipped learning approach to support student

engagement: A design-based research of secondary school
mathematics teaching

Chung Kwan Lo1 | Khe Foon Hew2

1
Department of Mathematics and Information
Technology, The Education University of Hong Abstract
Kong, Tai Po, Hong Kong The overarching goal of this research project is to develop a set of theoretical and
2
Faculty of Education, The University of Hong
empirical based design principles to support student engagement in flipped mathe-
Kong, Pok Fu Lam, Hong Kong
matics learning. We used a design-based research approach in which a series of stud-
Correspondence
ies was conducted in a secondary school. First, an analysis of practical problems was
Chung Kwan Lo, Department of Mathematics
and Information Technology, The Education performed via exploratory studies (Studies 1A and 1B), randomized experiments
University of Hong Kong, Tai Po, Hong Kong.
(Studies 2A and 2B), and a literature review (Study 3). Second, a basic flipped lesson
Email: chungkwanlo@eduhk.hk
prototype was developed based on our empirical findings and relevant theoretical
Peer Review
work of student engagement and self-determination theory. Third, we refined and
The peer review history for this article is
available at https://publons.com/publon/10. evaluated our practice with two empirical studies (Studies 4 and 5). Finally, we
1111/jcal.12474.
reflected on each stage of our research project to produce a set of design principles
for future practice. This study thus contributes to our knowledge of methods to sup-
port student engagement in flipped mathematics learning.

KEYWORDS

design-based research, flipped learning, mathematics education, self-determination theory,

student engagement

1 | I N T RO DU CT I O N cognitive learning outcomes relative to the traditional lecture format

in K-12 and higher education. Their results for mathematics education
In recent years, multiple studies examined flipped learning in mathe- suggested a small but significantly positive effect (k = 15; Hedges'
matics education (Lo, Hew, & Chen, 2017; Yang, Lin, & Hwang, 2019). g = .205; 95% CI [.120, .291]) of flipped learning.
In the two-component model of Bishop and Verleger (2013), flipped Although evidence supports the positive effects of flipped learn-
learning is a technology-enhanced instructional approach that (a) ing on student achievement (see Cheng et al., 2019; Lo et al., 2017;
delivers course materials before each class session using instructional van Alten, Phielix, Janssen, & Kester, 2019 for a review), challenges to
videos, text-based materials, and/or online exercises (pre-class) and supporting student engagement in this instructional approach have
(b) engages students in interactive learning activities inside the class- yet to be fully addressed. van Alten et al. (2019), for example, found a
room (in-class). In a flipped learning environment, students have non-significant effect on student satisfaction, one of the key indica-
access to the pre-class learning materials anytime and anywhere and tors of student engagement (Fredricks, Blumenfeld, & Paris, 2004),
have more class time to seek help from their teacher and peers (Lo across studies of flipped learning. In a broader context, their findings
et al., 2017). In a recent meta-analytic study, Cheng, Ritzhaupt, and are in line with the meta-analysis of Spanjers et al. (2015) on blended
Antonenko (2019) found that flipped learning increases students' learning. In fact, several teachers discovered that some students had a
negative perception of the flipped classroom approach despite
improvements in their learning achievement. One representative case
This manuscript has not been published in any language before; and is not under
consideration concurrently for publication elsewhere. was reported by van Sickle (2016), whose flipped-class students

142 © 2020 John Wiley & Sons Ltd wileyonlinelibrary.com/journal/jcal J Comput Assist Learn. 2021;37:142–157.
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LO AND HEW 143

significantly outperformed the students in her traditional class on the with the course than those in her traditional class. The research find-
final examination of her college algebra course. However, she noticed ings related to emotional engagement in flipped learning appear to be
that student perception on a few aspects (e.g., course interest) inconsistent.
decreased significantly after flipped learning began. How then can we Third, cognitive engagement concerns the level of investment in
support positive student engagement? In this paper, we report a 2- learning (Fredricks et al., 2004). This dimension can be conceptualized
year design-based research project that we conducted to address this as the students' desire to go beyond the course requirements and
very question. Before we describe the project in detail, this introduc- their preference for challenges (Connell & Wellborn, 1991; Newmann,
tory section first briefly explains the two theoretical perspectives (i.e., Wehlage, & Lamborn, 1992). In their research method course, Hew,
student engagement and self-determination theory) used in this study. Huang, Chu, and Chiu (2016) used optional learning tasks to evaluate
We then detail the rationale of using the design-based research students' cognitive engagement. They found that the use of game
approach and its potential contribution to the literature of flipped design elements (e.g., digital points and badges) could motivate stu-
learning. dents to complete more difficult tasks with higher quality relative to
students in a non-gamified learning environment. Their results thus
indicated an increase in students' cognitive engagement in their
1.1 | Theoretical perspectives gamified class.

1.1.1 | Student engagement

1.1.2 | Self-determination theory
According to Fredricks et al. (2004), student engagement is important
for learning because of its association with various desirable out- To support student engagement, Fredricks et al. (2004) suggested the
comes, such as lower dropout rates and students' increased willing- use of self-determination theory, which suggests that human beings
ness to go beyond the course requirements. The researchers have three fundamental psychological needs: autonomy, relatedness,
conceptualize student engagement as a multidimensional framework and competence (Ryan & Deci, 2000). If these needs are satisfied, stu-
of three dimensions: behavioural engagement, emotional engagement, dents are more likely to engage in the learning process (Fredricks
and cognitive engagement. Because this three-dimensional framework et al., 2004; Ryan & Deci, 2000). Abeysekera and Dawson (2015)
is derived from a comprehensive synthesis of educational research, it claimed that self-determination theory can be applied in the context
basically covers everything related to student engagement in school of flipped learning.
settings (Fredricks et al., 2004). Moreover, these dimensions are First, autonomy concerns whether an individual is free to decide
widely adopted by researchers in their investigation of student what to do and how to do it (Ryan & Deci, 2000). People generally
engagement (Bond, Buntins, Bedenlier, Zawacki-Richter, & prefer to have control over their choices and actions. Ryan and
Kerres, 2020). This framework thus offers a solid grounding for our Deci (2000) found that people with autonomy demonstrate more
discussion of student engagement in flipped learning. interest and confidence in an activity, leading to better performance
First, the core idea of behavioural engagement regards participa- and persistence. A sense of autonomy provides a motivational drive
tion (Fredricks et al., 2004). Both positive conduct (e.g., following for students' behavioural engagement in a course (Skinner, Furrer,
rules) and the absence of disruptive behaviour (e.g., skipping classes) Marchand, & Kindermann, 2008). Participants with a high level of
are related to this dimension (Finn, Pannozzo, & Voelkl, 1995; Finn & autonomy are also likely to enjoy their lessons (i.e., have a higher level
Rock, 1997). Behavioural engagement also includes completing the of emotional engagement) (Skinner et al., 2008). Autonomy is also
required pre-class learning tasks. In their review study, however, assumed to lead to more extensive cognitive engagement because it
Akçayır and Akçayır (2018) found that students' inadequate prepara- gives learners more time and space to determine and evaluate their
tion before class is one of the major challenges to flipped learning. goals and decisions (Rotgans & Schmidt, 2011). In a typical mathemat-
Another behavioural challenge is a decrease in student attendance ics flipped classroom, students can access their online learning mate-
(Giannakos, Krogstie, & Chrisochoides, 2014). In her flipped mathe- rials anytime and anywhere, so more time inside the classroom can be
matics course, for example, Larsen (2015) observed that adult learners spent on student-centred learning activities (Lo et al., 2017).
tended to withdraw from class discussion and even intentionally Abeysekera and Dawson (2015) argued that this active learning
skipped classes. In her case, flipped learning had a negative effect on nature of flipped learning can satisfy students' need for autonomy.
students' behavioural engagement. Second, relatedness concerns a person's connections with others
Second, emotional engagement concerns students' affective reac- (Ryan & Deci, 2000). According to Furrer and Skinner (2003), stu-
tions, such as interest, satisfaction, and level of anxiety (Connell & dents' sense of relatedness with peers contributes to their emotional
Wellborn, 1991; Fredricks et al., 2004; Skinner & Belmont, 1993). On engagement and is a key predictor of their learning motivation and
one hand, some studies (e.g., Dove & Dove, 2015) have reported that performance. Students are more engaged in learning activities if their
students' positive flipped learning experience helped decrease their needs for relatedness are supported (Fredricks et al., 2004).
anxiety in learning mathematics. On the other hand, van Sickle (2016) Abeysekera and Dawson (2015) noted that a flipped learning environ-
found that the students in her flipped algebra class were less satisfied ment facilitates student interaction and encourages the formation of
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
144 LO AND HEW

small learning groups, which provides the potential to increase their aimed to improve educational practices through iterative analysis,
sense of relatedness with peers. design, development, and implementation, based on collaboration
Third, competence concerns whether one's behaviour can be among researchers and practitioners in real-world settings, and lead-
effectively enacted (Ryan & Deci, 2000). In a school context, students ing to contextually sensitive design principles and theories” (pp. 6–7).
gain a sense of competence when they have the ability to meet the These anticipated research outcomes suit the current needs of flipped
challenges of schoolwork and are supported by capable others learning practice. Lo (2018) further argued that this research approach
(Niemiec & Ryan, 2009). Students also feel more competent when enables researchers to strengthen the theoretical foundations of
they can play an active role in disseminating what they have learned flipped learning. With reference to the recent review by Yang
(Prince, 2004). Their sense of competence contributes to a higher et al. (2019), however, there is an absence of design-based research in
level of cognitive engagement and supports their persistence when flipped mathematics education. This study thus has potential to con-
handling challenging tasks (Fredricks et al., 2004). tribute to the literature of flipped learning and open new dialog in
academia.
Reeves (2006) listed four sequential steps of design-based
1.2 | Design-based research research: (a) analysis of practical problems, (b) development of solu-
tions based on existing knowledge, (c) evaluation of the solutions in
The overarching goal of this research project is to investigate methods practice, and (d) reflection on the design principles produced.
to support student engagement in a flipped mathematics learning McKenney and Reeves (2012) reviewed existing models for design-
environment. To achieve this goal, we use a design-based research based research in education and proposed a general model that por-
approach to develop theoretical and empirical design principles to trays three main phases of design-based research: (a) analysis/explora-
help mathematics teachers create or refine their flipped courses. With tion, (b) design/construction, and (c) evaluation/reflection. During the
regard to design principles of flipped learning, Lo et al. (2017) pres- analysis/exploration phase, researchers work closely with practi-
ented some for mathematics flipped classrooms. However, these prin- tioners to gain an in-depth understanding of the educational problem.
ciples do not focus specifically on supporting student engagement in During the design/construction phase, potential solutions to the edu-
mathematics learning. No theories related to student engagement cational problem are generated and considered. A solution prototype
were drawn, and the principles were proposed through a synthesis of is further developed. During the evaluation/reflection phase, the solu-
research. Further empirical studies are required to develop and/or tion is tested and refined in authentic educational settings. These
refine any design principles in authentic mathematics flipped three phases have been reflected in Reeves's (2006) model which we
classrooms. employed in this study.
The design-based research approach is ideal for this research pro- Reports of the full design process can help researchers and practi-
ject as this research approach focuses on theory and practice tioners see the approaches to addressing the educational problem
(McKenney & Reeves, 2012). Wang and Hannafin (2005) defined throughout the entire process (McKenney & Reeves, 2012). However,
design-based research as “a systematic but flexible methodology Vanderhoven, Schellens, Vanderlinde, and Valcke (2016) observed

F I G U R E 1 Summary of the four steps

used in this design-based research project
[Colour figure can be viewed at
wileyonlinelibrary.com]
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LO AND HEW 145

TABLE 1 Course information of Studies 1A and 1B

Study Course topic Duration Pre-class learning activities In-class learning activities
1A Coordinate geometry 2 weeks (two 50-min Focusing on concepts and Focusing on problem-
lessons per week) formulas (e.g., distance solving
formula) with work • Review of pre-class
examples materials (10 min)
• Two to three instructional • Solving simple problems
videos (within 6 min) individually or in pairs
• Two online multiple- (20 min)
choice questions for each • Solving advanced
video problems in groups
(20 min)
1B Arithmetic and geometric 4 weeks (two 1-hr Focusing on concepts and Focusing on problem-
sequences and their lessons per week) formulas (e.g., general term solving
summations of geometric sequences) • Review of pre-class
with work examples materials (10 min)
• Two to three instructional • Solving simple problems
videos (within 6 min) individually or in pairs
• Two online multiple- (20 min)
choice questions for each • Solving advanced
video problems in groups
(30 min)

that previous reports of design-based studies usually documented 1A) and high-ability students (Study 1B). Working with these two
only one of these steps without presenting a comprehensive picture groups through the exploratory study approach gave us the experi-
of the study. The researchers therefore followed the steps of ence of handling students with different learning needs in the context
Reeves (2006) to conduct their study and organize their articles to of flipped learning, as well as understanding students' frustrations or
present their full research report (see Vanderhoven et al., 2016 for a challenges concerning flipped learning activities. Table 1 presents
review). more detailed information about the courses of Studies 1A and 1B.
Second, Study 2 focused on the design and construction of pre-
class learning activities. Instructional videos and online exercises are
2 | METHODS commonly used in flipped mathematics courses (Lo et al., 2017). How-
ever, in the words of some mathematics students, “The videos are
Following Vanderhoven et al. (2016), we adopted Reeves's (2006) very helpful but I sometimes have a hard time paying attention”
approach to design-based research as our methodological foundation. (Ogden, 2015, p. 786). As using the right video style can increase the
This section describes how the four steps of Reeves (2006) were exe- likelihood of students' video viewing, some researchers (e.g., Guo,
cuted in our research project (Figure 1). The ethical review board at Kim, & Rubin, 2014) have examined the possible effects of various
[anonymised] University has reviewed and approved the studies. The video styles on student engagement. In their study of massive open
main learning management system that we used was Moodle (https:// online courses (MOOCs), Guo et al. (2014) found that the natural
moodle.com). motion of human handwriting in Khan-style videos were more engag-
ing to MOOC students than static computer-rendered fonts in
PowerPoint narration or classroom lecture videos. However, other
2.1 | Step 1: Analysis of practical problems researchers discovered that some high school mathematics students
might prefer classroom lecture videos over Khan-style videos because
Reeves (2006) suggested that a problem can be analysed via multiple they were more familiar with a classroom setting (Ilioudi, Giannakos,
means, such as information sharing between researchers and practi- & Chorianopoulos, 2013). There is therefore an important need for us
tioners, studying the literature, and conducting exploratory studies. In to examine which video style (e.g., classroom lecture video,
this research project, the key starters of the practitioner (the first PowerPoint narration, or Khan-style video) may positively affect sec-
author) were the training activities and official documents (see Educa- ondary school students' mathematics learning.
tion Bureau, 2014 for a review) provided by the local education Besides pre-class videos, some teachers (e.g., Enfield, 2013)
bureau. With support from a faculty member of a local university (the assigned their students pre-class online exercises because they
second author), two empirical studies (Studies 1 and 2) and a literature believed that this kind of exercise is a strong motivator for students'
review (Study 3) were conducted during the first step of the design- class preparation. In their experiment of mathematics video lecturing,
based research project. First, Study 1 consisted of two sub-studies to Szpunar, Jing, and Schacter (2014) examined the effect of offering
test the use of flipped learning with underperforming students (Study short-answer quizzes on video content. They found that the students
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
146 LO AND HEW

with follow-up quizzes significantly outperformed those without the (e.g., a review of pre-class materials, a mini-lecture on new materials,
quizzes. However, requiring students to type mathematics answers in and problem-solving activities) informed by Studies 1A and 1B. Rele-
online quizzes may be difficult, especially for those involving symbols or vant possible strategies to support student engagement derived from
special characters. Therefore, besides short-answer quizzes, we evalu- Study 3 were also used to inform our flipped lesson prototype. As a
ated the feasibility of asking students to copy worked answers from side note, not all flipped learning materials were produced at this
online text-based materials and then annotating them. Study 2 was thus stage, thus allowing room for further refinement in Step 3.
conducted to identify a desirable video style (Study 2A) and pre-class
study strategy (Study 2B) for flipped learning in secondary school math-
ematics education. Table 2 presents more detailed information about 2.3 | Step 3: Evaluation of the solutions in practice
the video styles and study strategies of Studies 2A and 2B.
Upon the completion of Studies 1 and 2, we conducted a compre- After the development in Step 2, the lesson design and instructional
hensive systematic literature review with a specific focus on student materials were put into practice in the same school as Studies 1 and 2.
engagement in flipped learning (Study 3). The review followed the Two empirical studies (Studies 4 and 5) were conducted in the third
method of PRISMA (the preferred reporting of items for systematic step of the design-based research project. First, Study 4 used an
reviews and meta-analyses; Moher, Liberati, Tetzlaff, Altman, & the action research approach with an objective of gaining experience with
PRISMA Group, 2009). This method is frequently used in systematic the practice of flipped learning in regular classes and preparing for a
reviews to help researchers transparently report the major steps of long-term implementation (i.e., 1-year implementation in Study 5).
identifying articles for reviews. The major steps include article identifi- The two 2-week iterations (Iterations 1 and 2) both involved two clas-
cation, article screening, article eligibility, and article inclusion. ses and their mathematics teachers. The course topic of the two itera-
Researchers can use the PRISMA flow diagram to map out in detail tions was the same. Both iterations also had similar pre-class and in-
the number of articles identified, included and excluded, and the rea- class instructional materials (Table 4). However, some adjustments
sons for exclusions. The quantitative results (e.g., student survey) and were made to the materials in Iteration 2 based on the experiences
qualitative findings (e.g., teacher reflections and student interviews) of gained in Iteration 1. Student engagement was evaluated upon com-
the reviewed studies were synthesized using the engagement frame- pletion of each iteration. Second, Study 5 was the final iteration in this
work of Fredricks et al. (2004). Based on our research findings (Stud- research project. A quasi-experimental research design was adopted,
ies 1 and 2) and the relevant literature (Study 3), the first step of our aiming to compare students' mathematics achievement and engage-
research project was concluded by proposing a set of initial design ment in flipped learning and in the traditional lecture format. We
guidelines of flipped learning practice. Table 3 presents more detailed increased the duration of the study as the intervention had become
information about the methods of Studies 1–3. mature (McKenney & Reeves, 2012). The study was conducted in a
year-long mathematics enrichment course for Grade 9 students. With
a longer duration, Study 5 could minimize the novelty effect on stu-
2.2 | Step 2: Development of solutions dent engagement of being introduced into a new technology-
enhanced instructional approach (Clark, 1983). The validity of the
Based on the outcomes of Step 1 (analysis of practical problems), a study was thus improved (Maxwell, 2005). Table 4 presents more
prototype was developed for lesson design and relevant instructional detailed information about the courses of Studies 4 and 5.
materials in the second step of our research project. The prototype Student engagement was evaluated with a post-intervention sur-
encompassed several flipped learning activities, including video lec- vey and semi-structured interviews in Study 4 (i.e., Iteration 1 vs. Iter-
tures of the video style identified in Study 2A, the pre-class exercises ation 2) and Study 5 (i.e., flipped class vs. traditional class). According
of the study strategy identified in Study 2B, and the in-class activities to Fredricks and McColskey (2012), these are the most commonly

TABLE 2 Video styles and study strategies of Studies 2A and 2B

Study Course topic Duration Approaches to be explored

2A Rational and irrational numbers 1 lesson (four 6-min videos) Video styles
• Classroom lecture video
• PowerPoint narration with teacher's talking head
• Khan-style video with teacher's talking head
2B Deductive geometry 4 lessons (three 6-min videos per lesson) Study strategies
• Short-answer quizzes: Students completed two to
three online short-answer questions with
computerized feedback
• Copying of worked answers: Students copied the
full solution of questions from online text-based
materials for comprehension and memory encoding
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LO AND HEW 147

TABLE 3 Goals and design of the three studies in the first step of the design-based procedure

Data collection (analysing

Study Research goal Research design Participants (gender) techniques) Report and more details
1A To test the use of Exploratory study 13 Grade 12 students Pre-test and post-test (paired- Lo and Hew (2017b)
flipped learning for (F = 12; M = 1) samples t test), student
underperforming interviews (n = 5) and teacher
students reflections (thematic analysis)
1B To test the use of Exploratory study 24 Grade 12 students Pre-test and post-test (paired-
flipped learning for (F = 10; M = 14) samples t test), survey
high-ability (descriptive statistics), student
students interviews (n = 7) and teacher
reflections (thematic analysis)
2A To examine the Randomised experiment CLV: 43 Grade 8 students Post-test (one-way independent Hew and Lo (2018)
effects of various (F = 24; M = 19) ANOVA) and teacher
video styles PPT: 43 Grade 8 students reflections (thematic analysis)
(F = 25; M = 18)
KSV: 43 Grade 8 students
(F = 25; M = 18)
2B To examine the Randomised experiment SAQ: 44 Grade 8 students Post-test (independent t test)
effects of various (F = 25; M = 19) and teacher reflections
pre-class study CWA: 43 Grade 8 students (thematic analysis)
strategies (F = 25; M = 18)
3 To identify strategies Literature review NA 27 primary studies (content Lo and Hew (2017a)
to support student analysis)
engagement in
flipped learning

Abbreviations: CLV, classroom lecture video; CWA, copying of worked answers; KSV, Khan-style video with teacher's talking head; PPT, PowerPoint narra-
tion with teacher's talking head; SAQ, short-answer quizzes.

TABLE 4 Course information of Studies 4 and 5

Study Course topic Duration Pre-class learning activities In-class learning activities
4 Pythagorean theorem 2 weeks (three to Focusing on basic concepts (e.g., Focusing on difficult materials (e.g.,
four 35-min hypotenuse) and formulas different proofs of the Pythagorean
lessons per week) with work examples theorem) and problem-solving
• One to two instructional • Warm-up exercises and review of pre-
videos (within 6 min) class materials (10 min)
• Two online multiple-choice • Mini-lecture on advanced materials
questions for each video (5–10 min)
• Solving simple and advanced problems
in groups (15–20 min)
5 Algebra, geometry, 1 year (one 35-min Focusing on basic concepts and Focusing on difficult materials (e.g.,
and statistics lesson per week) formulas (e.g., mean and expected value) and problem-solving
range) with work examples • Warm-up exercises and review of pre-
• Two instructional videos class materials (10 min)
(within 6 min) • Mini-lecture on advanced materials
• Two online multiple-choice (5–10 min)
questions for each video • Solving simple and advanced problems
in groups (15–20 min)

used methods to examine student engagement. They allow student • Items 1–5: Behavioural engagement (e.g., I tried hard to do well in
participants to express their thoughts of being engaged or disengaged my study; Skinner et al., 2008, p. 781)
and “are particularly useful for assessing emotional and cognitive • Items 6–10: Emotional engagement (e.g., The class was fun; Skin-
engagement which are not directly observable” (p. 765). ner et al., 2008, p. 781)
The post-intervention survey (see Appendix A) was based on the • Items 11–14: Cognitive engagement (e.g., I was so involved that I
literature. All three dimensions of student engagement were forgot everything around me; Rotgans & Schmidt, 2011, p. 470)
considered: • Item 15: Free-response question soliciting additional comments
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
148 LO AND HEW

TABLE 5 Goals and design of the two studies in the third step of the design-based procedure

Report and more

Study Research goal Research design Participants (gender) Data collection (analysing techniques) details
4 To improve flipped Dual-cycle action research 3 mathematics teachers Pre-test and post-test (Mann–Whitney Lo (2017)
learning practice in (F = 1; M = 2) U test), survey (Mann–Whitney U
authentic Iteration 1:58 Grade 8 test), student interviews (n = 4 + 4)
classrooms students (F = 35; M = 23) and teacher reflections (thematic
Iteration 2:72 Grade 8 analysis)
students (F = 40; M = 32)
5 To evaluate the Quasi-experiment FC: 28 Grade 9 students Pre-test and post-test (Mann–Whitney Lo and Hew (2020)
effects of flipped (F = 15, M = 13) U test), survey (Mann–Whitney U
learning on student TC: 27 Grade 9 students test), optional assignment (descriptive
achievement and (F = 11, M = 16) statistics), student interviews (n = 6 in
engagement FC) and teacher reflections (thematic
analysis)

Abbreviations: FC, flipped class; TC, traditional class.

A 5-point Likert scale was used (ranging from 5 for “strongly reviews of the data had been done to ensure our understanding of
agree” to 1 for “strongly disagree”). The data were quantitatively each category. Some of the data were translated into English for
analysed using SPSS for Windows (Version 23.0). According to our reporting purposes. Table 5 presents more detailed information
preliminary exploration of data in Study 4, these sets of survey items about the methods of Studies 4 and 5.
had relatively high internal consistency (Cronbach's alpha > .8). For
each student, an overall engagement rating was obtained for each
dimension by averaging their ratings of all items within the dimension. 2.4 | Step 4: Reflection to produce design
To determine an appropriate statistical test for survey data analysis, principles
the Kolmogorov–Smirnov test was first used to check the normality
of the data (López, Valenzuela, Nussbaum, & Tsai, 2015). A deviation The Design-Based Research Collective (2003) emphasized that
from normality was found in the survey data of Studies 4 and 5, which “design-based research goes beyond merely designing and testing par-
indicated the need to use the Mann–Whitney U test (a non-paramet- ticular interventions” (p. 6). Using the design-based research of
ric test) for analysis. Vanderhoven et al. (2016) as an example by reflecting on their entire
In addition to quantitative data, our analysis of student engage- research procedure, the researchers proposed a set of design princi-
ment involved qualitative data, including interview data and free ples for the development of educational materials about risks on social
responses form the student survey. A topic guide (see Appendix B) network sites. Similarly, we reflected on our findings and the experi-
for student interviews about their engagement was designed based ence of Steps 1–3. Our research project was then concluded by pro-
on Fredricks et al. (2004). Their behavioural, emotional, and cogni- ducing theoretical and empirical design principles to support student
tive engagement topics were discussed during the interviews. The engagement in flipped learning.
student interview data and written comments were transcribed in
Chinese. The data were then analysed with a series of qualitative
data analysis procedures proposed by Creswell (2012). The first step 3 | RESULTS AND DISCUSSION
was an initial reading of all qualitative data to gain an overall idea of
the data. The coding was then started by picking the shortest inter- 3.1 | Step 1: Analysis of practical problems
view transcript. Codes were assigned to pieces of data. We were
open to any possibilities and there were no prefigured categories Table 6 summarizes the major findings of the three studies conducted
prior to our analysis. After completing the coding of the first tran- in the first step of our research project. Teacher reflections are
script, all of the codes assigned were reviewed and grouped by simi- described in the following paragraphs.
larity. Redundant codes were reduced to obtain a preliminary list of
codes. The list was then used to analyse the rest of the data. This
allowed for the identification of any emerging codes that enriched 3.1.1 | Exploratory studies to test the use of
the list of codes in addition to other “specific quotes from partici- flipped learning
pants that support the codes” (Creswell, 2012, p. 245). Finally, simi-
lar codes were organized into categories. To enhance the We conducted two exploratory studies to test the use of flipped
consistency of our classification, we identified several exemplary learning with underperforming students (Study 1A) and high-ability
quotes that clearly illustrated each category constructed. Multiple students (Study 1B). For Study 1A, the most important lesson we
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LO AND HEW 149

TABLE 6 Major findings of the three studies in the first step of the design-based procedure

Report and more

Study Research design Data collection (analysing techniques) Major findings details
1A Exploratory study • Pre-test and post-test (paired- • Significant learning gain (p < .05) Lo and
samples t test) Hew (2017b)
• Student interviews (n = 5; thematic • Major themes: (1) students' ability to learn at
analysis) their own pace (n = 3); and (2) students' inability
to understand video content (n = 3)
1B Exploratory study • Pre-test and post-test (paired- • Significant learning gain (p < .05)
samples t test)
• Survey (descriptive statistics) • More than 85% students agreed or strongly
agreed that (1) flipped learning was more
engaging than traditional lecturing, (2) they liked
to self-pace themselves through the course, and
(3) flipped learning improved their learning of
mathematics
• Student interviews (n = 7; thematic • Major themes: (1) students' ability to learn at
analysis) their own pace (n = 3); (2) increased peer
interactions (n = 6); and (3) students' demand of
more advanced problem-solving
exercises (n = 3)
2A Randomised experiment • Post-test (one-way independent • Test items on recall: CLV = PPT = KSV Hew and Lo (2018)
ANOVA) • Test items on application: CLV = PPT = KSV
2B Randomised experiment • Post-test (independent t test) • Test items on recall: SAQ = CWA
• Test items on application: SAQ > CWA (p < .05)
3 Literature review • 27 primary studies (content analysis) • Behavioural engagement: FC and TC appeared Lo and
to be similar in terms of attendance, effort, task Hew (2017a)
accomplishment, and attention
• Emotional engagement: There was broad but
weak evidence that FC improved student
interest, feeling, and satisfaction
• Cognitive engagement: Only four studies
examined this construct and provided evidence
that FC increased students' preference for
challenges and desire to go beyond the
requirement

Abbreviations: CLV, classroom lecture video; CWA, copying of worked answers; FC, flipped class; KSV, Khan-style video with teacher's talking head; PPT,
PowerPoint narration with teacher's talking head; SAQ, short-answer quizzes; TC, traditional class.

learned was about our selection of pre-class learning materials. A few solutions (i.e., Step 2 of our research project) for subsequent
students expressed an inability to understand the difficult materials (e. interventions.
g., finding the perpendicular bisector of a line segment) presented in
our instructional videos. As a result, the teacher was obliged to re-
teach those materials during class. Consistent with the literature, stu- 3.1.2 | Randomised experiments to inform the
dents' inability to understand video content presents a major chal- design of pre-class instructional strategies
lenge to flipped mathematics learning and can lead to students'
disengagement from their class preparation (Lo et al., 2017). In the We conducted two randomised experiments to examine the effects of
flipped calculus course of Palmer (2015), for example, one student various video styles (classroom lecture video, PowerPoint narration
admitted that “When I don't understand what you're saying, I turn the with a teacher's talking head, and Khan-style video with a talking
video off” (p. 890). As for Study 1B, we learned that the high-ability head) (Study 2A) and pre-class study strategies (short-answer quiz vs.
students demanded more challenging tasks. During class, some com- copying of worked answers) (Study 2B). From the experience of Study
pleted the learning tasks efficiently with peer collaboration and asked 2A, we found that the production of Khan-style (write-while-speak)
for more advanced application problems. The experience of these videos was relatively less demanding than some other commonly used
exploratory studies increased our awareness of lesson planning to video styles (e.g., PowerPoint narration and classroom lecture video).
cater for learner diversity. This influenced our development of Most importantly, our results suggested that various video styles had
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
150 LO AND HEW

similar effects on students' mathematics achievement. We therefore Tran, 2016), task accomplishment (e.g., Gundlach, Richards, Nelson, &
produced Khan-style videos for students' class preparation in the sub- Levesque-Bristol, 2015), or study time (e.g., He, Holton, Farkas, &
sequent interventions. For the pre-class study strategies, the results Warschauer, 2016). For students' emotional engagement, some evi-
of Study 2B suggested that the use of short-answer quizzes signifi- dence supported the positive effects of flipped learning on course
cantly improved students' application of pre-class learning materials interest (e.g., Blair, Maharaj, & Primus, 2016) and satisfaction (e.g.,
compared with interpolate copying of worked answers. By and large, Hung, 2015). However, several studies reported non-significant (e.g.,
the results echoed those of some other studies (e.g., Szpunar Whillier & Lystad, 2015) or even negative results (e.g., Whitman Cobb,
et al., 2014) in mathematics education. In practical terms, however, 2016). Whitman Cobb (2016) noted that some students dislike an
we learned that the production of short-answer quizzes required con- excessive pre-class workload. Whillier and Lystad (2015) further cau-
siderable effort from the teacher in setting the computerized checking tioned that the increased workload may have led to students' disen-
of students' free-responses. For example, if the answer is “ΔABC,” the gagement with flipped learning. Finally, we found that no studies
teacher must first work out and then manually type in all possible evaluated all three dimensions of student engagement in flipped
answers (i.e., “ΔABC,” “ΔACB,” “ΔBAC,” “ΔBCA,” “ΔCAB,” and “ΔCBA”) mathematics learning. Studies 4 and 5 could therefore address this
in Moodle. Under the constraint of limited resources, it was not feasi- research gap.
ble to apply this study strategy on a regular basis. We therefore used To conclude Step 1, we proposed a set of initial design guidelines
multiple-choice questions as our pre-class study strategy in the subse- to inform (a) course planning, (b) pre-class learning, and (c) in-class
quent interventions (i.e., Step 3 of our research project). learning of flipped mathematics courses. With regard to course plan-
ning, teachers should identify the learning items to be introduced via
video lecture based on their understanding of the students' ability
3.1.3 | Literature review and proposal of initial (Cheng et al., 2019). More advanced and/or complicated materials
design guidelines should be presented in a face-to-face setting in which the teacher
could immediately answer students' inquiries and resolve their learn-
Our literature review included multiple empirical studies found in the ing problems. This strategy could minimize students' disengagement
database search. Twenty-seven studies compared some aspects of during video lecturing (Study 1A; Lo et al., 2017; Palmer, 2015).
student engagement between flipped learning and traditional lectur- With regard to pre-class learning, we recommended the use of
ing. We found that students' behavioural engagement appeared simi- Khan-style videos with a teacher's talking head in secondary school
lar in these two instructional environments. The studies generally mathematics flipped courses (Study 2A). The write-while-speak nature
found no significant differences between flipped and traditional clas- of Khan-style videos could stimulate blackboard drawing that “makes
ses in terms of students' preparation rate (e.g., Balaban, Gilleskie, & visible the process of mathematical reasoning” (Greiffenhagen, 2014,

F I G U R E 2 Basic structure of
a flipped lesson to support
student engagement and
psychological needs [Colour
figure can be viewed at
wileyonlinelibrary.com]
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LO AND HEW 151

TABLE 7 Major findings of the two studies in the third step of the design-based procedure

Report and more

Study Research design Data collection (analysing techniques) Major findings details
4 Dual-cycle action • Pre-test and post-test (Mann– • Significant learning gain (p < .05) Lo (2017)
research Whitney U test)
• Survey (Mann–Whitney U test) • Behavioural engagement: Cycle 1 < Cycle Table 8
2 (p < .05)
• Emotional engagement: Cycle 1 = Cycle 2
• Cognitive engagement: Cycle 1 < Cycle 2 (p < .05)
• Student interviews (n = 4 + 4; • Main themes to inform study 5: (1) students'
thematic analysis) unpreparedness for pre-class learning tasks (n = 4);
and (2) students' demand of more advanced online
exercises (n = 3)
5 Quasi-experiment • Pre-test and post-test (Mann– • FC > TC (p < .05) Lo and Hew (2020)
Whitney U test)
• Survey (Mann–Whitney U test) • Behavioural engagement: FC = TC Table 9
• Emotional engagement: FC = TC
• Cognitive engagement: FC = TC
• Optional assignment (descriptive • Submission rate, number of questions attempted,
statistics) and quality: FC > TC
• Student interviews (nFC = 6; thematic • Main themes: (1) increased peer interactions (n = 6);
analysis) and (2) increased sense of competence in problem-
solving (n = 3)

Abbreviations: FC, flipped class; TC, traditional class.

p. 521). Talking-head videos were also more engaging because they (online) and in-class (a 35-min face-to-face lesson) learning compo-
could create an intimate and personal feel (Guo et al., 2014). After nents. For each learning activity, we describe how student engage-
watching the pre-class videos, multiple-choice questions on the con- ment (i.e., behavioural, emotional, and cognitive engagement) and the
tent were assigned to motivate students' class preparation and students' psychological needs (i.e., autonomy, relatedness, and compe-
enhance student learning (Study 2B). tence) were supported.
With regard to in-class learning, we suggested the use of small-
group activities with various levels of learning materials. Students
could discuss mathematics problems with their peers and support 3.3 | Step 3: Evaluation research of the solutions in
each other (Study 1B). One student of Larsen (2015) expressed that practice
“the other ideas and approaches that students had towards problems
allowed me to see different ways of understanding the questions and In Step 3, two studies (Studies 4 and 5) were conducted to evaluate
different techniques to use when finding an answer” (p. 57). Their dis- the solutions in practice. Table 7 summarizes the major findings of the
cussion process could also trigger further cognitive challenges that two studies conducted in the third step of our research project.
could deepen student learning (Topping & Ehly, 1998). Furthermore, Teacher reflections are described in the following paragraphs. Figure 3
teachers must prepare additional materials to cater to learner diver- summarizes the major refinements that we made across the studies.
sity. In the flipped transition-to-proof course of Talbert (2015), for
example, some students struggled with basic terminology, but others
completed their problem-solving exercises within a minute. To satisfy 3.3.1 | Action research to improve flipped learning
different students' learning needs, teachers should prepare extra basic practice
exercises for underperforming students (Study 1A) and more
advanced problems for high-ability students (Study 1B). Study 4 consisted of two iterations. Based on the findings of the
teacher reflections and student interviews in Iteration 1, we made a
few improvements to our flipped learning practice. First, we resolved
3.2 | Step 2: Development of solutions some technical problems encountered in Iteration 1 (e.g., students'
access to online materials). Second, our teacher participants observed
In the second step of our research project, the basic structure of our that a number of students did not engage in peer discussions during
flipped lessons was developed as informed by the aforementioned the in-class problem-solving activities but preferred to seek help from
guidelines and the two-component model of Bishop and Ver- their teacher or struggle alone on their own. In this regard, one
leger (2013). Figure 2 shows that each lesson consists of pre-class teacher participant suggested incorporating some game-like activities
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
152 LO AND HEW

F I G U R E 3 Summary of the
refinements across Studies 4 and 5
[Colour figure can be viewed at
wileyonlinelibrary.com]

that required student collaboration. We therefore used group compe- Toward the end of the course, we used a student survey to com-
tition activities to encourage peer interactions in Iteration 2. For pare student engagement in flipped learning and in the traditional lec-
example, each group member had to handle a different part of a prob- ture format. Table 9 shows that the students in the flipped and
lem and then discuss the overall solution. In Table 8, the results of the traditional classes rated the same statistically in the student survey
student survey indicate that the students in Iteration 2 (n = 70, regarding behavioural engagement (z = .870, p = .384), emotional
Mdn = 4.00) had significantly better overall behavioural engagement engagement (z = 1.506, p = .132), and cognitive engagement
than those in Iteration 1 (n = 58, Mdn = 3.80), z = 2.962, p = .003. (z = 1.028, p = .304). Although such non-significant results resonated
Their cognitive engagement in Iteration 2 (n = 70, Mdn = 3.75) was with the meta-analytic studies of Spanjers et al. (2015) and van Alten
also significantly better than that in Iteration 1 (n = 58, Mdn = 3.50), et al. (2019), we acknowledge that there is room for improvement
z = 2.068, p = .039. These results triangulated the teachers' observa- regarding our small-group learning activities. As informed by Study 4,
tion that “the students (in Iteration 2) paid great attention to the more group competition activities should have been used in Study 5
group activity” (Teacher A) and “were excited to share their ideas of because this kind of activity can promote student engagement. How-
problem-solving” (Teacher B). These findings provided evidence to ever, designing a group competition activity for every mathematics
support our refinements of the flipped learning practice in Study 4. lesson requires considerable effort from the teacher. Therefore, we
However, our teacher participants reported that not all students could not regularly offer group competition activities in the flipped
completed the pre-class learning tasks despite the use of online exer- class to support student engagement.
cises. Several students also admitted that they did not always prepare Nevertheless, it is worth noting that the flipped class performed
for the class. This challenge resonates with the research synthesis of significantly better than the traditional class in their post-test (Table 7)—
Akçayır and Akçayır (2018). Therefore, we further used game design a more objective instrument of students' cognitive engagement com-
elements (e.g., digital badges, points, and a leader board) as some pared with other self-reported measures (e.g., survey). To explain such
motivators of class preparation in Study 5. Following Hew et al. (2016), differences between the two classes, the qualitative findings of the stu-
the students in our course platform (i.e., Moodle) could earn badges dent feedback suggested the importance of peer interactions during
by completing their pre-class learning tasks on time, and they would problem-solving. In the words of one student, for example, “Through
receive digital points if they could answer the online questions cor- reminding each other, we can effectively identify the key to solve the
rectly. The digital points can also be regarded as computerized feed- problems. These also make me more active to learn the mathematics,
back for the teachers to confirm the students' accomplishments and willing to think and comprehend the complicated questions” (Student
support their sense of competence (Niemiec & Ryan, 2009; Sailer, A). As Dove and Dove (2015) observed, the students in the flipped class
Hense, Mayr, & Mandl, 2017). Like a competition, the five students could experience success in mathematics because they had more time
with the highest points were displayed on the leader board in Moodle. for practice and collaboration in class. This promoted their willingness
to solve difficult problems. To summarize, the result of the post-test
provides evidence that the students in the flipped class showed greater
3.3.2 | Quasi-experiment to examine the effects of cognitive engagement than those in the traditional class.
flipped learning and those of the traditional lecture
format
3.4 | Step 4: Reflection to produce design
In Study 5, we delivered our year-long mathematics enrichment principles
course in both flipped (n = 28) and traditional (n = 27) formats. In the
flipped class, we observed an improvement in the students' class prep- In this design-based research project, we drew upon the engagement
aration. The students generally prepared for the class by watching the framework of Fredricks et al. (2004) and self-determination theory
instructional videos. A few students occasionally failed to complete (Ryan & Deci, 2000). Through a series of studies (Studies 1–5), we
their pre-class learning tasks for personal reasons. They would then gradually improved our flipped learning practice. Based on the theo-
seek help from their classmates to catch up on the pre-class materials retical work and our empirical findings, our research project concludes
when doing the warm-up exercises at the start of a lesson. with the proposal of five design principles to support student
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LO AND HEW 153

TABLE 8 Students' overall engagement ratings of Study 4 by iteration

Iteration 1 (n = 58) Iteration 2 (n = 70)a

Engagement Mean (SD) Median Mean (SD) Median z p

Behavioural 3.80 (.56) 3.80 4.11 (.63) 4.00 2.962** .003
Emotional 3.66 (.79) 3.80 3.92 (.66) 4.00 1.727 .084
Cognitive 3.50 (.74) 3.50 3.76 (.68) 3.75 2.068* .039
a
Two students in Iteration 2 did not complete the survey.
*p < .05.
**p < .01.

T A B L E 9 Students' overall
Flipped class (n = 28) Traditional class (n = 27)
engagement ratings of Study 5 by class
Engagement Mean (SD) Median Mean (SD) Median z p
Behavioural 4.02 (.66) 4.00 4.15 (.45) 4.20 .870 .384
Emotional 3.81 (.81) 3.90 4.16 (.48) 4.20 1.506 .132
Cognitive 3.58 (.70) 3.50 3.75 (.50) 3.75 1.028 .304

engagement of flipped learning. These principles inform the design et al., 2019). Pre-class online exercises are an effective means to moti-
and implementation of the core instructional activities of a flipped vate and monitor students' class preparation (Enfield, 2013; Lo
mathematics lesson (see Figure 2), including pre-class video lecture et al., 2017). Principle 2 includes two elements to inform the design of
(Principle 1), pre-class online exercises (Principle 2), in-class warm-up the exercises. Ideally, teachers can produce the exercises in the form
exercises and a brief review (Principle 3), in-class mini-lecture (Princi- of short-answer questions. In Studies 4 and 5, we used multiple-
ple 4), and in-class small-group problem-solving (Principle 5). choice questions instead because the production of such questions
required relatively little effort. In addition to their effectiveness, both
question types can serve the purpose of supporting students' behav-
3.4.1 | Principle 1: Use Khan-style videos with ioural engagement (i.e., students' completion of learning tasks). Sec-
teacher's talking head to deliver basic materials before ond, game design elements can be embedded within the exercises to
class further encourage class preparation. For example, students can earn
digital points by answering the questions correctly (Hew et al., 2016).
In Principle 1, three elements are used to inform the design of a pre- According to Sailer et al. (2017), computerized feedback can evoke
class video lecture. First, the use of Khan-style (write-while-speak) students' feelings of competence because it directly communicates
videos is recommended because this kind of video can illustrate the their mastery of learning.
process of mathematics reasoning like blackboard drawings
(Greiffenhagen, 2014). Second, we suggest that basic or introductory
materials be presented in video lectures to avoid overwhelming and 3.4.3 | Principle 3: Allocate a portion of class time
upsetting students with learning difficulties during their online indepen- to follow-up students' pre-class learning
dent study (Cheng et al., 2019; Lo et al., 2017). From the perspective of
self-determination theory (Ryan & Deci, 2000), if students can follow The use of pre-class online exercises can provide information on stu-
the teacher's mathematics instructions and master the pre-class learning dents' mastery of pre-class materials (Lo et al., 2017; Schroeder &
materials, their sense of competence increases. Third, inserting the Dorn, 2016). Based on student performance, teachers are able to
teacher's talking head into the pre-class videos can induce students' inti- adjust their teaching plans. In Principle 3, we suggest the use of two
mate and personal feelings (Guo et al., 2014), thus increasing their emo- in-class activities to follow up students' pre-class learning. First,
tional engagement in the lesson (Fredricks et al., 2004). teachers can use basic warm-up exercises to help students recall their
pre-class learning. If they can complete the tasks on their own, their
sense of competence increases (Niemiec & Ryan, 2009); otherwise,
3.4.2 | Principle 2: Assign gamified online teachers can offer a brief review of the pre-class materials during
exercises on pre-class materials which they can resolve students' learning problems and provide them
with feedback at the point of need (Lo et al., 2017). From the perspec-
It is critical for teachers to use proper methods to encourage students' tive of self-determination theory, just-in-time teaching and timely
adequate preparation (Akçayır & Akçayır, 2018) and to keep students feedback can support students' sense of competence and thus their
on the right track of learning before their face-to-face lessons (Cheng cognitive engagement (Fredricks et al., 2004; Niemiec & Ryan, 2009).
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
154 LO AND HEW

3.4.4 | Principle 4: Present difficult materials in 4 | C O N C LU S I O N S A N D L I M I T A T I O N S

face-to-face lessons
This design-based research project explored ways to support student
The introduction of course materials before class via instructional engagement in flipped mathematics learning. Through a series of stud-
videos can free up class time from direct lecturing. However, Cheng ies, a set of design principles was developed to inform the design and
et al. (2019) and Lo et al. (2017) cautioned that not all materials are implementation of the core instructional activities of a flipped mathe-
suitable to help students learn independently, especially for difficult matics lesson. For example, we were faced with the reality that not all
mathematics concepts and complicated problems. When introducing mathematics concepts and problems are suitable for students to study
these advanced materials, it may be more convenient for teachers to independently online. We thus suggest that difficult materials be pres-
answer students' inquiries and provide timely feedback in a face-to- ented during face-to-face lessons, when the teacher can answer the
face environment. With their teacher's assistance, students' sense of students' inquiries and provide timely feedback (Principle 4). The prin-
competence can be supported (Niemiec & Ryan, 2009). Therefore, in ciples can offer mathematics teachers insights into practicing their
each component (i.e., pre-class and in-class) of a flipped lesson, math- flipped mathematics courses. Moreover, this research project
ematics teachers “have to be judicious with appropriate learning con- advances our understanding of how two theoretical perspectives (i.e.,
tent and requirements to successfully implement the flipped the three-dimensional framework of student engagement and self-
classroom” (Cheng et al., 2019, p. 816) based on their understanding determination theory) can be applied in the context of flipped
of the students' abilities. learning.
However, several limitations must be acknowledged with some
recommendations for future research. First, our research project
3.4.5 | Principle 5: Facilitate peer interactions was conducted in a single secondary school in Hong Kong, so cau-
using small-group problem-solving exercises tion should be exercised when applying our research outcomes in
other education contexts. Further research is thus required to test
Learning in groups can create an environment in which students expe- the use of the design principles in different contexts (e.g., primary
rience social relatedness (Abeysekera & Dawson, 2015; Hanze & Ber- school and college-level mathematic teaching). Second, the design
ger, 2007) and cultivate a sense of belonging inside the classroom principles that we developed may be specific to mathematics educa-
(Watkins, 2005). In Principle 5, two arrangements of small-group tion. In future studies, researchers can adopt our research outcomes
learning activities can be made to further support student engage- as a basis to formulate their own design principles in their subject
ment. First, teachers can allow their students to choose some of their discipline. Third, we did not focus on specific students' characteris-
learning tasks according to their learning needs. Based on the experi- tics (e.g., gender and anxiety about mathematics). We suggest fur-
ence of Clark (2015), students can form their groups (a) to work on ther research be conducted to identify instructional design
advanced problems, (b) to work on basic exercises first and then work principles for some particular groups of students. Finally, a lack of
on some advanced problems, or (c) to review the course materials first institutional support was one limitation to our research project. For
and then work on basic exercises and a few core advanced problems. example, group competition activities (which were found in Study 4
In this way, not only their sense of autonomy (i.e., having choice of to be effective in promoting student engagement) should have been
their learning tasks) but also their competence (i.e., their ability to used more frequently in Study 5. However, the time constraints on
complete the learning tasks) can be supported (Niemiec & Ryan, 2009; the practitioner somewhat hindered the implementation of our
Ryan & Deci, 2000). refined solution in our final year-long intervention. In future prac-
Second, teachers can design the class activities in the form of tice, more resources (e.g., additional manpower) should be provided
group competition, as in Study 4. A discussion of problem-solving to support teachers' start-up efforts to transform their courses into
strategies can result in greater cognitive engagement of students in a flipped format.
peer interactions. There are quite a few cooperative learning strate-
gies that can be applied in flipped lessons, such as Student Teams- AC KNOWLEDG EME NT S
Achievement divisions (STAD), Teams-Games-Tournament, and Jig- This research did not receive any specific grant from funding agencies
saw II, among others (Chang, Tsai, Li, Tsai, & Yu, 2018). For example, in the public, commercial, or not-for-profit sectors.
Kittur (2016) used STAD activities to enhance student learning. Stu-
dents worked in a team of four with different ability levels. The group- CONFLIC T OF INT ER E ST
ing can be achieved by referring to student performance of their pre- The author declares that no conflict of interest.
class online exercises. Kittur (2016) argued that the members in het-
erogeneous teams could help each other master learning content. DATA AVAILABILITY STAT EMEN T
Most importantly, we can avoid the high-ability students clustering The data that support the findings of this study are available on
together to dominate the group competition. Teachers can thus request from the corresponding author. The data are not publicly
ensure the fairness of activities (Chang et al., 2018). available due to privacy or ethical restrictions.
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LO AND HEW 155

ORCID Finn, J. D., & Rock, D. A. (1997). Academic success among students at risk
Chung Kwan Lo https://orcid.org/0000-0003-2305-9227 for school failure. Journal of Applied Psychology, 82, 221–234. https://
doi.org/10.1037/0021-9010.82.2.221
Khe Foon Hew https://orcid.org/0000-0003-4149-533X
Fredricks, J. A., & McColskey, W. (2012). The measurement of student
engagement: A comparative analysis of various methods and student
RE FE R ENC E S self-report instruments. In S. L. Christenson, A. L. Reschly, & C. Wylie
Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the (Eds.), Handbook of research on student engagement (pp. 763–782).
flipped classroom: Definition, rationale and a call for research. Higher New York: Springer. https://doi.org/10.1007/978-1-4614-2018-7_37
Education Research and Development, 34(1), 1–14. https://doi.org/10. Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engage-
1080/07294360.2014.934336 ment: Potential of the concept, state of the evidence. Review of Educa-
Akçayır, G., & Akçayır, M. (2018). The flipped classroom: A review of its tional Research, 74(1), 59–109. https://doi.org/10.3102/
advantages and challenges. Computers & Education, 126, 334–345. 00346543074001059
https://doi.org/10.1016/j.compedu.2018.07.021 Furrer, C., & Skinner, C. (2003). Sense of relatedness as a factor in children's
Balaban, R. A., Gilleskie, D. B., & Tran, U. (2016). A quantitative evaluation academic engagement and performance. Journal of Educational Psychol-
of the flipped classroom in a large lecture principles of economics ogy, 95, 148–162. https://doi.org/10.1037/0022-0663.95.1.148
course. The Journal of Economic Education, 47(4), 269–287. https://doi. Giannakos, M. N., Krogstie, J., & Chrisochoides, N. (2014). Reviewing the
org/10.1080/00220485.2016.1213679 flipped classroom research: Reflections for computer science education.
Bishop, J. L., & Verleger, M. A. (2013). The flipped classroom: A survey of the Proceedings of the Computer Science Education Research Conference, ACM.
research. Paper presented the 120th ASEE National Conference and Greiffenhagen, C. (2014). The materiality of mathematics: Presenting
Exposition, Atlanta, GA (Paper ID 6219). Washington, DC: American mathematics at the blackboard. The British Journal of Sociology, 65(3),
Society for Engineering Education. 502–528. https://doi.org/10.1111/1468-4446.12037
Blair, E., Maharaj, C., & Primus, S. (2016). Performance and perception in Gundlach, E., Richards, K. A. R., Nelson, D., & Levesque-Bristol, C. (2015).
the flipped classroom. Education and Information Technologies, 21(6), A comparison of student attitudes, statistical reasoning, performance,
1465–1482. https://doi.org/10.1007/s10639-015-9393-5 and perceptions for web-augmented traditional, fully online, and
Bond, M., Buntins, K., Bedenlier, S., Zawacki-Richter, O., & Kerres, M. flipped sections of a statistical literacy class. Journal of Statistics Educa-
(2020). Mapping research in student engagement and educational tion, 23(1), 1–33. https://doi.org/10.1080/10691898.2015.11889723
technology in higher education: A systematic evidence map. Interna- Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects stu-
tional Journal of Educational Technology in Higher Education, 17(2), 1– dent engagement: An empirical study of MOOC videos. Proceedings of
30. https://doi.org/10.1186/s41239-019-0176-8 the First ACM Conference on Learning@ Scale Conference, ACM.
Chang, C. T., Tsai, C. Y., Li, Y. J., Tsai, H. H., & Yu, P. T. (2018). Seamless Hanze, M., & Berger, R. (2007). Cooperative learning, motivational effects,
co-reading system for collaborative group learning. In S. K. S. Cheung, and student characteristics: An experimental study comparing cooper-
L. F. Kwok, K. Kubota, L. K. Lee, & J. Tokito (Eds.), Blended learning: ative learning and direct instruction in 12th grade physics classes.
Enhancing learning success (pp. 287–297). Cham: Springer. https://doi. Learning and Instruction, 17(1), 29–41. https://doi.org/10.1016/j.
org/10.1007/978-3-319-94505-7_23 learninstruc.2006.11.004
Cheng, L., Ritzhaupt, A. D., & Antonenko, P. (2019). Effects of the flipped He, W., Holton, A., Farkas, G., & Warschauer, M. (2016). The effects of
classroom instructional strategy on students' learning outcomes: A flipped instruction on out-of-class study time, exam performance, and
meta-analysis. Educational Technology Research and Development, 67 student perceptions. Learning and Instruction, 45, 61–71. https://doi.
(4), 793–824. https://doi.org/10.1007/s11423-018-9633-7 org/10.1016/j.learninstruc.2016.07.001
Clark, K. R. (2015). The effects of the flipped model of instruction on stu- Hew, K. F., Huang, B., Chu, K. W. S., & Chiu, D. K. (2016). Engaging Asian
dent engagement and performance in the secondary mathematics students through game mechanics: Findings from two experiment
classroom. Journal of Educators Online, 12(1), 91–115. https://doi.org/ studies. Computers & Education, 92, 221–236. https://doi.org/10.
10.9743/JEO.2015.1.5 1016/j.compedu.2015.10.010
Clark, R. E. (1983). Reconsidering research on learning from media. Review of Hew, K. F., & Lo, C. K. (2018). Comparing video styles and study strategies
Educational Research, 53(4), 445–459. https://doi.org/10.2307/1170217 during video-recorded lectures: Effects on secondary school mathe-
Connell, J. P., & Wellborn, J. G. (1991). Competence, autonomy, and relat- matics students' preference and learning. Interactive Learning Environ-
edness: A motivational analysis of self-system processes. In M. Gunnar ments. https://doi.org/10.1080/10494820.2018.1545671.
& L. A. Sroufe (Eds.), Minnesota symposium on child psychology (Vol. 23, Hung, H. T. (2015). Flipping the classroom for English language learners to
pp. 43–77). Chicago, IL: University of Chicago Press. foster active learning. Computer Assisted Language Learning, 28(1), 81–
Creswell, J. W. (2012). Educational research: Planning, conducting, and eval- 96. https://doi.org/10.1080/09588221.2014.967701
uating quantitative and qualitative research (4th ed.). Boston, MA: Ilioudi, C., Giannakos, M. N., & Chorianopoulos, K. (2013). Investigating
Pearson. differences among the commonly used video lecture styles. Proceed-
Dove, A., & Dove, E. (2015). Examining the influence of a flipped mathe- ings of the Workshop on Analytics on Video-Based Learning (WAVe2013).
matics course on preservice elementary teachers' mathematics anxiety New York, NY: The Association for Computing Machinery.
and achievement. Electronic Journal of Mathematics & Technology, 9(2), Kittur, J. (2016). Implementation of student–team–achievement–divisions
166–179. activity and flipped classroom to enhance student learning. Journal of
Education Bureau. (2014). The fourth strategy on information technology in Engineering Education Transformations, Special Issue, 0, 1–5. https://doi.
education: Realising IT potential, unleashing learning power: A holistic org/10.16920/jeet/2016/v0i0/85637
approach (consultation document). Hong Kong: Education Bureau. Larsen, J. (2015). Adult students' experiences of a flipped mathematics
Enfield, J. (2013). Looking at the impact of the flipped classroom model of classroom. Adults Learning Mathematics, 10(1), 50–67.
instruction on undergraduate multimedia students at CSUN. TechTrends, Lo, C. K. (2017). Examining the flipped classroom through action research.
57(6), 14–27. https://doi.org/10.1007/s11528-013-0698-1 Mathematics Teacher, 110(8), 624–627.
Finn, J. D., Pannozzo, G. M., & Voelkl, K. E. (1995). Disruptive and inatten- Lo, C. K. (2018). Grounding the flipped classroom approach in the founda-
tive withdrawn behavior and achievement among fourth graders. Ele- tions of educational technology. Educational Technology Research and
mentary School Journal, 95, 421–454. https://doi.org/10.1086/ Development, 66(3), 793–811. https://doi.org/10.1007/s11423-018-
461853 9578-x
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
156 LO AND HEW

Lo, C. K., & Hew, K. F. (2017a). Does the flipped classroom approach pro- Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Recip-
mote student engagement? A systematic literature review. In M. F. rocal effect of teacher behavior and student engagement across the
Chao & C. K. Li (Eds.), Proceedings of the 2017 International Conference school year. Journal of Educational Psychology, 85, 571–581. https://
on Education, Psychology, and Organizational Behavior (pp. 84–110) doi.org/10.1037/0022-0663.85.4.571
Taipei, Taiwan: International Business Academics Consortium. Skinner, E., Furrer, C., Marchand, G., & Kindermann, T. (2008). Engagement
Lo, C. K., & Hew, K. F. (2017b). Using “First Principles of Instruction” to and disaffection in the classroom: Part of a larger motivational
design secondary school mathematics flipped classroom: The findings dynamic? Journal of Educational Psychology, 100(4), 765–781. https://
of two exploratory studies. Educational Technology & Society, 20(1), doi.org/10.1037/a0012840
222–236. Spanjers, I. A. E., Könings, K. D., Leppink, J., Verstegen, D. M. L., de Jong, N.,
Lo, C. K., & Hew, K. F. (2020). A comparison of flipped learning with Czabanowska, K., & van Merrienboer, J. J. G. (2015). The promised land
gamification, traditional learning, and online independent study: The of blended learning: Quizzes as a moderator. Educational Research
effects on students' mathematics achievement and cognitive engage- Review, 15, 59–74. https://doi.org/10.1016/j.edurev.2015.05.001
ment. Interactive Learning Environments, 28(4), 464–481. https://doi. Szpunar, K. K., Jing, G. H., & Schacter, D. L. (2014). Overcoming over-
org/10.1080/10494820.2018.1541910 confidence in learning from video-recorded lectures: Implications of
Lo, C. K., Hew, K. F., & Chen, G. (2017). Toward a set of design principles interpolated testing for online education. Journal of Applied Research in
for mathematics flipped classrooms: A synthesis of research in mathe- Memory and Cognition, 3(3), 161–164. https://doi.org/10.1016/j.
matics education. Educational Research Review, 22, 50–73. https://doi. jarmac.2014.02.001
org/10.1016/j.edurev.2017.08.002 Talbert, R. (2015). Inverting the transition-to-proof classroom. Primus, 25
López, X., Valenzuela, J., Nussbaum, M., & Tsai, C. C. (2015). Some recom- (8), 614–626. https://doi.org/10.1080/10511970.2015.1050616
mendations for the reporting of quantitative studies. Computers & Edu- The Design-Based Research Collective. (2003). Design-based research: An
cation, 91, 106–110. https://doi.org/10.1016/j.compedu.2015.09.010 emerging paradigm for educational inquiry. Educational Researcher, 32
Maxwell, J. A. (2005). Qualitative research design: An interactive approach (1), 5–8. https://doi.org/10.3102/0013189X032001005
(2nd ed.). Thousand Oaks, CA: Sage Publications. Topping, K. J., & Ehly, S. W. (1998). In K. J. Topping & S. W. Ehly (Eds.),
McKenney, S., & Reeves, T. C. (2012). Conducting educational research Peer-assisted learning Introduction to peer-assisted learning (pp. 1–23).
design. New York, NY: Routledge. Mahwah, NJ: Lawrence Erlbaum Associates.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group. (2009). van Alten, D. C. D., Phielix, C., Janssen, J., & Kester, L. (2019). Effects of
Reprint-preferred reporting items for systematic reviews and meta- flipping the classroom on learning outcomes and satisfaction: A meta-
analyses: The PRISMA statement. Physical Therapy, 89(9), 873–880. analysis. Educational Research Review, 28, 100281. https://doi.org/10.
https://doi.org/10.1371/journal.pmed.1000097 1016/j.edurev.2019.05.003
Newmann, F. M., Wehlage, G. G., & Lamborn, S. D. (1992). The significance van Sickle, J. R. (2016). Discrepancies between student perception and
and sources of student engagement. In F. M. Newmann (Ed.), Student achievement of learning outcomes in a flipped classroom. Journal of
engagement and achievement in American secondary schools (pp. 11–39). the Scholarship of Teaching and Learning, 16(2), 29–38. https://doi.org/
New York: Teachers College Press. 10.14434/josotl.v16i2.19216
Niemiec, C. P., & Ryan, R. M. (2009). Autonomy, competence, and related- Vanderhoven, E., Schellens, T., Vanderlinde, R., & Valcke, M. (2016). Devel-
ness in the classroom: Applying self-determination theory to education oping educational materials about risks on social network sites: A design
practice. Theory and Research in Education, 7(2), 133–144. https://doi. based research approach. Educational Technology Research and Develop-
org/10.1177/1477878509104318 ment, 64(3), 459–480. https://doi.org/10.1007/s11423-015-9415-4
Ogden, L. (2015). Student perceptions of the flipped classroom in college Wang, F., & Hannafin, M. J. (2005). Design-based research and technol-
algebra. Primus, 25(9–10), 782–791. https://doi.org/10.1080/ ogy-enhanced learning environments. Educational Technology, Research
10511970.2015.1054011 and Development, 53(4), 5–23. https://doi.org/10.1007/BF02504682
Palmer, K. (2015). Flipping a calculus class: One instructor's experience. Watkins, C. (2005). Classrooms as learning communities: A review of
Primus, 25(9–10), 886–891. https://doi.org/10.1080/10511970.2015. research. London Review of Education, 3(1), 47–64. https://doi.org/10.
1050618 1080/14748460500036276
Prince, M. (2004). Does active learning work? A review of the research. Whillier, S., & Lystad, R. P. (2015). No differences in grades or level of sat-
Journal of Engineering Education, 93(3), 223–231. https://doi.org/10. isfaction in a flipped classroom for neuroanatomy. Journal of Chiroprac-
1002/j.2168-9830.2004.tb00809.x tic Education, 29(2), 127–133. https://doi.org/10.7899/JCE-14-28
Reeves, T. C. (2006). Design research from a technology perspective. In Whitman Cobb, W. N. (2016). Turning the Classroom Upside Down:
J. V. den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Experimenting with the Flipped Classroom in American Government.
Educational design research (pp. 52–66). London: Routledge. Journal of Political Science Education, 12, (1), 1–14. http://dx.doi.org/
Rotgans, J. I., & Schmidt, H. G. (2011). Cognitive engagement in the prob- 10.1080/15512169.2015.1063437.
lem-based learning classroom. Advances in Health Sciences Education, Yang, Q. F., Lin, C. J., & Hwang, G. J. (2019). Research focuses and findings
16(4), 465–479. https://doi.org/10.1007/s10459-011-9272-9 of flipping mathematics classes: A review of journal publications based
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilita- on the technology-enhanced learning model. Interactive Learning Envi-
tion of intrinsic motivation, social development, and well-being. Ameri- ronments. https://doi.org/10.1080/10494820.2019.1637351
can Psychologist, 55(1), 68–78. https://doi.org/10.1037//0003-066x.
55.1.68
Sailer, M., Hense, J. U., Mayr, S. K., & Mandl, H. (2017). How gamification
motivates: An experimental study of the effects of specific game How to cite this article: Lo CK, Hew KF. Developing a flipped
design elements on psychological need satisfaction. Computers in learning approach to support student engagement: A design-
Human Behavior, 69, 371–380. https://doi.org/10.1016/j.chb.2016. based research of secondary school mathematics teaching.
12.033
J Comput Assist Learn. 2021;37:142–157. https://doi.org/10.
Schroeder, L. B., & Dorn, B. (2016). Enabling and integrating online forma-
tive assessment in a flipped calculus course. Primus, 26(6), 585–602. 1111/jcal.12474
https://doi.org/10.1080/10511970.2015.1050619
 13652729, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12474 by Johannes Kepler Universitãt, Wiley Online Library on [27/12/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LO AND HEW 157

APPENDIX A: S URVEY ITEMS OF STUDENT APPENDIX B: I NTERVIEW TOPI C GUIDE ABOUT

ENGAGEMENT STU DENT EN GAGEMENT

1. Behavioural engagement
Supporting  Tasks completion (e.g., online learning tasks, homework,
Engagement Items resources
classwork)
Behavioural 1. I tried hard to do well in Skinner
 Persistence and concentration (e.g., video lectures, face-to-face
my studies. et al. (2008)
2. In my studies, I worked as lessons)
hard as I could.  Contribution to learning activities
3. I participated in class 2. Emotional engagement
discussions.
 Like or dislike (e.g., video lectures, in-class learning activities)
4. I paid attention to my
studies.  Interest or boredom
5. When I studied, I listened  Sense of belonging (e.g., individual pre-class learning, peer
very carefully. interaction)
Emotional 6. When I studied, I felt Skinner 3. Cognitive engagement
good. et al. (2008)
 Self-regulation (e.g., strategies to plan, monitor, and evaluate)
7. When we worked on
 Investment in learning (e.g., go beyond requirements, prefer
something in class, I felt
interested. challenge)
8. The class was fun.  Coping strategies (e.g., facing failure or challenge)
9. I enjoyed learning new
things.
10. When we worked on
something in class, I got
involved.
Cognitive 11. I was engaged with the Rotgans and
topic at hand. Schmidt (2011)
12. I put in a lot of effort.
13. I wished we could
continue with the work for
a while.
14. I was so involved that I
forgot everything around
me.