THE EFFECT OF BLENDED LEARNING

ON EFL STUDENTS’ PERFORMANCE IN

RESEARCH METHODOLOGY AND PRACTICE OF
21ST CENTURY SKILLS
Hadiyanto Hadiyanto, Universitas Jambi
Urip Sulistiyo, Universitas Jambi
Amirul Mukminin, Universitas Jambi
Eddy Haryanto, Universitas Jambi
Syaiful Syaiful, Universitas Jambi

ABSTRACT
This study aims to determine the effect of blended learning methods on EFL students’ performance in
research methodology and their practice of 21st century skills. A quasi-experimental design was used for
the study. The sample included two groups of 25 students divided into a control group for conventional
learning and an experimental group for blended learning. A test and take-home essay of short research
proposal writing was used to measure the students’ performance in research methodology, while their
practice of 21st century skills was measured via a self-report questionnaire. The results showed that
the blended learning method significantly influenced their performance in both areas. The experimental
class showed a significantly higher performance in research methodology and 21st century skills than the
control. The study outcome expatiates on the value of the implementation of blended learning in enhancing
research methodology performance and modern skills practice in the classroom.

Keywords: Soft skills, online Learning, English learning, 21st century skills, covid 19 pandemic.
INTRODUCTION students to interact physically, emotionally, and
Blended learning, which integrates face-to- expressively. Many researchers stated that blended
face and online classes, has been applied in higher learning is an opportunity for students to engage
education worldwide for over 10 years. Positive more, update resources, communicate, discuss,
issues related to the application of online learn- share, and use other additional applications in
ing, such as teaching strategies; students’ interests, presenting their assignments online. However,
motivations, and attitudes; and learning effective- some teachers are not interested in online learning
ness, efficiency, and quality had been discussed because they believe online students fail to partici-
widely (Khan et al., 2012; Wuryaningsih et al., pate in classroom activities, deliver course content,
2019). Furthermore, some studies showed that communicate with others, and engage in various
online learning effectively supports and poten- academic functions, thereby affecting their acqui-
tially replaces face-to-face classes. Online classes sition of knowledge and skills so that students do
offered via elearning effectively enhance learn- not acquire adequate knowledge and skills through
ing capacity while face-to-face classes allow online learning. This negative attitude cannot be

JOURNAL OF EDUCATORS ONLINE

countered by theory alone but needs evidence pro- Blended Learning not only provides more
vided by research. options for students, but it also has other benefits,
Blended learning is an extension of the physical such as improving access to learning materials
and online learning approaches where the teach- and enhancing the quality of learning. Various
ers facilitate students to engage in activities that academic activities such as lecturing, group and
fulfill their educational objectives. Online learn- individual projects, presentations, resource shar-
ing software include Edmodo, Moodle, Easyclass, ing, and free discussions can be applied in a
and others with features that support students in combination of face-to-face classroom and online
communicating, sharing information and learning learning (Khan et al., 2012).
resources, working in groups, submitting assign- Blended learning activities provide an explicit
ments, and performing other learning activities. focus on course content that improves students’
Online learning helps the conventional classroom academic performance and encourages 21st cen-
to adequately fulfill the educational requirements of tury skills, including soft and hard skills. These
students. In blended learning, students are allowed are both improved when students interact through
to initiate interaction, learn from others, enhance various activities inside the classroom and out-
their own learning outcomes, improve their intel- side through online learning (Singh & Singh,
lectual capacity, and develop their knowledge and 2017; Witherspoon, 2011). Delialioğlu, (2012)
skills. Teachers also have the ability to manage and and Schober et al. (2008) also stated that blended
control an online class in a much easier and more learning gives the opportunity and enables the
flexible manner (Broadbent, 2017). Furthermore, environment for students to communicate actively
through blended learning the students are expected with other students, which ultimately leads to
to optimally explore and acquire course knowl- communication, teamwork, problem-solving, and
edge and practice 21st century soft skills such as learning hard skills.
communication, IT, numeracy, Learning, problem- As a part of blended learning, online learn-
solving, and teamwork. Hard skills are equally ing allows the students to connect with classroom
acquired through the online learning process. activities at any time in any place (Nathan &
We hypothesized that applying blended learn- Rajamanoharane, 2016). Singh and Singh (2017)
ing enhances the performance of students in reiterated that it allows students to work together
research methodology as well as their acquisition and report back or present to the class as a whole,
of 21st century skills (Hadiyanto, 2019a; Schober thereby encouraging student-to-student interaction.
et al., 2008). This experimental research involved Furthermore, teachers are able to provide clear
applying blended learning to obtain evidence and directions and realistic goals for group and indi-
further prove the hypothesis. vidual assignments. Their roles include designing
blended activities for face-to-face and online learn-
LITERATURE REVIEW ing and giving directions and instruction to enable
Blended Learning, Students Course Performance, students to acquire 21st century skills through aca-
and the Practice of 21st Century Skills demic activities within and outside the classroom.
Blended learning is a combination of a face- Hadiyanto (2019b) and Shulamit and Yossi (2011)
to-face and an online class designed according to stated that these activities can be managed through
pedagogical principles (Bryan & Volchenkova, online learning to boost their 21st century skills.
2016; Ellis et al., 2008; Kara, 2016). It is an Students interact, read, question, and discuss the
extension of elearning models where a student- resources provided by the teacher. A class mem-
centered approach is applied in the transmission ber can upload multiple resources to share with
of knowledge. Combining online and face-to-face all class members to learn, compare, analyze, and
lessons enables students to continue learning via extract the necessary information from them.
a virtual class. It also gives students flexibility Figure 1 shows the conceptual framework of
in terms of time and place for interacting, com- blended learning applied in this research. It points
municating, working in groups, discussing, and out the three main phases: (a) course design, (b)
sharing resources (Fischer & Hanze, 2019; Glowa teaching and learning activities involving the inte-
& Goodell, 2016). gration of face-to-face and online learning, and (c)

JOURNAL OF EDUCATORS ONLINE

learning outcomes (Delialioğlu, 2012; Hadiyanto, organizing small discussions, posting and playing
2019c; Kara, 2016). video files, and providing links to online resources.
Figure 1. A Conceptual Framework of Blended Learning The role of teachers is important in facilitating
students to be more investigative and interactive
through elearning. Creative and innovative learn-
ing designs are also applied in online learning
activities.
21st Century Skills
The term “21st century skills” is currently used
by researchers, academicians, educators, and pro-
fessionals to refer to the essential skills that are
useful to employees in the workplace and there-
fore have to be acquired by university graduates
before entering the job market. The interchange-
able terms for “21st century skills” used in several
countries today include key skills; core compe-
tencies; adaptive, soft, generic, global, life, core,
and interpersonal skills; etc. The components of
each are closely associated with the other, such
as communication, IT, critical thinking, analyt-
ics, learning, intrapersonal relationship, subject
skills, etc., (Bialik et al., 2015; Hadiyanto, 2019b;
ILO, 2014; Lippman et al., 2015; Ontario Public
Services, 2016; Partnership for 21st Century Skills,
2008; Person et al., 2009). In short, 21st century
skills mainly consist of course knowledge and
Edmodo as a Tool for Online Learning soft skills. According to the analysis of the pres-
We used Edmodo during blended learning ent study, these skills are defined as the soft and
classes (the experiment class), and it is a highly hard skills required for the global job market that
have to be acquired by students through learning
relevant elearning platform used by teachers world-
activities at the university level (Bialik et al., 2015;
wide. The interface is very intuitive and allows for
Ontario Public Services, 2016; Partnership for 21st
easy navigation. It also allows the integration of a
Century Skills, 2008).
wide variety of resources as well as links to gen- Soft skills are the practical activities applied
uine sources. The features provided by Edmodo to generate and develop hard skills in the student
support blended learning activities such as creat- learning context and former student working con-
ing small groups to enable students to discuss and text. They include communication, IT, numeracy,
work cooperatively and collaboratively (Hadiyanto, problem-solving skills, and working with oth-
2019a; Lin et al., 2016; Singh & Singh, 2017). ers (Akyıldız & Çelik, 2020; Hadiyanto, 2019a;
Alternatively, Edmodo supports teacher-student Marando, 2012; Washer, 2007; Zalizan & Azman,
and student-student interactions while facilitating 2005). Furthermore, in this study, soft skills fur-
activities that enhance their learning 21st century ther include how-to skills, problem solving, and
skills (Kara, 2016; Witherspoon, 2011). Using it teamwork in learning activities (Hadiyanto, 2019b).
as a medium of online learning will give a wide Hard skills deal with specific knowledge
opportunity for students to improve their academic and subject content and the technical skills of a
performance. Edmodo also allows teachers to add student’s major (Marando, 2012; Washer, 2007;
learning resources and features such as links, Wilson-Ahlstrom et al., 2014). In the context of this
video presentations, and other supporting appli- study “hard skills” are defined as the practice of
cations. Solangi et al. (2018) stated that teachers specific knowledge through methods partly or fully
can optimize these features of elearning through blended with soft skills. Specifically, they relate to

JOURNAL OF EDUCATORS ONLINE

the knowledge of research methodology and the skills of soft skills and, consequently, the acquisition of
practiced by students through blended learning activ- hard skills and the fulfillment of course objectives
ities (Hadiyanto, 2019c; Marando, 2012; Ristekdikti, (Dunne et al., 2000; Hadiyanto, 2019a).
2015; Washer, 2007; Wilson-Ahlstrom et al., 2014). The 21st century skills in the present study are
Improving the 21st Century Skills of Students not a means for measuring students’ performance.
Through Blended Learning Activities This study characterizes the learning activities
Student-centered learning approaches widely within a course context and then compares them
recognize that blended learning is a strategy for between the conventional and blended learning
implementing academic activities in face-to-face classes. The hypothesis behind the comparison is
and online learning (Bryan and Volchenkova, that blended learning promotes a student-centered
2016). This is an innovative method applied to approach in both face-to-face classes and online
deliver course content to students in order to obtain learning, while conventional learning involves
a measurable learning outcome. Not only does it the physical classroom only. The indicators of
develop their academic performance, but it also skill practice were developed carefully and tested
improves communication, critical thinking, prob- repeatedly by Hadiyanto et al. (2018) and applied
lem-solving, numeracy, teamwork, and IT skills in this study to gather data from the students.
during the learning process and activities. Teachers METHODOLOGY
can observe and guide students while they engage, This is an experimental research study that aims
communicate, discuss, and work as a team to fulfill to determine the effect of the blended learning method
learning objectives. (Partnership for 21st Century on students’ research methodology performance, and
Skills, 2008), Wilson-Ahlstrom et al. (2014) to ascertain the difference between students in a con-
observed that the development of 21st century soft ventional and a blended learning class in terms of 21st
and hard skills occurred during the learning pro- century skills practice. Ninety-six students enrolled
cess. Appropriate teaching and learning methods in the research methodology course, and 50 of the stu-
create an opportunity for students to practice soft dents were divided into two groups according to their
skills by which they eventually gain hard skills.
respective CGPA equivalence, with 25 participants
Students are expected to possess observable 21st
in the control class and 25 in the experimental class.
century skills and measurable academic perfor-
There were three students with CGPA 3.75 and above
mance as discussed previously.
in each group, eight students between 3.50 and 3.74,
Teaching scenarios have to be specialized
11 students between 3.00 and 3.49, and three students
for active learning activities such as interacting,
exploring, surfing the internet, using applications, at 2.99 and below. The students’ research methodol-
sharing, suggesting, responding, discussing, pre- ogy performance and 21st century skills practice were
senting, writing reports, and working as a team. the major variables tested. For further data analysis,
These scenarios allow students to practice their both variables were divided into subvariables.
21st century skills. The role of the teacher involves Instruments and Their Reliability
guiding and facilitating the students to actively Table 1 illustrates the design of the study.
engage in learning and develop their own skills and X1=Test of Research Methodology
capacities. Student engagement through learning X2=Take-home essay of short research proposal
strategies and methods will enhance the practice writing
Table 1. Classroom Experimental Design of Blended Learning
Class Preassessment Experimentation- Postassessment (Dependent Variables)
(Dependent Variables) (Independent Variables)
Control X1 Y1 X1
X2 X2
X3
Experiment X1 Y2 X1
X2 X2
X3

JOURNAL OF EDUCATORS ONLINE

 X3=Self-Evaluation questionnaire of 21st cen- culturally relevant strategies for interpreting
tury skills practice and reporting the results of research.
Y1=Control Group (Conventional) A scoring rubric was used by two assessors to
Y2=Experimental Group (Blended Learning) judge the score of an element. Each element is scored
This study used three instruments: (a) a research from 1 to 20. Inter-rater reliability was used to test
methodology test, (b) a take-home essay of short the degree of consistency and agreement between
research proposal writing and (c) a self-evaluation both assessors. Cohen (1960) and Hair et al. (2006)
questionnaire of 21st century skills practice. The suggested that both the homogeneity and consensus
research methodology test and take-home essay values between assessors has to be 70% and above.
were applied to assess the students’ performance. In this study, the overall pre-take-home essay of short
Both assessments were taken before and after proposal writing yielded Cronbach alpha consensus
experimentation. The research methodology test of 92% and intraclass correlation or homogeneity in
was subdivided into three categories of questions: terms of single and average measures above 80%.
(a) matching, with a maximum score of 10 for 10 Meanwhile, the overall postwriting take-home essay
questions, (b) multiple choice, with a maximum of short proposal yielded a Cronbach alpha of 86%
score of 40 for 40 questions, and (c) essay, with a and intraclass correlation (homogeneity) in single
maximum score of 50 that consisted of five ques- measures and average measures above 75%. This
tions. Furthermore, the take-home essay of short means that the reliability or consistency of judgment
research proposal writing was expected to include, between both raters is at a very good level.
minimally, the following elements: The self-evaluation questionnaire was used
• A description of the study problem occurring to assess the students’ 21st century skills practice.
in a practice setting and rationale of the study. It was distributed to students after the experiment
• A preliminary literature review (6–8 articles) and consisted of indicators of skills practiced by
analyzing existing information that will help the students from before and after experimentation.
the researcher understand the problem. Students in both face-to-face and blended learning
classes were required to rate the level for each skill
• A description of the research methods, indicator practiced during the learning process. Each
such as qualitative, quantitative, single-case component of soft and hard skills was described by
designs, action, and outcome-based research. some statements. The Likert scale interval from 1
• A description of information that the research (very poor) to 5 (very good) was applied in judging
potentially provides. their 21st century skills practice during the teach-
• A description and analysis of ethical and ing and learning process. The questionnaire was a
Table 2. The Reliability of Self-evaluation Questionnaire

Corrected item-total
The 21st Century Skills Practices Number of Items Cronbach Alpha
correlation
Communication 9 .328–.592 .757
IT 6 .323–.406 .643
Numeracy 8 .314–.673 .758
Learning 11 .335–.598 .813
Problem-Solving 7 .414–.529 .745
Teamwork 8 .305–.681 .708
SOFT SKILLS 41 – .903
Knowledge of Research Acquisition 5 .378–.545 .684
Skills of Research Methodology Practices 5 .325–.401 .695
Hard Skills Practices 10 – .747
Overall 21st Century Skills 59 – .920

JOURNAL OF EDUCATORS ONLINE

Table 3. Independent t-test Result Between Control and Experiment Group in Pretest

Mean
Pretest Group N Mean t Sig.
Different
Control 25 4.84 .280 1.091 .281
Matching (Max. Score 10)
Experiment 25 4.56
Control 25 19.20 .560 .938 .353
Multiple choice (Max. Score 40)
Experiment 25 18.64
Control 25 17.48 −.320 −.367 .715
Essay Questions (Max. Score (50)
Experiment 25 17.80
Control 25 41.52 .520 .332 .741
Total Pretest
Experiment 25 41.00

piloted study with 50 randomized respondents to equality of the students’ research methodology
obtain consistency and correct item-total correla- performance between the control and experiment
tion. Pallant (2011) and Hair et al. (2006) suggested before experimentation. There was a significant
that a Cronbach alpha coefficient of 60% for a improvement due to the combination of the research
construct consisting of 10 items and fewer, while methodology test and the take-home essay of short
70% was recommended for a construct consist- research proposal writing. These were the two
ing of more than 10 items. A corrected item-total types of assessment conducted before and after
correlation at 30% is acceptable. The results of the experimentation to determine the effect of blended
consistency analysis had a Cronbach alpha coeffi- learning on students’ course performance.
cient of 72% and corrected item-total correlation
Pretest of Research Methodology
at 30% and above, with all subcomponents of soft
Three parts of the test were given to students:
and hard skills obtaining α level more than 60%
matching, multiple choice, and essay questions. An
and corrected item-total correlation more than 30%
equal pretest score was observed between the con-
(Pallant, 2011). The specification of the question-
trol and experiment class in the matching section (t
naire and the reliability is shown in Table 2.
An observer took observations during blended = 1,091, sig. = .281 > .05), multiple choice (t = .938,
learning activities in both the physical classroom sig. = .353 > .05), and essay questions (t = −.367,
and online. The focus of the observation was the sig. = .715 > .05), and the overall score of research
teaching method and task, the main online tool methodology (t = −.332, sig. = .741 > .05). This
used, the online activities of the students, and the implies that an equal competence was obtained
knowledge gained and skills practiced. between the two groups prior to experimentation.
The results are shown in Table 3.
Data Analysis
The independent t-test was applied in data Pre-take-home Essay of a Short Research
analysis to determine the differences between the Proposal Writing
control and experimental classes on the students’ A take-home essay of short research proposal
performance in research methodology and 21st writing was given to the control and experiment
century skills practice. Pallant (2011) suggested groups before experimentation. Afterwards, an
that this test is appropriate for comparing two independent sample t-test was performed to inves-
groups with a sample size below 30. A significant tigate the equivalence between the control and
value at 0.05 to 0.1 was used to ascertain the differ- experimental groups in terms of students’ ability
ences between the scores. in research methodology. The results in Table 4
showed that there is an equivalence between the
RESULTS overall score of both groups regarding the take-
Comparison of Students’ Research Methodology home essay of short research proposal writing
Performance before Experimentation (mean different = −2.150, t = .974 and sig. = 335
A pretest was conducted to ascertain the < 05). An equivalence is also obtained between

JOURNAL OF EDUCATORS ONLINE

Table 4. Independent Sample t-test Conducted to Control and Experiment Group toward Pre-take-home Essay of Short Research Proposal Writing.

Pre-A short research proposal Mean

Group N Mean t Sig.
writing (Take-home essays) Different
Control 25 12.10 −.450 −.692 .492
Description of the study problem occurring in a
practice setting and rationale for the study.
Experiment 25 12.55
A preliminary literature review (6–8 articles) 25 12.00 .100 .160 .874
Control
analyzing existing information that will help the
researcher to understand the problem. Experiment 25 11.90
A description of the research methods, such as 25 11,60 −.100 −.161 .873
Control
qualitative, quantitative, single-case designs,
action, and outcome-based research. Experiment 25 11.70

A description of information that the Control 25 11.00 −.700 −1.22 .227

research potentially provides. Experiment 25 11.70
A description and analysis of ethical and 25 11.35 −1.00 −1.80 .078
Control
culturally relevant strategies for interpreting
and reporting the results of research. Experiment 25 12.35
Control 58.05 −2.15000 −.974 .335
Overall
Experiment 60.20

groups in the following elements: (mean different = −.100, t = −.161, and sig. =
• A description of the study problem occurring .873 < 05).
in a practice setting and a rationale for the • A description of information that the
needs of the study (mean different = −.450, t research potentially provides (mean different
= −.692, and sig. = .492 > 05). = −.700, t = −1.223, and sig. = −.223 < 05).
• A preliminary literature review of 6–8 • A description and analysis of ethical and
articles analyzing existing information that culturally relevant strategies for interpreting
will help the researcher to understand the and reporting the results of research (mean
problem (mean different = −.100, t = −.691, different = −1.00, t = −1.800, and sig. = .078
and sig. = .874 < 05). < 05).
• A description of the research methods such These results implied that there was an equiva-
as qualitative, quantitative, single-case lence between the control and experimental groups
designs, action, and outcome-based research in the pre-take-home essay of short research
Table 5. The Independent t-test Result Between Control and Experiment Groups at Posttest

Post Test Group N Mean Mean t Sig.

Different
Control 8.52 −.440 −1,08 .285
Matching (Max. Score 10)
Experiment 8.96
Multiple Choices (Max. Score 40) Control 32.0 −2.24 −2.20 .032
Experiment 34.2
Essay Questions (Max. Score 50) Control 32.8 −3.88 −2.26 .028
Experiment 36.7
Total Posttest Control 73.3 −6.560 −2.21 .031
Experiment 79.9

JOURNAL OF EDUCATORS ONLINE

proposal writing. Table 6 shows a significant difference in the
Comparison of Students’ Research Methodology overall posttest score on take-home essay of short
research proposal writing between the control
Performance after Experimentation Posttest of
and the experimental groups (mean difference =
Research Methodology −8.400, t = −7.606, and sig. = .000 < 05). Significant
An independent sample t-test was conducted to differences were also observed between groups on
determine the statistical differences between the the description of the problem occurring in a prac-
control and experiment groups towards the posttest tice setting and the rationale of the study (mean
research methodology. As shown in Table 5, there difference = −1.65, t = —3.914, and sig. = .000 <
was a significant difference in the overall scores 05), a preliminary literature review (6–8 articles)
when compared with the posttest in both groups analyzing existing information that will help the
(t = −2.21, sig. = .031 < .05). This significant dif- researcher to understand the problem (mean differ-
ference was also observed in its subcategories of ence = −2.40, t = −4.642, and sig. = .000 < .05), a
multiple-choice (t = −2.26, sig. = .032 < .05) and description of the research methods such as quali-
essay questions (t = −2.20, sig. = .032 < .05). The tative, quantitative, single-case designs, action,
experiment group yielded a higher mean score in and outcome-based research (mean difference =
the overall test, particularly in its subcategories −.100, t = −.161, and sig. = .873 < 05), a descrip-
of multiple-choice and essay questions. However, tion of information that the research potentially
there was no significant difference between groups provides (mean difference = −.850, t = −2.798, and
in the matching category. sig. = −.007 < 05), and a description and analy-
Post-take-home Essay of a Short Research sis of ethical and culturally relevant strategies for
Proposal Writing interpreting and reporting the results of research

Table 6. Independent Sample t-test Conducted to Control Group and Experiment Group toward Post-take-home Essay of Short Research Proposal Writing

Post-take-home essay of short

Group N Mean Mean Different t Sig.
research proposal writing

Description of the study problem occurring in Control 25 15.60

−1.65 −3.914 .000
a practice setting and rationale for the study.
Experiment 25 17.25
A preliminary literature review (6–8 articles)
Control 25 15.25
analyzing existing information that will help −2.40 −4.642 .000
the researcher to understand the problem. Experiment 25 17.65

A description of the research methods, such as

Control 25 15.30
qualitative, quantitative, single-case designs, −1.30 −3.194 .002
action, and outcome-based research.
Experiment 25 16.60

A description of information that the Control 25 15.00

−.850 −2.798 .007
research potentially provides. Experiment 25 15.85
A description and analysis of ethical and
Control 25 15.05
culturally relevant strategies for interpreting −2.20 −5.047 .000
and reporting the results of research. Experiment 25 17.25
Control 25 76.20
Overall −8.400 −7.606 .000
Experiment 25 84.60
Significant at 0.05 to 0.1

JOURNAL OF EDUCATORS ONLINE

Table 7. Independent Sample t-Test Between Control and Experiment Group Toward Soft Skills practice

Soft Skills Group N Mean Mean Different t Sig.

Control 25 3.64 −.434 -4.59 .000
Communication
Experiment 25 4.08
Control 25 3.59 −.320 -2.73 .000
IT practice
Experiment 25 3.91
Control 25 3.42 −.520 -5.70 .000
Numeracy
Experiment 25 3.94
Control 25 3.90 −.272 -2.22 .031
Learning
Experiment 25 4.17
Control 25 3.87 −.377 -3.22 .002
Problem Solving
Experiment 25 4.25
Control 25 3.91 −.250 -2.63 .011
Teamwork
Experiment 25 4.16
Control 25 3.80 −.351 -4.98 .012
Soft Skills
Experiment 25 4.16

and/or program evaluation studies (mean differ- groups, specifically communication (mean dif-
ence = −2.20, t = −5.047 and sig. = .00 < 05). These ference = −.434, t = −4.59, sig. = .000 < .05), IT
results showed that blended learning significantly practice (mean difference = −3.20, t = −2.73, sig. =
increases students’ competency in take-home .000 < .05), numeracy (mean difference = −.520, t
essays of short proposal writing. = −5.70, sig. = .000 < .05), learning (mean differ-
ence = −.272, t = −2.22, sig. = .031 < .05), problem
solving (mean difference = −.377, t = −3.22, sig.
Differences in Students’ Perception toward
= .002 < .05), and teamwork (mean difference =
the Practice of 21st Century Skills between −.250, t = −2.63, sig. = .011 < .05). The experimen-
Conventional Classroom and Blended Learning tal group had a higher mean score in soft skills
Table 7 shows the result of an independent practice and its components than the control group.
t-test conducted to determine the mean score These results imply that blended learning gives
of 21st-century skills practice, which consist of more opportunities to students to practice their soft
soft and hard skills. A significant difference was skills in the methodology course.
observed between the control and experimental Table 8 shows a significant difference in the
groups in the overall level of soft skills practice overall hard skills practice between the control
(mean difference = −.351, t = −4.98, sig. = .012 and experiment groups (mean difference = −.312,
< .05). Looking at each component of soft skills, t = −2.78, sig. = .008 < .05). Significant differ-
significant differences were also observed in both ences were also observed between the control

Table 8. Independent Sample t-Test Conducted in the Control and Experiment Groups on the Practice of Hard Skills

Hard Skills Practices Group N Mean Mean Different t Sig.

Knowledge and Practice of Control 25 3.98 −.312 -2.78 .008
Research Methodology Experiment 25 4.29
Specific Skills of Research Control 25 4.14 −.355 -5.04 .000
Methodology practice Experiment 25 4.32
Control 25 4.06 −.272 -2.99 .000
Overall Research Methodology practice
Experiment 25 4.31

JOURNAL OF EDUCATORS ONLINE

Table 9. Independent Sample t-Test on the 21st Century Skills Practice Conducted on the Control and Experiment Groups After Evaluation

The 21st Century Skills N Mean Mean Difference t Sig.

Control 25 3.86 .0704 −5.04 .000
Experiment 25 4.13

and experimental groups in knowledge and prac- conventional methods.

tice of research methodology (mean difference Table 9 shows the statistical comparison of
= −.184, t = −1.83, sig. = .073 < .05) and specific overall students’ 21st century skills between the
skills of research methodology practices (mean control and experimental groups. A significant dif-
difference = −.248, t = −2.73, sig. = .009 < .05). ference was observed in both 21st century skills
The experimental group practiced hard skills and practice (mean difference = .0704, t = —5.04, sig.
their components more intensively than the con- = .000 < .05). The experimental group yielded
trol group. Therefore, blended learning allows a higher mean score in skills practice than the
students to practice and acquire research method- control group. Therefore, blended learning signifi-
ology knowledge and skills much better than the cantly enhances the 21st century skills in students
Table 10. Observation Summary of Blended Learning Activities
Students’
Delivery method Main online
Topic of discussion online What do the students practice and gain?
and task tool used
activities
The notion of social science Lecturing in the eLearning wall Discussing, Soft Skills
research, type and current classroom. Reply button asking a Communication, IT, teamwork. Learning
issues of research in language Online discussion Messages question, Hard Skills
teaching and learning. in the group. Assignment sharing ideas knowledge of social research, and awareness
and resources. about interesting issues of research in ELT

Research background, Video resources Video Watching and Soft Skills

current issues related to the posting. eLearning Wall commenting on Communication. IT. Working with others,
need for the study, Research Online discussion Reply button video resources. problem-solving, learning. Numeracy
problems and objectives in group. Assignment Hard Skills
Group project report. Online submission Knowing how to find key issues for
Classroom a research background.
presentation. Practice writing a research background,
Individual report.
Writing a literature review, E-resource sharing. Group Wall and Online searching, Soft Skills
finding relevant theory and Free online discussion. Reply Button resources Communication. IT. Working with
related research, types Group classroom sharing. others, problem-solving, learning.
of sources of literature, discussion and Reading Hard Skills
writing in APA Style. presentation. resources. Knowing relevant theories of their research
Individual report. Commenting topic. Practice writing literature reviews.
Sharing content knowledge of the topic.
A description of the research Work in groups in Elearning Wall and Online Soft Skills
methods, such as qualitative, and outside the Reply Button. questioning and Communication. IT. Teamwork,
quantitative, single-case classroom (Offline). answering. problem-solving, learning skills.
designs, action, and Group online Hard Skills
outcome-based research; presentation. Knowing the appropriate research method for a
study. Selecting a research method for a proposal.
Writing a short Teachers direction Elearning Wall - Softs Skills.
research proposal. in classroom. Assignment IT, problem-solving and learning skills.
Individual online Online submission Hard Skills
presentation. Applying knowledge of research methodology.

JOURNAL OF EDUCATORS ONLINE

better than conventional methods. essay questions categories. However, the match-
Observation Summary of Blended Learning ing test category showed no difference between
Activates in the Experimental Classroom the conventional and blended learning classes. The
Table 10 shows the results of the observation results also showed that students in the blended
in the experimental classroom group. Five main learning class showed higher performance in the
topics of methodology course content were taught take-home-essay of short research proposal writ-
by the teachers in the blended learning class as ing. Specifically, blended learning resulted in a
well as the control class. The teachers employed higher performance for a description of the study
the following learning methods in face-to-face problem occurring in a practice setting and ratio-
classes: lectures, classroom group discussions, nale for the study; a preliminary literature review
group projects, and presentations in both blended (6–8 articles) analyzing existing information that
and conventional learning (control class). The fol- will help the researcher to understand the prob-
lowing learning methods were applied for online lem; a description of the research methods, such
learners in the blended learning class: wall post- as qualitative, quantitative, single-case designs,
ings, resource sharing, free discussions, individual action research and outcome-based research; a
posting and presentations, and group discussions. description of information that the research poten-
The main elearning tools used in transmitting tially provides; and a description and analysis of
knowledge were the elearning wall, reply button, ethical and culturally relevant strategies for inter-
group wall, messages, assignments, and online preting and reporting the results.
submission. Students were made to participate Students in the blended learning class par-
through topic discussions, asking questions, shar- ticipated in the lessons without being physically
ing ideas and resources, presentations through present in the class. Their eagerness to learn was
video, observing, commenting on videos, online continually channeled towards elearning. They
searching, and resource sharing. Furthermore, the participated by posting, asking questions, respond-
sequences involved in blended learning enabled ing, and following class updates. Ma, et al., (2019)
the students to practice and therefore enhance both revealed that blended learning yields more oppor-
their soft and hard skills. tunities for students to experience and interact
flexibly in an online classroom after receiving gen-
DISCUSSION eral knowledge of topic discussions in the physical
This study investigated the effect of blended classroom. These methods significantly develop stu-
learning activities on students’ performance in dents’ content knowledge and obtain better results.
research methodology and their practice of 21st Online learning activities such as group and indi-
century skills. Blended learning was implemented vidual discussions, online presentations, and sharing
in the experimental class to determine these resources, significantly enhanced students’ perfor-
effects. The study contributes to the area of blended mance in research methodology. This is reflected
learning implementation. It also contributes to by the score of the blended learners, which showed
the nature, concept, and method of implementing a higher posttest and post-take-home essay scores
blended learning in different types of education. than the conventional classroom students. The find-
The study showed that blended learning activi- ings are in line with a study conducted by Broadbent
ties are more effective for enhancing students’ (2017) that concluded that blended learning signifi-
methodology course performance and 21st century cantly improved students’ academic performance,
skills practice than the conventional face-to-face and one by Hadiyanto et al. (2021) that revealed that
classroom. Their performance in this study was blended classes show higher GPA achievement than
used as a measurable dependent variable. Students’ the conventional category.
performance in both test and take-home essays In connection with 21st century skills practice,
showed statistically significant improvement. An the integral aspects of blended learning involve com-
improved score was observed for blended learn- munication, the use of IT, arranging work to meet a
ing students in both test and take-home essays deadline, learning from others, updating resources,
in research methodology. A significant differ- solving problems, and working in groups. These
ence was also found in the multiple-choice and soft-skill practices, taught through blended learning,

JOURNAL OF EDUCATORS ONLINE

enable students to gain course knowledge and skills different ways and enhance their practice of 21st
while significantly improving their academic perfor- century skills. For instance, a group discussion
mance (Fischer & Hanze, 2019; Glowa & Goodell, helps to improve students’ communication and
2016; Hadiyanto et al., 2018). Furthermore, the teamwork skills. Particularly, it enables the teach-
results showed that the students’ 21st century soft ers to practice blended learning, thereby allowing
skills in the blended learning class, such as commu- the students to practice soft and hard skills related
nication, IT, numeracy, learning, problem solving, to the subject. Teacher innovation and creativity in
and teamwork, were more improved than in the combining learning methods such as group discus-
conventional class. Students were able to build and sion, individual learning, and group and individual
enhance their individual course knowledge and hard presentations and projects, is another important
skills as well. Learning activities in the classroom factor for success in using a blended learning strat-
were saved to an online platform to prevent stu- egy that significantly impacts students’ learning
dents losing the information they obtained during outcome.
classroom learning. Through the practice of skills From the theoretical perspective, this research
and knowledge of research methodology, various adds more value to the available literature on
blended learning activities lead students to attain blended learning usage in different education con-
their learning goals. texts. The practice of 21st century skills during
The results also imply that the students’ 21st the learning processes is an argument for blended
century skills may be developed not only in the learning usage. In the past, blended learning has
conventional classroom but through online learn- contributed to the area of teaching and learn-
ing as well. Academic activities online provide ing, and this research gives more information on
wider flexibility for students to think critically, the theory and innovative concept of its imple-
contribute, and share ideas in a course topic discus- mentation. Furthermore, this study showed that
sion. As a result, the students have more chances to blended learning enhances student academic per-
practice and enhance their 21st century skills, both formance and improves their propensity to use
soft and hard skills, while improving their aca- IT applications, work together, search for solu-
demic performance (Hadiyanto, 2019c; Hadiyanto tions to problems, discover better ways to learn,
et al., 2021; Singh & Singh, 2017). Students showed and present and share information and resources.
higher participation in online learning activi- Therefore, blended learning’s methods enable stu-
ties after conventional classroom learning. They dents to practice and develop their 21st century
maintain and develop their capacity by discussing, skills through learning activities. A hypothesis can
communicating, searching, sharing, question- be created from this notion: The use of blended
ing, answering, suggesting, and responding. This learning by more teachers ensures an improvement
agrees with Nathan & Rajamanoharane (2016) that in the academic performance and skills of students.
online learning supplements and supports students However, further studies have to include a wider
in gaining more knowledge and soft skills while population and multiple classes for the develop-
improving their attitude towards learning. ment of experimental research on blended learning
Contribution of the Research in this context of the study. Mixed qualitative and
The research contributes to the theoretical and quantitative methods are needed to ascertain the
practical sides of blended learning strategies. For effectiveness of blended learning activities and
a practical contribution, it provides new infor- ways of improving the 21st century skills of stu-
mation that increases teachers’ advantages when dents based on the current research definition.
using online learning strategies while inviting As part of a larger research push, these results
other teachers to apply them in more creative ways have been forwarded to the policymakers of our
to fulfill course objectives. It also helps teachers university. Only a few universities worldwide pro-
choose the appropriate content delivery method vide extracurricular training to equip students with
to enable students to acquire better academic 21st century skills. The practice of these skills
performance and 21st century skills. The study through blended learning activities will be help-
results also show educators that different learn- ful in reducing the number of unskilled graduates.
ing methods improve students’ performances in Furthermore, it gives students the advantage of

JOURNAL OF EDUCATORS ONLINE

acquiring skills while in school and rendered as of blended learning research and inspire other
unnecessary spending more funds to acquire basic teachers to use it to fulfill the objective of learn-
skills after graduation. This is why policymakers ing. As more universities worldwide are enacting
in the university suggest that teachers have to be policies to move to fully online learning due to the
trained in the strategic and effective use of blended COVID-19 pandemic, further research has to focus
learning strategy in the classroom. on seeking and designing blended learning to meet
Implication and Limitation pedagogical principles, and be more interactive
These results disagree with the assumption and effective, particularly in the foreign language
that online learning fails to help students catch teaching context.
up with their classroom functions, gain adequate
knowledge, and engage in academic activities.
Previously, many educators and teachers stated that
academic activities, skills, and knowledge cannot
be enhanced through online learning. However,
some researchers have attempted to ascertain
the extent of teacher competence in designing
and implementing blended learning activities, as
many online learning platforms presently have
adequate features that support various learning
activities that equally develop students’ 21st cen-
tury skills. To address this issue, teachers should
be trained in designing and implementing online
learning activities that engage and expose students
to 21st century skills while learning (Shulamit &
Yossi, 2011). However, making teaching and learn-
ing fully online decreases the effectiveness of the
educational process. Aside from exposing students
to modern-day skills, the choice of blended learn-
ing is an appropriate solution to problems of time
constraints and unavailability of physical space at
campuses and schools. However, online teaching
without face-to-face learning neglects the objec-
tive of actual learning, which includes emotion,
behavior, body language, and sense or feeling.
Therefore, face-to-face and online learning com-
plement each other (Ellis et al., 2008; Nathan &
Rajamanoharane, 2016).
CONCLUSION
Blended learning was implemented in the
experiment class to investigate its effect on stu-
dents’ course performance and their practice of
21st century skills, specifically in a Research
Methodology course, by diving the class into
conventional and blended learning classes. This
implementation yielded a significant improvement
in students’ performance in the blended learning
class, and their practice of 21st century skills, when
compared to the conventional class. Therefore,
this study is expected to contribute to the scope

JOURNAL OF EDUCATORS ONLINE

 References applying blended cooperative e-learning (BCEL) in teaching
and learning process. Advances in Social Science, Education
Akyıldız, S. T. & Çelik, V. (2020). Thinking outside the box: and Humanities Research, 253, 169–173. https://doi.
Turkish EFL teachers’ perceptions of creativity. Thinking org/10.2991/aes-18.2019.40
Skills and Creativity, 36, 100649. https://doi.org/10.1016/j. Hadiyanto, H. (2019b). Constructing a prototype of developing
tsc.2020.100649 students’ 21st century skills: Soft skills, hard skills and
Bialik, M., Bogan, M., Fadel C., & Horyathova, M. (2015). Character competitiveness at a university in Indonesia. In Proceedings
education for the 21st century: What should students learn? of the 5th International Conference on Education in Muslim
Center For Curriculum Redesign. http://curriculumredesign. Society, ICEMS 2019, 30 September–01 October 2019,
Jakarta, Indonesia. EAI. https://doi.org/10.4108/eai.30-9-
org/wp-content/uploads/CCR-CharacterEducation_
2019.2291171
FINAL_27Feb2015.pdf
Hadiyanto, H. (2019c) The EFL students’ 21st century skill
Broadbent, J. (2017). Comparing online and blended learner’s
practices through e-learning activities. Indonesian Research
self-regulated learning strategies and academic performance.
Journal in Education [IRJE], 3(2), 461–473. https://doi.
Internet and Higher Education, 33, 24–32. https://doi.
org/10.22437/irje.v3i2.8036
org/10.1016/j.iheduc.2017.01.004
Hadiyanto, H., Failasofah, F., Armiwati, A., Abrar, M., &
Bryan A., & Volchenkova, K. N. (2016). Blended learning:
Thabran, Y. (2021) Students’ practices of 21st century
Definition, models, implications for higher education.
skills between conventional learning and blended
Bulletin of the South Ural State University. Ser. Education.
learning. Journal of University Teaching & Learning
Educational Sciences. 8(2), 24–30. https://doi.org/10.14529/
Practice, 18(3), 2021. https://doi.org/10.53761/1.18.3.7
ped160204
Hadiyanto, H., Syahrial, Fajaryani., N., Masbirorotni, & Juwita.,
Cohen, J. (1960). A coefficient of agreement for nominal scales.
M. (2018). Constructing the measurement of EFL students’
Educational and Psychological Measurement, 20(1), 37–46.
core competencies practices in learning activities. TOJET:
https://doi.org/10.1177/001316446002000104
The Turkish Online Journal of Educational Technology, 17(3),
Delialioğlu, Ö. (2012). ‘Student engagement in blended learning 16–25. http://www.tojet.net/articles/v17i3/1732.pdf
environments with lecture-based and problem-based Hair, J. F., Black, B., Babin, B., Anderson., R. E., & Tatham, R. L.
instructional approaches’, Educational Technology & Society, (2006). Multivariate data analysis (6th ed.). Pearson Prentice
Vol. 15, pp. 111-228. http://citeseerx.ist.psu.edu/viewdoc/down Hall.
load?doi=10.1.1.298.8303&rep=rep1&type=pdf
ILO. 2014. “Survey of ASEAN Employers on Skills and
Dunne, E., Bennett, N., & Carré, C. (2000). Skill development Competitiveness. Emerging Markets Consulting,” http://www.
in higher education and employment. In F. Coffield (Ed.). ilo.org/wcmsp5/groups/public/---asia/---ro-bangkok/---sro-
Differing visions of learning society (pp. 105–138). Policy bangkok/documents/publication/wcms_249982.pdf
Press. https://doi.org/10.2307/j.ctt1t891gf.7
Kara, S. (2016). How and why? Edmodo as a blended learning
Ellis, R. A., Goodyear, P., Calvo, R. A., & Prosser, M. (2008) tool: A brief overview of usage and research. In Proceedings
Engineering Students’ conceptions of and approaches of ISERD International Conference, Bali, Indonesia, 9th May
to learning through discussion in face-to-face and online (pp. 10–12). https://www.worldresearchlibrary.org/up_proc/
contexts. Learning and Instruction, 18(3), 267–282. https://doi. pdf/300-146503126810-12.pdf
org/10.1016/j.learninstruc.2007.06.001 Khan, A. I., Noor-ul-Qayyum, Shaik, S., Ali, A. M., & Bebi, Ch.
Fischer, E., & Hänze, M. (2019). Back from “guide on the side” to V. (2012) Study of Blended Learning Process in Education
“sage on the stage”? Effects of teacher-guided and student- Context. International Journal of Modern Education and
activating teaching methods on student learning in higher Computer Science, 4(9), 23–29. https://doi.org/10.5815/
education. International Journal of Educational Research, 95, ijmecs.2012.09.03
26–35. https://doi.org/10.1016/j.ijer.2019.03.001 Lin., M., Preston, A., Kharrufa, A. Z., Huang, K., (2016). Making L2
Glowa, L., & Goodell, J. (2016) Student-centered learning: learners’ reasoning skills visible: The potential of Computer
functional requirements for integrated systems to optimize Supported Collaborative Learning Environments. Thinking
learning. International Association for K–12 Online Learning Skills and Creativity, 22, 303–322. https://doi.org/10.1016/j.
(iNACOL). https://aurora-institute.org/wp-content/uploads/ tsc.2016.06.004
iNACOL_FunctionalRequirementsForIntegratedSystems.pdf Lippman, L. H., Ryberg, R., Carney, R., & Moore, K. A. (2015).
Hadiyanto, H. (2019a). Enhancing students’ core competencies by Workforce connections: Key “soft skills” that foster youth

JOURNAL OF EDUCATORS ONLINE

 workforce success: Toward a consensus across fields. Singh, A., & Singh, L. B. (2017). E-learning for employability skills:
Child Trends. https://www.usaid.gov/sites/default/files/ Students perspective. Procedia Computer Science, 122,
documents/1865/KeySoftSkills.pdf 400–406. https://doi.org/10.1016/j.procs.2017.11.386
Ma, J., Li, C., & H-Ning, L. (2019). Enhancing students’ Blended Solangi, Z. A., Al Shahrani, F., & Pandhiani, S. M. (2018). Factors
Learning Experience through Embedding Metaliteracy. affecting successful implementation of elearning: Study of
Education Research International, 2019, 6791058. https://doi. colleges and institutes sector RCJ Saudi Arabia. International
org/10.1155/2019/6791058 Journal of Emerging Technologies in Learning (iJET) 13(6),
Marando, A. (2012). Balancing project management hard 223–230. https://doi.org/10.3991/ijet.v13i06.8537
skill and soft skills. Master of Science in Management Washer, P. (2007). Revisiting key skills: A practical framework for
of Projects and Programs Rabb School of Continuing higher education. Journal of Quality in Higher Education,
Studies Division of Graduate Professional Studies 13(1), 57–67. https://doi.org/10.1080/13538320701272755
Brandeis University. https://www.guanaitong.com/ Wilson-Ahlstrom, A., Yohalem, N., David, Ji, P., Hillaker, P., &
uploadfile/2018/0904/201809041536039118.pdf Weikart, D. P. (2014). From soft skills to hard data: Measuring
Nathan, S. K., & Rajamanoharane, Sw. (2016). Enhancement of youth program outcomes. Forum for Youth Investment. https://
skills through e-learning: Prospects and problems. The Online forumfyi.org/knowledge-center/from-soft-skills-to-hard-data/
Journal of Distance Education and E-learning, 4(3), 24–32. Witherspoon, A. (2011). Edmodo . . . A learning management
Ontario Public Services. (2016). Towards defining 21st century system. Powered with Technology. https://www.
competencies for Ontario. Winter 2016 Edition. Queen’s poweredwithtechnology.com/2011/01/edmodoa-learning-
Printer for Ontario http://www.edugains.ca/resources21CL/Ab management-system.html
out21stCentury/21CL_21stCenturyCompetencies.pdf Wuryaningsih, W., Susilastuti, D. H., Darwin, M., & Pierewan, A.
Pallant, J. 2011. A Step by Step Guide to Data Analysis Using C. (2019), Effects of web-based learning and F2F learning
SPSS Program. Survival Manual. 4th Edition. China, Everbest on teachers achievement in teacher training program in
Printing. Indonesia. International Journal of Emerging Technologies in
Partnership for 21st Century Skills, 21st Century Skills (2008). Learning (iJET), 14(21), 143–147. https://doi.org/10.3991/ijet.
Education and competitiveness: A resources and policy v14i21.10736
guide,” [Online]. https://files.eric.ed.gov/fulltext/ED519337.pdf Zalizan, M. J., & Azman, N. (2005). Generic skills provision
Person, Ann, E. Moiduddin, M. Hague-Angus, M. and Malone, in higher education: A Malaysian perspective. The
L.M. 2009. “Survey of Outcomes Measurement in International Journal of Learning, 12(5), 200–210. https://doi.
Research on Character Education Programs (NCEE 2009- org/10.18848/1447-9494/CGP/v12i05/47746
006),” Washington, DC: National Center for Education
Evaluation and Regional Assistance, Institute of Education
Sciences, U.S. Department of Education. https://www.
semanticscholar.org/paper/Survey-of-Outcomes-
Measurement-in-Research-on-NCEE-Person-Moiduddin/
ff913679cf8be6bff3bc19cea9138d9b1ec3ed8f
Ristekdikti (2015). Standar nasional pendidikan tinggi “direktorat
penjaminan mutu,” [Online]. http://lldikti12.ristekdikti.go.id/
wp-content/uploads/2019/06/2.-Standar-Nasional-Pendidikan-
Tinggi-SN-Dikti.pdf
Schober, B., Wagner, P., Ralph, R., & Spiel, C. (2008). Vienna
e-Lecturing (VEL): Learning how to learn self-regulated in an
internet-based blended learning setting. International Journal
on E-learning, 7(4), 703–723. https://www.learntechlib.org/
primary/p/24292/
Shulamit, K., & Yossi, E. (2011). Development of e-learning
environments combining learning skills and science and
technology content for junior high school. Procedia Social and
Behavioral Sciences, 11, 175–179. https://doi.org/10.1016/j.
sbspro.2011.01.056

JOURNAL OF EDUCATORS ONLINE