See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/345428416

An Empirical Study on Vocabulary Recall and Learner Autonomy through

Mobile?Assisted Language Learning in Blended Learning Settings

Article in CALICO Journal · October 2020

DOI: 10.1558/cj.40436

CITATIONS READS

28 517

3 authors, including:

Takeshi Sato
Kobe University
41 PUBLICATIONS 271 CITATIONS

SEE PROFILE

All content following this page was uploaded by Takeshi Sato on 01 July 2023.

The user has requested enhancement of the downloaded file.

 calico journal (online) issn 2056–9017

Article

An Empirical Study on Vocabulary

Recall and Learner Autonomy through
Mobile-Assisted Language Learning in
Blended Learning Settings
Takeshi Sato1, Fumiko Murase2, and Tyler Burden3

Abstract
This study aims to examine the efficacy of Mobile-Assisted Language Learning
(MALL) of English as a foreign or second language (L2) through two perspec-
tives: learning gain and learner autonomy. Previous studies have shown that L2
learning combined with media could activate the learning processes, resulting
in an easier recall of the target vocabulary required in L2. In addition, mobile-
assisted L2 learning could also enhance autonomous learning inasmuch as suc-
cessful MALL would have to rely mainly on the autonomous learner even in
learning contexts where the goal and task are already fixed. Based on this stand-
point, the study hypothesizes that the engagement in L2 learning with mobile
devices along with a classroom-based writing course could make L2 learners
not only achieve the target L2 lexis effectively, leading to better L2 writing per-
formance, but also help them to be more autonomous even in a setting when
the task and goal are fixed. To test this hypothesis, both empirical and ques-
tionnaire studies were conducted for Japanese undergraduates (n=94). Based
on the results of three weeks of L2 academic writing practice between groups
learning with and without mobile devices, the findings of our t-test analyses of

Affiliations
1
Tokyo University of Agriculture and Technology, Japan.
email: tsato@cc.tuat.ac.jp

2
Ryukoku University, Japan.
email: fmurase@econ.ryukoku.ac.jp

3
Meisei University, Japan.
email: burden.tyler@meisei-u.ac.jp

calico journal vol 37.3 2020 254–276 https://doi.org/10.1558/cj.40436

©2020, equinox publishing
 Takeshi Sato, Fumiko Murase, and Tyler Burden 255

learners’ vocabulary recall and a questionnaire survey about learner autonomy

suggested that MALL significantly contributed not only to L2 vocabulary recall
in comprehensive and productive tests, but also to enhancing positive attitudes
towards autonomous learning.
Keywords:mobile-assisted language learning; vocabulary recall;
academic writing; learner autonomy; motivation; blended
learning.

1. Introduction
1.1 Advantages of MALL
Along with the rapid spread of mobile devices, the advantages of Mobile-
Assisted Language Learning (MALL) have been advocated. Many of the studies
claim that the MALL advantage lies in multimodal interfaces of such media as
pictures, animation, and sound (Sato & Suzuki, 2010; Sato, 2016; Yeh & Wang,
2003). Sato, Matsunuma and Suzuki (2013), for example, demonstrate that
prompt feedback of a mobile learning application can enhance the automa-
tization of vocabulary recall, allowing the vocabulary to be reapplied toward
reading activities and thus leading to successful L2 reading comprehension.
Mobility is also defined as one of the MALL advantages. As Laurillard (2007)
claims, the mobility of digital technologies provides learners a wide range of
choices of what, when, and how to learn; MALL offers different ways of learn-
ing from that in a classroom.
Several recent studies support the benefits of MALL, especially on L2 vocab-
ulary learning and the positive learning effects (Burston, 2015; Çakmak &
Erçetin, 2018; Chen, Liu, & Huang, 2019; Loewen et al., 2019; Rosell-Aguilar,
2018).
The successful L2 learning that these previous MALL studies demonstrated,
however, might have failed to consider what is happening to learners when
engaging in MALL. Therefore, the concept of the agency (Pachler, Bachmair, &
Cook, 2010) or autonomy (Holec, 1981) of learners seems to be vital in making
MALL a more positive and meaningful experience for learners. The concept of
learner autonomy is defined by Holec (1981, p. 3) as “the ability to take charge
of one’s own learning,” where the learners are expected to take responsibility
for decisions when determining the objectives of learning, evaluating what
has been acquired, and so on.
Learning with technology and learner autonomy are principally compatible
because technology affords the opportunity to carry out such processes (Dang,
2012; Lee, 2016; Reinders, 2011; Reinders & Hubbard, 2013; Ushioda, 2013)
because of the nature of mobile learning, such as prompt feedback outside the
256 Vocabulary Recall and Learner Autonomy

classroom. Therefore, the more learner autonomy is fostered through the use
of technology, the more actions and choices learners can take on their own
(Schwienhorst, 2003).
Using learners’ own devices also fosters learner autonomy (Kukulska-
Hulme, 2015). Choosing the contents and strategies for learning and actively
searching for resources can be seen as essential qualities of autonomous
learners (Benson, 2001). In that respect, MALL may require learners to be
autonomous while, at the same time, it may offer learners the opportunity
to autonomously engage in L2 learning (Kukulska-Hulme, 2015; Reinders &
White, 2011). Through such characteristics, MALL can encourage high-quality
involvement in learning (Ushioda, 2013) and higher learning gains.

1.2 Learner Autonomy in Blended Learning Settings

Despite the advantages of MALL shown above, autonomous learning with
mobile devices is rarely, if ever, combined with classroom-oriented education,
since classroom activities are conducted and controlled by instructors. They
tend not to give learners much responsibility for learning but ask the learners
to complete only the assigned tasks or exercises. This may lead to few trials of
implementing mobile technology in blended learning settings in which formal
and informal learning are integrated, especially to enhance learner autonomy
(Reinders & White, 2016).
However, along with the popularity of Bring Your Own Device (BYOD)
in recent years, MALL has been incorporated within blended learning set-
tings; learners are allowed to bring and use their mobile devices to efficiently
support activities in their language classrooms. As such, the shift facilitates a
combination of classroom-oriented learning with out-of-class learning using
mobile devices, and different types of autonomy could be fostered, as shown
from previous studies. Lyddon (2016), for example, describes the characteristics
of autonomous learners1 who are required to use mobile devices in classroom-
based compulsory learning contexts and demonstrates that the least autono-
mous learners do not participate in a classroom activity, whereas the more
autonomous ones try to find value from the activity and then strive to complete
it. Lyddon’s argument is that learners enhance their autonomy in the process
of blended learning combining classroom-based learning and MALL, even if
the task is assigned by teachers and their learning goal is also fixed.
Considering the increasing trend of BYOD in formal language learning, our
study explores the impact of MALL in blended learning settings in terms of
learner autonomy as well as learning effect. In this study we focus on mobile-
assisted vocabulary learning along with classroom-based writing activities.
This is primarily because previous MALL studies have not addressed writing
 Takeshi Sato, Fumiko Murase, and Tyler Burden 257

skills (Burston, 2015) and therefore have not investigated how MALL can help
to recall the target vocabulary in L2 writing.

1.3 Research Questions

Research on learner autonomy has faced several challenges, such as measuring
learner autonomy or the effectiveness of using technology in fostering learner
autonomy (Reinders, 2011; Reinders & White, 2011). This is due to the difficulty
of defining learner autonomy, so that different researchers have different views
of learner autonomy (Dang, 2012). Oxford (2003) defined learner autonomy
along four dimensions: psychological, technical, socio-cultural, and political-
critical, but no study was found to measure each dimension of autonomy in
an empirical manner.
This study, therefore, examines the effectiveness of MALL, based on our
claim that successful mobile-assisted L2 vocabulary learning enhances not
only L2 learning gains but also learner autonomy in blended learning settings.
Based on this claim, this study compared L2 learners who used mobile devices
with those who used paper-based lists of expressions for academic writing, fol-
lowed by research to measure improvements in their learner autonomy with a
questionnaire developed to measure learner autonomy (Murase, 2015).
Our focus on writing to examine learner autonomy follows the claim of Col-
lentine (2011) that writing as learners’ output reflects their linguistic awareness
derived from their autonomous learning. While we developed mobile-based
materials to help students learn the expressions required for writing academic
essays of several paragraphs, empirical research was conducted to examine the
following three research questions (RQs):

1. If, during a given period, L2 learners study expressions for academic

essays with a mobile-based application, would they recall more expres-
sions on the written test than those studying with a paper-based list did?
2. If L2 learners study these expressions with the application, would they
use more of those expressions in writing an essay than those with the
paper list?
3. If L2 learners study these expressions with the application, how would
their learner autonomy and attitudes towards MALL change, compared
with those with the paper list?
258 Vocabulary Recall and Learner Autonomy

2. Method
2.1 Participants
A total of 94 (80 male and 14 female) undergraduate students enrolled in a
compulsory English writing course participated in this research, most of whom
were sophomores from the faculty of engineering in the Japanese university
where the authors of this paper taught English as a foreign language. The
students’ majors, which included the life sciences, chemical sciences, physics,
and electrical engineering, are not related to English studies; yet their Eng-
lish language skills were sufficient for composing several English sentences
by themselves due to the fact that they had studied English for at least seven
years and passed the entrance examination which included an essay writing
component. Given all of the participants had a score approaching “mastery”
level in G-TELP Level 2 (two out of three skills exceed 75%), which corresponds
to a score of between 600–800 in the TOEIC test according to the official page
of G-TELP (n.d.), their English proficiencies were assumed to be approximately
at an intermediate level.
Participants were divided into two groups, a control group (n = 54) and
an experimental group (n = 40). They enrolled in three different classes and
were taught by two different instructors. One instructor taught one class of
the experimental group and one class of the control group, while the other
instructor taught only one class, which was divided into an experimental and
a control group. As the groups were divided according to their English writing
classes within their respective departments, the English language skill levels
in each group were expected to be equivalent, although no test was conducted
to corroborate this assumption.

2.2 Target Expressions

All the participants were asked to learn 100 expressions frequently used in aca-
demic writing (see Appendix A for the list of expressions). These were extracted
from several textbooks and reference books for L2 learners (Steinberg, 2008)
and consisted mainly of words or phrases used to clarify the logical flow of an
essay. To confirm the difficulty level of the expressions for the participants, we
conducted a paper-based questionnaire survey before the research; the partici-
pants were asked to answer on a four-point Likert scale the degree to which
they had already known each expression ( “know the expression well,” “know
it roughly,” “don’t know it well,” and “don’t know it at all”). According to the
survey results, we developed recall tests to ask the participants the expressions
they didn’t know well or at all before the treatment.
 Takeshi Sato, Fumiko Murase, and Tyler Burden 259

2.3 Treatment
Considering Kukulska-Hulme’s (2015) claim of the importance of MALL in
offering supplementary tasks to extend classroom-oriented learning, all par-
ticipants were asked to learn the expressions as their preparation for the end-
of-term writing test. Those in the control group were asked to memorize the
expressions with their corresponding Japanese translations from a paper-based
expression list (see Figure 1). The participants were supposed to memorize the
expressions and translations outside the classroom.

Figure 1. A screenshot of the paper-based phrase list.

Students in the experimental group, on the other hand, were asked to learn
the expressions on their smartphones. For that purpose, learning materials
were developed using Quizlet (https://quizlet.com/), a free online learning tool,
available on mobile devices such as iPhone and Android phones at the time of
the present study, that is used to generate vocabulary learning resources. As
shown in Figures 2 and 3, the online resource provides different kinds of quiz-
zes for the expressions, such as matching expressions with their translations.
These quizzes were available to any students who had Internet access on their
smartphones. After being provided with instructions on installing, registering,
and using the resource on their own mobile devices, the experimental group
was asked to learn the expressions outside the classroom. However, it was found
that some of the participants in the experimental group didn’t use Quizlet but
used the paper list to learn the expressions, so they were categorized as part
of the control group.
In both groups, the instructors did not instruct the students of the group
how to use their learning tools. The instructors only introduced “flashcard” as
the most popular exercise on Quizlet. This was because the present study aimed
to afford students the opportunity to use each learning tool in their own ways,
which is vital in fostering learner autonomy, as discussed above. To encourage
out-of-class learning in each treatment, however, the instructors announced
260 Vocabulary Recall and Learner Autonomy

that the test for the expressions would be held three weeks later, and the scores
would count as part of their grades in the writing class.

Figure 2. An example of the quizzes developed by Quizlet (matching).

Figure 3. Another example of the quizzes developed by Quizlet (fill-in-the-blanks).

 Takeshi Sato, Fumiko Murase, and Tyler Burden 261

2.4 Learner Autonomy Questionnaire Survey

Just after the introduction of the learning materials, all participants were asked
to answer an Internet-based questionnaire written in Japanese about their
attitudes toward and views of learning English, which was designed to meas-
ure the technical and psychological dimensions of learner autonomy (Murase,
2015). Students accessed the designated website developed by Google Forms
and then answered the questionnaire outside the classroom via their mobile
devices or PCs (see Appendix B for the English translation of the questionnaire
items). The same questionnaire was administered after the end-of-semester
writing test.
The questionnaire used in this study was designed to measure two dimen-
sions of learner autonomy, technical and psychological (Benson, 1997; Dang,
2012; Oxford, 2003; Pennycook, 1997), which consist of a total of 49 items
on a five-point Likert scale (strongly agree, agree, neither agree nor disagree,
disagree, strongly disagree). A response of “strongly agree” obtains the high-
est score (scored 5) and “strongly disagree” the lowest (scored 1); high scores
indicate high learner autonomy. To briefly summarize, the technical dimension
of autonomy refers to the learners’ act of learning a language on their own
outside the classroom without the aid of a teacher, and also to situations in
which learners are obliged to take control of their learning for some reason,
while the psychological dimension refers to the “capacity” of individual learn-
ers that “allows learners to take more responsibility for their own learning”
(Benson, 1997).
This questionnaire survey was developed based on extensive reviews of
existing definitions of learner autonomy in the literature. Furthermore, the
test for the internal consistency of each dimension showed a statistically rea-
sonable level of reliability (α = .936 for all the questionnaire items), while
the validity of the questionnaire was investigated by a series of confirmatory
factor analyses using structuring equation modeling and the Goodness-of-Fit
statistics showed an acceptable level of validity (Murase, 2015). Therefore, the
questionnaire can be seen as one of the most valid and reliable tools available
for measuring learner autonomy.

2.5 Procedure
Three weeks after the introduction of the materials and the questionnaire
survey on learner autonomy, a test of the expressions and an essay writing
task were conducted during a total period of 90 minutes. During the first ten
minutes of the class period, the participants were asked to answer 20 fill-in-
the-blank questions created from the 100 expressions. The expressions in the
questions were selected based on the results of the questionnaire carried out
262 Vocabulary Recall and Learner Autonomy

before the introduction of the materials, and consisted of the expressions that
participants had the least prior knowledge of; 80% of the participants had
answered “I don’t know it well” or “I don’t know it at all” for these items (in
bold in Appendix A). The test was graded according to the number of correct
answers (writing an appropriate word with correct spelling in each blank), so
the total possible score for the test was 20.
A timed essay writing task was then given. The participants were asked to
pick one of the following four topics given by the instructors:

• Science college students should learn English.

• Japanese universities should change to meet the needs of globalization.
• The Tokyo Olympics should be held as planned.
• The voting age should be lowered from 20 to 18.

They were then asked to write an essay of at least three paragraphs present-
ing their opinions about the topic they had chosen and to include as many
expressions they had studied as possible. As these topics were given on the
spot, the participants were not able to prepare beforehand. Although they were
not allowed to refer to any dictionaries, several keywords related to the essay
topics were given by the instructors. They were given 75 minutes for this task.
The essays were analyzed to determine the quality of writing. For this pur-
pose, all the essays were graded by one of the authors whose native language is
English, according to the IELTS band descriptors for the writing sections. In
the IELTS test, essays (in Writing Test 1) are graded on a band scale ranging
from one to nine, referring to the four criterion areas: task achievement, coher-
ence and cohesion, lexical resource, and grammatical range and accuracy. This
analysis was conducted to examine whether the quality of the essays changed
between the groups based on our presupposition that the quality of the essays
both groups wrote would not differ except for the use of the expressions they
learned in different ways.
After finishing the writing task, they were asked to answer the Internet-
based questionnaire again within a few days. In addition to the same 49 items
on learner autonomy as with the pre-questionnaire, a section was added asking
about their vocabulary learning experiences in terms of the frequency of the
students’ learning, the place of learning, and their motivation towards learn-
ing (see Appendix C for questions in the additional section). In the four-point
Likert scale questionnaire on frequency and motivation, more positive state-
ments (e.g., scored 4 for “I studied almost every day”) indicate a more positive
attitude towards learning.
 Takeshi Sato, Fumiko Murase, and Tyler Burden 263

3. Findings
All the data collected in this research were analyzed to investigate the dif-
ferences between the control and experimental groups, as well as changes in
learner autonomy within the groups.

3.1 Fill-In-The-Blank
First, in order to answer the first RQ, the scores on the fill-in-the-blank test
(total score: 20) were compared between the two groups, as shown in Table 1.
In the control group (n = 54), the mean score on the test was 6.48 (SD = 5.79,
max = 18, min = 0), whereas the mean score of the experimental group (n =
40) was 10.01 (SD = 6.50, max = 20, min = 0). A t-test showed significant differ-
ences between the groups with respect to the mean score (t(92) = 2.82, p <.05,
d = .58). This result demonstrates that the participants who studied the target
expressions with their mobile devices recalled significantly more expressions
than those who used the paper list.

Table 1
T-test Results of the Fill-In-The-Blank Test

n M SD df t Sig. d
Control 54 6.48 5.79 92 2.82 .006** .58
Experimental 40 10.01 6.5

*p < .05 ** p <.01

3.2 Essay Writing Test

To answer the second RQ, we analyzed the essays written by the participants
(n = 94) and compared them between the two groups. First, the number of
expressions appearing in each essay was counted and compared between the
groups, as shown in Table 2. In the control group (n = 54), the mean number of
the expressions used in an essay was 1.48 (SD = 1.71, max = 5, min = 0), while
in the experimental group (n = 40) it was 2.60 (SD = 1.70, max = 8, min = 0).
A t-test found a significant difference between the groups (t(92) = 3.15, p < .05,
d = .66). This result shows that the participants who studied with Quizlet used
more expressions in their essays than those who studied with the paper list.
264 Vocabulary Recall and Learner Autonomy

Table 2
T-test Result of the Number of Expressions Used in the Learners’ Essays

n M SD df t Sig. d

Control 54 1.48 1.71 92 3.15 .002** .66

Experimental 40 2.6 1.7

*p < .05 **p < .01

To verify our presupposition that the different treatments affect only the
expressions, but other qualities of the essays would not be different, the scores
on each criterion were compared between the two groups (see Table 3). Since
no statistically significant difference was observed between the two groups,
the qualities of the essays were not affected by the different treatments except
for the expressions they could recall.

Table 3
T-test Results of the Scores on Learners’ Essays based on the IELTS Criteria

M (SD)

Control (n = 54) Experimental (n = 40) df t Sig.

Task achievement 5.85 (0.76) 6.05 (0.81) 92 1.21 0.23

Cohesion and coherence 5.82 (0.55) 5.96 (0.62) 92 1.32 0.19
Lexical resources 5.85 (0.56) 6.05 (0.64) 92 1.59 0.11
Grammar accuracy 5.56 (0.54) 5.7 (0.56) 92 1.26 0.21

3.3 Questionnaire on Learner Autonomy

In order to answer the third RQ, the results of the Internet-based question-
naire about learner autonomy, which was administered before the treatment
(pretest) and after the treatment (posttest), were compared between groups
and also within groups over time.

3.3.1 Comparison Between Groups

In the pretest, 66 out of the 94 participants of this study responded to all the
questionnaire items and were therefore valid. In contrast, in the posttest, 76
participants answered all the questionnaire items and were valid.
When comparing the two groups on the 49 individual items, in the pretest,
a t-test found no significant difference between the control group (n = 33)
and the experimental group (n = 33). In the posttest, there was no significant
 Takeshi Sato, Fumiko Murase, and Tyler Burden 265

difference between the control group (n = 46) and the experimental group (n
= 30), except for two items (Q17 and Q21), for which the control group had
higher scores. For Q17 (“I take notes about how much time I spent on my
English study”), the mean score of the control group was 2.24 (SD = 1.78),
while the mean score of the experimental group was 1.77 (SD = .82). A t-test
found a significant difference between groups (t(74) = 2.06, p < .05). On Q21
(“I take notes of my feelings while I am studying English”), the mean scores
of the control and experimental groups were 1.76 (SD = 1.04) and 1.33 (SD =
.66), for which a t-test found a significant difference between groups (t(74) =
2.12, p < .05). As both items are concerned with taking notes while learning,
it may be assumed that it was easier for those working with the paper-based
list to physically take notes.
As described earlier, the 49 questionnaire items were originally designed to
measure two different dimensions of learner autonomy: the technical (Q1‒21)
and psychological (Q22‒49) dimensions. Thus, the results were also compared
between groups on each of the two dimensions. In the technical dimension,
the mean scores of the control and experimental groups were 2.60 (SD = .57)
and 2.38 (SD = .59), respectively. The t-test found no significant difference
between groups (t(74) = 1.67, p > .05). As for the psychological dimension, the
mean scores of the control and experimental groups were 3.34 (SD = .51) and
3.40 (SD = .40, respectively). A t-test showed no significant difference between
groups (t(74) = .491, p > .05).

3.3.2 Comparison Within Groups

In order to examine possible changes in learner autonomy over time, the scores
on the two dimensions in the pretest and the posttest were compared within
groups. For this part of the analysis, the data of 51 out of the 94 participants
who responded to both the pretest and posttest completely were analyzed:
32 participants in the control group and 19 participants in the experimental
group.
For the control group, there was no significant difference between the two
tests on either dimension (see Table 4). For the experimental group (see Table
5), there was no significant difference observed between the two tests on the
technical dimension (t(18) = .578, p > .05). However, there was a significant
difference between the two tests on the psychological dimension (t(18) = 2.36,
p < .05, d = .46).
Thus, when comparing the results of the pretest and the posttest, both
groups obtained higher scores (suggesting a higher autonomy) on the post-
test. However, there was no significant difference between the scores on the
two tests except for the experimental group, in terms of the psychological
dimension of autonomy.
266 Vocabulary Recall and Learner Autonomy

Table 4
T-test Results of Comparing Two Tests on Two Dimensions (Control Group, n = 32)

M SD df t Sig.
Technical
Pretest 2.40 .454 31 1.401 .171
Posttest 2.52 .477
Psychological
Pretest 3.16 .431 31 1.733 .093
Posttest 3.30 .466

Table 5
T-test Results of Comparing Two Tests on Two Dimensions (Experimental Group, n =
19)

M SD df t Sig. d
Technical
Pretest 2.41 .682 18 .578 .570 .05
Posttest 2.44 .567
Psychological
Pretest 3.25 .373 18 2.356 .030* .46
Posttest 3.42 .368

*p < .05

3.4 Vocabulary Learning Experiences

An additional section was added to the posttest questionnaire with questions
about vocabulary learning experiences (see Appendix C), and the results were
compared between groups in terms of frequency, place, and motivation. The
data collected from 76 participants, who answered all the posttest questions,
were analyzed.

3.4.1 Frequency of Learning

In order to examine the frequency of learning during the three weeks, the par-
ticipants’ responses to Q1 (“During the last three weeks, how often (on average)
did you study the expressions?”) were analyzed. A t-test showed that the mean
 Takeshi Sato, Fumiko Murase, and Tyler Burden 267

scores of the control and experimental groups were 1.78 (SD = .51) and 1.87
(SD = .51), respectively, and no significant difference was found between the
control group (n = 46) and the experimental group (n = 30) in the frequency
of their learning (t(74) = .702, p > .05).

3.4.2 Place of Learning

As for where the participants studied, their responses to the question “Where
did you mainly study?” were analyzed. As shown in Table 6, while 26% of the
students in the control group reported they studied at university, no students
in the experimental group studied at university. The results also showed that
there were more students in the experimental group who studied on the train
or bus than the control group. The results indicate that learning with mobile
devices could facilitate ubiquitous learning and help to combine formal learn-
ing with learning outside university, leading to blended learning, while paper-
based learning seems to be related more to formal learning settings such as
home and university.

Table 6
Responses about the Place of Learning (n = 76)

At home At university On the train or bus Other

Control 27 12 7 0
(n = 46) (59%) (26%) (15%)
Experimental (n = 30) 13 0 14 3*
(43.3%) (46.7%) (10%)

Note. Other responses included “In my free time” and “At a family restaurant.”

3.4.3 Motivation Towards Learning

Finally, to examine the motivation for vocabulary learning, students’ responses
to the question “By using a paper list (or mobile devices), did you feel moti-
vated towards learning essay phrases?” were analyzed. A t-test showed that
there was a significant difference concerning their motivation towards vocabu-
lary learning (t(74) = 2.01, p < .05, d = 0.47). This indicates that students in
the experimental group (M = 2.93, SD = .64) felt higher motivation towards
vocabulary learning on mobile devices than those who used the traditional
paper-based list (M = 2.63, SD = .65). This correlates to the improvement of
learner autonomy in the psychological dimension after three-weeks of mobile
learning, as described in 3.3.2.
268 Vocabulary Recall and Learner Autonomy

Table 7
Responses about the Motivation Towards Learning

n M SD df t Sig. d
Control 46 2.63 0.645 74 2.01 .048* .47
Experimental 30 2.93 0.64

*p < .05

4. Discussion
This section will discuss the three RQs addressed in this study, followed by
the limitations of the study.
The answer to our first RQ is yes. The analysis of the fill-in-the-blank test
data showed that those who used mobile devices could recall more expressions
than those who studied with the paper-based list. The better vocabulary recall
obtained here underpins the previous studies of mobile-assisted L2 vocabulary
learning (i.e. Burston, 2015). As for the second RQ, our answer is also yes. The
analysis of the essays showed that those who studied with Quizlet were able to
use more expressions when writing an essay than those studied with the paper
list. This seems reasonable, as those who studied with the application could
recall more expressions than those who studied with the paper list. Accord-
ing to the analysis of the essays based on the IELTS criteria, it was shown that
there was no significant difference between the groups in terms of the quality
of the writing, indicating that students in both groups had the same or similar
levels of writing and that the differences in recalling and using the expressions
were purely the results of the treatment. In sum, learning expressions with a
mobile application enhanced not only recall but also the participants’ ability
to apply the vocabulary to writing activities as well as reading activities (Sato
et al., 2013).
As for the third RQ. The results of the two questionnaire surveys indicated
different tendencies between the two groups. According to the pretest, there
was no significant difference between the groups, indicating that both groups
were homogeneous in terms of learner autonomy at the beginning of this study.
Comparing the pretest and the posttest, among those who learned the expres-
sions with the paper list, there was no significant difference between the two
tests, while there was a significant difference between the tests regarding the
psychological dimension of learner autonomy among those who learned on
the mobile devices. In addition, according to the additional part of the posttest
questionnaire, it was shown that the experimental group felt greater motiva-
tion towards learning the expressions than the control group. Considering
 Takeshi Sato, Fumiko Murase, and Tyler Burden 269

motivation and learner autonomy are closely related to each other and moti-
vation strengthens autonomy (Dörnyei, 2001), the significant improvement of
motivation through MALL to support classroom-based learning could trigger
an enhancement of learner autonomy.
The places where the devices were used are also suggestive. While paper-
based learning is connected with formal learning contexts, mobile-based learn-
ing seems to create new learning environments which traditional learning does
not offer. The shift of learning environment enhanced learners’ motivation
and the psychological dimension of autonomy, leading to successful blended
L2 learning.
This study was not free from limitations. One major difficulty in this study
was the grouping of the participants. After the posttest questionnaire which
asked whether the participants used the paper list or Quizlet, it turned out
that eleven students in the experimental group used the paper list. Therefore,
to reflect their actual learning experience, they had to be labeled as the control
group when analyzing the data.
What seemed to underlie this was the participants’ initial hesitancy or
resistance to the use of mobile devices. It can be assumed that those who were
in the experimental group but did not like to learn on mobile devices chose to
learn with the paper list, possibly obtaining the list from their classmates in the
control group. Although the treatment is a core part of the study that should
have been carefully controlled, this incident impressed on us the realization
that there might be some students who viewed using mobile devices for learn-
ing negatively. More careful division into the groups should be conducted in
any future study.
Furthermore, the difference in the number of valid participants in each
questionnaire analysis was a limitation. This was because answering 49 ques-
tions proved so time-consuming and troublesome that some of the partici-
pants did not answer all questions. Thorough instructions to complete the
questionnaire in any future study would make our analyses more convinc-
ing. The length of the research may also be worth considering because par-
ticipants might feel burdened in learning 100 expressions in only a relatively
short period.

5. Conclusions
This study examined the advantages of MALL from two perspectives: vocabu-
lary recall and autonomy. It explored whether blended L2 learning would help
learners to recall target expressions and also stimulate learner autonomy. To
this end, an experimental study and a questionnaire survey were conducted.
270 Vocabulary Recall and Learner Autonomy

The findings of the study showed that the advantages of MALL lay in bol-
stering the recall of the target expressions in both receptive and productive
tests. Furthermore, MALL brought about a significantly higher level of learner
autonomy in the psychological dimension and also higher motivation towards
L2 vocabulary learning, which would indicate the fostering of autonomy. Thus,
it appears likely that MALL has advantages in L2 vocabulary recall and, to
some degree, in the enhancement of learner autonomy in blended L2 learning
settings.
These findings appear to imply that L2 learning with advanced technology
should be examined not merely in respect of L2 learning gains but also moti-
vational effect, which would make the use of mobile devices for L2 learning
more effective, even in blended learning contexts. Thus, it is surely meaningful
to provide students with access to MALL regardless of their initial hesitancy or
resistance, as was experienced in this study. Nevertheless, a longer-term study
would be necessary to see more meaningful changes in learner autonomy.

Notes
1. Five characteristics of autonomous learners in a classroom are: compli-
ance, competence, cognizance, introspection, and diplomacy (Lyddon,
2016). Lyddon claims that mobile learning in a classroom generates dif-
ferent autonomous behaviors from mobile learning independently con-
ducted outside the classroom.

About the Authors

Takeshi Sato is an Associate Professor in the Institute of Engineering at Tokyo
University of Agriculture and Technology, Japan. His current academic inter-
ests are L2 vocabulary acquisition, CALL, and MALL. His articles can be found
in several journals such as TESOL Quarterly and ReCALL, and he recently
published a book on Implementing Mobile Language Learning Technologies in
Japan (Springer Education).

Fumiko Murase is an Associate Professor in the Faculty of Economics at Ryu-

koku University, Japan, where she teaches English to undergraduate students.
She holds a PhD in Linguistics from Macquarie University, Australia. Her
research interests include learner autonomy in language learning, assessment,
and out-of-class learning in the EFL context.

Tyler Burden is an Associate Professor in the Faculty of Education at Meisei

University, Japan. His research interests include L2 vocabulary acquisition
 Takeshi Sato, Fumiko Murase, and Tyler Burden 271

and testing. He has published various materials for EFL learners including
textbooks and graded readers.

References
Benson, P. (1997). The philosophy and politics of learner autonomy. In P. Benson & P. Voller
(Eds.), Autonomy and independence in language learning (pp. 18–34). London, England:
Longman.
Benson, P. (2001). Teaching and researching autonomy in language learning. Harlow,
England: Pearson Education.
Burston, J. (2015). Twenty years of MALL project implementation: A meta-analysis of
learning outcomes. ReCALL, 27(1), 4–20. https://doi.org/10.1017/S0958344014000159
Çakmak, F., & Erçetin, G. (2018). Effects of gloss type on text recall and incidental vocab-
ulary learning in mobile-assisted L2 listening. ReCALL, 30(1), 24–47. https://doi.
org/10.1017/S0958344017000155
Chen, C. M., Liu, H., & Huang, H. B. (2019). Effects of a mobile game-based English vocab-
ulary learning app on learners’ perceptions and learning performance: A case study of
Taiwanese EFL learners. ReCALL, 31(2), 170–188. https://doi.org/10.1017/
S0958344018000228
Collentine, K. (2011). Learner autonomy in a task-based 3D world and production. Lan-
guage Learning & Technology, 15(3), 50–67.
Dang, T. T. (2012). Learner autonomy: A synthesis of theory and practice. The Internet
Journal of Language, Culture & Society, 35, 52–67.
Dörnyei, Z. (2001). Motivational strategies in the language classroom. Cambridge, England:
Cambridge University Press.
G-TELP (n.d.). Overview G-TELP. Retrieved May 19, 2020 from https://g-telp.jp/english/
Holec, H. (1981). Autonomy and foreign language learning. Oxford, England: Pergamon
Press for Council of Europe.
Kukulska-Hulme, A. (2015). Language as a bridge connecting formal and informal lan-
guage learning through mobile devices. In L. H. Wong, M. Milrad, & M. Specht (Eds.),
Seamless learning in the age of mobile connectivity (pp. 281–294). Singapore: Springer.
Laurillard, D. (2007). Pedagogical forms for mobile learning: Framing research questions.
In N. Pachler (Ed.), Mobile learning: Towards a research agenda (pp. 153–175). London,
England: The Institute of Education.
Lee, L. (2016). Autonomous learning through task-based instruction in fully online lan-
guage courses. Language Learning & Technology, 20 (2), 81–97.
Loewen, S., Crowther, D., Isbell, D. R., Kim, K. M., Maloney, J., Miller, Z. F., & Rawal, H.
(2019). Mobile-assisted language learning: A Duolingo case study. ReCALL, 31(3), 293–
311. https://doi.org/10.1017/S0958344019000065
Lyddon, P. A. (2016). Mobile-assisted language learning and language learner autonomy.
In S. Papadima-Sophocleous, L. Bradley, & S. Thouësny (Eds.), Short papers from EURO-
CALL 2016 (pp. 302–306). Dublin, Ireland: Research-publishing.net.
Murase, F. (2015). Measuring language learner autonomy: Problems and possibilities. In
C. J. Everhard & L. Murphy (Eds.), Assessment and autonomy in language learning (pp.
35–63). London, England: Palgrave Macmillan.
Oxford, R. L. (2003). Toward a more systematic model of L2 learner autonomy. In D. Pal-
freyman & R. C. Smith (Eds.), Learner autonomy across cultures: Language education
272 Vocabulary Recall and Learner Autonomy

perspectives (pp. 75–91). Basingstoke, England: Palgrave Macmillan.

Pachler, N., Bachmair, B., & Cook, J. (2010). Mobile learning: Structures, agency, practices.
New York, NY: Springer.
Pennycook, A. (1997). Cultural alternatives and autonomy. In P. Benson & P. Voller (Eds.),
Autonomy and independence in language learning (pp. 35–53). London, England:
Longman.
Reinders, H. (2011). Learner autonomy and new learning environments. Language Learn-
ing & Technology, 15(3), 1–3.
Reinders, H., & Hubbard, P. (2013). CALL and learner autonomy: Affordances and con-
straints. In M. Thomas, H. Reinders, & M. Warschauer (Eds.), Contemporary computer
assisted language learning (pp. 359–375). London, England: Continuum Books.
Reinders, H., & White, C. (2011). Special issue commentary: Learner autonomy and new
learning environments. Language Learning & Technology, 15(3), 1–3.
Reinders, H., & White, C. (2016). 20 years of autonomy and technology: How far have we
come and where to next? Language Learning and Technology, 20(2), 143–154.
Rosell-Aguilar, F. (2018). Autonomous language learning through a mobile application: A
user evaluation of the busuu app. Computer Assisted Language Learning, 31(8), 854–881.
https://doi.org/10.1080/09588221.2018.1456465
Sato, T. (2016). Applicability of technology-enhanced visual glosses for explicit L2 vocab-
ulary learning: The enhancement of metaphoric competence through the learning of
English polysemous words. Ampersand, 3, 43–51. https://doi.org/10.1016/j.
amper.2016.03.003
Sato, T., Matsunuma, M., & Suzuki, A. (2013). Enhancement of automatization through
vocabulary learning using CALL: Can prompt language processing lead to better com-
prehension in L2 reading? ReCALL, 25(1), 143–158. https://doi.org/10.1017/
S0958344012000328
Sato, T., & Suzuki, A. (2010). Do multimedia-oriented visual glosses really facilitate EFL
vocabulary learning?: A comparison of planar images with three-dimensional images.
Asian EFL Journal, 12(4), 160–172.
Schwienhorst, K. (2003). Learner autonomy and tandem learning: Putting principles into
practice in synchronous and asynchronous telecommunications environments. Com-
puter Assisted Language Learning, 16, 427–443. https://doi.org/10.1076/
call.16.5.427.29484
Steinberg, R. G. (2008). Perfect phrases for the TOEFL speaking and writing sections. New
York, NY: McGraw-Hill.
Ushioda, E. (2013). Motivation matters in mobile language learning: A brief commentary.
Language Learning & Technology, 17(3), 1–5.
Yeh, Y., & Wang, C.-W. (2003). Effects of multimedia vocabulary annotations and learn-
ing styles on vocabulary learning. CALICO Journal, 21(1), 131–144.
 Takeshi Sato, Fumiko Murase, and Tyler Burden 273

Appendix A
The List of Expressions for Essay Writing
The following table shows the 100 expressions for essay writing that the stu-
dents were asked to learn. The expressions in bold were the 20 expressions the
participants had least knowledge of.
274 Vocabulary Recall and Learner Autonomy

Appendix B
Learner Autonomy Questionnaire (Murase, 2015)
Technical Dimension
1. I set long-term goals in learning English.
2. I make long-term plans for studying English.
3. I set goals for the day before I start studying English.
4. I make study plans for the day before I start studying English.
5. I set achievable goals in learning English.
6. I make study plans that match my goals in learning English.
7. I make realistic plans for studying English.
8. I revise my English study plans if they don’t work well.
9. If I have a limited amount of time available for study, I decide in what
order the things need to be done.
10. I reflect upon how I studied after I finish studying English for the day.
11. I reflect upon what I learned after I finish studying English for the day.
12. I try to create opportunities to use English outside the classroom.
13. I try to create the conditions under which I can study English best.
14. I evaluate the improvement in my ability to use English effectively.
15. I assess how much of my goal I have achieved.
16. I assess the effectiveness of my English study plans.
17. I take notes about how much time I spent on my English study.
18. I keep records of what kind of methods I used for my English study.
19. I write down what kinds of materials I used for my English study.
20. I keep records of what I learned from my English study.
21. I take notes of my feelings while I am studying English.

Psychological Dimension
22. All students ought to set their own goals in learning English.
23. Every student ought to set long-term goals in learning English.
24. All students ought to make long-term plans for studying English.
25. Every student ought to set goals for the day before he/she starts study-
ing English.
26. All students ought to choose the materials suitable for their goals in
learning English.
27. Every student ought to make study plans that match his/her goals in
learning English.
28. All students ought to make realistic plans for studying English.
29. Every student ought to create the conditions under which he/she can
study English best.
 Takeshi Sato, Fumiko Murase, and Tyler Burden 275

30. Every student ought to reflect upon how he/she studied after he/she
finishes studying English for the day.
31. All students ought to reflect upon what they learned after they finish
studying English for the day.
32. Every student ought to write down how he/she studied English.
33. A good learner of English keeps records of what he/she learned from
his/her English study.
34. Every student ought to evaluate the improvement in his/her ability to
use English effectively.
35. Every student ought to assess the effectiveness of his/her English study
plans.
36. I know what I need to study to improve my English.
37. I know what I am good at in learning English.
38. If I ask my teacher for help in learning English, I know how I want him/
her to help me.
39. I know the conditions under which I can study English best.
40. If I don’t feel like studying English, I know the reason.
41. If I don’t feel like studying English, I know how I can motivate myself.
42. I want to study overseas in the future.
43. I want to work overseas in the future.
44. I want to get a job where I use my English in the future.
45. I like the English language.
46. I like studying English.
47. I give a higher priority to studying English than studying other aca-
demic subjects.
48. The reason that I study English is to pass the exams for English classes.
49. The reason why I study English is that it is an obligatory part of the
course.
 276 Vocabulary Recall and Learner Autonomy

Appendix C
Items on Learning Experiences

Q1. During the three weeks, how often (on average) did you study the
expressions?

• Almost every day

• 3 to 4 times a week
• 1 to 2 times a week
• I hardly studied

Q2. Where did you mainly study?

• At home
• At university
• On the train or bus
• Other

Q3. (For the control group)

By using a paper list, did you feel motivated towards learning essay expressions?

• Very motivated
• A little motivated
• Not very motivated
• Not at all motivated

Q4. (For the experimental group)

By using mobile devices, did you feel motivated towards learning essay
expressions?

• Very motivated
• A little motivated
• Not very motivated
• Not at all motivated

View publication stats