TYPE Original Research

PUBLISHED 02 November 2023

DOI 10.3389/fpsyg.2023.1255594

Artificial intelligence-based
OPEN ACCESS language learning: illuminating
the impact on speaking skills and
EDITED BY
Xiaochun Xie,
Northeast Normal University, China

REVIEWED BY
Jill Burstein,
self-regulation in Chinese EFL
Duolingo, United States
Ranilson Oscar Araújo Paiva,
Federal University of Alagoas, Brazil
context
*CORRESPONDENCE
Aruna Zhao Hongliang Qiao 1 and Aruna Zhao 2*
60289@bttc.edu.cn 1
School of Foreign Languages, Northeast Petroleum University, Daqing, China, 2 Department of Foreign
RECEIVED 09 July 2023 Language, Baotou Teachers’ College, Inner Mongolia University of Science and Technology, Baotou,
ACCEPTED 11 October 2023 China
PUBLISHED 02 November 2023

CITATION
Qiao H and Zhao A (2023) Artificial Introduction: This study investigated the effectiveness of artificial intelligence-
intelligence-based language learning: based instruction in improving second language (L2) speaking skills and speaking
illuminating the impact on speaking skills and self-regulation in a natural setting. The research was conducted with 93 Chinese
self-regulation in Chinese EFL context.
Front. Psychol. 14:1255594. English as a foreign language (EFL) students, randomly assigned to either an
doi: 10.3389/fpsyg.2023.1255594 experimental group receiving AI-based instruction or a control group receiving
COPYRIGHT traditional instruction.
© 2023 Qiao and Zhao. This is an open-access
Methods: The AI-based instruction leveraged the Duolingo application,
article distributed under the terms of the
Creative Commons Attribution License (CC BY). incorporating natural language processing technology, interactive exercises,
The use, distribution or reproduction in other personalized feedback, and speech recognition technology. Pre- and post-tests
forums is permitted, provided the original
were conducted to assess L2 speaking skills and self-regulation abilities.
author(s) and the copyright owner(s) are
credited and that the original publication in this Results: The results of the study demonstrated that the experimental group,
journal is cited, in accordance with accepted
which received AI-based instruction, exhibited significantly greater improvement
academic practice. No use, distribution or
reproduction is permitted which does not in L2 speaking skills compared to the control group. Moreover, participants in the
comply with these terms. experimental group reported higher levels of self-regulation.
Discussion: These findings suggest that AI-based instruction effectively enhances
L2 speaking skills and fosters self-regulatory processes among language learners,
highlighting the potential of AI technology to optimize language learning
experiences and promote learners’ autonomy and metacognitive strategies in the
speaking domain. However, further research is needed to explore the long-term
effects and specific mechanisms underlying these observed improvements.

KEYWORDS

artificial intelligence, AI-based instruction, speaking skills, self-regulation, Duolingo,

EFL

1. Introduction
In recent years, the integration of information technology into education has revolutionized
instructional methodologies, with portable computers becoming ubiquitous tools in educational
organizations (Ruiz-Mercader et al., 2006; Gikas and Grant, 2013; Shatri, 2020; Sheikh et al.,
2021; Akhmedov, 2022; Fathi et al., 2023; Garcia and Garcia, 2023; Liu et al., 2023). This
technological surge has empowered educational institutions to enrich their curricula by
incorporating virtual learning environments for instructional tasks (Hamuddin, 2018;
Yamamoto et al., 2018). Among the array of technological advancements, artificial intelligence
(AI) has emerged as a potent force in the realm of education, particularly in the domain of

Frontiers in Psychology 01 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

spoken communication, which has witnessed remarkable progress demands, and optimize their learning outcomes (Pintrich, 2000). It
(Pokulevska, 2018). The integration of AI is driven by the aim to unfolds as a cyclical journey of forethought, performance, and self-
facilitate language learning within virtual environments, emancipating reflection, with individuals setting goals, applying strategies, and
learners from the constraints of time and physical classroom contexts, evaluating their progress, all of which significantly influence their
thus enabling seamless access to course materials and fostering academic achievements (Zimmerman, 2002).
communication with teachers and peers (Hamuddin, 2018; Ahmad In the realm of language learning, SRL plays a pivotal role in
et al., 2021; Gardner et al., 2021). shaping learners’ linguistic proficiency and autonomy. It empowers
Furthermore, AI has proven to be a transformative tool in them to actively engage with linguistic content, adeptly manage
kindling students’ enthusiasm and facilitating interactive language their learning resources, and navigate the intricacies of language
learning activities, indispensable in the contemporary educational tasks (Teng and Zhang, 2016). Scholars have underscored the
landscape (Wekke et al., 2017). Notably, the field of English as a critical importance of investigating SRL in language learning
Foreign Language (EFL) has witnessed the positive influence of AI in contexts as it holds the potential to enhance pedagogical practices
recent years (Hamuddin, 2018; Junaidi, 2020). Through automatic and foster learners’ self-directedness (Oxford, 2016; Bowen and
speech recognition technology, AI systems exhibit an ability akin to Thomas, 2022). Understanding how learners regulate their language
human comprehension, detecting and comprehending learners’ learning processes, establish goals, and employ strategies is
speech. This capacity is particularly valuable in environments where paramount for educators striving to design effective language
native English speakers may not be readily available, effectively instruction and cultivate autonomous language learners.
bolstering learners’ speaking skills (Junaidi, 2020; Zou et al., 2020; In this study, the operational definition of SRL is anchored in
Soleimani et al., 2022). It is pertinent to acknowledge that while the Seker’s (2016) model, which draws inspiration from well-established
application of AI in this context is promising, it remains in its nascent models of SRL in the realm of second language acquisition,
stages, with substantial strides in automatic speech recognition primarily building upon Boekaerts (1997) model and Oxford’s
technology materializing only in the early 2010s (Johnson and (1990) L2 learning strategy inventory. The examination of self-
Valente, 2009). regulated learning takes on pivotal significance within this study, as
The efficacy of AI in augmenting English language learning it serves as a linchpin for comprehending the intricate dynamics
performance has been subject to extensive investigation (Aljohani, that unfold when AI-based instruction intersects with learners’
2021; Sun et al., 2021; Zhang, 2022; Huang et al., 2023). Huang et al. active involvement, motivation, and cognitive strategies during
(2023) conducted a comparative study assessing the learning their language learning journey (Chang, 2005). This comprehensive
performance and engagement of learners in an AI-driven class, investigation not only sheds light on how AI impacts these critical
offering personalized video selection options, juxtaposed with a facets but also offers a deeper insight into the mechanisms that
non-AI class devoid of such choices. The results unveiled superior underpin effective language acquisition. Consequently, the findings
learning outcomes and heightened engagement levels among learners gleaned from this study are poised to offer invaluable guidance for
in the AI-driven class. refining and enhancing pedagogical approaches tailored to the
Moreover, a multitude of studies have delved into the impact of unique needs and preferences of language learners in AI-integrated
AI on the development of speaking skills among English language educational settings.
learners (Hill et al., 2015; Junaidi, 2020; Maknun, 2020; Divekar Despite previous research delving into the influence of AI on
et al., 2022; Kang, 2022; Suciati et al., 2022; Rustamova and the speaking skills of English language learners, a notable gap
Rakhmatullaeva, 2023). For instance, Hill et al. (2015) scrutinized persists in comprehending the impact of AI on the speaking skills
human-human interactions in contrast to AI-human interactions, and self-regulation of EFL learners. Consequently, further
revealing that learners exhibited prolonged engagement in investigation is imperative to elucidate the role of AI in enhancing
interactions with AI compared to their interactions with peers. both the speaking skills and self-regulation of EFL learners. This
Similarly, Kang (2022) compared learner-AI interactions with study, therefore, endeavors to bridge this research gap, aiming to
learner-native speaker interactions, uncovering the pivotal role of AI meticulously examine the effects of AI on the speaking skills and
in enhancing learners’ speaking skills. Junaidi (2020) further self-regulation of Chinese EFL learners. In this Randomized
substantiated these findings, indicating that AI-supported Controlled Trial (RCT), we employed two dependent variables:
instruction positively impacted learners’ overall speaking speaking skills, operationalized as a composite of skills
performance, including aspects of fluency, grammatical accuracy, encompassing fluency, vocabulary, accuracy, and pronunciation,
lexicon, and pronunciation. and self-regulation. Additionally, we considered speaking anxiety
Self-regulated learning (SRL) stands as a foundational concept in and global English proficiency as control variables, providing a
the educational landscape and has garnered considerable attention, comprehensive understanding of the multifaceted impacts of AI in
particularly within the domain of language learning. Zimmerman the educational landscape. Speaking anxiety, in the context of this
(1989) aptly defines SRL as the active process through which learners study, refers to the emotional and psychological discomfort or
proactively manage and oversee their cognitive, metacognitive, apprehension experienced by language learners when engaging in
motivational, and emotional dimensions in pursuit of their learning L2 spoken interactions. It encompasses feelings of nervousness, fear,
objectives. This multifaceted construct encompasses a wide array of or unease related to speaking tasks or situations in the L2 learning
strategies and processes, including goal establishment, self- environment (Ozdemir and Papi, 2022). The ensuing findings hold
monitoring, strategic planning, metacognitive awareness, and the substantial practical and pedagogical implications for the EFL
regulation of motivation (Zimmerman, 2002). SRL empowers learners context, furnishing valuable insights for educators and
to take control of their learning experiences, adapt to shifting practitioners alike.

Frontiers in Psychology 02 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

2. Literature review and intelligent decisions comparable to those made by humans. Kim
(2018) highlights the precise evaluation and enhanced features of AI
2.1. Theoretical framework applications, making them an effective means to improve oral skills,
particularly in listening and speaking.
The collaborative speaking exercises employed in this study align Several studies have been conducted addressing the impact of AI
with the principles of social constructivism, as proposed by Vygotsky on different language learning skills in English as a second and foreign
(1984), within both participant groups. Vygotsky (1984) argued that language contexts, including writing (Fitria, 2023), reading (Liu,
interactions with individuals who possess greater skills and intellect 2021), listening (Suryana et al., 2020), and speaking (Maknun, 2020;
enable learners to progressively internalize knowledge and develop Divekar et al., 2022; Rustamova and Rakhmatullaeva, 2023). With
higher levels of autonomous consciousness. According to Vygotsky regard to speaking skills, Suciati et al. (2022), for example, qualitatively
(1986), “every function in the child’s cultural development appears examined the impact of an AI-based program, namely Cake on
twice: first, on the social level, and later, on the individual level; first, learners’ language learning and their speaking abilities. Collecting the
between people (inter-psychological), and then inside the child (intra- required data via interview, observation, and documentation, the
psychological)” (p. 57). The concept of the zone of proximal findings indicated that AI-based instruction had a significant impact
development (ZPD), at the core of Vygotsky’s social constructivist on the learners’ language learning in general and their speaking
theory, refers to the disparity between learners’ current level of abilities in particular. Some features of AI-based instruction, such as
problem-solving ability and their potential for improvement when the user-friendliness of the program, its availability in different places
engaged in collaborative problem-solving activities with more and at different times, its wide repertoire of speaking topics, and the
competent peers. In the learning environment, learners can reach their ability to evaluate learners’ language learning, were believed to be the
ZPD by actively participating in activities and seeking assistance from reason behind the learners’ successful performance in language
others. By attaining their ZPD, learners are able to independently learning and speaking performance.
regulate their learning tasks and exhibit greater autonomy within the In a quasi-experimental study conducted by Maknun (2020), the
learning context. impact of an AI application called Orai on EFL students’ speaking
Consistent with the findings of Hsu et al. (2023) and Kim (2008), performance was examined. The experimental group utilized Orai for
collaborative learning in a group enables students to support each their communicative speaking activities, while the control group did
other in reaching their ZPD across various language learning abilities. not incorporate Orai into their group-based speaking interactions.
This is achieved by assuming roles as more proficient or less proficient The findings demonstrated that the experimental students exhibited
learners depending on the language learning tasks and activities. By higher speaking performance compared to the control students after
collaborating in pairs or groups to accomplish diverse language the intervention, underscoring the significant role of AI-based
learning tasks, students engage in co-constructing language learning instruction in enhancing EFL students’ speaking abilities.
competencies and reaching their ZPD. In the present study, both the Similarly, Safadi et al. (2022) investigated the influence of
experimental and control groups adhered to Vygotsky’s social AI-based instruction on the speaking skills of female English language
constructivist theory of learning, involving interactive speaking learners using a quasi-experimental research design. In this study, the
activities with peers. In the experimental group, students interacted experimental group engaged in interaction with AI to develop their
with AI to access their ZPD, while in the control group, students speaking performance, while the control group’s speaking performance
achieved their ZPD through collaborative speaking activities with was enhanced through interactive speaking activities with peers.
more proficient peers. Speaking skills tests were administered to collect the necessary data,
and the findings revealed that the experimental learners surpassed the
control learners in speaking performance, thereby confirming the
2.2. Artificial intelligence substantial effects of AI-based instruction in fostering the speaking
performance of female English language learners.
AI has emerged as an effective pedagogical approach in language In a similar vein, Junaidi (2020) investigated the role of
learning and instruction, offering language learners various AI-supported instruction using Lyra application in improving EFL
opportunities to enhance their language learning performance (Zhang learners’ speaking skills via an experimental, control group research
and Zou, 2020; Sun et al., 2021; Zhang, 2022) and foster positive design. The learners using their mobile phones to communicate with
perceptions and attitudes toward AI (Xia et al., 2022). Aldosari (2020) AI Lyra during class time in order to develop their speaking skills. The
defines AI as a programmed system that simulates and produces control group, however, did not utilize the AI Lyra during their
intelligent applications for computers and smartphones, capable of interactive speaking activities. The learners in both groups mainly
performing a wide range of tasks with human assistance. Luckin et al. focused on the sub-scales of speaking performance, namely fluency,
(2016) argue that AI can provide support in teaching, group learning, grammatical accuracy, lexicon, and pronunciation. The results
and virtual reality within language learning contexts. demonstrated that the AI-supported instruction outperformed its
Bibauw et al. (2019) suggest that AI-supported teaching, such as non-AI counterpart in developing the speaking subcomponents of the
chatbots, facilitates communication between learners and provides EFL learners. Makhlouf (2021) also examined the effectiveness of AI
both input and output. Chatbots promote authentic and meaningful instruction on EFL learners’ speaking performance in general and
social interactions (Clark, 2018) with various modalities, including fluency and accuracy, as the subcomponents of speaking performance,
text, audio, and visual elements, while delivering prominent and easily in particular. The study adopted a one-group pre-test, and post-test
comprehensible feedback (Bao, 2019). According to Akerkar (2014) research design and gathered the required data through some
and Ginsberg (2012), AI possesses the capability to make informed developed speaking skills tests. The study examined whether the

Frontiers in Psychology 03 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

learners’ interactions with AI using their mobile devices during class conversational skills, underscoring the effectiveness of AI in
time affected their speaking performance. The findings showed the language learning.
learners’ improvements in speaking performance in general and Also, Yan (2023) adopted a qualitative approach to explore the
fluency and accuracy in particular. impact of ChatGPT in L2 writing classrooms. The findings indicated
Kang (2022) examined the differences between AI-and native that ChatGPT had the potential to enrich L2 writing pedagogy by
speaker-supported instruction on speaking skills and affective factors introducing developmental features in learning activities and
of second language learners in a college in the United States. The facilitating efficient writing. This pioneering endeavor underscored
findings indicated that both AI-supported instruction and native the need for further research into ChatGPT’s applicability in L2
speaker instruction significantly developed the learners’ speaking learning and the formulation of corresponding pedagogical
skills. However, the learners who had interaction with AI adaptations. Jia et al. (2022) devised an AI system designed to enable
outperformed those who had interaction with a native speaker in authentic and ubiquitous language acquisition. Their study, involving
enhancing their speaking performance in general and speaking skills, 20 participants, employed a combination of research methods to
including accuracy, fluency, and coherence. The findings further assess the system’s usability and validity. The findings affirmed the
revealed that the learners with a low proficiency level benefited more efficacy of the AI system and yielded insights for potential
from interaction with AI in comparison with the learners with a high enhancements. This research contributes to the integration of AI in
proficiency level. On the other hand, the learners with a high language instruction and adheres to mobile learning principles,
proficiency level benefited more from interaction with a native emphasizing the significance of authentic learning environments.
speaker. Finally, the AI learners held positive perceptions toward With regard to Duolingo application, Li and Bonk (2023)
AI-based interaction, while native-speaker interaction did not conducted a study on online language learners using Duolingo outside
positively change the learners’ perceptions toward native-speaker- of formal classrooms. They found that learners employed various
based interaction. resources and self-monitored their learning process, relying on
In addition, El Shazly (2021) investigated the impact of AI on the Duolingo’s features. Intrinsic motivations, such as cultural interest and
speaking performance and anxiety of EFL learners via a quasi- travel, drove learners more than certificates or grades. Kessler (2023)
experimental, pretest–posttest design. Collecting the required data via addressed limitations in mobile-assisted language learning (MALL)
an anxiety test and IELTS speaking skill test, the results demonstrated applications by integrating reflective e-journal activities with
that AI significantly improved the learners’ speaking performance, Duolingo. The study, grounded in metacognition theory, revealed that
however, AI did not diminish the learners’ anxiety level. Hill et al. the journals enhanced students’ metacognitive awareness in various
(2015) compared human-human interaction with human-AI domains, with learners finding the activity beneficial and enjoyable.
interaction using the chatbot Cleverbot. The two types of interactions Also, Shortt et al. (2023) reviewed Duolingo’s gamified MALL
were explored by considering the number of words that occurred in application, highlighting its popularity and gamification elements.
each message and conversation, the number of messages in each They found that research focused on app design, quantitative methods,
conversation, and the uniqueness of the words that occurred in each and non-probability sampling, emphasizing tool creation over
utterance. The findings showed that human-AI interaction took a learning process and outcomes. The review identified preferences for
longer period of time in comparison with human-human interaction performance-based research questions, English language, and the
but contained shorter messages. The findings also demonstrated that United States as the main research context, revealing research gaps
human-human interaction contained rich vocabulary elements in with implications for MALL and gamification practitioners
comparison with human-AI interaction which lacked the richness of and researchers.
vocabulary in conversations. The reviewed studies collectively underscore the transformative
In Yang et al.’s (2022) study, they introduced an innovative task- potential of AI in language learning, particularly in honing speaking
based voice chatbot named “Ellie” as an English conversation partner. skills. Each of these interventions has demonstrated significant
The results indicated that students enthusiastically engaged with Ellie, enhancements in learners’ speaking performance, providing evidence
and the high task success rates demonstrated the appropriateness of of the substantial impact AI can have in language education. However,
the chatbot’s task design and operational intents. While the research despite the promising strides made in understanding the impact of AI
emphasized the positive potential of chatbots in EFL environments, it on language learning, a notable gap persists in comprehending its
also acknowledged specific limitations that warrant attention, offering effects in specific contexts, such as the Chinese EFL environment
valuable insights for the future integration of AI chatbots in examined in this study. While existing research provides valuable
language classrooms. insights, there remains a need for a more nuanced exploration of AI’s
Hsu et al. (2021) conducted an investigation into the influence of role in enhancing speaking skills within this specific demographic.
the Amazon Echo Show, a widely used Intelligent Personal Assistant This study aims to address this gap by conducting a rigorous
(IPA), on the listening and speaking skills of L2 learners. Their investigation into the impact of AI-powered language learning on
controlled experiment encompassed two groups, revealing notable speaking skills and self-regulation among Chinese EFL learners,
enhancements in speaking proficiency among learners who interacted contributing to the broader discourse on the integration of AI in
with the IPA. Furthermore, learners reported that IPAs provided language education.
increased opportunities for oral interactions and alleviated speaking
anxieties. Divekar et al. (2022) merged AI and extended reality (XR)
to create lifelike conversational interactions for the acquisition of 2.3. Purpose of the current study
foreign languages. Their seven-week evaluation, involving university
students learning Chinese as a foreign language, demonstrated The existing literature has demonstrated the positive influence of
significant enhancements in vocabulary, comprehension, and AI on the language learning performance of English language learners

Frontiers in Psychology 04 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

(Wang, 2022). Additionally, several studies have highlighted the collected before the intervention started, embedded within the first
significant role of AI in improving English language learners’ speaking two course sessions, and posttest measurements were collected in the
performance (Hill et al., 2015; Junaidi, 2020; Maknun, 2020; Kang, last course session. At each institute, one control group participated
2022; Suciati et al., 2022). However, there is a research gap regarding in a traditional speaking course, while the intervention group received
the investigation of AI’s impact on learners’ speaking skills and self- the AI-based speaking instruction. Both courses were offered
regulation, particularly in the EFL context. To address this gap, the simultaneously, with the AI-based instruction being delivered by a
present study aimed to examine the effects of AI on the speaking skills team of researchers and two trained English teachers who collaborated
and self-regulation of EFL learners. To achieve this goal, a randomized with the present researcher. The AI-based speaking instruction was
controlled trial (Benson and Hartz, 2000) was conducted. Participants identical for all groups.
were randomly assigned to either the treatment group or the control The two English teachers involved in delivering AI-based
group. Both groups received instruction across a span of 13 weeks, instruction possessed extensive experience in teaching English as a
with each week comprising one session. In the treatment group, foreign language, with a combined teaching experience of over
participants received AI-based instruction, while the control group 20 years. Both instructors held advanced degrees in TESOL (Teaching
received instruction without AI. To assess the effectiveness of the English to Speakers of Other Languages) and were well-versed in the
AI-based instruction on speaking skills, four measures including principles of language pedagogy and technology-enhanced language
fluency, vocabulary, accuracy, and pronunciation were used. learning. Moreover, they underwent specialized training in utilizing
Additionally, two control variables were measured to account for the AI-based language learning platforms and were proficient in managing
possible influence of students’ language proficiency and speaking and supporting learners within this context.
anxiety on the dependent variable. The first control variable was global To achieve randomization, the two courses, namely the
English proficiency, which was measured using College English Test AI-based course and the conventional course, were presented as
(CET). The second control variable was speaking anxiety, which was part of a course-tandem known as the ‘English Speaking Course.’
included due to its potential impact on speaking performance. This course-tandem approach involved students enrolling in both
To further explore the impact of the AI-based instruction, the the AI-based and conventional courses simultaneously, allowing
interaction term of the course and pretest score was included as an them to experience both instructional methods. To ensure a fair and
additional predictor variable. This allowed us to assess the differential unbiased distribution of students between the control and
effects of the intervention for EFL students with low versus high experimental groups across all participating institutes, a blocked
pretest scores on the dependent variable. Based on the literature, randomization technique, aided by computer-generated random
we expected that the AI-based speaking instruction would lead to numbers, was applied. This process was designed to establish an
improvements in students’ speaking skills and self-regulation, as equitable representation of participants in both instructional
reflected by the four measures used in this study. groups, thus minimizing potential bias in group assignments. In
total, 47 students were randomly assigned to the AI-based
instruction (age: M = 20.2, SD = 1.8; 45.5% female), and 46 students
3. Method were assigned to the traditional speaking course (age: M = 20.1,
SD = 1.7; 37.5% female). After the study, all students were invited to
3.1. Participants participate in the respective other course, allowing for a crossover
design to assess the persistence of the AI-based instruction’s effect
The present study involved 93 intermediate-level EFL students in on speaking skills.
Mainland China. The sample included both male (n = 41) and female
(n = 52) participants. These students were enrolled in one of four
conversation courses offered by five different institutes that offered the 3.2. Instruments
AI-based speaking instruction. The institutes participating in this
study were a mix of English language institutes and university 3.2.1. Speaking skills
programs, each recognized for their commitment to providing quality To assess the effectiveness of the AI-based speaking instruction
language education. Although there were no significant differences in on the students’ speaking skills, four measures of speaking skill
the demographic characteristics or English language proficiency levels components were used: fluency, vocabulary, accuracy, and
of students across the institutes, it is worth noting that they offered a pronunciation. The speaking abilities of Chinese EFL learners were
variety of supplementary resources and support services to facilitate evaluated using the IELTS speaking skill examination. This assessment
language learning. These included language labs, conversation clubs, encompassed four equally weighted components, namely fluency and
and access to digital learning platforms, all of which contributed to the coherence, vocabulary, grammatical range and accuracy, and
overall learning experience. pronunciation. The learners’ performance in each area was evaluated
The demographics of the participants were as follows: the mean based on the topics provided in the IELTS speaking test. The IELTS
age was 21.36 years old (SD = 2.86), with a range of 19 to 26 years. The Speaking Band Descriptors were employed to assign scores ranging
majority of participants reported Chinese as their first language (97%), from 1 to 9 to each learner in each speaking skill category. These
while the remaining 3% spoke other languages at home. individual scores were then aggregated and divided by four to
To evaluate the effectiveness of the AI-based speaking instruction, determine the overall speaking score for each student. To ensure
a randomized controlled trial with repeated measures was conducted consistency, the speaking skills of the learners were evaluated by two
(Deaton and Cartwright, 2018). Prior to the study, all participants proficient assessors, consisting of the researcher and another
provided informed written consent. Pretest measurements were experienced instructor specialized in teaching EFL speaking. The

Frontiers in Psychology 05 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

inter-rater reliability, assessed using the Cohen’s kappa coefficient, (1986), was thoughtfully adapted to specifically target anxiety within
yielded a satisfactory value of 0.87. speaking contexts. Participants were asked to rate their responses to
each item on a 6-point Likert scale, allowing them to express their
3.2.2. Self-regulated learning agreement or disagreement, ranging from ‘strongly agree’ to
To assess the participants’ L2 self-regulation, we employed the ‘strongly disagree.’
Self-Regulated Language Learning Questionnaire (SRLLQ), originally Finally, the pre-test scores were measured to account for any
developed and rigorously validated by Seker (2016). This Likert-scale differences in the participants’ initial speaking skills before the
instrument offers five response options and serves as a tool to gage intervention began. Including these control variables increased the
learners’ self-reported engagement in Self-Regulated Learning (SRL), precision of the regression coefficients and reduced any potential bias
drawing on well-established models within the field. The SRLLQ that may have been caused by differences between the two groups at
consists of a comprehensive set of 30 items, thoughtfully distributed the beginning of the study. The use of these control variables ensured
across five distinct subscales, namely internal motivation (n = 5), that any improvements in the participants’ speaking skills and self-
external motivation (n = 4), cognitive strategies (n = 7), metacognitive regulation could be attributed to the AI-based speaking instruction,
strategies (n = 10), and evaluation (n = 4). Notably, great care was taken rather than other factors such as overall English proficiency or
in crafting the questionnaire items, ensuring that they employ anxiety levels.
straightforward language and phrasing, making them easily accessible
to students without the need for translation. The SRLLQ demonstrated 3.2.4. AI-based language learning application:
excellent internal consistency reliability, with Cronbach’s alpha values Duolingo
of 0.85 at the pretest and 0.79 at the posttest, reaffirming its robustness Duolingo stands at the forefront of language learning applications,
within our study’s sample. acclaimed for its pioneering approach to language acquisition. Since
its inception in 2011, Duolingo has evolved into a globally recognized
3.2.3. Control variables and widely adopted language learning platform, earning its place as
To ensure the accuracy and reliability of the results, control an AI-powered language learning tool of choice.
variables were included in the study (Cohen et al., 2003). Firstly, the A cornerstone of Duolingo’s language learning curriculum is its
global English proficiency of participants was assessed using the immersive speaking exercises. These exercises immerse learners in
College English Test 3 (CET-3). CET-3 is a well-established spoken interactions with the application, requiring them to respond
standardized English proficiency examination widely recognized in to prompts and questions in their target language. These interactions
China. It is a part of the reputable College English Test (CET) series, play a pivotal role in honing speaking fluency as they compel learners
which is meticulously organized and overseen by the National College to articulate their thoughts and ideas verbally.
English Testing Committee, an authoritative body in English language A standout feature within Duolingo’s speaking component is its
assessment (Wang, 1999). CET-3 is specifically designed for college real-time feedback mechanism. As learners respond to prompts, an
students who have completed a three-year English study as part of AI-powered chatbot diligently evaluates various facets of their spoken
their undergraduate programs. This comprehensive test rigorously language, encompassing pronunciation, fluency, vocabulary usage,
evaluates students’ proficiency in key language skills including and grammatical accuracy. This instantaneous feedback mechanism
listening, reading, and writing. It encompasses various critical aspects is made possible by the application of machine learning algorithms,
of English language competence such as vocabulary, grammar, meticulously analyzing learner performance data. Consequently,
comprehension, and written expression. learners receive personalized feedback tailored to their unique
Importantly, CET-3 scores hold significant recognition and are linguistic requirements, enabling them to effect immediate
extensively utilized by Chinese universities as a pivotal criterion for refinements and improvements in their speaking skills (Kessler, 2023).
assessing students’ English proficiency levels. These scores play a Also, Duolingo places a strong emphasis on motivation through
crucial role in evaluating eligibility for graduation and various gamification elements seamlessly integrated into its platform. Learners
academic pursuits. embark on a journey where they earn points, conquer challenges, and
It is worth noting that the widespread use and acceptance of unlock new levels as they progress, nurturing a sense of achievement
CET-3 scores in Chinese universities have been affirmed by and fostering consistent practice. The application further empowers
numerous studies (Yan and Huizhong, 2006; Li, 2009). Additionally, learners to track their language learning odyssey, offering insights into
the test’s validity and reliability have been rigorously examined and their overall progress and areas that may warrant additional focus (Li
documented in a range of academic publications (Zheng and and Bonk, 2023).
Cheng, 2008). The endorsement of CET-3 by reputable institutions In addition to individual speaking exercises, Duolingo offers
further attests to its credibility and effectiveness in evaluating interactive group activities and discussions that replicate real-world
English language proficiency among Chinese students (Xu and conversational scenarios (Shortt et al., 2023). These features immerse
Liu, 2018). learners in dynamic and naturalistic contexts, refining their ability to
Furthermore, to account for the potential influence of anxiety on comprehend and respond to spoken language in real time. Engaging
speaking performance, we incorporated a measurement of speaking in dialogs and discussions enriches their speaking skills. The infusion
anxiety. We evaluated participants’ anxiety levels while engaging in L2 of AI technology into Duolingo represents a groundbreaking shift in
speaking activities using a meticulously validated 19-item language acquisition. It provides learners with uninterrupted,
questionnaire developed by Ozdemir and Papi in 2022. This scale, interactive, and personalized practice opportunities. Through its
stemming from the foundational Foreign Language Classroom speaking exercises and real-time feedback, Duolingo propels learners
Anxiety (FLCA) Scale originally conceptualized by Horwitz et al. toward speaking fluency by encouraging them to express their

Frontiers in Psychology 06 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

thoughts and ideas in the target language, simultaneously refining groups, rigorous procedures were implemented (Moncher and
their pronunciation and language production abilities. Prinz, 1991).
In our study, we harnessed the formidable capabilities of the Initially, comprehensive training sessions were conducted for both
Duolingo application, with its AI-driven speaking component, to the researchers and the English teachers responsible for delivering the
scrutinize its precise impact on speaking skills, prominently fluency, AI-based speaking instruction. This training encompassed detailed
within the context of Chinese EFL learners. By harnessing the rich instructions on how to effectively utilize the Duolingo application,
feature set of Duolingo, we endeavored to unveil the role of AI-based facilitate the speaking activities, and provide constructive feedback to
language learning in elevating speaking skills and fostering self- learners. Subsequently, an initial evaluation was conducted to assess
regulation among Chinese EFL learners. the teachers’ preparedness, knowledge of course materials, and
proficiency in using the AI-based platform.
Continuous monitoring of treatment fidelity was achieved
3.3. Procedure through a series of unannounced observations during the intervention
period. These observations were conducted randomly at various
In this study, we employed a comparative design to investigate the points throughout the study at each participating institute. The
impact of an AI-based language learning application, Duolingo, on primary objective was to verify that the speaking activities were being
speaking skills and self-regulation in the Chinese EFL context. The delivered in accordance with the intended design. To further ensure
participants were randomly assigned to either the intervention group consistency in delivery, a checklist (see the Appendix) was devised to
or the control group. monitor the execution of speaking instruction in both the intervention
The intervention group received instruction using the Duolingo and control groups. This checklist covered essential aspects such as the
application, which utilizes natural language processing technology to duration of the speaking activities, the specific types of activities
facilitate language learning across multiple skills, including speaking, employed, and the nature of feedback provided to learners. English
listening, reading, and writing. Specifically, our study focused on the teachers responsible for delivering the speaking instruction diligently
speaking component of the application. The Duolingo AI chatbot completed this checklist for each session.
played a central role in this intervention by providing learners with The proficiency of the English teachers in delivering the
prompts and questions in English. The learners were then required to intervention and control courses was regularly evaluated by the
respond to these prompts, and the chatbot provided real-time research team. This evaluation included assessing the teachers’
feedback on various aspects of their spoken language, including knowledge of course materials, their facilitation of group discussions,
pronunciation, fluency, vocabulary, and accuracy. This feedback was and their ability to provide effective feedback to learners. These
generated through machine learning algorithms that analyzed evaluations were carried out periodically to ensure the ongoing quality
learners’ performance data, enabling the chatbot to offer personalized of instruction. Throughout the study, strict adherence to the intended
feedback tailored to each learner’s specific needs. The intervention also intervention protocol was maintained. Any deviations from the
included group activities and discussions, allowing learners to practice prescribed intervention were meticulously documented and
their speaking skills in a more naturalistic setting. Furthermore, addressed. The rigorous monitoring of treatment fidelity was essential
learners had the opportunity to track their progress within the to uphold the validity and reliability of the study’s findings, ensuring
Duolingo application, which provided them with motivation and that any observed differences in outcomes between the intervention
feedback on their overall language learning journey. and control groups could be confidently attributed to the intervention
In contrast, the control group received instruction through a more itself rather than variations in instructional delivery.
traditional speaking course. This course focused on facilitating Additionally, in order to maintain experimental fidelity and
speaking skills development through group discussions, activities, role prevent potential contamination of the control group, experimental
plays, and presentations. Although this course provided learners with students were placed in separate classrooms from the control group.
opportunities to practice their speaking skills in a supportive and This physical separation ensured that experimental students did not
structured environment, it did not utilize AI technology or offer share access or inadvertently influence control group participants.
personalized feedback on learners’ speaking performance. Additionally, students were reminded of the importance of
To ensure the integrity and validity of our study, both the maintaining confidentiality and not discussing specific instructional
intervention and control groups received an equal amount of details or materials outside of their respective groups. The research
instructional time and were exposed to comparable learning contexts, team also conducted periodic checks to monitor and reinforce
except for the differing instructional methods described above. By compliance with these guidelines throughout the study duration.
comparing the outcomes of the intervention and control groups,
we aimed to examine the specific impact of AI-powered language
learning on speaking skills and self-regulation in the Chinese 3.5. Data analysis
EFL context.
The study used several analyzes to examine the effectiveness of the
intervention and ensure the groups were similar at the beginning of
3.4. Treatment adherence the study. First, two-tailed t-tests were conducted for all dependent
and control variables to examine baseline equivalence. The purpose
In this study, the consistent and accurate delivery of the AI-based was to ensure that any differences observed between the groups at the
speaking intervention was paramount. To ensure treatment adherence end of the study were due to the intervention and not
and monitor treatment fidelity across all participating institutes and pre-existing differences.

Frontiers in Psychology 07 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

To evaluate the usefulness of the intervention, multiple linear equivalence of the groups at the outset of the study, ensuring that any
regressions were used. Mplus Version 7 was used to conduct the subsequent changes in speaking skills and self-regulation can
analyzes employing maximum likelihood robust estimation (MLR). be attributed to the intervention rather than pre-existing disparities in
The amount of missing data varied between 1.8 and 5.7%, with the proficiency levels. The table also shows that, at pretest, the means for
higher rate resulting from students’ absence at posttest. However, each group on all variables were relatively similar, with no significant
there was no differential dropout between the treatment and control differences between the groups. This suggests that the groups were
group (χ2(1, 93) = 1.08; p = 0.263), which suggests that the missing data equivalent at the beginning of the study. After the intervention, the
were missing at random. To handle missing data, the full-information experimental group showed higher means than the control group on
maximum likelihood (FIML) estimator was employed. all speaking skills as well as self-regulation, suggesting a positive effect
For evaluating the effects of the training, directed hypotheses were of the intervention on these skills. However, the two groups did not
formulated, and one-tailed tests of significance were employed. The differ significantly on the control variable of speaking anxiety.
significance level (α) was set at 0.05 to determine the statistical Table 2 presents the correlations between variables at the pretest
significance of the findings. This approach was used to examine and posttest for the study. The results indicate that at the pretest,
whether the intervention had a positive effect on the speaking skill fluency is significantly correlated with any other variables. Also, at the
components based on IELTS: fluency, vocabulary, accuracy, and posttest, fluency is significantly correlated with vocabulary (r = 0.36,
pronunciation. Also, the effect of AI-based intervention on self- p < 0.01), accuracy (r = 0.24, p < 0.05), and pronunciation (r = 0.33,
regulation was investigated. In order to enhance the accuracy of the p < 0.01). In addition, at the posttest, vocabulary is significantly
regression coefficients and mitigate any potential bias stemming from correlated with fluency (r = 0.28, p < 0.05), accuracy (r = 0.34, p < 0.01),
initial between-group differences, control variables were incorporated. and pronunciation (r = 0.22, p < 0.05). Accuracy is significantly
These covariates encompassed overall English proficiency, speaking correlated with fluency (r = 0.27, p < 0.05) and vocabulary (r = 0.22,
anxiety, and pretest performance. p < 0.05) at the pretest, and with all other variables at the posttest
The dependent variables were measured as posttest measurements (r = 0.41 to 0.35, p < 0.01). Also, pronunciation is also significantly
for each of the four variables of speaking skills. To assess the effects of correlated with all other variables at the posttest (r = 0.37 to 0.35,
pretest differences and differential effects for participants with low p < 0.01). Likewise, self-regulation was positively correlated with all
versus high pretest scores on the dependent variable, the pretest score the other constructs both at pre-and posttest.
and interaction term of course and pretest score were included as Table 3 reports the results of examining the effects of an AI-based
additional predictor variables. If there was a significant interaction instruction on L2 speaking skills. The results indicate that the course
term, the effect of course participation differed for students depending had a significant positive effect on three of the four dependent
on their initial score on the dependent variable. In order to evaluate variables: fluency (B = 0.65, SE = 0.24, p = 0.006), vocabulary (B = 0.57,
the impact of initial score discrepancies and divergent outcomes SE = 0.29, p = 0.034), and accuracy (B = 0.46, SE = 0.23, p = 0.013). Also,
among participants with varying pretest scores, additional predictor the course had a significant effect on pronunciation (B = 0.42,
variables were incorporated. These variables consisted of the pretest SE = 0.21, p = 0.025). In addition, the intervention was found to have a
score itself and the interaction term between the course and pretest significant effect on self-regulation (B = 0.51, SE = 0.22, p = 0.018). This
score. In the event that a substantial interaction term was identified, suggests that controlling for the pretest scores and the interaction term
the influence of course participation varied for students based on their between the course and pretest score, global English proficiency, and
initial performance on the dependent variable (Cohen et al., 2003). All speaking anxiety, students who engaged in the speech training
continuous variables were standardized before analysis. Each course demonstrated significantly elevated scores in both speaking skills and
was binary coded, with speech training coded as 1 and the control self-regulation in comparison to students in the control group.
group as 0. The size of the course or treatment effect was indicated by The pretest score had a significant positive effect on accuracy
the standardized mean differences between the two groups, calculated (B = 0.42, SE = 0.17, p = 0.042), pronunciation (B = 0.53, SE = 0.21,
using Hedges (2007). p = 0.016), and self-regulation (B = 0.46, SE = 0.23, p = 0.027), but not
on fluency or vocabulary. Concerning the varying effects based on
students’ initial assessment scores, a statistically significant interaction
4. Results term between the pretest score and course was observed solely for the
vocabulary component (B = 0.41, p = 0.046). Accordingly, students
Table 1 shows the means and standard deviations for each group with higher vocabulary level at the pretest benefitted more from
in the pre-and post-tests. The table presents the data for the dependent the course.
variables, which are the four speaking skills (i.e., fluency, vocabulary, Among the control variables, speaking anxiety had a significant
accuracy, and pronunciation) and one control variable (i.e., speaking negative effect on fluency (B = −0.34, SE = 0.16, p = 0.036) and
anxiety). Additionally, the CET scores are reported, which were taken pronunciation (B = −0.28, SE = 0.16, p = 0.046). This suggests that
as a measure of English language proficiency. higher levels of speaking anxiety were associated with decreased
Upon close examination of the pre-test means presented in fluency and less accurate pronunciation during the speaking tasks.
Table 1, it is observed that there were slight group differences across This finding underscores the importance of addressing and mitigating
some variables. However, to determine the statistical significance of speaking anxiety in language learning contexts to enhance learners’
these variations, independent samples t-tests were conducted. The oral proficiency. In contrast, speaking anxiety did not demonstrate a
results of these tests indicated that there were no statistically significant significant effect on vocabulary, accuracy, or self-regulation. This
differences between the experimental and control groups on any of the implies that while speaking anxiety may exert a notable influence on
pre-test variables (p > 0.05). This rigorous analysis confirms the initial certain facets of speaking skills, it may not necessarily impact other

Frontiers in Psychology 08 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

TABLE 1 Means and standard deviations for each group in pre- and post-tests.

Pre-test Post-test
Experimental G Control G Experimental G Control G
M SD M SD M SD M SD
Fluency 5.12 0.62 5.24 0.59 5.94 0.49 5.52 0.67

Vocabulary 4.85 0.78 4.73 0.69 5.78 0.67 5.09 0.63

Accuracy 5.26 0.81 5.32 0.58 6.32 0.93 4.97 0.56

Pronunciation 4.77 0.64 4.63 0.73 5.91 0.77 5.13 0.92

Self-regulation 3.28 0.73 3.41 0.71 4.34 0.82 3.89 0.83

CET 42.35 11.26 43.58 12.06

Speaking anxiety 3.12 0.69 2.94 0.57

TABLE 2 Correlations between the variables at the pretest (below the provision of tailored feedback and practice materials (Dupoux,
diagonal) and the posttest (above diagonal).
2018; Yan, 2023). This individualized approach allows learners to
1 2 3 4 5 address their specific language needs and progress at their preferred
pace, thereby fostering enhanced language development (Yang et al.,
(1) Fluency – 0.36** 0.24* 0.33** 0.34**
2021). Furthermore, AI-based instruction provides learners with
(2) Vocabulary 0.31** – 0.28* 0.34** 0.22**
extensive and immersive language input through various means
(3) Accuracy 0.27* 0.22* – 0.41** 0.32** such as interactive simulations, virtual environments, and
(4) Pronunciation 0.37** 0.21* 0.35** – 0.38** AI-powered chatbots or language tutors (Jia et al., 2022). Engaging
in realistic and contextually rich speaking activities within these
(5) Self-regulation 0.32** 0.27** 0.36** 0.43** –
platforms exposes learners to authentic language use, which plays a
*p < 0.05; **p < 0.01.
pivotal role in facilitating the development of fluency, vocabulary,
and pragmatic skills necessary for effective oral communication
linguistic domains or learners’ self-regulation strategies. These results (Bahrani and Sim, 2012). In addition, AI-based instruction ensures
suggest that learners’ ability to select and deploy vocabulary, maintain learners receive continuous and immediate feedback on their
grammatical accuracy, and engage in self-regulated learning processes speaking performance (Dodigovic, 2007; Kim et al., 2019). By
may be less directly influenced by their level of speaking anxiety. leveraging AI technologies, such as advanced speech recognition
Also, CET did not have a significant effect on any of the dependent and language processing algorithms, learners’ pronunciation,
variables. The explained variance (R2) for each dependent variable was grammar, and discourse features can be analyzed, enabling the
moderate, ranging from 0.23 for pronunciation to 0.44 for fluency, provision of real-time corrective feedback (Dodigovic, 2005; Divekar
indicating that the independent variables accounted for a significant et al., 2022). This prompt feedback allows learners to promptly
portion of the variance in the dependent variables. identify and rectify errors, reinforcing accurate language production
and fostering the development of self-monitoring and self-correction
skills (Li, 2023; Loncar et al., 2023). The findings of this study
5. Discussion demonstrate that AI-supported interactions fostered the
development of EFL learners’ self-regulation and outperformed
This research employed a mixed-methods approach to gather and learner-learner interactions in the control group. These findings
analyze data, drawing upon Vygotsky’s (1984) social constructivism align with Vygotsky’s (1984) social constructivism, highlighting the
to investigate the effects of AI-supported instruction on EFL learners’ role of AI as a facilitator in the growth of students’ self-regulation.
speaking skills and self-regulation in speaking. The quantitative Consistent with Vygotsky’s recommendations, learners initially
findings initially demonstrated that AI-based instruction had a engaged in communicative speaking activities with AI, which likely
significant positive impact on learners’ speaking skills, surpassing its assisted them in regulating their own speaking performance.
non-AI counterpart. These results align with prior studies by Hill et al. Through these communicative activities, students gradually
(2015), Junaidi (2020), Kang (2022), and Suciati et al. (2022), which transitioned from other-regulation to self-regulation, demonstrating
also reported favorable effects of AI on students’ speaking abilities. It independent speaking performance. Notably, the students who
is plausible that learners’ engagement with AI in more stimulating and exhibited self-control were able to complete their speaking tasks
interactive ways played a role in enhancing their speaking skills. In without relying on AI or other students, indicating higher levels of
other words, the virtual experience with AI motivated the learners to self-regulation among the AI learners. More specifically, in relation
engage in communication in a novel environment, potentially to Vygotsky’s sociocultural theory (1986), the idea of scaffolding
contributing to the observed improvements in their emphasizes the importance of external support in aiding learners’
speaking proficiency. cognitive and linguistic growth. AI systems can serve as valuable
AI-based instruction offers learners personalized and adaptive scaffolds by offering learners customized prompts, reminders, and
learning experiences, facilitating the analysis of their performance feedback that address their specific needs. This personalized
and enabling the identification of areas for improvement, as well as assistance enables learners to regulate their learning, establish goals,

Frontiers in Psychology 09 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

TABLE 3 AI-based instruction effects on the L2 speaking skills (posttest).

Fluency Vocabulary Accuracy Pronunciation Self-regulation

B (SE) p B (SE) p B (SE) p B (SE) p B (SE) p
Course 0.65 0.24 0.006 0.57 0.29 0.034 0.46 0.23 0.013 0.42 0.21 0.025 0.51 0.22 0.018

Pretest 0.39 0.21 0.112 0.19 0.22 0.427 0.42 0.17 0.042 0.53 0.21 0.016 0.46 0.23 0.027
Score

Course × −0.31 0.22 0.346 0.41 0.29 0.046 0.11 0.14 0.573 −0.26 0.15 0.456 0.13 0.15 0.462
Pretest
Score

CET 0.28 0.16 0.308 0.19 0.11 0.296 −0.24 0.16 0.954 0.15 0.11 0.372 0.14 0.16 0.764

Speaking −0.34 0.16 0.036 0.12 0.13 0.826 −0.33 0.17 0.307 −0.28 0.16 0.046 −0.16 0.15 0.359
anxiety

Explained 0.44 0.33 0.26 0.23 0.28

variance
(R2)

track their progress, and adapt accordingly, ultimately facilitating the Factors such as prior experience with self-regulation techniques,
development of self-regulation skills. intrinsic motivation, and external support systems can influence the
Furthermore, according to Bandura’s (1989) social cognitive rate at which individuals develop and consolidate self-regulation skills
theory, learning occurs through the observation, imitation, and (Zimmerman, 2002).
modeling of others’ behaviors. In the context of interactions supported The observed improvements in students’ speaking skills and self-
by AI, learners have the opportunity to observe and engage with AI regulation can be attributed to the flexibility of engaging in
systems that demonstrate self-regulatory behaviors, such as offering communicative speaking activities with AI anytime and anywhere.
adaptive feedback or guiding learners in goal-setting and planning. By Unlike traditional classroom settings, learners were not limited by
observing these behaviors, learners can internalize and replicate self- time and location, allowing them to engage in interactive speaking
regulatory strategies, leading to the development of their own self- activities at their convenience. These findings align with Gardner et al.
regulation skills. Moreover, AI technologies provide certain (2021) and Hamuddin (2018), who affirmed the positive role of AI in
advantages, such as adaptive learning algorithms and real-time data providing learners with opportunities to communicate in various
analysis, which enable learners to receive immediate feedback and convenient settings and at flexible times. Furthermore, students were
monitor their performance. This prompt feedback allows learners to more inclined to communicate with AI due to the stress-free
evaluate their progress, identify areas in need of improvement, and environment it provided for collaborative speaking activities. Speaking
adapt their learning strategies accordingly. By engaging in self- anxiety often hinders learners from actively participating in interactive
reflection and making adjustments based on the feedback received speaking activities with instructors and peers. In this context, AI
from AI systems, learners can cultivate metacognitive awareness and facilitated greater engagement in communicative speaking activities,
self-regulatory behaviors (Zimmerman, 2002). which, in turn, contributed to the development of students’ speaking
In examining the effects of AI-based instruction on self- skills and self-regulation.
regulation, it is imperative to consider the stability of self-regulation These findings align with the research conducted by Kang (2022),
over time. Self-regulation is a dynamic construct that may evolve and which also confirmed learners’ favorable effects of AI-supported
fluctuate based on various factors, including the duration and intensity instruction. EFL students might have found interacting with AI to
of the intervention (Zimmerman and Moylan, 2009). Research be more enjoyable compared to communicating with their peers,
suggests that achieving meaningful improvements in self-regulation which likely contributed to their enhanced performance in speaking
capability often requires a sustained effort and a considerable activities. As highlighted by Sun et al. (2021) and Zhang (2022), the
investment of time (Metcalfe and Mischel, 1999; Zimmerman, 2013; AI environment offered learners various user-friendly features,
Duckworth et al., 2019). including portability and accessibility at any time and place. These
While our study demonstrates significant positive effects of features may have contributed to the AI learners’ greater improvements
AI-based instruction on self-regulation among Chinese EFL learners, in speaking skills and self-regulation compared to non-AI learners.
it is important to acknowledge that the duration and intensity of the The implementation of an AI environment is recommended due
intervention may influence the stability of these gains. Previous to its potential to create a motivating and engaging technological
studies have indicated that extended exposure to self-regulation setting, enabling students to interact more effectively with AI and their
interventions, along with consistent practice and reinforcement, is peers, thereby enhancing their speaking skills and self-regulation. The
crucial for enduring improvements in self-regulatory skills findings of this study indicate that EFL students actively participated
(Zimmerman and Schunk, 2011; Duckworth et al., 2019; Lei et al., in collaborative speaking activities with AI and their peers, leading to
2022). Moreover, individual differences in learners’ receptiveness to significant improvements in their speaking skills and self-regulation.
self-regulation strategies and their capacity for sustained effort may The interactive speaking tasks facilitated by AI likely played a role in
play a role in the duration required to observe significant changes. enhancing students’ ability to self-regulate their speaking skills.

Frontiers in Psychology 10 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

According to Vygotsky (1984), students with diverse abilities can assist significantly enhances EFL students’ speaking skills and self-
their peers in reaching higher levels of learning performance, regulation, its integration is recommended in interactive EFL speaking
including self-regulation. In the context of this study, AI also courses. EFL educators are encouraged to incorporate AI in their
supported students in successfully performing interactive speaking communicative speaking courses to facilitate the development of
tasks by fostering positive evaluations of their speaking abilities, speaking skills and self-regulation among EFL students. By
aligning with the findings of Suciati et al. (2022). implementing an AI-supported classroom, EFL teachers can design
Finally, the findings regarding the impact of speaking anxiety on engaging communicative speaking activities and tasks involving both
different aspects of speaking skills provide valuable insights into the AI and peers. This approach would enable EFL students to engage in
complexity of this phenomenon. Specifically, higher levels of speaking meaningful speaking interactions, thereby improving their speaking
anxiety were found to have a significant detrimental effect on fluency skills and self-regulation.
and pronunciation, indicating that learners experiencing greater EFL students can benefit greatly from an AI-infused course
anxiety tend to exhibit reduced fluency and less accurate pronunciation specifically designed for communicative speaking activities. By
during speaking tasks. This underscores the critical need to address participating in such a course, students can receive feedback from
and alleviate speaking anxiety in language learning environments, as both peers and AI, leading to improvements in their communication
it directly correlates with learners’ oral proficiency (Galante, 2018). abilities and self-regulation in speaking. Additionally, the engaging
However, it is noteworthy that speaking anxiety did not yield a nature of the AI environment’s communicative speaking activities is
significant effect on vocabulary, accuracy, or self-regulation. This likely to enhance EFL students’ speaking proficiency and foster their
suggests an intricate relationship between speaking anxiety and enthusiasm for further language learning endeavors. The findings
various linguistic and self-regulation domains. While anxiety may suggest that AI-based instruction holds promise for addressing the
exert a notable influence on the fluidity and pronunciation of speech, unique challenges faced by Chinese EFL learners in developing
it may not necessarily impede other facets of language acquisition, speaking skills and self-regulation. With the growing demand for
such as lexical selection, grammatical accuracy, or self-regulated English proficiency in China, integrating AI technologies into
learning processes (Pae, 2013). These results indicate that learners may language classrooms can provide learners with effective tools to
possess a degree of resilience in certain linguistic areas, demonstrating improve their speaking abilities and develop important self-
an ability to select and utilize vocabulary, maintain grammatical regulatory competencies.
precision, and engage in self-regulated learning practices, irrespective Despite its implications, this study has several limitations that
of their level of speaking anxiety. This nuanced understanding of the should be considered. Firstly, the generalizability of the findings may
relationship between anxiety and different aspects of speaking skills be restricted to the specific sample of Chinese EFL students in a
can inform targeted interventions to enhance overall oral proficiency natural setting, cautioning against applying these results to learners
and mitigate the adverse effects of speaking anxiety on from different language backgrounds or diverse cultural and
language learners. educational contexts. Secondly, the relatively small sample size of 93
participants might have compromised the statistical power and
confidence in the findings, highlighting the need for larger samples to
6. Conclusion and implications enhance generalizability and detect smaller effects more reliably.
Another limitation pertains to the duration of the AI-based
The primary aim of the present study was to investigate the instruction and the length of exposure to the intervention, as the
potential of AI in enhancing EFL students’ speaking skills and self- impact on speaking skills and self-regulation could vary depending on
regulation. The findings indicated that AI learners exhibited greater the intervention’s duration. Longer intervention periods may yield
improvements in both speaking skills and self-regulation compared different outcomes and provide a more comprehensive understanding
to non-AI learners. These positive outcomes can be attributed to the of the effects. Additionally, the control group in this study received
creative and engaging environment that AI provided for interactive traditional instruction, which introduces the possibility of
speaking activities. The use of the Duolingo application, which confounding factors influencing the observed differences between the
incorporates AI technologies such as natural language processing, experimental and control groups. To better isolate the effects of
interactive exercises, personalized feedback, and speech recognition, AI-based instruction, future research could incorporate an active
resulted in significantly greater improvement in L2 speaking skills control group that receives an alternative instructional approach.
than traditional instruction. This suggests that AI-based instruction While the study employed pre-and post-tests to assess L2 speaking
has the potential to enhance language learning by providing learners skills and self-regulation abilities, it is important to acknowledge that
with interactive and personalized learning experiences that target these measures may not fully capture the complexity and range of
specific language areas for improvement. Also, the participants in the these skills. Including multiple measures and qualitative assessments
experimental group, who received AI-based instruction, reported in future studies would provide a more comprehensive understanding
higher levels of self-regulation than the control group. This indicates of the impact of AI-based instruction. The study also recognizes the
that AI technologies can support learners in regulating their learning need for further research to investigate the long-term effects of
processes, setting goals, monitoring their progress, and making AI-based instruction and delve into the specific mechanisms
necessary adjustments. Having offered personalized feedback and underlying the observed improvements. Gaining insight into the
adaptive exercises, AI-based instruction empowers learners to take sustainability of the effects and understanding the underlying
control of their learning and develop metacognitive strategies that processes will contribute to a more robust body of evidence regarding
enhance their speaking skills. the effectiveness of AI-based instruction in language learning.
The results of this study have important implications for EFL Although our study primarily focused on the positive effects of
education. Given that AI aligns with student-centered approaches and AI-based instruction on speaking skills and self-regulation,

Frontiers in Psychology 11 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

we recognize the need for continued attention to the complex issue of University of Science and Technology, China. The studies were
speaking anxiety in language learning. It is important to acknowledge conducted in accordance with the local legislation and institutional
that while our study did not find a significant change in speaking requirements. The participants provided their written informed
anxiety as a result of the AI-based intervention, the negative impact consent to participate in this study. Written informed consent was
of speaking anxiety on fluency and pronunciation highlights the obtained from the individual(s) for the publication of any potentially
importance of addressing learners’ anxiety levels in language learning identifiable images or data included in this article.
contexts. Understanding its impact and exploring effective strategies
to alleviate it can further enhance the language learning experience
for students. Additionally, the potential differences in anxiety levels Author contributions
between practicing with an AI tool and a human instructor in real
conversations present an intriguing avenue for further research. HQ: Data curation, Investigation, Methodology, Resources,
Future studies could explore the comparative effectiveness of Software, Validation, Writing – original draft, Writing – review &
practicing with AI tools and human instructors in reducing anxiety editing. AZ: Conceptualization, Formal analysis, Project
and enhancing conversational proficiency. This research could shed administration, Supervision, Visualization, Writing – original draft.
light on the unique benefits and challenges associated with each
approach, ultimately providing valuable insights for language learners
and educators. Conflict of interest
The authors declare that the research was conducted in the
Data availability statement absence of any commercial or financial relationships that could
be construed as a potential conflict of interest.
The raw data supporting the conclusions of this article will
be made available by the authors, without undue reservation.
Requests to access these datasets should be directed to AZ, 60289@ Publisher’s note
bttc.edu.cn.
All claims expressed in this article are solely those of the authors
and do not necessarily represent those of their affiliated
Ethics statement organizations, or those of the publisher, the editors and the
reviewers. Any product that may be evaluated in this article, or claim
The studies involving humans were approved by Department of that may be made by its manufacturer, is not guaranteed or endorsed
Foreign language, Baotou Teachers’ College, Inner Mongolia by the publisher.

References
Ahmad, S. F., Rahmat, M. K., Mubarik, M. S., Alam, M. M., and Hyder, S. I. (2021). Bowen, N. E. J. A., and Thomas, N. (2022). Self-regulated learning and knowledge
Artificial intelligence and its role in education. Sustainability 13:12902. doi: 10.3390/ blindness: bringing language into view. Appl. Linguis. 43, 1207–1216. doi: 10.1093/
su132212902 applin/amac062
Akerkar, R. (2014). Introduction to artificial intelligence 2nd Edn. New Delhi: PHI Learning. Chang, M. M. (2005). Applying self-regulated learning strategies in a web-based
instruction—an investigation of motivation perception. Comput. Assist. Lang. Learn. 18,
Akhmedov, B. (2022). A new approach to teaching information technologies in
217–230. doi: 10.1080/09588220500178939
education. Cent. Asian J. Educ. Comput. Sci. 1, 73–78. Available at: http://www.cajecs.
com/index.php/cajecs/article/view/28 Clark, D. (2018). To Siri with love: How chatbots are becoming social companions and
Aldosari, S. A. M. (2020). The future of higher education in the light of artificial teachers. Donald Clark Plan B. Available at: http://donaldclarkplanb.blogspot.
intelligence transformations. Int. J. High. Educ. 9, 145–151. doi: 10.5430/ijhe. com/2018/06/social-learning-andchatbots.html.
v9n3p145 Cohen, J., Cohen, P., West, S. G., and Aiken, L. S. (2003). Applied multiple regression/
Aljohani, R. A. (2021). Teachers and students’ perceptions on the impact of artificial correlation analysis for the behavioral sciences. 3rd Edn. Mahwah, NJ: L.
intelligence on English language learning in Saudi Arabia. J. Appl. Linguist. Lang. Res. 8, Erlbaum Associates.
36–47. Deaton, A., and Cartwright, N. (2018). Understanding and misunderstanding
Bahrani, T., and Sim, T. S. (2012). Informal language learning setting: technology or randomized controlled trials. Soc. Sci. Med. 210, 2–21. doi: 10.1016/j.socscimed.2018.
social interaction? Turk. Online J. Educ. Technol. 2, 10–17. doi: 10.7456/10203100/002 04.046
Bandura, A. (1989). Human agency in social cognitive theory. Am. Psychol. 44, Divekar, R. R., Drozdal, J., Chabot, S., Zhou, Y., Su, H., Chen, Y., et al. (2022). Foreign
1175–1184. doi: 10.1037/0003-066X.44.9.1175 language acquisition via artificial intelligence and extended reality: design and
evaluation. Comput. Assist. Lang. Learn. 35, 2332–2360. doi: 10.1080/09588221.
Bao, M. (2019). Can home use of speech-enabled artificial intelligence mitigate foreign 2021.1879162
language anxiety: investigation of a concept. Arab World Engl. J. 5, 28–40. doi: 10.24093/
awej/call5.3 Dodigovic, M. (2005). Artificial intelligence in second language learning: raising error
awareness. Multiling. Matters 2005:319. doi: 10.21832/9781853598319
Benson, K., and Hartz, A. J. (2000). A comparison of observational studies and
randomized, controlled trials. N. Engl. J. Med. 342, 1878–1886. doi: 10.1056/ Dodigovic, M. (2007). Artificial intelligence and second language learning: an efficient
NEJM200006223422506 approach to error remediation. Lang. Aware. 16, 99–113. doi: 10.2167/la416.0
Bibauw, S., François, T., and Desmet, P. (2019). Discussing with a computer to practice Duckworth, A. L., Taxer, J. L., Eskreis-Winkler, L., Galla, B. M., and Gross, J. J. (2019).
a foreign language: research synthesis and conceptual framework of dialogue-based Self-control and academic achievement. Annu. Rev. Psychol. 70, 373–399. doi: 10.1146/
CALL. Comput. Assist. Lang. Learn. 32, 827–877. doi: 10.1080/09588221.2018.1535508 annurev-psych-010418-103230
Boekaerts, M. (1997). Self-regulated learning: a new concept embraced by researchers, Dupoux, E. (2018). Cognitive science in the era of artificial intelligence: a roadmap
policy makers, educators, teachers, and students. Learn. Instr. 7, 161–186. doi: 10.1016/ for reverse-engineering the infant language-learner. Cognition 173, 43–59. doi: 10.1016/j.
S0959-4752(96)00015-1 cognition.2017.11.008

Frontiers in Psychology 12 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

El Shazly, R. (2021). Effects of artificial intelligence on English speaking anxiety and Li, R. (2023). Investigating effects of computer-mediated feedback on L2 vocabulary
speaking performance: a case study. Expert. Syst. 38:e12667. doi: 10.1111/exsy.12667 learning. Comput. Educ. 198:104763. doi: 10.1016/j.compedu.2023.104763
Fathi, J., Rahimi, M., and Liu, G. Z. (2023). A preliminary study on flipping an English Li, Z., and Bonk, C. J. (2023). Self-directed language learning with Duolingo in an
as a foreign language collaborative writing course with video clips: its impact on writing out-of-class context. Comput. Assist. Lang. Learn. 39, 1–19. doi: 10.1111/jcal.12730
skills and writing motivation. J. Comput. Assist. Learn. 39, 659–675. doi: 10.1111/
Liu, Z. (2021). Sociological perspectives on artificial intelligence: a typological
jcal.12772
reading. Sociol. Compass 15:e12851. doi: 10.1111/soc4.12851
Fitria, T. N. (2023). Artificial intelligence (AI) technology in OpenAI ChatGPT
Liu, G. Z., Fathi, J., and Rahimi, M. (2023). Enhancing EFL learners’ intercultural
application: a review of ChatGPT in writing English essay. ELT Forum 12, 44–58. doi:
communicative effectiveness through telecollaboration with native and non-native speakers
10.15294/elt.v12i1.64069
of English. Comput. Assist. Lang. Learn. 1-31, 1–31. doi: 10.1080/09588221.2022.2164778
Galante, A. (2018). Drama for L2 speaking and language anxiety: evidence from
Loncar, M., Schams, W., and Liang, J. S. (2023). Multiple technologies, multiple
Brazilian EFL learners. RELC J. 49, 273–289. doi: 10.1177/0033688217746205
sources: trends and analyses of the literature on technology-mediated feedback for L2
Garcia, M. B., and Garcia, P. S. (2023). “Intelligent tutoring system as an instructional English writing published from 2015-2019. Comput. Assist. Lang. Learn. 36, 722–784.
Technology in Learning Basic Nutrition Concepts: an exploratory sequential mixed doi: 10.1080/09588221.2021.1943452
methods study” in Handbook of research on instructional Technologies in Health Luckin, R., Holmes, W., Griffiths, M., and Forcier, L. B. (2016). Intelligence unleashed:
Education and Allied Disciplines. eds. M. B. Garcia, L. Cabrera, M. Vanessa, A. De An argument for AI in education. London, UK: Pearson.
Almeida and R. P. Pereira (Hershey, PA: IGI Global), 265–284.
Makhlouf, M. K. I. (2021). Effect of artificial intelligence-based application on Saudi
Gardner, J., O'Leary, M., and Yuan, L. (2021). Artificial intelligence in educational preparatory-year students’ EFL speaking skills at Albaha university. Int. J. Engl. Lang.
assessment:‘breakthrough? Or buncombe and ballyhoo?’. J. Comput. Assist. Learn. 37, Educ. 9:87. doi: 10.5296/ijele.v9i2.18782
1207–1216. doi: 10.1111/jcal.12577
Maknun, L. L. (2020). The implementation of Orai as artificial intelligence for digital
Gikas, J., and Grant, M. M. (2013). Mobile computing devices in higher education: native students in English speaking learning. Indones. Technol. Enhanced Lang. Learn.
student perspectives on learning with cell phones, smartphones & social media. Internet 1, 131–138.
High. Educ. 19, 18–26. doi: 10.1016/j.iheduc.2013.06.002
Metcalfe, J., and Mischel, W. (1999). A hot/cool-system analysis of delay of gratification:
Ginsberg, M. (2012). Essentials of artificial intelligence. Oxford: Newnes. dynamics of willpower. Psychol. Rev. 106, 3–19. doi: 10.1037/0033-295X.106.1.3
Hamuddin, B. (2018). Blogs as powerful learning tools: the perception from EFL Moncher, F. J., and Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clin.
students in Riau main island Indonesia. IOP Conf. Ser. Earth Environ. Sci 156:012060. Psychol. Rev. 11, 247–266. doi: 10.1016/0272-7358(91)90103-2
doi: 10.1088/1755-1315/156/1/012060
Oxford, R. L. (1990). Language learning strategies: What every teacher should know.
Hedges, L. V. (2007). Effect sizes in cluster-randomized designs. J. Educ. Behav. Stat. Boston, MA: Heinle and Heinle.
32, 341–370. doi: 10.3102/1076998606298043
Oxford, R. L. (2016). Teaching and researching language learning strategies: self-
Hill, J., Ford, W. R., and Farreras, I. G. (2015). Real conversations with artificial regulation in context. Abingdon: Taylor and Francis.
intelligence: a comparison between human–human online conversations and
human–chatbot conversations. Comput. Hum. Behav. 49, 245–250. doi: 10.1016/j. Ozdemir, E., and Papi, M. (2022). Mindsets as sources of L2 speaking anxiety and
chb.2015.02.026 self-confidence: the case of international teaching assistants in the US. Innov. Lang.
Learn. Teach. 16, 234–248. doi: 10.1080/17501229.2021.1907750
Horwitz, E. K., Horwitz, M. B., and Cope, J. (1986). Foreign language classroom
anxiety. Mod. Lang. J. 70, 125–132. doi: 10.1111/j.1540-4781.1986.tb05256.x Pae, T. I. (2013). Skill-based L2 anxieties revisited: their intra-relations and the inter-
relations with general foreign language anxiety. Appl. Linguis. 34, 232–252. doi: 10.1093/
Hsu, T. C., Chang, C., and Jen, T. H. (2023). Artificial intelligence image recognition applin/ams041
using self-regulation learning strategies: effects on vocabulary acquisition, learning
anxiety, and learning behaviours of English language learners. Interact. Learn. Environ. Pintrich, P. R. (2000). “The role of goal orientation in self-regulated learning” in
1-19, 1–19. doi: 10.1080/10494820.2023.2165508 Handbook of self-regulation. eds. M. Boekaerts, P. R. Pintrich and M. Zeidner (San Diego,
CA: Academic Press), 452–502.
Hsu, H. L., Chen, H. H. J., and Todd, A. G. (2021). Investigating the impact of the
Amazon Alexa on the development of L2 listening and speaking skills. Interact. Learn. Pokulevska, A. I. (2018). Experience of using skype as a means of foreign languages
Environ. 1-14, 1–14. doi: 10.1080/10494820.2021.2016864 teaching efficiency improving. Inf. Technol. Learn. Tools 68, 155–165. doi: 10.33407/itlt.
v68i6.2150
Huang, A. Y., Lu, O. H., and Yang, S. J. (2023). Effects of artificial intelligence–enabled
personalized recommendations on learners’ learning engagement, motivation, and Ruiz-Mercader, J., Meroño-Cerdan, A. L., and Sabater-Sánchez, R. (2006). Information
outcomes in a flipped classroom. Comput. Educ. 194:104684. doi: 10.1007/978- technology and learning: their relationship and impact on organisational performance
981-99-2449-3 in small businesses. Int. J. Inf. Manag. 26, 16–29. doi: 10.1016/j.ijinfomgt.2005.10.003

Jia, F., Sun, D., Ma, Q., and Looi, C. K. (2022). Developing an AI-based learning Rustamova, A. E., and Rakhmatullaeva, S. N. (2023). How to improve speaking skill
system for L2 learners’ authentic and ubiquitous learning in English language. with the help of innovative technology. In: International conferences, pp. 1104–1106.
Sustainability 14:15527. doi: 10.3390/su142315527 Safadi, H. A., Shgair, M. S. A., and Qatawnih, K. S. (2022). The effectiveness of
designing e-learning environment based on mastery learning and artificial intelligence
Johnson, W. L., and Valente, A. (2009). Tactical language and culture training systems:
on developing English speaking skills among tenth graders in Palestine. J. Educ. Psychol.
using AI to teach foreign languages and cultures. AI Mag. 30, 72–83. doi: 10.1609/aimag.
Sci. 31, 475–507. doi: 10.33976/IUGJEPS.31.1/2023/22
v30i2.2240
Seker, M. (2016). The use of self-regulation strategies by foreign language learners and its
Junaidi, J. (2020). Artificial intelligence in EFL context: rising students’ speaking
role in language achievement. Lang. Teach. Res. 20, 600–618. doi: 10.1177/1362168815578550
performance with Lyra virtual assistance. Int. J. Adv. Sci. Technol. Rehabil. 29,
6735–6741. Shatri, Z. G. (2020). Advantages and disadvantages of using information technology
in learning process of students. J. Turk. Sci. Educ. 17, 420–428.
Kang, H. (2022). Effects of artificial intelligence and native speaker interlocutors on
ESL learners’ speaking ability and affective aspects. Multimed. Assist. Lang. Learn. 25, Sheikh, A., Anderson, M., Albala, S., Casadei, B., Franklin, B. D., Richards, M., et al.
9–43. doi: 10.15702/mall.2022.25.2.09 (2021). Health information technology and digital innovation for national learning health
and care systems. Lancet Digit. Health 3, e383–e396. doi: 10.1016/S2589-7500(21)00005-4
Kessler, M. (2023). Supplementing mobile-assisted language learning with reflective
journal writing: a case study of Duolingo users’ metacognitive awareness. Comput. Shortt, M., Tilak, S., Kuznetcova, I., Martens, B., and Akinkuolie, B. (2023).
Assist. Lang. Learn. 36, 1040–1063. doi: 10.1080/09588221.2021.1968914 Gamification in mobile-assisted language learning: a systematic review of Duolingo
literature from public release of 2012 to early 2020. Comput. Assist. Lang. Learn. 36,
Kim, Y. (2008). The contribution of collaborative and individual tasks to the 517–554. doi: 10.1080/09588221.2021.1933540
acquisition of L2 vocabulary. Mod. Lang. J. 92, 114–130. doi: 10.1111/j.1540-4781.2008.
00690.x Soleimani, H., Mohammaddokht, F., and Fathi, J. (2022). Exploring the effect of
assisted repeated reading on incidental vocabulary learning and vocabulary learning
Kim, J. (2018). Artificial intelligence and retention: The future of persistence in the age self-efficacy in an EFL context. Front. Psychol. 13:851812. doi: 10.3389/fpsyg.2022.851812
of narrow AI, or deep learning.
Suciati, S., Faridi, A., Mujiyanto, J., and Arifani, Y. (2022). Artificial intelligence-based
Kim, N. Y., Cha, Y., and Kim, H. S. (2019). Future English learning: Chatbots and application cake as the alternative for learning speaking: opportunities and challenges. In:
artificial intelligence. Multimed. Assist. Lang. Learn. 22, 1–11. International conference on science, education, and technology, pp. 202–205.
Lei, X., Fathi, J., Noorbakhsh, S., and Rahimi, M. (2022). The impact of mobile- Sun, Z., Anbarasan, M., and Praveen Kumar, D. J. C. I. (2021). Design of online
assisted language learning on English as a foreign language learners’ vocabulary learning intelligent English teaching platform based on artificial intelligence techniques. Comput.
attitudes and self-regulatory capacity. Front. Psychol. 13:872922. doi: 10.3389/ Intell. 37, 1166–1180. doi: 10.1111/coin.12351
fpsyg.2022.872922
Suryana, I., Asrianto, A., and Murwantono, D. (2020). Artificial intelligence to master
Li, H. (2009). Are teachers teaching to the test? A case study of the college English test English listening skills for non-English major students. J. Lang. Lang. Teach. 8, 48–59.
(CET) in China. Int. J. Pedagog. Learn. 5, 25–36. doi: 10.5172/ijpl.5.1.25 doi: 10.33394/jollt.v8i1.2221

Frontiers in Psychology 13 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

Teng, L. S., and Zhang, L. J. (2016). A questionnaire-based validation of Yang, H., Kim, H., Lee, J. H., and Shin, D. (2022). Implementation of an AI chatbot as
multidimensional models of self-regulated learning strategies. Mod. Lang. J. 100, an English conversation partner in EFL speaking classes. ReCALL 34, 327–343. doi:
674–701. doi: 10.1111/modl.12339 10.1017/S0958344022000039
Vygotsky, L. S. (1984). “The problem of age” in Child Psychology. ed. R. W. Rieb (Moscow: Yang, X., Kuo, L. J., Eslami, Z. R., and Moody, S. M. (2021). Theoretical trends of
Pedagogical) research on technology and L2 vocabulary learning: a systematic review. J. Comput.
Educ. 8, 465–483. doi: 10.1007/s40692-021-00187-8
Vygotsky, L. S. (1986). Thought and language. Cambridge: MIT Press.
Zhang, Y. (2022). Construction of English language autonomous learning center
Wang, Y. (1999). ‘College English’in China. Engl. Today 15, 45–51. doi: 10.1017/
system based on artificial intelligence technology. Math. Probl. Eng. 2022, 1–15. doi:
S0266078400010713
10.1155/2022/7055106
Wang, Z. (2022). Computer-assisted EFL writing and evaluations based on artificial
Zhang, R., and Zou, D. (2020). Types, purposes, and effectiveness of state-of-the-art
intelligence: a case from a college reading and writing course. Libr. HiTech 40, 80–97.
technologies for second and foreign language learning. Comput. Assist. Lang. Learn. 33,
doi: 10.1108/LHT-05-2020-0113
1–47. doi: 10.1080/09588221.2020.1744666
Wekke, I. S., Yandra, A., and Hamuddin, B. (2017). Learning strategy in class
Zheng, Y., and Cheng, L. (2008). Test review: college English test (CET) in China.
management: a reflection from Manado case. IOP Conf. Ser. Earth Environ. Sci.
Lang. Test. 25, 408–417. doi: 10.1177/0265532208092433
97:012053. doi: 10.1088/1755-1315/97/1/012053
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning.
Xia, Q., Chiu, T. K., Lee, M., Sanusi, I. T., Dai, Y., and Chai, C. S. (2022). A self-
J. Educ. Psychol. 81:329. doi: 10.1037/0022-0663.81.3.329
determination theory (SDT) design approach for inclusive and diverse artificial
intelligence (AI) education. Comput. Educ. 189:104582. doi: 10.1016/j. Zimmerman, B. J. (2002). Becoming a self-regulated learner: an overview. Theory
compedu.2022.104582 Pract. 41, 339, 64–70. doi: 10.1207/s15430421tip4102_2
Xu, Q., and Liu, J. (2018). A study on the washback effects of the test for English majors Zimmerman, B. J. (2013). From cognitive modeling to self-regulation: a social
(TEM): Implications for testing and teaching reforms. Berlin: Springer. cognitive career path. Educ. Psychol. 48, 135–147. doi: 10.1080/00461520.2013.
794676
Yamamoto, T., Liao, A., Wu, W. C. V., Shih, M., Shih, J. L., and Chu, H. C. (2018). A
proposal for the global and collaborative PBL learning environment where all global Zimmerman, B. J., and Moylan, A. R. (2009). “Self-regulation: where metacognition
members on different campuses are. In: 2018 conference on technologies and applications and motivation intersect” in Handbook of metacognition in education. eds. D. J. Hacker, J.
of artificial intelligence (TAAI). IEEE Computer Society, pp. 90–95. Dunlosky and A. C. Graesser (New York: Routledge), 299–315.
Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: an Zimmerman, B. J., and Schunk, D. H. (2011). Handbook of self-regulation of learning
exploratory investigation. Educ. Inf. Technol. 12, 1–25. doi: 10.1007/s10639-023- and performance. New York: Routledge/Taylor & Francis Group.
11742-4
Zou, B., Liviero, S., Hao, M., and Wei, C. (2020). “Artificial intelligence technology for
Yan, J., and Huizhong, Y. (2006). The English proficiency of college and university EAP speaking skills: student perceptions of opportunities and challenges” in Technology
students in China: as reflected in the CET. Lang. Cult. Curric. 19, 21–36. doi: and the psychology of second language learners and users. eds. M. R. Freiermuth and N.
10.1080/07908310608668752 Zarrinabadi, 433–463.

Frontiers in Psychology 14 frontiersin.org

Qiao and Zhao 10.3389/fpsyg.2023.1255594

Appendix
Treatment adherence observation checklist
Instructions for observers:
The following checklist was used to monitor the consistent and accurate delivery of the AI-based speaking intervention. Observers were
instructed to conduct unannounced observations at various points throughout the study to verify that the speaking activities were being
delivered in accordance with the intended design.
Checklist items:
Duration of speaking activity:
Met the prescribed time frame.
Deviated from the prescribed time frame (provide details):
Types of speaking activities:
Implemented the specified activities as outlined in the intervention protocol.
Deviated from the specified activities (provide details):
Nature of Feedback Provided to Learners:
Provided constructive and relevant feedback to learners.
Did not provide adequate feedback (provide details):
Additional comments:

Frontiers in Psychology 15 frontiersin.org