Impact of ChatGPT on Academic Performance of Undergraduate Students in Malaysia

Cheah Rin (SUKD2201368), Dharshini Jayasanker (SUKD 2101803), Kirithika Sri

Devan (SUKD2201560), Xia Fei (SUKD2102101)

Bachelor of Psychology (HONS), Segi University

Lecturer Name: Mr Lee Woon Ping

Due date: 19/11/2023

This study set out to find out how ChatGPT affects academic performance. One hundred

undergraduate psychology students from a private university participated in this study, which

used a single-factor between-subjects post-test design. Half of the participants used ChatGPT,

an AI, for their projects, while the other half were not allowed to utilize it. The results of both

student categories' academic achievement were recorded. The academic performance of the

AI group significantly declined, whereas the control group's performance increased,

according to the findings. This supported our hypothesis that the use of AI will lead to a

decrease in academic performance. In conclusion, AI-based activities could make human

academic performance poorer.

Keywords: Artificial intelligence, ChatGPT, Academic performance

1
 Table of Contents

Abstract................................................................................................................................................1
1.0 Impact of ChatGPT on Academic Performance of Undergraduate Students in Malaysia.3
1.1 Artificial Intelligence.................................................................................................................3

1.2 ChatGPT....................................................................................................................................4

1.3 Academic performance...............................................................................................................4

1.4 Theoretical framework..............................................................................................................5

1.5 Review of Past Literature..........................................................................................................7

1.6 Research Gap.............................................................................................................................8

1.7 Current Study............................................................................................................................9

2.0 Methodology.................................................................................................................................10
2.1 Research design........................................................................................................................10

2.2 Participants..............................................................................................................................10

2.3 Materials...................................................................................................................................11

2.4 Procedure.................................................................................................................................11

3.0 Result............................................................................................................................................13
4.0 Discussion.....................................................................................................................................16
4.1 Limitation and future research...............................................................................................18

4.2 Contribution and implication.................................................................................................18

5.0 Conclusion....................................................................................................................................19
References..........................................................................................................................................20

2
 1.0 Impact of ChatGPT on Academic Performance of Undergraduate Students in

Malaysia

This study had revealed how ChatGPT affects Malaysian undergraduate students' academic

achievement. The impact of using AI-integrated language in learning is examined. The

findings ought to encourage educational environments including government, academia, and

student bodies to think about the bad influence of using AI. Comprehending the effects of

artificial intelligence on students' academic performance holds great academic significance

and bears important ramifications for educational institutions, authorities, and society at

large.

1.1 Artificial Intelligence

In the contemporary globalization period, artificial intelligence (AI) is rapidly expanding and

is mostly heard among students and professors. Machine learning and artificial intelligence

are the two most powerful forces in today's world. Education is no exception to the growing

use of AI in all aspects of life. The ability of a computer or robot under computer control to

execute tasks normally carried out by intelligent living things is known as artificial

intelligence (AI).. (Perez, 2023) It's an algorithm with cognitive skills similar to those of

humans, including generalization, reasoning, meaning-finding, and experience-based

3
learning. (Akgun, 2022) AI is beneficial in numerous ways, such as individual learning and

assessment in real time, simpler lesson planning, deep virtual learning, and many more. Due

to the quick advancements in artificial intelligence (AI) and natural language processing

(NLP), language models are becoming more sophisticated and flexible. (Mathew, 2023)

1.2 ChatGPT

The internet has been captivated by ChatGPT, the latest craze that has amassed over a million

users in just one week of its launch. Its many features allow users to write poems, draft

letters, respond to exam-style questions, write essays, and respond to follow-up inquiries,

among many other things. Chatbots, or more particularly ChatGPT, that are powered by

artificial intelligence are among the most significant inventions made possible by this

technology. When the American corporation OpenAI launched it on November 30, 2022,

academics, journalists, and others were alarmed almost instantly because they believed it

would be hard to distinguish ChatGPT-generated text from human writing. (Gregersen, 2023)

The programme known as ChatGPT, or Chat Generative Pre-training Transformer, enables

users to pose inquiries to it in a conversational or natural manner. The amount of labour that

needs to be done by human intellect could be greatly decreased with ChatGPT. It can help

with a lot of things, such as writing essays, aiding students with tasks, answering exam-style

questions, creating contracts automatically, writing poems, and writing computer code, to

name a few.

1.3 Academic performance

4
Academic success is seen as a significant accomplishment for students throughout their

academic studies. Performance success has an impact on students' lives now and in the future.

(Kell, 2013). The research found that children's academic success is a net outcome from both

cognitive and non-cognitive qualities. (Khine, 2016). Teachers and school administration are

concerned that students are using ChatGPT to complete assignments and create papers in

violation of the honour code and plagiarism policies. It is almost impossible to distinguish

between ChatGPT and a real human assignment, which has raised worries. (Dragidella, 2023)

Similar to personal computers and graphic calculators, this has the potential to completely

transform education in the long run.

1.4 Theoretical framework

A complex and varied topic, the effect of ChatGPT on academic performance can vary

depending on how students use the technology and their unique situation. Cognitive and

learning theory include a wide range of cognitive functions and processes that people use to

gather, process, store, and use information. (MSEd, 2023) These abilities are essential for

efficient learning, solving problems, and general intellectual growth. Techniques and tactics

for gaining and keeping knowledge, such as taking notes, managing your time, and actively

listening, are examples of learning skills. Conversely, higher-order mental activities like

reasoning, information processing, memory, and critical thinking are referred to as cognitive

skills. (Stadler, 2016)

5
Low academic performance may be linked to the use of AI, such as ChatGPT, because to its

effects on learning and cognitive abilities. The excessive dependence of students on artificial

intelligence (AI) tools such as ChatGPT for homework assignments may result in a decrease

in the acquisition of critical cognitive and learning skills. They might grow reliant on the AI

to solve problems and retrieve information, which would impair their capacity for critical

thought and in-depth engagement with the subject matter.

Cognitive learning theory, however, covers wide ground. Several facets of cognitive learning

theory may be used as a framework to examine how ChatGPT and related technologies affect

academic achievement. Within the context of cognitive learning theory, the constructivist

concept is one relevant viewpoint.

Constructivism is a philosophy of learning that places an emphasis on how students actively

construct their own knowledge. (Constructivism, 2023) Constructivism is a learning method

that maintains that individuals actively create their own knowledge and that the learner's

experiences shape reality. (Elliott, 2000) The focus is on student-constructed knowledge via

engagement with material and active learning. With ChatGPT, for example, students may

interact with the model's comments to create meaning, but this will rely on how well the

technology is incorporated into the classroom.

Constructivism often emphasizes the creation of knowledge, teamwork, and active

participation. Hence, students may face several difficulties with their academic achievement

if they only duplicate ChatGPT's produced responses without going through more in-depth

thought processes.

6
Their academic performance may suffer if they just accept AI-generated stuff without

challenging or analysing it. Relying too much on AI to generate content or retrieve

information can impair students' ability to retain information and understand various courses.

They might put more emphasis on quick fixes than on long-term learning, which could hinder

their academic success on tests that are more difficult.

1.5 Review of Past Literature

A handful of researchers have previously studied how AI affects academic performance.

Among them is an article on ChatGPT's effects on Bangladeshi undergraduate students'

academic performance which was published in the International Journal of Research in

Science and Engineering in August 2023. This research investigates ChatGPT's effects on

Bangladeshi undergraduate students' academic achievement in relation to motivation,

innovation, and plagiarism. The study's two main hypotheses were that students' creativity is

impacted by plagiarism, which ChatGPT facilitates and there is a connection between

students’ motivation and creativity. They have concluded that the capacity of ChatGPT to

allow plagiarism lowers students' academic performance. The other hypothesis was also

accepted since academic performance and creativity are positively correlated .It also has a

significant negative impact on students' creativity since it makes using AI-generated content

easier and may cause users to rely too much on the tool to accomplish written assignments.

(Tanvir, 2023)

7
The next research that we looked onto is an article titled ‘The impact of Artificial Intelligence

in academia: Views of Turkish academics on ChatGPT’. This article aims to investigate the

effects of ChatGPT, an AI and machine learning technology, on the academic domain and

ascertain the opinions of academics about it. In order to do this, ten academics were

interviewed in-depth, and their opinions on the matter were examined. It is evident that

academics think ChatGPT will be a useful instrument for scientific research, instructional

procedures, and as a source of ideas for new study themes. Academics face ethical

challenges, including plagiarism and disinformation, in spite of these benefits. According to

the study, ChatGPT is seen favourably as a helpful tool for scientific research and teaching,

but ethical issues like plagiarism and disinformation need to be taken into consideration.

(Livberber, 2023)

1.6 Research Gap

There is much research on ChatGPT's versatility and ability to improve people's lives from all

walks of life, but none specifically examines ChatGPT's effects on Malaysian undergraduate

students' academic achievement. The study evaluates the impact of open AI in an effort to

reduce the likelihood that it will be abused, given that the majority of colleges have

implemented policies prohibiting its usage. The minister of higher education recently

approved the use of ChatGPT in the classroom, which has drawn our attention to this study.

While visiting Universiti Malaysia Sarawak (Unimas) yesterday, he told reporters, "We do

not reject it, but we need to know its limitations and so on." (Fam, 2023) This has been a

8
topic of much recent discussion among both students and educators. one of the justifications

for the investigation. Through this research, we were able to demonstrate to the students the

drawbacks of improper usage of ChatGPT. This research was necessary for us to distinguish

between the various chatgpt use methods.

1.7 Current Study

Even though many studies on the impact of AI on academic performance have been

published, there is a paucity of research on the impacts of ChatGPT on the academic

accomplishment of Malaysian students. This study is a replication to make sure the existing

research publications are valid. It's significant since ChatGPT is now among Malaysia's most

sophisticated writing tools. The purpose of our research is to find out how ChatGPT affects

academic achievement. "How does the quality and accuracy of information provided by

ChatGPT influence the academic outcomes and grades of students who use the tool?" is the

research question for this study. According to a prior study conducted among Bangladeshi

undergraduate students, the hypothesis that ChatGPT-facilitated plagiarism affects students'

creativity was approved. (Tanvir, 2023) Thus, the hypothesis of this study is that using

ChatGPT would have a negative impact on students' academic performance.

9
2.0 Methodology

2.1 Research design

This is a single factor between-subjects experimental study involving one independent

variable (IV) and one dependent variable (DV). The impact of artificial intelligence

(ChatGPT) is the independent variable, and the academic performance of undergraduate

students (CGPA) is the dependent variable. The DV was operationally defined as good

academic performance based on the overall CGPA score, measured using the usage of

ChatGPT, an artificial intelligence (AI) by undergraduate students.

2.2 Participants

One hundred undergraduate students (50 males, 50 females) were recruited from Segi

University. The minimum sample size was determined based on past research analysis which

indicated a minimum of 100 participants. The participants were aged around 18-25 years old

(M=3.38, SD=0.38). Students were recruited through a probability simple random sampling

method by posting the advertisement on the experiment in the university’s blackboard e-

learning portal. The participants were given credit for their participation. However, only

students who met the criteria of being age between 18 to 25 years old, currently studying as

undergraduate students, and going to sit for their final exam were recruited. Students were

10
recruited from psychology course. They will be sitting for their final exam at the end of the

semester. There were no other inclusion or exclusion criteria in this study.

2.3 Materials

In this experimental study, artificial intelligence (AI), ChatGPT was used to explore

the impact on the academic performance of undergraduate students by analyzing their CGPA

of an academic semester. ChatGPT is an advanced language model designed to provide

human-like responses for various purposes such as answering questions and providing

recommendations. To ensure a representative sample, a probability simple random sampling

method was used to capture a diverse range of experiences and perspectives. The

experimental group will receive access to an AI system, ChatGPT, and the control group will

not have access to any AI system.

2.4 Procedure

In order to tract the student's academic performance, we observed their online

assessments, physical presentation and physical exam throughout the semester. Participants

who signed up for the experimental portal were sent an email with their participant ID and

examination docket a week before the test. The test was held for four days consisting of four

different subjects in the Multi-Purpose Hall, Segi University. Additionally, the participants

were first required to read and indicate consent on the Informed Consent form, which was

administered to them using a Google form link. It was to ensure the participant's privacy,

11
confidentiality, purpose of the research, risk and benefits and the voluntary nature of

participation for this experimental study.

The experimental group was allowed to use ChatGPT to do their assignments,

presentations and to do revision throughout the semester. Meanwhile, the control group was

restricted from using any AI tools. It was to test the undergraduate psychology student's

academic performance with or without the help of artificial intelligence. The hypothesis of

this experiment was control group was more likely to obtain a higher CGPA compared to the

experimental group.

During the exam, all the students were given a similar set of questions. Each test

duration was four hours. The participants were instructed to complete all the questions within

the time given. When all the participants had submitted their test papers to the examiner, they

were thanked and allowed to leave the examination hall. All the responses were then marked

by the examiners. The overall result was calculated by combining the assignments,

presentations, and test marks gained by each individuals for a semester. The results were

tabulated by the impact of artificial intelligence, ChatGPT (Yes/No) and academic

performance (CGPA) of the undergraduate students.

12
3.0 Result

There were two groups in this experiment which were control group and treatment

group, each has 50 people in a group without gender specific. An independent-sample t-test

was run to determine if there were differences in CGPA to undergraduate students using

ChatGPT (treatment) and those didn’t used (control). The control group had mean of 2.97 and

standard deviation of 0.412 while treatment group had mean of 2.81 and standard deviation of

0.411 (table 1) which indicate that individual who used ChatGPT had a lower CGPA

compared to those didn’t used.

Control and treatment group were normally distributed, as assessed by Shapiro-Wilk's

test which is p>0.05 (table 2). There was also homogeneity of variance for both control and

treatment group, as assessed by Levene’s test which is p>0.05 (table 3). There was a

statistically significant difference in mean CGPA between control and treatment group, with

control group had higher CGPA than treatment group, 0.16 (95% CI, -0.32 to 0.01), t (98) = -

1.927, p = 0.057. (table 4)

Table 1

Group Statistics

ChatGP Std. Std. Error

T N Mean Deviation Mean

CGP ChatGP 50 2.8114 .41000 .05798

A T

Control 50 2.9698 .41197 .05826

13
 Table 2

Tests of Normality

ChatGP Kolmogorov-Smirnova Shapiro-Wilk

T Statistic df Sig. Statistic df Sig.

CGP ChatGP .096 50 .200* .964 50 .127

A T

Control .127 50 .044 .967 50 .176

*. This is a lower bound of the true significance.

a. Lilliefors Significance Correction

Table 3

Independent Samples Test

Levene's Test for t-test for Equality of

Equality of Variances Means

F Sig. t df

CGPA Equal .143 .707 -1.927 98

variances

14
 assumed

Equal -1.927 97.998

variances

not

assumed

Table 4

Independent Samples Test

t-test for Equality of Means

95%

Confidence

Interval of

the

Sig. (2- Mean Std. Error Difference

tailed) Difference Difference Lower

CGP Equal variances .057 -.15840 .08220 -.32152

A assumed

Equal variances not .057 -.15840 .08220 -.32152

assumed

15
4.0 Discussion

The purpose of this study was to investigate the effect of ChatGPT (a natural language

processing model) on student achievement, especially at the time of the examinations. By

collecting and analysing the data, we found that students who normally used ChatGPT

frequently performed significantly lower on the exam than those who did not use ChatGPT,

this has raised some concerns about the potential negative impact of technology on academic

achievement.

At the outset, our data demonstrated that the group of students who regularly used

ChatGPT (Experimental Group) were generally lower in grades than those who did not use

the AI tool (Control Group), which has a P-Value of 0.057. Although this level of significance

is only slightly above the traditional 0.05 threshold value, it can still be considered

significant. This further emphasizes the potential negative impact of ChatGPT on student

achievement. Based on our speculation, this group of students generally performed poorly,

which may be related to ChatGPT's usage habits. Students using ChatGPT tend to rely on the

tool to provide answers without actively thinking and solving problems as without proper

regulation, the chance of potential misused is high. (Enkelejda K. & et al., 2023) Besides, this

dependence may have led to their inability to cope with the difficulties they encountered

during the exam, affecting their performance.

Second, we believe that the use of ChatGPT may also result in students lacking the

necessary skills in learning and memorization. Students get answers by interacting with

ChatGPT, but they may not have the opportunity to understand and internalize the

knowledge. This may have a negative influence on their academic performance in the long

run, as they may be deficient in depth and breadth of knowledge. However, the impact on

learning may vary as it depends on individual profession. Research done by Pradhan S. K. &

16
et al. had shown logistic regression between impact of ChatGPT and effective learning, the

significant value was –0.525 and 0.34. The result was mixed; correlation analysis was further

conducted, and significant value was less than 0.05. The test did not categories individual

profession, thus impact of ChatGPT on learning may difference based on their profession.

Another key issue is that the use of ChatGPT may weaken students' problem-solving and

critical-thinking skills. Because they have easy access to answers, they may no longer feel the

need to think about the complexity of the problem or develop the ability to solve issues.

(Sayed F. A. & et al., 2023) According to research done by Kazi T. & et al., their result

supported our hypothesis. The overly dependent on ChatGPT negatively affect their

creativity, and the motivation of wanting to learn more which lower their problem-solving

skills. This is a troubling trend in the academic field, as developing critical thinking and

problem-solving skills is critical to academic success.

Furthermore, unreliable sources were also one of the problems from ChatGPT that may

cause lower CGPA among the students. (Kyoung Won S., 2021) ChatGPT is unable to

reference real-life documents, instead, it creates a fake reference. (Katie L., 2023) It raises

questions against the integrity of the information, since the reputation of ChatGPT is so

strong, students may believe everything they receive from ChatGPT. Students without fact-

checking information from ChatGPT may receive it as actual knowledge and apply in

examination which causes lower CGPA in the treatment group.

While our research revealed some of the possible detrimental effects of ChatGPT in the

academic arena, we do not advocate a complete ban on its usage. Instead, we believe it is

necessary to take steps to manage the use of ChatGPT to minimize its negative impact. This

may include clear rules for exams, encouraging students to engage in independent learning

and thinking, and providing more support and resources to help students improve their

academic skills.

17
 4.1 Limitation and future research

In discussing the limitations of this study, we must consider the constraints of the

experimental design. First, the sample size was small and not sufficiently representative of

the general student population. Secondly, the experiment was of limited duration and could

not capture the effects of long-term ChatGPT utilization. In addition, differences in subject

matter and grade levels may affect the effectiveness of using ChatGPT. Furthermore,

treatment groups were only instructed of using ChatGPT on homework and assignments,

without proper guideline, students mostly used it to find direct answers. Future research could

be done is having ChatGPT to assists students instead of spoon feeding them such as used on

summarize notes for easier revision and measure their academic performance.

4.2 Contribution and implication

From the experimental result, overly dependent on ChatGPT indeed could be detrimental

to academic performance which is consistent with previous research. The spoon feed method

of completing homework and assignments doesn’t provide students actual learning and

weaken their critical thinking skill. However, the significant value of 0.057 in T-test and

research by Pradhan & et al. indicate that ChatGPT have positive results to learning as well

and individual profession seems to be one of the factors. It is possible that ChatGPT enhance

learning experience of certain profession by translating difficult scientific words to simple

words. It would motivate students to learn more as they can understand easily which directly

improve their CGPA.

18
 An implication of the study’s finding is although ChatGPT can be used to provide

information quickly but couldn’t enhance student performance on examination. This indicates

the current education method should change and embrace ChatGPT as application of AI in

education is inevitable. Educators could incorporate ChatGPT as a way of self-learning and

teach them the proper way to use it. For example, fact check information from ChatGPT, it

improves student’s critical thinking skill as they must determine if the information is valid.

Homework and assignment may also implement more open-ended questions, ChatGPT is

unable give opinion, thus reduce its negative effects.

5.0 Conclusion

To summarize, the use of ChatGPT may have a negative impact on student performance,

especially at test time. However, the issue is complex and requires a combination of

academic, educational, and technical areas. Educators and students should wisely utilize this

technology in conjunction with traditional learning methods to maximize knowledge

acquisition and critical thinking. Future research could more fully explore the impact of the

ChatGPT on students across different profession and the way of using ChatGPT to better

understand its potential impact.

19
References

Ahmad, S. F., Han, H., Alam, M. M., Rehmat, M. K., Irshad, M., Arraño-Muñoz, M., &

Ariza-Montes, A. (2023). Impact of artificial intelligence on human loss in

decision making, laziness and safety in education. Humanities and Social

Sciences Communications, 10(1), 1–14. https://doi.org/10.1057/s41599-023-

01787-8

Akgun, S. &. (2022). Artificial intelligence in education: Addressing ethical challenges in K-

12 settings. . AI and ethics, , 431–440.

Constructivism. (2023, April 4). Office of Curriculum, Assessment and Teaching

Transformation - University at Buffalo.

https://www.buffalo.edu/catt/develop/theory/constructivism.html#title_205963

0958

Dragidella, K. (2023, June 12). ChatGPT and its impact on education. Crossplag.

https://crossplag.com/chatgpt-and-its-impact-on-education/

Elliott S. N. Kratochwill T. R. Cook J. L. & Travers J. F. (2000). Educational psychology :

effective teaching effective learning (3rd ed.). McGraw Hill, 256-267

Gregersen, E. (2023, November 18). ChatGPT | Definition & Facts. Encyclopedia

Britannica. https://www.britannica.com/technology/ChatGPT

20
Fam, C. (2023, June 14). Minister: Higher education ministry to allow ChatGPT use at local

universities, guidelines must be followed. The Star.

https://www.thestar.com.my/tech/tech-news/2023/06/14/minister-higher-

education-ministry-to-allow-chatgpt-use-at-local-universities-guidelines-must-

be-followed

Kalal, P. S., S, N. M., & Vijay, V. (2023, June 28). EFFECT OF CHATGPT ON STUDENTS

CREATIVITY FROM ACADEMICIAN’S AND STUDENTS PERSPECTIVE.

ResearchGate.

https://www.researchgate.net/publication/371912289_EFFECTOF_CHATGP

T_ON_STUDENTS_CREATIVITY_FROM_ACADEMICIAN

%27S_AND_STUDENTS_PERSPECTIVE

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser,

U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G.,

Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel,

T., & Stadler, M. (2023). ChatGPT for good? On opportunities and challenges

of large language models for education. Learning and Individual Differences,

103(102274). https://doi.org/10.1016/j.lindif.2023.102274

Kell, H. J., Lubinski, D., & Benbow, C. P. (2013). Who Rises to the Top? Early Indicators.

Psychological Science, 24(5), 648-659.

https://doi.org/10.1177/0956797612457784

21
Khine, M. S. (2016). Non-Cognitive skills and factors in educational success and academic

achievement. Non-cognitive Skills and Factors in Educational Attainment, 3–

9. https://doi.org/10.1007/978-94- 6300-591-3_1

Lai, K. (2023). How Well Does ChatGPT Handle Reference Inquiries? An Analysis Based on

Question Types and Question Complexities | Lai | College & Research

Libraries. Crl.acrl.org, 84(6). https://doi.org/10.5860/crl.84.6.974

Livberber, T., & Ayvaz, S. (2023). The impact of Artificial Intelligence in academia: Views

of Turkish academics on ChatGPT. Heliyon, 9(9), e19688.

https://doi.org/10.1016/j.heliyon.2023.e19688

Mathew, A. (2023). Is artificial intelligence a world changer? A case study of OpenAI’s chat

GPT. recent Progress in Science and Technology Vol.5, 35-42.

MSEd, K. C. (2023, April 18). Cognition in Psychology. Verywell Mind.

https://www.verywellmind.com/what-is-cognition-2794982

Perez, J. (2023, October 26). Artificial Intelligence (AI) in education: Impact &

Examples. QuestionPro. https://www.questionpro.com/blog/ai-in-education/

Seo, K., Tang, J., Roll, I., Fels, S., & Yoon, D. (2021). The impact of artificial intelligence on

learner–instructor interaction in online learning. International Journal of

22
 Educational Technology in Higher Education, 18(1).

https://doi.org/10.1186/s41239-021-00292-9

Stadler, M. A. (2016). Choosing between what you want now and what you want most: Self-

control explains academic achievement beyond cognitive ability. . Personality

and Individual Differences, 168-172.

Tanvir, K. &.-J. (2023). Impact of ChatGPT on Academic Performance among Bangladeshi

Undergraduate Students. . International Journal of Research In Science &

Engineering. , 18-28 .

23
24