617797
617797
MASTER OF ARTS
KARS – 2020
T.C.
KAFKAS ÜNĠVERSĠTESĠ
SOSYAL BĠLĠMLER ENSTĠTÜSÜ
BATI DĠLLERĠ VE EDEBĠYATLARI ANABĠLĠM DALI
ĠNGĠLĠZ DĠLĠ VE EDEBĠYATI BĠLĠM DALI
KARS – 2020
1
KAFKAS UNIVERSITY
APPROVAL
The acceptance of this thesis study was registered by the committee of Graduate
School of Social Sciences of Kafkas University in session …………. on
……/……../………..
I
KAFKAS ÜNĠVERSĠTESĠ
SOSYAL BĠLĠMLER ENSTĠTÜSÜ MÜDÜRLÜĞÜNE
ONAY
II
SCIENTIFIC ETHIC STATEMENT
I declare that I complied with the rules of academic and scientific ethics from the
proposal stage to the process of completion of the study titled “Academic Staff’s
Perceptions on Washback Effect of YDS and YÖKDĠL: The Case of Ardahan
University” as a Master Thesis I prepared, that I obtained all information in term
Project with the framework of scientific ethics and traditions, that I showed sources
to the each quotation I made directly or indirectly in this study I prepared as a term
Project in accordance with the writing rules and works which I used have been
shown in the bibliography.
02/01/2020
III
TABLE OF CONTENTS
ACKNOWLEDGEMENT .......................................................................................... i
ABSTRACT ................................................................................................................ ii
ÖZET.......................................................................................................................... iii
LIST OF TABLES .................................................................................................... iv
LIST OF FIGURES .................................................................................................. vi
LIST OF ABBREVIATIONS ................................................................................. vii
1. INTRODUCTION .................................................................................................. 1
1.1. Background to the Study ....................................................................................... 1
1.2. Statement of the Problem ...................................................................................... 2
1.3. Purpose and Significance of the Study.................................................................. 3
1.4. Limitations of the Study ........................................................................................ 3
2. LITERATURE REVIEW...................................................................................... 4
2.1. Testing ................................................................................................................... 4
2.1.1. High-stakes Tests ............................................................................................... 5
2.1.2. Low- stakes Tests ............................................................................................... 5
2.2. Washback Effect ................................................................................................... 6
2.2.1. Positive Washback ............................................................................................. 9
2.2.2. Negative Washback Effect ............................................................................... 10
2.3. Washback Effect Studies..................................................................................... 11
2.4. Foreign Language Proficiency Exams in Turkey................................................ 14
2.4.1. KPDS-ÜDS ...................................................................................................... 15
2.4.2. YDS and YÖKDĠL........................................................................................... 16
2.4.2.1. YDS ............................................................................................................... 17
2.4.2.2. YÖKDĠL ....................................................................................................... 17
3. METHODOLOGY............................................................................................... 20
3.1. Introduction ......................................................................................................... 20
3.2. The Setting and Participants................................................................................ 20
3.3. Research questions .............................................................................................. 26
3.4. Data Collection Procedures ................................................................................. 26
3.5. Data Analysis ...................................................................................................... 27
4. RESULTS ............................................................................................................. 28
IV
4.1. Introduction ......................................................................................................... 28
4.2. Quantitative Results ............................................................................................ 28
4.3. Qualitative Results .............................................................................................. 40
4.3.1. Academic Staff‟s Perceptions About Foreign Language in General ............... 40
4.3.2. Academic Staff‟s Perceptions About YDS and YÖKDĠL Exams ................... 41
4.3.3. Academic Staff‟s Perceptions About the Effects of YDS and YÖKDĠL on
Academic Context ...................................................................................................... 43
5. DISCUSSION AND CONCLUSION ................................................................. 45
5.1. Introduction ......................................................................................................... 45
5.2. Discussion ........................................................................................................... 45
5.2.1. What do academic staff think about the content of YDS and YÖKDĠL? ........ 45
5.2.2. What do academic staff think about the effect of YDS and YÖKDĠL?........... 46
5.3. Conclusion .......................................................................................................... 47
5.4. Suggestions for Further Studies .......................................................................... 48
REFERENCES ......................................................................................................... 49
APPENDICES .......................................................................................................... 54
Appendix A: ............................................................................................................... 54
Appendix B: ............................................................................................................... 58
Appendix C: ............................................................................................................... 62
Appendix D: ............................................................................................................... 63
Appendix E: ............................................................................................................... 64
CURRICULUM VITAE .......................................................................................... 65
V
ACKNOWLEDGEMENT
I would also like to express my sincere appreciation to jury members Assist. Prof.
Dr. Turan Özgür GÜNGÖR and Asst. Prof. Dr. Oktay YAĞIZ for their
recommendations, comments, and feedback.
I want to express my thanks to Prof. Dr. Mustafa ÖZDEMĠR for his valuable
contributions throughout MA degree program.
I am grateful to my parents who have never failed to support me. I also thank my
parents-in-law for their encouragement.
Special thanks go to my husband Doğan SALTAġ for his continuous support, love,
and encouragement. I could not complete this study without his help.
i
KAFKAS UNIVERSITY
THE INSTITUTE OF SOCIAL SCIENCES
DEPARTMENT OF WESTERN LANGUAGES AND LITERATURE
DIVISION OF ENGLISH LANGUAGE AND LITERATURE
ACADEMIC STAFF’S PERCEPTIONS ON WASHBACK EFFECT OF YDS
AND YÖKDĠL: ARDAHAN UNIVERSITY CASE
Emine KASAP SALTAġ
MASTER’S THESIS
Supervisor
Prof. Dr. Gencer ELKILIÇ
2020 – VII + 65 PAGES
ABSTRACT
The aim of the study is to investigate the perceptions of academic staff on washback
effect of YDS and YÖKDĠL exams. The study was conducted at Ardahan University.
Both quantitative and qualitative methods were used to collect data. The
questionnaire was applied to 158 academic staff working at Ardahan University and
for the qualitative data, focus group interview was administered and the statements of
the participants were analysed according to the descriptive analysis procedure. SPSS
version 16.0 was used to analyse the quantitative data. According to results of the
study, perceptions toward YDS and YÖKDĠL exams are negative. Since YDS and
YÖKDĠL do not include speaking, listening and writing, academicians do not trust
the validity of these exams. It was also concluded that academics have negative
feelings on content and washback effects of YDS and YÖKDĠL. Although the
academicians believe in the necessity of foreign language proficiency in Turkey, they
do not think that these exams help candidates to be competent in the target foreign
language.
Key Words: YDS, YÖKDĠL, washback effect, academic staff, Ardahan University.
ii
KAFKAS ÜNĠVERSĠTESĠ
SOSYAL BĠLĠMLER ENSTĠTÜSÜ
BATI DĠLLERĠ VE EDEBĠYATLARI ANABĠLĠM DALI
ĠNGĠLĠZ DĠLĠ VE EDEBĠYATI BĠLĠM DALI
AKADEMĠSYENLERĠN YDS VE YÖKDĠL SINAVLARININ
ETKĠSĠNE YÖNELĠK ALGILARI: ARDAHAN ÜNĠVERSĠTESĠ ÖRNEĞĠ
Emine KASAP SALTAġ
YÜKSEK LĠSANS TEZĠ
DanıĢman
Prof. Dr. Gencer ELKILIÇ
2020- VII + 65 SAYFA
ÖZET
Anahtar Kelimeler: YDS, YÖKDĠL, ket vurma etkisi, akademik personel, Ardahan
Üniversitesi.
iii
LIST OF TABLES
iv
Table 21. All the questions of YDS and YÖKDĠL are related to each other in a
structural integrity ...................................................................................................... 32
Table 22. YDS and YÖKDĠL are necessary exams for the professional development
of all academics regardless of their field.................................................................... 33
Table 23. Since listening questions are not asked in YDS and YÖKDĠL, listening-
comprehension skills in foreign languages are not given importance ....................... 33
Table 24. YDS and YÖKDĠL make the candidates feel that learning a foreign
language is not difficult .............................................................................................. 34
Table 25. Frequently asking grammar questions in YDS and YÖKDĠL causes
foreign language learning to be perceived as grammar learning ............................... 34
Table 26. Before and after YDS and YÖKDĠL, candidates experience anxiety due to
the exam ..................................................................................................................... 35
Table 27. For YDS and YÖKDĠL, some tactics and test strategies need to be known
.................................................................................................................................... 36
Table 28. YDS and YÖKDĠL create a desire to learn foreign languages more
intensively for the candidates who will take the exam .............................................. 36
Table 29. Because of the name YDS and YÖKDĠL, it creates an unwanted (negative)
perception on the candidates ...................................................................................... 37
Table 30. YDS and YÖKDĠL affect foreign language learning negatively .............. 37
Table 31. Since advanced vocabulary is required in YDS and YÖKDĠL, vocabulary
memorization is a common study method ................................................................. 38
Table 32. YDS and YÖKDĠL effect foreign language learning positively ............... 38
Table 33. The candidate studying for YDS and YÖKDĠL will be more successful in
using that foreign language ........................................................................................ 39
Table 34. In YDS and YÖKDĠL preparation, it is useful to hire a tutor to teach test
techniques................................................................................................................... 39
v
LIST OF FIGURES
vi
LIST OF ABBREVIATIONS
MA : Master of Arts
vii
1. INTRODUCTION
This chapter covers background to the study, statement of the problem, purpose and
significance of the study and limitations.
English, as a global language, has been dominating the world with its power in
social, economic and academic life. Especially, it is an essential language in most
professions when compared with other languages. English has become an inseparable
part of our daily lives thanks to its connection to every social and professional field.
If one wants to be aware of what is happening in the world, then it becomes a must
for him/her to know a foreign language, most importantly English as the global one.
Turkey has been giving much emphasis on English teaching for many years. From
1997 to 2013, primary school students used to begin learning English in the fourth
year. But, English has started to be thought in Turkey as a foreign language starting
from the second year of primary school to university within an educational reform by
the state since 2013. The purpose of this reform is to improve language acquisition
with all skills.
English is taught in Turkey as a foreign language starting from primary school untill
university. Even though the lesson hours spent on English vary from one school to
another according to the types of schools (i.e. Anatolian high schools, Ġmam Hatip
high schools, etc.), English is offered as a compulsory course, not selective.
Therefore, students are expected to develop their proficiency in English as the years
pass.
Beside learning a foreign language at school, students are given exams by their
teachers in order to be assessed in terms of lingual competence. This is a common
process in all kinds of teaching activities.
1
This exam process, either the beginning or the end, affects students positively or
negatively which in result affects the educational process as well. This effect of
exams, which can be either, positive, negative, even weak or strong are called
washback or backwash (Biggs, 1995). Terms like feedback, impact, effect can also
be used instead of the washback term. These terms mainly cover the field of
language testing.
Alderson and Wall (1993) state that both backwash and washback terms can be used
with the same meaning. Besides, Pan (2009) says that researchers use both terms to
define the outcomes or effects of the examinations.
The Foreign Language Examination for Civil Servants (KPDS) and the Inter-
University Foreign Language Examination (ÜDS) were administered by ÖSYM until
2013. In January 2013, ÖSYM announced that these two exams would replace with
YDS and it was planned to be the only language exam officially administered. While
the content didn‟t change, it continued to measure the language proficiency levels of
test takers.
Although English is taught in Turkey as a foreign language from the second year of
primary school untill university, it is believed that foreign language education lags
2
behind European standards. To solve the problem, reforms have been made like
changing curriculum, providing a lot of materials, increasing the number of foreign
language teachers. However, this is not enough to eliminate the problem. Today,
even the academic staff in Turkey have difficulties and anxiety about foreign
language exams which is a must to get a new degree or be promoted. Trying to get
prepared and spend extra time on a language exam, as it is not believed to be taught
efficiently before, academics in Turkey seem to tackle psychologically mostly during
this exam preparation or entrance process.
The aim of the study is to search the washback effects of Foreign Language Exam
(YDS) and Higher Education Institutions Foreign Language Examination
(YÖKDĠL) conducted in Turkey to assess foreign language levels, and discover out
the impacts of these exams on academic staff in Ardahan University. The main
purpose of this study is to investigate and analyze the opinions of academics about
the washback effect on foreign language exams currently held in Turkey known as
YDS and YÖKDĠL.
Although there are some studies about the problem of foreign language testing, there
are not enough studies investigating the washback effects of these two newly
designed and organized exams (YDS and YÖKDĠL). As these exams can be regarded
as new in Turkey, this study will help scholars studying in this field to get in-depth
data. Besides, this study will also contribute to the existing literature with more
recent data.
There are two limitations of the present study, first of which is that the study is
limited to only Ardahan University with 158 academic personnel. Therefore, the
findings cannot be generalized to all populations. This research is also limited in
terms of participants who took any foreign language exam before.
3
2. LITERATURE REVIEW
This chapter presents the conceptual framework of testing, high-stakes test, and low
stakes test, washback effect, positive and negative washback effect, washback effect
studies. Finally, foreign language proficiency exams in Turkey, KPDS and ÜDS,
YDS and YÖKDĠL will be high lightened.
2.1. Testing
Tests have been significant tools that present information about educators‟ teaching
methods and learners‟ performances and skills in the foreign language taught. The
instructors can assess their teaching performances whether they are effective or not.
In addition, teachers can evaluate the learners‟ language proficiency according to the
test scores. Sheng-ping and Chong-ning (2004) describe tests as “the main sources
for both teachers and students to get feedback, which enables them to reflect on their
teaching and learning activities and thus to help improve the jobs of both sides”
(p.1).
There are a lot of English language tests all around the world. TOEFL (Test of
English as a Foreign Language), IELTS (International English Language Testing
System) and PTE (Pearson Test of English-Academic) are the most popular language
tests in the English language. All these international tests are also valid and officially
accepted in Turkey. Furthermore, Turkey has its national foreign language tests;
4
YDS, e-YDS, and YÖKDĠL which are currently conducted and valid. According to
YÖK (Council of Higher Education), participants who want to be accepted MA or
PhD degree programs, assistant professors who want to be promoted to associate
professorship or people who want to get an academic position at a university have to
take one of these exams with a minimum score ranging according to qualification
conditions for each position or program.
Coombe, Folse and Hubley (2007) defined high-stakes tests as the tests that have an
important impact on test takers lives on a large scale. According to Madaus (1988),
tests are called as high-stakes when the scores of these tests or the tests themselves
are used for important decisions or positions in test-takers‟ lives. Cizek (2001), and
similarly Resnick (2004) described the high-stakes tests term as “the exams which
students need to pass in order to finish a school, to enter an education program, to
attend a university, to be able to get a scholarship, or to obtain proficiency for an
application” (as cited in KumandaĢ & Kutlu, 2010 p.759). Casbarro (2005) similarly
explains high-stakes tests as “the exams or tests, whose results are very important for
the examinees, and which have high concern and anxiety on them” (p.37). And
Bachman and Palmer (1996) characterized the high-stakes test as “high-stakes
decisions are likely to have a major impact on the lives of individuals or programs”
(p.147). Taking all these literature into account, the tests conducted by ÖSYM in
Turkey can be regarded as examples of high-stakes tests as the scores of these tests
have a crucial effect on test-takers academic or professional lives.
Cole (2007) defined the term low stakes tests as whose outcomes have almost no
significant effect on the test takers, instructors, and administrators. Bachman and
Palmer (1996) stated that “low stakes decisions have relatively minor impact on the
lives of relatively small numbers of individuals or small programs and can be
reversed quite easily” (p.5). Roever (2001) states that low stakes tests include
teacher-made tests, quizzes or exams being held at the end of the year. Such tests are
prepared and conducted by teachers in the classroom (McKay, 2006, p. 5).
5
Instructors utilize these tests to instructional plans, to control students learning and to
place test takers in a special program (Goertz & Duffy, 2003).
The term washback has been defined in a different ways stressing similar meaning by
scholars. As Spratt (2005) points out, „washback‟ is used in the literature with
various meanings, which reveal differences in scope, actor and intentionality (p.8).
Washback and backwash are used as the same meaning referring to the effects of
testing on learning and teaching a foreign language (Alderson & Wall, 1993).
Similarly, Buck (1988) identifies washback term as “a natural tendency for both
students and teachers to tailor the classroom activities to the demands of the test,
especially when the test is particularly important for test takers” (p. 17). Dorobat‟s
(2007) statement is similar, as he states that “washback is the effect of a test has on
teaching in the classroom” (p. 29). With a close explanation of what Buck says;
Bailey (1996) similarly describes the term as the impact of testing on learning and
teaching.
Alderson and Wall (1993) take the term from a different perspective by stating “the
extent to which the test influences language teachers and learners to do things that
they would not necessarily otherwise do” (p.2). Some other researchers such as
Spolsky (1994), Alderson and Hamp-Lyons (1996), Bachman and Palmer (1996),
Shohamy, Donitsa-Schmidt and Ferman (1996), Watanabe (2004), McNamara
(2000) and Hughes (2003) state that washback is the effect of testing on teaching and
learning. Green (2013) also describes the washback term as “the interaction between
tests, teaching and learning” (p.48). The term „washback‟ is more commonly used in
applied linguistics, language education, and language testing and „backwash‟ is used
in the general education field (Hughes, 1989).
For the last decades, high-stakes tests have increased in terms of numbers and which
eventually resulted in gaining importance as well. That means these tests have
become an important step for test takers as they have multiple effects in different
circumstances. For instance, most of the institutions ask the individuals for a
language certificate of qualification. And these tests are still among the requirements
to take advantage of employment and position promotion in many countries.
6
Hughes (1989) categorizes the components of the washback process in three phases;
participant, process and product. The test-taking process has an effect on test takers
teaching and learning strategies. Having the vital importance on test-takers‟ future
plans, the participants, in other words, the test-takers, will plan their study process
accordingly. Markee (1997), in his study, illustrated a model which consisted of
participants, process, innovation, context, duration, rationale and the strategies used
in innovation.
The scholars studying language testing as a research field, have sympathized the
washback term with great interest, but they used the terms with different names with
the same aim in terms of meaning. For instance, while Bachman and Palmer (1996)
name it as test impact, Hughes (2003) prefers to use the term as backwash. Similarly,
Karabulut (2007) found out that backwash is used in the educational field where the
effects of tests on learning and teaching are investigated, and washback used in the
Applied Linguistics field in general. That means, it is not abnormal to use two terms
interchangeably as they belong to the same research field. However, the impact term
is generally used by scholars in education and educational assessment apart from the
term washback. So, it can be stated that the impact term is used in the general aspect
while washback is used limitedly in terms of scientific study fields. For an in-depth
comparison between two terms, it can be said that the washback term has more to do
with the classroom, teaching process and effect of these three phases into teaching.
Nevertheless, many scholars take impact term as the effect of tests on policies,
society, people and practices in general (Bachman and Palmer, 1996; McNamara,
2000; Taylor, 2005). Moreover, Alderson and Wall (1993) use washback terms in a
narrower meaning, referring it as the descriptive outcome of the teaching outcomes
and addressing the use of the term only in classroom activities like the beliefs and
perceptions of students towards tests.
Bachman and Palmer (1996) state that the testing process has two levels of impacts,
micro, and macro. The micro impact includes the effect of tests on only teachers and
students, while macro impact covers the effect of tests on the educational system
including policies, application and the society whole test taker population. Regarding
the micro level impact, washback was defined by Buck (1992) as the effect of the
testing process on teachers and students in the classroom. Similarly, Pearson (1988)
7
states that the tests affect the teaching and learning process, which emphasizes the
micro level impact of washback as well. However, when it comes to defining the
macro level impact Cohen (1994) explains it briefly as “how assessment instruments
affect educational practices and beliefs” (p.41). Moreover, Pierce (1992) uses the
term “the impact of a test on classroom pedagogy, curriculum development, and
educational policy” (p.687).
After implementing a course, the examination is carried out, this is the case for
public examinations as well, but the flow of the influence is in backward positions
which means that these exams affect motivation, behaviour, and attitudes of learners,
teachers and parents at the end of the course taking process (Pearson, 1988). This
backward effect can be seen in figure 1 as follows;
As shown in the Figure 1, the direction of the washback effect is in backwards. But
with the time the direction of the effect is expected to be in forward as shown below.
The public exams have important outcomes as they are used for different purposes
like acquiring a position, degree or payment. These exams are also called high-stakes
tests as they have vital importance for test-takers in their future plans which may
include important steps like getting a job, being promoted or getting a degree as
8
mentioned before. Qi (2005) states that these high-stakes public examinations cause
washback as they have an influence on the education process in general.
The effect of high-stakes tests on language teaching and learning process has always
been clear. As the washback issue is a complicated process, many researchers claim
that the impact of the test is either positive or negative.
Bachman (1990) states that “positive washback would result when the testing
procedure reflects the skills and abilities that are taught in the course” (p.283). Gates
(1995) claims, „if washback is strong, students and teachers will tend to alter their
classroom behaviours in order to achieve good marks in the test‟ (p. 101). In contrast,
a weak washback occurs when a test is not an important one and its impact is little. If
the test and the course objectives fulfill each other, positive washback occurs, if vice-
versa happens, negative washback becomes inevitable.
9
In another study, Hughes (2003) examined the effects of an English proficiency test
at Boğaziçi University in Ġstanbul. He discovered that the test has positive washback
as it has encouraged students to study more to be competent in terms of English
Language proficiency.
Alderson and Wall (1993) define negative washback as “the undesirable effect on
teaching and learning of a particular test deemed to be poor” (p.6). In other words, in
consequence of teaching and learning process, unintended effects occur. According
to Pearson (1988), negative washback happens if a test doesn‟t cover learning
principles and course objectives with regard to the test. For instance, the content of
the test and testing strategies contrast with the goal of the course. Alderson and Wall
(1993) state that “negative washback effect can be observed in circumstances in
which learners do not want to learn and teachers do not want to teach.” (p.29).
Vernon (1956) (as cited in Demirogları, 2014 p.18) claims “teachers tend to ignore
subjects and activities that are not directly related to passing the exam, and tests
change the curriculum in a negative way”. Wiseman (1961) reports that “paid
coaching classes, which are intended for preparing student for exams, are not a good
use of the time because students are practicing exam techniques rather than language
learning activities” (p. 9), and Davies (1968) backs up Wiseman‟s and Vernon‟s
thoughts that “testing devices had become teaching devices that teaching and
learning are effectively being directed to past examination papers, making the
educational experience narrow and uninteresting.” (p.9).
Pearson (1988) points out that washback either positive or negative is related to the
quality of the test. He proposes that the washback effect of a test will be negative in
the event that it does not mirror the learning principles and course goals to which it is
dealing with. However, the washback effect will be positive when the impacts are
valuable and support the entire changes aimed. Nevertheless, Alderson and Wall
(1993) affirm that the nature of the washback effect may not have anything to do
with the quality of a test. Along with these lines, any test, positive or negative, can be
said to have beneficial or harmful washback related to the curriculum. Davies (1985)
10
claims that a good test must be “an obedient servant of teaching” so that it may result
in positive washback (p.10).
Yıldırım (2010) carried out a study on the Washback Effects of English Section of
the University Entrance Exam on candidate English Language Teachers. He
discovered that the test-taker students spent most of their time in English classes, but
they rarely or never interested in writing, reading and speaking skills as the exam
consists of multiple choice tests with grammar, vocabulary and reading
comprehension in the foreign language.
There are number of studies conducted on the washback effect on foreign and second
language learning. Yıldırım (2010) studied students‟ and teachers‟ teaching and
learning practices in the preparation process for ECFLUEE (English Component of
the Foreign Language University Entrance Exam). The purpose of the study was to
discover EFL (English as a Foreign Language) students‟ and teachers‟ perspective on
ECFLUEE. Yıldırım (2010) in this study, found out that ECFLUEE had a negative
washback effect on students and instructors. The exam focused on test takers
reading skill, grammar and vocabulary knowledge but writing, speaking and listening
skills were neglected. And these students had trouble in speaking and writing English
when they attended the university.
11
money on KPSS course, because of their professional future plans which they aimed
at first. Furthermore, with regard to the results gained from the study, students who
were both studying KPSS exam and who have academic future career plans were
discovered to have no time and effort to focus more on academic exams to be
accepted for MA Degree programs.
Özmen (2011b) also studied the probable washback effect of ÜDS (Inter-University
Foreign Language Examination) in Turkey. He wanted to learn how candidate
academicians and graduate students‟ language learning processes were effected by
ÜDS. It was found out that ÜDS had a negative washback effect on test takers‟
foreign language learning. The participants thought that ÜDS was a procedure in
order to attend a graduate degree program or to be promoted at the university.
Besides, test takers added that ÜDS was like game on vocabulary which needs
concentration on just vocabulary and grammar. The results also showed that the
exam has both a micro and macro level effect depending on the size, while micro
level emphasizing the effect on individual, macro level as the effect on large
population as academic staff working at universities.
Yavuzer and Göver (2012) studied the academics‟ language proficiency and their
attitudes on foreign language exams (KPDS and ÜDS) at NevĢehir University in
Turkey. The aim of the research was how Turkish academic staff were affected by
KPDS and ÜDS. The results of the study showed that academics were negatively
affected by KPDS and ÜDS as these exams caused them to focus only at language
learning apart from their scientific studies. In the study, the participants stated that
they couldn‟t focus on their academic studies since they were expected to pass the
language exams to have degrees or promotion. Similarly, KPDS and ÜDS caused
academics to worry about their future career plans. The exams just evaluated
participants‟ vocabulary, grammar and reading skills and listening, speaking and
writing skills were not tested or scored. Considering all these, it could be said that
KPDS and ÜDS had a negative washback effect on academic staff studying in
Turkey although it is not generalizable to all universities in the country.
12
was that KPDS and ÜDS had a positive washback effect on reading skills of test
takers. Since these exams required participants to study reading skills and knowledge
of grammar more rather than other skills, test takers focused only on developing their
reading skills to get high scores. Moreover, as these exams only focused on receptive
skills like grammar and reading, it was observed that there was negative washback
effect on test-takers with regard to productive language skills like writing, speaking
and receptive skill; listening which is not tested at these foreign language exams as
well. The results and implications were in line with Yavuzer and Göver‟s studies
(2012).
Kılıçkaya (2016) investigated the washback effect of the foreign language section of
TEOG (Transition Examination from Primary to Secondary Education). The purpose
of the research was to examine the negative or positive washback effect of TEOG
and 8th grade instructors teaching methods. The content of TEOG influenced widely
language teaching practices in the school. Additionally, TEOG has a fundamental
negative washback effect on writing, speaking and listening skills which were not
tested. Moreover, TEOG affected course book choice, learning environment,
classroom evaluation and students‟ apprehension. Also, the students were placed to
high schools according to scores of this exam. This situation made a negative
washback effect on test takers as well as teachers, parents and managers all. The
results of the research showed that there was a need for a change in TEOG and the
practises of the exam at that time as it had negative washback effect on students at a
nation-wide level.
Sayın and Aslan (2016) examined the washback effects of the language section of
LYS exam on ELT (English Language Teaching) undergraduate students. The
researchers discovered that listening, speaking and writing skills were not tested or
scored as the exam consisted of only grammar and reading skills which resulted in
giving less emphasis on speaking and writing instead of all skills of the foreign
language learning process. Sayın and Aslan (2016) also emphasized that “for the
correct acquisition of language four skills must be included in the courses and
assessment tests in the same proportion” (p.38). The test takers concentrated
distinctly on grammar and vocabulary and most of them didn‟t feel well qualified
13
when they began the university. So LYS seemed to have a negative washback effect
on students.
Külekçi (2016) studied possible washback effects of YDS (Foreign Language Exam).
In the study, it was found that there was a positive washback effect as YDS
encouraged participants to study more so as to be promoted or to get extra money in
their salaries according to scores. However, the test was found to have negative
washback effect on test takers as they spent most of their time studying exam and
multiple choice techniques. Külekçi (2016) claimed that “the construct analysis of
YDS-English reveals that the test mainly focuses on grammatical and textual
knowledge more than pragmatic knowledge, and by its nature it neglects
communicative and productive language abilities of test takers” (p.311). Since
ÖSYM shared a limited number of the questions (with a percentage of 10), the
researcher couldn‟t evaluate all the questions in the exam that they answered.
14
Turkey are YDS (Foreign Language Examination), e-YDS, and YÖKDĠL (Higher
Education Institutions Foreign Language Examination). These exams are conducted
by ÖSYM (Center for Assessment Selection and Placement).
2.4.1. KPDS-ÜDS
Until 2013, ÖSYM was administrating two different language proficiency tests: The
Foreign Language Examination for Civil Servants (KPDS) and the Inter-University
Foreign Language Examination (ÜDS). In January 2013, ÖSYM issued a press
release announcing that it would not continue offering these two proficiency tests
(i.e., KPDS and ÜDS), but administrate only the Foreign Language Examination
(YDS) to measure test takers‟ English language proficiency levels (ÖSYM, 2019).
KPDS was conducted to determine the level of foreign language knowledge of the
civil servants who aim to assess foreign language proficiency. It was published in the
Official Gazette in accordance with ÖSYM that the exam was going to be held twice
a year in May and November in 16 different cities, more than 20 languages. The
exam consisted of 100 multiple choice questions until 2011, and after spring 2011,
the test included 80 multiple choice questions. The exam included grammar,
vocabulary, reading skills and translation questions. Listening, speaking and writing
skills were not measured in the exam. Depending on the score achieved,
academicians had a raise in their salaries (ÖSYM,2019).
ÜDS was a national high-stakes academic English test that was developed primarily
for individuals holding a PhD degree and aiming to apply for associate professorship
examination that is offered by Inter-University Board under the auspices of Council
of Higher Education (ÖSYM, 2019). It was offered in three languages, English,
French and German. ÜDS may also be taken by candidates who want to get a
graduate degree in national universities and institutions. In practice, test takers were
dominantly academic staff of universities who either want to start a PhD program or
aim to apply for associate professorship. ÜDS was conducted biannually, in spring
and in fall terms since 2000. The test was conducted in 17 different cities. Also, the
test had three different versions that are designed for three major fields of study;
Social Science, Natural and Applied Sciences and Medicine Science.
15
Of all the aforementioned national foreign language high-stakes exams, KPDS and
ÜDS were the most preferred ones in Turkey. Most common use of these two
examinations for academic personnel could be divided into four categories; first of
which was; using the scores of these exams to attend a PhD program. Universities in
Turkey required students to get at least 55 points to be admitted as a PhD candidate.
Secondly, these exams were used as a prerequisite for the academic promotion.
Assistant professors in Turkey were supposed to get at least 65 points (since 2018,
minimum 55 points) to be promoted, which was one of the several requirements of
being promoted to associate professorship. Thirdly, when applying to a master‟s
degree program, applicants were also asked to provide their scores of these exams.
Although there was no minimum limit, the higher scores the candidates had from
these exams, the more chance they would have for admission. Finally, the staff who
work for the government were supposed to attend to KPDS to get some amount of
raise in their salaries depending on their foreign language proficiency level.
The content and format of both KPDS and ÜDS were almost similar but there were
still a few differences between them. There were 80 questions both in KPDS and
ÜDS. The time given for both of the exams was 180 minutes. KPDS consisted of one
type of exam whereas ÜDS was divided into three majors: Social Science, Natural
and Applied Sciences and Medicine Science. KPDS and ÜDS shared eight main
parts under which the types of questions fitted: Vocabulary, grammar, sentence
completion, reading comprehension, translation, dialogue, paragraph completion and
irrelevant sentence. However, there were two parts which only exist in KPDS, which
were situational response and paraphrasing the sentences.
YDS (Foreign Language Examination) and YÖKDĠL (Foreign Language Exam for
Higher Education Institutions) are the current foreign language proficiency exams
held in Turkey. With the changes in policies and laws, the names of the exams have
changed but the content of these exams remained nearly unchanged.
16
2.4.2.1. YDS
Since 4th January 2013, YDS has been started to be conducted in Turkey with the
"Regulations on Procedures and Principles about Foreign Language Placement in
Turkey” which was published in Official Gazette numbered 28518 in the 2 nd item of
delegated legislation numbered 375. YDS is administered twice a year in autumn
(September) and spring (April) as a paper based form in 47 different cities by ÖSYM
(Student Selection and Placement Center) in Turkey. For two years, YDS has been
implemented three times a year. According to regulation; 2019-YDS/1 Exam is held
German, Arabic, Albanian, Bosnian, Bulgarian, Chinese, Danish, Armenian, Persian,
French, Georgian, Dutch, English, Spanish, Italian, Japanese, Korean, Polish,
Hungarian, Portuguese, Romanian, Russian, Serbian, Ukrainian and Greek, 2019-
YDS/2 exam is held in German, Arabic, French, English and Russian. 2019-YDS/3
exam is held only in English. In German, Arabic, Bulgarian, Persian, French,
English, Spanish, Italian, Russian and Greek languages are prepared in a multiple
choice exam format with 80 questions.
For these languages, the exam includes vocabulary, grammar, translation and reading
comprehension questions. Albanian, Bosnian, Chinese, Danish, Armenian, Georgian,
Dutch, Japanese, Korean, Polish, Hungarian, Portuguese, Romanian, Serbian,
Ukrainian language tests are scored via translation into Turkish and from Turkish to
foreign language with approximately 200 words written by the test-takers. And the
results are evaluated by the academic jury determined by ÖSYM. Ever since, from
September 2014, once in every month, that is 12 times a year, a computer-based
language test (e-YDS) is applied/held. In the beginning, the exam was conducted
only in English, currently it is conducted in 6 different languages (German, Arabic,
French, English and Russian and Persian). e-YDS is held at the OSYM Electronic
Exam Centers (e-Exam Center) in Ankara, Ġstanbul and Ġzmir. e-YDS is equal to
YDS. The candidates who have entered E-YDS have the same rights earned by
candidates entering YDS (ÖSYM,2019).
2.4.2.2. YÖKDĠL
17
University, Anadolu University and the Council of Higher Education. The Higher
Education Council also announced that the YÖKDĠL exam scores were to be used
for all academic issues, like to be accepted for graduate and PhD programs and
academic positions and for the minimum foreign language proficiency score to apply
for Associate Professorship, except for language allowance payment (ÖSYM, 2019).
However; since 2017, YÖKDĠL has been being conducted in three main fields of
science including Social Science, Natural and Applied Sciences and Medicine
Science.
YDS and YÖKDĠL have also the same content. They both include 80 questions, 180
minutes to answer the questions and each question is scored 1.25 points. YDS is
carried out with no regard to any scientific field, but YÖKDĠL is carried out in three
scientific fields. YDS is used for foreign language proficiency in all levels of the
state, YÖKDĠL is used only for academic purposes. Moreover, question distributions
are different. While YDS includes dialogues and restatement questions, YÖKDĠL
includes relative/noun clauses questions. And other types of the questions are the
same but, the number of the questions in categories are different (ÖSYM,2019). The
content is presented in Table 1.
18
Table 1. A Comparison of Distribution of YDS and YÖKDĠL Questions
YDS YÖKDĠL
Type of Questions Number of Number of
Questions Questions
Tenses, Passive voice, Modals, If 3 4
clauses
Vocabulary 4 5
Phrasal verbs 2 1
Prepositions 2 1
Noun clause & Relative clause - 2
Conjunctions and Clauses 5 7
Cloze test 10 10
Sentence completion 10 11
English-Turkish translation 3 6
Turkish-English translation 3 6
Finding out irrelevant sentences in 5 6
a paragraph
Paragraph completion 4 6
Paragraph reading 20 15
Dialogues 5 -
Restatement 4 -
Total 80 80
19
3. METHODOLOGY
3.1. Introduction
This chapter presents description of the methodology used in the research. The
setting and participants, research questions, data collection procedures and data
analysis are presented.
The study is conducted at Ardahan University which was founded in 2008. It has six
faculties: Faculty of Humanities and Letters, Faculty of Fine Arts, Faculty of
Economics and Administrative Sciences, Faculty of Theology, Faculty of
Engineering, Faculty of Health Sciences. Ardahan University has also six vocational
schools. As of November 2019, the university has 328 academic staff.
The study consists of both quantitative and qualitative data collection instruments.
The aim of using both qualitative and quantitative methods is to support the results.
Firestone (1987) states that “each method-type does provide different kinds of
information. Their strengths and weaknesses are complementary” (p. 19). The
quantitative data were gathered by questionnaire, and the qualitative data were
obtained by focus group interview.
Demographic tables of the participants who took part in the study are as follows:
20
Table 2. Distribution of the Participants' Gender
Valid
Frequency Percent Percent
Valid Female 65 41,1 41,1
Male 93 58,9 58,9
Total 158 100,0 100,0
As stated in Table 2, 58,9% of the participants are male and 41,1% of the participants
are female. When the gender variable is considered, it can be seen that male
participants are in the majority.
Valid
Frequency Percent Percent
As shown in Table 3 that most participants (48,1%) are between 31-40 years old, and
the second most crowded group (26,6%) includes people who are between 20-30
years old. There are only 5 participants who are 51-60 years old and 6 participants
who are 61 and above in this research.
21
Table 4. Distribution of the Participants' Unit at work
Valid
Frequency Percent Percent
Valid Vocational
41 25,9 25,9
School
School 34 21,5 21,5
Faculty 82 51,9 51,9
Other 1 0,6 0,6
Total 158 100,0 100,0
Valid
Frequency Percent Percent
Valid Instructor 58 36,7 36,7
Research Assistant 36 22,8 22,8
Asst. Prof. Dr. 45 28,5 28,5
Assoc. Prof. Dr. 10 6,3 6,3
Prof. Dr. 9 5,7 5,7
Total 158 100,0 100,0
Regarding the percentage of the participants of the study, instructors are the most
frequent participants with the total number of 58, and Assoc Prof. Dr. with the total
number of 10 and Prof. Dr. with the total number of 9 are the least frequent
participants in the study. However, 28,5% of the participants are Asst. Prof. Dr. and
22,8% of the participants are Research Assistants.
22
Table 6. Distribution of the Participants' Field of Study
Valid
Frequency Percent Percent
Valid Social Science 109 69,0 69,0
Life Science/ Natural
22 13,9 13,9
and Applied Science
Health Science 21 13,3 13,3
Educational Sciences 4 2,5 2,5
Other 2 1,3 1,3
Total 158 100,0 100,0
It is observed that most of the participants (69,0%) studies in the field of Social
Sciences, and Life Science/ Natural and Applied Science (13,9%) and Health
Science‟ s participants (13,3%) are nearly equal to each other, whereas there are very
few people from the field of Educational Sciences in this study.
Valid
Frequency Percent Percent
Valid 0 8 5,1 5,1
1-2 44 27,8 27,8
3-4 47 29,7 29,7
5 and above 59 37,3 37,3
Total 158 100,0 100,0
Most of the participants (37,3%) with the number of 59 took the foreign language
exam 5 and above. 29,7% of the participants took the exam 3-4 times and 27,8 of the
participants took the exam 1-2; whereas 5,1% of the participants never took the
exam.
23
Table 8. Distribution of the Participants' Reason for Taking the Exam
Valid
Frequency Percent Percent
As for the participants‟ reasons for taking the exam, 34,8% of the participants took
the exams for graduate or postgraduate education. 27,2 of the participants took the
exams for Associate Professorship, 17,1% of the participants took the exams for
Academic Recruitment, 8,9% of the participants took the exam for Learning Foreign
Language Level, 7,0% of the participants took the exam for Receiving Foreign
Language Allowance and 5,1% of the participants took the exam for other reasons.
24
Table 9. Distribution of the Participants' How Prepared for the Exam
Valid
Frequency Percent Percent
Valid I have never studied 34 21,5 21,5
I studied and prepared
57 36,1 36,1
myself
I attend to the courses 25 15,8 15,8
I took private courses 10 6,3 6,3
I attended online
29 18,4 18,4
courses
Other 3 1,9 1,9
Total 158 100,0 100,0
It can be concluded from Table 9 that 36,1% of the participants studied and prepared
themselves, 21,5% of the participants never studied, 18,4% of the participants
attended online courses, 15,8% attended to courses, 6,3% of the participants took
private courses, and 1,9% of the participants prepared themselves with other ways.
Table 10. Distribution of the Exams Participants Attended apart from YDS and
YÖKDĠL
Valid
Frequency Percent Percent
Valid I have never
131 82,9 82,9
attended
TOEFL 4 2,5 2,5
IELTS 14 8,9 8,9
PTE 1 ,6 ,6
Other 8 5,1 5,1
Total 158 100,0 100,0
As seen in Table 10, most of the participants (82,9%) with the number of 131 never
attended any of the foreign language exams except for YDS and YÖKDĠL. 8,9% of
25
the participants took IELTS, 5,1% took other international foreign language exams
and only a person took PTE.
Major Research Question 1: What do academic staff think about the content of
YDS and YÖKDĠL?
Major Research Question 2: What do academic staff think about the effect of YDS
and YÖKDĠL?
For the present study, both quantitative (via the questionnaire) and qualitative (via
the interview) data were collected. Firstly, questionnaire which was suitable for the
topic was determined and permission was taken from Polat (2017) (creator of the
questionnaire). In order to conduct the study, legal permission was taken from
Ardahan University (see Appendix C, D). After legal permission procedure, the
questionnaires were distributed to participants with a short visit to each department
and unit they currently work, they were given information about the content of the
study and asked to complete the survey. Both written and oral consent were
presented at the beginning of the questionnaire filling out process. After 2 weeks, the
answered questionnaires were collected. The questionnaire was aimed to cover 328
academicians while only 158 of them were either available or volunteering to
participate. The reason why the number of the participants is so low when compared
with the total number of academic staff at Ardahan University is that the number
covers all the academic staff including research assistants who are conducting their
MA or PhD studies in a different city or country, and the academic staff who have
never taken YDS or YÖKDĠL before, like foreign academic staff and some
instructors working at vocational schools at Ardahan University.
26
selected voluntarily. A group of 9 people including at least 1 academic staff from
each academic title. They were interviewed face to face with the researcher as
moderator. Interviews were audio-taped and transcribed. The questionnaires and
interviews were both conducted in Turkish language so as to avoid
misunderstandings.
The quantitative data were analysed by using Statistical Package for Social Sciences
(SPSS) 16.0 for Windows. The analysis included means, standard deviations,
percentages, frequency rates, and the values showing reliability. Likert scale items
were given values from 1 to 5 to calculate the mean and standard deviation. The
collected data were analysed for 25 different survey items by taking participants‟
genders, ages, marital status, unit at work, academic title, field of study, how many
times they took YDS and YÖKDĠL, final score from the exam, reason for taking the
exam and how they prepared for the exam, titles into considerations. With regard to
the questionnaire used in the study, the reliability analysis is as follows;
Reliability Statistics
Cronbach's
Alpha N of Items
,667 23
Two items were deleted in order to increase the Cronbach‟s Alpha Level. Because,
the Cronbach‟s Alpha level is expected to be 0,5 and more to define the scale used in
the study as reliable.
After finishing quantitative data analysis, focus group interview statements were
analysed. The qualitative data were thematically categorized, and analysed according
to the qualitative data analysis techniques. Bryman (2006) states that using both
quantitative and qualitative methods help the researcher to compare the results
efficiently. Taking this into consideration, the researcher combined two methods of
data collection to get in-depth results.
27
4. RESULTS
4.1. Introduction
Table 12. YDS and YÖKDĠL questions are appropriate for measurement of the
desired foreign language skills in Turkey
Valid
Frequency Percent Percent
Valid Strongly
25 15,8 15,8
Disagree
Disagree 77 48,7 48,7
Neutral 34 21,5 21,5
Agree 20 12,7 12,7
Strongly Agree 2 1,3 1,3
Total 158 100,0 100,0
Table 12 shows that most of the participants (64,6%) do not agree with the opinion
that YDS and YÖKDĠL questions are appropriate for measurement of the foreign
language skills. Only a small number of participants (14%) stated that they agree
with the idea.
Table 13. YDS and YÖKDĠL's score will be similar to the language proficiency
score obtained from any other internationally valid foreign language test
Valid
Frequency Percent Percent
Valid Strongly
22 13,9 13,9
Disagree
Disagree 84 53,2 53,2
Neutral 33 20,9 20,9
Agree 14 8,9 8,9
Strongly Agree 5 3,2 3,2
Total 158 100,0 100,0
Regarding the score similarities, most of the participants (67,1%) stated that they do
not believe that the international foreign language exam scores are similar to those
28
got from YDS and YÖKDĠL. However, a small number of participants with a
percentage of 12,1% gave positive response to the statement.
Table 14. YDS and YÖKDĠL are easy for candidates who have learned the foreign
language by living abroad or staying abroad for a while
As seen in Table 14, most of the participants (40,5%) have no idea about the YDS
and YÖKDĠL are easy for candidates who have learned the foreign language by
living abroad or staying abroad for a while. 34,2% of the participants agree that
living or staying abroad for a while will help participants to succeed in YDS and
YÖKDĠL. And 25,3% of the academic staff think that an abroad experience in order
to get a valid point from YDS and YÖKDĠL is not necessary.
Table 15. YDS and YÖKDĠL are exams that can measure the foreign language level
Valid
Frequency Percent Percent
Valid Strongly
22 13,9 13,9
Disagree
Disagree 55 34,8 34,8
Neutral 57 36,1 36,1
Agree 19 12,0 12,0
Strongly Agree 5 3,2 3,2
Total 158 100,0 100,0
In total, 48,7% of the academicians, nearly half of them, checked the disagree option,
which means YDS and YÖKDĠL cannot measure the foreign language level. Only
29
15,2% of the academic staff claim that YDS and YÖKDĠL can measure the foreign
language level.
Table 16. YDS and YÖKDĠL's scores show the candidates' actual foreign language
level
Valid
Frequency Percent Percent
Valid Strongly
25 15,8 15,8
Disagree
Disagree 73 46,2 46,2
Neutral 50 31,6 31,6
Agree 6 3,8 3,8
Strongly Agree 4 2,5 2,5
Total 158 100,0 100,0
As clearly seen in Table 16, the percentage of „disagree‟ in total is 62% with YDS
and YÖKDĠL‟s score shows the candidate's actual foreign language level. And just
6,3% of the participants agree with the statement.
Table 17. In order to answer YDS and YÖKDĠL questions, it is enough to know the
foreign language well
Valid
Frequency Percent Percent
Valid Strongly
12 7,6 7,6
Disagree
Disagree 39 24,7 24,7
Neutral 74 46,8 46,8
Agree 21 13,3 13,3
Strongly Agree 12 7,6 7,6
Total 158 100,0 100,0
Table 17 shows that 46,8% of the participants had neutral ideas about answer YDS
and YÖKDĠL questions, it is enough to know the foreign language well. 32,3% of
the participants do not agree with the item. 20,9% of the participants have positive
ideas to answer questions, knowing foreign language well is sufficient.
30
Table 18. All YDS and YÖKDĠL questions are suitable to measure the foreign
language level
Valid
Frequency Percent Percent
Valid Strongly
22 13,9 13,9
Disagree
Disagree 65 41,1 41,1
Neutral 55 34,8 34,8
Agree 13 8,2 8,2
Strongly Agree 3 1,9 1,9
Total 158 100,0 100,0
As evidently seen in Table 18, more than half of the academicians (55,1%) disagree
that all YDS and YÖKDĠL questions are suitable to measure the foreign language
level. 10,1% of the academicians agree that YDS and YÖKDĠL are good tools to
measure the foreign language level.
Table 19. YDS and YÖKDĠL's score should be valid for decisions on foreign
language level alone
Valid
Frequency Percent Percent
Valid Strongly
25 15,8 15,8
Disagree
Disagree 70 44,3 44,3
Neutral 43 27,2 27,2
Agree 14 8,9 8,9
Strongly Agree 6 3,8 3,8
Total 158 100,0 100,0
As clearly seen in Table 19, most of the participants (60,1%) are in the view that
YDS and YÖKDĠL‟s scores should not be valid for decisions on foreign language
level alone, while very few (12,7%) support the idea.
31
Table 20. At the end of YDS and YÖKDĠL, a candidate who gets 90 and above can
use that foreign language at an advanced level
Valid
Frequency Percent Percent
Valid Strongly
18 11,4 11,4
Disagree
Disagree 58 36,7 36,7
Neutral 52 32,9 32,9
Agree 24 15,2 15,2
Strongly Agree 6 3,8 3,8
Total 158 100,0 100,0
It has been shown in Table 20 that 48,1% of the participants disagree with the idea
that a candidate who gets 90 and above can use that foreign language at an advanced
level. 19% of the participants stated that a participant getting 90 and above can be
regarded as proficient in terms of foreign language.
Table 21. All the questions of YDS and YÖKDĠL are related to each other in a
structural integrity
Valid
Frequency Percent Percent
Valid Strongly
10 6,3 6,3
Disagree
As illustrated in Table 21, it can be figured out that the percentage of „disagree‟ in
total is 48,7. 15,8% of the participants agreed that all the questions of YDS and
32
YÖKDĠL are related to each other in a structural integrity. And 35,4% of the
participants have no idea about the statement.
Table 22. YDS and YÖKDĠL are necessary exams for the professional development
of all academics regardless of their field
Valid
Frequency Percent Percent
Valid Strongly
11 7,0 7,0
Disagree
It has been shown in Table 22 that 50,6% of the academic staff have neutral feelings
about YDS and YÖKDĠL‟s necessity for the professional development of all
academics regardless of their field. While 27,2% of the academic staff do not agree
with the statement, 22,2% of the academicians agree with it.
Table 23. Since listening questions are not asked in YDS and YÖKDĠL, listening-
comprehension skills in foreign languages are not given importance
Valid
Frequency Percent Percent
Valid Strongly
4 2,5 2,5
Disagree
Disagree 6 3,8 3,8
Neutral 24 15,2 15,2
Agree 64 40,5 40,5
Strongly Agree 60 38,0 38,0
Total 158 100,0 100,0
33
Table 23 clearly shows that 78,5% of academicians expressed that they don‟t do
anything to develop their listening skills because of the fact that YDS and YÖKDĠL
do not test it. YDS and YÖKDĠL can be said to have negative effect on candidates
listening skills. Only 6,3% of them responded the statement negatively.
Table 24. YDS and YÖKDĠL make the candidates feel that learning a foreign
language is not difficult
Valid
Frequency Percent Percent
Valid Strongly
15 9,5 9,5
Disagree
As can be seen from the Table 24, 42,4% of the participants gave positive response
to item 14. 32,9% of the participants responded the statement negatively.
Table 25. Frequently asking grammar questions in YDS and YÖKDĠL causes
foreign language learning to be perceived as grammar learning
Valid
Frequency Percent Percent
Valid Strongly
5 3,2 3,2
Disagree
Disagree 14 8,9 8,9
Neutral 53 33,5 33,5
Agree 53 33,5 33,5
Strongly Agree 33 20,9 20,9
Total 158 100,0 100,0
34
Table 25 shows that 54,4% of the participants believe that frequently asking
grammar questions in YDS and YÖKDĠL causes foreign language learning to be
perceived as grammar learning. The participants‟ responses show that they study
only grammar since knowledge of grammar is tested in YDS and YÖKDĠL. 12% of
the participants disagree with statement.
Table 26. Before and after YDS and YÖKDĠL, candidates experience anxiety due to
the exam
Valid
Frequency Percent Percent
Valid Strongly
5 3,2 3,2
Disagree
Disagree 2 1,3 1,3
Neutral 26 16,5 16,5
Agree 85 53,8 53,8
Strongly Agree 40 25,3 25,3
Total 158 100,0 100,0
As clearly seen in Table 26, 79,1% of the participants feel anxious during the YDS
YÖKDĠL exam process both before and after. Only 4,4% of them state that the
exams do not make them feel stressed.
35
Table 27. For YDS and YÖKDĠL, some tactics and test strategies need to be known
Valid
Frequency Percent Percent
Valid Strongly
5 3,2 3,2
Disagree
As evidently seen from the Table 27, the percentage of disagree in total is just 8,2%
which is a very small amount. Totally, 74,7% of the participants, which is a very
high amount, do agree that in order to get a valid point from YDS and YÖKDĠL
knowing and applying the strategies and techniques of these exams are necessary.
Table 28. YDS and YÖKDĠL create a desire to learn foreign languages more
intensively for the candidates who will take the exam
Valid
Frequency Percent Percent
Valid Strongly
16 10,1 10,1
Disagree
Disagree 40 25,3 25,3
Neutral 68 43,0 43,0
Agree 21 13,3 13,3
Strongly Agree 13 8,2 8,2
Total 158 100,0 100,0
36
Table 28 shows that percentage of neutral and disagree are close to each other;
43,0% and 35,4% respectively. It can be said that some academicians (43%) think
that YDS and YÖKDĠL have no impact on motivation to learn a foreign language
while similar number of academics (35,4%) think that these exams effect their
learning processes negatively as well. However, a small number of academic staff
(21,5%) states that they agree with the item.
Table 29. Because of the name YDS and YÖKDĠL, it creates an unwanted
(negative) perception on the candidates
Valid
Frequency Percent Percent
Valid Strongly
6 3,8 3,8
Disagree
Disagree 37 23,4 23,4
Neutral 45 28,5 28,5
Agree 56 35,4 35,4
Strongly Agree 14 8,9 8,9
Total 158 100,0 100,0
As illustrated in Table 29, 44,3% of the participants give positive response to the
idea that the names of YDS and YÖKDĠL exams create an unwanted perception on
test-takers. 27,2% of the participants disagree that YDS and YÖKDĠL terms do not
have negative impact on them.
Table 30. YDS and YÖKDĠL affect foreign language learning negatively
Valid
Frequency Percent Percent
Valid Strongly
12 7,6 7,6
Disagree
Disagree 29 18,4 18,4
Neutral 54 34,2 34,2
Agree 54 34,2 34,2
Strongly Agree 9 5,7 5,7
Total 158 100,0 100,0
37
As can be seen from Table 30, most of the participants (39,9%) think that YDS and
YÖKDĠL affect foreign language learning negatively, while 25,9% of them think that
these exams do not affect them in a negative way.
Table 31. Since advanced vocabulary is required in YDS and YÖKDĠL, vocabulary
memorization is a common study method
Valid Strongly
5 3,2 3,2
Disagree
Table 32. YDS and YÖKDĠL effect foreign language learning positively
Valid
Frequency Percent Percent
Valid Strongly
13 8,2 8,2
Disagree
Disagree 28 17,7 17,7
Neutral 55 34,8 34,8
Agree 52 32,9 32,9
Strongly Agree 10 6,3 6,3
Total 158 100,0 100,0
38
As can be seen from Table 32, 39,2% of the participants agree that YDS and
YÖKDĠL effect foreign language learning positively, whereas 25,9% of participants
think that these exams have negative impact on learning foreign language.
Table 33. The candidate studying for YDS and YÖKDĠL will be more successful in
using that foreign language
Valid
Frequency Percent Percent
Valid Strongly
14 8,9 8,9
Disagree
Disagree 36 22,8 22,8
Neutral 67 42,4 42,4
Agree 33 20,9 20,9
Strongly Agree 8 5,1 5,1
Total 158 100,0 100,0
Table 33 shows that 31,6% of the participants disagree with the idea that a candidate
studying for YDS and YÖKDĠL will be more successful in using that foreign
language while 26% of the participants agree with the statement.
Table 34. In YDS and YÖKDĠL preparation, it is useful to hire a tutor to teach test
techniques
Valid
Frequency Percent Percent
Valid Strongly
33 20,9 20,9
Disagree
Disagree 24 15,2 15,2
Neutral 40 25,3 25,3
Agree 49 31,0 31,0
Strongly Agree 12 7,6 7,6
Total 158 100,0 100,0
As illustrated in Table 34, percentages of agree and disagree are very close to each
other as in item number twenty-five; 38,6% and 36,1% respectively. 38,6% of the
participants believe that in order to get valid point from YDS and YÖKDĠL extra
help is needed. 36,1% of the participants do not agree with the item.
39
4.3. Qualitative Results
After gathering data from questionnaire, the researcher conducted a focus group
interview with the academics at Ardahan University. There were 9 participants from
at least one academic title, including 2 professors,1 associate professor, 3 assistant
professors, 1 research assistant and 2 lecturers. The focus group interview questions
were as follow;
1. What do you think about English (or any other foreign languages) in general?
2. Do you believe that an academic staff should know a foreign language
regardless of their field of study?
3. What do you think about YDS and YÖKDĠL in general?
4. Do you think that YDS and YÖKDĠL exams help candidates to improve
themselves in terms of foreign language?
5. Do you believe that getting a higher score from YDS and YÖKDĠL means
that you are competent at that language?
6. How did you study for YDS and YÖKDĠL exams? What kind of strategies
did you use?
7. Do you believe that YDS and YÖKDĠL have more importance for specific
academic titles?
8. Do you believe that the foreign language proficiency must be proved in order
to apply for associate professorship?
9. Do you believe that preparation process affected your level of foreign
language that you use while writing an abstract or academic work, or
presenting in English at conferences?
Firstly, the participants were asked to express their ideas about foreign language in
general. The responses were positive mostly. Most of the participants stated that
foreign languages, especially English as the global language, have importance in
both academic and social life. However, some participants claimed that knowing a
foreign language is a sign of intellectual competence which makes an individual
different from their peers or colleagues.
40
As the world is being globalized day by day, every individual should
know English as it has become the language of the world. P1
English language has integrated into our lives so intensely that with
the regulations at primary schools offering English courses from
second year help children to be aware of the English language which
is good. P3
The self-reports of participants show that they perceive English language positively
in general sense. While some of them state that English is a need instead of being just
a tool, many other participants stated that it is the medium that they need in their
lives. When the question is about the necessity of foreign language proficiency for
academics, the opinions didn‟t seem to change. It was found out during the interview
that, they strongly believe that academic staff should be English proficient so as to
cope with the academic world all around the world.
It is seen that participants are in the same idea that an academician should know a
foreign language especially the English, as it is regarded as the language of the
global world. It can be said that being English proficient is regarded as the crucial
feature of an example academician in a global context.
In Turkey, getting a language score from national foreign language exams, as well as
accredited international ones, is an important requirement for candidates applying for
graduate programs, academic positions or associate professorships. It has been a real
41
debate for many years about whether these national exams really measure the
language level or not. Taking this information into consideration, the participants
were asked to express their ideas about YDS and YÖKDĠL exams in terms of
content, necessity and effects on candidates in general. Most of the participants
stated that these exams are not good tools to measure the language proficiency
mostly because they lack productive skills like speaking, writing and receptive skill
as listening.
These exams do not measure skills like speaking, writing as these are
important skills for an academician that he/she will use it in most of
his academic activities like writing an article, or presenting at an
international conference. P4
The names of the exams have been changed over the years. For me,
it is proof that these exams are not efficient in measuring the
proficiency, if they were so, the names and the contents wouldn’t be
changed that often. P8
As seen from the statements of participants, the perceptions toward YDS and
YÖKDĠL exams are negative. The participants trust neither the measurability nor
credibility. This negativity‟s main reason is that these exams do not measure other
skills rather than grammar and reading. The participants expect these exams to cover
all skills which will in return help candidate to use the foreign language effectively in
all aspects. But, as the candidates only focus on specific strategies and vocabulary
42
memorisation, the effect of these exams results in being negative which in turn
demotivates participants in terms of foreign language use in general.
4.3.3. Academic Staff’s Perceptions About the Effects of YDS and YÖKDĠL on
Academic Context
In order to get an academic position or academic degree you need to prove your
foreign language level either with national foreign language exams or international
ones. Apart from being a need for academicians to know a foreign language, it is also
a necessity to get a score determined by the institutions according to qualification of
the position or degree. So, it is no doubt to state that, foreign language proficiency is
important in all phases of the academic world in Turkey. Taking this information into
consideration, the participants were asked to express their ideas on the effects of
YDS and YÖKDĠL exams on academic life in Turkey. The statements were similar
to the previous ones, most of the participants believed that foreign language
competence is a must but it needs to measure the language in all aspects.
Knowing a foreign language is very important but, for me, YDS and
YÖKDİL exams should measure all the skills including speaking,
writing because an academician is expected to use the foreign
language effectively. P4
The participants were also asked to express their ideas about the academic positions
application, graduate program and associate professorship processes that include the
necessity of proving the foreign language proficiency score with at least 55 points.
The statements were as follow;
43
If you are working as an academic staff in Turkey, you should know
English well. The legal way to prove that is to show the score you get
from YDS or YÖKDİL. But it should be kept in mind that the
academician should use the foreign language effectively regardless
of the score he/she got from those exams. P5
As seen above, the participants emphasized the credibility of the YDS and YÖKDĠL
exam as being low when compared worldwide. It can be concluded from the
statements that English proficiency should be given importance first and then asked
to be measured. Because the system just wants to see the academic staff‟s foreign
language level temporarily, more motivational support seems to be needed in order to
establish a continuous presentation of foreign language proficiency.
44
5. DISCUSSION AND CONCLUSION
5.1. Introduction
This study aims to explore the effect of YDS and YÖKDĠL exams on academic staff
in Ardahan University. As these foreign language exams have been major issues for
the professional and academic lives of academic staff in Turkey, this small scaled
study will help to explore the washback effects of national foreign language exams in
detail. In this chapter, the major finding of the results will be presented with
discussion and suggestions for further studies.
5.2. Discussion
5.2.1. What do academic staff think about the content of YDS and YÖKDĠL?
The questionnaire used in the study had two factors. While the items in the first part
with 12 statements covered the content of the YDS and YÖKDĠL, the second part
included statements about the washback effects of those exams. The results were
surprisingly indifferent in terms of demographic information like gender, academic
title and age. In order to see if there is a significant difference between these groups,
an independent t-test was administered. As stated before, there was not a significant
difference between the genders, academic titles and ages.
As the first part of the questionnaire was about the content of the YDS and YÖKDĠL,
it was interesting to see that most of the academic staff (67,1%) did not agree with
the idea that the score taken from these national exams would be similar to those
taken from international foreign language exams. This can be proof that the academic
staff in Turkey do not believe the international validity of the YDS and YÖKDĠL
exams although they were supposed to get some specific points to get either an
academic title or MA/PhD degree. However, a great number of the participants
(64,6%) similarly disagreed with the idea that YDS and YÖKDĠL questions are
appropriate for the measurement of the desired foreign language skills in Turkey.
The reason behind this strong disapproval may be that the exams do not assess the
basic language skills like speaking, listening and writing. Moreover, with regard to
the content of these exams, a majority of participants stated that these exams are not
good tools to measure foreign language level because the skills as mentioned before,
45
are not asked in the exam. Similarly, in the study conducted by Yavuzer and Göver
(2012), the researchers concluded that the test takers are in the same view that the
foreign language exams in Turkey are not powerful enough in terms of measurement,
as they lack to measure productive skills like speaking and writing. So it can easily
be said that as four skills are crucial components of a foreign language, it is
impossible to assess a language proficiency without measuring them.
5.2.2. What do academic staff think about the effect of YDS and YÖKDĠL?
Pearson (1988) states that the quality of the test determines the effect of the test
itself. As for the results from quantitative and qualitative data, participants stated that
they feel anxious both before and after the YDS and YÖKDĠL exam. The main
reason is, as they also stated during the interviews, that the psychological pressure is
too high as these exams have vital roles in participants‟ lives. The test-takers will use
the scores either to be accepted for an academic position or post-graduate programs.
The role of the foreign language becomes more important with the exam itself. As
mentioned before, the foreign language learning process is a long journey in Turkey,
starting from primary school till university. This long process creates an atmosphere
where individuals are accepted to use language skills effectively after learning it for
more than 15 years. This can be another reason why participants feel so stressed
during exam period.
Akpınar and Çakıldere (2013) found out that the candidates of foreign language tests
in Turkey tend to focus more on reading and grammar. Similarly, Sayın and Aslan
(2016) concluded that the test takers used strategies that give emphasis on grammar
46
and vocabulary, which in fact did not help them to feel fulfilled in terms of foreign
language competence. Considering the strategies participants use while studying for
the foreign language exam, most of them agreed that they give emphasis on grammar
and vocabulary skills more as the exams do not cover the other skills like listening,
speaking or writing. They tend to focus more on grammar and vocabulary as they
believe that specific strategies are needed to get a higher score from the exams. This
side is interesting, because, during the interview, some of the participants stated that
they want to learn foreign language to communicate with foreigners easily or present
their studies during international conferences. This is somehow paradoxical because
although they are aware of the fact that they need the foreign language with all the
elements including speaking, writing and listening, they just focus on getting a valid
score from the exam without concentrating on other skills. This can be again
regarded as the official psychological pressure because the system and the
regulations ask academicians for the language score which assess only grammar and
reading skills, but the same system also encourages academic staff to participate in
international conferences, symposiums and similar activities that they are supposed
to use language skills that are not tested with the exams that the same system
organizes.
5.3. Conclusion
This study investigated the washback effects of YDS and YÖKDĠL exams on
academic staff at Ardahan University with all the results obtained from both
qualitative and quantitative data. It is also examined what academic staff at Ardahan
University think about the content and effect of YDS and YÖKDĠL exams. The
results showed that the perception toward these exams are negative. Although they
are important exams for academicians professional and academic lives, the
academicians believe that the validity and the content of the exams are not as they
are supposed to be. It means that, although they spend time for more than 15 years
with the foreign language, starting from primary school, they feel psychologically
under pressure and this leads to feeling anxious demotivating them about the exams.
It is important to note that the foreign language is a must in terms of both speaking,
writing and reading during academic career. For instance, in order to apply for
associate professorship, the candidate is expected to publish his article in an
47
international journal, or the candidate is expected to participate in international
academic conferences and symposiums where he/she can use all the basic skills of
the foreign language. But when it comes to assess the foreign language, the system
only measures the grammar and reading skills which in turn demotivates the
candidate in terms of foreign language learning. The best way to assess the foreign
language proficiency, is to conduct exams that cover all the skills including speaking,
reading, writing and listening. By this way, the candidate may feel that he/she is
internationally proficient in terms of foreign language. Encouraging academic staff to
integrate the global world by sending them abroad for conferences or language
courses, may also decrease the anxiety level towards the exams if they are exposed to
foreign languages more.
This study was conducted in Ardahan University. The participants of the study were
the academic staff currently working at Ardahan University. Further research with
more participants from different universities can be conducted with a bigger number
of data.
However, gender, age and academic title were not compared or taken into account in
the study. A study with these elements with a comparison can be carried out for more
in-depth results.
The study investigated the thoughts of academic staff towards YDS and YÖKDĠL
exams together. A study that compares the effect of both national and international
foreign language exams can be carried out to see if there is any significant difference.
48
REFERENCES
Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14,
115–129.
Alderson, J.C. & Hamp (Lyons, L. (1996). TOEFL preparation courses, a study of
washback. Language Testing, 13(3), 280-297.
Bachman, L.F. and Palmer, A.S. (1996). Language Testing in Practice. Oxford:
Oxford University Press.
Cohen, A. D. (1994). Assessing language Ability in the classroom (2nd Ed.). New
York: Heinle and Heinle.
49
Cole, J. (2007). Motivation to do well on low-stakes tests. Unpublished doctoral
dissertation, University of Missouri, Columbia.
Coombe, C. A., Folse, K. S., & Hubley, N. J. (2007). A practical guide to assessing
English language learners. (pp. 89-110). University of Michigan: University
of Michigan Press.
Davies, A. (1985). Follow my leader: Is that what language tests do? In: LEE Y.P.
and[others] (Eds.) New Directions in Language Testing. Oxford: Pergamon
Press, pp-10.
Gates, S. (1995). Exploiting washback from standardized tests. In Brown, J.D. and
Yamashita, S.O. (Eds.), Language testing in Japan (pp. 107-112). Tokyo:
Japan Association for language Teaching.
Goertz, M., & Duffy, M. (2003). Mapping the landscape of high-stakes testing and
accountability. Theory into Practice, 42(2), 4-11.
50
Hughes, A. (2003). Testing for language teachers (2nd edition). Cambridge:
Cambridge University Press
KumandaĢ, H., & Kutlu, Ö. (2010). High stakes testing: does secondary education
examination involve any risks? Procedia Social and Behavioral Sciences,
9,758-764. Doi: https://doi.org/10.1016/j.sbspro.2010.12.230
51
ÖSYM. (2019) (a). YDS: Yabancı Dil Bilgisi Seviye Tespit Sınavı. Retrieved
November 5, 2019, from https://www.osym.gov.tr/TR,8860/hakkinda.html
Pearson, I. (1988). Tests as levers of change (or „putting first things first‟). In D.
Chamberlain & R. Baumgartner (Eds.), ESP in the classroom: Practice and
Evaluation (pp. 98-107). London: Modem English Publications in
association with the British Council.
52
Sheng-ping, T. & Chong-ning, X. (2004). On washback of testing to general English
education. Paper presented at the Fourth International Conference in ELT,
China.
Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test Impact revisited:
Washback effect over time. Language Testing, 13, 298–317.
Spratt, M. (2005). Washback and the classroom: The implications for teaching and
learning of studies of washback from exams. Language Teaching Research,
9(1), 5- 29.
53
APPENDICES
Appendix A:
Dear Participant;
This questionnaire is prepared to collect data for the master‟s thesis with the title
“Academic Staff’s Perceptions on Washback Effect of YDS and YÖKDİL: Ardahan
University Case” within the scope of the Master's Program of the Department of
Western Languages and Literature, Department of English Language and Literature,
Kafkas University, Institute of Social Sciences. The first part includes your
demographic information and questions about YDS and YÖKDĠL that you attended
before; the second part includes statements about the content of YDS and YÖKDĠL
and the last part includes statements about washback effect of YDS and YÖKDĠL.
Please mark the appropriate option with (X). Answering the questions will also mean
that you consent to the use of the information you provide in the study, and your
answers will be used for only scientific purposes.
6. Field of Study: Social Science ( ) Life Science/ Natural and Applied Science ( )
Health Science ( ) Educational Sciences ( ) Other ( )
7. How many times have you participated in YDS or YÖKDĠL:0 ( ) 1-2 ( ) 3-4 ( )
5 and above ( )
54
8. The final score from the exam: ………….
10. While preparing for the exam: I never studied ( ) I studied and prepared myself
( ) I attend to the courses ( ) I took private lessons ( ) I attended online courses ( )
Other ( )
11. Have you taken any of the following exams that are accepted to YDS and
YÖKDĠL, except YDS and YÖKDĠL, and if so, how many points have you
received last?
Strongly disagree
Strongly agree
Sıra Yabancı Dil Bilgisi Seviye Tespit Sınavı (YDS) ve
Disagree
Neutral
55
5 YDS and YÖKDĠL’s score shows the candidate's
actual foreign language level.
6 In order to answer YDS and YÖKDĠL questions, it
is enough to know the foreign language well.
56
3.Washback Effect of YDS and YÖKDĠL
Strongly disagree
Strongly agree
Sıra
Disagree
Neutral
Agree
13 Since listening questions are not asked in YDS and
YÖKDĠL, listening-comprehension skills in foreign
languages are not given importance.
14 YDS and YÖKDĠL make the candidates feel that
learning a foreign language is not difficult.
15 Frequently asking grammar questions in YDS and
YÖKDĠL causes foreign language learning to be
perceived as grammar learning.
16 Before and after YDS and YÖKDĠL, candidates
experience anxiety due to the exam.
57
Appendix B:
Sayın Katılımcı;
5. Akademik Ünvan: Öğr. Gör. ( ) ArĢ. Gör. ( ) Dr. Öğr. Üyesi ( ) Doç. Dr. ( ) Prof.
Dr. ( )
6. Alanınız: Sosyal Bil ( ) Fen Bil ( ) Sağlık Bil ( ) Eğitim Bil ( ) Diğer ( )……..
9. Sınava giriĢ nedeni: Yabancı Dil Tazminatı Almak ( ) Lisans Üstü Eğitim ( )
58
Diğer ( )
Katılmadım ( ) TOEFL ( ) Puan: …….. IELTS ( ) Puan: …….. PTE ( ) Puan: ……..
Diğer ( ) Puan: ……..
Kesinlikle katılmıyorum
Kesinlikle katılıyorum
Kısmen katılıyorum
Sıra Yabancı Dil Bilgisi Seviye Tespit Sınavı (YDS) Katılmıyorum
Katılıyorum
ve Yüksek Öğretim Kurumları Yabancı Dil
Sınavı (YÖKDĠL);
59
7 sorularının bütünü yabancı dil seviyesini ölçmeye
uygundur.
Kesinlikle katılmıyorum
Kesinlikle katılıyorum
Kısmen katılıyorum
Katılmıyorum
60
18 sınava girecek adaylarda daha yoğun yabancı dil
öğrenme isteği oluĢturur.
61
Appendix C:
62
Appendix D:
63
Appendix E:
64
CURRICULUM VITAE
KĠġĠSEL BĠLGĠLER
Adı Soyadı Emine KASAP SALTAġ
ĠLETĠġĠM
Adres Ardahan Üniversitesi
E-mail eminekasap45@gmail.com
65