153 159

2001

Putting English Language Proficiency Tests on the Curriculum

Clive S. Langham

This paper describes how an English language proficiency test was put on the curriculum. In particular, investigation is made of the oral interview and the differences between the paired speaking test and the standard one-to-one format. It also details how a group of students at a university in Japan prepared for and took the test. Student reaction to both the preparation classes and the test was elicited by means of a brief questionnaire.

The University of Cambridge Local Examination Syndicate (hereafter referred to as UCLES) has a range of five examinations: level 1 Key English Test (KET), level 2 Preliminary English Test (PET), level 3 First Certificate in English (FCE), level 4 Certificate in Advanced English (CAE), level 5 Certificate of Proficiency in English (CPE). Cambridge level 1 is that of the Council of Europe Waystage 1990 specifications, as set out in van Ek & Trim 1991. Candidates at the KET level have a basic command of the spoken language and are able to convey basic meaning in very familiar or highly predictable situations. Candidates utterances are likely to be marked by short interactions with a tendency to rely on rehearsed or set phrases with limited ability to produce spontaneous utterances. Additionally, pronunciation is considerably influenced by L1 features. It is considered that level 1 (KET) may be attained after approximately 180 to 200 hours of study. Success at KET level demonstrates that candidates have the linguistic ability to handle basic communication required in everyday situations where they may come into contact with either native or non-native speakers of English. KET consists of three parts : Reading & Writing (1 hour 10 minutes), Listening (25 minutes), including 8 minutes transfer time, and a Speaking component which lasts 8 to 10 minutes. The Listening Test is made up of five parts. In all parts, candidates hear each taped item twice. Part 1 of the test consists of five short conversations. Before each conversation, candidates hear a question. This question is also printed in the test booklet. Candidates are required to choose the correct answer from a series of three line drawings. In part two of the listening test, candidates hear a single, slightly longer conversation and are required to match five items with the appropriate items, of which there are eight to choose from. Part three of the listening test consists

153

2001

of five conversions. For each conversation, students have to complete a sentence by choosing one of three items. Parts four and five require candidates to complete missing items of information with the content retrieved from conversations, announcements, recorded messages etc. The marks for the speaking component are 25% of the total marks obtainable over the whole test. The KET speaking test consists of the following two parts. 1. Interlocutor Frame 2. Question activity prompt cards and scripted instructions In part one of the speaking test, candidate output may be described as follows.

Discourse Features
Responding to questions (including one extended response)

Functions
Giving factual information about self Talking about present circumstances Expressing opinions Explaining and giving reasons Talking about future plans Talking about past experiences During this part of the interview, if candidates are unable to respond, a back-up question is used. This part of the test takes five to six minutes. In part one, each candidate interacts with the interlocutor in turn for a period of between five to six minutes. In part two, however, the candidates interact with each other and are semiindependent of the interlocutor. Candidate output in the second part of the test can be described as follows. Initiating and responding appropriately Requesting/giving information of a personal/non-personal kind. Requesting/giving information on present circumstances Likes/dislikes Habits Past experiences Factual information on dates/times/prices etc One candidate is shown a card with about five prompts in the form of incomplete questions and several line drawings. The topics cover those such as friends, family, favorite teacher, a day trip, a hobby etc. From the prompts on the card, students A formulates questions which are in turn answered by candidate B. The process is repeated with a different card from which candidate

154

Clive S. Langham Putting English Language Proficiency Tests on the Curriculum

B formulates questions. This part of the test takes three to four minutes. Following that, student A is given a card with five outline questions along with line drawings. Student B is given a card with information. Through question and answer between the candidates, the information is exchanged. The assessment criteria for the test involve marks being awarded on the basis of the consideration of the following factors 1. Completion of task 2. communicative competence 3. appropriateness of language 4. linguistic resources 5. pronunciation 6. fluency 7. independence of interlocutor ( in part 2 only) The assessment scale for the KET speaking test consists of the following five points, details of which are listed below. 5. Deals well with tasks. Communicates effectively. 4. Some of the features of 5 and some of the features of 3 3. Communicates appropriately most but not all of the time. 2. Some of the features of 3 and 1. 1. Does not achieve most tasks. Generally cannot communicate effectively.

The Paired Speaking Test Versus the One-to-one Format

In recent years, there has been a move towards placing considerable emphasis on a face-to-face speaking test. The common format for such tests has traditionally been on a basis of one-to-one. In other words, a single candidate with one examiner in an oral interview. This is still the case for the STEP or EIKEN tests which are widely recognized in Japan. However, it should be noted that the oral interview test in these examinations is only taken by candidates who have already attained a certain level on reading, writing and listening tests, and as such is not treated as an integral part of the test. In the field of foreign language testing, there is considerable interest in evaluating factors which affect test performance in the language learner, particularly as it affects the oral proficiency interview Bachman 1990, McNamara 1996. In the early 1980s, investigation of the potential of the two candidates two examiners and three candidates two examiners format for oral tests was undertaken. It is worth pointing out that this movement was in part derived from developments in the 1980s in the teaching and learning of English as a Foreign Language. Researchers and teachers at this time had developed a better understanding of the strategies required to communicate. With advances in the field of Applied Linguistics moving ahead rapidly, it had been demonstrated that knowledge about language was of lesser importance than the ability

155

2001

to use language for communicative purposes. The direct consequence of this was that the focus in the classroom changed to one where pairwork, communication and meaningful exchanges, including discussion, role-play and debate became objectives to be achieved. With this revolution in the classroom, there followed a significant change in focus in terms of the assessment of students oral proficiency. The paired candidate format was accordingly introduced in the Certificate in Advanced English (CAE) in 1991, in the Key English Test (KET) in 1993 and for the revised version of the Preliminary English Test (PET) in 1995 and for revised First Certificate in English (FCE) in 1996. The adoption of the paired format, as outlined above, has also been as a result of the findings of a number of research projects on spoken language discourse which have been carried out over the past ten years or so, and which have highlighted the degree to which language produced in oral test situations is influenced by its goals and participants. The main points of this research may be summarized as follows. Recent work by Osullivan 2000 has shown how the gender of the person with whom the learners interact in proficiency interview affects the performance of the those learners. Research carried out by Hughes 1989 suggested a potentially serious problem with the standard interview format. He showed that the power relationship existing between tester and candidate resulted in only a single style of speech being represented, as well as failure to elicit many other language functions. Additionally, he found that discussions between candidates can be a valuable source of information concerning students speaking ability. Another study by Ross and Berwick (1992) concentrated on how oral interviewers use features of control such as topic nomination and accommodation (modification of speech) for different purposes. Young and Milanovics (1992) study found that examiner-candidate discourse was highly asymmetrical in terms of features of dominance, contingency and goal orientation. It seems likely that as research in this area develops, more information will be made available to teachers and test makers that will lead to fairer oral interview test situations and formats. In a one-to-one test format, the relationship between examiner and candidate necessarily makes it hard for candidates to work freely outside the above mentioned asymmetrical system. Research has shown, however, that on the other hand the paired candidate format yields vastly greater potential for various patterns of interaction. These being between candidate and examiner, between the two candidates taking the test and also between the three participants (candidate and interlocutor). The three-way potential for communication in the paired speaking test format is illustrated below.

156

Clive S. Langham Putting English Language Proficiency Tests on the Curriculum

Candidate

Interlocutor (participant)

global assessment

Candidate

Assessor (Observer)

analytical assessment

Comparisons of student production in the paired and one-to-one format produced some interesting findings. While it might be reasonable to expect that the quantity of language produced in a one-to-one situation would exceed that in a paired format, it was found in transcription studies of interviews that the volume of production in the paired format was larger. Additionally, it was found that the contribution of the examiner (in terms of the numbers of words and turns) was reduced in the paired format test, with relative contributions of the two candidates increasing. These findings suggest that the asymmetrical nature of standard speaking test formats, as revealed by previous studies, is greatly reduced, indicating a more balanced interaction between candidates being achieved, and also with the examiner taking a far less dominant role. Additionally, the considerable increase in the number of turns from paired candidates along with greater variation in turn length clearly indicate that the paired format produces a sample of spoken language that is far more indicative of students actual level than is commonly achieved in the oneto-one format. In short, the paired format allows greater degrees of interactive communication. Work on speaking ability has identified a total of about 30 communicative language functions that make up spoken discourse. Among the categories, we find informational (expressing an opinion), interactional (persuading), interaction management (terminating a discussion). Analysis of the one-to-one speaking test has indicated that twenty out of the above mentioned thirty functions are likely to be in evidence in a candidates spoken output, whereas in a paired format the situation shows potential for eliciting twenty-eight out of the thirty functions which is a significantly higher proportion.

History of KET at Tsukuba Womens University

In 1996, Tsukuba Womens University (TWU) became a closed test centre for both the Key English Test and the Preliminary English Test. At this time, although preparation classes for the tests were not timetabled in regular classes, a number of study groups met on a weekly basis. An average of twelve students has taken KET each year. Of the twelve students who sat for the examination in November 1999, the results are as follows: Candidates 12 Pass with Merit 1 Pass 8 Narrow Fail 1 Fail 2

157

2001

The two candidates who failed were particularly weak in paper 1, reading and writing. Analysis of the candidates results over the period of four years showed that of the failing candidates, a large proportion were particularly weak in reading and writing. This result would at first glance appear to be somewhat surprising, since it is a generally held belief among teachers in Japan that students have higher levels in reading and writing than in listening and speaking. From the authors own experience of preparing students for KET, weaknesses in listening and speaking can more easily be compensated for with intensive remedial work than similar weakness in reading which are still largely underestimated by both teachers and students alike, and which seriously damage students chances of attaining good results.

The KET 2000 Session

Revision of the curriculum for the academic year 2000 meant that a preparation course for KET was scheduled in the timetable for the first time. This was in response to increased demand for formal qualifications in English, particularly in the light of a very competitive job market for graduating students. The KET preparation class was of the duration of 90 minutes, meeting once a week. Twenty three students enrolled in the class which started in April 2000. No specific textbook was used. Past papers were reviewed and teacher produced material made up the bulk of the teaching material. In September, approximately two months before the test took place, students were asked to write responses to the following four questions as an assignment.

Survey
1. Why did you choose to take KET? 2. What is the most difficult part of KET for you? 3. What do you like best about KET? 4. Do you plan to take another English examination? Responses to survey question number one showed unequivocally that students took the course in order to gain a formal qualification in English, particularly in the hope that it would improve their prospects in the job market. Comments such as It might help me to get a job were common. Survey question number two revealed that the majority of students rated the listening paper and speaking component to be a greater obstacle than the reading and writing paper. This, as I have already pointed out, does not correspond to the level of difficulty as reflected by the results over the past four years, where almost all failing students have scored badly in the reading and writing section. The survey illustrated that students rated the oral interview as the best thing about KET. For example, one students wrote, I liked it because I could study for an exam and still practice English conversation. This was typical of students answers to this part of the survey. In the final question of the survey, most students seemed keen to continue taking English language proficiency tests, particularly if they are part of the curriculum and have good preparation classes

158

Clive S. Langham Putting English Language Proficiency Tests on the Curriculum

with adequate teacher support and back-up material in the form of units in the self access center.

Conclusion
It is likely that many private universities in Japan will go down the route of offering examination classes on the curriculum, and that there will be a movement away from English language classes with no specific goals to those that are more goal oriented. With declining student numbers and an increasingly competitive job market, this kind of movement will become more widespread. Examinations that incorporate a speaking component will continue to attract students since they feel that they are able to make progress in all areas of their English studies. The results of the survey showed that students consider listening and speaking to represent greater levels of difficulty than other aspects of the test. However, results from previous test takers showed that in reality the reading and writing components represent a far greater challenge for students.

Bibliography
Bachman, L. 1990 Fundamental Considerations in Language Testing, Oxford University Press, Oxford Hughes, A. 1989 Testing for language teachers, Cambridge University Press McNamara, T. F. 1996 Item response theory and the validation of an ESP test for health professionals. Language Testing 7 (1) 52-75 OSullivan B. 2000 Exploring gender and oral proficiency interview performance, System 28 (2000) 373-386 Ross, S. and Berwick, R. 1992 The discourse of accommodation in oral proficiency interviews, Studies in Second Language Acquisition, 14/2 159-176 Taylor, T. 2000 Investigating the paired speaking test format, Research Notes 2, University of Cambridge Local Examination Syndicate, English as a Foreign Language (EFL) Young, S. and Milanovic, M 1992 Discourse variation in oral proficiency interviews, Studies in Second Language Acquisition 14/4 , 403-424

159