Koo Yew Lie and Harriet Wong
Universiti Kebangsaan Malaysia

Background

The B. A. Integrated English Proficiency course was designed primarily for first year undergraduates to upgrade their general language proficiency so that they could cope with the content courses in the B. A. in English Studies programme. Apart from objectives related to reading and writing, another objective of the course is to equip students with oral communication skills for interacting in everyday situations, hence, the corresponding need to assess oral proficiency. Several assessment procedures have been tried out, including brief oral presentations on given topics and oral interviews with the rater as the interviewer. However, these were found to be generally unsatisfactory because they were not sufficiently communicative: they did not reflect the kind of oral interaction activities and tasks students were involved in while following the English course. More recently, a different testing procedure was developed which takes into account the dynamic, interactive quality of real-life communication. The core of this testing procedure is what we call 'an oral interaction task'.

An Oral Interaction Task

The justification for adopting the task as a testing unit rests on the premise that authentic language use develops when the participants have a purpose for communicating. Johnson (1979:20) when arguing for 'task-oriented teaching' says:

We cannot expect listeners to approach interaction in a state of readiness, to learn how to scan for pragmatic information, unless we provide them with a reason for scanning; nor can we expect them to evaluate incoming information against a speaker aim, unless we provide them with a speaker aim (a communicative intent). Finally, we cannot expect them to make appropriate selection from meaning potential unless they have an intention from which to derive meaning.

The same arguments that Johnson puts forward for 'task-orientated teaching' also apply to communicative language testing.

For our purpose, an oral interaction task is one which involves at least two participants who have to draw upon their linguistic resources and communicative strategies to achieve a well-defined goal (e.g. deciding on the menu for a dinner party, organizing a picnic, etc). Therefore, a task would necessarily consist of the following components:

Statement of the problem

The task is essentially problem-solving, e.g. participants have to come to a decision regarding a problem after some discussion. An example of a problem-solving task reads as follows:

You and Student A are about to move to a new flat in Bandar Baru Bangi. A lot of things have to be done before you can move in. Decide on the things to be done and who is going to do what. Try to distribute the work fairly between your flatmate and you.

The task is broken down into several enabling stages so that the participants will know when it is their turn to speak. A task sequence is provided to guide and facilitate the activity but students need not adhere to it strictly. They can, based on addressor/addressee responses, modify the sequence somewhat as long as there is an outcome, a clearly defined product. Examples of possible task sequences for a simple and a more complex task are given below:

For a simple task (See Sample Task 1 in Appendix 1):

For a more complex task (See Sample Task 5 in Appendix 1):

We are aware that in real-life conversations, the interaction sequences are much more interlocking and unpredictable than those given above.

Porter (1983:192), when listing the characteristics associated with communicative testing, identifies 'unpredictability' as an important feature. 'lt is a central characteristic of natural communicative interaction that both form and, to a considerable extent, content of language produced is unpredictable. The design of a test should be such as to allow for such creative unpredictability.'

In our tests 'unpredictability' is also a concern but it is scaled down to make the task more manageable for the majority of our students and to save time in the actual assessment.

Information Input

In addition to the statement of the problem and the task sequence, some information necessary for the completion of the task is also provided. It may take the form of pictures, tables, charts or short written texts. For most tasks, the information input may not be accessible to both participants, thus creating an information gap. This makes it necessary for the participants to exchange information or in other words, to bridge the gap in order to complete the task (Johnson, 1979). Sometimes, an opinion gap is built into the task so that students have to express their opinions on an issue and come to a consensus or work out an acceptable compromise.

Context

As Morrow (1979:149) points out, 'any use of language will take place in a context, and the language forms which are appropriate will vary in accordance with this context'. Context refers to the context of situation, e.g. physical environment, role/status of participants and attitude/formality. The context and the topic will influence the kind of language produced in the interaction, therefore it is necessary to specify them in the task. The table below provides some examples of possible contexts:

Some General Considerations

Based on feedback from raters of this oral proficiency test, we have found the following considerations to be pertinent:

1. The tasks should revolve around topics that are socially and culturally familiar to students. It would be unrealistic to expect students to handle socially and culturally alien topics. For example, students who have never set foot in a five-star hotel or a fancy French restaurant would not have adequate content to cope with a task involving such settings. Instead, topics which are close to the students' experience like personal relationships, accommodation and their studies would be more relevant and, therefore, motivating.

2. The tasks should be pitched at the conceptual level of the students so that they will be able to draw upon their prior knowledge, if, and when necessary. It would, for example, be conceptually too demanding to ask them to plan a programme for visiting professors to UKM or to project future developments for UKM, especially within the time constraints of a test.

3. The students should be carefully paired in terms of personality and language proficiency. It was found that a student who had a more aggressive personality when paired with a timid person tended to monopolize the interaction. In an extreme case, the latter would be so threatened that he spoke minimally or merely echoed his partner. Similarly, it was found that when a good student was paired with a very poor one, the same pattern of interaction occurred. It is recognized that in real-life conversations, the speakers are not balanced in terms of personality and proficiency. But for testing purposes, the conflict of practicality versus authenticity has to be resolved and in this case, practical constraints are considered more pertinent.

4. Student-student ratio of speaking time should be reasonably balanced. Firstly, this is to ensure that neither participant is unfairly burdened with the brunt of the task. Secondly, for testing purposes, it is essential that both participants be given enough opportunity to speak so that raters will get sufficient oral production on which to make a fair assessment. In view of this, the task sequence has to be carefully staged. Admittedly, this staging of the task would affect the authenticity and unpredictability of communication but these features of natural language use have to be compromised. A completely authentic, global communicative task would be too time-consuming and unmanageable.

Testing Procedures

At this point, it is necessary to consider the following aspects in relation to the oral interaction test:

Test format

1. The two participants involved in an interaction are each given an activity card. Included in the card is a description of the task, some information for carrying out the task and the stages of the interaction. The information given in both the activity cards need not be the same. (See Sample Tasks in Appendix 1.)
2. The number of participants is limited to two. This is done, firstly, because students at this level of proficiency may have difficulty interacting with more than one person. Secondly, this is to facilitate raters' assessment of students. Based on previous experience, raters have found it difficult to reliably assess the many dimensions of language use when there were more speakers.

Administration of the test

The students are tested in pairs. They are given about two minutes to study the activity card on their own and are not allowed to communicate with one another at this stage. Each pair of students is then given eight to ten minutes to complete the task. Our raters have generally found this time period adequate, even for the weaker students.

The moderator

Every class in the B. A. Integrated programme is taught by two instructors and one of them will serve as the moderator in the test. The role of the moderator is to help reduce anxiety in the test situation since she is known to the students. In the event of a student misinterpreting the task, or finding it difficult to cope with the demands of the task, the moderator steps in and directs the activity.

The raters

The raters are also the instructors of the course although we make sure that no instructor evaluates her own students. The raters therefore, share a common understanding of the skills involved in oral communication at this level. This shared understanding and intuitive sense of what is 'good', 'fair' and 'weak' communicative ability have helped the raters to be reasonably consistent in their judgement. Nevertheless, there is still a need to train the raters to ensure greater inter-rater reliability. We would concur with Morrow (1979:154) that 'informal experience suggests that standardization meetings (should) enable fairly consistent scorings to be achieved'.

Several training sessions were organized where raters listened to recordings of actual student interaction and assessed them using the rating scale. The students selected for these recordings were roughly representative of students with 'good', 'fair' and 'weak' language ability, illustrating the different points on the rating scale.

After listening to every interaction activity, the raters discussed the students' performance and decisions were reached as to the relative communicative ability of each. In addition to this, the raters modified the rating scale slightly, defining more specifically the guidelines for each subcategory in the scale. (See Oral Interaction Assessment Scale in Appendix 2)

Instructors of the course were involved in the development of the tests right from the beginning to exploit the washback effect of testing on classroom practice. Communicative language teaching is much more likely to occur when there is communicative testing (A. Hughes in Porter, 1983). As Swain (1984) points out: 'It has frequently been noted that teachers will teach to a test; that is, if they know the content of a test and/or the format of a test, they will teach their students accordingly.' In fact, in our experience at UKM, the instructors have on their own developed oral interaction activities for their classes and made student learning more communicative.

The oral interaction assessment scale

The scale is essentially analytic in nature whereby student performance is scored in terms of each specified criterion of oral production and these are totaled to give an overall score. However, there is one component in the scale which is holistic in nature and requires overall impressionistic marking. This holistic component is incorporated to accommodate general features of communicative ability that are not accounted for in the specific criteria. (See Scale in Appendix 2.)

In the past, our raters have used different types of rating scales but have found the analytic scale easier to work with. As the scale specifies exactly what to look for in an oral interaction, scoring can be quickly and reliably (at least, reasonably) done. The test scores obtained, when using this scale, have been found to be close to instructors' ranking of students' oral ability.

This scale does not take into consideration higher-level communicative criteria like 'flexibility' and 'range' because most of our students do not demonstrate these at their level of proficiency. It is unrealistic to expect them, at this stage, to come close to near-native proficiency.

Future Projections

This oral interaction task-based test is still at its experimental stage. It is envisaged that with increased experience and insights gained from further trials of the test, a more valid and reliable instrument will be developed. Future tests will include different and more complex task-types to assess students at higher levels of proficiency. Although many questions regarding the assessment of oral production have yet to be answered, it is hoped that our experience at Pusat Bahasa, UKM, has made some contribution to the problem of evaluating oral proficiency of ESL learners.

Appendix 1

Sample Task 1

You are Student A.

You are back home on holiday. Your ex-schoolmate visits you to find out about your experiences as a first year student in UKM. He hopes to join UKM next year as an Arts Student.

1. Respond to B's greeting.
2. When asked, give him this information:

1. the nature of your course
2. some problems you faced in your first year

1. Advise him on how to prepare for life in a university.

You are Student B.

You are visiting an ex-schoolmate who is back home for holiday from UKM after his first year. You hope to join him in UKM next year as an Arts student.

1. Greet him.
2. Ask him for information about:

1. the nature of his course
2. some problems he faced in the first year

3. Respond to his advice, when given.
4. End the conversation.

Sample Task 2

You are Student A.

As organizers of a tea-party for the English Club, discuss and decide on the types of food you are going to order from the canteens. You have about $2.50 to spend on each person.

The table below provides information on the varieties of food and their price in Canteen A.

1. Present the information on the food available at Canteen A to your partner.
2. Note down in the worksheet the information given by your partner. If necessary, ask questions to clarify the information given.
3. Discuss and decide on the types of food you are going to order for the tea-party.

You are Student B.

As organizers of a tea-party for the English Club, discuss and decide on the types of food you are going to order from the canteens. You have about $2.50 to spend on each person.

The table below provides information on the varieties of food and their prices in Canteen B.

1. Present the information on the food available at Canteen B to your partner.
2. Note down in the worksheet the information given by your partner. If necessary, ask questions to clarify the information given.
3. Discuss and decide on the types of food you are going to order for the tea-party.

You are Student A.

As student representative of hostel Z, discuss and decide on improvements to the facilities before you make your recommendations to the official in charge of the hostel.

The table below provides information on the most popular leisure-time activity of students in hostel Z in UKM.

4. Present the information on the most popular leisure-time activity to your partner.
5. Note down in the worksheet the information given by your partner. If necessary, ask questions to clarify the information given.
6. Discuss and decide on improvement to the facilities.

You are Student B.

As student representatives of hostel Z, discuss and decide on improvements to the facilities before you make your recommendations to the official in charge of the hostel.

Below is a list of the facilities available at hostel Z.

Facilities available at hostel Z

1. One television set in a small room with 20 chairs.
2. Three carrom boards, one in poor condition.
3. Five badminton courts but only one with lighting.
4. Two new squash courts.
5. One table-tennis table but without a net.

A. You are Student A.

The second semester is ending soon. Your B.A. Integrated English Class has plans to hold a tea-party for lecturers and students.

You and student B are organising the party.

B. You are Student B.

The second semester is ending soon. Your B. A. Integrated English class has plans to hold a tea-party for lecturers and students.

You and student A are organising the party.

1. When you meet, present to him your ideas when asked: - 2. Listen to his suggestions and note them down.

3. Come to a compromise and finalise your plans.

Student A

You are the chairman of a committee in charge of organising an educational tour to a local hotel.

1. Find out from a committee member the following information and note them down.

1. number of students going
2. interests of the students
3. time available for the visit

2. Work out a suitable programme with your committee member.

Student B

You are a member of a committee in charge of organising a visit to a local hotel.

1. When asked, provide your chairman with the following information:

1. number of students going - 20
2. students' interests:

1. prospects of a career in the hotel industry
2. meeting people in the hotel industry

3. students are free for half a day

2. Together with your chairman, work out a suitable programme.

References

Alderson, C. J. (1985), 'Evaluation', Lancaster Practical Papers in English Education, Pergamon Institute of English.

Brumfit, C. J. and Johnson, K. (eds.) (1979), The Communicative Approach to Language Teaching, English Language Book Society/Oxford University Press.

Swain, M. (1984), 'Large Scale Communicative Language Testing', in S. J. Savignon, M. S. Berns (eds.), Initiatives in Communicative Language Teaching, Addison-Wesley.

Porter, D. (1983), 'Assessing Communicative Proficiency: The Search for Validity', in K. Johnson, and 0. Porter (eds.), Perspectives in Communicative Language Teaching, Academic Press.

Van Els et.al. (1984), Applied Linguistics and the Teaching of Foreign Languages, Edward Arnold.