The English Teacher Vol XIX July 1990





The Relationship of Cloze Test Format to Motivation with regard to Performance

JAYANTHI THASARATHAPANY
University of Hawaii

 

 

Introduction

Motivation has not been proven to have a direct causal effect in second language acquisition. However, the importance of it has to be taken into account. As there have been differences in the way researchers define motivation, I have chosen to look at it as a stimulus in a learning situation, My study moves away from the traditional approach of linking motivation with attitude. Researchers who have advocated a non-traditional approach to motivation have surmised that content or format could have an effect on motivation. By administering a doze test with different formats, I will attempt to test if format does indeed effect motivation.

 

Background

The literature available on the topic of motivation, shows that the very definition of motivation differs. Gardner and Lambert (1968) linked motivation with attitudes towards the community of speakers of the target language. The Gardner and Lambert model have influenced how researchers have looked at motivation. They expound on this further by drawing a distinction between integrative and instrumental orientation to L2 learning. Brown (1981) distinguishes motivation into three types: global motivation, situational motivation and task motivation. McDonough (1981, p.143) states that motivation has been seen as,

"...a general cover term - a dustbin - to include a number of possible distinct concepts, each of which may have different origins and different effects and require classroom treatment."
The concept that I would tend to agree with would consider motivation in terms of (1) the micro level; (2) the classroom level; (3) syllabus level and (4) informal level, including out of class and more long term factors (Schmidt and Crookes 1988). This view of motivation takes into account the fact that although motivation is an affective factor there are non-affective considerations.

The implications of current conceptions of motivation suggested by Schmidt and Crookes (1988) are to do with the micro level, the classroom level, the syllabus/curriculum level and factors outside the classroom. They have indicated that there are sub-components which have relevance to the current conceptions. Preliminaries, materials, activities, feedback and effects of student evaluation are those factors identified at the classroom level.

The focus of this paper is on motivation at the classroom level in terms of the tests used.

In a learning situation, there obviously will be a variety of interpretations that could be matched to any given situation. Age, personality, socio-economic background are a few of the factors which can affect individual differences in motivational patterns. However, the factors mentioned above are not the only ones which determine motivational patterns. Performance level is a product of a variety of the factors mentioned above. The choices made, the activity level and material preferences are likely to be reflected in the performance level. Though it has been argued that performance is a crude measure of motivation, it is a behavioural pattern that is typically taken seriously in the discussion of motivation. It has been said that performance level is the "bottom line" in the rationale for studying motivation. The task employed in the teaching situation can carry several features that may affect meaning. One feature could be that the task itself might elicit personal information. The other feature is the information that is made available through a variety of ways. Obviously, the conveying of information influences the options perceived by the person.

Oddly enough, there have been no studies showing that factors like content or format are direct indications of motivation in second language learning. Schmidt and Crookes (1988) state that recent textbooks which are communicatively structured use varied typographical layouts, colour illustrations, photographs and page formats which were said to be taken from journalism. This paper attempts to find out if format can induce motivation An terms of performance in a doze test as there do not seem to be many studies that have done so in the format of tests. The study done for this paper will attempt to show a relationship between test format and motivation in terms of performance.

 

Procedure

A placement test used by an English Language Program at the University of Hawaii was administered to a class of 20 students. The cloze test consisted of 35 items which had been revised several times prior to this. For this study it was presented in two formats. Cloze test A had the passage on the left and had blanks which were on the far right of the passage. These blank spaces were numbered to correspond with the numbers in the passage which indicated where answers were required. Cloze test B had the blank spaces incorporated into the passage.

A class of 20 was randomly divided into two groups: group 1 and group 2. The two formats of the same test was administered to the two groups but the sequence in which they were given was different. Group 1 was given Cloze test A first and after they had completed the test then they answered Cloze test B. Simultaneously Group 2 had been administered Cloze test B and then Cloze test A. They were given 30 minutes on both the tests. This method of counter balancing was used to integrate the scores unambiguously. Upon completing both formats, the students were requested to respond to a questionnaire in which they could indicate their preference.

The scores from the tests and questionnaire measure were recorded. Then the mean was calculated for the scores from test A and test B respectively for the two groups. The overall mean for both the tests was also calculated. The t-test for independent orders was used and the probability was recorded. A Pearson correlation coefficient was calculated to show correlation between scores on Cloze test A and Cloze test B.

 

Sample

The cloze test with two different formats was administered to a class of 20 students in an ESL class at a private college. The students were between the ages of 19 - 23 and had been in Hawaii for less than a year. A majority of the students were from Hongkong. There were also a number of students from the People's Republic of China. They had all had some instruction in English prior to coming to Hawaii. They had scores of 500-550 in the TOEFL examination.

 

Results

The majority of the students preferred Cloze test B. This was true for both groups of students regardless of which test they answered first. Ninety percent of Group 1, who did Cloze test A first, preferred Cloze test B to Cloze A. Only ten percent preferred test A. Group 2, who were given Cloze test B first, also preferred Cloze test B. Seventy percent of the students in Group 2 chose Cloze test B. Thirty percent chose Close test A (Table 1).


Preference Group 1 Group 2
A 10% 30% B 90% 70%
Table 1 Percentage of preference measure

 

A t-test shows the difference between means for the scores of Cloze test A and Cloze test B for all 20 students. There is a significant difference of 0.03 (p. = 0.5) probability. This indicates that there is only a small probability of scores between Cloze A and Cloze test B due to chance. In turn, it implies that there is a real difference. As all other factors are the same except for the format, I conclude that it is the format which brings about this difference.

The mean for the preference measure also indicates that Cloze test B is preferred to Cloze test A. Group 1 had a mean of 36.20 for Cloze test B and only 14.40 for Cloze test B but had a mean of 21.10 for Cloze test A. For the whole of test A, the mean is only 17.70 (see Table 2).


Group Preference A Preference B
GP 1 14.40 36.20 GP2 21.10 28.60 GP1+2 17.70 32.40
Table 2 Mean for Preference Measure

The majority of the students seem to have done better on Cloze test B regardless of whether they did the test first or last. Table 3 shows the mean for the test scores for Cloze test A and B. The mean for Cloze test B is larger than the mean of Cloze test A for both groups. Group 1 showed a greater mean for Cloze test B. It seems logical that if they sat for the same test their scores would increase. But Group 2 who took Cloze test B first, also had a higher mean. The students from Group 2 had a mean of 13.00 for Cloze test B. They only had a mean of 11.90 for Cloze test A even though they did it after Cloze test B. The whole class had a higher mean of 32.00 for Cloze test B. The mean for Cloze test A for the group of 20 students is only 17.00.

	
	 Sequence	Group		Cloze A		Cloze B
	 A/B		GP1		9.40		10.00
	 B/A		GP 2		11.90		13.00
			GP 1+2		7.00		32.00
Table 3 Mean for Cloze test scores

 

The Pearson correlation coefficient reveals a high, positive correlation of 0.87 (p.c.=0.5) (see scatterplot) for the scores of Cloze test A and B. This shows a positive relationship between the two sets of scores. This indicates that there is no large difference between the two tests if performance can be used as a crude measure.

 

Discussion

Based on the results of this study, format does make a difference to overall performance. Format of the doze test can be seen as a stimulus for the performance of the students. As such, keeping in mind the most simple definition of motivation as a stimulus, we have a link between format and motivation. This definition of motivation is tested against the current conceptions of motivation for second language learning. However, to say that it is the only link or it is in itself a direct link for language learning would be simplistic.

A decision to change the format of the doze test does not seem to be an obvious factor which could affect performance. The change was based on the fact that Cloze test A could be found to be distracting. The idea to change the format was then presented to a number of ESL graduate students. They were informally asked if they preferred one format over the other. Many of them said they did, but did not know why. Some felt that it did not make a difference.

It is interesting to note that it is exactly the same test content-wise. The print and the font size was also the same. Both tests were administered simultaneously to the students from one class. The format was the only physical difference. It could be argued that since Group 1 had taken Cloze test B after they had done Cloze test A, it is obvious that they preferred Cloze test B. However, Group 2 had also preferred Cloze test B though they had taken that test first. The scores also showed that both groups had performed better on Cloze test B, regardless of the sequence the tests were administered. By using a counter-balanced design, I was able to deal with the argument stated above.

Ten percent of the students had a higher total preference.

Ninety percent of the students thought that Cloze test B was clearer and easier to read. Though the students were given the same amount of time for both tests, they said in the questionnaire that Cloze test B was quicker to answer. It also allowed them to check their answers better. They also thought that writing the answers in the right column was distracting. At this point, I would like to reaffirm that the only obvious difference between the two tests was the format.

Based on the findings, I conclude that answer blanks which are incorporated into the doze test are less distracting. It makes the test on the whole easier to read. I can only assume that the continuity of the sentences in the test is not disrupted and this plays a part in the overall impression of the student. This is in contrast to writing the answers of the test in a column on the right. Therefore, a difference in format can affect performance and performance itself is a rationale for studying motivation. It can act as a stimulus as presented by the preference and test scores. It would be interesting to try this study with a large sample and other formats.

 

Conclusion

The current implication of motivation for second language learning has started a trend which takes into account the teacher's and researcher's perspective on motivation. It also seems to give us an opportunity to actually test factors which are more controlled. These factors in turn could be employed more readily in the class room. This could bring a wealth of information to the teacher and the learner.

 

References

Ames,C. (1984). "Effective motivation: the contribution of the learning environment." In R.S.Feldman (ed.) The social psychology of education (pp. 235-256) Cambridge. Cambridge University Press.

Au, S.Y. (1988). A critical appraisal of Gardner's social-psychological theory of second language (L2) learning. Language Learning, 38(1), 75-100.

Crookes, G. (in press) "Planning and interlanguage development" in Studies in second language acquisition, (11) 4.

Crookes and Schmidt. (1988). "Motivation: Reopening the agenda." Working Papers, University of Hawaii, Hawaii.

Ellis, R. (1985). Understanding second language learning. Oxford: Oxford University Press.

Gardner, R.C. (1979). Social Psychological aspects of second language acquisition. In H. Giles, & R. St. Clair, (eds.) Language and social psychology. Oxford: Basil Blackwell.

Maehr, M.L. and Archer J.(1987). "Motivation and school achievement." In L.G. Katz (ed.) Current topics in early childhood education." (pp. 85-107). Norwood, NJ: Ablex.

McDonough, S.H. (1981). Psychology in foreign language teaching. London: Alien and Unwin.

Oiler, J.W. (1981). Research on the measurement of affective variables: some remaining questions. In R.W. Anderson (ed.). New Dimensions in Second Language Acquisition Research (p.14-27). Rowley, Mass: Newbury House Publishers.



© Copyright 2001 MELTA