Curriculum Innovation, Testing and Evaluation: Proceedings of the 1st Annual JALT Pan-SIG Conference.
May 11-12, 2002. Kyoto, Japan: Kyoto Institute of Technology.

English language entrance examinations: A progress report

by James Dean Brown (University of Hawai'i at Manoa)



How do you Japanese teachers of English feel about guns in America? Do you have definite opinions? Perhaps strong negative views? I'm sure some of you do. Well, the reverse is true too: at least some foreigners have strong negative views about university entrance exams in Japan. Is that "cultural imperialism"? Of course not. It is perfectly alright for people to have opinions about aspects of other countries without them becoming cultural imperialists, unless, of course they try to impose their ideas on the other country.
To tell you the truth, despite the fact that I have become known as an outspoken critic of the university entrance examinations in Japan, I must admit that I started out thinking I should just ignore the entrance exams because they are basically a Japanese problem and therefore none of my business. What changed my point of view? Well, I've been coming to Japan for about fourteen years now, and everywhere I've given lectures or workshops, no matter what the topic was, the audience in Japan always seems to steer the discussion around to the university entrance examinations. This went on for a numbers of years during my frequent visits to Japan. People would ask my opinion of the entrance exams, but what they really wanted me to do was make statements about how bad the exams were for communicative language teaching, or how dreadful they were for the youth of the country, or how expensive or unfair they were, and so forth. Through all this, I staunchly argued that the entrance exams are not a problem for an outsider to deal with. This is a Japanese problem, one that only Japanese will ever be able to solve. People argued back that maybe I should speak out on the issue because I can't be fired. In other words, I could say whatever I liked, perhaps stirring up useful discussion of the issues, and then leave, without suffering the consequences that teachers in Japan might face.
Somewhere along the line, another factor began to play on my attitudes toward the entrance examinations. I was given an opportunity over several years to observe the effects of the entrance examinations on the two sons of a colleague here in Japan. I'll call them Toshi and Yuki. Yuki was the older brother, so he was the first to face the problem. I watched as Yuki, a very bright kid, took eight entrance exams that cost his parents 30,000 to 40,000 yen each over the course of one examination hell season. He failed them all. He was crushed. Naturally, he did not give up. He became a ronin and spent a year at an expensive yobiko cram school (another expense for his poor parents) preparing to do better the second year. This time he picked a smaller number of universities of varying levels of prestige. Ultimately, he succeeded in passing the exam for the international studies department at a well-known university that I will call University A. Yuki thought he passed that exam for that particular department because he was a returnee and the English part of the exam was double weighted. Yuki ended up going to University A and studying a major that didn't interest him at all, simply because that was the entrance examination that he managed to pass. According to him, he didn't care about learning anything; he just needed a degree from University A. He just needed that piece of paper in order to go on to study law, his real interest.

[ p. 95 ]

". . . a great deal has been written about the quality and appropriateness of Japanese university English language entrance examinations."

His brother Toshi was also a bright kid. In fact, he was so bright that he refused to go through the same nonsense that his brother had suffered. Instead, he went straight to a community college in Canada and side-stepped the Japanese university entrance examinations altogether. He is now studying at the University of British Columbia and making plans to go to grad school at a prestigious university in the United States. Seeing all this, his older brother is now also making plans to go the United States to study. Thank you Japan for devising an entrance exam system that is driving some of your best, brightest, and most creative people to leave for North America. We will welcome them and give them plenty of opportunities to contribute to our society.
There is a part of me that says, maybe it would be better to keep the entrance exam system the way it is in Japan, so that bright Japanese kids keep coming to my country. But another part of me realizes that I am watching a terrible injustice. Not just one kid, but many wasted a great deal of time and effort, suffering through many exams, failing, and finally passing only to major in something that wasn't at all interesting to them. Perhaps I should speak out. Maybe I can serve as a catalyst, and in any case, I cannot be fired. I can just say what's on my mind and go back to Hawaii to my regular life.
That is how I decided to speak out on the issue of the English language entrance examinations in Japan, and I will continue to do so here by (a) briefly reviewing the literature on English language entrance examinations, (b) exploring some of the most prominent problems with the entrance examinations, and (c) suggesting some potential solutions to those problems.

The literature on English language entrance exams

During the last fifteen years, a great deal has been written about the quality and appropriateness of Japanese university English language entrance examinations. For any readers interested in the topic, I will simply list some of the key articles on the topic and add a brief note to explain what they were about:
In the articles above, a number of criticisms were leveled and problems were identified, but also, a few solutions were offered. Next, expand on the problems with the entrance examinations and then provide my suggestions for solving some of those problems.

Problems with English language entrance exams

The questions raised about the Japanese entrance examinations were first discussed in Brown (1999). Here, I will offer an expanded list of those questions organized into five main topics (purposes, effects, roles, responsibilities, and perspectives) within the larger framework of Japanese education and society as a whole.

A. Purposes of assessment B. Effects of assessment: Watanabe (1996a & b) made a good start on researching washback effects in Japan, but clearly, many questions remain to be answered. For instance:

[ p. 98 ]

C. Roles of assessment D. Responsibilities of assessment [Questions raised by Messick's (1996) validity ideas]:

[ p. 99 ]

E. Perspectives on assessment [Based on Cronbach's (1988) perspectives on validity]:
These and many other questions need answers in the coming years. They need to be answered not only for the entrance examinations, but also for the many other types of language tests used in Japan if these tests are to be productively reformed along with the rest of the educational system in Japan (as suggested in Mombusho, 2000).

[ p. 100 ]

Some solutions to the problems

The literature offers a number of possible strategies for addressing the questions listed in the previous section. These strategies are summarized here in four sets (test design, test content, logistical, and interpretation strategies) in the hope that some or all of these ideas will be used to help ameliorate the current negative effects of the university entrance examinations (adapted from Brown 2000a):

A. Test design strategies B. Test content strategies C. Logistical strategies D. Interpretation strategies
Conclusions
". . . admission by entrance exam is far from being the only way to get into a Japanese university. Increasingly, there are other possibilities . . ."

I have explored some of the problems with the university entrance examinations in Japan and have also suggested some potential solutions to those problems. However, it is important to recognize that to some degree these issues have become less important in recent years. Various demographic (e.g. the plummeting numbers of applications to Japanese universities), economic (e.g. the dwindling employment opportunities after graduation), and social forces (e.g. the need to accommodate returnees, the emigration of bright students like Toshi and Yuki, and so forth) are leading to changes in admissions policies. In fact, admission by entrance exam is far from being the only way to get into a Japanese university. Increasingly, there are other possibilities: entrance 'by recommendation'; special policies and exams for returnees; automatic admissions for students who attended feeder high schools; the use of multiple sources of information in admitting students; and so forth. Clearly exams are no longer the only way to get into a Japanese university nor are they any longer an isolated issue. The entrance examinations that were originally designed to create a 'level playing field' for all applicants to universities can now be circumvented in a number of ways. In other words, the situation has become considerably more complex and perhaps even less fair than how they were originally conceived.

[ p. 102 ]


I would also like to make it clear that I recognize that problems in admissions decisions are not just an issue in Japan. We have persistent problems in the American admissions systems as well. However, using hundreds of different university entrance examinations developed by amateurs to help raise money for the universities is not on any list of our problems. Naturally, admissions problems also exist elsewhere in the world, especially in Asia. Indeed the stakes are particularly high in Asia because the country that first figures out how to test and teach English effectively may have a distinct economic advantage over other countries. In recent years, both China and Korea have developed unified nationwide professionally developed English entrance exam systems, and thus, they may have developed an important advantage over Japan with its hundreds of entrance exams developed by amateurs. It will be interesting to see if the simpler, more cogent admissions systems in China and Korea do indeed confer an economic advantage during the next couple of decades.
So let me end at the beginning by asking you once again: What do I think about gun control in the United States? And more importantly, what do you think about the university entrance examination system in Japan? I hope you have found something in this paper that has made you think more deeply about the issues involved. The future of Japanese young people is far too important to be left in the hands of the hundreds of teams of amateur test developers in the many universities all over Japan. Think about it, and don't be afraid to express your opinion when the need arises. Maybe I can get you a job in Hawaii.

References

Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing. Language Testing, 13, 257-279.

Brown, J. D. (1987). False beginners and false starters: How can we identify them? JALT Journal, 11, 14, 9-11.

Brown, J. D. (1990). Where do tests fit into language programs? JALT Journal, 12, 1, 121-140.

Brown, J. D. (1993). Language test hysteria in Japan? The Language Teacher, 17, 12, 41-43.

Brown, J. D. (1995a). English language entrance examinations in Japan: Myths and facts. The Language Teacher, 19, 10, 21-26.

Brown, J. D. (1995b). A gaijin teacher's guide to the vocabulary of entrance examinations. The Language Teacher, 19, 12, 25.

Brown, J. D. (1996a). English language entrance examinations in Japan: Problems and solutions. In G. van Troyer (Ed.) JALT '95: Curriculum and evaluation (pp. 272-283). Tokyo: Japan Association for Language Teaching.

Brown, J. D. (1996b). Japanese entrance exams: A measurement problem? The Daily Yomiuri (Educational Supplement, February 5, 1996), 15.

Brown, J. D. (1997). Do tests washback on the language classroom? TESOLANZ Journal, 5, 63-80.

Brown, J. D. (1998). University entrance examinations and their effect on English language teaching in Japan. In J. Kahny & M. James (Eds.), Perspectives on secondary EFL education: A publication in commemoration of the 30th anniversary of the Language Institute of Japan (pp. 20-27). Odawara, Japan: LIOJ.

Brown, J. D. (1999). The roles and responsibilities of assessment in foreign language education, JLTA Journal, 2, 1-21.

Brown, J. D. (2000a). University Entrance Examinations: Strategies for creating positive washback on English language teaching in Japan. Shiken: JALT Testing & Evaluation SIG Newsletter, 3, 2, 4-8. Retrieved October 10, 2002 from the World Wide Web: http://jalt.org/test/bro_5.htm.

[ p. 103 ]

Brown, J. D. (2000b). Give second chances in education. The Daily Yomiuri (Educational Supplement, January 3, 2000), 6.

Brown, J. D., & Christensen, T. (1987, November). Interview: James D. Brown. The Language Teacher, 11, 7, 6-10.

Brown, J. D., & Gorsuch, G. (1995). An inteview with J.D. Brown: Analyzing the value, meaning of entrance examinations. The Daily Yomiuri (Educational Supplement, October, 30, 1995), 15.

Brown, J. D., & Kay, G. S. (1995, November). English language entrance examinations at Japanese universities: An interview with James Dean Brown. The Language Teacher, 19, 11, 7-11.

Brown, J. D., & Yamashita, S. O. (1995a). English language entrance examinations at Japanese universities: What do we know about them? JALT Journal, 17, 1, 7-30.

Brown, J.D., & Yamashita, S. O. (1995b). The authors respond to O'Sullivan's letter to JALT Journal: Out of criticism comes knowledge. JALT Journal, 17, 2, 257-260.

Brown, J. D., & Yamashita, S. O. (1995c). English language entrance examinations at Japanese universities: 1993 and 1994. In J. D. Brown & S. O. Yamashita (Eds.) Language Testing in Japan (pp. 86-100). Tokyo: Japan Association for Language Teaching.

Cheng, L., & Watanabe, Y. (In press). Context and method in washback research: The influence of language testing on teaching and learning. Mahwah, NJ: Lawrence Earlbaum Associates.

Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 3-17). Hillsdale, NJ: Lawrence Erlbaum Associates.

Heyneman, S. P., & Ransom, A. W. (1990). Using examinations and testing to improve educational quality. Educational Policy, 177-192.

Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press.

Irinoda (forthcoming MA thesis).

Kellaghan, T. & Greaney, V. (1992). Using examinations to improve education: A study of fourteen African countries. Washington, DC: The World Bank.

Messick, S. (1996). Validity and washback in language testing. Language Testing, 13, 241-256.

Mombusho (2000). Daigaku nyuushi no kaizen ni tsuite. [About improving university entrance examinations]. Retrieved October 10, 2002 from the World Wide Web: http://www.mext.go.jp/b_menu/shingi/12/daigaku/toushin/001102.htm.

Mulvey, B. (1999). A myth of influence: Japanese university entrance exams and their effect on junior and senior high school reading pedagogy. JALT Journal, 21, 1, 125-142.

Mulvey, B. (2001, July). The role and influence of Japan's university entrance exams: A reassessment. The Language Teacher, 25, 7, 11-17.

Murphey, T. (2000, June). International TESOL encourages assessment literacy among test makers. The Language Teacher, 24, 6, 1-3.

Murphey, T. (2001, October). Nonmeritorious features of the entrance exam system in Japan. The Language Teacher, 25, 10, 1-4.

O'Sullivan, B. (1995). A reaction to Brown and Yamashita "English language entrance exams at Japanese univerities: What do we know about them?" JALT Journal, 17, 2, 255-257.

[ p. 104 ]


Shohamy, E. (1992). Beyond performance testing: A diagnostic feedback testing model for assessing foreign language learning. Modern Language Journal, 76, 4, 513-521.

Stapleton, P. (1996, March). A reaction to J. D. Brown's recent inquiry on the English entrance exam. The Language Teacher, 20, 3, 29-32.

Wall, D. (1996). Introducing new tests into traditional systems: Insights from general education and from innovation theory. Language Testing, 13, 234-354.

Watanabe, Y. (1992). Washback effects of college entrance examinations on language learning strategies. JACET Bulletin, 23, 175-194.

Watanabe, Y. (1996a). Investigating washback in Japanese EFL classrooms: Problems of methodology. The language testing cycle: From inception to washback (Series S, Number 13) (pp. 208-239). Canberra, Australia: Applied Linguistics Association of Australia.

Watanabe, Y. (1996b). Does grammar translation come from the entrance examination? Preliminary findings from classroom-based research. Language Testing, 13, 318-333.

Yoshida, K. (1996a). Language testing in Japan: A cultural problem? The Daily Yomiuri (Educational Supplement, January 15, 1996), 15.

Yoshida, K. (1996b). Testing the bounds of culture. The Daily Yomiuri (Educational Supplement, February 12, 1996), 15.




2002 Pan SIG-Proceedings: Topic Index Author Index Page Index Title Index Main Index
Complete Pan SIG-Proceedings: Topic Index Author Index Page Index Title Index Main Index

[ p. 105 ]
Last Next