These proceedings present a record of some of the papers presented at the JALT
Pan-SIG Mini-Conference held on May 11th-12th 2002 at the Kyoto Institute of Technology.
Two of the SIGs involved: CUE (College and University Educators) and TEVAL (Testing
and Evaluation) combined their efforts to produce this volume because they saw a firm
link between their themes, Curriculum Innovation and Testing and Evaluation in the
21st Century.
Most universities and colleges in Japan are changing their curricula as a whole
and their foreign language curricula in particular for a number of reasons. There is
increased competition for students due to the changing population structure. Being seen
as innovative and producing new and interesting curricula is an attempt to attract more
students. There is also a realisation that students go through university language programs
without much change in their language ability. Many institutions are trying to rectify this
through structural and methodological changes at curricular, course and class delivery
levels. A third influence is the Japanese Ministry of Education, Culture, Sports, Science
and Technology's desire to improve tertiary level education standards through policy
changes at tertiary, secondary and elementary levels. Integral to the innovations they are
recommending are changes in entrance exams, placement procedures for levelling or
streaming students and assessment procedures for monitoring student progress.
The papers in this volume are divided into four sections: curriculum innovation issues,
pro-autonomy curriculum innovations, entrance examinations and placement, testing
within the curriculum. A brief summary of each section follows.
Curriculum innovation issues
Richard Blight opens this section with a description of the processes involved in
moving from grammar-translation-based teaching to communicative methods in a national
university. This paper illustrates that simple methodological changes imply alternative
structural arrangements for schools and deep policy shifts in order to be effective.
Next, Anna Seabourne explicates the changes in a private university attempting
to move toward a content-based program. She also recounts what was learned from
this methodological shift, emphasizing how all changes within the curriculum should be
monitored and evaluated.
Gregory D. O'Dowd then explores different ways in which curriculum change can
be implemented and presents a case study of the evaluation of curriculum innovation
at a private university.
Exploring concerns teachers have about curriculum change, Neil Cowie details
the impact of curriculum change on a faculty in a national university. He emphasises the
need to develop more collaborative cultures in the staffroom in order to involve faculty
members more in the curriculum change process. Similarly, Charles Kowalski investigates
the causes of teacher stress and suggests ways in which schools can become more
caring places that listen to teachers rather than merely tell them what to do.
[
p. 1
]
Finally, Alan S. Mackenzie explains the changes currently underway in Thailand's
national curriculum providing a different perspective on how another country is reforming
EFL teaching and learning.
Pro-autonomy curriculum innovations
Echoing the 2001 CUE conference theme of Developing Autonomy, some presenters
discussed curriculum innovations that focus on development of autonomy in its many
guises.
The first paper by Dr. Alan Brady moves from personal reflection to detailed exposition
concerning autonomy in higher education. He outlines a framework for encouraging
students to explore, think critically and study authentic language use.
Stacy Vye et al continue this theme from the teacher's viewpoint by describing a
workshop in which they facilitated from the audience ways to become aware of curriculum
constraints and how we can operate within them on a personal level.
Finally Ikuko Kondo and Carolyn Obara and Robert Kirkpatrick demonstrate classroom-
level techniques for respectively stimulating personal interest through videos and bringing religion into the classroom.
Entrance examinations and placement
JD Brown starts this section by describing how his attitudes toward Japan's entrance
examination system have changed. Emphasizing that foreign teachers should not be afraid
of taking pro-active roles in voicing concerns about unprofessional testing practises, he
encourages people to consider test impact more deeply. Brown then outlines six general
problems with Japan 's testing system and some specific ways to remedy the situation.
Tomoyasu Akiyama then considers how the English section of the high-school
entrance exams in Tokyo are conducted. He mentions a proposal to make such exams
more interactive and authentic by including direct speaking tests as an exam component.
Akiyama asserts that this would enhance the construct validity of the exam and create
more beneficial washback. He further stresses the need to develop more data about
junior high school student performance and build up a roleplay task bank.
The next three papers in this section came from the CUE Roundtable on Leveling,
Streaming and Placement Testing. Deborah Bollinger first describes how Tokai University's
streaming system was developed, listing questions administers need to reflect on in
implementing a similar system. After this, Amanda Gillis-Furutaka and Nobuko Sakurai
describe how the curricula at their university changed recently as streaming was
introduced. The need to inform students clearly about the purpose of streaming and to
obtain their consent when changing levels is underscored. Finally, Alan S. Mackenzie
mentions how his school evolved from a two-level into a three-level streamed program
in order to meet the needs of students with limited foreign language skills. He points out
that streaming is worthwhile if the criteria for each level are clear and well understood
by students.
Testing within the curriculum
The next five articles explore the testing-curriculum interface further, and also mention
some innovative approaches to language assessment.
Richard Blight's article points out how one section of a test of English proficiency for
Australian immigrants was evaluated. Underscoring that test revision should be conceived
of as a cyclical process, he emphasises the need to constantly reanalyze the degree to
which test results, test specifications, and program objectives correlate.
[
p. 2
]
After this, Mohd. Sallehhudin Abd. Aziz specifies how a test for law students at a
university in Malaysia was designed, constructed, and validated. Factors to consider in
developing a large-scale EAP proficiency test are discussed in detail. The model for test
development outlined in this article provides many helpful tips about how to accurately
assess English academic ability.
Alistair van Moere and Francis C. Johnson provide an overview of how a video-
mediated proficiency test at a private university relates to the overall curriculum and
academic goals of that institution. The test they outline is used not only for streaming,
but also to allow students to bypass some courses. Moreover, the school uses it to set
performance standards for required courses. Ways that the curriculum at their school is
being individualized as a result of this streaming process are also recounted.
After overviewing how verbal reporting (VR) techniques are generally used to obtain
respondent data, Bob Gibson mentions some of the shortcomings of such procedures.
He then suggests an alternative approach to obtaining feedback by having informants
categorize their behaviors in terms of a set of a priori set of codings. The benefits and
drawbacks of this procedure are then outlined. Related to this, Ellen Head briefly overviews
how cloze tests differ from C-tests, mentioning some resources to learn more about C-tests.
The final article is this volumes concerns how to best assess EFL writing compositions.
Since it is often impractical to have multiple raters and seldom reliable to have only
one rater, Yuuji Nakamura suggests a technique of using paired raters. Various factors
related to the rating process are then statistically analyzed through a many-faceted
Rasch measurement (FACETS) model. The results indicate that a paired rating system
can provide a practical and reliable way to assess compositions.
Conclusion
What this book attempts to do is outline some of the curricular innovations taking
place in Japan today and demonstrate how the connection between testing and curriculum
is both far-reaching and interactive. When so many institutions are becoming change-
oriented, it is important to monitor those changes and examine the processes they, their
faculties and their students go through. As educators, we should be striving for congruence
between what we purport to teach and how we purport to measure the results of that
teaching. If there is a high degree of alignment between these two important facets, then
beneficial washback can be enhanced. The fact that there is often a lack of congruence
between curricular content components and testing is borne out by many of the articles
herein. Nonetheless, some innovative and practical ways to foster that connection are
underscored. This volume demonstrates that the relationship between curriculum and
testing is a vibrant research area. It also indicates many key areas in the testing-curriculum
matrix which remain to be explored.
Main Index