Keisen University's Center for English Education and Research by Alan S. Mackenzie

Curriculum Innovation, Testing and Evaluation: Proceedings of the 1st Annual JALT Pan-SIG Conference.
May 11-12, 2002. Kyoto, Japan: Kyoto Institute of Technology.

Keisen University's Center for English Education and Research

by Alan S. Mackenzie

What is CEER?

The Center for English Language Education and Research (CEER) is responsible for coordinating Keisen University's two-year Communicative English (CE) program. The members of the center are drawn from the British and American Studies Department and the International Socio-cultural Studies department. The staff numbers have fluctuated a lot but at time of writing there are five native English speaker and two Japanese members as well as one full-time administrative assistant from the university's administration. All decisions are discussed thoroughly by the center members and careful attention is paid to predicting possible repercussions of university policy on the CE program and possible adaptations of the program within the framework of the university curriculum. CEER is directly accountable only to the General Academic Affairs Committee of the university, which is rare in Japan for a non-departmental body. It has strived to maintain close contact with each department directly by asking departmental committees for input and program feedback. CEER completes all administrative procedures connected with testing, leveling, and separating students into classes for all first and second-year students for around 400-450 students in each year. Once this procedure is complete, the resulting class lists are given to the academic administration office.

What form of leveling does Keisen employ and why?

The CE program has had two levels: Regular (R) and Challenge (C) since its inception in 1995. The initial rationale behind the two-tier level system was to allow the top ten to twenty percent of students to be challenged to develop their English ability as much as they could. In other words, there was a feeling that students with better language ability were held back in some ways by their classmates with lower level language abilities. It was also felt that one of the aims of the CE program was to help students with already well-developed abilities, so that they could develop them as further as much as possible. Any student in R classes that showed promise, could be moved into the C level, while those having difficulty in C were moved into R at the end of the year.

[ p. 120 ]

In 2000 the university doubled the student intake at a time of great economic stress for the country and the education system as a whole, leading to a marked decrease in the academic ability of Keisen students. Coupled with poorly timed infrastructure expenditures, the financial pressures on the school lead to the need to accept as many students as it could to survive.
In the same year Keisen was labeled as a virtual free entry by Kawai Jyuku, a leading Japanese cram school. The failure rate that the university published consisted of the number of students who registered to take the entrance exam but did not turn up. Anyone who sat the entrance exam, except for one or two very extreme cases, received an offer of a place.
The consequent marked decrease in general academic and English language level of students entering the university lead to calls from the full-time teachers to teach English (i.e. non-EFL specialists largely from the British and American Studies department) for a third level of English classes. This level, termed Support (S), was intended to cater for the lowest ten to twenty percent of students and was greeted favourably by the CEER. Plans were made to introduce a more thorough placement system as well as an orientation program for first-year students.

How are students placed within the CE program?

Before academic year 2002, students were separated into two levels using the ITP TOEFL^® test. Aware that the TOEFL^® was not best suited to this task, the university also wanted a standardized measure of students' English language ability to enable external comparison and a measure of student ability over time as part of CE program evaluation. It was also thought that students needed to know their own ability as judged by an internationally recognised measure. The decision to use TOEFL^® was made before the TOEIC's^® rise in popularity among the business sector in Japan and because it was deemed the most reliable method of mass-testing available.
Other options were considered and rejected at different times in the placement procedure's history such as generating our own placement test or interviewing students (too time-consuming and labor intensive for the small full-time faculty); using a published placement test like the Oxford Quick Placement Test (not adopted because of technical issues among the IT staff).
In 2001, the placement procedure consisted of the following:

TOEFL^® scores were sorted and the top and bottom thirty or so students defined.
Students' writing samples were graded as either C, R or S and compared to their TOEFL^® placement level.
A self-assessment of students' confidence of being able to participate in the given level was then taken into account. Due to the unreliability of this instrument however, the self-assessment was given less weight than the other two measures and was used primarily as an arbiter in borderline cases and only when there was an extreme difference between the self-assessment and level at which the students tested.

During the first two weeks of classes (leading up to Golden Week) teachers and students were given the opportunity to change classes within the time period in which their class too place. In order to change classes, students had to complete a "level change form" on which they had to collect the signatures of both class teachers and explain their reasons for changing class. Likewise teachers had to collect the signatures of their co-teacher and the student in question after having discussed the reasons for level change with all relevant parties. Contentious situations were arbitrated by CEER members in an interview with both teacher and student and resolved to the satisfaction of both parties. The total number of student placements changed was eight out of a total of 460 students.

[ p. 121 ]

". . . leveling is worthwhile so long as the criteria for those levels are concrete, and the students understand what they are and what they need to do to be able to progress through them. Streaming students, on the other hand, tends to restrict their options and may lead to self-fulfilling prophecies."

However, in 2002, CEER discovered that TOEFL^® no longer discriminated very well between C and R levels and did not discriminate at all for the S level and the decision to change to Pre-TOEFL^® was made. It remains to be seen whether this is any more valid as a placement test for this program.

Conclusion

Like any good discussion, this one raised more questions than it answered. However, for the most part, we concluded that although both leveling and streaming label students, which may be seen as disadvantageous to lower level students, leveling is worthwhile so long as the criteria for those levels are concrete, and the students understand what they are and what they need to do to be able to progress through them. Streaming students, on the other hand, tends to restrict their options and may lead to self-fulfilling prophecies.
Any form of leveling or streaming must also be in-line with program goals and the measures used to place students should be direct measures of the skills used on the courses the students are to take.
In-house testing or interviewing is the most labor intensive. These are also most likely to be executed without thorough examination of the tests validity, accuracy and reliability. However with appropriate attention to these, they are probably the best solution to ensure that the assessment fits the program. Published tests can be expensive but may offer useful information. However, they may not fit program goals and cannot as yet be used for measuring production skills. There is also some doubt as to whether tests like the TOEIC^® or TOEFL^® really save any administrative labor.
What all of this means is that each institution needs to work out for itself which methods are best for them based on the principles that underpin their program. Hopefully, the descriptions by the presenters above will help to clarify the processes taken in their institutions and help others to make appropriate decisions based on their own language programs and institutional constraints.

2002 Pan SIG-Proceedings: Topic Index

Complete Pan SIG-Proceedings: Topic Index

[ p. 122 ]