The eight articles in these proceedings are from a conference co-sponsored by the Pragmatics SIG,
Testing & Evaluation SIG, CUE SIG, and the Tokyo and West Tokyo Chapters of the Japan Association
for Language Teaching (JALT). This event was held at Tokyo Keizai University on May 22 - 23, 2004.
Dr. Andrew D. Cohen of the University of Minnesota and Dr. Jenny Thomas of the University of Wales were
plenary speakers and about 40 others presentations and poster sessions were featured during this event.
The relationship between testing, pragmatics, and fluency has been an ongoing research topic for many
language teachers and researchers across the world.
The 2002 and 2003 Conference Proceedings reflect
some of this research. The proceedings of those conferences offered a rich source
of information about various ways testing and pragmatics and university education interlink.
This year's conference proceedings continues to explore themes raised in earlier
conferences.
The opening essay by Dr. Andrew D. Cohen
considers some facets of fluency. After describing early approaches to speech act performance assessment,
some recent efforts to assess speech act ability were mentioned.
Cohen also highlighted some factors often used when assessing speech
acts and described his own work in developing online pragmatics materials.
Kazumi Kimura and Masako Kondo next explore how
Japanese danraku and English paragraphs vary. Kimura and Kondo
note that many Japanese EFL students use danraku-like structures
in their essays. This results in a lack of focus and overall organization.
The authors suggest that the essential features of an English paragraph
need to been explained to EFL writers, and clearly distinguished
from Japanese danraku.
H. P. L. Molloy and Mika Shimura then describe an interlanguage pragmatics
research study which explores how sensitive L2 English users are
to situational differences when making scripted complaints.
After analyzing data via G- and D-studies and some ad hoc scatterplots, they
found that situational variance accounted for less than 3% of the total variance
when making scripted complaints. Their data suggests that participant idiosyncrasies
were likely the major reason for variation in L2 pragmatic performance.
The authors concluded that situational sensitivity may have a less high-profile
role in determining pragmatic behavior than previously suggested.
Keita Kikuchi describes how a needs analysis was conducted
at a university in Tokyo. Since learners, teachers, and program coordinators
tend to have differing views regarding appropriate curricular targets, Kikuchi
underscored the value of obtaining information from each of these groups
via interviews, questionnaires, and observational data.
Rather than basing curriculum development ontop-down management ultimatums,
Kikuchi argues for combining multiple sources of information.
Specifically, Kikuchi attempted to assess six components of Aoyama Gakuin's English program:
(1) curricular goals, (2) salient problems, (3) learners' preferences concerning materials,
(4) the overall English level of most students, (5) the attitudes
towards English among most students, and (6) suggestions for improving the program.
Responses were varied and gave program designers some valuable data
with which to plan the future curriculum.
Joseph Falout then explores ways to teach the TOEIC® communicatively.
A number of practical activities designed to help learners
progress from mere declarative knowledge of linguistic forms towards
a more practical "procedural knowledge" in which
linguistic tasks are performed in real time without too much deliberation are described.
Examples of focused, communicative tasks designed to foster automaticity
are highlighted in his article.
Holistic and analytic writing assessment methods are then constrasted
by Yuji Nakamura, using Weigle's
adaptation of Bachman and Palmer's framework for test usefulness. The author
suggests that "the best practice is to have multiple raters and multiple rating items."
The least recommended practise is to have just one rater with a single-item holistic
scale. In between these two options, one compromise
might be to have multiple raters use a holistic scale. Nakamura echoed Weigle
in pointing out how construct validity and reliability issues should be of primary concern
in testing, and practicality and test impact issues should be of lesser weight.
The article by Matthew Apple and Etsuko Shimo
describes learner attitudes towards portfolio assessment writing projects.
The authors contrast traditional assessment methods with portfolio assessment.
They point out that though portfolio assessment is generally more time-consuming
and difficult than traditional assessment procedures, it is also more
rewarding and seems to promote greater awareness of meta-cognitive skills.
Miyoko Kobayashi's investigation of reading comprehension test performance examines
two of the test method facets proposed by Bachman, namely the nature of the input and the nature of the expected response.
Meyer's model of prose analysis was used to manipulate the nature of the input to create four different types of text
organization. Three different types of response format were used: cloze test items, open-ended questions and summary
writing. Kobayashi concludes that these two variables influence reading comprehension test performance. Furthermore,
it was shown that the influence of the text organization and response format varies between the different proficiency
levels. A follow-up study examined the texts and response formats used for the Center Examination and shows that some
students may have been disadvantaged by these tests.
We would like to conclude by offering special thanks to the following people for their help in organizing this conference:
Chiyo Hayashi, Jeff Hubbell, Tadashi Ishida, Megumi Kawate-Mierzejewska,
Marish Mackowiak, Yoko Mochizuki, Peter Ross, Stan Pederson, Steve Ross,
Chizuko Sato, Chris Sullivan, Sayoko Yamashita, and Junko Fujio.
Without their kind assistence, this event would never have taken place.
- T Newfields, Yvonne Ishida, Mark Chapman, & Mayumi Fujioka
2004 Conference Proceedings Co-editors
