Appendix C, Part 3 of _Teacher development and assessment literacy

Teacher development and assessment literacy

Appendix C:

Assessment Literacy Test for High School Foreign Language Teachers

Part 3:
Test Interpretation

PART III. Test Interpretation

(A) Matching exercise

37. Here is one multiple-choice TOEIC^® Part 3 listening item. The task is to select the best response (A-D)
for the question. Please write at least two points about why this test item is problematic.

	Man:	This job is the worst, isn't it?
	Woman:	I know I shouldn't complain, but things could be better.
	Man:	I don't think so. I've never had a more interesting job than this one.

Q: What does the man think of the job?
(A) It's a bad job.	(C) It's a great job.
(B) It could be better.	(D) It could be more interesting.

Source: Longman Preparation Series for the TOEIC Test - Introductory Course,
(3rd. Edition, 2005, Pearson Education.) (p. 71 of text and p. 205 of tape script)

Problematic Issue #1:
Problematic Issue #2:

38. Here is a multiple-choice item from an entrance exam.
The task is to select the correct accent of the two underlined words.

A group of international journalists published (a) photographs of whales in a magazine which discusses (b) ecological changes.
(1)	(a) phótographs	(b) ecólogical
(2)	(a) phótographs	(b) ecológical
(3)	(a) photógraphs	(b) ecólogical
(4)	(a) photógraphs	(b) ecológical

Source: 2006 Japanese University Center English Examination, Section 1-A, Version 2110.
What construct, if any, is this test item measuring?

39-41. Here is a table comparing how 47 Japanese university students performed on two different listening tests.
Notice the way the table is constructed and the way the data appear, then complete the tasks below.

Table 1: Subjects' listening scores

Test	No. of Items	Average
TOEFL	50	17.96 (35.91%)
IELTS	40	13.32 (33.32%)
TOEFL	90	31.28 (34.75%)

Source: Squires, T. (2004, Spring). Classroom implications of authentic listening tasks on standardized tests of English proficiency. On CUE, (12), 1, 10.

Mention at least two problematic points with the information that is presented in the table above
and also two additional statistics that should be probably included with this data.

39. Problematic Issue #1:
Problematic Issue #2:

40. Missing Statistic #1:
Missing Statistics #2:

41. Based on the data presented here, what sort of interpretations can be made about how closely
the three tests in the table above correlate for this survey group?

42. If 25-30 students take a placement test, what's the optimum number of standard deviations within the range?

43. If the range of a placement test for 30 students is only 2 standard deviations, what's this suggest about the test?

44. If the standard error of measurement for a test equals or exceeds the difference for the grade divisions
in a test, what does this say about the test?

cont'd.

Main Article

Appendix A: I II III IV

Appendix B

Appendix C: I II III IV

2006 Pan SIG-Proceedings: Topic Index

Complete Pan SIG-Proceedings: Topic Index

[ p. 71 ]