So what are we listening for?
by Kristie Sage (Komazawa University & Gakushuin Universities) |
Through comparing and contrasting the English listening sections of the Test of English as a Foreign Language Internet Based Test (TOEFL® iBT) and the 2006 Japanese National Centre Test (J-NCT), the development of a test higher in construct validity for the later is proposed by this research. The J-NCT has a significant gate-keeping function and this is the first study to systematically examine the test items used in the J-NCT English listening section utilizing both empirical and judgmental analyses. This study examines issues such as construct breadth and response formats. It concludes that the listening score of the TOEFL® iBT is a better indication of students' English listening ability than the J-NCT as it is more integrative and representative of both academic and conversational discourse domains. Keywords: Japanese National Centre Test (J-NCT), construct validity, item analysis, empirical analysis, judgemental analysis, constructs, test items, integrative testing, TOEFL® iBT (Test of English as a Foreign Language Internet Based Test)
|
[ p. 74 ]
"In objective-format tests which are favoured for norm-referenced tests, reliability is often upheld – but not necessarily validity. . ." |
[ p. 75 ]
Number of items: | 50* | Mode: | 38 | Standard Deviation: | 7.45 |
High score: | 38 | Median: | 37 | Skewness: | -0.02 |
Low score: | 22 | Mean: | 38 | Kurtosis: | -0.95 |
* 25 items, weighted 2 points per item | Range: | 22-50 |
[ p. 76 ]
Section 1 sample question: |
Question: What does the man order? ![]() Recorded Dialogue:
|
[ p. 77 ]
• Section 2Section 2 sample question: |
Recorded Dialogue:
![]() ![]() ![]() ![]() |
[ p. 78 ]
• Section 3Section 3 sample question: |
Written Schedule:
![]() ![]() ![]() ![]() ![]() ![]() |
[ p. 79 ]
• Section 4Section 4 sample question: |
Spoken Monolog: "... Suddenly, the weather forecasters were shocked to find out that there was not one, but two powerful storms approaching the island. In fact, the first one was being followed by an even more powerful one." Spoken Question:
Why were the weather forecasters shocked? ![]() ![]() ![]() ![]() |
[ p. 80 ]
The TOEFL® iBT in Japan". . . the J-NCT test scores do not clearly reflect how well examinees' can perform in academic settings." |
[ p. 81 ]
[ p. 82 ]
"A procedure that incorporates consistent piloting, analysing and revising of items can only enhance a test's validity." |
[ p. 83 ]
• Proposal 3: Pilot the J-NCT and delete items which perform poorly[ p. 84 ]
[ p. 85 ]
Main Article | Appendix A | Appendix B | Appendix C | Appendix D | Appendix E |
Appendix F | Appendix G | Appendix H | Appendix I | Appendix J | Appendix K |