May 22-23, 2004. Tokyo, Japan: Tokyo Keizai University.
Learners to Teacher: Portfolios, please!
|
![[ 学習者から教師へ「お願い！ポートフォリオを！」―英語の授業におけるポートフォリオ評価― ]](../Graphics/AppleShimoT.gif){kind=small}
|
Abstract Keywords: : portfolio assessment, learning processes, learner autonomy, active learning, meta-cognitive skills
|
Learners to Teacher: Portfolios, please!
|
|
Abstract Keywords: : portfolio assessment, learning processes, learner autonomy, active learning, meta-cognitive skills
|
[ p. 53 ]
Introduction and research background
Portfolios have a long history of use in L1 situations, finding popularity especially in U.S. elementary schools (Stefanakis, 2002) and at the tertiary level as "exit interviews" for writing programs (Hamp-Lyons & Condon, 2000). Recently, in Japan portfolios have been used as an alternate means of assessment to more traditional testing methods (Mineishi, 2002; Apple, 2003; Shimo, 2003). The current pilot study examines students' perceptions of portfolio creation in an EFL setting, at the tertiary level in Japan.
For this pilot study, questionnaires were collected from sixty-one students in two separate four-year universities in western Japan attending English writing courses. In each course, in addition to class participation and attendance, a portfolio of student-selected work was used as the primary means of assessment. As tests were not used at all for assessment, students were required to submit portfolios in order to pass the courses. Because students from three different EFL writing classes participated, the classes themselves posed the greatest obstacle to the research due to differing course objectives, course materials, and English proficiency levels of students. Unlike traditional tests, however, the portfolio measures individual students' progress rather than the product; when learners are of varying proficiency levels, essay exams following a single standard may not actually assess what they are intended to, placing the validity construct at risk. In this instance, the portfolio fits such a validity construct better due to its ability to assess individual learner progress in the target language.
What exactly is a "portfolio"?
Portfolios can be described as a purposeful collection of students' work. O'Malley and Pierce (1996) differentiate portfolios in three separate categories: "Showcase portfolios," "collection portfolios" (also called "working portfolios"), and "assessment portfolios". In a showcase portfolio, students select pieces which they believe represent their best work. Collection portfolios, on the other hand, contain all the drafts and the final product of all student works. Finally, assessment portfolios require students to select works for assessment according to certain criteria given by the instructor.
| "Rather than focusing on grammar and vocabulary usage, activating the processes of meta-cognitive language awareness is more important when using portfolio assessment." |
These categories, however, are merely neat divisions for purposes of research; in practice, more complicated, and conversely more simple combinations are also possible. For the classes surveyed in this study, the learners were asked to choose selections from their weekly journals, one or two essays including all pre-writing and drafts, and a reflective writing essay on their work for the course. In one class of the three writing classes surveyed, due to concern over English proficiency level students were allowed to write their reflective essays in Japanese. However, writing a reflective essay in one's mother tongue rather than the target language does not detract from the value of the reflection, and in fact can even add to it. Rather than focusing on grammar and vocabulary usage, activating the processes of meta-cognitive language awareness is more important when using portfolio assessment.
The requirements for the portfolio assessment in this study follow the recommendations of Hamp-Lyons and Condon (2000), who identify the three elements of successful portfolios as collection, selection, and reflection. For a portfolio to work effectively as an assessment tool, it must first of all include samples of a student's work rather than all the work done. Furthermore, students must individually choose which pieces to include, though this principle of learner autonomy is tempered by the obvious requirement of the portfolio and the writing class in the first place (see "Learner autonomy through portfolio creation" below). Finally, self-evaluation by the learners is crucial. A reflexive learner actively engages meta-cognitive processes, enabling monitoring and preparing the learner's mind for feedback from both the instructor and peers (Breen & Mann, 1997).
Encouraging learners to become active, reflexive learners is only one of the many strengths of portfolio assessment. Because portfolios contain a record of concrete examples of student work done over time, they can accurately demonstrate a learner's progress in the target language, give learners the opportunity to reflect on their own progress and work collaboratively with peers even after the actual assessment ("grade") has been given, and help learners take responsibility for their own progress toward both class-oriented and personal learning goals.
[ p. 54 ]
Among the weaknesses of portfolios is the major factor of amount of time required. Unlike essay exams, which must be done in a certain amount of time such as a half-hour, portfolios are created during the course of an entire semester or even academic year. For portfolio assessment to be effective, learners must receive feedback from instructors during the creation process. Additionally, most learners in an L2 setting must be guided through the portfolio creation, as they most likely have never created a portfolio before. Finally, there remains an overall perception by both instructors and learners alike that portfolios are "easy," simply because they are not exams. This last point will be addressed in this paper's research.
| ". . . portfolios offer a collaborative assessment . . . partly determined by the instructor and partly determined by the learner." |
Assessment of the portfolio itself can be done either holistically or analytically. Holistic assessments give the entire portfolio a single grade based on set criteria such as voice, grammar, structure, and so on, whereas analytic assessments give each of these criteria a separate grade. Generally, the most effective way to assess portfolios combines both of these assessment methods; individual pieces included in the portfolio can be assessed analytically, while the portfolio itself receives an overall holistic grade. The most important point concerning portfolio assessment is that the assessment is not just based on the quality of the product, for example, an essay on certain specific themes. For classes with students of widely varying proficiency levels, setting a specific standard for essays may turn out to be neither beneficial to the learner nor entirely fair. Though it has so far been difficult to judge the reliability of portfolio assessment, portfolios do appear to promote a "greater awareness of what good writing may be and how it might be best achieved" (Hyland, 2002, p.146) thus raising the validity of the assessment.
Rather than judging a single moment in time, as does an exam, portfolios emphasize individual progress towards goals which the learners themselves help establish. In that sense, portfolios offer a collaborative assessment, an assessment partly determined by the instructor and partly determined by the learner. This assessment includes "not only an assessment of what the result was, but the how and the why: how a student reached the result and why the result came about" (Apple, 2004, p.88). This shift from "producing correct English rather than selecting, organizing, and presenting ideas for effective communication to a reader" (Shih, 1999, p.20) allows both instructor and learner to evaluate progress and effort over time.
Research questions and methodology
Three research questions (RQ's) were devised for this pilot study:
Of the 14 items on the self-report questionnaire, three items were bio-data questions. Eight items consisted of 5-point Likert scale questions combined with an open response follow-up question. These items were based on Shimo (2003) and adapted to measure learner responses from an assessment point of view. The final three items were open response only. The questionnaire was written originally in Japanese by a native speaker of the language and given to students during the final January class of the fall 2002 semester. All student responses to open-ended were written first in Japanese, then translated into English by the researchers. Likert scale question results are tabulated in Table 1.
| Question | N | 5 | 4 | 3 | 2 | 1 | Mean | SD |
| Q4: Did you have opportunities to consider your strong and weak points? | 61 | 12 | 31 | 12 | 3 | 3 | 3.75 | 0.99 |
| Q5: Did you have opportunities to establish reachable goals and objectives? | 61 | 3 | 17 | 6 | 4 | 3 | 3.15 | 0.91 |
| Q6: Did you plan your own learning process to achieve your objectives? | 59 | 2 | 22 | 21 | 9 | 5 | 3.12 | 1.13 |
| Q7: Did you choose your own materials, activities, and class contents? | 59 | 9 | 20 | 11 | 10 | 9 | 3.17 | 1.41 |
| Q8: Did you work without being supervised by another person? | 61 | 19 | 20 | 3 | 14 | 5 | 3.56 | 1.36 |
| Q9: Did you evaluate your progress in English ability? | 61 | 9 | 28 | 15 | 6 | 3 | 3.56 | 1.03 |
| Q10: Has the process of portfolio creation led you to become more active in learning? | 60 | 14 | 24 | 18 | 2 | 2 | 3.77 | 1.08 |
| Q11: Has creating a portfolio led you to become a more autonomous learner? | 59 | 12 | 29 | 16 | 0 | 2 | 3.83 | 1.1 |
[ p. 54 ]
There were three open response items:
Q12 "What was beneficial/what have you learned through portfolio creation?" Q13 "What would you improve next time? Why?" and Q14 "Please write anything about portfolios as a means of assessment."
Of these, the final question elicited responses answering our first research question. It also directly addressed a primary concern many educators have regarding portfolio assessment, namely the belief that students perceive portfolios as being "easier" than traditional tests. As student responses to Q14 will demonstrate, these fears are unfounded.
Student perceptions of portfolio assessment and traditional tests
How do students respond to portfolio assessment as it compares to traditional testing? (RQ1)
Traditional writing tests are usually in these formats: translation, grammar, and short essay exercises. In most short essay tests, students usually do not re-write their essays. Their first draft is accepted as the final product and sole basis for evaluating a student's essay writing ability. There is no way for the teacher to assess the student's grasp of the writing process.
Students themselves indicated in their responses the perceived benefits of portfolio assessment as:
Many students wrote positive comments on portfolio assessment in their responses to Q14 ("Please write any thing about portfolios as a means of assessment."). Out of the 50 statements in the responses, 33 clearly indicate that the student preferred portfolio assessment to testing. Ten more students pointed out how beneficial portfolio assessment was, while not explicitly stating that they liked it better than tests. These statements and some comments written in the responses to other questions (e.g., Q12 "What was beneficial/what have you learned through portfolio creation?") indicate that the students appreciated the above features of portfolio assessment.
Only seven students gave relatively negative comments on portfolio assessment, stating reasons such as portfolio construction was "time-consuming," "too much work," "complicated," and "difficult." This point is important, as it flies in the face of the common "drawback" to portfolio assessment, namely, the perception that portfolios are "easy."
In addition to the above learner-perceived benefits, portfolio use also allows the teacher to see L2 output and "real" language use that students could not demonstrate through traditional testing. Portfolios give instructors another tool to give ample, appropriate feedback to students, to add new criteria to assessment [e.g., attitude (how active and serious is the student?), process (how did the student work on revising processes?), progress (how much has the student improved the skill?)], and to incorporate strategies and skills useful or even necessary for students to become successful language learners (e.g, cognitive, meta-cognitive, and social/affective strategies and skills) in the curricula.
[ p. 55 ]
Active learning, autonomy, and portfolio use
Does portfolio creation promote "active learning"? Do students perceive a connection between portfolio construction and increased autonomy? (RQ2 & RQ3)
Students who engaged themselves with learning actively are usually motivated learners. Conversely, motivated students are likely to exhibit active attitudes toward learning. Students with active attitudes take responsibility and control over their learning and develop greater degrees of autonomy. Such autonomous learners become successful learners, and this success can lead to more enhanced motivation. Portfolio assessment can support this cycle by contributing positively to each of the three factors, "enhanced motivation," "active learning," and "autonomous learning."
Nearly two thirds of the students answered that they (strongly) thought that portfolio creation promoted "active learning" (Q10 "Has creating a portfolio led to active involvement in learning?"). Student responses indicate four reasons for their active involvement: Portfolio construction provided them with (a) a joy of creation and ownership, (b) goal-awareness, (c) individual accountability, and (d) continuous and extended learning opportunities. These reasons are closely related to students' affect, cognition, and meta-cognition, and indicate the connections between enhanced motivation, active learning, and autonomous learning.a. Joy of creation and ownership o "I felt like studying harder because the portfolio is my own [product]." o "Creating a kind of booklet is fun and so I felt like working more actively."
Students seemed to appreciate that they owned the assessment tool and that they could implement their original ideas in it. They enjoyed making choices and decisions (e.g., writing-topics and entries for their portfolio), which allowed them to take responsibility and control over their learning. The tasks for the portfolio assignment did not constrain them from expressing their ideas as much as traditional testing would. Many of the portfolios turned out in the end to be the student's unique, original, special single piece of work.
b. Goal-awareness o "I was conscious that I have to turn in the portfolio at the end of the semester." o "I thought I would have to work on all the assignments [in the semester]."
All the in- and out-of- class activities in the semester were supposed to lead to the students' portfolios in one way or another. If students did not do each activity properly and could not complete their portfolio to submit at the end of the semester, it would have meant that they would not pass the course. It seems that students were more aware of this final stage of portfolio submission while being engaged with each task, probably because they can see how it is related to their final product, which is rather unclear in the case of the traditional, end-of-the-semester exam. One student said, "For tests, we study only for one week, but for portfolios, we make the same amount of effort all throughout the semester."
c. Individual accountability o "Nobody can do it for me." o "I worked hard so I wouldn't make my group members and partners in trouble." o "A portfolio is not something you create by yourself. You have to consider others' opinions. You write articles in a group. I think completing a work with others rather than completing it by yourself is more likely to lead to active learning."
Many collaborative learning procedures were integrated in the process of portfolio creation (e.g., peer review activities), and this seems to have promoted students' sense of responsibility for their individual roles so that their group-mates would not get disadvantaged. This individual accountability seems to have kept them more active in learning. The last of the above three comments clearly indicates that collaboration was greatly appreciated.
[ p. 56 ]
d. Continuous and extended learning opportunities o "I had more opportunities to write in English." o "I have come to use my dictionary more often." o "I have come to like English and watch NHK English programs, study independently at home, and so on." o "I came not to dislike emailing in English and I email in English every week now.
The above statements of students were sample actions showing their "active learning" that they said were caused by portfolio creation. It is more of the tasks and assignments that made the difference in students' acts outside the class. Yet, portfolio creation seems to have eventually increased autonomous learning opportunities outside the class. It is probably because portfolio construction requires long-term work (e.g., one-semester-long), and therefore students continuously need to make efforts for it. Some students tried different English learning tools. Some also seem to have developed strategies for independent learning, such as dictionary-using strategy, as they often had to work on their own, for example, on the revision processes after receiving feedback from others.
In summation, portfolio assessment can help students become motivated because of the enjoyment, responsibility, and goal-awareness that they experience through the construction process. It can also allow students to develop cognitive skills and meta-cognitive abilities (i.e. abilities to plan ways of learning, to choose learning materials, monitor learning process, and evaluate their progress) through their active involvement with learning.
Learner autonomy through portfolio creation
As discussed above, "active learning" or active attitudes towards learning is closely related to autonomous learning. Then, what did the students actually think autonomy means?
Forty out of sixty-one students agreed that portfolio creation had led to more autonomous learning, and such students explained autonomy by referring to meta-cognitive abilities. Their definitions include planning yourself and achieving the plan, taking responsibility for your own role, setting goals for yourself, heading toward them and doing what you have to do, and making a decision about something on your own.
| ". . . the concept of 'compulsory work' and 'autonomous work' are not necessarily dichotomous." |
A couple of students in the study, however, pointed out that a portfolio is what you are forced to make after all, that it is no different from other types of homework, and that it is just a collection of pieces of homework. They seem to be insisting that "you don't call compulsory work autonomous" (cf. Shimo, 2003).
In the formal instruction or classroom settings in language learning, however, the concept of "compulsory work" and "autonomous work" are not necessarily dichotomous, where the choice A or B should be made. One can develop the sense of autonomy even in working on "compulsory" tasks provided by a teacher.
Little (2004) points out the main differences between the concepts of "independence" and "autonomy": independence connotes the ability to do things on our own — totally free from outside help or interference — whereas autonomy means doing things for ourselves, "being self-governing, self-managing, self-regulating." Students who made the above comment that the portfolio, being required for a course grade, was no different from other homework may have confused the idea of independence (i.e., being able to choose not to do any homework, yet pass the course) with autonomy (i.e., being "forced" to do homework, yet having the ability to determine which items to complete within a certain framework).
Such students probably did not have enough opportunities to learn what a portfolio was and did not have deep reflection in the process of making one. Learners should have opportunities to discover what a portfolio is and the purposes of portfolio creation. They can do so by examining sample portfolios and sharing each other's work in progress. The instructor can also explain the purposes explicitly to them. Moreover, Shimo (2003, p. 175) suggests that providing "optional tasks which allow students to make choices" and "decision-making tasks which enable students to plan and organize their learning" will help learners feel the sense of autonomy.
[ p. 57 ]
The importance of integrating collaborative learning processes in portfolio creation should also be pointed out. This can increase the chance to get feedback, to reflect on one's own activities, as well as increase opportunities to examine others' work critically. When learners work on a group project, they often do so through decision and choice making procedures, through which learners should be able to enhance their autonomy.
Conclusion
Learners in this pilot study agreed that portfolios offer an assessment not available through traditional testing: learners can have their learning processes assessed and are provided with the chance to reflect about their own work. Portfolios also can provide learners with ample feedback from both peers and the instructor, increasing cooperative learning and motivation.
| ". . . the benefits of portfolio assessment clearly outweigh the negative aspects." |
On the other hand, possible drawbacks to portfolio assessment include the length of time required both for the learner to assemble the portfolio and for the instructor to assess it. Learners need specific, clear directions about what the instructor expects from the portfolio. Some learners, especially those accustomed to traditional testing methods of evaluation, may be frustrated when expected to set their own learning goals, to choose their own work for portfolio inclusion, and to reflect on their work or that of their peers. Future studies will examine aspects of cooperative learning and the benefits and drawbacks of using CL processes for portfolio creation in the second language classroom.
Despite these concerns, the benefits of portfolio assessment clearly outweigh the negative aspects. Portfolio assessment helps students to enjoy their assignments, while assisting learners in developing cognitive and meta-cognitive abilities. Rather than a mere course grade or percentage in an instructor's class book, portfolio assessment can be an enjoyable learning tool, one which students can reflect upon long after the course itself has finished.
References
Topic Index
Author Index
Page Index
Title Index
Main Index
Topic Index
Author Index
Page Index
Title Index
Main Index [
p. 58
]
