Content-based methods in peer assessment of open-response questions to grade students as authors and as graders

详细信息    查看全文

• 作者：Oscar Luaces ; ^a ; ^{oluaces@uniovi.es} ; Jorge Dí ; ez^a ; ^{jdiez@uniovi.es} ; Amparo Alonso-Betanzos^b ; ^{ciamparo@udc.es} ; Alicia Troncoso^c ; ^{atrolor@upo.es} ; Antonio Bahamonde^a ; ^{abahamonde@uniovi.es}
• 关键词：Peer assessment ; Factorization ; Preference learning ; Grading graders ; MOOCs
• 刊名：Knowledge-Based Systems
• 出版年：2017
• 出版时间：1 February 2017
• 年：2017
• 卷：117
• 期：Complete
• 页码：79-87
• 全文大小：612 K
• 卷排序：117

文摘

Massive Open Online Courses (MOOCs) use different types of assignments in order to evaluate student knowledge. Multiple-choice tests are particularly apt given the possibility for automatic assessment of large numbers of assignments. However, certain skills require open responses that cannot be assessed automatically yet their evaluation by instructors or teaching assistants is unfeasible given the large number of students. A potentially effective solution is peer assessment whereby students grade the answers of other students. However, to avoid bias due to inexperience, such grades must be filtered. We describe a factorization approach to grading, as a scalable method capable of dealing with very high volumes of data. Our method is also capable of representing open-response content using a vector space model of the answers. Since reliable peer assessment requires students to make coherent assessments, students can be motivated by their assessments reflecting not only their own answers but also their efforts as graders. The method described is able to tackle both these aspects simultaneously. Finally, for a real-world university setting in Spain, we compared grades obtained by our method and grades awarded by university instructors, with results indicating a notable improvement from using a content-based approach. There was no evidence that instructor grading would have led to more accurate grading outcomes than the assessment produced by our models.