In this blog post you wil learn:
- What is massive peer assessment
- Its implications for student feedback
Imagine a university professor, sitting by himself in a cozy café. In front of him lay three dozen student projects. He looks at them and sighs. They will take him all weekend to read, comment and grade.
It is hard work, but someone has to do it.
Imagine a different situation now. What if the teacher were not faced with three dozen projects, but three hundred ?
How about three thousand?
Enter Coursera HCI Mooc
This was the problem faced by the staff at Coursera and Stanford during 2013’s Human Computer Interaction Mooc. They had to grade and provide feedback to homework submissions from over 3,000 students.
They solved it by outsourcing the classmates assesment to other classmates. Every student submitted a homework and received specific instructions and a short training on ‘how to evaluate a peer’s homework’. If they succeeded the training, they were trusted with grading five of their peer’s papers.
How reliable was it?
Some papers were also evaluated by staff at Stanford, so as to provide some ‘ground truth’At the end of the assessment the absolute difference between peer and staff grades was 3%. Under- and over-grading sort of balanced out.
Not perfect, but still amazing.for the first time peer assessment had been used a such a massive scale.
Ever since, calibration techniques have been devised to improve graders’ accuracy. Massive Peer Assessment (MPA from now on) is now used in hundreds of Moocs in several platforms.
Call it ‘crowdsourcing of feedback and grading’
Call it ‘assessment by the students, for the students’
I call it a killer application.
To see why this is the case, let’s fist talk about student feedback.
One of the most interesting aspects of the technology is its implication on student feedback. It has the potential to make it more timely. And a lot, a lot more abundant. These are exciting news for several reasons:
- First, feedback is precious. In the now classic 2-Sigma paper , Benjamin B. Bloom observed that one-on-one -tutoring using effective techniques can help any student raise to the top 2% of her class. Yes, any student. The availability of immediate, personalized feedback is at the core of the value added by a tutor, and can indeed make or break a student.
- Respective to motivation, feedback ist crucial. In Flow Theory, Mihaly Csikszentmihalyi established that immediate feedback is a crucial element for a person to concentrate fully on a task , a state of enjoyment also called being in the zone. There is no better state of mind for learning than being in the zone.
- On the flipside, lack of feedback is dangerous. Research has found that ignoring a person’s efforts is almost as bad for motivation as destroying their work.
Massive Peer Assessment and student feedback.
Indeed, I believe most learning today is largely feedbackless task. Classrooms and training needs are too large. Incredibly, not even after major educational milestones like final exams do students get substantial feedback. Whatever the context, feedback for students is extraordinarily scarce.
I believe there are very juicy, low hanging fruits in terms of student achievement and motivation if we get more timely and abundant feedback. Massive peer assessment has now made those fruits accessible.
I believe MAP can allow educators to leverage abundant, quality feedback into the creation of learning experiences that approach Bloom’s ‘One-on-one tutoring’ in terms of achievement and that are engineered in a manner that students can really enjoy and get in the zone.
But that is not all. Oh no, that is not all.
MAP might be precious for the student receiving it, but the same is true for the students generating it.
Indeed, assessing someone else’s work and explaining things to others are first-class learning experiences. Massive Peer assessment democratizes such experiences.
Part 2: Why assessing peers is a world-class learning experience
Read more about Coursera and Stanford’s use of MPA here