I’d like you to close your eyes for a moment and think back to every piece of writing that the children in your class did for you last year.
All of the stories, the setting descriptions, the information texts. Now I have a simple question: out of all of that writing, which piece was the best? It’s not a simple question, of course.
Daniel Hahn, one of the judges of the International Man Booker Prize, believes that what makes a ‘good’ book is “a book that has found the unimprovable form of expression for itself.”
Although perhaps not on the same scale, each time we read a set of stories written by our children, I like to think of teachers as mini Man Booker judges.
My favourite piece from last year – the story to which I would award the mini Man Booker prize – was a narrative based on the Apollo 11 moon landing, written by a child in Year 2. Notice how I said ‘my favourite’ piece of writing.
There appears to be something hopelessly subjective about choosing the best writing in your class. And is this the same as ‘securing age-related expectations’? What makes good writing, well, ‘good’?
Playing favourites
The orthodoxy is for teachers to have a checklist of criteria against which they judge each piece of writing, before plonking the child responsible into the corresponding band or level.
There is a great deal wrong with this approach, which I won’t get into here.
Suffice to say that ‘criterion-based’ assessment is unreliable (people don’t agree) and invalid (it doesn’t identify ‘good’ writers or writing).
The new interim assessment framework suffers from same problems that national curriculum levels did, and most new assessment systems do too.
Whilst wrestling with these problems, I stumbled across an innovative and radical approach to assessing writing that claimed to evade these flaws. Known as ‘comparative judgement’, it relies on a single piece of criteria: which is the best piece of writing?
Aided by a website called No More Marking (nomoremarking.com), the work of an entire class is electronically scanned, and then two pieces of writing are displayed side by side.
The teacher then simply clicks on which piece of writing they deem to be the better piece. No criteria, no tick boxes, just the simple question that we’re really interested in: which is better? Instead of judging against criteria, we are judging against another piece of writing – something humans are naturally good at.
More and more pairs of writing pop up, and teachers complete a few hundred judgements for each class, which takes around an hour or so. At the end of this process, an algorithm is used to rank all of the children’s work from best to worst, assigning a standardised score to each child (we set ours at 85-115 to marry with the KS1 scores following the SATs).
This gamification of assessment made the whole process very addictive, and teachers were especially keen to learn and improve their ‘infit’ score, which tells them how consistently they are judging against themselves.
The process had the added benefit of taking a fraction of the time compared with levelling each child’s work and then moderating these judgements across the year group.
Surprisingly consistent
Is it really possible to agree on which writing is better, though? After judging the Pulitzer Prize, Michael Cunningham wrote that “Utter objectivity is not only impossible when judging literature, it’s not exactly desirable. Fiction involves trace elements of magic.
It works for reasons we can explain, and also for reasons we can’t.”
It may be surprising, then, that the reliability of comparative judgement is so high; teachers consistently agree on the ranking over 90% of the time.
Although we may not be able to articulate why, we have an implicit gauge of what make good writing.
We enjoyed the same high levels of reliability during our experimentation with the scheme last year.
This continued even when we invited other schools to submit work so that we could judge a much larger sample of abilities.
The final results gave us an insight into how our kids stack up against kids in other schools – crucial information to which we were previously blind.
Coordinating across the schools was a bit of a pain but, thankfully, No More Marking has launched a ‘sharing standards’ tool on its website for this coming year which automatically compares your children’s writing against kids from schools across the country.
But what about next steps?
Although we were delighted with the data that comparative judgement (or CJ, as the cool kids now refer to it) was giving us, we did wrestle with the ‘now what’ questions.
For all its flaws, criterion-based referencing does give a teachers a clear roadmap of next steps or targets for children.
We have addressed this by asking our children to take part in the CJ process themselves.
Following the judging, we display two pieces of real writing and ask the children which they think was judged to be better piece, and why.
Such an approach transcends the tick-box approach. Piece A and B may both include expanded noun phrases, but the description in B is more precise and visceral (‘the rusting, blood-soaked dagger’), whereas ‘the incredible, wonderful door’ in A makes little sense.
It’s my hope that in future, No More Marking publishes exemplar pieces from each year group to give teachers a clearer idea of what good writing looks like across different genres for different year groups.
I know that I would find top, middle and bottom exemplar Year 4 stories infinitely more useful than a list of criteria which, to be honest, looks very similar to the criteria used by Years 3 and 5.
Of course, the elephant has not left the room. Statutory assessment remains stubbornly criterion-based. It would be tempting to continue to fall into line, but we have made the choice to do the right thing, not the easy thing.
We want to focus on great writing, and helping our children produce it. In this endeavour, CJ has transformed the way that we think about writing at our school. Could it do the same for you?
Jon Brunskill is head of curriculum and Year 4 teacher at Reach Academy Feltham
@jon_brunskill
pedfed.wordpress.com
