Assessment in Practice Conference at Glasgow School of Art

Last week I attended a conference at Glasgow School of Art on the subject of Assessment in Art, Design and Architecture. I also led a “workshop” session on the subject of grades and their negative impact on intrinsic motivation. It always amazes me how hooked people are on grades despite the mountain of evidence demonstrating how corrosive of creativity and risk taking grades can be. In fact, as the day progressed it became more and more obvious - to me at least - that the majority of complexities and issues that arise in relation to assessment are entirely the product of a pervasive obsession with grades that distracts from the far more important issues and potential of feedback (let’s not forget that grades are a form of feedback, just an incredibly impoverished one).

Susan Orr gave the first keynote paper of the day with reference to what she terms “connoisseurship in practice” and “Zing” (surely for want of a better term?). She argues that the assessment process in art and design involves "undefinable elements" that cannot be captured in Learning Outcomes or assessment criteria. She uses the word “zing” as a means to describe this special quality that is sought by art and design teachers during assessment of student work (though I think the word “excellence” might be a little less cringeworthy). I've written about Susan Orr's work previously on this blog (here) and my conclusion then was to question the "why", since it seems to me that this research, whilst sound, ends up becoming unnecessarily focused upon justifying the practice of grading and how grades are negotiated and agreed amongst different subject experts rather than looking more closely at the content, language, psychology and impact of feedback on student learning and achievement.

Following this keynote we split into 2 different sessions and I did 'my little turn', although in retrospect I wish I had taken my own advice from a couple of posts ago about not being too certain of myself (I'll need to bear this in mind in future). The next session I attended was a fascinating but information-rich presentation by Susan Roan and Elise Allan of the Visual Communication Department at Glasgow School of Art. Roan and Allan have been collaborating on a new initiative to encourage and instill metacognitive skills (“thinking about thinking”) in studio practice with design students. They have evidently brought a great many different creative ideas and learning theories together in innovative ways for this initiative and the results appear to have been very positive. Nonetheless, I couldn't help wondering why such innovations are not the norm within the art and design education as opposed to the exception, especially as you would think that art and design education would be a veritable hothouse of creative innovation - not so!

Alison Shreeve presented a keynote after lunch about a body of research she has undertaken into the application of audio recording as a feedback tool. She has gathered a variety of interesting responses through student interviews but I was a little unconvinced by these 'reports' as evidence of the genuine impact of this form of feedback. For instance, one student said that they would like “both” audio AND written feedback (and why wouldn’t they?). At the beginning and end of her presentation Shreeve asked the audience about any particular experiences they remembered of being assessed. I was reminded of the fact that we are all assessed by our students on an annual basis through the NSS survey which seeks to gather student perceptions of satisfaction with the education they have received. Fortunately growing numbers of teachers are beginning to speak out that we need to be careful that we do not pander to do this idea of ‘student satisfaction’ since there are many instances where genuine learning requires difficult challenges and may involve struggle and difficulties which may not - in the first analysis - be satisfying to students but which may equip them with greater confidence and skill than their self-perception might indicate. In other words "self reports" are an important tool but only if they are backed up by other more concrete evidence of development/achievement. Happy students don't necessarily correlate with the high achievement… though they are great for NSS surveys.

The final session was presented by Ben Craven of the Product Design Engineering course at Glasgow School of Art. Ben presented an excellent critique of Criterion-Based Assessment. Criterion-Based Assessment is promoted by many in the field of education theory and has been universally adopted as gospel by universities. Criterion-based assessment (AKA standards-based assessment) is a form of assessment whereby criteria are laid down in advance to define the levels of achievement that students must attain in order to achieve any given grade level. The intention behind this practice is to provide an objective alternative to “norms based assessment” where students are ranked in relation to their peers. Craven argues persuasively that such ‘objective’ criteria can never be laid down unambiguously because language is too inherently vague to ensure of faultless intended learning outcomes or assessment criteria. He even demonstrated how the hallowed Biggs and Tang (who are strong advocates of Criterion-Based Assessment in their PGCert HE bible) had used the word “average” three times in their example Learning Outcomes. As Craven pointed out “average” in this context is indistinguishable from the word “norm”. He also discussed a type of psychological bias known as the “Centering Bias” where people tend to distribute their evaluations (of artworks for example) in a bell curve across given criteria as in the following image:

Image: courtesy of Ben Craven, 2012

However, if you use the exact same criteria but add another at the top (ie: “Outstanding” in the below example) the bell curve remains in exactly the same place relative to the highest and lowest possible evaluations:

Image: courtesy of Ben Craven, 2012

Craven’s conclusion was that we are in fact assessing students using a form of Norms-Based assessment dressed up as Criterion-Based Assessment. In order to further emphasise his thesis he then split us into groups and gave us a challenge to devise some unambiguous criteria for a stage 1 student “A” and a fail. Needless to say, nobody succeeded. I also think it’s very significant that Craven is considering dropping grading in future with the students he teaches.