30% Less than Perfect

Inherited protractor (partially polished)

Anyone who follows this blog will know that I am opposed to grading the work of art students for a variety of reasons of which this post is a continuation. On the other hand, attempting to assign numerical equivalents to more concrete subjects like Engineering or Healthcare for instance seems not unreasonable since the achievement of specific outcomes can both be more clearly set and measured. But without a clear example of what constitutes a top performance (ie: 100%) in a subject, there is little of substance for students to aim at or for staff to use as a benchmark by which other evaluations may be made. As a result it becomes almost impossible to agree that any particular work represents all that is required at any given level and therefore the highest marks simply languish unused waiting for that ‘perfect’ student who, of course, is simply an impossible amalgam of all the very best performances we have seen in the past.

Is a First Class Honours degree a sign of perfect achievement? Not in the least. Even a 100% achievement would only be perfect at Honours level, otherwise why have Post Graduate, Masters and Doctoral study? A First Class Honours degree is simply a reflection of attainment somewhere in the top 30% of the Honours degree classification.

So why on earth are there not more First Class Honours degrees being awarded to graduates each year?

In the UK around 10% of graduating students achieve “a first” while around 20% attain the "third" (third Class) category - the lowest category of Honours degree (graded at 40% or above). The other 70% of students achieve either a "two one" (Second Class Upper), a "two two" (Second Class Lower) or an outright fail. Very few students actually fail, and I’ve never heard of an Honours student getting less than 20%, though I’m sure it must happen.

That means that on average only 10% of students achieve anything higher than 69% at their degree assessment. Can it be true that the other 9 out of 10 students are really so ‘unexceptional’ that more of them can’t even reach the low 70s? Or is the education provided so poor that only the very strongest students manage to keep their heads above water? Neither. The reason there are so few firsts, I suggest, is because the general perception amongst staff is that the highest category should be reserved for the very best performances, or, as a colleague once said to me when I began teaching: “We don’t hand out firsts like sweeties here Jim.” Indeed.

There is a contradiction in this though. If honours level is not the very highest level of study and if a first class honours is not an indication of perfect performance (ie: it should logically contain work that is fully 30% less than perfect at that level) then it is quite clear that the expectations of the highest category of the Honours classification are a distortion.

Surely 100% performance on a degree course should be achievable through the production of the highest standard of work expected at that level, not some unachievable notional pinnacle of excellence. In most cases a top ranking student means a graduate who is fully equipped to begin engaging in a professional context - though it is recognised that even there they will still have a lot to learn. Instead it would seem that the very highest grades attainable (90%+) are being reserved for those one or two geniuses that might, by chance, grace our hallowed halls. And of course those individuals never do arrive and we probably wouldn't even recognise them if they did.

An alternative approach is to dispense with percentages as much as possible and to simply assign each component of work to a band and to add everything up at the end. Whilst this is an improvement it ultimately runs into the same difficulties of vagueness since, without a clear understanding of what represents the highest expected performance, it is impossible to accurately situate that all important cutoff between a First Class and a Second Class degree? Furthermore, this does nothing to remedy or highlight the fact that a First Class Honours degree can and should contain work that is in each and every respect 30% less than the top level expected.

Intuition and Prejudice

“Assessments are based, not on whether the decisions made are any good, but on whether they were made in accordance with what is deemed to be an appropriate process. We assume, not only that good procedure will give rise to good outcome, but also that the ability to articulate the procedure is the key to good outcomes.” -John Kay

It could be argued that one of the principle differences between a novice and an expert is the degree to which experts are able to explain the reasons for any judgement made and to supply these retrospectively. But this begs a question: if experts make intuitive judgements – which they often do - then how can we be sure that these are not simply preferences dressed up in the trappings of deft argumentation?

A standard reply would go something like this: if the justification for an intuitive judgement stands up to rigourous debate or close scrutiny then we have no alternative than to accept the justification as valid. If such valid judgements are performed repeatedly then we need no further proof of expertise. Novices, on the other hand, are less competent in teasing out or debating their reasoning process and their judgements tend to be more inconsistent. So, even if a novice makes the same choice as an expert, we cannot say that they have exercised judgement, since judgement - in order to be judgement - needs to be justifiable.

But is this not simply an intellectual sleight of hand that surreptitiously places the quality of the justification above the quality of the decision? And whilst we may accept that instantaneous decisions are justifiable post-hoc, this still doesn’t answer the question of how such decisions can be made without calling upon resources that require far more time-consuming and considered deliberation. In other words: what is actually going on when judgements are made in an intuitive instant?

Dylan Wiliam writes of experts having what he calls “scripts” that allow them to make rapid decisions and improvise within their domain of expertise. He cites the work of Chase and Simon and their studies comparing expert and novice chess players. Chase and Simon found that expert chess players were significantly more accurate at quickly memorizing the positions of chess pieces placed in positions of actual games whereas they were significantly worse at memorizing the positions of randomly arranged chess pieces:

“Chase and Simon suggest that the much better performance of experts with ‘real’ chessboards stems from an ability to ‘read’ a chessboard in terms of a series of standard configurations of pieces that they have learnt through their experience as chess players.” 

Such expertise is the product of prolonged experience within a domain and cannot be acquired through shortcuts, formulae or crash-courses. Nonetheless we are all experts in the art of everyday life and the vast majority of the decisions we make are instantaneous and automatic. If pressed, we would have no difficulty in providing reasons for our having stopped at a red light. But can it be said that the background familiarity amassed by experts - us included - constitutes a form of criteria, or are criteria simply a means to externalize and make explicit what is essentially a tacit process that works in an entirely different way and therefore allows many decisions to be made intuitively rather than demanding deeper reflection?

"It is fairly clear that teachers acquire notions of ‘standards’ much more effectively when presented with actual samples of students’ work that exemplify the standards being promulgated than when given criterion-based descriptions of the standards. … I would like to suggest that the explanation of this phenomenon is that the grading of pieces of work with respect to a set of internal standards involves a far greater use of unconscious processing than has previously been acknowledged." -Dylan Wiliam

Psychologists, like Daniel Kahneman, distinguish between two kinds of decision making systems: System 1 and System 2. System 1 is roughly what we would think of as intuition. System 2 could be described as deliberative thinking:

“System 2 is the one who believes that it's making the decisions. But in reality, most of the time, System 1 is acting on its own, without your being aware of it. It's System 1 that decides whether you like a person, which thoughts or associations come to mind, and what you feel about something. All of this happens automatically. You can't help it, and yet you often base your decisions on it. […] System 2, on the other hand, is lazy and only becomes active when necessary. Slow, deliberate thinking is hard work. It consumes chemical resources in the brain, and people usually don't like that.” [my emphasis]

From the discussion so far it would seem that experts do not have sufficient time to formulate let alone deliberate over criteria in their intuitive judgements. This is not to say that they never pause to contemplate things more closely or even to reverse their initial judgements. They can and sometimes do. But what it does suggest is that criteria are not the primary tools of experts: experience is. By experience I mean the repertoire of exemplars that have been acquired through studying something closely – through comparing different instances and considering the relative merits of various examples, both good and bad. One nectarine alone would never be sufficient to furnish anyone with a clear idea of what constitutes a good nectarine. We need to experience a range of qualities of an experience in order to formulate an awareness of what might enhance it or detract from it, of whether the juice running down our chin is a sign of succulence or overripeness.

The point I’m making is that experts do not use criteria or deliberative thinking to form their initial judgements, instead they draw on experience and the more extensive this experience, the more well informed their judgements. However, an intuitive judgement, no matter how well informed, is not - strictly speaking - a judgement at all, but a pre-judgement; a pre-judice: an evaluation made without due consideration of the specific issues or circumstances at hand. It is a shortcut based on an unconscious repository of prior examples and is therefore limited to the known and familiar.

If the prevailing conditions in any situation that urgently demands judgement bear a close resemblance to previously encountered circumstances then it is obviously appropriate not to waste valuable time and energy logically weighing up the options.

“Intuitive intelligence is the ability to know directly, to perceive and appreciate whole or hidden patterns beyond (or faster than) logic.” -Tom Atlee (The Co-Intelligence Institute) 

But where there is no urgency, and especially where decisions impact on other people – students for instance – it seems prudent to be wary of the tendency to cut cognitive corners. As the research of Daniel Kahneman, Philip Tetlock and others increasingly shows (although it is not impossible to find exceptions), even the judgements of experts can be far from reliable, especially when making predictions about intrinsically complex and unpredictable domains. In the arts (which are well known for their unpredictability), as in other creative fields, intuition is frequently revered as a unique and often indispensable route to problem solving and discovery. But in arts education, where the judgements of teachers are directed not just towards artworks but towards individuals, too great a reliance upon intuitive judgement or gut-feeling risks undermining the putative integrity of academic assessment. And there’s the rub: what works for the production of art may well be at odds with its evaluation.

You might also be interested in the following article (Thanks to Tor for the link)

Philosopher's Stone or Crystal Ball?

“Should Universities Give Preference to Poor Applicants?”

This question was asked by the American philosopher Michael Sandel a few days ago in a debate broadcast on BBC Radio 4: http://www.bbc.co.uk/podcasts/series/r4sandel

The discussion is engaging and is excellently fielded and summarised by Sandel, though some listeners may find the overall debate somewhat inconclusive. Even admitting that the issues are immensely fraught, and may even be irreconcilable, I suspect that Sandel holds a position that he refrained from putting forward. So whilst there is much to consider in what is presented, there is also an underlying sense that we have only touched the surface of something much deeper.

Though it wasn’t mentioned directly during the debate, it seems to me that the whole issue hinges on the difference between attainment on the one hand and potential on the other. These two things are by no means equivalent, but since potential is nigh-on impossible to accurately predict in advance we have to use the next best thing, which turns out to be track record: usually in the form of attainment (most often measured in grades). In art schools we take relatively little account of grades so long as the minimum are covered (though the minimum seems to have risen in recent times). Art students are accepted or rejected, in the main, according to their ‘readiness’ suggested by their application portfolio. But once again, this is still to some degree a rough substitute for that philosopher’s stone that we call potential.

I can’t imagine that anyone seriously doubts that uncertain quantities of applicants simply slip through the net because of this problem? The question, of course, is how might universities deal with it better? The University of Texas have an interesting solution. They offer places to any students in the top 10% from any state school regardless of grades, which means that no matter how poor your school is and how low your grades are on a state level, you’d still be offered a place if you are in the top 10% of students.

The greatest gift an education can provide is an enduring desire to learn

What would it be like if, instead of evaluating achievement at art schools, we decided to assess fulfilment? It’d be absurd of course - fulfilment is a perception of an internal condition and, as such, it can only be evaluated internally. But the question seems to point to the heart of the problem with grading (in art schools at least) since it exposes exactly the mismatch between two conceptions of the role that art plays: one of which is predicated upon notions of performance (or rather performativity) and the other upon ‘being’.

Just imagine though, if it were possible to gather a clear picture of how fulfilled students were with their studies on completing their education, what enormously valuable data that would be. And if it turns out that students have less interest in their chosen field than when they started, perhaps we should ask ourselves if we’ve failed to do what education should strive to do above all else: to foster their desire to go on learning.

References:

“Measuring Happiness – some practical advice for David Cameron’s latest idea.”

“Step outside the Rogerian circle” (Gail Kinman asks whether a condemnation of 'therapeutic education' examines all the evidence

Sticky Carrot

When I was studying towards a teaching qualification last year I was permitted to see the assessment and feedback guidelines given to my ‘mentor’. Much of the advice seemed perfectly reasonable, well founded and relevant, however, there was one thing nestled amongst the bullet points that caught my attention:

The feedback provided on course participant’s work should be:

· encouraging and motivating: e.g. give praise when and where appropriate. This will help motivate the learner.

It seems sound enough doesn’t it? What could be wrong with a healthy dose of praise? I’m going to argue here though that the answer to this question is far from obvious. Indeed, the use of praise is so commonplace that we’re practically blind to its subtleties to such a degree that it seems almost churlish to even cast it into doubt. So, whilst we might be critical, and even a little suspicious, of effusive or unwarranted praise, in most cases we tend to assume that praise is a good and even a wholesome thing.

“However, in study after study, measures of teacher praise failed to correlate with other classroom process variables, or with outcome variables, in ways that would be expected if such praise were in fact functioning as reinforcement.” -Jere Brophy

Ok, it doesn’t mean that praise is bad exactly, but does mean that in a wide variety of studies praise has not been found to reinforce learning. In addition:

“It is important, however, to distinguish between praise that directs attention away from the task to the self (because such praise has low information value to achievement and learning) and praise directed to the effort, self-regulation, engagement, or processes relating to the task and its performance (e.g., “You’re really great because you have diligently completed this task by applying this concept”).” -Hattie and Timperley

Here we find the first notable weakness of praise: its “low information value.” Possibly due to this very this lack of clarity, praise can also have differential effects depending on a whole range of circumstances:

“Research demonstrates that various forms of praise can have different kinds of effects on different kinds of students. Students from different socioeconomic classes, ability levels, and genders may not respond in the same way to praise. The use of praise is further complicated by the fact that it may have differential effects depending on the type of achievement being measured. For example, praise may be useful in motivating students to learn by rote, but it may discourage problem solving.” -Hitz and Driscoll

Although the majority of studies of praise have been carried out with children, there is certainly no reason to assume that the problems become any more straightforward with adults:

“older students perceived praise after success… as an indication that the teacher perceived their ability to be low. When given… neutral feedback after success, they perceived that the teacher had estimated their ability to be high and their effort low.” -Hattie and Timperley

So what’s the best response then if a student has been successful but you consider their ability to be high? You could certainly be forgiven for being confused here. But perhaps the issue brings us directly back to the poor information value of praise. If praise provides little information then it would seem wise to make every effort to supply this lacking information and to explain exactly what it is that merits praise in a successful student’s work. The valuable thing about such feedback is that it also ensures that both teacher and student are fully aware of what is working and most importantly that the praise isn’t simply the result of teacher insecurity or social convention:

“Much teacher praise is determined more by teachers' perceptions of student needs than by the quality of student conduct or performance.” -Jere Brophy

And here we encounter the obstacle of extrinsic motivation and dependence upon approval and reward. Increasingly educators agree that a fundamental goal of education should be to instill in students a desire to go on learning beyond formal education. But if students are hooked upon teacher approval, praise and external evaluations of their performance (grades) then how sustainable is their learning? And is praise really the most effective way to go about transforming it.

The Failure of Success

I was interviewed yesterday by a colleague about the Co-Creativity of Hand and Mind research group that has recently formed at Gray’s School of Art. One of the questions was “Tell me about a failed work - Why was it a failure?”. If you follow this blog you’ll have noticed that I've written about failure quite a lot - I guess it’s because failure seems to be such a commonly encountered concept, and issue, in art and art education (and not just a concept: for some students it's a genuine threat). But when I began to think of my own failures I realised that these are relatively few and far between. I'm not saying I've never made mistakes, broken things or made things in the studio that didn't work or that I'm not proud of. On the contrary, I've made more than my fair share of such 'failures', but I realise that I don't actually consider these failures - they're just sketches, rejects, breakages or experiments. The only true failures I feel I've produced are the works that have been exhibited in public but that have broken down in some way or that didn’t function as intended. And even the works that haven’t failed technically, but which I’m not proud of, are not so much failures as simply not my best works.

But it's not really as simple as a few inferior or malfunctioning artworks, but rather the damage to my reputation that emerges as the real failure. Artworks can be repaired or replaced by better ones but reputation damage is much harder to salvage. You might well ask why I bother then to complicate the issue by making the kind of art that can break down or “fail” in this way - why not just stick with inanimate work? When I think back over my practice as an artist, it’s evident that I've tended to take fewer such risks in recent years. This might be thought of as an admission that I've become somewhat more risk averse in my middle age. Perhaps, but then again there is an alternative interpretation that one can take, and inevitably it’s the one that I will claim. In my youth the use of materials and processes that could recognisably fail - in the obvious sense of the word - meant that I had a clear yardstick by which my achievements could be gauged: if the work worked then it worked and whilst it might be conceptually or aesthetically flawed, at least I could feel that it did what it was supposed to do. The parameters of success and failure were therefore fairly transparent. And in a context where concepts and aesthetics are about as fixed and predictable as the British weather, a certain amount of control seemed to count for a great deal.

But 20 years down the line, I'm not so much in need of the reassurance of fixed parameters, so the potential for genuine failure (especially in terms of reputation harm) has become far less of a threat. I'm quite content with the ambiguity and contestability of conceptual and aesthetic success and failure; in fact it’s a liberation. I don’t see myself as risk averse - I just don't measure my artistic achievements by other people's yardsticks to the same degree any more. Now, I realise that this is no doubt partly due to the relative security of my position as a teacher, but on the other hand, I'd argue that I’m simply no longer concerned about failure because I'm no longer bent on success. As I've said here before, success is a false aspiration in my view - we'd be far better off aiming at fulfillment in life.

This brings me back to education - that set of institutions tasked with the responsibility of providing support, resources and guidance for learning but that also take it upon themselves to enshrine peoples’ future reputations in those impoverished forms of feedback known as grades and which, despite legions of highly educated employees, still chime in with the mainstream misconception that success is the pinnacle of achievement.

Failure is an event, not a person.

Interesting isn’t it, how we confuse what people do for what they are? I mean by this the way we describe someone as being, for example, original: an original thinker, an original artist or an original writer rather than simply describing what they actually do as being original. Clearly this is completely inaccurate. No one is more original than anyone else.

This kind of subtle misattribution might be ok when we’re dealing with job titles: secretary, waiter, artist etc. but the real problems start when we begin to internalize negatively weighted attributions: “I’m unoriginal”, “I’m a failure”, “I’m no good at that.” Such self-perceptions are only ever something we resign ourselves to: they’re a declaration that we’ve given up trying and that we’ve come to the conclusion that further effort is futile. Indeed, further effort would simply reinforce the negative perception. It’s not surprising therefore that people avoid putting themselves in such circumstances and consequently avoid the kinds of risks that might lead, not just to disappointment, but to growth.

The problem, of course, is thinking that a failed thing, of our own making, is representative of who we are and – crucially - who we might become: that creations define not just internal states but potentialities. We can see this same deception in education, in which students are constantly under the critical eye of evaluation and assessment. What better way to encourage self-criticism and crippling self-consciousness?

In education there’s a widely held assumption that we assess learning. In fact we don’t assess learning at all. What we asses are the products of study, which we take as proof of learning. Assessment, and grades in particular, perpetuate the notion that what defines people is that which is created by them but which is external to them. Whilst this may necessarily be true within the view of others, it also, arguably, has the side effect of turning individuals into observers of their own performance when what education should be trying to foster is unselfconscious critical engagement with the objects of study at the very deepest level.

The Road to Mastery is Littered with the Wreckage of Past Failures

What do you say to a student who is taking every opportunity to learn and extend their competence and understanding but who, as a consequence, is accumulating a catalogue of hard-won errors? Most especially, how do you encourage them and convincingly explain the less than excellent grade you've given the work they've produced? If it’s true that we learn from our mistakes, what sense does it make to discourage failure and to commend success? Whilst the commendation of success necessarily acknowledges the overcoming of error, it also tends to encourage conservatism by focussing attention on what can be achieved with certainty rather than what might not; between what is a genuine challenge and what is merely difficult.

“The greater danger for most of us is not that our aim is too high and we miss it,
but that it is too low and we reach it.” -Michelangelo

The Prejudice of Grading Painting Students

Image Credit: Calum MacLeod

Thanks to James A for sharing a link to an article entitled: “The Justice of Grading Painting Students”.

With barely concealed smugness, Professor Laurie Fendrich discusses how she works with and grades "beginning painting students". Any teacher who uses terms like “natural talent” without the slightest hesitation has made a fundamental mistake, in my view, by assuming that some students have it and some just don’t. You don’t have to know anything about the Pygmalion Effect to realise just how self-fulfilling such prejudices can be.

Whilst frequently using the possessive form (“my students”, “my course” etc), Fendrich begins by describing how she found it difficult to grade her students this semester. However, as the article progresses, it becomes quite clear that she doesn’t find it particularly difficult at all, and certainly not because she perceives any problems with the nature of grades.

“Yet the moment of a final grade in a course is not a small matter. It generates anxiety for students as well as for professors who take the act of judgment seriously. Rough and imperfect though they are, grades function as a form of justice, the meting out of which is a solemn occasion.”

What? ...“grades function as a form of justice.” Does she think she's some kind of high priestess who meets out judgement to praise and punish her minions?

“After all the friendliness and soft competition in a studio course, students are often shocked when it’s time for judgment. […] The answer is that the time has come. A final grade must be given. A judgment must be made. This, dear students, is life.”

That last patronising sentence had me instantly reaching for the comments box:

That’s the voice of unexamined orthodoxy speaking. Yes this is “life” as you call it (or “the real world” as others think of it too). However, the point that needs to be scrutinised very carefully is whether it could be different, or even better? The creation of art is very much an expression of this belief (faith even) in the power of creativity to transform experience. If we are simply to accept the status quo, as your article seems to suggest your students should, then what point is there to make art at all?

Rewards and punishments (of which grades are a common form) of all kinds are corrosive to creativity. This has been shown in numerous studies conducted across cultures and generations (lookup intrinsic motivation or check out the research and writings of Alfie Kohn).

Your article says more about your own acceptance of grades as unavoidable (“a final grade must be given”) than it does about your students naïveté in the face of life. They, on the contrary, seem to understand, if only intuitively, that there’s some hypocrisy at work in the way they’re being treated. You’d do well to examine your “dear” students’ shock a little more carefully. Perhaps then you’d realise that it is not “life” they are dealing with so much as 'an institution' and there is nothing inevitable about that, only what we unquestioningly accept as being so.

Why Bother? (Effort, Achievement and Risk)

Lucy Gunning: “Climbing around my room”. Video, 1993

"...effort is nothing more, and also nothing less, than tension between means and ends in action, and that the sense of effort is the awareness of this conflict." -John Dewey

I was involved in a really interesting series of studio discussions with some of the Photographic and Electronic Media students last week. They were sharing experiences from over the Summer break as a method of reintroducing themselves to thinking about how these experiences and observations might be placed within the context of reflecting upon, discussing and working towards art making. One thing in particular struck me amongst the many fascinating things we discussed - this was the extent to which almost everyone, myself included, had invested time, energy, money and especially expectations in the pursuit of specific experiential encounters: holidays, weddings, childbirth etc. Somehow almost all of us (though there were some exceptions) had bought into to the conventional notion that only these kinds of experiences are worthy of aspiring towards, making an effort for or sharing with other people, whilst everything else - jobs, relationships and a multitude of other 'commonplace' experiences - are merely a necessary part of - or, at best, a pleasant distraction from - our striving to acquire money or opportunities to achieve these other more 'unique' experiences. By way of contrast (though I didn’t mention it at the time) I was reminded of Xavier de Maistre’s extraordinary, but at the same time extraordinarily modest, expeditions:

“…in the spring of 1790, a twenty-seven-year-old Frenchman, Xavier de Maistre, undertook a journey around his bedroom, later entitling the account of what he had seen Journey around My Bedroom. Gratified by his experiences, in 1798, De Maistre undertook a second journey. This time he traveled by night and ventured out as far as the window ledge, later entitling his account Nocturnal Expedition around My Bedroom. […] …it was not Xavier’s intentions to cast aspersions on the heroic deeds of the great travelers of the past: Magellan, Drake, Anson and Cook. …it was just that (de Maestre) had discovered a way of traveling that might be infinitely more practical for those neither as brave or as wealthy as they.” -Alain de Botton

Much as I love this idea of a unique journey amongst the commonplace and familiar and much as I admire artworks that transform our understanding of such experiences, I’m nonetheless tempted to think of this particular case as an inventive but somewhat extreme example of the Principle of Least Effort. Indeed, De Botton goes on to say of De Maistre that: “He particularly recommended room-travel to the poor and to those afraid of storms, robberies, and high cliffs.” This seems a form of travel most suited to the risk averse, the unadventurous and the lazy:

“The most indolent beings won’t have any reason to hesitate before setting off to find pleasures that will cost them neither money nor effort.” -Xavier de Maistre

But is this really laziness? The great beauty of this idea (though it was patently intended as a parody) is it’s wonderfully unconventional simplicity, but would it be true to say that no effort is required? Effort has nothing to do with the grandeur of the ends but rather, as Dewey suggests, the conflict or tension between ends and means. As the games of every child clearly attest, it’s quite possible to create a genuine and engaging challenge even with the most meager of materials or environment. The effort in this instance then, becomes entirely a labour of the imagination: to embark on the journey anew, as if never before travelled and to voyage over each encounter, savoring every intricacy with the curiosity of the most inquisitive and observant tourist.

So what correlations may we draw between effort expended and our sense of achievement?

In psychology there’s a process known as Effort Discounting, which describes how our sense of reward diminishes as effort increases. We naturally tend to prefer low-effort routes to rewards. An analogue of this effect has also been observed in a neurological study published in 2009 by Botvinick, Huffstetler and McGuire. From a survival perspective, effort discounting makes evolutionary sense since there’s very little point in pursuing food, for example, if the resources expended in order to acquire it outweigh the benefit gained. But human tastes are far more complex than this alone would suggest. We often expend inordinate quantities of energy in seeking things to satisfy all manner of strange and exotic desires, despite the fact that these things provide very little in the way of energy return. It’s the promise of pleasure which drives many of these desires - desires that are essentially aesthetic in nature: food, music, art, literature, dance and even holidays. However, even with these unique experiences, we almost always have an underlying sense of how much effort is deserving of the end result and we quickly tire of what we anticipate to be unrewarding pursuits. Equally, when we gain something with unexpected ease the pleasure is often multiplied.

These observations strongly suggest that, as beings, we’re very highly attuned to the economics of effort and reward - to our own individual cost-benefit analysis. But there’s one further set of experiences for which we’re frequently prepared to exert ourselves physically or cognitively and these are processes that are in themselves pleasurable - that provide stimulating feedback or the release of mood elevating endorphins. Many areas of human endeavor exploit this tendency, from sport and exercise to art and craft. There’s little doubt though, that many of these pleasures are acquired tastes and to expect them to arise spontaneously or to be appreciated as immediately gratifying is frequently unrealistic or frustrating. For example, the pleasures of exercise are relatively easily acquired by comparison with the pleasures of creation which require skills and sensitivities that take time and practice to develop:

“Children who have not learned that drawing skill is based on practiced observation will be very frustrated when they reach the next developmental stage. They will wish they could draw more realistically. As they get older, they mistakenly believe that they lack talent while others are gifted in drawing. They give up because they see others who can do better. Art educators refer to this as the "crisis of confidence." -Marvin Bartel

In all this talk of effort and reward there is one further consideration that demands our attention: risk. I’ve written about risk previously, but in that instance it seems I neglected a vital aspect which has only become clear to me through a recent exchange in which I was asked if a grade for effort might be a legitimate method of encouraging students to take risks “particularly in the creative arts, where the "willingness" to fail is such a hard thing to encourage?”

The important thing to bear in mind in relation to effort and risk is that the teacher and student both need to understand, and agree upon, exactly what is being assessed (the "ends" for which effort is being expended and risks taken) and this ideally shouldn't be set by the teacher but rather negotiated between tutor and student (though, of course, this isn't always possible, especially when there's a fixed curriculum to follow).

Much as it might appear to make intuitive sense to grade students on their effort, their “willingness” to expend effort or to take risks should never be c0-opted by the temptation or threat of a grade but rather by the aim of improving what they do to the greatest possible degree. Success should therefore not depend upon an artificial extrinsic reward, but rather on achieving what they set out to achieve or, ideally, even more than they set out to achieve. This is the value of self-efficacy and intrinsic motivation, as opposed to fixation upon rewards, or conversely, being continually reminded of others' “evaluation” of how much effort needs to be put in. Ultimately it comes down to establishing what exactly the student is aiming for, so that the teacher can support them to achieve this - or, if they're aiming for something unrealistically difficult, to encourage them set their sights on something a little more achievable and to help them to see how this is something genuinely worth working towards instead.

Willingness to fail can only come about when students (or anyone for that matter) feels secure enough to take risks. So it's the teacher's - indeed Education's - duty to ensure that every possible measure is taken to alleviate pressure upon students and certainly not to add a little extra in the form of an intimidating or distracting grade for effort.

References:

BARTEL, M., Teaching Observation Drawing to Young Children. Goshen College Publication.[Online]. Available from:http://www2.goshen.edu/~marvinpb/PreSchool/aboutschematic.html

BOTVINICK, M. M. & HUFFSTETLER, S., McGUIRE, J. (2009). Effort discounting in human nucleus accumbens. Cognitive, Affective and Behavioral Neuroscience, 9(1), 16-27. [Online]. Available from:http://www.princeton.edu/~matthewb/Publications/BotvinickETAL_CABN09.pdf

DE BOTTON, A., (2002) The Art of Travel. Penguin. p.243-245

DEWEY, J., (1897). The Psychology of Effort, Philosophical Review 6: 43-56. [Online]. Available from: http://www.brocku.ca/MeadProject/Dewey/Dewey_1897.html

Revolting Students and Unqualified Teachers

I notice that on the 27^th and 28^th of this month BBC2 will be airing a two-part documentary, presented by Dylan Wiliam (deputy director of the UK’s Institute of Education), about implementing formative assessment with a class in a Hertfordshire comprehensive. (Now available on iPlayer here). I’m very intrigued, especially by part two in which:

“There is a classroom revolt when the teachers remove grades from work. The idea is to make the students actually read the comments on their work in order to help them improve, but they are left confused and angry after becoming so used to the traditional grading system.”

It might be thought that this issue is unique to school kids, but not so. I have two colleagues studying, like me, for a post graduate HE teaching qualification. One is, what might be called in the nicest possible terms, “a bit of a swot”, and the other has been so inundated during the last year that very little time has been available for going the extra mile in studying. When they were both given marks for their first essay submission earlier this year it was immediately clear that the gap between them had suddenly become a gulf in terms of motivation. The change was most apparent in the over-worked colleague who immediately lost all interest in engaging in the course other than to aim for a bare pass. And this is from a dedicated, innovative and experienced teacher!

A few months ago I read an academic paper also by Dylan William entitled “The Half Second Delay: what follows?” (2006). In the paper William musters a variety of research to demonstrate that experts, in all fields - but most specifically in this case: teachers - employ a sophisticated range of unconscious scripts or “filters” which can only be acquired through experience rather than through instruction and teaching qualifications:

“The availability of these data requires a thorough grounding in experience, which it appears is extraordinarily difficult to acquire without prolonged immersion in the relevant settings. In the context of the research findings presented here the idea of intuition remains mysterious, but can be viewed as an exquisite and largely unconscious sensitivity to very small details.”

So, not only do grades demotivate intrinsically motivated students but they widen the achievement gap and tell us absolutely nothing about how “good” a teacher is.

Blowing the Whistle on Theory and Practice

It’s possible that I’m wrong, but I strongly suspect that we have a injustice (or perhaps it would be less inflammatory to describe it as an inconsistency) at work in the way that Art students are being assessed at universities in the UK. I realise that this is a significant claim with potentially serious ramifications so I offer this as an observation and discussion to be further examined, explained and hopefully resolved or proven to be unfounded.

All Fine Art degree courses employ an assessment process which evaluates each student’s final degree exhibition as well as their portfolio of coursework produced during their final year of study. In most cases the assessed material consists of a public exhibition as well as a portfolio submission of supporting work (sketchbooks, notebooks, drawings, supplementary images, maquettes etc). All such courses also assess an academic component in the form of an essay, dissertation, thesis, critical evaluation, or other form of written submission and this forms a fixed portion of the overall mark (usually between 12% and 25%).

Due to the specific nature of fine art courses, the differing staffing levels involved and the demands of assessing text in comparison with visual material, it has become established practice to assess this written component differently to the studio work. Inevitably these assessments have to utilise certain assessment methods which, in contrast to studio assessment, take comparatively little account of the preparatory work produced in order to arrive at the final submission. In other words, the assessment of this component doesn’t do what the practical “studio” component does: it doesn’t assess the process and the evidence of this process to the same degree. This privileging of ends over means; of outcome over endeavour is patently at odds with the approach to assessment involving studio work and sometimes leaves students with significant amounts of material unconsidered, unacknowledged and un-assessed. If we marked studio work in the same manner we’d simply assess the exhibition and nothing besides, and the marks that students received at the end of their course would often be significantly different: sometimes better, sometimes worse.

So what’s going on in this irregular approach to assessment and why is this important aspect of the student’s work not being sufficiently recognised? I would argue that this derives from longstanding differences of opinion - prejudices even - about what constitutes assessable material and in particular a privileging of academic practices over practical. In academic circles generally, essays are rarely, if ever, assessed in terms of the process that leads to them. Essays are seen as the culmination of the student’s endeavour and it is often believed that this is sufficient to provide a complete picture of the student’s achievement. In Fine Art, as I have described, we have a more process oriented approach to the assessment of studio work and it would be claimed that this is due to the practical nature of this discipline. But is this distinction between theory and practice really a fair and justified one?

Whether in practice or theory, the relationships between engagement, inspiration and outcome are remarkably similar. We all know that someone can work for years, pouring energy, research, thought and skill into their work only to arrive at a mediocre result, whilst someone else might produce something astonishing and truly insightful with only the barest application. Creativity, originality, innovation, insight and inspiration cannot be scripted, much as we might wish them to be. The landscape of discovery is notoriously treacherous with many complex and unfamiliar obstacles and few, if any, guarantees of success.

In academia, the formal organisation, structure and presentation of essays is carefully supported (usually) and administered in order to ensure that as much of the process is captured and incorporated into the final piece as possible. Bibliographies, references, footnotes etc are intended to give a comprehensive indication of the material gathered and assimilated in order to produce an essay, but do we really believe that this represents a foolproof method of demonstrating what has really been read and understood let alone conceptualised and applied? From all appearances it would certainly seem so. However, we could just as easily contend that a degree exhibition is the culmination of all that preceded it. However, few art school teachers would seriously assert this, because they’ve come to recognise the importance of examining the journey, not just the destination.

The Fall from Grades

“What would it be like if one of today's art schools offered a single prize so lucrative and prominent that the winner would be virtually assured of making a living? The whole school, I think, would become obsessed with the prize, and suddenly the non-competitive atmosphere of post modern practice would evaporate.” -James Elkins

If you start typing the word “intrinsic” into Google you’re likely to see a drop-down list of suggestions appear. Second down the list will be “intrinsic motivation”. You can do exactly the same for the word “extrinsic” with the same result. These two forms of motivation have a huge role to play in education but almost everybody would hope that the principle motivator for anything students (or staff) do is intrinsic rather than extrinsic. Intrinsic motivation is the desire to do something for its own sake whereas extrinsic motivation seeks approval - or the avoidance of disapproval - or other kinds of rewards (money, prizes accolades, praise etc) from external sources. It’s very well documented that intrinsically motivated people consistently perform better at creative and cognitively challenging tasks, are more persistent and generally enjoy what they do more than extrinsically motivated people.

Of course, it would be an oversimplification to say that people are either intrinsically or extrinsically motivated. Most individuals are intrinsically AND extrinsically motivated by different things to different degrees depending on a variety of factors. Nonetheless we all recognize that some things just motivate us whereas other things, you couldn’t even pay us to do.

There's a rather curious phenomenon which is closely related to these forms of motivation known as the “Overjustification Effect” where people who are intrinsically motivated lose interest in an activity if it becomes associated with rewards. Most particularly they lose the ability to sustain their motivation after rewards are removed. In other words, the introduction of rewards actually destroys peoples’ intrinsic motivation. It's speculated that rewards are closely associated with typical means of persuading people to do dreary work or carry out chores, and this begins to form negative perceptions around any related activity.

Surprise rewards, on the other hand, do have a vaguely positive influence on people's subsequent motivation but such rewards have to be both genuinely unexpected and justified. Rewarding someone for no clear reason or for something which they feel is undeserving can simply lead to confusion.

“These findings are consistent across a variety of subject populations, rewards, and tasks, with the most destructive effects occurring in activities that require creativity or higher-order thinking. That this effect is produced by the extrinsic motivators known as grades has been documented with students of different ages and from different cultures. Yet the findings are rarely cited by educators.” (Alfie Kohn 1994)

Teachers have to tread a very narrow line when it comes to motivation - between challenging and potentially demotivating students. They need to galvanise students’ motivation as a means of challenging them. Challenges are one of the primary means by which people learn and develop but without motivation on the part of students it's often impossible to present challenges meaningfully. One of a teacher's fundamental skills therefore is to accurately gauge the level at which a challenge can be pitched without dulling the pleasure of engagement. For this reason, encouragement is one of the most useful tools at their disposal, but it’s important that this encouragement isn’t some form of disguised praise or the offer of future reward but rather a genuine involvement in the student’s striving to improve what they do.

Generally speaking then, rewards are inadvisable (unless, of course, you want to manipulate someone, and don’t care about the longer term consequences). In ideal circumstances it’s preferable to create as many opportunities as possible for extrinsically motivated people to develop intrinsic motivation. Supporting autonomy, providing choice and alternative ways of achieving successful outcomes, making projects relevant, achievable and challenging and creating an environment which is enabling and supportive, all help people internalize their motivation. In other words anything that promotes, clarifies and makes something rewarding in itself, has the potential to transform an extrinsically motivated individual into an intrinsically motivated one.

Ultimately, it’s not important whether I, as a teacher, think that what a student has done is good or not. What’s important is whether they’ve achieved something which is valuable to their development and whether the goals they’re setting for themselves, and that are set for them, are relevant and challenging enough to maintain their interest and engagement. Everybody strives to improve what they do - we should be building on this desire, not stifling it. The aim of education should be to promote students' independence and self-efficacy as much as possible, not to cultivate self-doubt and dependence. Grades especially, are the last thing that we should be offering as a means of motivating students. Grades are little more than an addictive and pernicious drug peddled by schools and universities to maintain dependency and conformity to their antiquated systems of assessment. Grades are a drug students would do better without – literally.

References:

I've also written on the problem of grading here.

ELKINS, J., 2001. Why Art Cannot be Taught. University of Illinois Press, Urbana and Chicago.

DEAN, J., 2009. How Rewards Can Backfire and Reduce Motivation. [online]. Available from: http://www.spring.org.uk/2009/10/how-rewards-can-backfire-and-reduce-motivation.php

HATTIE, J. and TIMPERLEY, H., 2007. The Power of Feedback. [online]. Available from: http://rer.sagepub.com/content/77/1/81.full.pdf+html

HITZ, R. and DRISCOLL, A., (undated). Praise in the Classroom. [online]. Available from: http://www.ericdigests.org/pre-9213/praise.htm

KAPLAN, A. 2009. Intrinsic and Extrinsic Motivation. [online]. Education.com. Available from: http://www.education.com/reference/article/intrinsic-and-extrinsic-motivation/

KOHN, A. 1994. GRADING: The Issue Is Not How but Why. [online]. Available from: http://www.alfiekohn.org/teaching/grading.htm

KOHN, A., 1999. From Degrading to De-Grading. High School Magazine. [online] Available from: http://www.alfiekohn.org/teaching/fdtd-g.htm