Reading, Fast and Slow

A week or so ago I finished reading “The Craftsman” by Richard Sennett. There is much in it to recommend it but I can’t say that I enjoyed the read, which is to say that I found it disappointing. Admittedly I’d had very high hopes of what I would find in its pages and this is perhaps not the most favourable attitude to bring to bear upon any new experience, even if it comes highly recommended. Of course I’m sympathetic with the underlying premise and it is undoubtedly well researched and a rich source of ideas and references but ultimately I came away feeling little better armed with insights and arguments for the championship of craftsmanship than I’d felt when I picked it up.

No doubt, with greater patience, I could go back and furnish myself with a fuller repertoire of critical tools and examples, but this leads me to wonder what it was about the initial reading that couldn’t provide such things in the first place? Did I read it too quickly perhaps? (I’m a plodding reader at the best of times so I doubt it), is it, dare I say, a fault in craftsmanship on Sennett’s part or should the responsibility fall more squarely upon my own shoulders, on my readercraft?

Rather than rushing at an answer, perhaps it would be more interesting to take a brief detour via a discussion raised by Sennett in relation to IQ tests. Sennett points out that IQ tests are timed and he cautions that there are occasions when questions demand more consideration than a cursory, albeit correct, answer:

“In the spirit of craftsmanship you want to dwell on this problem, making it more specific, puzzling over it – but time is running out. You have to answer as many questions as you can to raise your score, so you guess and go on. Intuitive leaps that open up a problem are impossible to test using multiple-choice questions.”

And:

“Invidious comparison of speed has distorted the measure of quality.”

Yesterday I chanced upon an almost opposing sentiment on the part of Friedrich Nietzsche:

 “’The ‘Plodders.’ –Persons slow of apprehension think that slowness forms part of knowledge.”

Sure, slowness would seem an unlikely contender as a component of knowledge, but conceived instead as ponderousness the complexion of the statement changes considerably, and whilst ponderousness may not be a part of knowledge either, few would disagree that it forms a venerable route towards it. Moreover, if slowness isn’t part of knowledge, logically speaking nor is rapidity (as Sennett wisely cautions). Ponderousness isn’t exactly slow but it demands whatever time is required. It will not be profitably hurried.

So, it’s not really a case of how fast or how slow you apprehend something that matters but how deeply you consider it, how thoroughly you weigh (ponder) what it has to offer. And this leads me back to The Craftsman.

To have gained the maximum from the book I would indeed have to had read it more closely, to have made more notes and perhaps even to have written my own summaries of the most salient passages. But it’s not a one way street, there is still a requirement that the craft of the reader is met by the craft of the author. If there is too great a disparity between these two forms of craftsmanship - or the purposes that underlie them - then the work of reading will never be as fully satisfying, pleasurable or edifying as might otherwise be the case.

"Admittedly, to practice reading as an art … requires one thing above all, and it is something which today more than ever has been thoroughly unlearnt … it is something for which one must be practically bovine and certainly not a ‘modern man’: that is to say, rumination…" — Friedrich Nietzsche, "On the Genealogy of Morals."

Paddling in the Shallows

Teaching Through Failure

George Bernard Shaw’s famous dictum.

There are many routes to becoming an expert – some are littered with failures whilst others emerge with surprisingly little effort. Being fascinated by a subject is certainly a great advantage since it predisposes the individual to devote concentrated time and attention to their chosen interest whilst at the same time providing the impetus to overcome any difficulties encountered (and what more lamentable student is there than the one with proven ability but not the slightest enthusiasm?)

Learning can be a struggle or equally it can seep into consciousness unimpeded. Some would call the propensity to absorb a subject or its components without difficulty a “talent” and though this might be true it rarely explains anything about what underlies talent.

As students, most teachers did well in education and whilst they may have struggled with some subjects, their area of specialism is one in which they most likely excelled. Nonetheless, this achievement may have come about through anything from significant difficulties to effortless ease. They may have failed often, occasionally or only rarely.

But who seems likely to make the better teacher – the one who failed often but eventually succeeded or the one who sailed through easily? And who might make the better practitioner?

A teacher may be brilliant in their subject, but might this brilliance not preclude them to some extent from being a great teacher? If they lack experience of common mistakes are they likely to be able to recognise the difficulties and struggles of those who frequently fail? Moreover, if they have little experience of failure themselves, are they likely to empathize with the associated feelings of confusion, helplessness or demoralisation felt by students in this situation and to know how these might best be overcome? 

If Bernard Shaw was right and those who can, do, and those who can’t, teach, perhaps this is all to the better, since those who "can", may well not be able to teach as well as those who cannot.

Or alternatively as Woody Allen once said:

“Those who can't do, teach. Those who can't teach, teach gym.”

Flag

30% Less than Perfect

Inherited protractor (partially polished)

Anyone who follows this blog will know that I am opposed to grading the work of art students for a variety of reasons of which this post is a continuation. On the other hand, attempting to assign numerical equivalents to more concrete subjects like Engineering or Healthcare for instance seems not unreasonable since the achievement of specific outcomes can both be more clearly set and measured. But without a clear example of what constitutes a top performance (ie: 100%) in a subject, there is little of substance for students to aim at or for staff to use as a benchmark by which other evaluations may be made. As a result it becomes almost impossible to agree that any particular work represents all that is required at any given level and therefore the highest marks simply languish unused waiting for that ‘perfect’ student who, of course, is simply an impossible amalgam of all the very best performances we have seen in the past.

Is a First Class Honours degree a sign of perfect achievement? Not in the least. Even a 100% achievement would only be perfect at Honours level, otherwise why have Post Graduate, Masters and Doctoral study? A First Class Honours degree is simply a reflection of attainment somewhere in the top 30% of the Honours degree classification.

So why on earth are there not more First Class Honours degrees being awarded to graduates each year?

In the UK around 10% of graduating students achieve “a first” while around 20% attain the "third" (third Class) category - the lowest category of Honours degree (graded at 40% or above). The other 70% of students achieve either a "two one" (Second Class Upper), a "two two" (Second Class Lower) or an outright fail. Very few students actually fail, and I’ve never heard of an Honours student getting less than 20%, though I’m sure it must happen.

That means that on average only 10% of students achieve anything higher than 69% at their degree assessment. Can it be true that the other 9 out of 10 students are really so ‘unexceptional’ that more of them can’t even reach the low 70s? Or is the education provided so poor that only the very strongest students manage to keep their heads above water? Neither. The reason there are so few firsts, I suggest, is because the general perception amongst staff is that the highest category should be reserved for the very best performances, or, as a colleague once said to me when I began teaching: “We don’t hand out firsts like sweeties here Jim.” Indeed.

There is a contradiction in this though. If honours level is not the very highest level of study and if a first class honours is not an indication of perfect performance (ie: it should logically contain work that is fully 30% less than perfect at that level) then it is quite clear that the expectations of the highest category of the Honours classification are a distortion.

Surely 100% performance on a degree course should be achievable through the production of the highest standard of work expected at that level, not some unachievable notional pinnacle of excellence. In most cases a top ranking student means a graduate who is fully equipped to begin engaging in a professional context - though it is recognised that even there they will still have a lot to learn. Instead it would seem that the very highest grades attainable (90%+) are being reserved for those one or two geniuses that might, by chance, grace our hallowed halls. And of course those individuals never do arrive and we probably wouldn't even recognise them if they did.

An alternative approach is to dispense with percentages as much as possible and to simply assign each component of work to a band and to add everything up at the end. Whilst this is an improvement it ultimately runs into the same difficulties of vagueness since, without a clear understanding of what represents the highest expected performance, it is impossible to accurately situate that all important cutoff between a First Class and a Second Class degree? Furthermore, this does nothing to remedy or highlight the fact that a First Class Honours degree can and should contain work that is in each and every respect 30% less than the top level expected.

Some stand taller on the shoulders of giants than others

On The Ignorant Schoolmaster

A certain toddler adds notes to my book.

“We can thus dream of a society of the emancipated that would be a society of artists. Such a society would repudiate the division between those who know and those who don’t, between those who possess or don’t possess the property of intelligence.”
— Jacques Rancière, The Ignorant Schoolmaster, (1991)

Recently I finished reading "The Ignorant Schoolmaster" by the French philosopher Jacques Ranciere. Despite having some reservations about confronting another French descendant of the Heideggerian line, I actually found the book immensely thought provoking and almost entirely accessible. Indeed it was heartening to read: “Man is a being that knows very well when someone speaking doesn’t know what he is talking about.” In fact Ranciere does a ‘masterful’ job of interweaving historical record and philosophy in what could be considered a 21^st Century manifesto.

The book centers around the work of Joseph Jacotot, a 19C schoolmaster who developed a “panecastic” method of teaching what he didn't know  - hence the "Ignorant" in the title. One of Jacotot’s five fundamental precepts in his “Universal Education” was that everyone is of equal intelligence. Ranciere makes a fantastic case in championing Universal Teaching – so much so that I found myself making copious notes in the margins. It just so happens though that I've also been reading "The Blank Slate" by Steven Pinker which makes a yet more convincing case for the reverse, backed up by overwhelming evidence and clear - but characteristically lengthy - argument. Ranciere does nonetheless concede that:

“It is true that we don’t know that men are equal. We are saying that they might be. But we know that this might is the very thing that makes a society of humans possible.”

If, as Pinker’s research seems to prove beyond all doubt, we are not of equal intelligence, is it then the case that Ranciere and Jacotot’s arguments entirely crumble? Whilst I would like to have had a good deal more evidence to be convinced that Jacotot’s method really had the impact that is claimed for it, I think it would be a mistake to simply dismiss it. Instead I think we have to seek an alternative explanation for the apparent success of Universal Education. The conclusion I am left with after weighing up the evidence and arguments presented on both sides is that where learning is concerned it is not so important what you are as what you believe. In other words, if you believe you are just as able as everyone else (i.e. of equal intelligence) then this belief is far more likely to lead you to persist where anyone else would desist and to have confidence where anyone else would have doubt. As is becoming increasingly clear; talent is certainly advantageous, but when compared with gritty determination its influence upon achievement is nowhere near as profound. There are exceptions of course, but as Stefan Zweig has pointed out in his book “Casanova: A Study in Self-Portraiture”, talent can be little more than superfluous unless it is allied to persistence and determination:

“He excelled in mathematics no less than in philosophy. He was a competent theologian, preaching his first sermon in a Venetian church when he was not yet 16 years old. As a violinist, he earned his daily bread for a whole year in the San Samuele theatre. When he was eighteen… he became doctor of laws at the University of Padua... He was well informed in chemistry, medicine, history, philosophy, literature, and, above all, in the more lucrative (because perplexing) sciences of astrology and alchemy... As universal dilettante, indeed, he was perfect, knowing an incredible amount of all the arts and all the sciences; but he lacked one thing, and this lack made it impossible for him to become truly productive. He lacked will, resolution, patience.” - Stefan Zweig (2009)

Another important precept in Jacotot’s teaching is this: “every man has received from God the faculty of being able to instruct himself.” i.e. given an opportunity and properly facilitated - especially with a meaningful subject - people are very adept at self education. This may have been a radical claim in its time, but in the present day it appears to be little more than a commonplace. More to the point, it doesn’t in the least explain why in some cases teachers do make a difference. We’ve all been taught by ignorant teachers in our lives but the results were rarely, if ever, positive.

When reading The Ignorant Schoolmaster I was frequently reminded of something Goethe wrote:

"Treat a man as he is, he will remain so. Treat a man the way he can be and ought to be, and he will become as he can be and should be." 

Once again it comes down to the idea of self-belief mentioned above. It seems to me that what really drives the success of Jacotot’s  teaching - indeed all good teaching - is the way that people are treated. Self-belief is an incredibly potent instrument, whether it comes directly from the individual or whether encouraged in them by the conviction of a respected parent, friend or teacher. If it comes from the individual, as seems to have been the case with Jacotot’s students, then all to the better. But if it can be facilitated, enabled or encouraged - so long as this doesn’t engender dependence - then all to the better too.

So, in spite of all the evidence in Pinker's favour, how you "treat" people seems more profoundly crucial for the vast majority of people than those rare and fickle gifts of talent. I don't think I know anyone who hasn't at some point in their lives been inspired by someone - whether a teacher, parent or friend - that made them feel as though they had a special ability - a “talent” if you will - that was worth serious attention and effort. Whether these teachers were in fact correct in spotting some unique gift is perhaps peripheral to the immense force of conviction that comes from being encouraged by the evident faith of someone one respects or admires – by being treated as you can be and ought to be.

Dependents

Visiting aliens could easily be forgiven for thinking those precious little bundles we carry everywhere are just another form of demanding infant – one that screams at us at inopportune moments, making us drop everything else we’re doing, nestling them to our cheeks and reassuring them with gentle murmurings. We pour our hearts into them, dote on them and spend more money on them than we ever intended to. We even feed them regularly via their own umbilical cord. But despite all the long conversations, topups and upgrades they never seem to grow up, nor do we cease to be fascinated staring into their bright little eyes.

Assessors

TEACHER 1: So, before we get down to looking through this student’s work what's your gut feeling about their mark… a “B”?

TEACHER 2: Sorry, I don’t think this is appropriate – we haven’t even looked at the backup work yet.

TEACHER 1: Sure, but you must have an idea of where they stand in terms of mark.

TEACHER 2: No, I don’t know that that I do – that’s what this process is supposed to be about surely.

TEACHER 1: It is. I was just trying to get a sense of whether we were in the least bit in agreement about the overall achievement of this student.

TEACHER 2: How can we even know that without having looked at their work?

TEACHER 1: We have though, throughout the year. Okay, I haven’t gone through it in detail yet but we’ve known them for several years now and we’ve got a pretty good sense of their track record surely?

TEACHER 2: So you think that by plucking a mark out of the air that we will be more likely to award the correct mark?

TEACHER 1: Steady on! No, not in the least. I just thought it might be useful if we knew where we were starting from, that’s all.

TEACHER 2: Well I’d prefer if we didn’t start by pigeonholing anybody and I’d also prefer not to be influenced by suggested marks before I’ve even had a chance to go through the work and form my own opinion.

TEACHER 1: That’s totally fine by me. But you seem to be suggesting that starting with a provisional mark is wrong.

TEACHER 2: I think it is wrong.

TEACHER 1: Really, why?

TEACHER 2: Because you’re starting with a bias and the likelihood is that you’ll just end up confirming the bias.

TEACHER 1: I don’t think that’s true at all. I start with a sense of what the student has achieved in the past and how their current performance compares with that. I’m also comparing their work with other work I have seen over the years at this level. As I look through their work I’m obviously looking for evidence to contradict my initial assumptions. If I find it I adjust my evaluation. If I don’t find any, my evaluation stays the same. I don’t see how that’s wrong.

TEACHER 2: And how can you be sure that the mark you start with isn’t subtly influencing your evaluation?

TEACHER 1: I’m not using it to reinforce my opinions. I’m using it as a starting point. I’m quite happy for it to change – radically if necessary. But are you telling me that you never come to the assessment of work with a ballpark idea of a mark?

TEACHER 2: I think it’s important to try to be as objective as possible.

TEACHER 1: Sure, but that’s not really what I asked. Are you saying that you treat each student’s work as though you’ve never seen it before?

TEACHER 2: I try to divorce myself from any preferences I might have formed in the time that I have known the students and yes I try to approach each student’s work as if I’ve never marked it before. Their track record is totally irrelevant as far as I’m concerned. If they have done really well in the past or really badly it has no baring on what they have submitted here. I have to mark it in relation to what is expected at the level.

TEACHER 1: So it doesn’t matter if they are seriously struggling or have improved massively since their last assessment?

TEACHER 2: Of course it matters, and we should mention it in our feedback, but that’s not what we’re marking. Marks aren’t rewards or punishments for getting better or worse. They’re an indication of the attainment of a specific level of learning.

TEACHER 1: Yes yes I know that, but do you think that we should completely ignore their journey?

TEACHER 2: In the marks, yes.

TEACHER 1: Going back to something you said a moment ago - you said you gauge the students’ work in relation to the level that is expected?

TEACHER 2: Of course.

TEACHER 1: Then I don’t really see how that is significantly different from having a ballpark idea of the mark at the beginning. Surely as you look through their work you begin to form an idea of the mark and then you keep looking until the evidence becomes overwhelming one way or the other?

TEACHER 2: No, that’s not how I evaluate things at all. I start with what is expected at the level and as I go through the work I’m looking for how well it achieves the required level. The marks are secondary.

TEACHER 1: But that sounds almost identical to what I do except I’ve already converted the level outcomes to a grade in my mind beforehand, but I‘m still looking for what is expected at the level.

LATER:

TEACHER 1: So what mark would you award this work?

TEACHER 2: I think it’s a “C”.

TEACHER 1: Yeah, I think it’s a “C” too. So, why do you think it’s a C?...

Any resemblance in the preceding script to characters or situations real or imagined is purely coincidental.

Intuition and Prejudice

“Assessments are based, not on whether the decisions made are any good, but on whether they were made in accordance with what is deemed to be an appropriate process. We assume, not only that good procedure will give rise to good outcome, but also that the ability to articulate the procedure is the key to good outcomes.” -John Kay

It could be argued that one of the principle differences between a novice and an expert is the degree to which experts are able to explain the reasons for any judgement made and to supply these retrospectively. But this begs a question: if experts make intuitive judgements – which they often do - then how can we be sure that these are not simply preferences dressed up in the trappings of deft argumentation?

A standard reply would go something like this: if the justification for an intuitive judgement stands up to rigourous debate or close scrutiny then we have no alternative than to accept the justification as valid. If such valid judgements are performed repeatedly then we need no further proof of expertise. Novices, on the other hand, are less competent in teasing out or debating their reasoning process and their judgements tend to be more inconsistent. So, even if a novice makes the same choice as an expert, we cannot say that they have exercised judgement, since judgement - in order to be judgement - needs to be justifiable.

But is this not simply an intellectual sleight of hand that surreptitiously places the quality of the justification above the quality of the decision? And whilst we may accept that instantaneous decisions are justifiable post-hoc, this still doesn’t answer the question of how such decisions can be made without calling upon resources that require far more time-consuming and considered deliberation. In other words: what is actually going on when judgements are made in an intuitive instant?

Dylan Wiliam writes of experts having what he calls “scripts” that allow them to make rapid decisions and improvise within their domain of expertise. He cites the work of Chase and Simon and their studies comparing expert and novice chess players. Chase and Simon found that expert chess players were significantly more accurate at quickly memorizing the positions of chess pieces placed in positions of actual games whereas they were significantly worse at memorizing the positions of randomly arranged chess pieces:

“Chase and Simon suggest that the much better performance of experts with ‘real’ chessboards stems from an ability to ‘read’ a chessboard in terms of a series of standard configurations of pieces that they have learnt through their experience as chess players.” 

Such expertise is the product of prolonged experience within a domain and cannot be acquired through shortcuts, formulae or crash-courses. Nonetheless we are all experts in the art of everyday life and the vast majority of the decisions we make are instantaneous and automatic. If pressed, we would have no difficulty in providing reasons for our having stopped at a red light. But can it be said that the background familiarity amassed by experts - us included - constitutes a form of criteria, or are criteria simply a means to externalize and make explicit what is essentially a tacit process that works in an entirely different way and therefore allows many decisions to be made intuitively rather than demanding deeper reflection?

"It is fairly clear that teachers acquire notions of ‘standards’ much more effectively when presented with actual samples of students’ work that exemplify the standards being promulgated than when given criterion-based descriptions of the standards. … I would like to suggest that the explanation of this phenomenon is that the grading of pieces of work with respect to a set of internal standards involves a far greater use of unconscious processing than has previously been acknowledged." -Dylan Wiliam

Psychologists, like Daniel Kahneman, distinguish between two kinds of decision making systems: System 1 and System 2. System 1 is roughly what we would think of as intuition. System 2 could be described as deliberative thinking:

“System 2 is the one who believes that it's making the decisions. But in reality, most of the time, System 1 is acting on its own, without your being aware of it. It's System 1 that decides whether you like a person, which thoughts or associations come to mind, and what you feel about something. All of this happens automatically. You can't help it, and yet you often base your decisions on it. […] System 2, on the other hand, is lazy and only becomes active when necessary. Slow, deliberate thinking is hard work. It consumes chemical resources in the brain, and people usually don't like that.” [my emphasis]

From the discussion so far it would seem that experts do not have sufficient time to formulate let alone deliberate over criteria in their intuitive judgements. This is not to say that they never pause to contemplate things more closely or even to reverse their initial judgements. They can and sometimes do. But what it does suggest is that criteria are not the primary tools of experts: experience is. By experience I mean the repertoire of exemplars that have been acquired through studying something closely – through comparing different instances and considering the relative merits of various examples, both good and bad. One nectarine alone would never be sufficient to furnish anyone with a clear idea of what constitutes a good nectarine. We need to experience a range of qualities of an experience in order to formulate an awareness of what might enhance it or detract from it, of whether the juice running down our chin is a sign of succulence or overripeness.

The point I’m making is that experts do not use criteria or deliberative thinking to form their initial judgements, instead they draw on experience and the more extensive this experience, the more well informed their judgements. However, an intuitive judgement, no matter how well informed, is not - strictly speaking - a judgement at all, but a pre-judgement; a pre-judice: an evaluation made without due consideration of the specific issues or circumstances at hand. It is a shortcut based on an unconscious repository of prior examples and is therefore limited to the known and familiar.

If the prevailing conditions in any situation that urgently demands judgement bear a close resemblance to previously encountered circumstances then it is obviously appropriate not to waste valuable time and energy logically weighing up the options.

“Intuitive intelligence is the ability to know directly, to perceive and appreciate whole or hidden patterns beyond (or faster than) logic.” -Tom Atlee (The Co-Intelligence Institute) 

But where there is no urgency, and especially where decisions impact on other people – students for instance – it seems prudent to be wary of the tendency to cut cognitive corners. As the research of Daniel Kahneman, Philip Tetlock and others increasingly shows (although it is not impossible to find exceptions), even the judgements of experts can be far from reliable, especially when making predictions about intrinsically complex and unpredictable domains. In the arts (which are well known for their unpredictability), as in other creative fields, intuition is frequently revered as a unique and often indispensable route to problem solving and discovery. But in arts education, where the judgements of teachers are directed not just towards artworks but towards individuals, too great a reliance upon intuitive judgement or gut-feeling risks undermining the putative integrity of academic assessment. And there’s the rub: what works for the production of art may well be at odds with its evaluation.

You might also be interested in the following article (Thanks to Tor for the link)