The Structure of Representations

[C]lassification cannot be embodied in, or reduced to, or dissolved in, discriminant behaviour directed towards the classified objects, but involves the use of symbols specific to the purpose. These symbols will be explicit symbols: they will not be mistaken for the things classified; they will stand off from the world whose contents are being classified. (Raymond Tallis 2000)

It’s almost a platitude to say that representations come in all shapes and sizes, but it isn’t a platitude to observe that there is no representation without structure. Even in their most ephemeral and ethereal forms, words, images, models and performances are far from immaterial, and when we consume them, their contours rarely escape our notice. If not for this awareness of the differences between representations and the things they are used to represent, there could be little, if anything in the way of representations or acts of representing. A clone of an apple is just another apple[1]. If on waking, we were to find ourselves in an identical copy of this world, we would accept it as readily as this one. Interestingly, such a copy would not qualify as an illusion because the concept of sameness logically excludes illusion. If you surreptitiously swap your dog’s dinner bowl with another of the same design, there is no illusion involved and no representation either. So much for the suggestion that we might be dwelling in a simulated universe!

Not only are representations structured, they must be crafted, manufactured or otherwise performed, and thus the skills necessary to do so must first be discovered, developed and refined through social and environmental feedback. Nothing as sophisticated as representation could just spring out of the evolutionary sandbox fully formed. Deaf songbirds never learn to sing, let alone to pick-up the localised “dialects” paraded by their chirruping friends (Marler 2004). Without feedback, there could be no opportunity for the shaping or “grounding” (Harnad 1990) of communication. There could be no right way and no wrong way of depicting, characterising or even mimicking aspects of the environment. Even camouflage, which is arguably not a form of representation[2], would be impossible, because there would be no constraining mechanisms guiding its evolutionary development.

Perceptibility in Principle

Must representations always be perceptible? Can imperceptibly small images, models or words still be representations? So long as a candidate representation remains perceptible in principle, it seems perfectly reasonable to say that it qualifies as a representation. But if we divide a representation into fragments and rearrange these in an unrecognisable form, then I think it is fair to say that they would no longer qualify as representations.

"Imagine a painting cut up into small, almost monochromatic bits which are then used as pieces in a jig-saw puzzle. Even when such a piece is not monochromatic it would not indicate any three-dimensional shape, but should appear as the flat colour patch. Only together with the other pieces does it become a bit of blue sky, a shadow, a highlight, transparent or opaque, etc. Do the individual pieces show us the real colours of the parts of the picture?" (Wittgenstein, “Remarks on Colour” §60, 1977)

Data Processing and Storage

It might be argued that the data stored on a hard drive is representational. At best, this would be “representation” dressed up in a lab coat, not representation in its ordinary non-technical attire. Many objects and patterns have representational utility (i.e. they are apt for representational interpretation and use), but they do not constitute representations unless we treat them as such. So for example, the highly patterned structure of DNA is widely regarded as a code, but few self-respecting biologists would wager that the “genetic code” is anything other than a convenient metaphor for the exquisitely computable structures of Guanine, Adenine, Thymine and Cytosine. Similarly, the facial movements of ordinary speech are not representations in addition to the words we produce, but they are nonetheless a visible component of speech and can be reliably interpreted as representations by lip readers.

So when we say that photographs, videos and emails etc. are stored on digital media, it needn’t follow that this storage is representational too. Computation relies on the fact that we can convert analogue signals into the meaningless binary states of digital processing and storage media. Think of Wittgenstein’s jigsaw again. When the picture is disassembled, it no longer represents anything. Only when the fragments are reassembled do they become an image once more. So even though computers disassemble their various inputs in highly structured ways (into what we commonly refer to as “data’), this structuring alone is not sufficient to constitute representation. Data stored on digital media has to be converted back into representational form before we can recognise it as such.

According to Jerry Fodor (1981): “There is no computation without representation.” This assertion lies at the foundations of computationalism; the doctrine that our brains are biological computing mechanisms. Fodor is half right. There can be no computation without representation because computation is a human invention necessarily dependent upon a history of cultural innovations involving the manipulation of symbolic representations. If not for this thoroughly human knowhow we would never have discovered how to create machines capable of saving us the trouble of crafting, producing and manipulating (including computing) representations in the first place.

---

[1] This is not to suggest that apples are any less apt for representational use than anything else of course.

[2] Because it isn’t performed for the purposes of communication.

Thanks to John Ragin, for some very helpful discussions about digital processing and storage.