Unconscious Vision

Melvyn Goodale and David Milner describe the extraordinary case of patient DF, a woman whose bizarre perceptual deficits led to a new understanding of how the brain processes visual information

The Biologist 65(4) p20-23

Thirty years ago in a small village in northern Italy, a young Scottish woman known as Dee* was taking a shower in the cottage she and her partner were renovating. She was unaware that their water heater was improperly vented, and carbon monoxide began to slowly and insidiously accumulate in the bathroom.

This colourless, odourless and deadly gas displaces oxygen from the haemoglobin in red blood cells, starving the body and the brain. Dee’s brain quickly became hypoxic and she passed out. Miraculously, her partner, Carlo, found her before she stopped breathing and rushed her to hospital.

When Dee regained consciousness some 24 hours later, it was clear that something was badly wrong. She seemed totally blind, unable to see Carlo standing beside her or anything else in the room except light and dark.

Over the next few days, her vision gradually improved, but although Dee could tell the colour of objects, and even the material they were made from, she could not identify anything on the basis of its shape or form. She couldn’t even recognise her partner or mother, except when Carlo was wearing a pullover with a familiar and distinctive colour. On one puzzling occasion, she remarked that she could clearly see the small hairs on the back of her mother’s hand, yet could not recognise the hand as a hand.

Some months later, when Dee came back to Scotland for a visit, we had an opportunity to assess her visual abilities in our laboratory at the University of St Andrews. It became clear she could not discriminate between shapes as simple as a square and a triangle.

Her failure to recognise objects was not because she had some sort of disconnection between the visual appearance of an object and its name or meaning. When asked to copy simple line drawings of objects (drawings that she could not identify), her copies bore very little resemblance to the originals, yet when asked to draw the same objects from memory, her drawings were quite recognisable. She also could not recognise her own drawings when shown them later.

Dee’s object recognition did benefit from the addition of colour and shading. We found that Dee was also quite unable to identify letters of the alphabet or digits. In short, Dee appeared to have a fundamental deficit in the visual processing of form, rather than a higher-level deficit in understanding what an object is. The damage in Dee’s hypoxic brain had left her with a rare neurological disorder called visual form agnosia, an inability to perceive the form or shape of objects. Moreover, her deficit was purely visual: she had no problem identifying objects from touch.

One day, however, we were given a hint that would lead us to change our assessment of Dee’s visual abilities – and to change our thinking about how the human visual system might be organised. To demonstrate to a colleague just how profound Dee’s visual deficit was, we held up a pencil at different orientations in front of her, and asked her to tell us whether the pencil was oriented vertically, horizontally or diagonally. Even though she could report (correctly) that the pencil was yellow, she had absolutely no idea what its orientation was. Her guesses were completely at chance.

Then something extraordinary happened. She suddenly said “Wait a minute, let me see that” and reached out and grasped the pencil. The orientation of her grasping hand as it approached the pencil corresponded exactly to its orientation, even though a moment before she had been unable to tell which way it was oriented (see Fig. 1, below). We tested her again with a different orientation and once more she rotated her hand ‘in flight’ so that her fingers closed around the pencil perfectly. This dramatic observation led us to test her on a range of different visually guided motor tasks, from posting a card through a slot to picking up rectangular objects with different widths.

Fig. 1. The examiner held a pencil vertically, slanted or horizontally. Even though Dee could only guess the orientation of the pencil, she always grasped it perfectly.

In all these tests, her behaviour was essentially normal: she rotated her wrist appropriately to insert a card into a slot presented at different orientations (see Fig. 2, below) and she scaled the opening of her fingers in flight to objects of different widths in order to pick them up adeptly. Yet at the same time she was completely unable to use vision to indicate either verbally or manually the orientation of the slot or the width of the object.

The fact that Dee could use vision to control limb and hand movements toward objects whose features she could not perceive presented us with a real puzzle. How could she act on visual information she apparently could not perceive?

Fig. 2. Dee was asked to hold a card so that it matched the orientation of the slot, without reaching out toward the display (‘matching’). She was later asked to ‘post’ the card into the slot (‘posting’). Dee performed almost randomly when matching the card to the slot, but had no problem posting it at the right angle. (Although the slot was presented in several orientations, the diagrams show ‘correct’ as vertical.)

One clue came from the brain scans that she had undergone in Italy after her accident. Although the scans showed the typical pattern of diffuse brain damage associated with carbon monoxide poisoning, it was evident that Dee had particularly dense damage on both sides of her brain in the so-called ventral stream of visual projections, one of two major visual pathways in the primate brain that arise in early visual areas (See Fig. 3, below).

The other pathway, the dorsal stream, appeared to be relatively unscathed. Could it be that it was the damage to her ventral stream that was disrupting her ability to ‘see’ the shape of objects, while her spared dorsal stream allowed her to grasp those same objects quite normally?

There was certainly tantalising evidence from anatomical and electrophysiological work in monkeys to support this idea. It had been known for a long time, for example, that monkeys with large lesions of the occipital and temporal lobes, largely destroying the ventral stream, could no longer recognise objects, even though they had no trouble in picking them up quite deftly. And while we were testing Dee in Scotland, a group in Japan had shown that neurons in the monkey’s dorsal stream were ‘tuned’ to the size, shape and orientation of goal objects that the monkey had been trained to grasp.

There was also supporting evidence from other human studies. A group working in France had shown that patients with lesions of the parietal lobe, damaging areas within the dorsal stream, exhibited a pattern of behaviour that appeared to be the exact opposite of that shown by Dee. These patients could recognise the orientation of a slot, for example, but had great difficulty rotating their wrist so that they could insert their hand into the slot – until they made contact with it.

Fig. 3. The two streams of visual processing in a primate cerebral cortex. The ventral stream receives nearly all of its visual input from the primary visual cortex (V1), which in turn receives its input from the lateral geniculate nucleus (LGNd) of the thalamus. The dorsal stream also receives input from V1, but also gets substantial inputs from the superior colliculus (SC) via the pulvinar, another nucleus in the thalamus that projects to areas (V3 and MT) in the cerebral cortex that eventually carry visual information to the posterior parietal cortex. The arrows on the inset photograph of the brain show the approximate route of the two streams within the cerebral hemispheres.

We too had an opportunity to test individuals with dorsal-stream damage and found that, in stark contrast to Dee, many of them could easily identify the shape and size of objects, while unable to shape their fingers into the correct posture as they reached out to pick up those same objects.

At the time, all of this evidence was pointing to one inevitable conclusion: the ventral and dorsal visual streams seemed to be doing very different jobs. But why should that be the case? To answer this question, we realised, one had to step back and ask another fundamental question: why do we need vision in the first place? A compelling argument could be made that we need vision for two quite different but complementary reasons.

On the one hand, we need vision to understand the world beyond our bodies, to identify and remember objects and places, and to plan different courses of action. On the other hand, we also need vision to guide our actions in that world at the very moment we make them. These are quite different job descriptions, requiring very different computations – and, as a consequence, two different visual systems have evolved to carry them out.

In 1992, in a brief essay published in Trends in Neurosciences, we proposed that it is the ventral visual stream, a network that is intimately interconnected with the cognitive areas of the brain, that enables us to recognise objects and build up a ‘database’ of the world.

Thus, when we talk about ‘seeing’ something, it is the ventral stream that allows us to have that perceptual experience.

By contrast, we argued, it is the dorsal stream that provides the visual control of our actions, enabling us to move about and interact with objects, often with exquisite timing and control, even though we might have no conscious experience of the visual information that is providing that control. Although we can see that we are playing a game of tennis, for example, the visual information that allows us to hit the approaching tennis ball with the sweet spot of the racquet is quite inaccessible to consciousness.

The idea of two visual systems in a single brain might seem initially counter-intuitive or even absurd. Our visual experience of the world is so compelling and apparently unitary that it is hard to believe that some other quite independent visual system – one that operates completely outside of consciousness – is guiding our movements. Indeed, at the time we began testing Dee’s visual abilities, it was the near-unanimous belief of neuroscientists, including researchers specialising in vision, that vision was served by a single system.

Although the inputs to the system were known as being extremely complex and to some degree handled through separate neural channels, the consensus was that they were somehow combined in the brain to provide us with a unified visual representation of the world that then served as a perceptual foundation for all our thoughts and actions. The striking dissociation we had observed in Dee challenged this monolithic conception of how vision works, and we now know that there is a fundamental difference in the way in which vision for perception and vision for the control of action are organised in the human brain.

Decision-making in sports like tennis may be based on a conscious picture of what is happening on the court. But a large amount of visual information is processed unconsciously in order to precisely execute each shot

Since we put forward these ideas 25 years ago, a lot has happened. The advent of functional magnetic resonance imaging (MRI), in particular, has revealed in exquisite detail the functional organisation of the human visual system. The story that is emerging from imaging studies largely confirms the conclusions of the earlier neuropsychological studies in patients and the neurophysiological work in monkeys. Thus, when the brains of healthy volunteers are scanned while they are simply looking at objects, there is robust activation in the ventral stream, and when the same volunteers are asked to reach out and grasp the object, there is activation in the dorsal stream.

Functional MRI has also shown that within the ventral system there are separate channels for analysing object geometry and surface properties. Dee’s brain damage spares the areas serving colour and texture perception, whereas other patients lose their perception of such surface properties while retaining good shape perception.

The division of labour between the ventral and dorsal stream is one ultimately geared towards the requirements of the output systems that the two streams serve. Natural selection slowly moulded the neural circuitry in these pathways to reflect the different demands the environment presented to our distant mammalian ancestors. We have talked about dissociations between perception and action but, of course, perception is shorthand for the more elaborate visual processing that higher mammals can exploit in order to improve on their choice and planning of behavioural acts.

In other words, the ventral stream contributes to action every bit as much as the dorsal stream – it just does it in a more circuitous way, by engaging other cognitive modules with which it is connected. If it did not promote more adaptively successful behaviour, the ventral stream would simply never have evolved.

It should be emphasised that this new conception of unconscious vision playing a critical role in the online control of movements in no way denies that we often have a full visual awareness of the objects upon which we are acting. What it does claim is that the visual processing that controls our movements is segregated from the processing that furnishes our awareness. The two kinds of visual processing are, at least to a great extent, going on quietly in parallel. Our common-sense belief that our conscious vision controls our actions is an illusion.

Yet even a cursory glance at the anatomical connections between different visual areas in the brain shows that the two streams are not hermetically sealed from one another. Indeed, they need to work together constantly in everyday life. To illustrate one important kind of interaction, we may draw an analogy with tele-assistance, whereby a human operator acts to guide semi-autonomous robots on the surface of a distant planet. The robot can efficiently gather rock samples, but is guided by the operator as to which samples would be most worthy of attention. This has proven to be a highly successful mode of operation, and is reflected in the way the ventral and dorsal streams interact.

In the case of the human brain, inter-stream interactions take several forms. For example, the ventral stream through its close links with memory systems can provide the dorsal stream with information not only about recently encountered visual objects, but also with functional or ‘semantic’ knowledge about objects. These long-term memories include, for example, the material properties that characterise particular objects and the uses to which manufactured objects such as tools can be put.

None of this, of course, sheds much light on the age-old concern of philosophers to ‘explain’ consciousness. Although modern cognitive neuroscience is constantly offering clues as to what kinds of brain processing are and are not associated with visual awareness, we are still far away from an understanding of the neural basis of consciousness.

Professor Melvyn Goodale FRSC FRS FRSB is distinguished university professor in visual neuroscience and director of the Brain and Mind Institute at Western University, Canada.

Professor David Milner FRSE FRS is emeritus professor at Durham University and an honorary professor at the University of Edinburgh.