As a part of social cognition, people automatically construct rich models of other people's vision. Here we show that when people judge the mechanical forces acting on an object, their judgments are biased by another person gazing at the object. The bias is consistent with an implicit perception that gaze adds a gentle force, pushing on the object. The bias was present even though the participants were not explicitly aware of it and claimed that they did not believe in an extramission view of vision (a common folk view of vision in which the eyes emit an invisible energy). A similar result was not obtained on control trials when participants saw a blindfolded face turned toward the object, or a face with open eyes turned away from the object. The findings suggest that people automatically and implicitly generate a model of other people's vision that uses the simplifying construct of beams coming out of the eyes. This implicit model of active gaze may be a hidden, yet fundamental, part of the rich process of social cognition, contributing to how we perceive visual agency. It may also help explain the extraordinary cultural persistence of the extramission myth of vision.

In the attention schema theory, awareness is an impossible, physically incoherent property that is described by a packet of information in the brain. That packet of information is an internal model and its function is to provide a continuously updated account of attention. It describes attention in a manner that is accurate enough to be useful but not so accurate or detailed as to waste time or resources. In effect, subjective awareness is a caricature of attention. One advantage of this theory of awareness is that it is buildable. No part of it requires a metaphysical leap from chemistry to qualia. In this article we consider how to build a conscious machine as a way to introduce the attention schema theory.

Many people show a left-right bias in visual processing. We measured spatial bias in neurotypical participants using a variant of the line bisection task. In the same participants, we measured performance in a social cognition task. This theory-of-mind task measured whether each participant had a processing-speed bias toward the right of, or left of, a cartoon agent about which the participant was thinking. Crucially, the cartoon was rotated such that what was left and right with respect to the cartoon was up and down with respect to the participant. Thus, a person's own left-right bias could not align directly onto left and right with respect to the cartoon head. Performance on the two tasks was significantly correlated. People who had a natural bias toward processing their own left side of space were quicker to process how the cartoon might think about objects to the left side of its face, and likewise for a rightward bias. One possible interpretation of these results is that the act of processing one's own personal space shares some of the same underlying mechanisms as the social cognitive act of reconstructing someone else's processing of their space.

Visual attention and awareness can be experimentally separated. In a recent study (Webb , Cortical networks involved in visual awareness independently of visual attention. 2016a;113:13923-8), we suggested that awareness was associated with activity in a set of cortical networks that overlap the temporoparietal junction. In a comment, Morales (Measuring away an attentional confound? 2017;3:doi:10.1093/nc/nix018) suggested that we had imperfectly controlled attention thereby jeopardizing the experimental logic. Though we agree that attention behaves differently in the presence and absence of awareness, we argue it is still possible to roughly equate the level of attention between aware and unaware conditions, and that an imbalance in attention probably does not explain our experimental results.

The purpose of the attention schema theory is to explain how an information-processing device, the brain, arrives at the claim that it possesses a non-physical, subjective awareness, and assigns a high degree of certainty to that extraordinary claim. The theory does not address how the brain might actually possess a non-physical essence. It is not a theory that deals in the non-physical. It is about the computations that cause a machine to make a claim and to assign a high degree of certainty to the claim. The theory is offered as a possible starting point for building artificial consciousness. Given current technology, it should be possible to build a machine that contains a rich internal model of what consciousness is, attributes that property of consciousness to itself and to the people it interacts with, and uses that attribution to make predictions about human behavior. Such a machine would “believe” it is conscious and act like it is conscious, in the same sense that the human machine believes and acts.

The neural basis of autism spectrum disorder (ASD) is not yet understood. ASD is marked by social deficits and is strongly associated with cerebellar abnormalities. We studied the organization and cerebellar connectivity of the temporoparietal junction (TPJ), an area that plays a crucial role in social cognition. We applied localized independent component analysis to resting-state fMRI data from autistic and neurotypical adolescents to yield an unbiased parcellation of the bilateral TPJ into 11 independent components (ICs). A comparison between neurotypical and autistic adolescents showed that the organization of the TPJ was not significantly altered in ASD. Second, we used the time courses of the TPJ ICs as spatially unbiased "seeds" for a functional connectivity analysis applied to voxels within the cerebellum. We found that the cerebellum contained a fine-grained, lateralized map of the TPJ. The connectivity of the TPJ subdivisions with cerebellar zones showed one striking difference in ASD. The right dorsal TPJ showed markedly less connectivity with the left Crus II. Disturbed cerebellar input to this key region for cognition and multimodal integration may contribute to social deficits in ASD. The findings might also suggest that the right TPJ and/or left Crus II are potential targets for noninvasive brain stimulation therapies.

Information processing in specialized, spatially distributed brain networks underlies the diversity and complexity of our cognitive and behavioral repertoire. Networks converge at a small number of hubs - highly connected regions that are central for multimodal integration and higher-order cognition. We review one major network hub of the human brain: the inferior parietal lobule and the overlapping temporoparietal junction (IPL/TPJ). The IPL is greatly expanded in humans compared to other primates and matures late in human development, consistent with its importance in higher-order functions. Evidence from neuroimaging studies suggests that the IPL/TPJ participates in a broad range of behaviors and functions, from bottom-up perception to cognitive capacities that are uniquely human. The organization of the IPL/TPJ is challenging to study due to the complex anatomy and high inter-individual variability of this cortical region. In this review we aimed to synthesize findings from anatomical and functional studies of the IPL/TPJ that used neuroimaging at rest and during a wide range of tasks. The first half of the review describes subdivisions of the IPL/TPJ identified using cytoarchitectonics, resting-state functional connectivity analysis and structural connectivity methods. The second half of the article reviews IPL/TPJ activations and network participation in bottom-up attention, lower-order self-perception, undirected thinking, episodic memory and social cognition. The central theme of this review is to discuss how network nodes within the IPL/TPJ are organized and how they participate in human perception and cognition.

The attention schema theory offers one possible account for how we claim to have consciousness. The theory begins with attention, a mechanistic method of handling data in which some signals are enhanced at the expense of other signals and are more deeply processed. In the theory, the brain does more than just use attention. It also constructs an internal model, or representation, of attention. That internal model contains incomplete, schematic information about what attention is, what the consequences of attention are, and what its own attention is doing at any moment. This “attention schema” is used to help control attention, much like the “body schema,” the brain’s internal simulation of the body, is used to help control the body. Subjective awareness – consciousness – is the caricature of attention depicted by that internal model. This article summarizes the theory and discusses its relationship to the approach to consciousness that is called “illusionism.”

The attention schema theory is a proposed explanation for the brain basis of conscious experience. The theory is mechanistic, testable, and supported by at least some preliminary experiments. In the theory, subjective awareness is an internal model of attention that serves several adaptive functions. This chapter discusses the evolution of consciousness in the context of the attention schema theory, beginning with the evolution of attentional mechanisms that emerged more than half a billion years ago and extending to human consciousness and the social attribution of conscious states to others.

It is now well established that visual attention, as measured with standard spatial attention tasks, and visual awareness, as measured by report, can be dissociated. It is possible to attend to a stimulus with no reported awareness of the stimulus. We used a behavioral paradigm in which people were aware of a stimulus in one condition and unaware of it in another condition, but the stimulus drew a similar amount of spatial attention in both conditions. The paradigm allowed us to test for brain regions active in association with awareness independent of level of attention. Participants performed the task in an MRI scanner. We looked for brain regions that were more active in the aware than the unaware trials. The largest cluster of activity was obtained in the temporoparietal junction (TPJ) bilaterally. Local independent component analysis (ICA) revealed that this activity contained three distinct, but overlapping, components: a bilateral, anterior component; a left dorsal component; and a right dorsal component. These components had brain-wide functional connectivity that partially overlapped the ventral attention network and the frontoparietal control network. In contrast, no significant activity in association with awareness was found in the banks of the intraparietal sulcus, a region connected to the dorsal attention network and traditionally associated with attention control. These results show the importance of separating awareness and attention when testing for cortical substrates. They are also consistent with a recent proposal that awareness is associated with ventral attention areas, especially in the TPJ.

Previous studies show that it is possible to attend to a stimulus without awareness of it. Whether attention and awareness are independent or have a specific relationship, however, remains debated. Here, we tested three aspects of visual attention with and without awareness of the visual stimulus. Metacontrast masking rendered participants either subjectively aware or not aware of the stimulus. Attention drawn to the stimulus was measured by using the stimulus as a cue in a spatial attention task. We found that attention was drawn to the stimulus regardless of whether or not people were aware of it. However, attention changed significantly in the absence of awareness in at least three ways. First, attention to a task-relevant stimulus was less stable over time. Second, inhibition of return, the automatic suppression of attention to a task-irrelevant stimulus, was reduced. Third, attention was more driven by the luminance contrast of the stimulus. These findings add to the growing information on the behavior of attention with and without awareness. The findings are also consistent with our recently proposed account of the relationship between attention and awareness. In the attention schema theory, awareness is the internal model of attention. Just as the brain contains a body schema that models the body and helps control the body, so it contains an attention schema that helps control attention. In that theory, in the absence of awareness, the control of attention should suffer in basic ways predictable from dynamical systems theory. The present results confirm some of those predictions.

The map of the body in the motor cortex is one of the most iconic images in neuroscience. The map, however, is not perfect. It contains overlaps, reversals, and fractures. The complex pattern suggests that a body plan is not the only organizing principle. Recently a second organizing principle was discovered: an action map. The motor cortex appears to contain functional zones, each of which emphasizes an ethologically relevant category of behavior. Some of these complex actions can be evoked by cortical stimulation. Although the findings were initially controversial, interest in the ethological action map has grown. Experiments on primates, mice, and rats have now confirmed and extended the earlier findings with a range of new methods.

The temporoparietal junction (TPJ) is activated in association with a large range of functions, including social cognition, episodic memory retrieval, and attentional reorienting. An ongoing debate is whether the TPJ performs an overarching, domain-general computation, or whether functions reside in domain-specific subdivisions. We scanned subjects with fMRI during five tasks known to activate the TPJ, probing social, attentional, and memory functions, and used data-driven parcellation (independent component analysis) to isolate task-related functional processes in the bilateral TPJ. We found that one dorsal component in the right TPJ, which was connected with the frontoparietal control network, was activated in all of the tasks. Other TPJ subregions were specific for attentional reorienting, oddball target detection, or social attribution of belief. The TPJ components that participated in attentional reorienting and oddball target detection appeared spatially separated, but both were connected with the ventral attention network. The TPJ component that participated in the theory-of-mind task was part of the default-mode network. Further, we found that the BOLD response in the domain-general dorsal component had a longer latency than responses in the domain-specific components, suggesting an involvement in distinct, perhaps postperceptual, computations. These findings suggest that the TPJ performs both domain-general and domain-specific computations that reside within spatially distinct functional components.

The organization of the motor cortex has been studied and debated for more than 130 years. Although it contains a map of the body, the map is overlapping and fractured and therefore additional principles of organization may be needed to explain the topography. Recently, a growing body of evidence suggests one such principle. The motor cortex appears to be partly organized as a map of complex, behaviorally useful actions that compose the animal’s movement repertoire. In the action-map perspective, the statistical complexity of the movement repertoire leads to the complexity of the cortical map.

Three main views of the primate motor cortex have been proposed over the 140 years of its study. These views are not necessarily incompatible. In the homunculus view, the motor cortex functions as a rough map of the body’s musculature. In the population-code view, populations of broadly tuned neurons combine to specify hand direction or some other parameter of movement. In the recently proposed action map view, common actions in the movement repertoire are emphasized in different regions of cortex. In the action map view, to fully understand the organization of the motor cortex, it is necessary to study the structure and complexity of the movement repertoire and understand how that statistical structure is mapped onto the cortical surface. This chapter discusses the action map in the primate brain and how some of the complex actions represented there may play a role in social behavior.

We recently proposed the attention schema theory, a novel way to explain the brain basis of subjective awareness in a mechanistic and scientifically testable manner. The theory begins with attention, the process by which signals compete for the brain's limited computing resources. This internal signal competition is partly under a bottom-up influence and partly under top-down control. We propose that the top-down control of attention is improved when the brain has access to a simplified model of attention itself. The brain therefore constructs a schematic model of the process of attention, the 'attention schema,' in much the same way that it constructs a schematic model of the body, the 'body schema.' The content of this internal model leads a brain to conclude that it has a subjective experience. One advantage of this theory is that it explains how awareness and attention can sometimes become dissociated; the brain's internal models are never perfect, and sometimes a model becomes dissociated from the object being modeled. A second advantage of this theory is that it explains how we can be aware of both internal and external events. The brain can apply attention to many types of information including external sensory information and internal information about emotions and cognitive states. If awareness is a model of attention, then this model should pertain to the same domains of information to which attention pertains. A third advantage of this theory is that it provides testable predictions. If awareness is the internal model of attention, used to help control attention, then without awareness, attention should still be possible but should suffer deficits in control. In this article, we review the existing literature on the relationship between attention and awareness, and suggest that at least some of the predictions of the theory are borne out by the evidence.

The human temporoparietal junction (TPJ) is a topic of intense research. Imaging studies have identified TPJ activation in association with many higher-order functions such as theory-of-mind, episodic memory, and attention, causing debate about the distribution of different processes. One major challenge is the lack of consensus about the anatomical location and extent of the TPJ. Here, we address this problem using data-driven analysis to test the hypothesis that the bilateral TPJ can be parcellated into subregions. We applied independent component analysis (ICA) to task-free fMRI data within a local region around the bilateral TPJ, iterating the ICA at multiple model orders and in several datasets. The localized analysis allowed finer separation of processes and the use of multiple dimensionalities provided qualitative information about lateralization. We identified four subdivisions that were bilaterally symmetrical and one that was right biased. To test whether the independent components (ICs) reflected true subdivisions, we performed functional connectivity analysis using the IC coordinates as seeds. This confirmed that the subdivisions belonged to distinct networks. The right-biased IC was connected with a network often associated with attentional processing. One bilateral subdivision was connected to sensorimotor regions and another was connected to auditory regions. One subdivision that presented as distinct left- and right-biased ICs was connected to frontoparietal regions. Another subdivision that also had left- and right-biased ICs was connected to social or default mode networks. Our results show that the TPJ in both hemispheres hosts multiple neural processes with connectivity patterns consistent with well developed specialization and lateralization.

My son and I have a pet orangutan named Kevin. He talks to us almost every day and usually asks for a banana. All right, he’s not a pet, he’s a large hairy puppet, and I make him talk using the trick of ventriloquism. But it’s pretty fun anyway. When the puppet moves his mouth, a squeaky voice seems to come out of him, not me.

Recently we proposed a theory of consciousness, the attention schema theory, based on findings in cognitive psychology and systems neuroscience. In that theory, consciousness is an internal model of attention or an ‘attention schema.’ Consciousness relates to attention the same way that the internal model of the body, the ‘body schema,’ relates to the physical body. The body schema is used to model and help control the body. The attention schema is used to model and help regulate attention, a data-handling process in the brain in which some signals are enhanced at the expense of other signals. We proposed that attention and the attention schema co-evolved over the past half billion years. Over that time span, the attention schema may have taken on additional functions such as promoting the integration of information across diverse domains and promoting social cognition. This article summarizes some of the main points of the attention schema theory, suggests how a brain with an attention schema might conclude that it has a subjective awareness, and speculates that the same basic properties can be engineered into machines.

This study tested the possible relationship between reported visual awareness ("I see a visual stimulus in front of me") and the social attribution of awareness to someone else ("That person is aware of an object next to him"). Subjects were tested in two steps. First, in an fMRI experiment, subjects were asked to attribute states of awareness to a cartoon face. Activity associated with this task was found bilaterally within the temporoparietal junction (TPJ) among other areas. Second, the TPJ was transiently disrupted using single-pulse transcranial magnetic stimulation (TMS). When the TMS was targeted to the same cortical sites that had become active during the social attribution task, the subjects showed symptoms of visual neglect in that their detection of visual stimuli was significantly affected. In control trials, when TMS was targeted to nearby cortical sites that had not become active during the social attribution task, no significant effect on visual detection was found. These results suggest that there may be at least some partial overlap in brain mechanisms that participate in the social attribution of sensory awareness to other people and in attributing sensory awareness to oneself.

The "attention schema" theory provides one possible account of the biological basis of consciousness, tracing the evolution of awareness through steps from the advent of selective signal enhancement about half a billion years ago to the top-down control of attention, to an internal model of attention (which allows a brain, for the first time, to attribute to itself that it has a mind that is aware of something), to the ability to attribute awareness to other beings, and from there to the human attribution of a rich spirit world surrounding us. Humans have been known to attribute awareness to plants, rocks, rivers, empty space, and the universe as a whole. Deities, ghosts, souls--the spirit world swirling around us is arguably the exuberant attribution of awareness.

We proposed a theory of consciousness in which the machinery for social perception constructs awareness, and awareness is a perceptual model of the process of attention. One can attribute awareness to others or to oneself. Awareness of X is the brain's perceptual metaphor for the deep attentive processing of X. A set of ten comments on our hypothesis are included in this issue. Each comment raises specific points some of which directly challenge the hypothesis. Here we respond to these specific points and challenges.

Early in the physiological study of the motor cortex, one experimental question began to dominate the research. How are points in cortex connected to muscles? The question fosters a simplistic, feed-forward view of motor cortex in which its intrinsic processing is ignored and its function is assumed to be defined almost entirely by the cables that run down to the spinal cord, relay onto motor neurons, and thus cause muscle contraction. This perspective still pervades almost all modern thinking about the motor cortex. As a result, a more realistic view of motor cortex as a control network has been hindered. The study by Capaday et al. (2011), examining the lateral interactions among neurons in motor cortex, represents an important step beyond the limited muscle-map conception and toward a better understanding of the processing network within the cortex itself.

A common modern view of consciousness is that it is an emergent property of the brain, perhaps caused by neuronal complexity, and perhaps with no adaptive value. Exactly what emerges, how it emerges, and from what specific neuronal process, is in debate. One possible explanation of consciousness, proposed here, is that it is a construct of the social perceptual machinery. Humans have specialized neuronal machinery that allows us to be socially intelligent. The primary role for this machinery is to construct models of other people's minds thereby gaining some ability to predict the behavior of other individuals. In the present hypothesis, awareness is a perceptual reconstruction of attentional state; and the machinery that computes information about other people's awareness is the same machinery that computes information about our own awareness. The present article brings together a variety of lines of evidence including experiments on the neural basis of social perception, on hemispatial neglect, on the out-of-body experience, on mirror neurons, and on the mechanisms of decision-making, to explore the possibility that awareness is a construct of the social machinery in the brain.

An exciting new experiment on the motor cortex of monkeys, by Shenoy and colleagues, begins to elucidate how the neuronal ensemble travels in a systematic fashion through state space. This trajectory through state space may help to explain how the motor cortex sets up and then triggers arm movements.

How is the macaque monkey extrastriate cortex organized? Is vision divisible into separate tasks, such as object recognition and spatial processing, each emphasized in a different anatomical stream? If so, how many streams exist? What are the hierarchical relationships among areas? The present study approached the organization of the extrastriate cortex in a novel manner. A principled relationship exists between cortical function and cortical topography. Similar functions tend to be located near each other, within the constraints of mapping a highly dimensional space of functions onto the two-dimensional space of the cortex. We used this principle to re-examine the functional organization of the extrastriate cortex given current knowledge about its topographic organization. The goal of the study was to obtain a model of the functional relationships among the visual areas, including the number of functional streams into which they are grouped, the pattern of informational overlap among the streams, and the hierarchical relationships among areas. To test each functional description, we mapped it to a model cortex according to the principle of optimal continuity and assessed whether it accurately reconstructed a version of the extrastriate topography. Of the models tested, the one that best reconstructed the topography included four functional streams rather than two, six levels of hierarchy per stream, and a specific pattern of informational overlap among streams and areas. A specific mixture of functions was predicted for each visual area. This description matched findings in the physiological literature, and provided predictions of functional relationships that have yet to be tested physiologically.

Electrical stimulation of the motor cortex of monkeys elicits complex movements that combine many muscles and joints and that resemble fragments of the animal’s normal movement repertoire. Hand-to-mouth movements, reaching movements, defensive movements, and other ethologically relevant actions can be evoked. Different movements are evoked from different locations in motor cortex. The movement repertoire of the monkey appears to be mapped on the cortical sheet in a manner that preserves local continuity. Simple map schemes that have been proposed in the past, such as a map of the body, or a segregation of the motor cortex into separate areas that process different aspects of movement, explain only some aspects of motor cortex organization. More of the subtlety and complexity of the motor cortex topography can be explained by the principle of a highly dimensional movement repertoire that is flattened onto the cortical surface.

Much of the research on the neuronal basis of prehension focuses on macaque monkeys. Yet most of the behavioral description of grip types pertains to humans and apes. The purpose of the present study was to provide a catalogue and description of basic grip behavior in macaque monkeys. The observational study explored the diversity of grasping behavior in 157 semi-free ranging rhesus macaques. Video footage of monkeys grasping objects ad libitum was analyzed frame-by-frame, and grips were classified based on the skin surface areas that contacted the object. When monkeys held objects for manipulation, 15 distinct grip categories were observed. When monkeys held support points during climbing, two grip categories were observed. Not all grips were performed with the hand. Some involved the mouth, the foot, or an opposition between the forearm and chest. Grip in macaque monkeys is more diverse than the narrow range of grip that is typically studied.

Macroscopic (millimeter scale) functional clustering is a hallmark characteristic of motor cortex spatial organization in awake behaving mammals; however, almost no information is known about the functional micro-organization (approximately 100 microm scale). Here, we optically recorded intracellular calcium transients of layer 2/3 neurons with cellular resolution over approximately 200-microm-diameter fields in the forelimb motor cortex of mobile, head-restrained mice during two distinct movements (running and grooming). We showed that the temporal correlation between neurons was statistically larger the closer the neurons were to each other. We further explored this correlation by using two separate methods to spatially segment the neurons within each imaging field: K-means clustering and correlations between single neuron activity and mouse movements. The two methods segmented the neurons similarly and led to the conclusion that the origin of the inverse relationship between correlation and distance seen statistically was twofold: clusters of highly temporally correlated neurons were often spatially distinct from one another, and (even when the clusters were spatially intermingled) within the clusters, the more correlated the neurons were to each other, the shorter the distance between them. Our results represent a direct observation of functional clustering within the microcircuitry of the awake mouse motor cortex.

A traditional view of the human motor cortex is that it contains an overlapping sequence of body part representations from the tongue in a ventral location to the foot in a dorsal location. In this study, high-resolution functional MRI (1.5x1.5x2 mm) was used to examine the somatotopic map in the lateral motor cortex of humans, to determine whether it followed the traditional somatotopic order or whether it contained any violations of that somatotopic order. The arm and hand representation had a complex organization in which the arm was relatively emphasized in two areas: one dorsal and the other ventral to a region that emphasized the fingers. This violation of a traditional somatotopic order suggests that the motor cortex is not merely a map of the body but is topographically shaped by other influences, perhaps including correlations in the use of body parts in the motor repertoire.

Human subjects practiced navigation in a virtual, computer-generated maze that contained 4 spatial dimensions rather than the usual 3. The subjects were able to learn the spatial geometry of the 4-dimensional maze as measured by their ability to perform path integration, a standard test of spatial ability. They were able to travel down a winding corridor to its end and then point back accurately toward the occluded origin. One interpretation is that the brain substrate for spatial navigation is not a built-in map of the 3-dimensional world. Instead it may be better described as a set of general rules for manipulating spatial information that can be applied with practice to a diversity of spatial frameworks.

A traditional view of the motor cortex in the primate brain is that it contains a map of the body arranged across the cortical surface. This traditional topographic scheme, however, does not capture the actual pattern of overlaps, fractures, re-representations, and multiple areas separated by fuzzy borders. Here, we suggest that the organization of the motor cortex, premotor cortex, supplementary motor cortex, frontal eye field, and supplementary eye field can in principle be understood as a best-fit rendering of the motor repertoire onto the two-dimensional cortical sheet in a manner that optimizes local continuity.

The activity of single neurons in the monkey motor cortex was studied during semi-naturalistic, unstructured arm movements made spontaneously by the monkey and measured with a high resolution three-dimensional tracking system. We asked how much of the total neuronal variance could be explained by various models of neuronal tuning to movement. On average, tuning to the speed of the hand accounted for 1% of the total variance in neuronal activity, tuning to the direction of the hand in space accounted for 8%, a more complex model of direction tuning, in which the preferred direction of the neuron rotated with the starting position of the arm, accounted for 13%, tuning to the final position of the hand in Cartesian space accounted for 22%, and tuning to the final multijoint posture of the arm accounted for 36%. One interpretation is that motor cortex neurons are significantly tuned to many control parameters important in the animal's repertoire, but that different control parameters are represented in different proportion, perhaps reflecting their prominence in everyday action. The final posture of a movement is an especially prominent control parameter although not the only one. A common mode of action of the monkey arm is to maintain a relatively stable overall posture while making local adjustments in direction during performance of a task. One speculation is that neurons in motor cortex reflect this pattern in which direction tuning predominates in local regions of space and postural tuning predominates over the larger workspace.

One way to understand the topography of the cerebral cortex is that "like attracts like." The cortex is organized to maximize nearest neighbor similarity. This principle can explain the separation of the cortex into discrete areas that emphasize different information domains. It can also explain the maps that form within cortical areas. However, because the cortex is two-dimensional, when a parameter space of much higher dimensionality is reduced onto the cortical sheet while optimizing nearest neighbor relationships, the result may lack an obvious global ordering into separate areas. Instead, the topography may consist of partial gradients, fractures, swirls, regions that resemble separate areas in some ways but not others, and in not a lack of topographic maps but an excess of maps overlaid on each other, no one of which seems to be entirely correct. Like a canvas in a gallery of modern art that no two observers interpret the same way, this lack of obvious ordering of high-dimensional spaces onto the cortex might then result in some scientific controversy over the true organization. In this review, the authors suggest that at least some sectors of the cortex do not have a simple global ordering and are better understood as a result of a reduction of a high-dimensional space onto the cortical sheet. The cortical motor system may be an example of this phenomenon. The authors discuss a model of the lateral motor cortex in which a reduction of many parameters onto a simulated cortical sheet results in a complex topographic pattern that matches the actual monkey motor cortex in surprising detail. Some of the ambiguities of topography and areal boundaries that have plagued the attempt to systematize the lateral motor cortex are explained by the model.

Motor cortex in the primate brain was once thought to contain a simple map of the body's muscles. Recent evidence suggests, however, that it operates at a radically more complex level, coordinating behaviorally useful actions. Specific subregions of motor cortex may emphasize different ethologically relevant categories of behavior, such as interactions between the hand and the mouth, reaching motions, or defensive maneuvers to protect the body surface from impending impact. Single neurons in motor cortex may contribute to these behaviors by means of their broad tuning to idiosyncratic, multijoint actions. The mapping from cortex to muscles is not fixed, as was once thought, but instead is fluid, changing continuously on the basis of feedback in a manner that could support the control of higher-order movement parameters. These findings suggest that the motor cortex participates directly in organizing and controlling the animal's behavioral repertoire.

In the monkey brain, two interconnected cortical areas have distinctive neuronal responses to visual, tactile, and auditory stimuli. These areas are the ventral intraparietal area (VIP) and a polysensory zone in the precentral gyrus (PZ). The multimodal neurons in these areas typically respond to objects touching, near, or looming toward the body surface. Electrical stimulation of these areas evokes defensive-like withdrawing or blocking movements. These areas have been suggested to participate in a range of functions including navigation by optic flow, attention to nearby space, and the processing of object location for the guidance of movement. We suggest that a major emphasis of these areas is the construction of a margin of safety around the body and the selection and coordination of defensive behavior. In this review, we summarize the physiological properties of these brain areas and discuss a range of behavioral phenomena that might be served by those neuronal properties, including the ducking and blocking reactions that follow startle, the flight zone of animals, the personal space of humans, the nearby, multimodal attentional space that has been studied in humans, the withdrawal reaction to looming visual stimuli, and the avoidance of obstacles during self-motion such as locomotion or reaching.

Motor cortex neurons in the monkey brain were tested with a diverse and naturalistic arm movement set. Over this global set of movements, the neurons showed a limited but significant degree of tuning to the multijoint posture attained by the arm at the end of each movement. Further supporting the hypothesis that the neurons are partially tuned to end posture, the postures preferred by the neurons significantly matched the postures evoked by electrical stimulation of the same cortical sites. However, much of the variance in neuronal activity remained unexplained even by the end-posture model, and thus other variables must have contributed to the response profile of the neurons. One possibility is that motor cortex neurons become tuned to the wide variety of movement parameters that are relevant to the animal's normal behavioral repertoire, and, therefore, any one parameter accounts for only a limited amount of neuronal variance.

We propose that some of the features of the topographic organization in motor cortex emerge from a competition among several conflicting mapping requisites. These competing requisites include a somatotopic map of the body, a map of hand location in space, and a partitioning of cortex into regions that emphasize different complex, ethologically relevant movements. No one type of map fully explains the topography; instead, all three influences (and perhaps others untested here) interact to form the topography. A standard algorithm (Kohonen network) was used to generate an artificial motor cortex array that optimized local continuity for these conflicting mapping requisites. The resultant hybrid map contained many features seen in actual motor cortex, including the following: a rough, overlapping somatotopy; a posterior strip in which simpler movements were represented and more somatotopic segregation was observed, and an anterior strip in which more complex, multisegmental movements were represented and the somatotopy was less segregated; a clustering of different complex, multisegmental movements into specific subregions of cortex that resembled the arrangement of subregions found in the monkey; three hand representations arranged on the cortex in a manner similar to the primary motor, dorsal premotor, and ventral premotor hand areas in the monkey; and maps of hand location that approximately matched the maps observed in the monkey.

A new study in this issue of Neuron shows that when monkeys reach to a visual target, neurons in the dorsal premotor cortex compare the location of the target, the hand, and the point of visual fixation. The neurons therefore encode space through a combination of eye-centered and hand-centered coordinates.

We used electrical microstimulation to study the organization of motor cortex in awake monkeys. Stimulation on a behaviorally relevant time scale (0.5-1 s) evoked coordinated, complex postures that involved many joints. Postures that involved the arm were arranged across cortex to form a map of hand positions around the body. This map encompassed both primary motor and lateral premotor cortex. Primary motor cortex appeared to represent the central part of the workspace, where monkeys most often manipulate objects with their fingers. These findings suggest that primary motor and lateral premotor cortex might not be arranged in a hierarchy, but instead might operate in parallel, serving different behavioral functions in different parts of the workspace. This hypothesis is also consistent with some of the previous data from motor and premotor cortex.

Electrical stimulation of the motor cortex in monkeys can evoke complex, multijoint movements including movements of the arm and hand. In this study, we examined these movements in detail and tested whether they showed adaptability to differing circumstances such as to a weight added to the hand. Electrical microstimulation was applied to motor cortex using pulse trains of 500-ms duration (matching the approximate duration of a reach). Arm movement was measured using a high-resolution three-dimensional tracking system. Movement latencies averaged 80.2 ms. Speed profiles were typically smooth and bell-shaped, and the peak speed covaried with movement distance. Stimulation generally evoked a specific final hand position. The convergence of the hand from disparate starting positions to a narrow range of final positions was statistically significant for every site tested (91/91). When a weight was fixed to the hand, for some stimulation sites (74%), the evoked movement appeared to compensate for the weight in that the hand was lifted to a similar final location. For other stimulation sites (26%), the weight caused a significant reduction in final hand height. For about one-half of the sites (54%), the variation in movement of each joint appeared to compensate for the variation in the other joints in a manner that stabilized the hand in a restricted region of space. These findings suggest that at least some of the stimulation-evoked movements reflect relatively high-level, adaptable motor plans.

If a hornet flies toward your face, you might duck, squint, and lift your hand to block it. If the insect touches your hand, you might withdraw your hand, even pulling it behind your back. These defensive movements have a reflexive quality. They are fast and can occur without conscious planning or thought. They are similar in all people (see Figure 1). However, although they seem reflexive, defensive movements are also highly sophisticated. They can be elicited by touch, sight or sound. They involve coordination between different body parts, such as the arm and head. They are spatially specific: the body parts that move and the direction of movement are appropriate for the location of the threat. The movements can be stronger or weaker depending on external context or the internal state of the person. For example, someone whose “nerves are on edge” may give an exaggerated alerting response to an unexpected stimulus. What sensory-motor pathways in the brain coordinate this rich and complex behavior? We suggest that a special set of interconnected areas in the monkey brain monitors the location and movement of objects near the body and controls startle, flinch and defensive responses. This hypothesized “defensive” system, shown in Figure 2, includes the ventral intraparietal area (VIP), parietal area 7b, the polysensory zone (PZ) in the precentral gyrus, and the putamen. These brain areas are monosynaptically interconnected (Cavada and Goldman-Rakic 1989a,b; Cavada and Goldman-Rakic 1991; Kunzle 1978; Matelli et al. 1986; Mesulam et al. 1977; Parthasarathy et al. 1992; Weber and Yin 1984; Luppino et al., 1999). Of the four areas, PZ is closest to the motor output, sending direct projections to the spinal cord (Dum and Strick 1991). Electrical stimulation of PZ evokes defensive movements, such as withdrawal of the hand, squinting and turning of the head, ducking, or lifting the hand as if to defend the side of the head (Graziano, Taylor and Moore 2002).

Recently it was shown that electrical stimulation of the precentral gyrus of monkeys can evoke complex, coordinated movements. In the forelimb representation, stimulation of each site caused the arm to move to a specific final posture, and thus the hand to move to a location in space. Among these stimulation-evoked hand locations, certain regions of the hand's workspace were more represented than others. We hypothesized that a similar non-uniform distribution of hand location should be present during a monkey's spontaneous behavior. The present study examined the distribution of hand location of monkeys in their home cages. This distribution was similar to that found by stimulation of the precentral gyrus. That is, arm postures that were over-represented in spontaneous behavior were also over-represented in the movements evoked by cortical stimulation.

This experiment used cortical microstimulation to probe the mapping from primary motor cortex to the biceps and triceps muscles of the arm in monkeys. The mapping appeared to change depending on the angle at which the elbow was fixed. For sites in the dorsal part of the arm and hand representation, the effects of stimulation were consistent with initiating a movement of the elbow to an extended angle. Stimulation evoked more triceps activity than biceps activity, and this difference was largest when the elbow was fixed in a flexed angle. For sites in the ventral part of the arm and hand representation, stimulation had the opposite effect, consistent with initiating a movement of the elbow to a flexed angle. For these sites, stimulation evoked more biceps activity than triceps activity, and the difference was largest when the elbow was fixed in an extended angle. For sites located in intermediate positions, stimulation evoked an intermediate effect consistent with initiating a movement of the elbow to a middle, partially flexed angle. For these sites, when the elbow was fixed at a flexed angle, the evoked activity was largest in the triceps, and when the elbow was fixed at an extended angle, the evoked activity was largest in the biceps. These effects were obtained with 400-ms-long trains of biphasic pulses presented at 200 Hz and 30 microA. They were also obtained by averaging the effects of individual, 30-microA pulses presented at 15 Hz. How this stimulation-evoked topography relates to the normal function of motor cortex is not yet clear. One hypothesis is that these results reflect a cortical map of desired joint angle.

The precentral gyrus of monkeys contains a polysensory zone in which the neurons respond to tactile, visual, and sometimes auditory stimuli. The tactile receptive fields of the polysensory neurons are usually on the face, arms, or upper torso, and the visual and auditory receptive fields are usually confined to the space near the tactile receptive fields, within about 30 cm of the body. Electrical stimulation of this polysensory zone, even in anesthetized animals, evokes a specific set of movements. The movements resemble those typically used to defend the body from objects that are near, approaching, or touching the skin. In the present study, to determine whether the stimulation-evoked movements represent a normal set of defensive movements, we tested whether they include a distinctive, nonsaccadic, centering movement of the eyes that occurs during defensive reactions. We report that this centering movement of the eyes is evoked by stimulation of sites in the polysensory zone. We also recorded the activity of neurons in the polysensory zone while the monkey made defensive reactions to an air puff on the face. The neurons became active during the defensive movement, and the magnitude of this activity was correlated with the magnitude of the defensive reaction. These results support the hypothesis that the polysensory zone in the precentral gyrus contributes to the control of defensive movements. More generally, the results support the view that the precentral gyrus can control movement at the level of complex sensorimotor tasks.

In a restricted zone of the monkey motor cortex, neurons respond to objects near, approaching, or touching the body. This polysensory zone was hypothesized to play a role in monitoring nearby stimuli for the guidance of defensive movements. To test this hypothesis, we chemically manipulated sites within that zone by injecting bicuculline (increasing neuronal activity) or muscimol (decreasing neuronal activity). Bicuculline caused the monkey to react in an exaggerated fashion to an air puff on the face and to objects approaching the face, whereas muscimol caused the monkey to react in a reduced fashion. The effects were expressed partly as a motor abnormality (affecting movement of the musculature contralateral to the injection site) but also partly as a sensory enhancement or sensory neglect (affecting responses to stimuli contralateral to the injection site). These findings suggest that the polysensory zone contributes to the ethologically important function of defense of the body.

Most neurons in the ventral intraparietal area (VIP) of the macaque brain respond to both visual and tactile stimuli. The tactile receptive field is usually on the face, and the visual receptive field usually corresponds spatially to the tactile receptive field. In this study, electrical microstimulation of VIP, but not of surrounding tissue, caused a constellation of movements including eye closure, facial grimacing, head withdrawal, elevation of the shoulder, and movements of the hand to the space beside the head or shoulder. A similar set of movements was evoked by an air puff to the monkey's cheek. One interpretation is that VIP contributes to defensive movements triggered by stimuli on or near the head.

Electrical stimulation of two connected cortical areas in the monkey brain, the ventral intraparietal area (VIP) in the intraparietal sulcus and the polysensory zone (PZ) in the precentral gyrus, evokes a specific set of movements. In one interpretation, these movements correspond to those typically used to defend the body from objects that are near, approaching, or touching the skin. The present study examined the movements evoked by a puff of air aimed at various locations on the face and body of fascicularis monkeys to compare them to the movements evoked by stimulation of VIP and PZ. The air-puff-evoked movements included a movement of the eyes from any initial position toward a central region and a variety of stereotyped facial, shoulder, head, and arm movements. These movements were similar to those reported on stimulation of VIP and PZ. One difference between the air-puff-evoked movements and those evoked by stimulation of VIP and PZ is that the air puff evoked an initial startle response (a bilaterally symmetric spike in muscle activity) followed by a more sustained, lateralized response, specific to the site of the air puff. In contrast, stimulation of VIP and PZ evoked mainly a sustained, lateralized response, specific to the site of the receptive fields of the stimulated neurons. We speculate that VIP and PZ may contribute to the control of defensive movements, but that they may emphasize the more spatially specific reactions that occur after startle.

To reach for the computer mouse, sit upright in a chair or hold a journal in order to read it, indeed, to do most of the actions that we commonly perform, we rely on a representation of the spatial configuration of the body. How and where in the brain is the body represented and what are the psychological properties of this body schema? In this article we review first the neurophysiology and then the psychology of the body representation. One finding that has emerged from both approaches is that the body representation is not merely a registration of proprioceptive inputs about joint angle. Instead, the brain contains a sophisticated model of the body that is continually updated on the basis of multimodal input including vision, somesthesis and motor feedback. Neurophysiological studies in the monkey brain show that parietal area 5 is a critical node for processing the body’s configuration. Neurons in area 5 combine signals from different modalities in order to represent limb position and movement. Psychological studies show that the body schema is used to cross-reference between different senses, as a basis for spatial cognition and for movement planning.

In a News and Views piece (“Stimulating research on Motor Cortex” 2002, 5:714), Strick comments on our recent finding that microstimulation of motor cortex evokes complex, coordinated behavior (1).  A major concern that he raises is that, “one might ask whether electrical stimulation of the cortex is capable of revealing its function.”  We agree that one should always ask such questions about all experimental methods.  However, a large body of recent work, conspicuously not cited in Strick’s piece, successfully probes cortical function using electrical stimulation.  For example, Newsome and colleagues (2) stimulated monkey visual area MT and influenced the monkey’s perceptual decisions about the direction of motion of visual stimuli.  Romo and colleagues (3) stimulated primary somatosensory cortex and influenced the monkey’s perceptual decisions about tactile stimuli. Shadlen and colleagues (4) stimulated the frontal eye fields and influenced the monkey’s target selection.  Many researchers have used electrical stimulation to study functional maps of eye and head movement (5-7).  We took the well-established protocol of stimulating on a behaviorally relevant time scale and applied it to motor cortex.  The stimulation durations that we used are within the range of these previous studies, and the current intensities are within the range used in the oculomotor studies.  As in previous studies, we evoked meaningful behaviors.

Neurons in a restricted zone in the precentral gyrus of macaque monkeys respond to tactile, visual, and auditory stimuli. The tactile receptive fields of these multimodal cells are usually located on the face, arm, or upper torso. In the present study, in awake monkeys sitting in a primate chair, the neurons responded to a tactile probe touching the skin within the tactile receptive field. However, the same neurons did not respond when the tactile receptive field was touched by the primate chair, to which the monkey was habituated.

Electrical microstimulation was used to study primary motor and premotor cortex in monkeys. Each stimulation train was 500 ms in duration, approximating the time scale of normal reaching and grasping movements and the time scale of the neuronal activity that normally accompanies movement. This stimulation on a behaviorally relevant time scale evoked coordinated, complex postures that involved many joints. For example, stimulation of one site caused the mouth to open and also caused the hand to shape into a grip posture and move to the mouth. Stimulation of this site always drove the joints toward this final posture, regardless of the direction of movement required to reach the posture. Stimulation of other cortical sites evoked different postures. Postures that involved the arm were arranged across cortex to form a map of hand positions around the body. This stimulation-evoked map encompassed both primary motor and the adjacent premotor cortex. We suggest that these regions fit together into a single map of the workspace around the body.

Recently, we found that electrical stimulation of motor cortex caused monkeys to make coordinated, complex movements. These evoked movements were arranged across the cortex in a map of spatial locations to which the hand moved. We suggest that some of the subdivisions previously described within primary motor and premotor cortex may represent different types of actions that monkeys tend to make in different regions of space. According to this view, primary and premotor cortex may fit together into a larger map of manual space.

Damage to restricted parts of the brain can cause spatial confusion and even eliminate awareness of large parts of space around the body.  The precise brain areas responsible for spatial awareness, however, are still in debate.

There are currently three main views on the neural basis of visually guided reaching: 1) neurons in the superior parietal lobe guide arm movements in a spatial framework that is centered on the body; 2) neurons in the intraparietal sulcus guide arm movements in a spatial framework that is centered on the eye; 3) neurons in the caudal part of premotor cortex guide arm movements in a spatial framework that is centered on the arm and hand. The three viewpoints are mutually compatible and may fit into a larger pattern. Eye-centered representations of target position, and body-centered representations of arm and hand position, may be integrated to form a hand-centered representation close to the output stage in caudal premotor and primary motor cortex.

The primate cerebral cortex has traditionally been divided into separate territories for vision, touch, audition and movement.  These functions are known to overlap in many parts of cortex, but until recently the regions of overlap were not well studied.  In this issue of Neuron, Bremmer et al. report a major advance in understanding at least one set of areas in the human brain in which the senses are integrated.  This finding joins a growing set of work in monkeys and humans on the integration of the senses with each other and with the control of movement.

In 1870, Fritsch and Hitzig first studied primary motor cortex in the monkey brain using electrical stimulation, and in 1881, Hermann Munk used lesion methods to localize the primary visual cortex in the occipital lobe (cited in Gross 1998). Only now, more than one hundred years later, has neuroscience begun to identify the neuronal pathways that connect these two areas. We are finally beginning to understand the routes through which vision is transformed into action.

Area 5 in the parietal lobe of the primate brain is thought to be involved in monitoring the posture and movement of the body. In this study, neurons in monkey area 5 were found to encode the position of the monkey's arm while it was covered from view. The same neurons also responded to the position of a visible, realistic false arm. The neurons were not sensitive to the sight of unrealistic substitutes for the arm and were able to distinguish a right from a left arm. These neurons appear to combine visual and somatosensory signals in order to monitor the configuration of the limbs. They could form the basis of the complex body schema that we constantly use to adjust posture and guide movement.

Neurons in the premotor cortex of macaques respond to tactile, visual and auditory stimuli. The distribution of these responses was studied in five anesthetized monkeys. In each monkey, multiunit activity was studied at a grid of locations across the precentral gyrus. A cluster of sites with polysensory responses was found posterior to the genu of the arcuate sulcus. Tactile and visual responses were represented in all five monkeys, while auditory responses were rarer and found in only two monkeys. This polysensory zone (PZ) was located in the caudal part of premotor cortex. It varied in extent among the monkeys. It was mainly ventral to the genu of the arcuate, in the dorsal and caudal part of the ventral premotor cortex (PMv). In some monkeys it extended more dorsally, into the caudal part of dorsal premotor cortex (PMd). Sensory responses were almost never found in the rostral part of PMd. We suggest that the polysensory zone may contribute to the guidance of movement on the basis of tactile, visual and auditory signals.

Area V6A, on the anterior bank of the parieto-occipital sulcus of the monkey brain, contains neurons sensitive both to visual stimulation and to the position and movement of the eyes. We examined the effects of eye position and eye movement on the activity of V6A neurons in monkeys trained to saccade to and fixate on target locations. Forty-eight percent of the neurons responded during these tasks. The responses were not caused by the visual stimulation of the fixation light because extinguishing the fixation light had no effect. Instead the neurons responded in relation to the position of the eye during fixation. Some neurons preferred a restricted range of eye positions, whereas others had more complex and distributed eye-position fields. None of these eye-related neurons responded before or during saccades. They all responded postsaccadically during fixation on the target location. However, the neurons did not simply encode the static position of the eyes. Instead most (88%) responded best after the eye saccaded into the eye-position field and responded significantly less well when the eye made a saccade that was entirely contained within the eye-position field. Furthermore, for many eye-position cells (45%), the response was greatest immediately after the eye reached the preferred position and was significantly reduced after 500 ms of fixation. Thus these neurons preferentially encoded the initial arrival of the eye into the eye-position field rather than the continued presence or the movement of the eye within the eye-position field. Area V6A therefore contains a representation of the position of the eye in the orbit, but this representation appears to be dynamic, emphasizing the arrival of the eye at a new position.

In primates, prefrontal, inferior temporal, and posterior parietal cortex are important for cognitive function. It is shown that in adult macaques, new neurons are added to these three neocortical association areas, but not to a primary sensory area (striate cortex). The new neurons appeared to originate in the subventricular zone and to migrate through the white matter to the neocortex, where they extended axons. These new neurons, which are continually added in adulthood, may play a role in the functions of association neocortex.

Humans can accurately perceive the location of a sound source-not only the direction, but also the distance. Sounds near the head, within ducking or reaching distance, have a special saliency. However, little is known about this perception of auditory distance. The direction to a sound source can be determined by interaural differences, and the mechanisms of direction perception have been studied intensively; but except for studies on echolocation in the bat, little is known about how neurons encode information on auditory distance. Here we describe neurons in the brain of macaque monkeys (Macaca fascicularis) that represent the auditory space surrounding the head, within roughly 30 cm. These neurons, which are located in the ventral premotor cortex, have spatial receptive fields that extend a limited distance outward from the head.

A central problem in motor control, in the representation of space, and in the perception of body schema is how the brain encodes the relative positions of body parts. According to psychophysical studies, this sense of limb position depends heavily on vision. However, almost nothing is currently known about how the brain uses vision to determine or represent the location of the arm or any other body part. The present experiment shows that the position of the arm is represented in the premotor cortex of the monkey (Macaca fascicularis) brain by means of a convergence of visual cues and proprioceptive cues onto the same neurons. These neurons respond to the felt position of the arm when the arm is covered from view. They also respond in a similar fashion to the seen position of a false arm.

Neurons in the ventral premotor cortex of the monkey encode the locations of visual, tactile, auditory and remembered stimuli. Some of these neurons encode the locations of stimuli with respect to the arm, and may be useful for guiding movements of the arm. Others encode the locations of stimuli with respect to the head, and may be useful for guiding movements of the head. We suggest that a general principle of sensory-motor integration is that the space surrounding the body is represented in body-part-centered coordinates. That is, there are multiple coordinate systems used to guide movement, each one attached to a different part of the body. This and other recent evidence from both monkeys and humans suggest that the formation of spatial maps in the brain and the guidance of limb and body movements do not proceed in separate stages but are closely integrated in both the parietal and frontal lobes.

The ventral premotor cortex (PMv) of the macaque monkey contains neurons that respond both to visual and to tactile stimuli. For almost all of these "bimodal" cells, the visual receptive field is anchored to the tactile receptive field on the head or the arms, and remains stationary when the eyes fixate different locations. This study compared the responses of bimodal PMv neurons to a visual stimulus when the monkey was required to fixate a spot of light and when no fixation was required. Even when the monkey was not fixating and the eyes were moving, the visual receptive fields remained in the same location, near the associated tactile receptive field. For many of the neurons, the response to the visual stimulus was significantly larger when the monkey was not performing the fixation task. In control tests, the presence or absence of the fixation spot itself had little or no effect on the response to the visual stimulus. These results show that even when the monkey's eye position is continuously changing, the neurons in PMv have visual receptive fields that are stable and fixed to the relevant body part. The reduction in response during fixation may reflect a shift of attention from the visual stimulus to the demands of the fixation task.

We find it effortless to reach toward or avoid nearby objects.  However, the spatial and visuo-motor computations must be quite complicated, especially since our eyes, head, limbs, body, and the objects themselves may be continually changing positions. How does the brain construct a representation of the visual space surrounding the body, and how does this representation guide movement?

The ventral premotor cortex in primates is thought to be involved in sensory-motor integration. Many of its neurons respond to visual stimuli in the space near the arms or face. In this study on the ventral premotor cortex of monkeys, an object was presented within the visual receptive fields of individual neurons, then the lights were turned off and the object was silently removed. A subset of the neurons continued to respond in the dark as if the object were still present and visible. Such cells exhibit "object permanence," encoding the presence of an object that is no longer visible. These cells may underlie the ability to reach toward or avoid objects that are no longer directly visible.

In macaque ventral premotor cortex, we recorded the activity of neurons that responded to both visual and tactile stimuli. For these bimodal cells, the visual receptive field extended from the tactile receptive field into the adjacent space. Their tactile receptive fields were organized topographically, with the arms represented medially, the face represented in the middle, and the inside of the mouth represented laterally. For many neurons, both the visual and tactile responses were directionally selective, although many neurons also responded to stationary stimuli. In the awake monkeys, for 70% of bimodal neurons with a tactile response on the arm, the visual receptive field moved when the arm was moved. In contrast, for 0% the visual receptive field moved when the eye or head moved. Thus the visual receptive fields of most "arm + visual" cells were anchored to the arm, not to the eye or head. In the anesthetized monkey, the effect of arm position was similar. For 95% of bimodal neurons with a tactile response on the face, the visual receptive field moved as the head was rotated. In contrast, for 15% the visual receptive field moved with the eye and for 0% it moved with the arm. Thus the visual receptive fields of most "face + visual" cells were anchored to the head, not to the eye or arm. To construct a visual receptive field anchored to the arm, it is necessary to integrate the position of the arm, head, and eye. For arm + visual cells, the spontaneous activity, the magnitude of the visual response, and sometimes both were modulated by the position of the arm (37%), the head (75%), and the eye (58%). In contrast, to construct a visual receptive field that is anchored to the head, it is necessary to use the position of the eye, but not of the head or the arm. For face + visual cells, the spontaneous activity and/or response magnitude was modulated by the position of the eyes (88%), but not of the head or the arm (0%). Visual receptive fields anchored to the arm can encode stimulus location in "arm-centered" coordinates, and would be useful for guiding arm movements. Visual receptive fields anchored to the head can likewise encode stimuli in "head-centered" coordinates, useful for guiding head movements. Sixty-three percent of face + visual neurons responded during voluntary movements of the head. We suggest that "body-part-centered" coordinates provide a general solution to a problem of sensory-motor integration: sensory stimuli are located in a coordinate system anchored to a particular body part.

In the macaque, neurons in ventral premotor cortex and in the putamen have tactile receptive fields on the face or arms, and corresponding visual receptive fields which extend outward from the tactile fields into the space near the body.  For cells with tactile receptive fields on the arm, when the arm is moved, the corresponding visual receptive fields move with it.  However, when the eyes move, the visual receptive fields remain stationary, “attached” to the arm.  We suggest that these “arm-centered" visual responses play a role in visuo-motor guidance.  We predict that other portions of the somatotopic map in premotor cortex and the putamen contain similar receptive fields, centered on the corresponding body parts.  This "body-part-centered" representation of space is only one of several ways in which space is represented in the brain.

How are we able to reach accurately toward objects near us and avoid ones that are approaching, even though the objects and our own eyes, head, limbs and body may be continually changing positions? How does the brain construct a representation of the visual space surrounding the body, and how does this representation guide movement?

Lesions of posterior parietal cortex have long been known to produce visuo-spatial deficits in both humans and monkeys. Yet there is no known “map” of space in parietal cortex. Posterior parietal cortex projects to a number of other areas which are involved in specialized spatial functions. In these areas space is represented at the level of single neurons and in many of them there is a topographically organized map of space. These extra-parietal areas include premotor cortex and the putamen, involved in visuomotor space, the frontal eye fields and the superior colliculus, involved in oculomotor space, the hippocampus, involved in environmental space, and dorsolateral prefrontal cortex, involved in mnemonic space. In many of these areas, space is represented by means of a coordinate system that is fixed to a particular body part. Thus, the processing of space is not unitary, but is divided among several brain areas and several coordinate systems in addition to those in posterior parietal cortex.

We propose that extrapersonal space is represented in the brain by bimodal, visual-tactile neurons in: 1) inferior area 6 in the frontal lobe, 2) area 7b in the parietal lobe, and 3) the putamen.  In each of these areas, there are cells which respond both to tactile and visual stimuli.  In each area, the tactile receptive fields are arranged to form a somatotopic map.  The visual receptive fields are usually adjacent to the tactile ones and extend outward from the skin about 20 cm.  Thus each area contains a somatotopically organized map of the visual space that immediately surrounds the body.  These three areas are monosynaptically interconnected, and may form a distributed system for representing extrapersonal visual space.  For many neurons with tactile receptive fields on the arm or hand, when the arm was moved, the visual receptive field moved with it.  Thus, these neurons appear to code the location of visual stimuli in arm centered coordinates.  More generally, we suggest that the bimodal cells represent near, extrapersonal space in a body part centered fashion, rather than in an exclusively head centered or trunk centered fashion.

Cells in the dorsal division of the medial superior temporal area (MSTd) have large receptive fields and respond to expansion/contraction, rotation, and translation motions. These same motions are generated as we move through the environment, leading investigators to suggest that area MSTd analyzes the optical flow. One influential idea suggests that navigation is achieved by decomposing the optical flow into the separate and discrete channels mentioned above, i.e. expansion/contraction, rotation, and translation. We directly tested whether MSTd neurons perform such a decomposition by examining whether there are cells which are preferentially tuned to intermediate spiral motions, which combine both expansion/contraction and rotation components. The finding that many cells in MSTd are preferentially selective for spiral motions indicates that this simple 3 channel decomposition hypothesis for MSTd is incorrect. Instead, there appears to be a continuum of patterns to which MSTd cells are selective. In addition, we find that MSTd cells maintain their selectivity when stimuli are moved to different locations in their large receptive fields. This position invariance indicates that MSTd cells selective for expansion cannot give precise information about the retinal location of the focus of expansion. Thus individual MSTd neurons cannot code the direction of heading by using the location of the focus of expansion. The only way this navigational information could be derived from MSTd is through the use of a coarse, population encoding. Positional invariance and selectivity for a wide array of stimuli suggest that MSTd neurons encode patterns of motion per se, regardless of whether these motions are generated by moving objects or by motion induced by observer locomotion.

The left fielder squints at the baseball as it curves toward him.  He adjusts his hand and body, and the ball lands in his mitt.  Somehow, the changing pattern of light on his retina was transformed into a motor command which  brought his hand to the correct location for catching the ball.  How was this accomplished?  Is there a map of visual space in the brain which encodes the location of the ball and the fielder's glove?  In this article, we review some recent experiments using monkeys, on visuo-motor transformations in the brain.  We ask how neurons represent the location of a stimulus for the purposes of looking at it, reaching toward it, or avoiding it.

In primates, the premotor cortex is involved in the sensory guidance of movement. Many neurons in ventral premotor cortex respond to visual stimuli in the space adjacent to the hand or arm. These visual receptive fields were found to move when the arm moved but not when the eye moved; that is, they are in arm-centered, not retinocentric, coordinates. Thus, they provide a representation of space near the body that may be useful for the visual control of reaching.

Two monkeys were trained on an auditory-visual (AV) delayed matching-to-sample (DMS) task with auditory cues serving as sample stimuli and visual cues serving as comparison stimuli. To determine whether the monkeys were remembering auditory or visual information during the delay period, auditory and visual interference were presented following the sample stimulus. Auditory interference had little effect on AV DMS performance. In contrast, visual interference severely impaired AV DMS performance, indicating that the monkeys were remembering visual information during the delay period. This finding may reflect a predisposition of monkeys toward remembering information via their dominant visual modality.

The macaque putamen contains neurons that respond to somatosensory stimuli such as light touch, joint movement, or deep muscle pressure. Their receptive fields are arranged to form a map of the body. In the face and arm region of this somatotopic map we found neurons that responded to visual stimuli. Some neurons were bimodal, responding to both visual and somatosensory stimuli, while others were purely visual, or purely somatosensory. The bimodal neurons usually responded to light cutaneous stimulation, rather than to joint movement or deep muscle pressure. They responded to visual stimuli near their tactile receptive field and were not selective for the shape or the color of the stimuli. For cells with tactile receptive fields on the face, the visual receptive field subtended a solid angle extending from the tactile receptive field to about 10 cm. For cells with tactile receptive fields on the arm, the visual receptive field often extended further from the animal. These bimodal properties provide a map of the visual space that immediately surrounds the monkey. The map is organized somatotopically, that is, by body part, rather than retinotopically as in most visual areas. It could function to guide movements in the animal's immediate vicinity. Cortical areas 6, 7b, and VIP contain bimodal cells with very similar properties to those in the putamen. We suggest that the bimodal cells in area 6, 7b, VIP, and the putamen form part of an interconnected system that represents extra personal space in a somatotopic fashion.

Dr. Stein begins by claiming "very little evidence has been found for the existence of a topographic map of perceptual space" and ends by stating that "indeed there is no evidence for a region...where egocentric space is represented topographically."  In lieu of a map, he offers us a "neural network," that is,  "a distributed system of rules for information processing that can be used to transform signals from one coordinate system to another."  Such a computational scheme might indeed work; however, it is quite unnecessary since a neuronal topographic map of visual space does exist, at least for the region adjacent to the body, i.e. immediate extrapersonal space.  As there is good evidence for more than one such map in the primate brain the question would seem to be what are their different functions rather than how can we erect a computational network to do without them.

Recent work on the visual system of primates has delineated several cortical fields involved in the processing of visual motion. These cortical areas appear to be connected anatomically in stages, which suggests that there is a hierarchy in the machinery for motion perception. In this paper, we outline experiments that we have performed along the most prominent pathway for motion analysis, which begins in area V1 and proceeds through the middle temporal area (MT) to the medial superior temporal area (MST).