 |
Information Processing in the Visual PathwayStanley B. Garrett
|
|
|
The global goal of our research program is to understand how information about the outside world is encoded in the neuronal activity in the central nervous system. The effects of a continuous exogenous input (the visual world) are manifest through discrete electrical events in the early visual pathway. The basic anatomy is shown in Figure 1, where light entering the eye falls onto the photoreceptors of the retina and is transduced into electrical signals that are processed through layers in the retina, and then passed through the lateral geniculate nucleus (LGN) to the visual cortex. Individual neurons in each of these areas respond to light within a restricted region of visual space, known as the receptive field (RF). More generally, we refer to the spatiotemporal RF as the characteristics of the integration of visual input over space and time, giving rise to the neuronal response.
Figure 1. The early mammalian visual pathway. It
is important to quantify the manner in which information is encoded in
these stages of the early visual pathway, so that we may, in turn,
precisely control neural function to produce desired visual percepts,
in situations where function has been lost to trauma or disease. Note
that current attempts at visual prosthetics are still in the nascent
stages, and there are major signal processing, estimation, prediction,
and control-related problems that must be overcome before such
technology becomes truly viable. Despite the fact that the anatomy and
physiology of the visual pathway have been studied for some time, much
is still not known about the true nature of the neural code that
enables us to interact with the external visual world. Encoding of Natural Scenes.
Figure 2.
Reconstruction of natural scenes from LGN activity. Left shows actual
(black) and reconstructed (magenta) light intensity of 4 adjacent
pixels (0.2 degrees/pixel), from 8 LGN neurons. Right shows actual
(top) and reconstructed (bottom) frames of natural scene movie (approx
6 degrees), from 177 LGN neurons. From Stanley et al. (1999). Taking this perspective, we decoded or reconstructed natural visual inputs from population activity in the visual thalamus (which is an intermediate stage of processing between the retina and cortex) (Stanley et al., 1999), providing a description of the information about raw light intensity being encoded in specific cell types within the early pathway. Figure 1 shows the reconstructed light intensity of 4 adjacent pixels from 8 LGN neurons on the left, and a reconstruction of a much larger region of visual space on the right (from Stanley et al., 1999; PDF) This work was critical in defining our direction of research for the next several years. The large majority of experimental work on the functional aspects of coding in the visual pathway has utilized artificial classes of visual stimuli. To understand the true functionality of the visual pathway and thus to make long term clinical impact, it is absolutely imperative that we explore more behaviorally and practically relevant scenarios. Despite the surprising amount of detail that was extracted from the ensemble neuronal activity in the LGN, as shown in Figure 2, there are many unresolved issues. Specifically, when studying neural encoding in the natural environment, questions are raised concerning the role of adaptation in the transient natural environment, the effects of spatial and temporal correlation structure on neural coding, the relationship between functional encoding properties and the information carrying properties of the pathway, and the role of the early visual pathway in the detection of salient features in the environment. Figure 3 shows examples of the differences between the evolution of the light intensity at a point in visual space for natural vs. artificial (white noise) visual stimuli, along with the corresponding second order statistics (power spectra) in both temporal and spatial domains.
Figure 3. White noise and natural scene stimuli. a. Frames of spatiotemporal binary noise (left), natural scenes from the woods (middle), and a Hollywood movie (right). b. Light intensity over time for a single spatial location within each movie. c. Power spectral density (PSD) as a function of temporal frequency for white noise (black) and natural scenes (gray). d. PSD as a function of spatial frequency. From Lesica & Stanley (2004). The overall perspective that we have taken in this work is that the early visual pathway has (at least) two distinct roles in processing of information about the outside world: transmission and detection, as shown in Figure 4.
Figure 4. The early visual pathway serves to detect changes in the outside visual world, and to transmit fine details about relevant features. In certain contexts, it is important for the visual pathway to transmit fine details concerning features of the outside visual world. We may think of this as the what question: given that something of interest is in my visual field, what is it? Is it predator or prey, or perhaps a potential mate? In other ethologically relevant contexts, it is important to detect the presence of an object or to signal novelty, in a "bottom-up" context, potentially for the "top-down" allocation of attentional resources. We may think of this as a yes or no question: is something of interest there or not? The fast and robust detection of changes in the external environment may be critical for the survival of the organism. The startling aspect of this dichotomy is that the seemingly disparate tasks may in fact be accomplished by the same neuronal circuitry. |
||
AdaptationA common thread through much of our current work is in understanding how encoding properties of the neuronal pathways change over time. One of the most striking properties of the visual system is that it can faithfully encode visual stimuli over an enormous operating range of light intensity. This important property exists as a result of adaptive mechanisms that effectively shift the neuronal sensitivity in response to changes in the light level. The adaptive mechanisms are continually active in all but the most artificial of laboratory conditions, as we move our eyes across the visual field, and objects move in and out of our view. These aspects of encoding are ignored in many studies, and are the subject of much controversy as to what the functional significance might be. This is an extremely critical issue in prosthetics, and in other types of biomimetic applications which seek to process visual information with the efficiency of the true biological systems. Adaptation mechanisms affect the encoding of information in the pathway, which is reflected in the properties of the spatio-temporal receptive field and corresponding spatial and temporal frequency properties. We have developed novel approaches to track changes in the linear and nonlinear encoding dynamics in the visual pathway through adaptive estimation schemes (Stanley, 2002; Lesica et al., 2003; Lesica and Stanley, 2005a; Lesica and Stanley 2005b), testing in the retina, LGN, and visual cortex. A block diagram of the encoding framework is shown in Figure 5, along with the evolution of a spatial receptive field (RF) of an LGN X cell over several seconds of exposure to a visual stimulus that induces adaptation.
Figure 5. Block diagram of the assumed encoding, with spatiotemporal receptive field (RF), g, followed by a rectifying nonlinearity. Bottom panels, adaptation of spatial properties of an OFF LGN X cell RF. Spatial RF at peak in temporal kernel at 8, 16, 24 and 32 s after stimulus onset (approx. 1.8 degrees of visual space). From Lesica et al. (2003). Figure 6 shows more recent work in which the algorithms were refined using extended recursive least-squares (ERLS), which is similar in nature to the framework in which the Kalman filter is used for parameter estimation.
Figure 6. (A) 3 minutes of the spatially uniform Gaussian white-noise contrast-switching stimulus. (B) The gain of the RF estimate. (C) The offset estimate. (D) The average gain of the RF estimate over 24 repeats of the same contrast transitions with (black) and without (gray) simultaneous estimation of the offset. (E) The average of the offset estimate. Data from Baccus and Meister; Figure from Lesica and Stanley (2005b). Specifically, this particular example is of a retinal ganglion cell response in a contrast switching experiment (data provided by Baccus and Meister). The left column demonstrates that the gain and offset (theta) of the imposed model (top of figure) can be estimated over a single trial of experimental data, as the contrast switching in panel A invokes dramatic adaptation. The second column illustrates that the proper estimation/representation of linear and nonlinear components of the encoding process is necessary to accurately capture the true dynamics of the adaptation process. We have established a recent collaboration with the laboratory of Dr. Jose-Manuel Alonso in the Department of Optometry at SUNY, in Manhattan. In this collaboration, we have been able to design experiments and collect data from the early visual pathway that specifically address issues related to encoding during adaptation, both with artificial stimuli designed to specifically probe contrast adaptation, and with natural scene stimuli. Related Publications D. A. Butts, C. Weng, J. Z. Jin, C. I. Yeh, N. A. Lesica, J. A. Alonso, and G. B. Stanley. Changes in functional properties of LGN neurons with contrast and their significance in information transmission, to be presented at SFN, Washington DC, 2005. PDF N. A. Lesica, J. Z. Jin, C. Weng, C. I. Yeh, D. A. Butts, G. B. Stanley, and J. A. Alonso. The effects of contrast adaptation during natural stimulation, to be presented at SFN, Washington DC, 2005. PDF N. A. Lesica and G. B. Stanley. Decoupling functional mechanisms of adaptive encoding in the visual pathway, in press, Network: Computation in Neural Systems, 2005. PDF N. A. Lesica and G. B. Stanley. Functional identification of adaptive visual encoding, to appear in Neural Engineering/Signal Processing, edited by Metin Akay, IEEE/Wiley, 2005. N. A. Lesica and G. B. Stanley. Improved tracking of time-varying encoding properties of visual neurons by extended recursive least-squares, IEEE Trans. Neural Sys. Rehab., 2005. PDF N. A. Lesica, A. S. Boloori, and G. B. Stanley. Adaptive encoding in the visual pathway, Network: Comput. in Neural Systems, 14:119-135, 2003. PDF G. B. Stanley. Adaptive spatiotemporal receptive field estimation in the visual pathway, Neural Computation, 14:2925-2946, 2002. PDF |
||
DetectionThe lateral geniculate nucleus (LGN) of the thalamus is the gateway to the visual cortex, controlling the flow of visual information from the retina [for a review of LGN function, see Sherman (2001a)]. Understanding the neural code of the LGN is an essential first step in characterizing the processing of visual information in higher-level neurons. After prolonged periods of hyperpolarization, voltage-dependent calcium channels in the membrane of the neuron are de-inactivated, and subsequent depolarization causes a stereotyped burst of closely spaced action potentials. It has been suggested that bursts serve to signal the appearance of a salient stimulus (detection), whereas tonic firing relays detailed features of the stimulus (transmission) (Crick, 1984; Guido et al., 1995). Bursts may serve as a wake-up call, alerting the visual cortex to the presence of a stimulus in the receptive field (RF) and signaling the beginning of tonic relay (Sherman, 2001b). We investigated the role of LGN bursts in encoding correlated natural stimuli by analyzing the responses of LGN neurons to natural scene movies. Across a sample of LGN X-cells, a significant increase in bursting was observed during natural scene stimulation (relative to white noise stimulation). As shown in left of Figure 7, bursts were triggered by specific stimulus features, such as the movement of objects into the RF.
Figure 7. Burst events are triggered by the appearance of objects during natural stimulation. LEFT: Frames 1, 3, 5, and 7 of an eight-frame (256 msec) sequence of the natural scene movie stimulus corresponding to the responses in b. The white circle indicates the RF of the neuron, the responses of which are shown in b. A 48x48 pixel region of the entire stimulus for each frame is shown. The inset shows the magnified stimulus inside the RF center. The timeline indicates the onset of each frame (F1–F8). b, A raster plot of the response of a typical neuron to eight repeats of the stimulus shown in a. Spikes that are part of burst events are gray. RIGHT: The temporal evolution of the average white noise stimulus (triggered average) in the center of the RF preceding burst events (gray) and tonic spikes (black) for a sample of 18 cells. Burst events were marked by the time of the first spike in the burst. Triggered averages from OFF cells were reflected about the mean luminance. Error bars represent one SD. From Lesica & Stanley (2004). The stimulus features preceding burst events and tonic spikes were characterized, revealing that the type of visual stimulus evoking a burst was fundamentally different from that evoking tonic activity, and was in fact a feature characteristic of the correlation structure of natural scenes. Taken together, our results support the detect/transmit framework described above, and suggest that LGN bursts may be an important part of the neural code of the LGN, providing an amplification of stimulus features that are typical of correlated natural scenes. The LGN resting membrane potential can vary widely according to behavioral state, from its lowest level during sleep to its highest level during active waking (Hirsch et al., 1983). We hypothesized that the resting potential would affect the particular features of the visual stimulus that evoke bursts, and therefore be a function of behavioral state. In a recent study with our collaborators (the laboratory of Dr. Jose-Manuel Alonso at SUNY), we characterized the visual features that evoke bursts at different resting potentials using simulated and experimental LGN responses to natural scene movies, and our results support this claim. To investigate the functional consequences of the effects of resting potential on burst generation, we tested the effects of changes in resting potential on the extent to which bursts enhance the detection of different visual features (Guido et al., 1995; Sherman, 2001a; Smith and Sherman, 2002). Although comparing the LGN response with and without bursts in vivo is not possible (Porcello et al., 2003), these experiments can be simulated using an integrate-and-fire-or burst (IFB) model, which can accurately reproduce the LGN response during both tonic and burst firing (Smith et al., 2000; Lesica and Stanley, 2004). We simulated the LGN response to different visual features at different resting potentials with and without the burst mechanism, and compared the results using signal detection theory. Our results show that bursts enhance detection of the onset of excitatory features at low resting potentials and the offset of inhibitory features at high resting potentials, suggesting that bursts may play a dynamic role in visual processing that changes with behavioral state. Most recently, we have also developed algorithms that are inspired by the transmit/detect framework of the early visual pathway, specifically for the robust relay of visual information in situations with constraints on bandwidth. Related Publications N. A. Lesica, C. Weng, J. Jin, C. Yeh, J. M. Alonso, and G. B. Stanley. LGN bursts enhance detection of specific stimulus features in natural scenes, submitted, 2005. N. A. Lesica and G. B. Stanley. An LGN inspired detect/transmit framework for high fidelity relay of visual informatoin with limited bandwidth, to be presented at the 1st International Symposium on Brain, Vision, and Artificial Intelligence, Naples, 2005. PDF N. A. Lesica and G. B. Stanley. Encoding of natural scene movies by tonic and burst spikes in the lateral geniculate nucleus, J. Neurosci., 24:10731-10740, 2004. PDF |
||
Information TransmissionFor decades, the visual receptive field (RF) has served as the fundamental building block for our current understanding of the visual pathway (Kuffler, 1953; Hubel and Weisel, 1962). Spatiotemporal integration of visual stimuli, when combined with functional mechanisms representative of non-linear spike generation, has been shown to be a good predictor of firing rate for many neurons in the early visual pathway (Dan et al., 1996; Stanley, 2002). However, the temporal resolution of the stimulus representation is limited by the photoreceptor transduction process and takes place over a time course of tens to hundreds of milliseconds (Barlow, 1952), limiting a receptive-field-based description of the firing rate to this relatively coarse temporal scale. In contrast, information theoretic studies of these same neurons reveal significant temporal structure in the neural responses on the order of one millisecond (Reinagel and Reid, 2000; Liu et al., 2001). The discrepancy of temporal scales between stimulus representation and neural response has been the seed of a far ranging debate concerning temporal precision and variability of neural responses, and their corresponding role in neural coding of dynamic stimuli (e.g., de Ruyter van Steveninck et al. (1997); Warzecha and Egelhaaf (1999)). Figure 8 shows the ability of a simple linear-nonlinear (LN) model to capture the firing activity of an LGN neuron at a coarse temporal resolution (bottom), while failing at finer time resolutions (middle).
Figure 8. Raster
of the spike times of a simulated LGN neuron to full-field Gaussian
white noise, with the resulting instantaneous spike rate (below, solid
line) and LN prediction (dashed line), at a binsize of 0.3 ms. Dashed
horizontal line shows the mean firing rate. Bottom, actual (solid) and
LN model prediction (dashed) of firing rate at binsize of 8.3 ms. In recent work, we have established a direct link between receptive field descriptions of neurons and their information encoding properties by incorporating elements that capture finer time resolutions into the relatively coarse representation of the receptive field. Such a framework results in “sparse” representations with large fluctuations in firing rates that match experimental observations, and thus provides an understanding of how elements of neural encoding directly relate to information transmission. Furthermore, these sparse representations translate into structure in the neuronal response on time scales of the order of a millisecond. We have demonstrated that such structure conveys a significant fraction of the total information about the stimulus. As a result, two neurons with subtle differences in their receptive fields could produce distinct responses that would be indistinguishable at coarse times scales, allowing information about small differences in visual stimuli to be easily read out by downstream neurons. Related Publications D. A. Butts, C. Weng, J. Z. Jin, C. I. Yeh, N. A. Lesica, J. A. Alonso, and G. B. Stanley. Changes in functional properties of LGN neurons with contrast and their significance in information transmission, to be presented at SFN, Washington DC, 2005. PDF D. A. Butts and G. B. Stanley. Temporal hyperacuity in visual neurons: connecting receptive fields to information transmission, submitted, 2005. D. A. Butts and G. B. Stanley. The important aspects of visual encoding: relating receptive fields to mutual information rates in visual neurons, COSYNE, Salt Lake City, 2005. PDF D. A. Butts, A. E. Desjardins, and G. B. Stanley. Quick and accurate information calculations based on linear characterizations of sensory neurons, SFN, New Orleans, 2003. HTML
| ||
