Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | Previous revision | ||
|
shared:publications [2020/04/15 11:26] |
shared:publications [2023/08/01 12:32] (current) justin |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Publication List ====== | ====== Publication List ====== | ||
| + | Fox, K. J., Birman, D. and Gardner, J. L. (2023) Gain, not concomitant changes in spatial receptive field properties, improves task performance in a neural network attention model //eLife// [[ https://doi.org/10.7554/eLife.78392|DOI]] 12:e78392 ++Abstract| | ||
| + | \\ | ||
| + | \\ | ||
| + | Attention allows us to focus sensory processing on behaviorally relevant aspects of the visual world. One potential mechanism of attention is a change in the gain of sensory responses. However, changing gain at early stages could have multiple downstream consequences for visual processing. Which, if any, of these effects can account for the benefits of attention for detection and discrimination? Using a model of primate visual cortex we document how a Gaussian-shaped gain modulation results in changes to spatial tuning properties. Forcing the model to use only these changes failed to produce any benefit in task performance. Instead, we found that gain alone was both necessary and sufficient to explain category detection and discrimination during attention. Our results show how gain can give rise to changes in receptive fields which are not necessary for enhancing task performance.++[[https://www.biorxiv.org/content/10.1101/2022.03.04.483026v2.full.pdf|pdf]] | ||
| + | |||
| + | Himmelberg, M. M., Gardner, J. L. and Winawer, J. (2022) What has vision science taught us about functional MRI? //Neuroimage// 262:119536. [[https://doi.org/10.1016/j.neuroimage.2022.119536|DOI]] ++Abstract| | ||
| + | \\ | ||
| + | \\ | ||
| + | In the domain of human neuroimaging, much attention has been paid to the question of whether and how the development of functional magnetic resonance imaging (fMRI) has advanced our scientific knowledge of the human brain. However, the opposite question is also important; how has our knowledge of the visual system advanced our understanding of fMRI? Here, we discuss how and why scientific knowledge about the human and animal visual system has been used to answer fundamental questions about fMRI as a brain measurement tool and how these answers have contributed to scientific discoveries beyond vision science.++{{:reprints:1-s2.0-s1053811922006516-main.pdf|pdf}} | ||
| + | |||
| + | Jagadeesh, A. V., and Gardner, J. L. (2022) Texture-like representation of objects in human visual cortex //Proceedings of the National Academy of Sciences// 119 (17) e2115302119. [[https://doi.org/10.1073/pnas.2115302119|DOI]] ++Abstract| | ||
| + | \\ | ||
| + | \\ | ||
| + | Task-optimized convolutional neural networks (CNNs) show striking similarities to the ventral visual stream. However, human-imperceptible image perturbations can cause a CNN to make incorrect predictions. Here we provide insight into this brittleness by investigating the representations of models that are either robust or not robust to image perturbations. Theory suggests that the robustness of a system to these perturbations could be related to the power law exponent of the eigenspectrum of its set of neural responses, where power law exponents closer to and larger than one would indicate a system that is less susceptible to input perturbations. We show that neural responses in mouse and macaque primary visual cortex (V1) obey the predictions of this theory, where their eigenspectra have power law exponents of at least one. We also find that the eigenspectra of model representations decay slowly relative to those observed in neurophysiology and that robust models have eigenspectra that decay slightly faster and have higher power law exponents than those of non-robust models. The slow decay of the eigenspectra suggests that substantial variance in the model responses is related to the encoding of fine stimulus features. We therefore investigated the spatial frequency tuning of artificial neurons and found that a large proportion of them preferred high spatial frequencies and that robust models had preferred spatial frequency distributions more aligned with the measured spatial frequency distribution of macaque V1 cells. Furthermore, robust models were quantitatively better models of V1 than non-robust models. Our results are consistent with other findings that there is a misalignment between human and machine perception. They also suggest that it may be useful to penalize slow-decaying eigenspectra or to bias models to extract features of lower spatial frequencies during task-optimization in order to improve robustness and V1 neural response predictivity.++{{:reprints:pnas.2115302119.pdf|pdf}} | ||
| + | |||
| + | Kong, N. C. L., Margalit, E., Gardner, J. L., and Norcia, A. M. (2022) Increasing neural network robustness improves match to macaque V1 eigenspectrum, spatial frequency preference and predictivity //PLoS Computational Biology// 18:e1009739. [[https://doi.org/10.1371/journal.pcbi.10009739|DOI]] ++Abstract| | ||
| + | \\ | ||
| + | \\ | ||
| + | Task-optimized convolutional neural networks (CNNs) show striking similarities to the ventral visual stream. However, human-imperceptible image perturbations can cause a CNN to make incorrect predictions. Here we provide insight into this brittleness by investigating the representations of models that are either robust or not robust to image perturbations. Theory suggests that the robustness of a system to these perturbations could be related to the power law exponent of the eigenspectrum of its set of neural responses, where power law exponents closer to and larger than one would indicate a system that is less susceptible to input perturbations. We show that neural responses in mouse and macaque primary visual cortex (V1) obey the predictions of this theory, where their eigenspectra have power law exponents of at least one. We also find that the eigenspectra of model representations decay slowly relative to those observed in neurophysiology and that robust models have eigenspectra that decay slightly faster and have higher power law exponents than those of non-robust models. The slow decay of the eigenspectra suggests that substantial variance in the model responses is related to the encoding of fine stimulus features. We therefore investigated the spatial frequency tuning of artificial neurons and found that a large proportion of them preferred high spatial frequencies and that robust models had preferred spatial frequency distributions more aligned with the measured spatial frequency distribution of macaque V1 cells. Furthermore, robust models were quantitatively better models of V1 than non-robust models. Our results are consistent with other findings that there is a misalignment between human and machine perception. They also suggest that it may be useful to penalize slow-decaying eigenspectra or to bias models to extract features of lower spatial frequencies during task-optimization in order to improve robustness and V1 neural response predictivity.++{{:reprints:journal.pcbi.1009739.pdf|pdf}} | ||
| + | |||
| + | Gardner, J. L., and Merriam, E.M. (2021) Population models, not analyses, of human neuroscience measurements //Annual Review of Vision Science// 7:1-31. [[https://doi.org/10.1146/annurev-vision-093019-111124|DOI]] ++Abstract| | ||
| + | \\ | ||
| + | \\ | ||
| + | Selectivity for many basic properties of visual stimuli, such as orientation, is thought to be organized at the scale of cortical columns, making it difficult or impossible to measure directly with noninvasive human neuroscience measurement. However, computational analyses of neuroimaging data have shown that selectivity for orientation can be recovered by considering the pattern of response across a region of cortex. This suggests that computational analyses can reveal representation encoded at a finer spatial scale than is implied by the spatial resolution limits of measurement techniques. This potentially opens up the possibility to study a much wider range of neural phenomena that are otherwise inaccessible through noninvasive measurement. However, as we review in this article, a large body of evidence suggests an alternative hypothesis to this superresolution account: that orientation information is available at the spatial scale of cortical maps and thus easily measurable at the spatial resolution of standard techniques. In fact, a population model shows that this orientation information need not even come from single-unit selectivity for orientation tuning, but instead can result from population selectivity for spatial frequency. Thus, a categorical error of interpretation can result whereby orientation selectivity can be confused with spatial frequency selectivity. This is similarly problematic for the interpretation of results from numerous studies of more complex representations and cognitive functions that have built upon the computational techniques used to reveal stimulus orientation. We suggest in this review that these interpretational ambiguities can be avoided by treating computational analyses as models of the neural processes that give rise to measurement. Building upon the modeling tradition in vision science using considerations of whether population models meet a set of core criteria is important for creating the foundation for a cumulative and replicable approach to making valid inferences from human neuroscience measurements.++{{:reprints:annurev-vision-093019-111124.pdf|pdf}} | ||
| + | |||
| + | Lin, Y., Zhou, X., Naya, Y., Gardner, J. L., and Sun, P. (2021) Voxel-wise linearity analysis of increments and decrements in BOLD responses in human visual cortex using a contrast adaptation paradigm //Frontiers in Human Neuroscience// 15:541314.[[https://doi.org/10.3389/fnhum.2021.541314|DOI]] ++Abstract| | ||
| + | \\ | ||
| + | \\ | ||
| + | The linearity of BOLD responses is a fundamental presumption in most analysis procedures for BOLD fMRI studies. Previous studies have examined the linearity of BOLD signal increments, but less is known about the linearity of BOLD signal decrements. The present study assessed the linearity of both BOLD signal increments and decrements in the human primary visual cortex using a contrast adaptation paradigm. Results showed that both BOLD signal increments and decrements kept linearity to long stimuli (e.g., 3 s, 6 s), yet, deviated from linearity to transient stimuli (e.g., 1 s). Furthermore, a voxel-wise analysis showed that the deviation patterns were different for BOLD signal increments and decrements: while the BOLD signal increments demonstrated a consistent overestimation pattern, the patterns for BOLD signal decrements varied from overestimation to underestimation. Our results suggested that corrections to deviations from linearity of transient responses should consider the different effects of BOLD signal increments and decrements.++{{:reprints:fnhum-15-541314.pdf|pdf}} | ||
| + | |||
| Lin W-H., Gardner J. L., Wu S-W. (2020) Context effects on probability estimation. //PLoS Biology// 18:e3000634.[[https://doi.org/10.1371/journal.pbio.3000634|DOI]] ++ Abstract| | Lin W-H., Gardner J. L., Wu S-W. (2020) Context effects on probability estimation. //PLoS Biology// 18:e3000634.[[https://doi.org/10.1371/journal.pbio.3000634|DOI]] ++ Abstract| | ||
| \\ | \\ | ||
| Line 13: | Line 43: | ||
| \\ | \\ | ||
| \\ | \\ | ||
| - | Attention can both enhance and suppress cortical sensory representations. However, changing sensory representations can also be detrimental to behavior. Behavioral consequences can be avoided by flexibly changing sensory readout, while leaving the representations unchanged. Here, we asked human observers to attend to and report about either one of two features which control the visibility of motion while making concurrent measurements of cortical activity with BOLD imaging (fMRI). Extending a well-established linking model to account for the relationship between these measurements, we found that changes in sensory representation during directed attention were insufficient to explain perceptual reports. A flexible downstream readout was also necessary to best explain our data. Such a model implies that observers should be able to recover information about ignored features, a prediction which we confirmed behaviorally. Thus, flexible readout is a critical component of the cortical implementation of human adaptive behavior.++{{:reprints:Nat_Commun_2019_Birman.pdf|pdf}} | + | Attention can both enhance and suppress cortical sensory representations. However, changing sensory representations can also be detrimental to behavior. Behavioral consequences can be avoided by flexibly changing sensory readout, while leaving the representations unchanged. Here, we asked human observers to attend to and report about either one of two features which control the visibility of motion while making concurrent measurements of cortical activity with BOLD imaging (fMRI). Extending a well-established linking model to account for the relationship between these measurements, we found that changes in sensory representation during directed attention were insufficient to explain perceptual reports. A flexible downstream readout was also necessary to best explain our data. Such a model implies that observers should be able to recover information about ignored features, a prediction which we confirmed behaviorally. Thus, flexible readout is a critical component of the cortical implementation of human adaptive behavior.++{{:reprints:nat_commun_2019_birman.pdf|pdf}} |
| Fukuda H., Ma N., Suzuki S., Harasawa N., Ueno K., Gardner J.L., Ichinohe N., Haruno M., Cheng K., Nakahara H. (2019) Computing social value conversion in the human brain. //The Journal of Neuroscience// 39(26):5153-72 [[http://doi.org/10.1523/JNEUROSCI.3117-18.2019|DOI]] ++Abstract| | Fukuda H., Ma N., Suzuki S., Harasawa N., Ueno K., Gardner J.L., Ichinohe N., Haruno M., Cheng K., Nakahara H. (2019) Computing social value conversion in the human brain. //The Journal of Neuroscience// 39(26):5153-72 [[http://doi.org/10.1523/JNEUROSCI.3117-18.2019|DOI]] ++Abstract| | ||
| Line 28: | Line 58: | ||
| \\ | \\ | ||
| \\ | \\ | ||
| - | The foundation for modern understanding of how we make perceptual decisions about what it is that we see or where to look comes from considering the optimal way to perform these behaviors. While statistical computation is useful for deriving the optimal solution to a perceptual problem, optimality requires perfect knowledge of priors and often complex computation. Accumulating evidence, however, suggests that optimal perceptual goals can be achieved or approximated more simply by human observers using heuristic approaches. Perceptual neuroscientists captivated by optimal explanations of sensory behaviors will fail in their search for the neural circuits and cortical processes that implement an optimal computation whenever that behavior is actually achieved through heuristics. This article provides a cross-disciplinary review of decision-making with the aim of building perceptual theory that uses optimality to set the computational goals for perceptual behavior, but through consideration of ecological, computational and energetic constraints incorporates how these optimal goals can be achieved through heuristic approximation.++{{:reprints:Gardner-2019-Nature_Neuroscience.pdf|pdf}} | + | The foundation for modern understanding of how we make perceptual decisions about what it is that we see or where to look comes from considering the optimal way to perform these behaviors. While statistical computation is useful for deriving the optimal solution to a perceptual problem, optimality requires perfect knowledge of priors and often complex computation. Accumulating evidence, however, suggests that optimal perceptual goals can be achieved or approximated more simply by human observers using heuristic approaches. Perceptual neuroscientists captivated by optimal explanations of sensory behaviors will fail in their search for the neural circuits and cortical processes that implement an optimal computation whenever that behavior is actually achieved through heuristics. This article provides a cross-disciplinary review of decision-making with the aim of building perceptual theory that uses optimality to set the computational goals for perceptual behavior, but through consideration of ecological, computational and energetic constraints incorporates how these optimal goals can be achieved through heuristic approximation.++{{:reprints:gardner-2019-nature-neuroscience.pdf|pdf}} |
| Birman, D., and Gardner, J. L. (2018) A quantitative framework for motion visibility in human cortex. //Journal of Neurophysiology// 120:1824-1839. [[https://www.physiology.org/doi/abs/10.1152/jn.00433.2018|DOI]] [[https://osf.io/s7j9p/|DATA]]++Abstract| | Birman, D., and Gardner, J. L. (2018) A quantitative framework for motion visibility in human cortex. //Journal of Neurophysiology// 120:1824-1839. [[https://www.physiology.org/doi/abs/10.1152/jn.00433.2018|DOI]] [[https://osf.io/s7j9p/|DATA]]++Abstract| | ||
| Line 56: | Line 86: | ||
| \\ | \\ | ||
| \\ | \\ | ||
| - | When making choices under conditions of perceptual uncertainty, past experience can play a vital role. However, it can also lead to biases that worsen decisions. Consistent with previous observations, we found that human choices are influenced by the success or failure of past choices even in a standard two-alternative detection task, where choice history is irrelevant. The typical bias was one that made the subject switch choices after a failure. These choice-history biases led to poorer performance and were similar for observers in different countries. They were well captured by a simple logistic regression model that had been previously applied to describe psychophysical performance in mice. Such irrational biases seem at odds with the principles of reinforcement learning, which would predict exquisite adaptability to choice history. We therefore asked whether subjects could adapt their irrational biases following changes in trial order statistics. Adaptability was strong in the direction that confirmed a subject’s default biases, but weaker in the opposite direction, so that existing biases could not be eradicated. We conclude that humans can adapt choice history biases, but cannot easily overcome existing biases even if irrational in the current context: adaptation is more sensitive to confirmatory than contradictory statistics.++{{:reprints:PNAS-2016-Abrahamyan-E3548-57.pdf|pdf}} | + | When making choices under conditions of perceptual uncertainty, past experience can play a vital role. However, it can also lead to biases that worsen decisions. Consistent with previous observations, we found that human choices are influenced by the success or failure of past choices even in a standard two-alternative detection task, where choice history is irrelevant. The typical bias was one that made the subject switch choices after a failure. These choice-history biases led to poorer performance and were similar for observers in different countries. They were well captured by a simple logistic regression model that had been previously applied to describe psychophysical performance in mice. Such irrational biases seem at odds with the principles of reinforcement learning, which would predict exquisite adaptability to choice history. We therefore asked whether subjects could adapt their irrational biases following changes in trial order statistics. Adaptability was strong in the direction that confirmed a subject’s default biases, but weaker in the opposite direction, so that existing biases could not be eradicated. We conclude that humans can adapt choice history biases, but cannot easily overcome existing biases even if irrational in the current context: adaptation is more sensitive to confirmatory than contradictory statistics.++{{:reprints:pnas-2016-abrahamyan.pdf|pdf}} |
| Birman, D. & Gardner, J. L. (2016) Parietal and prefrontal: categorical differences? //Nature Neuroscience// 19: 5-7 [[http://dx.doi.org/10.1038/nn.4204|DOI]] | Birman, D. & Gardner, J. L. (2016) Parietal and prefrontal: categorical differences? //Nature Neuroscience// 19: 5-7 [[http://dx.doi.org/10.1038/nn.4204|DOI]] | ||
| Line 94: | Line 124: | ||
| How does our brain detect changes in a natural scene? While changes by increments of specific visual attributes, such as contrast or motion coherence, can be signaled by an increase in neuronal activity in early visual areas, like the primary visual cortex (V1) or the human middle temporal complex (hMT+), respectively, the mechanisms for signaling changes resulting from decrements in a stimulus attribute are largely unknown. We have discovered opposing patterns of cortical responses to changes in motion coherence: unlike areas hMT+, V3A and parieto-occipital complex (V6+) that respond to changes in the level of motion coherence monotonically, human areas V4 (hV4), V3B, and ventral occipital always respond positively to both transient increments and decrements. This pattern of responding always positively to stimulus changes can emerge in the presence of either coherence-selective neuron populations, or neurons that are not tuned to particular coherences but adapt to a particular coherence level in a stimulus-selective manner. Our findings provide evidence that these areas possess physiological properties suited for signaling increments and decrements in a stimulus and may form a part of cortical vigilance system for detecting salient changes in the environment.++ | How does our brain detect changes in a natural scene? While changes by increments of specific visual attributes, such as contrast or motion coherence, can be signaled by an increase in neuronal activity in early visual areas, like the primary visual cortex (V1) or the human middle temporal complex (hMT+), respectively, the mechanisms for signaling changes resulting from decrements in a stimulus attribute are largely unknown. We have discovered opposing patterns of cortical responses to changes in motion coherence: unlike areas hMT+, V3A and parieto-occipital complex (V6+) that respond to changes in the level of motion coherence monotonically, human areas V4 (hV4), V3B, and ventral occipital always respond positively to both transient increments and decrements. This pattern of responding always positively to stimulus changes can emerge in the presence of either coherence-selective neuron populations, or neurons that are not tuned to particular coherences but adapt to a particular coherence level in a stimulus-selective manner. Our findings provide evidence that these areas possess physiological properties suited for signaling increments and decrements in a stimulus and may form a part of cortical vigilance system for detecting salient changes in the environment.++ | ||
| \\ | \\ | ||
| - | {{reprints:2012Costagli.pdf|pdf}} | + | {{reprints:2012costagli.pdf|pdf}} |
| Merriam, E. P., Gardner, J. L., Movshon, J. A., and Heeger, D. J. (2013) Modulation of visual responses by gaze direction in human visual cortex. //The Journal of Neuroscience// 33: 9879-9889 [[http://dx.doi.org/10.1523/JNEUROSCI.0500-12.2013|DOI]] ++Abstract| | Merriam, E. P., Gardner, J. L., Movshon, J. A., and Heeger, D. J. (2013) Modulation of visual responses by gaze direction in human visual cortex. //The Journal of Neuroscience// 33: 9879-9889 [[http://dx.doi.org/10.1523/JNEUROSCI.0500-12.2013|DOI]] ++Abstract| | ||