Perception Beyond Gestalt

How does the brain piece together the information required to achieve object recognition, figure-ground segmentation, object completion in cases of partial occlusion and related perceptual phenomena?

This book focuses on principles of Gestalt psychology and the key issues which surround them, providing an up-to-date survey of the most interesting and highly debated topics in visual neuroscience, perception and object recognition.

The volume is divided into three main parts: Gestalt and perceptual organization, attention aftereffects and illusions, and color vision and art perception. Themes covered in the book include:

a historical review of Gestalt theory and its relevance in modern-day neuroscience

the relationship between perceptive and receptive fields

a critical analysis of spatiotemporal unity of perception

the role of Gestalt principles in perceptual organization

self-organizing properties of the visual field

the role of attention and perceptual grouping in forming non-retinotopic representations

figural distortions following adaptation to spatial patterns

illusory changes of brightness in spatial patterns

the function of motion illusions as a tool to study Gestalt principles in vision

conflicting theories of color vision and the neural basis of it

the role of color in figure-ground segmentation

chromatic assimilation in visual art and perception

the phenomena of colored shadows.

Including contributions from experts in the field, this book will provide an essential overview of current research and theory on visual perception and Gestalt. It will be key reading for researchers and academics in the field of visual perception and neuroscience.

Adam Geremek received his PhD at the Brain Research Unit, Freiburg, and is now a paediatrician specializing in child psychiatry in Schleswig, Germany.

Mark W. Greenlee is Professor of Experimental Psychology at the University of Regensburg, Germany.

Svein Magnussen is Professor Emeritus of Cognitive Psychology at the University of Oslo, Norway.

Explorations in Cognitive Psychology series

Books in this series:

Perception Beyond Gestalt

Progress in vision research

Edited by Adam Geremek, Mark W. Greenlee and Svein Magnussen

Perception Beyond Gestalt

Progress in vision research

Edited by

Adam Geremek, Mark W. Greenlee and Svein Magnussen

First published 2014
by Psychology Press
27 Church Road, Hove, East Sussex, BN3 2FA

Simultaneously published in the USA and Canada
by Psychology Press
711 Third Avenue, New York, NY 10017

Psychology Press is an imprint of the Taylor & Francis Group, an informa business

© 2014 A. Geremek, M. W. Greenlee and S. Magnussen

The right of the editors to be identified as the authors of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers.

Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.

British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library

Library of Congress Cataloging in Publication Data
Perception beyond Gestalt : progress in vision research / edited by
Adam Geremek, Mark Greenlee, and Svein Magnussen.

    pages cm

  Includes index.

1. Gestalt psychology. 2. Perception. I. Geremek, Adam editor of compilation. II. Greenlee, Mark editor of compilation. III. Magnussen, Svein editor of compilation.
BF203.P27 2014
153.7–dc22                           2013008418

ISBN: 978–0–415–65801–0 (hbk)
ISBN: 978–0–203–07632–3 (ebk)

In honor of
Prof. Lothar Spillmann
on the occasion of his 75th birthday

List of contributors

Prof. Bruno G. Breitmeyer

Director, Visual Cognition Lab

Department of Psychology

University of Houston

Houston, Texas 77204, USA

Frédéric Devinck, PhD

Department of Psychology

University Rennes 2

CRPCC, EA 1285

35000 Rennes, France

Erica L. Dixon

Department of Psychology

College of Arts and Science

American University

4400 Massachusetts Avenue, NW

Washington D.C. 20016, USA

Dr. Birgitta Dresp-Langley

Centre National de la Recherche

Scientifique UMR 5508

University of Montpellier, France

Dr. Adam Geremek

HELIOS Klinikum Schleswig

Clinic for Child and Adolescent Psychiatry

Friedrich-Ebert-Str. 5

24837 Schleswig, Germany

Simone Gori, PhD

Department of General Psychology

University of Padua

Via Venezia 8

35131 Padova, Italy

Prof. Mark W. Greenlee

Institute for Experimental Psychology

Faculty for Psychology, Educational und Sports Science

University of Regensburg

Universitässtrasse 31

93053 Regensburg, Germany

Dr. Kai Hamburger

Experimental Psychology and Cognitive Science

Justus Liebig University Giessen

Otto-Behaghel-Str. 10F

35394 Giessen, Germany

Prof. Lewis O. Harvey, Jr.

Chair, Department of Psychology and Neuroscience

University of Colorado, 345 UCB

Boulder, Colorado 80309-0345 USA

Prof. Michael H. Herzog

Head, Laboratory of Psychophysics

École Polytechnique Fédérale de Lausanne

1015 Lausanne, Switzerland

Dr. Julian Kallmann

Institute of Zoology

Johannes-Gutenberg-University Mainz

55128 Mainz, Germany

Prof. Svein Magnussen

Professor emeritus

Department of Psychology

University of Oslo

0371 Oslo, Norway

Prof. Christa Neumeyer

Institute of Zoology

Johannes-Gutenberg-University Mainz

55128 Mainz, Germany

Prof. Haluk Öğmen

Director, Center for Neuro-Engineering and Cognitive Science

Department of Electrical & Computer Engineering

N 308 Engineering Building 1

Houston, Texas 77204, USA

Prof. Baingio Pinna

Director, Department of Architecture, Design & Planning

University of Sassari at Alghero, Sardegna

Palazzo del Pou Salit, Piazza Duomo 6

07041 Alghero (SS), Italy

Prof. Adam Reeves

Psychology Department

Northeastern University

125 Nightingale Hall, 360 Huntington Ave.

Boston, Massachusetts 02115, USA

Dr. Eric K. Schmidt, M.A., D.D.S.

Alameda Family Dentistry

2049 Central Avenue

Alameda, California 94501, USA

Dr. Jürgen Schramme

Institute of Zoology

Johannes-Gutenberg-University Mainz

55128 Mainz, Germany

Prof. Arthur G. Shapiro

Department of Psychology

College of Arts and Science

American University

4400 Massachusetts Avenue, NW

Washington D.C. 20016, USA

Prof. Alan Stubbs

Department of Psychology

University of Maine

5742 Clarence Cook Little Hall

Orono, Maine 04469-5742, USA

Prof. James P. Thomas

Professor emeritus

Department of Cognitive Psychology

UCLA Los Angeles

CA 90095-1563, USA

Prof. Arne Valberg

Professor emeritus

Norwegian University of Science and Technology

Section of Biophysics and Medical Technology

N-7491 Trondheim, Norway

Prof. Johan Wagemans

Faculty of Psychology and Educational Sciences

Laboratory of Experimental Psychology

Tiensestraat 102 – box 3711

3000 Leuven, Belgium

Prof. John S. Werner

Department of Neurobiology, Physiology & Behavior, and

Department of Ophthalmology & Vision Science

University of California, Davis, 4860 Y Street, Suite 2400

Sacramento, California 95817, USA

Preface

The underlying theme of the present volume is perception beyond Gestalt. This volume offers new insights into the principles underlying perceptual organization. Each section deals with new research on topics that are of interest to the Gestalt theorists, such as how is the percept influenced by the individual parts of the image, how do dynamic processes influence the way we perceive static and moving stimuli, how does past experience (adaptation) and attention affect the way we perceive or how does color influence our perception and aesthetic appreciation?

Overall, this volume summarizes recent developments in the study of visual perception and Gestalt principles. As such it should provide essential reading to students and researchers interested in visual perception, sensory neuroscience and the relationship between visual perception and art appreciation. In addition, this book pays tribute to one of the pioneers of modern Gestalt theory, Professor Lothar Spillmann, who directed the psychophysics laboratory at the Neurologische Universitäsklinik, at the University of Freiburg, Germany, over a 35-year period (1970–2005).

The Freiburg psychophysics laboratory under Spillmann’s leadership was a center for vision scientists from all over the world. In the spirit of Richard Jung, the legendary director of the Neurologische Universitäsklinik, Abteilung fur Neurophysiologie, Spillmann attracted a large number of international scientists, funded by the German Research Foundation, the Alexander von Humboldt Foundation and the German Academic Exchange Service, including several of the authors of the present volume. Many stayed in Freiburg for periods of months or even years, since Spillmann’s almost childlike fascination for vision research was contagious. Long nocturnal experimental runs and on-going vivid discussions with international colleagues, often ending up in the local wine tavern – these collaborations led to hundreds of publications, making the Freiburg psychophysics laboratory a top address among international perception research, and possibly one of the most influential laboratories on the European continent. Lothar Spillmann was extremely generous in his support and paved the way for many of the current generation of vision scientists. In contrast to many laboratory directors nowadays, he never expected to be co-author of papers that came out of his lab, but rather only on those he really contributed to. Above all, his greatest achievement was bringing together scientists from all over the world to focus on the research issues at hand – a quality without which this volume would not have come about.

We hope we have captured some of the excitement we experienced while working in the Spillmann Laboratory and we want to express our admiration of Lothar Spillmann with this book, rightfully entitled: Perception Beyond Gestalt – Progress in Vision Research.

Adam Geremek, Mark W. Greenlee, Svein Magnussen

Kiel, Regensburg, Oslo in February 2013

1    Perception beyond Gestalt

Introductory remarks

Adam Geremek, Mark W. Greenlee and Svein Magnussen

At the beginning of the twentieth century, the Berlin school of Gestalt psychology headed by Christian von Ehrenfels and Carl Stumpf, with its pioneering members Kurt Koffka, Max Wertheimer, Wolfgang Köhler, Kurt Lewin and Wolfgang Metzger, developed a theory of perceptual organization that is now known as Gestalt theory. This school of thought proposed that the percept of the whole was more than the sum of its parts. At the focus of this idea are the principles of perceptual grouping, consisting of the laws of closure, proximity, similarity, good continuation, symmetry, common fate and good Gestalt. These principles or laws were put forth as general tendencies of perceptual organization that can be best studied when visual images were incomplete or the definition of figure-ground is ambiguous. Perceptual phenomena like figure-ground segmentation (i.e. which parts of the image belong to the object and which belong to the background), bistable percepts (exemplified in the famous Necker cube or the Rubin vase-face figure) and visual illusions of brightness (e.g. the Hermann gird or Ehrenstein figure), motion (the phi-phenomenon and other forms of apparent motion) or perspective (the Ponzo figure) could be better understood if the underlying Gestalt principles were consulted. When viewing a matrix of dots, we tend to perceive a single row of these dots as belonging to a perceptual group when they share a common color, are close in proximity to each other, or when their arrangement leads to the formation of a good shape (indeed the German term Gestalt means shape or form, see Figure 1.1). Phenomenological in nature, the Gestalt approach was influenced also by the insights of Weber and Fechner and their laws of psychophysics. Precise description of perceptual phenomena and the stimulus conditions leading to these phenomena were essential prerequisites for their research in perception.

Figure 1.1 a) Demonstration of Gestalt principle of similarity. The 7 x 7 array of gray dots is aligned to be equidistant. The central row forms a collinear group since their color is slightly brighter than those of the other dots. b) Demonstration of Gestalt principle of good Gestalt. As in Figure 1.1(a), the 7 x 7 array of gray dots is aligned to be equidistant. The central square forms two collinear rows and columns of dots that share a slightly brighter color compared to the background dots (see Chapter 5 by Pinna, and Chapter 7 by Hamburger, Dixon and Shapiro for more detailed accounts of Gestalt principles underlying perceptual grouping).

Initially, the Gestaltists had little or no knowledge of the underlying neural mechanisms responsible for these sensory and perceptual processes. The later work of Hartline (1940), Jung et al. (1952), Hubel and Wiesel (1959) and other researchers had yet to be conducted to reveal concepts like that of receptive fields of sensory neurons, the channels for brightness and darkness perception and orientation selectivity. Indeed, the electrical field theory proposed by Köhler, where electrical fields were thought to propagate along the surface of the cortex forming the Gestalt percept, was rejected by American researchers like Lashley, who conducted critical experiments disproving this assumption (see Lashley et al., 1951 and the chapter by Wagemans in this volume). Despite the shortcomings of that time, Gestalt theory provided an important theoretical foundation for researchers by outlining a set of organizing principles that guide our visual perception. Unfortunately, the political developments taking place at the end of the Weimar Republic led to an abrupt halt of these academic activities in 1933, when many of the leading Gestalt scientists fled Nazi Germany and emigrated to the United States of America or elsewhere.

When a century ago the paper on phi-motion (i.e. illusory motion seen, for example, in a circular array of dots, where the location of a single missing dot is displaced sequentially along the array) was published by Max Wertheimer, probably only a few were able to foresee that a new theory of vision was about to be founded. This new point of view on perception rapidly developed into the Gestalt school of psychology. Why do we see our surrounding world the way we see it and what are the basic principles guiding our perception of the world? This issue has been investigated using psychophysical, neurophysiological or computational methods over the past century. Now, at the beginning of the twenty-first century, we want to summarize to what extent Gestalt theory has survived in modern neuroscience after decades of research in neurophysiology, single-neuron recordings and neuroimaging research. We want to determine to what extent the ideas of the early Gestaltists still survive in modern vision research.

Modern perceptual science builds on the findings from neurophysiology and systems neuroscience to understand principles of perceptual organization as a direct consequence of the underlying neural architecture of the sensory system. The sensory modality receiving the most attention so far is vision. Experiments are now designed to test specific ideas arising from theories of perception, best exemplified by the work of von der Heydt, Roelfsema and others (reviewed in the chapter by Wagemans). The present volume focuses on these visual processes and provides new insights into our present understanding of principles guiding perceptual organization, the role of visual neuron’s receptive fields in determining our sensitivity to patterns of light, the way motion is perceived, the way color perception is affected by context, etc. Hence, we have put together contributions from experts in visual perception, visual illusions, visual adaptation, visual motion perception, color perception, visual attention, functional imaging, and electrophysiology of the visual system.

The volume is divided into three parts: I) Gestalt and perceptual organization, II) Attention, aftereffects and illusions, and III) Color vision and art perception.

In Part I we introduce the reader to Gestalt theory and how modern neuroscientific research has impacted on this approach of perceptual organization with a review by Johan Wagemans on Gestalt theory and its development throughout the past century. Importantly, this chapter compares the original observations of the Gestalt school to those made later by means of psychophysics and electrophysiology. This chapter is followed by James Thomas’s historical analysis of the research on receptive/perceptive fields and their relationship to each other and to visual sensitivity. Here, the reader is introduced to the concept of receptive and perceptive fields as the basis for sensory encoding. The Gestalt psychologists were also interested in the dynamic principles underlying perceptual organization. Along these lines, Bruno Breitmeyer further pursues the question of dynamic perceptual processes underlying objects’ contour and surface properties. His chapter describes recent research on visual masking, where a briefly presented stimulus is masked by a subsequently presented stimulus. This work points to the sequential nature of perception that requires time for a percept to be formed. The final two chapters in Part I describe recent research on the concept of object perception and self-organizing principles underlying spatial perception. In his chapter on object perception, Baingio Pinna describes the Gestalt principles underlying figure-ground segmentation and perceptual grouping. The concepts of good Gestalt and stimulus saliency (Prägnanz) are essential in our understanding of object perception. In the chapter by Lewis Harvey and Eric Schmidt we learn about the dynamic interactions among visual elements. According to these authors, perceptual experience is a result of dynamic interactions among stimulus elements and the functional interactive properties of the nervous system. Gestalt principles are used in media, web design and art, the latter being the focus of attention in the final section of this volume.

In Part II we introduce the reader to the research domain of the role of attention in perceptual grouping, to research on perceptual aftereffects of adaptation and to research on perceptual illusions, showing how certain stimulus configurations lead to illusions in brightness and shape. The chapter by Haluk Öğmen and Michael Herzog examines the factors involved in the information selection and the role of attention in perceptual grouping. According to them attention allows the articulation of more differentiated and more detailed perceptual representations that can be non-retinotopic, i.e. spatially distance groups of elements that share the same motion direction and speed are grouped. The succeeding chapter by Mark W. Greenlee and Svein Magnussen reviews research on the technique of spatial adaptation, referred to as the ‘psychophysicist’s microelectrode’. Among other issues, in this chapter the Fourier theory of vision is criticized by showing that adaptation to the higher harmonics of complex spatial waveforms had little or no effect on the detection of that pattern after adaptation. Their results conflict with simple linear summation models of vision and suggest the existence of more complex interactions that determine our sensitivity to complex spatial patterns of light and contrast. Studies on spatial adaptation contribute to our understanding of the mechanism of short-lived plasticity effects as well as to our understanding of the development of spatial channels in the human visual system. The chapter by Kai Hamburger, Erika Dixon and Arthur Shapiro introduces the reader to research on illusory changes in brightness evoked in spatial patterns. They compare brightness illusions evoked by traditional Hermann grids to the so-called Spillmann weaves under different conditions of spatial filtering. The final chapter in Part II by Simone Gori and Alan Stubbs describes recent work on motion illusions as a window into the neural processing of dynamic stimuli. These authors compare illusions of motion in stationary patterns to the misperception of real motion, such as that found for the Barber Pole Illusion. They conclude that our brains are able to process motion directions accurately by relying on hierarchically organized visual cortical areas.

The final part in this volume deals with color perception and the role of color in the perception of art. The underlying brain mechanisms are discussed and the role of color in figure-ground segmentation is investigated. The relationships between art and visual perception was always central in Gestalt psychology – Rudolph Arnheim’s Art and Visual Perception (1954) is a classic monograph, the famous art historian Ernest Gombrich (1960) was heavily influenced by Gestalt psychology and the painter Paul Klee was inspired by patterns from Wertheimer (Teuber, 1976). Since color and its sensation has always been a key issue in art, we start this section with Arne Valberg’s historical analysis of the conflicting theories of Hering and Helmholtz on color perception. Valberg outlines how our understanding of this visual ability has developed over the last two centuries and which pathways in primate vision participate in color vision. He concludes by stating that search for neural correlates to color perception, a promising physiological model of low-level color vision, has come about. We further learn about color qualia as a Gestalt factor and its interactions with figural properties in the chapter by Birgitta Dresp-Langley and Adam Reeves. This chapter contributes to our understanding of the processes of perceptual organization needed to extract ‘qualia’ from complex visual scenes. Finally, Frédéric Devinck, Baingio Pinna and John Werner introduce the reader to the neural foundations of the color assimilation and color contrast effects in scientific investigation as well as in classical and contemporary art. The final chapter by Julian Kallmann, Jürgen Schramme and Christa Neumeyer deals with the phenomenon of color shadows in classical and modern art.

The present volume offers a summary of past and current research on Gestalt principles in perceptual organization with an emphasis on the human visual system. We hope we have been able to capture some of the excitement in this fast moving area of visual research. Perception Beyond Gestalt presents an extensive overview of modern Gestalt principles in the context of contemporary visual neuroscience.

References

Arnheim, R. (1954) Art and Visual Perception: A psychology of the creative eye, Berkeley, CA: University of California Press.

Gombrich, E. H. (1960) Art and Illusion, Princeton, NJ: Princeton University Press.

Hartline, H. K. (1940). The receptive fields of optic nerve fibers. American Journal of Physiology, 123: 690–699.

Hubel, D. H., and Wiesel, T. N. (1959). Receptive fields of single neurones in the cat’s striate cortex. The Journal of Physiology, 148, 574–591.

Jung, R., Baumgarten, R. V., and Baumgartner, G. (1952). Archiv für Psychiatrie und Nervenkrankheiten, vereinigt mit Zeitschrift fur die gesamte [Microleads of single nerve cells in visual cortex of cat: photo-activated neurons]. Neurologie und Psychiatrie, 189(6): 521–539.

Lashley, K. S., Chow, K. L., and Semmes, J. (1951). An examination of the electrical field theory of cerebral integration. Psychological Review, 58(2): 123–136.

Teuber, M. L. (1976) ‘“Blue night” by Paul Klee’, in M. Henlse (ed.) Vision and artifact, New York, NY: Springer, pp. 131–151.

Part I

Gestalt and perceptual organization

2    How much of Gestalt theory has survived a century of neuroscience?

Johan Wagemans

Introduction

Exactly 100 years ago Wertheimer (1912) published his paper on so-called phi motion, which many consider to be the start of Gestalt psychology as an important school of thought. This historical occasion has been the impetus for two extensive reviews of the empirical and theoretical contributions of Gestalt psychology regarding visual perception (Wagemans et al., 2012a, 2012b), and for Lothar Spillmann’s efforts to bring Wertheimer’s seminal work under the attention of contemporary scientists again, by publishing good English translations of the original German papers, along with some additional chapters to clarify their meaning and importance (Spillmann, 2012).

In this chapter, I take another perspective, starting from a narrower focus. The notion of ‘Gestalts’ and the adagio that ‘the whole is different from the sum of its parts’ were two central contributions by Gestalt psychology. However, the golden decades of Gestalt psychology ended before anything was known about neurophysiology in the visual cortex, resulting in a large discrepancy between the classic theories of Gestalt psychologists and current knowledge within the neurosciences. This raises the question as to how much of Gestalt theory has survived the century of research on the neural mechanisms of visual perception. I will present my own view on the history of Gestalt psychology and visual neuroscience, centered on the issue of elementalism versus holism. I will argue for a renewed interest in global processes of cortical dynamics, which could do justice to the original Gestalt position within a modern view of how the brain is working. Because I assume that readers are more familiar with current neuroscience than with the history of Gestalt theory, I will elaborate somewhat more on the older literature than on the more recent literature. However, even then, this is only a sketchy, cursory review; for a more substantial historical perspective, I refer to Ash (1995).

How Gestalt psychology started

What Max Wertheimer discovered was a special case of apparent motion called ‘phi motion’. The conventional view of apparent motion perception maintained that we see an object on several positions successively and something is then added subjectively. If this were correct, then an object would have to be seen moving, and at least two positions, the starting and end points, would be required to produce seen motion. Neither of these conditions held in the case of phi motion. In the key experiment, a white strip was placed on a dark background in each of two slits in the wheel of a tachistoscope, and the rotation speed was adjusted to vary the time required for the light to pass from one slit to the next (i.e. the interval between the two). Above a certain threshold value (~200 ms), observers saw the two lines in succession. With much shorter intervals (~30 ms), the two lines appeared to flash simultaneously. At the optimal stage (~60 ms), observers perceived a motion that could not be distinguished from real motion. When the interval was decreased slightly below 60 ms, after repeated exposures, observers perceived motion without a moving object – that is, pure phenomenal or phi motion. In the same paper, Wertheimer proposed a physiological model described in terms of a short circuit and a flooding back of the current flow, described as ‘transverse functions of a particular kind’ (Wertheimer, 1912/2012: 75), which produced what he called ‘a uniform overall process’ (Wertheimer, 1912/2012: 79). He then extended this theory to the psychology of pure simultaneity (for the perception of form or shape) and of pure succession (for the perception of rhythm or melody). These extensions were decisive for the emergence of Gestalt theory.

Gestalt theory

The phi phenomenon was the perception of a pure process, a transition that could not be composed from more primitive percepts of a single object at two locations. In other words, perceived motion was not added subjectively after the sensory registration of two spatiotemporal events but had its own phenomenological characteristics and ontological status. From this phenomenon, Wertheimer concluded that structured wholes or ‘Gestalten’, rather than sensations, are the primary units of mental life. This was the key idea of the new and revolutionary Gestalt theory, developed by Max Wertheimer and his colleagues Kurt Koffka and Wolfgang Köhler in Berlin.

The notion of ‘Gestalt’ had already been introduced into psychology by Christian von Ehrenfels in his essay ‘On “Gestalt qualities”’ (1890/1988). Based on the observation that humans can recognize two melodies as identical even when no two corresponding notes in them have the same frequency, von Ehrenfels argued that these forms must possess a ‘Gestalt quality’ – a characteristic that is immediately given, along with the elementary sensations that serve as its foundation, a characteristic that is dependent on its constituent objects but rises above them. For von Ehrenfels, Gestalt qualities rest unidirectionally on sense data: wholes are more than the sums of their parts, but the parts are the foundation (‘Grundlage’) of the whole. In contrast, Wertheimer and his colleagues claimed that functional relations determine what will appear as the whole and what will appear as parts (i.e. reciprocal dependency). Often the whole is grasped even before the individual parts enter consciousness. The contents of our awareness are by and large not additive but possess a characteristic coherence. They are structures that are segregated from the background, often with an inner center, to which the other parts are related hierarchically. Such structures or ‘Gestalten’ are different from the sum of the parts. They arise from continuous global processes in the brain, rather than combinations of elementary excitations.

Some early Gestalt history

Two major developments are generally considered as highlights in the history of Gestalt psychology: Köhler’s discussion of ‘physical Gestalten’ (1920/1938) and Wertheimer’s proposal of ‘Gestalt laws of perceptual organization’ (1923/2012). Köhler (1920/1938) extended the Gestalt concept from perception and behavior to the physical world, thus attempting to unify holism (i.e. the doctrine stressing the importance of the whole) and natural science. In addition, Köhler tried to construct a specific testable theory of brain processes that could account for perceived Gestalten in vision. He thought of visual Gestalten as the result of an integrated process in what he referred to as ‘the entire optical sector’, including retina, optical tract and cortical areas, as well as transverse functional connections among conducting nerve fibers (i.e. a recurrent neural network in modern terms). He proposed an electrical field theory, in which

the lines of flow are free to follow different paths within the homogeneous conducting system, and the place where a given line of flow will end in the central field is determined in every case by the conditions in the system as a whole.

(Köhler, 1920/1938: 50)

In modern terms, Köhler had described the visual system as a self-organizing physical system. These ideas led Köhler to postulate a psychophysiological isomorphism between the psychological reality and the brain events underlying it. Experiments to establish the postulated connections between experienced and physical Gestalten in the brain were conducted only decades later (see ‘From speculation to facts’ below).

Around the same time, Max Wertheimer further developed his Gestalt epistemology and outlined the research practice of experimental phenomenology that was based on it. He first stated the principles publically in a manifesto published in Volume 1 of PsychologischeForschung in 1922 (the same journal which is now published as Psychological Research). Wertheimer called for descriptions of conscious experience in terms of the units people naturally perceive, rather than the artificial ones imposed by standard scientific methods. By assuming that conscious experience is composed of elemental units, psychologists constrain themselves to a piecemeal inquiry into the contents of consciousness, building up higher entities from constituent elements, using associative connections. In fact, such and-summations (‘Und-Summe’), as Wertheimer called them, appear ‘only rarely, only under certain characteristic conditions’ (Wertheimer 1922/1938: 13). Rather, what is given in experience ‘is itself in varying degrees “structured” (“gestaltet”), it consists of more or less definitely structured wholes and whole-processes with their whole-properties and laws, characteristic whole-tendencies and whole-determinations of parts’ (Wertheimer 1922/1938: 14). The perceptual field does not appear to us as a collection of disjointed sensations, but possesses a particular organization of spontaneously combined and segregated objects.

In 1923, Wertheimer published a follow-up paper, which was an attempt to elucidate the fundamental principles of that organization. The most general principle was the so-called ‘law of Prägnanz’, stating, in its most general sense, that the perceptual field and objects within it will take on the simplest and most encompassing (‘ausgezeichnet’) structure permitted by the given conditions. For Köhler (1920/1938), this tendency towards the Prägnanz of the Gestalt was just another example that phenomenal Gestalten were like physical Gestalten: as shown by Maxwell and Planck, all processes in physical systems, left to themselves, show a tendency to achieve the maximal level of stability (homogeneity, simplicity, symmetry) with the minimum expenditure of energy allowed by the prevailing conditions. More specific principles that determine perceptual organization according to Wertheimer were proximity, similarity, uniform density, common fate, direction, good continuation and ‘whole properties’ (or ‘Ganzeigenschaften’) such as closure, equilibrium and symmetry.

The focus on the role of organization in perceptual experience was turned into a complex open-ended research program aimed at the discovery of the laws or principles governing perceptual organization in both its static and dynamic aspects. It is this research program that Wertheimer, Koffka and Köhler started to work on with their students, once they had acquired professorships at major universities in Germany in the 1920s and 1930s. This period marked the high point but not the end of Gestalt psychology’s theoretical development, its research productivity, and its impact on German science and culture. Around this time, Gestalt theory also started to have some impact on research in the U.S., mainly owing to Wolfgang Köhler and Kurt Koffka (see King and Wertheimer, 2005)

However, very devastating to the further development of Gestalt psychology was the emergence of the Nazi regime in Germany from 1933 to World War II. In this period, many of the psychology professors at German universities lost their posts because of the discrimination and prosecution of Jews, so they emigrated to the U.S. to take on new positions there. The works by German psychologists who stayed were largely ignored outside Germany. For instance, Metzger’s synoptic account of research on the Gestalt theory of perception entitled Gesetze des Sehens (Laws of seeing), first published in 1936 and later reissued and vastly expanded three times, was only in 2006 translated into English by Lothar Spillmann and his colleagues.

From speculation to facts

After emigrating to the U.S., Köhler, who had taken up physiological psychology, used electroencephalography (EEG) and other methods in an attempt to verify his isomorphism postulate directly. Initially, his results with Hans Wallach on so-called figural aftereffects appeared to support his interpretation in terms of satiation effects of cortical currents (Köhler and Wallach, 1944) Afterwards, he was able to directly measure the cortical currents – as EEG responses picked up from electrodes at the scalp – which flow in directions corresponding to some bright objects moving in the visual field (Köhler and Held, 1949). In all of these experiments, vertically oriented bright bars were moving slowly in the horizontal direction. The occipital electrode showed downward deflections, which were maximal (about 25 to 40 micro-volts) in the middle of the exposures, when the stimuli passed the fixation mark. Remarkably, the deflection began when the object moved at a distance of almost 50 degrees from the fovea, indicating that the cortical areas involved were interrelated by currents across considerable distances. Although these experiments are very simple measurements of visual evoked potentials from a modern perspective, for Köhler, they offered convincing empirical evidence for the existence of the theoretically postulated electrical flows in the visual cortex, accompanying visual percepts.

Encouraged by these findings, Köhler further developed his electrical field theory, arguing that the flow of current is ‘relational’ and that the field of distribution of the current flow is ‘molar’. More importantly, the characteristics of perceptual organization were argued to correspond to the distribution of current, for instance, a dense, continuous flow corresponding to the percept of a figure and the interruption of flow or a sharp intensity gradient corresponding to a contour (Köhler et al., 1952).

End of electrical field theory

Soon after that breakthrough, Lashley and colleagues (Lashley et al., 1951) performed a more critical test of Köhler’s electric field theory (and its underlying postulate of isomorphism). If the flows of current picked up from the scalp in Köhler and Held’s experiments were supposed to reflect the organized pattern of perception and not merely the applied stimulation, and if that pattern of perception would result from a global figure-field across the whole cortex, a marked alteration of the currents should distort visual figures and make them unrecognizable. By inserting metallic strips and metal pins in large regions of the visual cortex of rhesus monkeys, Lashley and co-workers could short-circuit the cortical currents. Surprisingly, the monkeys could still perform the learned shape discriminations, which demonstrated that global cortical currents were not necessary for pattern perception. In subsequent experiments, Sperry and colleagues (Sperry et al., 1955) performed extensive subpial slicing and dense impregnation with metallic wires across the entire visual cortex of cats, and showed that these animals too could still perform rather difficult shape discriminations (e.g. between a prototypical triangle and several different ones with small distortions). Together, these two studies effectively ruled out electrical field theory as an explanation of cortical integration, and therefore removed the empirical basis of isomorphism between cortical flows of current and organized patterns of perception.

Of course, Köhler (1965) reacted to these experiments but his rather convincing counterarguments and suggestions for further experiments were largely ignored, and for most scientists at the time (especially physiological psychologists), the matter was closed and electrical field theory, which was one of the pillars of Gestalt psychology’s scientific basis, was considered dead and buried.

Single-neuron doctrine

At the same time, in the footsteps of Hartline, Kuffler, Jung and others, Hubel and Wiesel (1959, 1962, 1968) began to start mapping the responses of single neurons in lateral geniculate nucleus (LGN), striate and extrastriate cortex in cat and monkey (see Hubel and Wiesel, 2005, for reprints and discussion). These neurons were shown to be tuned to simple stimulus attributes (e.g. orientation, motion direction) and these single neurons were therefore interpreted as ‘feature detectors’ (e.g. line detectors, edge detectors). Following Hubel and Wiesel, single-unit recording flourished in the 1960s, 1970s and 1980s. Studies with this technique revealed the tuning properties of different types of cells in different areas of the brain.

This whole enterprise was a big success (later awarded the Nobel Prize in 1981). It led to what Barlow (1972) called ‘the single neuron doctrine’ of perceptual psychology and visual neuroscience, consisting of two basic principles (if not dogmas):

our perceptions are caused by the activity of a small number of neurons and

the activity of a single neuron is related quite simply to our subjective experience.

Within a few decades, these developments within neurophysiology gave rise to an overwhelmingly reductionist, elementalist approach which the Gestalt theorists had banished. It seemed as if the pendulum had swung completely from elementalism (Wundt’s and Titchener’s structuralism) to holism (Berlin Gestalt theory) and back to elementalism again (single-neuron doctrine).

Further developments

Based on this line of work, the predominant view of the visual system became one of functional specialization and hierarchical organization, with successive stages with gradually decreasing retinotopy, increasing receptive field size (see Chapter 3 by Thomas in this volume), increasing selectivity of neurons, and increasing complexity of the ‘features’ across the sequence of cortical areas in the ‘ventral stream’ (see Maunsell and Newsome, 1987, for a review). In the 1990s, this view was further corroborated by functional imaging studies in humans, leading up to the notion of specialized ‘modules’ and ‘cortical maps’ (e.g. Grill-Spector and Malach, 2004; Op de Beeck et al., 2008). Similar work continued to be conducted in animal neurophysiology (Felleman and Van Essen, 1991) and all of this led to what I would call ‘the standard view’ of visual neuroscience, also shared by computational modelers proposing a predominantly feedforward architecture with successive feature extraction stages (e.g. Serre et al., 2007).

Re-emergence of Gestalt issues

In spite of the overwhelming dominance of the standard view, several single-cell studies in the 1980s and 1990s obtained results that seemed to be more compatible with a more global and interactive view. I briefly discuss four of these studies.

First, Allman and colleagues (1985) recorded from the middle temporal (MT) area in the owl monkey, an area known to be selective for motion. When they stimulated the classical receptive field (cRF) with motion of a random dot pattern in the preferred direction, they obtained the normal inverted U-shaped function for response strength plotted against motion direction (centered on the optimal direction). When they simultaneously stimulated a large surround region with a neighboring random dot pattern, the responses from these target cells in MT strongly depended on the surround stimulation, far beyond their cRF. When the random dots in the surround moved in directions opposite to that of the cRF, responses were enhanced by more than 50 per cent and when they were in the same direction, responses were reduced by more than 50 per cent. Apparently, the strength of the figure-ground segregation affected the neuron’s response profile, indicating that a neuron is not just signaling an absolute (elemental) stimulus property within its receptive field, but rather a relational (Gestalt) property. A single neuron’s activity is thus strongly influenced by its neighbors, raising doubts to the very notion of a receptive field (since that appears to stretch far beyond the area in the visual field from which the neuron can normally be stimulated with isolated stimuli).

Second, in a remarkable Science paper, von der Heydt and colleagues (1984) showed that cells in area V2 in the visual cortex of the macaque monkey responded not only to physical line, bar or edge stimuli (as shown by Hubel and Wiesel) but also to so-called illusory (or subjective) contours. When two line segments were arranged in such a way that they were perceived as one single continuous line (with the gap between them being filled-in perceptually), the firing of typical V2 cells was almost half as strong as with an actual physical line, despite the fact that their receptive field was not actually stimulated by any physical discontinuity (no actual line or edge present). When the line segments were slightly repositioned so that an illusory contour was not perceived, the firing rates dropped to chance levels. A similar finding was obtained with an illusory edge of a white rectangle. When a vertical illusory contour was created by two abutting patterns consisting of horizontal line segments, the response strength even doubled at the target (vertical) orientation compared to an actual line, and the response function degraded smoothly away from vertical, as with normal orientation tuning functions. Although we may now find it quite natural to establish a correspondence between a perceptual attribute and a cell’s tuning properties, the fact that this tuning concerned a purely phenomenal, purely subjective characteristic which arises from the processing of the configuration of the whole stimulus array, and not just a small, local, physical stimulus attribute, was quite striking at the time (see Chapter 9 by Hamburger and colleagues in this volume).

Third, in one of the interesting follow-up studies from the same laboratory (Zhou et al., 2000), responses of these V2 cells were shown to be modulated strongly by the context within which a borderline was embedded. When the same light-to-dark edge was part of a light shape on a dark background, for instance, the response could be twice as strong than when it was part of a dark shape on a light background. These response rates could easily be inverted for the same polarity, same receptive field and same cell, when the surround was configured differently, implying a different figure-ground relationship. This pattern of results clearly showed that these cells code for the so-called border-ownership of an edge, not just for the edge as such. Hence, these cells are not local edge detectors (as previously assumed), but they code for the specific interpretation of a localized fragment, depending on the global configuration surrounding the receptive field, including the figure-ground assignment, which is determined by relatively remote parts of the image. This indicates that these cells receive and integrate signals from far-away cells, probably even higher up in the stream of processing.

Fourth, the influence of context and the role of feedback on responses of neurons in low-level visual areas, even V1, were demonstrated convincingly in work by Lamme and colleagues (e.g. Lamme, 1995). When presenting macaque monkeys with dense textures of oriented line segments and recording from the cRF of their V1 cells, the firing rates for optimal and no-optimal texture orientations differed already around 50 ms after stimulus onset. When comparing the responses for RFs on the figure boundary with those on the background, the difference became visible after about 75 ms. For the same position in the visual field, a cell whose cRF was on a region which was seen as part of the figure could fire more vigorously than when the same region belonged to the background, and this difference emerged only after 100 ms following stimulus onset. This pattern of results suggests that a cell’s local responsiveness is not only influenced by the context within which its cRF is embedded (as shown in the studies discussed before), but also that a cell’s response profile can change over time, from a strictly local, absolute encoding of an elemental stimulus attribute, to a more relational, perceptual attribute, with a gradual filling-in from a figure-ground boundary to the interior of a figure. All of this is very suggestive for the role of top-down feedback on the activity of these low-level neurons.

Larger implications: a modern view on a radical vision

I think it should be obvious that a simple, localist, elementalist view on single neuronal activity can no longer be maintained in the light of the evidence from studies like these. Clearly, cells in the visual cortex encode stimulus attributes in a more global, configural way, and their activity results as much from top-down influences from higher level areas as from purely bottom-up encoding of the incoming input images. A more dynamic, interactive view is needed to understand how the visual cortex is giving rise to meaningful percepts. As argued before (e.g. Ehrenstein et al., 2003; Spillmann, 1999, 2009), neuroscientists should try to get beyond their spontaneous, implicit perspective on neuronal activity, which is often reductionist and localist, and should try to do justice to the Gestalt point of view. Specific models for figure-ground segregation (e.g. Roelfsema et al., 2002), surface completion, border-ownership and subjective contours (e.g. Kogo et al., 2010) are now being formulated and tested, which do justice to the neural evidence as well as to a more interactive and dynamic perspective. I would even argue that something like a modern reinterpretation of the Gestalt view is emerging in the recent literature. I give two examples of these modern Neo-Gestaltist views on cortical processing.

First, Hochstein and Ahissar (2002) proposed Reverse Hierarchy Theory, in which they argue that a fast feedforward sweep quickly activates global percepts (e.g. the overall gist of a scene) in high-level areas with large receptive fields, whereas feedback from these higher areas to lower areas and recurrent processing in the low-level areas with small receptive fields is necessary for fine-grained processing of local details (e.g. a difficult visual search task). Hence, this theory distinguishes the anatomical, structural aspects of the hierarchy of the visual system (low-versus high-level representations) from the temporal, functional aspects of it (early versus late stages of processing, see Chapter 4 by Breitmeyer in this volume). Second, Bar and colleagues (e.g. Bar, 2003; Bar et al., 2006) have proposed a general theory of visual object recognition, in which two streams of processing initially occur in parallel, one in which the high spatial frequencies of input images are processed in a relatively slow, feedforward sequence according to the standard hierarchy of stages (V1, V2, V4, …), and another one in which the low spatial frequencies are quickly transmitted to areas in the prefrontal cortex to generate hypotheses about possible identities of the object as well as their most likely scene context, after which these two streams are integrated and refined in an iterative, re-entrant way.

Both of these views are very Gestalt-like in spirit because they both postulate that global processing comes first, both are highly interactive and dynamic views on cortical processing, and they are not even limited to purely visual areas (cf. Köhler’s notion of ‘the entire optical sector’). Indeed, Bar has later extended this view to include cognitive interpretation and emotional significance (e.g. Bar, 2007; Barrett and Bar, 2009).

Remaining challenges

Although I think it is fair to conclude that the pendulum has turned away again from a strictly reductionist, elementalist position (à la Hubel and Wiesel) back in the more globalist, Gestalt-like direction, it is not the case that neuroscience is completely holistic now. On the contrary, modern vision science appears to be incommensurate with ‘the fundamental “formula” of Gestalt theory’ as postulated by Wertheimer: ‘There are wholes, the behavior of which is not determined by that of their individual elements, but where the part-processes are themselves determined by the intrinsic nature of the whole’ (Wertheimer, 1924/1938: 2).

Like the Gestaltists’ rivals in the early days, much of contemporary science is analytic rather than holistic. Even when configural effects are shown to occur time and time again, and even when strictly bottom-up models are replaced by more realistic models emphasizing feedforward–feedback loops and re-entrant processing (e.g. Hochstein and Ahissar, 2002; Kogo et al., 2010; Lamme, 1995; Roelfsema et al., 2002), this is still a far cry from recognizing the primary nature of structured wholes in experience, the importance of two-sided dependency between parts and wholes, and global field dynamics. In fact, Gestalt phenomena are still not very well integrated into mainstream thinking about the visual system’s operating principles (e.g. selectivity and tuning of single cells, V1 as a bank of filters or channels, segregated ‘what’ and ‘where’ streams, increasing receptive field size and invariance at higher levels of the hierarchy, specialized modules for separate object categories such as faces and places). Or, formulated more positively, establishing such an integration continues to provide serious challenges.

I am personally strongly convinced that a more fully dynamic and holistic view on cortical processing is called for, not like Köhler’s naïve idea of electrical field theory (which was proven wrong), but doing justice to what we now know about complex adaptive systems and cortical dynamics, including well-established phenomena such as synchronization, coherence intervals and travelling waves (e.g. Nikolaev et al., 2010; van Leeuwen, 2007).

Summary/conclusion

Gestalt theory as developed in the Berlin school (Wertheimer, Köhler, Koffka) was a radical vision. Arguing against elementalism and associationism, Gestalt theory maintained that experienced objects and relationships are fundamentally different from collections of sensations. Going beyond the notion of Gestalt qualities proposed by von Ehrenfels, which involved one-sided dependence on sense data, true Gestalts are dynamic structures in experience that determine what will be wholes and parts, figure and background. From the beginning, this two-sided dependence was believed to have a neural basis. Already in 1912, in his seminal paper on phi motion, Wertheimer hypothesized some kind of physiological short circuit, and an electric flow theory. In 1920, Köhler stated that the brain events underlying perception follow a dynamic, self-organizing principle, which he likened to the tendency of physical systems to approach maximum order with minimum expenditure of energy. After some apparently successful attempts to provide empirical support (e.g. measurements of cortical currents in a Science paper in 1949), the electrical field theory was considered rejected by famous experiments by Lashley and Sperry in the 1950s. When Hubel and Wiesel discovered simple cells and complex cells in primary visual cortex shortly thereafter, using single-unit recordings, neuroscience went almost exclusively in the direction of a reductionist, elementalist approach which the Gestalt theorists had banished.

A century later, Gestalt phenomena are of central interest again in modern neuroscience but how to understand them within our current views on the visual system’s hierarchy? Can we decode Gestalts in the neural signals somewhere along the stream, or do they emerge in the interplay between bottom-up processing, lateral connections and feedback processing? Or do we need to look for traveling waves across the whole brain or for synchronization of EEG activity between brain areas? These are important questions for a contemporary approach to ‘Gestalts in the brain’.

Acknowledgments

I am supported by long-term structural funding from the Flemish Government (METH/08/02). This chapter was written during my sabbatical, with support from the Research Foundation–Flanders (K8.009.12N). I would also like to acknowledge the hospitality of the Department of Psychology at the University of California, Berkeley, the ‘Institut d’études avancées’ (IEA), Paris and the Department of Experimental Psychology at the University of Oxford.

References

Allman, J., Miezin, F., and McGuinness, E. (1985) ‘Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT)’, Perception, 14: 105–126.

Ash, M. G. (1995) Gestalt psychology in German culture 1890–1967: Holism and the quest for objectivity, Cambridge: Cambridge University Press.

Bar, M. (2003) ‘A cortical mechanism for triggering top-down facilitation in visual object recognition’, Journal of Cognitive Neuroscience, 15: 600–609.

Bar, M. (2007) ‘The proactive brain: Using analogies and associations to generate predictions’, Trends Cognitive Sciences, 11: 280–289.

Bar, M., Kassam, K. S., Ghuman, A. S., Boshyan, J., Schmid, A. M., Dale, A. M., Hämäläinen, M. M., Marinkovic, K., Schacter, D. L., Rosen, B. R., and Halgren, D. J. (2006) ‘Top-down facilitation of visual recognition’, Proceedings of the National Academy of Science of the USA, 103: 449–454.

Barlow, H. (1972): ‘Single units and sensation: A neuron doctrine for perceptual psychology’, Perception, 1: 371–394.

Barrett, L. F., and Bar, M. (2009) ‘See it with feeling: Affective predictions in the human brain’, Philosophical Transactions of the Royal Society, Series B, 364: 1325–1334.

Ehrenstein, W. H., Spillman, L., and Sarris, V. (2003) ‘Gestalt issues in modern neuroscience’, Axiomathes, 13: 433–458.

Felleman, D. J., and Van Essen, D. C. (1991) ‘Distributed hierarchical processing in the primate cerebral cortex’, Cerebral Cortex, 1: 1–47.

Grill-Spector, K., and Malach, R. (2004) ‘The human visual cortex’, Annual Review of Neuroscience, 27: 649–677.

Hochstein, S., and Ahissar, M. (2002) ‘View from the top: Hierarchies and reverse hierarchies in the visual system’, Neuron, 36: 791–804.

Hubel, D. H., and Wiesel, T. N. (1959) ‘Receptive fields of single neurons in the cat’s striate cortex’, Journal of Physiology, 148: 574–591.

Hubel, D. H., and Wiesel, T. N. (1962) ‘Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex’, Journal of Physiology, 160: 106–154.

Hubel, D. H., and Wiesel, T. N. (1968) ‘Receptive fields and functional architecture of monkey striate cortex’, Journal of Physiology, 195: 215–243.

Hubel, D. H., and Wiesel, T. N. (2005) Brain and visual perception: The story of a 25-year collaboration, New York, NY: Oxford University Press.

King, D. B., and Wertheimer, M. [Michael] (2005) Max Wertheimer and Gestalt theory, New Brunswick, NJ: Transaction Publishers.

Kogo, N., Strecha, C., Van Gool, L., and Wagemans, J. (2010) ‘Surface construction by a 2-D differentiation-integration process: A neurocomputational model for perceived border ownership, depth, and lightness in Kanizsa figures’, Psychological Review, 117: 406–439.

Köhler, W. (1920) Diephysischen Gestalten in Ruhe und im stationären Zustand. Eine natur-philosophische Untersuchung. Braunschweig, Germany: Friedr. Vieweg und Sohn. Translated extract reprinted as: ‘Physical Gestalten’ in W. D. Ellis (ed.) (1938) A source book of Gestalt psychology, London, UK: Routledge and Kegan Paul Ltd., pp. 17–54.

Köhler, W. (1965) ‘Unsolved problems in the field of figural after-effects’, Psychological Record, 15: 63–83.

Köhler, W., and Wallach, H. (1944) ‘Figural after-effects: An investigation of visual processes’, Proceedings of the American Philosophical Society, 88: 269–357.

Köhler, W., and Held, R. (1949) ‘The cortical correlate of pattern vision’, Science, 110: 414–419.

Köhler, W., Held, R., and O’Connell, D. N. (1952) ‘An investigation of cortical currents’, Proceedings of the American Philosophical Society, 96: 290–330.

Lamme, V. A. F. (1995) ‘The neurophysiology of figure-ground segregation in primary visual cortex’, Journalof Neuroscience, 15: 1605–1615.

Lashley, K. S., Chow, K. L., and Semmes, J. (1951) ‘An examination of the electrical field theory of cerebral integration’, Psychological Review, 58: 123–136.

Maunsell, J. H. R., and Newsome, W. T. (1987) ‘Visual processing in monkey extrastriate cortex’, Annual Reviewof Neuroscience, 10: 363–401.

Metzger, W. (1936) Gesetzedes Sehens. Frankfurt am Main, Germany: Kramer, Translated as Laws of seeing (2006) by L. Spillmann, M. Wertheimer, and S. Lehar, Cambridge, MA: MIT Press.

Nikolaev, A. R., Gepshtein, S., Gong, P., and van Leeuwen, C. (2010) ‘Duration of coherence intervals in electrical brain activity in perceptual organization’, Cerebral Cortex, 20: 365–382.

Op de Beeck, H. P., Haushofer, J., and Kanwisher, N. (2008) ‘Interpreting fMRI data: Maps, modules, and dimensions’, Nature Reviews Neuroscience, 9: 123–135.

Roelfsema, P. R., Lamme, V. A. F., Spekreijse, H., and Bosch, H. (2002) ‘Figure-ground segregation in a recurrent network architecture’, Journal of Cognitive Neuroscience, 14: 525–537.

Serre, T., Oliva, A., and Poggio, T. (2007) ‘A feedforward architecture accounts for rapid categorization’, Proceedings of the National Academy of Science of the USA, 104: 6424–6429.

Sperry, R. W., Miner, N., and Myers, R. E. (1955) ‘Visual pattern perception following subpial slicing and tantalum wire implantations in the visual cortex’, Journal of Comparative and Physiological Psychology, 48: 50–58.

Spillmann, L. (1999) ‘From elements to perception: Local and global processing in visual neurons’, Perception, 28: 1461–1492.

Spillmann, L. (2009) ‘Phenomenology and neurophysiological correlations: Two approaches to perception research’, Vision Research, 49: 1507–1521.

Spillmann, L. (ed.) (2012) On perceived motion and figural organization. Cambridge, MA: MIT Press.

van Leeuwen, C. (2007) ‘What needs to emerge to make you conscious?’ Journal of Consciousness Studies, 14: 115–136.

von der Heydt, R., Peterhans, E., and Baumgartner, G. (1984) ‘Illusory contours and cortical neuron responses’, Science, 224: 1260–1262.

von Ehrenfels, C. (1890) ‘Über “Gestaltqualitäten”’. Vierteljahrsschrift fur wissenschaftliche Philosophie, 14: 224–292. Translated as ‘On “Gestalt qualities”’ in B. Smith (ed.) (1988) Foundations of Gestalt theory, Munich, Germany/Vienna, Austria: Philosophia Verlag, pp. 82–117.

Wagemans, J., Elder, J. H., Kubovy, M., Palmer, S. E., Peterson, M. A., Singh, M., and von der Heydt, R. (2012a) ‘A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization’, Psychological Bulletin, 138: 1172–1217.

Wagemans, J., Feldman, J., Gepshtein, S., Kimchi, R., Pomerantz, J. R., van der Helm, P., and van Leeuwen, C. (2012b) ‘A century of Gestalt psychology in visual perception: II. Conceptual and theoretical foundations’, Psychological Bulletin, 138: 1218–1252.

Wertheimer, M. (1912) ‘Experimentelle Studien über das Sehen von Bewegung’, Zeitschrift für Psychologie, 61: 161–265. Translated as ‘Experimental studies on seeing motion’ in L. Spillmann (ed.) (2012) On perceived motion and figuralorganization, Cambridge, MA: MIT Press, pp. 1–91.

Wertheimer, M. (1922) ‘Untersuchungen zur Lehre von der Gestalt, I: Prinzipielle Bemerkungen’, Psychologische Forschung, 1: 47–58. Translated extract reprinted as ‘The general theoretical situation’ in W. D. Ellis (ed.) (1938) A sourc ebook of Gestalt psychology, London, UK: Routledge and Kegan Paul Ltd., pp. 12–16.

Wertheimer, M. (1923) ‘Untersuchungen zur Lehre von der Gestalt, II’, Psychologische Forschung, 4: 301–350. Translated as ‘Investigations on Gestalt principles, II’ in L. Spillmann (ed.) (2012) On perceived motion and figural organization, Cambridge, MA: MIT Press, pp. 127–182.

Wertheimer, M. (1924) ‘Über Gestalttheorie’. Lecture delivered to the Kant-Gesellschaft, Berlin, December 1924. Translated extract reprinted as ‘Gestalt theory’ in W. D. Ellis (ed.) (1938) A source book of Gestalt psychology, London, UK: Routledge and Kegan Paul Ltd., pp. 1–11.

Zhou, H., Friedman, H. S., and von der Heydt, R. (2000) ‘Coding of border ownership in monkey visual cortex’, Journal of Neuroscience, 20: 6594–6611.

3    Perceptive fields and receptive fields

James P. Thomas

Introduction

The receptive field and its psychophysical analog, labeled perceptive field by Spillmann and colleagues (Jung and Spillmann, 1970; Spillmann, 1971; Spillmann et al., 1987), are fundamental concepts in the study of visual processing. This chapter examines the relationship between receptive and perceptive fields in the context of their historical development and with particular attention to the related assumptions that have been made about the nature of visual processing and representations.

Sherrington (1906) used the term receptive field to refer to the region of skin that elicited a spinal reflex when stimulated. In modern usage the term refers to the region of stimulus space that modulates the activity of a particular neuron. The activity of the neuron is measured more or less directly, typically (for the research discussed here) as reflected in spike impulses. Following Yeh et al. (2009), this chapter defines visual space broadly to include the spatial, Fourier, spectral, and temporal domains.

The term perceptive field refers to the visual space that affects a particular perceptual response and is measured psychophysically. As discussed below, the field is an inferred entity that depends upon assumptions about visual processing. The goal is often that the perceptive field provides an estimate of a corresponding receptive field. For example, for many years the best estimates of the spectral sensitivities of cones came from psychophysical research (Smith and Pokorny, 1975; Stiles, 1959). However, the perceptive field often reflects a complex of neural activity rather than the properties of a single neuron. Nevertheless, there are major parallels, in concept and empirical findings, between receptive and perceptive fields. This chapter discusses those parallels.

The sections of this chapter discuss:

a conceptual schema that provides a context for perceptive fields and their measurements

and into which receptive fields can also be fitted;

similarities in the conceptual architectures of receptive and perceptive fields;

parallel historical evolution of thinking and findings about the two types of fields;

quantitative comparisons between the fields.

A general schema

Perceptive fields are measured in the context of some task, generally detection of a target or discrimination between two or more stimuli. A general model posits four components:

the stimulus;

a battery of transformation operations that yield

the representation of the stimulus at the decision level;

the decision algorithm (for a formal example of this type of model see Abbey and Eckstein, 2002).

Perceptive fields are the transforms that link the stimulus to its representation at the decision level. In one approach (e.g. Blackwell, 1963; Graham et al., 1939; Spillmann et al., 1987; von Bekesy, 1960), the link between stimulus and representation is convolution of the former with a linear filter or spread function described by the weighting function of the perceptive field. The decision process operates on this representation, for example by finding the maximum and comparing it to a critical value or threshold (Blackwell, 1963). In another approach, there are multiple representations, each produced by convolution with a different filter or perceptive field function. The decision process examines each representation for a critical feature, e.g. judging a stimulus present if threshold is reached in one or more representations (Sachs et al., 1971). A third approach drops the neural image concept and treats each neural pathway as a detector, modeled as a linear filter followed by a monotonic transform yielding a single value as the detector response. The perceptive field is the weighting function of the filter. Each local region of the visual field is represented by multiple detectors with differently tuned filters (Thomas, 1970; Watson, 1983; Wilson and Bergen, 1979). The stimulus is represented at the decision level by a population of detector responses. The decision process calculates some statistic that determines the judgment [e.g. the max (Hines, 1976) or weighted sum (Abbey and Eckstein, 2002; Thomas et al., 1982)].

In the context of this schema, receptive fields are the transforms that operate on the stimulus, creating a representation in the form of neural impulses transmitted to the next level of processing. Only in the case that the same single neuron determines the perceptual response on each trial can the receptive and perceptive fields be identical. Even when it is the single maximum neural response that determines the percept, noise will cause different neurons to dominate from trial to trial, and the perceptive field will be some combination of their receptive fields.

Parallel conceptual architectures

The conceptual architectures of perceptive and receptive fields share many properties. The most basic is that they gather multiple sensory inputs and yield a single output. For a neuron in the visual system, the receptive field maps the points in visual space that modulate the instantaneous response of the neuron. Similarly, a perceptual field is the map of points in visual space that influence a particular perceptual event.

A second common property is summation of inputs within the receptive field. Hartline (1940a, b) established this property for the receptive fields of frog optic nerve fibers. He demonstrated that when multiple elements of the receptive field were illuminated simultaneously, their stimulating effects summed. Subsequent investigators have confirmed this property in a wide variety of species. Ricco’s law, that the product of area and intensity is constant at the detection threshold, expresses summation as a property of perceptual fields.

A third property, widely adopted in both receptive field and perceptive field models, is that of a linear filter. In these models, the filter output is a weighted sum of the stimuli delivered to different elemental regions of the field. A monotonic (and usually nonlinear) function relates the neural or perceptual response to the filter output. The weighting function quantifies the field map and is the formal description of the field. Helmholtz (1962) introduced the filter model in his treatment of color vision, modeling each color receptor as taking a weighted sum of spectral inputs and generating its specific color sensation whose intensity is a monotonic function of the weighted sum. Rodieck (1965) provided a formal statement of the filter model as a description of the receptive field of visual neurons. Subsequently, many psychophysical investigators of spatial vision adopted versions of the filter model (e.g. Blackwell, 1963; Graham et al., 1939; Mostavi and Sackrison, 1976; Thomas et al., 1982; Wilson and Bergen, 1979; von Bekesy, 1960).

Although these concepts undoubtedly reflect only a part of what happens in a neuron (Branco, 2011; Bullock et al., 2005) or in perception, they have proved useful in the study of visual processing. The range of their usefulness is limited, however. Yeh et al. (2009) argue that the receptive field model, as a fixed weighting function applied to visual space, loses its usefulness above the input layers of V1 cortex. Similar limitations must also occur for perceptual fields as tasks become more complex. For example, the spectral sensitivity functions of cones well explain color matches between isolated disks of light, but prove inadequate when it comes to predicting perceived colors in more complex scenes (Land, 1959).

Parallel historical developments

Concepts of receptive and perceptive fields evolved in a parallel and closely interrelated fashion. In no small part this relationship reflected the efforts of psychophysicists to model perceptive fields after the emerging knowledge of receptive fields.

Hartline (1940a, b) pioneered the study of visual receptive fields, recording nerve impulses of single optic nerve fibers in the frog and alligator. He established several basic properties that influenced subsequent conceptualizations of both receptive fields and perceptive fields. Activity in each fiber is affected by many receptor cells that lie within a fixed region of the retina, the receptive field. The spatial location of the receptive field of a given fiber is fixed, but varies from one fiber to another. The fields of different fibers overlap extensively. By comparing responses to a large stimulus patch with responses to subdivisions of the patch, he established that spatial summation occurs within the receptive field of a fiber. The effectiveness of stimulation in activating the fiber is greatest in the center of the field, falling off toward the edges of the field. Although Hartline did not make the point, his data support the proposition that the spatial summation is approximately linear. Anticipating later findings, Hartline noted on-, off-, and on-off-responses, but not any effect of these differences in response on how the receptive field was organized.

Early models of perceptive fields incorporated Hartline’s findings (Blackwell, 1963; Graham et al., 1939). In these models, detection of the stimulus is determined by the weighted sum of the luminances of the elements of the stimulus, the weights becoming smaller with distance from the center of the stimulus. Weighting functions were derived from functions relating luminance thresholds to the diameters of disk stimuli. Each weighting function can be interpreted as the spatial sensitivity map of its perceptive field.

Following World War II, investigators exploited wartime advances in electronics to make more refined studies of retinal ganglion cells and their receptive fields (Barlow, 1953; Kuffler, 1953). Using microelectrodes, they recorded spike responses from cells in the intact eyes of frogs and cats. They used stimuli of different sizes and shapes, including small spots of light that were moved about the receptive field to systematically explore its sensitivities and response properties. These studies confirmed Hartline’s findings and added new ones. The receptive fields of most ganglion cells were found to consist of antagonistic center and surround regions, light falling in one tending to suppress response to light falling in the other. More information was gathered about on- and off-responses and their relationship to the center-surround organization of receptive fields. Some cat ganglion cells, X-cells, were found to be linear, in the sense that cell response was a function of a linear weighted sum of the illuminances of local elements of the receptive fields, weights for center and surround having opposite signs (Enroth-Cugell and Robson, 1966). Other cells, Y-cells, could not be so modeled.

Rodieck (1965) formulated an influential model of ganglion cell receptive fields. The model contains two separable parts: a two-dimensional spatialsensitivity map of the receptive field and a temporal profile of the response of each local element to a step change in illumination. To capture center-surround organization, he proposed that the sensitivity map is the difference between two concentric, circularly symmetrical sensitivity functions, both having values that decline with distance from the center, and with one function having a wider diameter and lower peak value than the other. This configuration yields a map that has positive values in the center and negative values in the surround and what is informally called a Mexican hat profile. To facilitate computation, Rodieck proposed that these two functions are bivariate Gaussians differing in variance and peak values. Enroth-Cugell and Robson (1966) provided extensive evidence that X-cells, but not Y-cells, are well fit by this model. The Difference of Gaussians, DOG, became widely used as both stimulus format and a basic function for the description of receptive fields.

The findings of antagonistic center-surround organization in the receptive fields of retinal ganglion cells were soon followed by analogous findings with respect to perceptive fields. Traditional studies of spatial summation, measuring detection threshold as a function of test stimulus size, had failed to reveal such antagonism, and investigators turned to other psychophysical paradigms and stimulus configurations. Von Bekesy (1960), building on the work of Mach, analyzed viewers’ drawings of Mach bands and related perceptual phenomena. Applying a linear model, he concluded that the perceived distributions of lightness and darkness resulted from convolution of the stimuli with a perceptive field having an excitatory center and inhibitory surround.

In a related approach, Spilllmann (Spillmann, 1971; Spillmann et al., 1987) derived perceptive fields by analysis of the Hermann grid illusion. This illusion occurs when the observer views a grid of white intersecting bars on a black background or a grid of black intersecting bars on a white background. Faint grey spots are seen at the intersections of the white bars and faint light spots are seen at the intersections of the black bars. Spillmann attributed these spots to the operation of perceptive fields with excitatory centers and inhibitory surrounds. By manipulating the widths and lengths of the intersecting bars, Spillmann estimated the diameters of perceptive field centers and their overall widths.

Westheimer demonstrated in 1967 center-surround antagonism by varying the diameter of an adapting stimulus rather than the test target. The test was a small spot of light superimposed on the center of a disk of light that served as the adapting stimulus. As the diameter of the adapting disk increased, the luminance threshold for the test first increased and then declined. He interpreted these changes in the adapting effect of the background as reflecting spatial summation within a central excitatory area, followed by inhibitory interactions from surrounding areas.

Other investigators used inducing stimuli to demonstrate center-surround antagonism. In one type of experiment the visibility of a narrow test line, as indicated by luminance thresholds or lightness/darkness matches, was measured as the proximity or width of neighboring inducing fields varied (Hines, 1976; Thomas, 1968; Thomas et al., 1968).

All of the foregoing studies, using foveally viewed small spots or narrow lines as test stimuli, yielded similar antagonistic center-surround perceptive field organizations, with excitatory centers a few minutes of visual angle wide and inhibitory surrounds extending 10 or more minutes beyond. However, the results in several studies led the authors to propose that there are multiple perceptive fields, differing in size, at each retinal location (Hines, 1976; Thomas et al., 1968; Troscianko, 1982). Similarly, Enroth-Cugell and Robson (1966), comparing contrast sensitivity functions of cat ganglion cells and of human observers, suggested that the central human retina contained receptive fields with a wide range of sizes.

Consistent with the dominance of linear models, the 1960s saw increasing use of Fourier concepts and methods to the study of both receptive fields (Enroth-Cugell and Robson, 1966) and perceptive fields (Campbell and Robson, 1968). Sinusoidal gratings and Gabor patches came into wide use as stimuli and a key measure became the contrast sensitivity function (CSF): the reciprocal of the contrast threshold for detection of a grating as a function of its spatial frequency. The contrast threshold is the contrast required to elicit a criterion number of spikes in the case of a neuron, or a criterion level of detection, in the case of an observer. Given a linear model, such as Rodieck’s, the CSF is the Fourier transform of the weighting function of the receptive or perceptive field. The antagonistic center-surround organization is reflected as band-pass sensitivity in the CSF, and the peak and bandwidth of the pass-band are basic Fourier parameters of both types of field.

Multiple pathways

By the mid-twentieth century, it was well accepted that each small region of the retina contains multiple classes of receptors and neurons with different tuning and response characteristics. Except in the central fovea, there are mixtures of rods and of cones of three different types. Beginning in the 1950s, microelectrode studies led to the identification of more classes, including on- and off-cells, X- and Y-cells, and the analogues of the latter, parvo- and magno-cells (see review by Schiller, 1986). The distinctions among the classes are preserved in the pathways from eye to brain, with the consequence that each small region of the visual field has multidimensional representation in the brain. As microelectrode studies advanced to single cells in the visual cortex, new increases in the dimensionality of the representation were discovered and explored with recordings from cells and with psychophysical studies of corresponding perceptual fields.

In the 1960s, Hubel and Wiesel (1962, 1965, 1968) published a series of papers on neurons in primary visual cortex that altered concepts of how the brain processes visual information. The receptive fields of cortical neurons differ in shape and organization from those of retinal ganglion cells and lateral geniculate body cells:

cortical fields are oblong and oriented;

consist of antagonistic parallel, abutting strips;

the number and width of the strips and their orientation vary greatly from one cell to another;

antagonism is more evenly balanced, with the result that cells do not respond at all to uniform illumination and are selective with respect to the patterns of modulation to which they do respond;

perhaps most striking, each small area of the retina or visual field is monitored by multiple cells whose receptive fields vary widely in these properties and, hence, in the domains of stimuli to which they respond (DeValois and DeValois, 1988).

Psychophysical demonstrations of multiple perceptive fields followed. As noted above, the experiments of several investigators led them to propose the existence of multiple perceptive fields at a given location in the visual field. Thomas and colleagues, summarized in Thomas (1970), demonstrated multiple patterns of spatial summation in central vision and proposed that the traditional spatial summation function, showing the change in increment threshold as target diameter increased, is the envelope of summation functions for a range of perceptive fields, tuned to different sizes. Thomas also proposed a multiple pathway explanation of Mach bands, later elaborated (Fiorentini et al., 1990).

At much the same time, investigators using grating stimuli developed evidence for multiple mechanisms with differing band-pass sensitivities in the Fourier domain. Campbell and Robson (1968) measured contrast detection thresholds for single sinusoids, square wave, and triangular waves as a function of the spatial frequency of the fundamental component. Under a single mechanism (or single ‘channel’) model, thresholds for the square and triangular waves are well predicted by convolution of each with the CSF obtained with simple sinusoids. This expectation was not confirmed. Instead, there was evidence that the fundamental and third harmonic components of the complex waves are separately and independently detected by mechanisms with their own, non-overlapping CSFs. Campbell and Robson concluded:

Each filter and detector would constitute a separate ‘channel’ and each channel would have its own contrast-sensitivity function. On this basis, the envelope of the contrast-sensitivity functions of all the channels would be the contrast-sensitivity function of the overall visual system.

(Campbell and Robson, 1968: 564)

Sachs and colleagues (1971) followed up using complex gratings comprised of two sinusoids whose frequencies and amplitudes were independently varied. The design of the study enabled quantitative tests of whether detection of the complex grating was determined by convolution of a single CSF with the sum of the two stimulus components, as a single mechanism model would predict, or whether the two components were separately and independently detected, as a multiple mechanism model would predict, or whether some intermediate process obtained. Their results supported the multiple mechanism model when the components of the grating differed in spatial frequency by a factor of about two or more.

Blakemore and Campbell (1969) used selective adaptation to demonstrate the existence of multiple mechanisms with band-pass sensitivity in the Fourier domain. Adapting to a high contrast sinusoid of one spatial frequency decreased the sensitivity to subsequently viewed gratings, but only if the frequencies of the latter were within about an octave of the adapting grating (see Chapter 8 by Greenlee and Magnussen in this volume). When the frequency of the adapting grating was varied, the dip in sensitivity shifted along the spatial frequency axis, remaining centered on the adapting frequency. Blakemore and Campbell estimated that each tuned mechanism had a half-amplitude bandwidth of about one octave.

Legge and Foley (1980) used masking to demonstrate multiple, differently tuned mechanisms and estimated their bandwidths with respect to spatial frequency. They modeled detection as mediated by local mechanisms with halfamplitude half-bandwidths of about one octave.

Although the estimates of perceptive field bandwidths varied significantly from one study and methodology to another, the estimated half-amplitude full bandwidths were on the order of one to two octaves for spatial frequency and 10 to 20 degrees for orientation (Olzak and Thomas, 1986).

Reverse correlation and classification images

Reverse correlation uses noise patterns varying over time and space to estimate the spatiotemporal sensitivity function of a neuron (Ringach and Shapley, 2004). Typically, the noise is presented in a sequence of display frames, the spatial pattern varying randomly from frame to frame. Depending on the goals of the experiment, the noise pattern can consist of random pixel luminances or more constrained variation, such as a sequence of sinusoidal gratings varying in spatial frequency and phase (Bredfeldt and Ringach, 2002). Each time the neuron spikes, each of the preceding noise patterns is recorded as a function of the temporal delay between its presentation and the occurrence of the spike. After many sequences, the noise patterns at each delay are averaged, and the distribution of averages over time is modeled as the spatiotemporal sensitivity function of the neuron’s receptive field. In the case of most V1 neurons, the function cannot be factored into separable spatial and temporal components. Ringach and Shapley (2004) suggested that each V1 neuron is multiplexing information, signaling somewhat different information as its response evolves.

Classification images are an analogous psychophysical procedure (Ahumada, 2002; Ahumada and Lovell, 1971). Stimuli to be judged are imbedded in spatiotemporal noise that varies from trial to trial. The noise patterns are aggregated over trials and analyzed to determine which components of the noise are systematically associated with one or the other of the observer’s responses. These components can be interpreted as the perceptive field mediating the psychophysical judgment involved.

Comparisons

Spillmann et al. (1987) compared measurements of perceptive fields in rhesus monkeys with receptive field measurements of broad-band ganglion cells in the same species. The monkeys were trained to detect small light spots against backgrounds of varied sizes in the Westheimer paradigm. The diameters of perceptive field centers and the dependence of the diameters on retinal eccentricity were derived from the data and compared with corresponding measures derived from ganglion cell recordings. The measures of perceptive field and receptive field diameters agree closely at all eccentricities measured. The authors also reported close correlation with anatomical measures of ganglion cell receptive fields. Because of these correspondences, the authors concluded that in the rhesus monkey the perceptive field and receptive field are close correlates of each other. The authors also compared the perceptive field measurements obtained from monkeys with perceptive field measurements obtained from human observers with the same paradigm. Because of the close agreement in these measurements, the authors proposed that a like correspondence between perceptive and receptive fields exists in the human.

In the Fourier domain, DeValois and DeValois (1988) compared the spatial frequency bandwidths of perceptive fields, estimated from psychophysical studies of channel tuning functions, to receptive field bandwidths derived from single cell recordings in the macaque striate cortex. The precision of such a comparison is limited by the wide variation in bandwidths from one cell to another; the variation in psychophysical estimates produced by different experimental procedures; and significant variation from one observer to another even using identical procedures (e.g. Thomas et al., 1982). However, the ranges of the two classes of estimates overlap greatly, supporting the concept of a close relationship between the tuning functions of psychophysically derived perceptive fields and the selectivity of striate cells.

Neri and Levi (2006) compared perceptive field functions estimated with classification image procedures to selected receptive field functions estimated from single cell recordings. They considered examples of tuning with respect to orientation, retinal disparity, and direction of motion and cited many similarities between the two types of fields, including the manner in which they were altered by selective attention, i.e. changing the peaks but not the bandwidths.

An important comparison concerns information: Do receptive fields and perceptive fields mediate comparable amounts of information?

One way to address this question is to record the responses of a single cell to the stimuli used in a psychophysical detection or discrimination task and determine whether an ideal observer, using only the information provided by the cell’s responses, could perform the task with the same accuracy as a psychophysical observer. Britten and colleagues (1992) performed one of the best of these assessments. They recorded the responses of a single macaque MT neuron while the monkey performed a direction of motion discrimination. Psychometric functions relating the accuracy of the monkey’s direction judgments to task difficulty were constructed and fitted with Weibull functions. Ideal observer analyses yielded analogous psychometric functions based only on the information provided by the cell’s responses. Discrimination thresholds determined from the functions varied from cell to cell, with the mean closely approximating the psychophysically determined threshold of the individual monkey. The slopes of the functions also agreed closely.

Parker and Hawken (1985) performed similar analyses on recordings from striate cells in anesthetized old world monkeys. They estimated thresholds for optimally stimulated single neurons with respect to high spatial frequency resolution, discrimination of small differences in spatial frequency, and small shifts in spatial position. They found these thresholds to be comparable to the corres ponding psychophysical thresholds of human observers. These studies indicate that perceptive fields and the receptive fields of optimally stimulated neurons do, in at least some situations, mediate comparable information.

A significant number of cells in the Britten et al. study had calculated thresholds that were lower than the behaviorally based thresholds. Also, DeValois and DeValois (1988) noted that some macaque striate cells had spatial frequency bandwidths that were narrower than the psychophysical estimates of bandwidths for human observers. Shadlen and coworkers (1996) discussed some of the possible reasons for the discrepancy. From their analyses of the Britten et al. data they concluded that the monkey’s decision processes in the motion discrimination task used data from large pools of neurons that varied widely in their tuning properties, rather than relying on the responses of a small number of optimally selective neurons.

Summary/conclusion

Spillmann and his colleagues (Jung and Spillman, 1970; Spillmann et al., 1987) defined perceptive fields as the psychophysical analogs of receptive fields. This chapter has discussed some of the conceptual, historical, and quantitative parallels that make the analogy apt. To a large extent, the correlations result from efforts of psychophysicists to design experiments that will isolate small groups of similar neural pathways.

The fruitful interplay between perception and physiology is not limited to the simple realms of detection, discrimination, and lightness–darkness modulations. As described by Spillmann (2006, 2009), there is a symbiotic relationship between the study of even more complex perceptual phenomena, including Gestalt organizational principles, and the study of neuronal mechanisms that mediate the phenomena.

References

Abbey, C. K., and Eckstein, M. P. (2002) ‘Classification image analysis: estimation and statistical inference for two-alternative forced-choice experiments’, Journal of Vision, 2: 66–78.

Ahumada, A. J. Jr. (2002) ‘Classification image weights and internal noise level estimation’, Journal of Vision, 2: 121–131.

Ahumada, A. J. Jr., and Lovell, J. (1971) ‘Stimulus features in signal detection’, Journal of the Acoustical Society of America, 49: 1751–1756.

Barlow, H. B. (1953) ‘Summation and inhibition in the frog’s retina’, Journal of Physiology, 119: 69–88.

Blackwell, H. R. (1963) ‘Neural theories of simple visual discrimination’, Journal of the Optical Society of America, 53: 129–160.

Blakemore, C., and Campbell, F. W. (1969) ‘On the existence of neurons in the human visual system selectively sensitive to the orientation and size of retinal images’, Journal of Physiology, 202: 237–260.

Branco, T. (2011) ‘The language of dendrites’, Science, 334: 615–616.

Bredfeldt, C. E., and Ringach, D. L. (2002) ‘Dynamics of spatial frequency tuning in macaque V1’, Journal of Neuroscience, 22: 1976–1984.

Britten, K. H., Shadlen, M. N., Newsome, W. T., and Movson, J. A. (1992) ‘The analysis of visual motion: a comparison of neuronal and psychophysical performance’, Journal of Neuroscience, 12: 4745–4765.

Bullock, T. H., Bennett, M. V., Johnston, D., Josephson, R., Marder, E., and Fields, R D. (2005) ‘The neuron doctrine, redux’, Science, 310: 791–793.

Campbell, F. W., and Robson, J. G. (1968) ‘Application of Fourier analysis to the visibility of gratings’, Journal of Physiology, 197: 551–566.

DeValois, R L., and DeValois, K. K. (1988) Spatial Vision, New York, NY: Oxford University Press.

Enroth-Cugell, C., and Robson, J. G. (1966) ‘The contrast sensitivity of retinal ganglion cells of the cat’, Journal of Physiology, 187: 517–552.

Fiorentini, A., Baumgartner, G., Magnussen, S., Schiller, P. H., and Thomas, J. P. (1990) ‘The perception of brightness and darkness: relations to neuronal receptive fields’, in L. Spillmann and J. S. Werner (eds) Visual Perception: The Neurophysiological Foundations, San Diego, CA: Academic Press.

Graham, C. H., Brown, R H., and Mote, F. A. Jr. (1939) ‘The relation of size of stimulus and intensity in the human eye: I. Intensity thresholds for white light’, Journal of Experimental Psychology, 24: 555–573.

Hartline, H. K. (1940a) ‘The receptive fields of optic nerve fibers’, American Journal of Physiology, 130: 690–699.

Hartline, H. K. (1940b) ‘The effects of spatial summation in the retina on the excitation of the fibers of the optic nerve’, American Journal of Physiology, 130: 700–711.

Helmholtz, H. V. (1962) Physiological Optics, Volume II, J. P. C. Southall (ed.), New York, NY: Dover Publications.

Hines, M. (1976) ‘Line spread function variation near the fovea’, Vision Research, 16: 567–572.

Hubel, D. H. and Wiesel, T. N. (1962) ‘Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex’, Journal of Physiology, 160: 106–154.

Hubel, D. H. and Wiesel, T. N. (1965) ‘Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat’, Journal of Neurophysiology, 28: 229–289.

Hubel, D. H., and Wiesel, T. N. (1968) ‘Receptive fields and functional architecture of monkey striate cortex’, Journal of Physiology, 195: 215–243.

Jung, R., and Spillman, L. (1970) ‘Receptive-field estimation and perceptual integration in human vision’, in F. A. Young and D. B. Lindsley (eds) Early Experience and Visual Information Processing in Perceptual and Reading Disorders, Washington, DC: National Academy of Sciences Proceedings, pp. 181–197.

Kuffler, S. W. (1953) ‘Discharge patterns and functional organization of mammalian retina’, Journal of Neurophysiology, 16: 37–68.

Land, E. H. (1959) ‘Color vision and the natural image, part II’, Proceedings of the National Academy of Sciences of the USA, 45: 636–644.

Legge, G. E., and Foley, J. M. (1980) ‘Contrast masking in human vision’, Journal of the Optical Society of America, 70: 1458–1471.

Mostavi, H., and Sakrison, D. J. (1976) ‘Structure and properties of a single channel in the human visual system’, Vision Research, 16: 957–968.

Neri, P., and Levi, D. M. (2006) ‘Receptive versus perceptive fields from the reverse-correlation viewpoint’, Vision Research, 46: 2465–2474.

Olzak, L. A., and Thomas, J. P. (1986) ‘Seeing spatial patterns’, in K. R. Boff, L. Kaufman, and J. P. Thomas (eds) Handbook of Perception and Human Performance Volume I: Sensory Processes and Perception, New York, NY: Wiley-Interscience.

Parker, A., and Hawken, M. (1985) ‘Capabilities of monkey cortical cells in spatial-resolution tasks’, Journal of the Optical Society of America A, 2: 1101–1114.

Ringach, D., and Shapley, R. (2004) ‘Reverse correlation in neurophysiology’, Cognitive Science, 28: 147–166.

Rodieck, R. W. (1965) ‘Quantitative analysis of cat retinal ganglion cell response to visual stimuli’, Vision Research, 5: 583–601.

Sachs, M. B., Nachmias, J., and Robson, J. G. (1971) ‘Spatial-frequency channels in human vision’, Journal of the Optical Society of America, 61: 1176–1186.

Schiller, P. H. (1986) ‘The central visual system’, Vision Research, 26: 1351–1386.

Shadlen, M. N., Britten, K. H., Newsome, W. T., and Movshon, J. A. (1996) ‘A computational analysis of the relationship between neuronal and behavioral responses to visual motion’, Journal of Neuroscience, 15: 1486–1510.

Sherrington, C. (1906) The Integrative Action of the Nervous System, New Haven, CT: Yale University Press.

Smith, V. C. and Pokorny, J. (1975) ‘Spectral sensitivity of the foveal cone photopigments between 400 and 500 nm’, Vision Research, 15: 161–171.

Spillmann, L. (1971) ‘Foveal perceptive fields in the human visual system measured with simultaneous contrast in grids and bars’, Pflügers Archiv, 326: 281–299.

Spillmann, L. (2006) ‘From perceptive fields to Gestalt’, Progress in Brain Research, 155: 67–92.

Spillmann, L. (2009) ‘Phenomenology and neurophysiological correlations: two approaches to perception research’, Vision Research, 49: 1507–1521.

Spillmann, L., Ransom-Hogg, A., and Oehler, R. (1987) ‘Comparison of perceptive and receptive fields in man and monkey’, Human Neurobiology, 6: 51–62.

Stiles, W. S. (1959) ‘Color vision: the approach through increment threshold sensitivity’, Proceedings of the National Academy of Sciences USA, 45: 100–114.

Thomas, J. P. (1968) ‘Linearity of spatial integrations involving inhibitory interactions’, Vision Research, 8: 49–60.

Thomas, J. P. (1970) ‘Model of the function of receptive fields in human vision’, Psychological Review, 77: 121–134.

Thomas, J. P., Rourke, D. L. K., and Wilder, D. G. (1968) ‘Inhibitory effect of less intense stimuli upon the increment threshold for a narrow test line’, Vision Research, 8: 537–542.

Thomas, J. P., Gille, J., and Barker, R. A. (1982) ‘Simultaneous visual detection and identification: theory and data’, Journal of the Optical Society of America, 72: 1642–1651.

Troscianko, T. (1982) ‘A given visual field location has a wide range of perceptive field sizes’, Vision Research, 22: 1363–1369.

von Bekesy, G. (1960) ‘Neural inhibitory units of the eye and skin. Quantitative description of contrast phenomena’, Journal of the Optical Society of America, 50: 1060–1070.

Watson, A. B. (1983) ‘Detection and recognition ofsimple spatial forms’, in O. J. Braddick, and A. C. Sleigh (eds) Physical and Biological Processing of Images, Berlin, Germany: Springer-Verlag.

Westheimer, G. (1967) ‘Spatial interaction in human cone vision’, Journal of Physiology, 190: 139–154.

Wilson, H. R and Bergen, J. R (1979) ‘A four mechanism model for threshold spatial vision’, Vision Research, 19: 19–32.

Yeh, C. I., Xing, D., Williams, P. E., and Shapley, R M. (2009) ‘Stimulus ensemble and cortical layer determine v1 spatial receptive fields’, Proceedings of the National Academy of Sciences of the USA, 106(34): 14652–14657.

4    Spatiotemporal unity of perception

Given or derived?

Bruno G. Breitmeyer

Introduction

Our perception of the world appears to be a spatiotemporally seamless feat. Gestalt psychologists took such spatiotemporal unity to be a basic phenomenal datum rather than an assemblage derived from more elementary processes. Even the spatiotemporal unity of perception of moving objects, extensively investigated by Gestalt psychologists like Wertheimer (1912) and Ternus (1926), is characterized by a smoothness and a continuity that are subject to the same overarching factors of sensory organization giving rise also, when viewing static objects, to good shape and good continuation (Koffka, 1935). As more fully discussed by Öğmen and Herzog (see Chapter 7 in this volume), the factors governing the perception of objects in motion apply to the preservation as well as transformation of their phenomenal identity in various dynamic settings. As interesting as these phenomena are, the present chapter takes a different, microgenetic approach to the dynamics of the object perception. It will demonstrate that underlying the apparently seamless feat of the unitary perception of visual objects is a series of specifiable temporally successive transformations, some of which, not accessible to phenomenal report at all, must be inferred from results of more recent theoretical, psychophysical and electrophysiological research.

The microgenetic approach taken here is premised on the well-founded assumption that an interval of time intervenes between the onset of visual stimulus and its registering in conscious perception. What happens during that interval is the question that the microgenetic approach seeks to answer. Various methods exist to study the microgenesis of object perception, among them the use of various visual masking techniques (Breitmeyer and Öğmen, 2006), transcranial magnetic stimulation (TMS) (Corthout et al., 1999; Ro et al., 2003), and electro- and magneto-encephalographic (EEG and MEG) recordings (Koivisto and Revonsuo, 2010; Liu et al., 2002; Rieger et al., 2005), all of which allow the temporal tracking of visual processing within millisecond accuracy.

In a recent study, Breitmeyer et al. (2006) explored how metacontrast masking differentially affects the perception of the brightness (a surface feature) and the shape (a form feature) of a target stimulus. Metacontrast masking occurs when one brief stimulus, the target, is followed at variable onset asynchronies (SOAs) by another, brief and spatially surrounding stimulus serving as the mask. A key feature of metacontrast is that it yields maximal suppression of target features at intermediate SOAs, while preserving their visibility at the shortest SOAs near 0 ms and the longer ones beyond about 150 ms. As shown in the upper panel of Figure 4.1, in one task, using a psychophysical tracking method, observers were asked to match the perceived brightness of a black target’s surface relative to an unmasked comparison stimulus; in the second task the same observers were asked to identify one of three disk-like targets that differed in the shape delineated by their contours (a complete disk, a disk with an upper contour deletion as shown, and a disk with a lower contour deletion). Normalized visibilities of the targets for the two tasks are shown in the lower panel of Figure 4.1. Note that metacontrast generally produces a decrease of the visibilities of the target’s surface contrast and of its form. What is also readily apparent is that the SOA at which peak contour masking occurs is 10 ms, 30 ms shorter than the SOA of 40 ms at which peak surface contrast masking occurs. These results conform to the latency differences of about 30 ms – reported by Lamme and colleagues (1999) and by Scholte and co-workers (2008) – between cortical boundary and surface processing. Moreover, as indicated by the solid grey arrow in the lower panel of Figure 4.1, the shorter SOAs ranging up to about 40 ms reveal a dissociation between the contour and surface visibilities. All four observers participating in the study noted this dissociation. In particular, at the SOA of 10 ms at which the form of the disk-like target was not seen or very difficult to see, the surface of the target appeared quite dark, matching about 70 per cent of the contrast of the unmasked target. This indicates that the mask strongly suppressed the processing of the target’s contour needed to discriminate the forms of the three disk-like targets while leaving largely intact the processing of its surface brightness. As shown in the inset depicting two of the three possible targets, this dissociation was reflected in the phenomenal appearance of a target area that was not completely filled from its center to its contours, that is, of a dark target with distorted, missing or blurry contours. A third feature, as demonstrated by Öğmen et al. (2006), is that the mechanism of metacontrast suppression resides at nonconscious levels of processing. Hence, the processes of contour formation and surface completion that are suppressed by metacontrast also occur at nonconscious, pre-phenomenal levels of processing.

Figure 4.1 Top: The spatial layout of stimuli used to study a target’s surface and contour processing during metacontrast. Bottom: Target visibility, relative to a baseline visibility of 1.0 obtained when the targets were presented without the following mask, shown separately for surface contrast and for contours, as a function of the stimulus onset asynchrony between the target and the mask. The inset shows two targets and how they might be perceived at a 10-ms SOA. See text for further details. (Adapted from Breitmeyer et al. (2006), with kind permission from Elsevier Ltd.)

Neural network approaches to object vision

The upshot is that form and surface properties of visual objects are processed as spatiotemporally separate features prior to the conscious perception of a unitary object. In line with this finding, biologically realistic models of vision (e.g. Biederman and Ju, 1988; Grossberg, 1994; Marr, 1982) incorporate separate modules processing the contours that delimit objects and those processing the surface properties within the contours. The evolving versions of Grossberg’s model (1994, 2003; Grossberg and Yazdanbakhsh, 2005) provide particularly apt illustrations of these processes. The model includes a Boundary Contour System (BCS) and a Feature Contour System (FCS). The BCS specifies an object’s boundary. It delineates, for example, the two-dimensional form outline of an object. The FCS specifies the surface features that fill in the area delimited by the BCS. When combined, the perceived object is rendered in terms of its form attributes (the geometric qualia of orientation, width, length, etc.) and of its surface attributes (the sensory qualia of color, lightness, etc.). Thus, we can perceptually distinguish two photo images of, say, an Anjou pear and a Bartlett pear by color; of others, such a green clover leaf and a green dandelion leaf by shape; and of still others, such as a banana and a pomegranate by color and shape. A schematic depiction of the contributions of the form-processing BCS and the surface-processing FCS to object perception is illustrated in Figure 4.2 for two objects, a square and a rhombus with differing, black and white, surface lightnesses.

Figure 4.2 Depictions of a black square and a white diamond on a grey background (top), the implicit representation of their contours by the BCS (middle), and their explicit representation via filling in of surface brightness by the FCS (bottom).

An important aspect of the BCS is that its processing of form is implicit, that is, occurs at nonconscious levels (Grossberg, 2003), as indicated by the dashed lines delineating the contour outline of the objects. The interactive role of the BCS and the FCS in object perception has been stated as follows by Grossberg:

A boundary that is completed within the segmentation system (denoted BCS) does not generate visible contrasts within the BCS. In this sense, all boundaries are invisible [original emphasis]. Visibility is a property of the surface filling-in system (denoted FCS). The completed BCS boundary can directly activate the object recognition system (ORS) whether or not it is visible within the FCS [emphasis added]. In summary, a boundary may be completed within the BCS, and thereby improve pattern recognition by the ORS, without necessarily generating a visible brightness or color difference within the FCS.

(Grossberg, 1994: 59)

These model-based properties of the form- and the surface-processing systems resonate with the finding that form or shape is processed before surface details are filled in (e.g. Humphreys et al., 2000; Spillmann and De Weerd, 2003) and have important implications for our understanding of object perception (Biederman and Ju, 1988; Marr and Nishihara, 1978; Ullman, 1984).

Neurobiological substrate for form-processing and surface-processing systems in primate visual cortex

Accumulating evidence indicates that there are anatomically identifiable pathways and areas in the early and intermediate cortical object-processing systems (see Figure 4.3) that process primarily the surface properties of color and lightness on the one hand and the form properties of contour and edge orientations on the other (Conway et al., 2007; Felleman et al., 1997; Roe and Ts’o, 1999; Wang et al., 2007; Xiao and Felleman, 2004; Xiao et al., 2003, 2007). For instance, based on Felleman et al.’s (1997) work, visual area 4 (V4), as well as areas 1 and 2 (V1 and V2), have separate compartments for shape and surface processing. Supporting this scheme, Girard and colleagues (2002) showed that reversible deactivation of V4 in macaque monkey can impair shape discrimination while leaving hue discrimination intact.

Figure 4.3 Schematic of the BCS and the FCS in relation to the ventral object-processing pathway. Note that the FCS (see long-dashed lines) consists of projections to V1-blob/V2-thin stripes and beyond; the BCS (see short-dashed lines) consists of projections to V1-interblobs/V2-interstripes and beyond. The FCS processes wavelength and luminance properties (designated respectively by the and the symbols) of object surfaces. The BCS processes object contours defined by isoluminant-wavelength and by luminance contrast (designated respectively by the and the symbols). See text for further details. (Adapted from DeYoe and Van Essen (1988), with kind permission from Elsevier Ltd.)

In line with proposals also suggested by others (Kiper, 2003; Roe and Ts’o, 1999), neurons tuned jointly to color and orientation may be especially well suited for processing contours defined by wavelength differences. These would be especially important to the form-processing system when contours are defined by isoluminant wavelength differences. Non-oriented wavelength-specific neurons would instead contribute to the surface-processing system. Of course, luminance-defined borders and achromatic surface properties such as lightness are also processed by the separate cortical form- and surface-processing systems. In a prior study of cortical chromatic processing, Xiao et al. (2003) showed that the thin stripes in V2, which receive input from V1’s color-selective blob areas, contain functional color-specific subregions in which variations of stimulus color are systematically mapped onto varying locations within the subregions. Recent findings reported by Xiao et al. (2007) indicate that cortical color maps exist as early as in the V1 blob areas. These could provide input to the spatially more extensive V2 thin-stripe color maps (Xiao et al., 2007). Wang and colleagues (2007) additionally found that adjacent to the color maps within the V2 thin stripes are neurons responding differentially to positive (light-on-dark) and negative (dark-on-light) brightness. According to these authors the thin stripes of V2 receive input from V1 blob areas and comprise neural modules for processing chromatic and achromatic surface properties of visual stimuli; whereas neurons found in the adjacent V2 interstripes respond selectively to contour or edge orientation of stimuli and receive input from V1 interblob areas (see Figure 9 from Wang et al., 2007). Additionally, electrophysiological (Pasupathy and Connor, 2002; Tsunoda et al., 2001) and psychophysical (Treisman, 1988) findings indicate that more complex form features such as vertices, corners, and intersections, are constructed at later stages of visual processing by conjoining simpler form features such as oriented lines or edges processed at earlier stages.

Evidence for separate but interactive form-processing and surface-processing systems in human vision

Consistent with these neurophysiological properties found in monkey, studies of neurological patients with impairments of visual perception reveal that the shape and surface properties of visual objects also are processed by dissociable cortical systems in humans. Several studies (Barbur et al., 1994; Kentridge et al., 2004) showed that, despite loss of hue perception of surfaces, achromatopsic (cortically color-blind) patients can detect isoluminant chromatic contours. Moreover, an achromatopsic patient investigated by Heywood and co-workers (1987), while grossly impaired in discriminating isoluminant hues, was able to discriminate different achromatic grays. This is consistent with the above-noted finding in monkeys (Wang et al., 2007) of separate lightness- and color-processing submodules within V2 thin-stripe surface-processing modules. In addition, recalling that selective loss of shape discrimination with intact hue discrimination was reported by Girard et al. (2002) when V4 was reversibly deactivated in macaque monkey, Zeki and colleagues (1999) similarly showed that a patient, though all but form blind, was able to name objects’ colors.

Critchley (1965) reported some neurological patients who perceive the color of an object irradiating outward beyond, and sometimes at great distances from, its boundaries, which in turn are often seen as fuzzy or blurred. In other patients the color of an object is perceived as not adhering to its surface, but instead as free-floating in space, in a plane usually closer to the patient than the perceived object. Related phenomena have been reported recently by Hong and Blake’s (2009) study of binocular suppression. Here, rapidly changing grey Mondrian patterns were flashed to one eye while a stationary chromatic bar was presented to the other eye. Although observers failed to see the shape of the bar, they did report its color in a free-floating, cloudlike constellation. Both the neurological symptoms reported by Critchley (1965) and the phenomena reported by Hong and Blake (2009) indicate either that the form-processing BCS does not construct or provide the necessary constraints for the filling-in process of the color-FCS system or that such BCS constraints are not communicated to the FCS system. In either case, the result is a perception of blurred or missing boundaries along with spatially diffused colors.

The spatiotemporal dynamics of form and surface processing

Nonconscious level

Given that contour and surface properties are processed at early, nonconscious levels such as V1, what is the evidence indicating that the processing of contour precedes the processing of surface properties at these levels?

First, as noted, electrophysiological recordings from V1 neurons in macaque reveal separate processing of contour and surface properties of stimuli, with neural responses to surface properties lagging those to contour properties by about 30 ms (Lamme et al., 1999). Cortically evoked potential and psychophysical studies of human observers also indicate slower surface than contour processing (Caputo et al., 1999; Romani et al., 1999; Rossi and Paradiso, 2003).

Conscious level

A Reversal of temporal order

The temporal priority of the processing of contour relative to surface at non-conscious, pre-perceptual cortical levels reverses, as indicated by recent findings that at the perceptual level abrupt changes of a stimulus’s color occur about 40 ms earlier than (Zeki and Bartels, 1998), or at best nearly simultaneously with (Viviani and Aymoz, 2001), the abrupt changes of its form. Consistent with this temporal priority of perceived color, in their study of interocular continuous flash suppression, Hong and Blake (2009) found that the color of a chromatic bar presented to the (temporarily) suppressed eye nearly always emerged into phenomenal awareness before the orientation (form) of the bar was perceived. Rather than revealing a contradiction or inconsistency, we take these results to actually point out a key feature of the transition from pre-perceptual phase to a perceptual phase of form processing. According to Grossberg’s (1994) model articulated above, the conscious percept of form depends on the prior completion of surface features. Adding Grossberg’s (2003: 19) further claim that ‘surfaces are for seeing’, it follows that the seeing or perception of form occurs only after the perceptual completion of surface features such as color commences. For this reason the temporal dynamics of surface and contour processing, though characterized by a clear lag of surface processing at the nonconscious, pre-perceptual level, is characterized by a lead (or near synchrony) of surface (e.g. color) processing at the conscious, perceptual level.

Evidence for surface filling-out

Recently Breitmeyer and Jacob (2012) obtained evidence for a microgenetic process of surface completion that can be characterized by filling out rather than filling in. The spreading of surface properties such as color can be considered as proceeding in a direction away from contours. Since visual objects are typically defined by contours that enclose their surfaces, such spreading will consequently result in a filling in that begins at the edges of the object and spreads toward its center (Paradiso and Nakayama, 1991; Rossi and Paradiso, 2003; Spillmann and de Weerd, 2003). However, using a metacontrast paradigm in which a rectangular target was flanked by two smaller rectangular masks, as shown in the upper panel of Figure 4.4, Breitmeyer and Jacob (2012) demonstrated that when the target-mask SOA assumed a value of 80 ms, barely above the one yielding maximal surface suppression, only a small central region of the target’s surface was (barely) visible. As the SOA increased, all observers reported that a progressively wider central region became visible until the target reached full surface visibility at the upper SOAs (see lower panels of Figure 4.4).

Figure 4.4 Upper panel: Representation of target rectangle and the two flanking metacontrast mask rectangles. Lower panels: Facsimiles of the appearance of the target at target-mask onset asynchronies (SOAs) increasing from 80 to 240 ms. (Reprinted from Breitmeyer and Jacob (2012), with kind permission from Elsevier Ltd.)

Noting this, Breitmeyer and Jacob (2012) varied the width of the foveally centered target to track the change of the upper SOA at and above which the target appeared filled out. Shown in the upper panel of Figure 4.5 are:

1)  

the changes of averaged SOAs, at each of which the target was judged by observers to appear complete on 90 percent of the target-mask trials, as target width (in retinal degrees of visual angle) increases, along with

2)  

a best fitting power function describing the relationship between criterion SOA and target width.

Figure 4.5 Upper panel: The SOA, at which the target was judged by observers to be complete on 90 percent of target-mask presentations, as a function of the target width in retinal degrees of visual angle, along with the best-fitting power function. Lower panel: The relationship between the same SOAs and cortical distance along with the best-fitting linear function. (Reprinted from Breitmeyer and Jacob (2012), with kind permission from Elsevier Ltd.)

Such power functions indicate the presence of cortical magnification (the extent of cortical surface devoted to each degree of visual angle). Using values of cortical magnification provided by Duncan and Boynton (2003), Breitmeyer and Jacob (2012) converted the target widths given in retinal distances to cortical distances to obtain the best fitting linear function describing the relationship between criterion SOA and cortical distance in millimeter units. The slope of this function, 0.353mm/ms (0.353m/s), provides an estimate of a cortical filling-out speed.

In settings outside the experimental laboratory these microgenetic fillings out generally go unnoticed. Assuming that

this speed of 0.353m/s is uniform across retinotopic cortical surfaces (such as V1) and knowing that

that cortical magnification decreases as retinal eccentricity increases (Duncan and Boynton, 2003), one would expect the filling-out time for a stimulus of constant extent in retinal coordinates to decrease with eccentricity.

For example, a 1-degree diameter disk, centered, say, at 0.5 degree from the fovea, should be filled out within 20 ms; whereas when centered at 8 degrees, it should be filled out within only 3 ms. Hence even visual objects of large spatial extent should fill out nearly instantaneously, and thus unnoticeably, as long as they fall in peripheral parts of the visual field.

Moreover, noting that processing of contours at progressively finer spatial scales requires progressively longer time (Breitmeyer, 1975; Lupp et al., 1976; Schyns and Oliva, 1994; Vassilev and Mitov, 1976), Breitmeyer and Jacob (2012) presented a model in which, over the course of few tens of milliseconds, the evolution of the phenomenal filling out results from progressively finer spatial-scale contour processing, followed by its corresponding filling-in process. Such a model is consistent with other extant models of surface completion that incorporate processing at varying spatial scales (Grossberg, 1994, 2003; Watt, 1988).

Summary/conclusion

The fundamental ideas discussed in this chapter are that the visual processing of an object is partitioned along parallel channels into the processing of its surface properties and its form or contour properties. At cortical levels, its form/contour properties are processed by the ventral P-dominated interblob/interstripe pathway and its surface properties are processed by the ventral P-dominated blob/thin-stripe pathway. At pre-perceptual, nonconscious levels the processing of form/contour properties precedes the processing of surface properties by several tens of milliseconds. However, if, as proposed (see Grossberg’s quote above), object vision ultimately depends on the completing of surface properties, then at conscious, phenomenal levels the asynchrony disappears or is reversed by several tens of milliseconds. Moreover, while form-feature primitives can be conjoined at nonconscious levels to establish representations of vertices, corners, etc. or the entire shape of an object (Breitmeyer et al., 2005), the conjunction of form and surfaces attributes appears to emerge only at the conscious level of processing. Because the Gestalt approach relied methodologically on phenomenal descriptions and analyses, it was limited to studying the properties of conscious vision and thus could not access the nonconscious surface- and form-processing levels to which the sophisticated electrophysiological and psychophysical methods developed since the heyday of Gestalt science have allowed us access. Nonetheless, the results shown in Figure 4.5 do illustrate nicely a particular application of the Gestalt principle of psychoneural isomorphism, according to which the spatiotemporal filling out phenomenologically described in terms of visual, and thus of retinal, space map in a one-to-one manner onto a corresponding spatio-temporal filling out in cortical space.

References

Barbur, J. L., Harlow, A. J. and Plant, G. (1994) ‘Insights into the different exploits of colour in the visual cortex’, Proceedings of the Royal Society London B, 258: 327–334.

Biederman, I. and Ju, G. (1988) ‘Surface versus edge-based determinants of visual recognition’, Cognitive Psychology, 20: 38–64.

Breitmeyer, B. G. (1975) ‘Simple reaction time as a measure of the temporal response properties of transient and sustained channels’, Vision Research, 15: 1411–12.

Breitmeyer, B. G. and Öğmen, H. (2006). Visual Masking: Time Slices Through Conscious and Unconscious Vision. Oxford, UK: Oxford University Press.

Breitmeyer, B. G. and Jacob, J. (2012) ‘Microgenesis of surface completion in visual objects: Evidence for filling-out’, Vision Research, 55: 11–18.

Breitmeyer, B. G., Öğmen, H., Ramon, J. and Chen, J. (2005) ‘Unconscious and conscious priming by forms and their parts’, Visual Cognition, 12: 720–736.

Breitmeyer, B. G., Kafaligönül, H., Öğmen, H., Mardon, L., Todd, S. and Ziegler, R. (2006) ‘Meta- and paracontrast reveal differences between contour and brightness-processing mechanisms’, Vision Research, 46: 2645–58.

Caputo, G., Romani, A., Callieco, R., Gaspari, D. and Cosi, V. (1999) ‘Amodal completion in texture visual evoked potentials’, Vision Research, 39: 31–38.

Conway, B. R., Moeller, S. and Tsao, D. Y. (2007) ‘Specialized color modules in macaque extrastriate cortex’, Neuron, 56: 560–573.

Corthout, E., Uttl, B., Ziemann, U., Cowey, A. and Hallett, M. (1999) ‘Two periods of processing in the (circum)striate visual cortex as revealed by transcranial magnetic stimulation’, Neuropsychologia, 37: 137–145.

Critchley, M. (1965) ‘Acquired anomalies of colour perception of central origin’, Brain, 88: 711–724.

DeYoe, E. A. and Van Essen, D. C. (1988) ‘Concurrent processing streams in monkey visual cortex’, Trends in Neuroscience, 11: 219–226.

Duncan, R. O. and Boynton, G. M. (2003) ‘Cortical magnification within human primary visual cortex correlates with acuity thresholds’, Neuron, 38: 659–671.

Felleman, D. J., Xiao, Y. and McClendon, E. (1997) ‘Modular organization of occipitotemporal pathways: Cortical connections between visual area 4 and visual area 2 and posterior inferotemporal ventral area in macaque monkeys’, Journal of Neuroscience, 17: 3185–3200.

Girard, P., Lomber, S. G. and Bullier, J. (2002) ‘Shape discrimination deficits during reversible deactivation of area V4 in the macaque monkey’, Cerebral Cortex, 12: 11146–56.

Grossberg, S. (1994) ‘3-D vision and figure-ground separation by visual cortex’, Perception & Psychophysics, 55: 48–120.

Grossberg, S. (2003) ‘Filling-in the forms: Surface and boundary interactions in visual cortex’, in L. Pessoa and P. De Weerd (eds) Filling-in: From Perceptual Completion to Cortical Reorganization, Oxford, UK: Oxford University Press.

Grossberg, S. and Yazdanbakhsh, A. (2005) ‘Laminar cortical dynamics of 3D surface perception: Stratification, transparency, and neon color spreading’, Vision Research, 45: 1725–43.

Heywood, C. A., Wilson, B, and Cowey, A. (1987) ‘A case study of cortical colour “blindness” with relatively intact achromatic discrimination’, Journal of Neurology, Neurosurgery, and Psychiatry, 50: 22–29.

Hong, S. W. and Blake, R. (2009) ‘Interocular suppression differentially affects achromatic and chromatic mechanisms’, Attention, Perception, & Psychophysics, 71: 403–411.

Humphreys, G. W., Cinel, C., Wolfe, J., Olson, A. and Klempen, N. (2000) ‘Fractioning the binding process: Neurophsychological evidence distinguishing binding of form from binding of surface features’, Vision Research, 40: 1569–96.

Kentridge, R. W., Heywood, C. A. and Cowey, A. (2004) ‘Chromatic edges, surfaces, and constancies in cerebral achromatopsia’, Neuropsychologia, 42: 821–830.

Kiper, D. C. (2003) ‘Colour and form in the early stages of cortical processing’, Journal of Physiology, 548 (2): 335.

Koffka, K. (1935) Principles of Gestalt Psychology, New York, NY: Harcourt, Brace & World, Inc.

Koivisto, M. and Revonsuo, A. (2010) ‘Event-related brain potential correlates of visual awareness’, Neuroscience and Biobehavioral Reviews, 34: 922–934.

Lamme, V. A. F., Rodriguez-Rodriguez, V. and Spekreijse, H. (1999) ‘Separate processing dynamics for texture elements, boundaries, and surfaces in primary visual cortex of the macaque monkey’, Cerebral Cortex, 9: 406–413.

Liu, J., Harris, A. and Kanswisher, N. (2002) ‘Stages of processing in face perception: An MEG study’, Nature Neuroscience, 5: 910–916.

Lupp, U., Hauske, G. and Wolf, W. (1976) ‘Perceptual latencies to sinusoidal gratings’, Vision Research, 16: 969–972.

Marr, D. (1982) Vision, San Francisco, CA: W. H. Freeman.

Marr, D. and Nishihara, H. K. (1978) ‘Representation and recognition of the spatial organization of three-dimensional shapes’, Proceedings of the Royal Society of London B, 200: 269–294.

Öğmen, H., Breitmeyer, B. G., Todd, S. and Mardon, L. (2006) ‘Target recovery in metacontrast: The effect of contrast’, Vision Research, 46: 4726–4734.

Paradiso, M. A. and Nakayama, K. (1991) ‘Brightness perception and filling-in’, Vision Research, 31: 1221–1236.

Pasupathy, A. and Connor, C. E. (2002) ‘Population coding of shape in area V4’, Nature Neuroscience, 5: 1332–1338.

Rieger, J. W., Braun, J., Bülthoff, H. H. and Gegenfurtner, K. R. (2005) ‘The dynamics of visual pattern masking in natural scene processing: A magnetoencephalography study’, Journal of Vision, 5: 275–286.

Ro, T., Breitmeyer, B., Burton, P., Singhal, N. S. and Lane, D. (2003) ‘Feedback contributions to visual awareness in human occipital cortex’, Current Biology, 11: 1038–1041.

Roe, A. W. and Ts’o, D. Y. (1999) ‘Specificity of color connectivity between primate V1 and V2’, Journal of Neurophysiology, 82: 2719–2730.

Romani, A., Caputo, G., Callieco, R., Schintone, E. and Cosi, V. (1999) ‘Edge detection and surface “filling in” as shown by texture visual evoked potentials’, Clinical Neurophysiology, 110: 86–91.

Rossi, A. F. and Paradiso, M. A. (2003) ‘Surface completion: Psychophysical and neurophysiological studies of brightness interpolation’, in L. Pessoa and P. De Weerd (eds) Filling-in: From Perceptual Completion to Cortical Reorganization, Oxford, UK: Oxford University Press.

Scholte, H. S., Jolij, J., Fahrenfort, J. J. and Lamme, V. A. F. (2008) ‘Feed forward and recurrent processing in scene segmentation: Electroencephalography and functional magnetic resonance imaging’, Journal of Cognitive Neuroscience, 20: 2097–2109.

Schyns, P. G. and Oliva, A. (1994) ‘From blobs to boundary edges: Evidence for time and spatial-scale-dependent scene recognition’, Psychological Science, 5: 195–200.

Spillmann, L. and de Weerd, P. (2003) ‘Mechanisms of surface completion: Perceptual filling-in of texture’, in L. Pessoa and P. de Weerd (eds) Filling-in: From Perceptual Completion to Cortical Reorganization, New York, NY: Oxford University Press.

Ternus, J. (1926) ‘Experimentelle Untersuchungen über phänomenale Identität’, Psyhologische Forschung, 7: 81–136.

Treisman, A. (1988) ‘Feature binding, attention, and object perception’, Philosophical Transactions of the Royal Society B, 353: 1295–1306.

Tsunoda, K., Yamane, Y., Nishizaki, M. and Tanifuji, M. (2001) ‘Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns’, Nature Neuroscience, 4: 832–838.

Ullman, S. (1984) ‘Visual routines’, Cognition, 18: 97–159.

Vassilev, A. and Mitov, D. (1976) ‘Perception time and spatial frequency’, Vision Research, 16: 89–92.

Viviani, P. and Aymoz, C. (2001) ‘Colour, form, and movement are not perceived simultaneously’, Vision Research, 41: 2909–2918.

Wang, Y., Xiao, Y. and Felleman, D. J. (2007) ‘V2 thin stripes contain spatially organized representations of achromatic luminance change’, Cerebral Cortex, 17: 116–129.

Watt, R. (1988) Visual Processing, Hove, UK: Lawrence Erlbaum Associates.

Wertheimer, M. (1912) ‘Experimentelle Studien über sas Sehen von Bewegung’, Zeitschrift für Psychologie, 61: 161–265.

Xiao, Y. and Felleman, D. J. (2004) ‘Projections from primary visual cortex to cytochrome oxidase thin stripes and interstripes of macaque visual area 2’, Proceedings of the National Academy of Sciences, USA, 101: 792–804.

Xiao, Y., Wang, Y. and Felleman, D. J. (2003) ‘A spatially organized representation of colour in macaque cortical area V2’, Nature, 421: 535–539.

Xiao, Y., Casti, A., Xiao, J. and Kaplan, E. (2007) ‘A spatially organized representation of colour in macaque cortical area V2’, Neuro Image, 35: 771–786.

Zeki, S. and Bartels, A. (1998) ‘The asynchrony of consciousness’, Proceedings of the Royal Society B, 265: 1583–1585.

Zeki, S., Aglioti, S., McKeefry, D. and Berlucchi, G. (1999) ‘The neurological basis of conscious color perception in a blind patient’, Proceedings of the National Academy of Sciences, USA, 96: 14124–14129.

5    What is a perceptual object?

Beyond the Gestalt theory of perceptual organization

Baingio Pinna

Introduction: on the problem of visual object

The Gestalt approach to figure-ground segregation

Gestalt psychologists first investigated and developed a theory of object perception. They approached the problem of visual objects in terms of figure-ground segregation and grouping. Rubin (1921) studied the object formation by investigating what appears as a figure and what as a background. He discovered the following well-known figure-ground principles: surroundedness, size, orientation, contrast, symmetry, convexity, and parallelism. According to Rubin, these principles depend on three main phenomenal attributes, belonging to the figure but not to the background:

The contour belongs unilaterally to the figure (see also Pinna, 2010a, 2010b, 2012a; Spillmann and Ehrenstein, 2004), namely it is only the figure which takes on the shape traced by the contour, not the background.

The figure color/brightness shows a way of appearance (Erscheinungsweise, Katz, 1930) similar to a surface, i.e. denser than the same physical color/brightness of the background that is, instead, perceived completely transparent and empty.

The figure is always seen placed in depth, closer to the observer than the background.

The three main properties are clearly perceived in Figure 5.1a, where the two regions of the square, separated by a wavy contour, are perceived reversibly as a figure or as a background. If one region is perceived as a figure, the other is seen as a background. The wavy contour appears as the boundary of the figure, while the background perceived without a boundary, completes amodally behind the figure and fills the square frame as an empty space. Moreover, the figure-ground segregation is accompanied by a clear brightness differentiation: the figure shows a surface color/brightness property, namely the chromatic paste appears solid, impenetrable, and epiphanous as a surface; while, the background is seen empty, penetrable and diaphanous as a void (Katz, 1930). Finally, the region perceived as a figure appears in front of the background and popping out from the square. All these properties together elicit the perception of the square shape like a frame or a window. If the square is seen as a window, then its surrounding region is perceived like a solid surface. In summary, the figure-ground segregation reveals a complex set of object-components each related and dependent on the other: the square window-frame, the undulated filled surface amodally continuing behind the window-frame, and the complementary empty space.

Figure 5.1 Conditions illustrating the role of the figure-ground segregation and the unilateral belongingness of the boundaries in object formation.

Under the conditions of Figure 5.1a, the figure-ground segregation is related to the concave–convex alternation along the contour and to the surroundedness principle. Nevertheless, the same complementary figure-ground differentiation emerges by replacing the wiggly contour with a straight line (Figure 5.1b), thus annulling the concave-convex alternation. Furthermore, the segregation persists when the square frame is removed (Figure 5.1c), thus annulling the role of the surroundedness principle. These results suggest that a single line/contour is like a figure-ground watershed, i.e. a source of phenomenal asymmetry of figure-ground attributes on both sides (Pinna, 2012b, 2012c).

It is worth noticing that in Figure 5.1c there are two antagonistic phenomenal results: the figure-ground differentiation imparted by the contour that appears as the boundary belonging to only one of the two segregated regions and the perception of the wiggly contour alone without any figure-ground segregation. The two possible results emerge alternately – i.e. if the wiggly contour emerges, the figure-ground differentiation is invisible and vice versa – although the wiggly contour only is, at first sight, the most prominent.

By perceiving the figure-ground differentiation, this result persists even when the contour only is enhanced by interweaving/overlapping its terminations, as shown in Figure 5.1d. The undulated central component of the contour, not interweaved, persists as the boundary of the segregated figure, although it is weaker than the one of Figure 5.1a. This result does not prevent possible inner contradictions or antinomies within the contour continuation (see also Pinna and Reeves, 2006). This suggests that different portions of the same object can be perceived in different ways, even opposite (Tanca et al., 2010), as corroborated by other well-known figures, e.g. Penrose’s impossible objects.

The Gestalt approach to perceptual grouping

The complex set of figure-ground segregations, emerging from Figure 5.1a, depends also on another process considered crucial by Gestalt psychologists to understand object perception: grouping. As a matter of fact, the formation of the square frame/window is due to the grouping of segments with orthogonal orientations and, at the same time, to their ungrouping from the wiggly contour. Wertheimer (1923) first approached the problem of object perception in terms of grouping. The questions were:

How do the elements in the visual field ‘go together’ to form an integrated percept?

How do individual elements create larger wholes?

He answered these questions through the following well-known principles: proximity, similarity, good continuation, closure, symmetry, convexity, Prägnanz, past experience, common fate, and parallelism.

In Figure 5.2a, an eight-pointed star is illustrated. The perception of the star is the result of the grouping of 16 short segments that in pairs create eight points/protrusions or, at the same time, indentations. According to Wertheimer, the principles of good continuation, Prägnanz, and closure are mainly responsible for this result. It is worth noticing that the protrusions appear much more salient than the indentations. The former are in fact responsible for defining what the perceived object is, i.e. the star, although theoretically the indentations are equally likely. Only the points are perceived as parts of the object, not the indentations that are almost invisible. The immediate description ‘eight-pointed star’ is a clear linguistic-phenomenological demonstration of the perceptual main role of the protrusion. This suggests a possible and plausible visual constraint/assumption according to which, all else being equal, objects tend to appear mostly convex rather than concave. This assumption agrees with the figure-ground principle of convexity, stating that, all else being equal, convex patterns rather than concave tend to appear as figures. The asymmetry between protrusions/indentations underlines the problem of multiple possibilities of an object, a problem that will be dealt in greater depth in the next sections.

Figure 5.2 Different objects created by the Gestalt principles of grouping.

The object formation, as a result of the grouping of individual segments, can be phenomenally demonstrated in Figure 5.2b, where two instances of the eight-pointed star have been partially overlapped. Under these conditions, the single segments break the expected organization, i.e. two overlapped stars, and group in three juxtaposed closed regions. Among them, the central one pops out more saliently. The closure and good continuation principles are mainly responsible for this result. By overlapping more than two stars the grouping interaction among segments becomes even more unexpected as shown in Figure 5.2c, where the stars are disguised within a set of other possible objects through which the visual system switches spontaneously and reversibly. The stars can be made popped out by alternating their shape with opposite contrast polarity, i.e. by drawing them one white and one black (Figure 5.2d). The grouping principle here involved is the similarity of the achromatic color or of the contrast polarity of the segments, according to which, all else being equal, similar segments group together to create whole objects. The multiple possibilities of creating many different groupings and, thus, different objects on the basis of the similarity principle is illustrated in Figures 5.2e–g.

The two main issues raised by Gestalt psychologists – figure-ground segregation and grouping – can be easily connected as shown in Figure 5.3a, where eight-pointed stars and crosses can be alternately and reversibly perceived. When the stars are perceived the crosses are invisible and vice versa, due to the unilateral belongingness of the boundaries. A tessellation can also be seen although much weaker than the alternation stars/crosses. In Figures 5.3b–c, just the stars and the crosses pop out while, respectively, the complementary crosses and stars become totally invisible. In Figure 5.3d, by regrouping the same pattern of components on the basis of the similarity, new objects emerge: a white square overlapping a black lattice of oblique parallel and orthogonal lines amodally completing behind the white squares. In Figures 5.3e–f, new intersected objects are formed according to the similarity principle. Finally, in Figures 5.3g–h, the wholeness of the shapes is broken by splitting the grouping in small segments each with a different contrast polarity. In other words, by using the same principle of similarity/dissimilarity, the grouping previously perceived so clearly is now ungrouped. This entails that, to form a clear object, i.e. a whole unique object, the similarity and homogeneity of its components is required. This is the assumption of homogeneity.

Figure 5.3 Reversible stars and crosses becoming new objects due to the similarity principle.

Figure-ground and grouping issues

Although the conditions illustrated in Figure 5.3 build a bridge between figure-ground segregation and grouping they raise, at the same time, some logical and epistemological issues.

From a logical point of view, figure-ground segregation should operate before grouping (Palmer, 1999). For example, dot or line elements on which grouping acts must be already segregated as a figure from the background, otherwise the visual system would not know which elements to group. Under these logical conditions, the elements cannot show the figural properties of holistic organized and segregated objects, but should appear as basic components necessary to create whole objects, namely like ‘bricks’ necessary to create an emerging and holistic object. At the same time, the opposite is also true. As single elements segregated from the background, they do not appear like whole objects (see Figures 5.3g–h). To be perceived as such, they have first to be grouped, thus creating an emergent Gestalt.

In spite of the apparent relations between figure-ground segregation and grouping, what phenomenally comes up from the previous figures is their phenomenological intimate intertwining. A sharp and univocal distinction both neural and phenomenal is maybe impossible and likely useless in scientific terms. This is related to the fact that one becomes the row material of the other and vice versa, creating different perceptual levels with different categories of visual objects (see Pinna, 2010b).

The epistemological problem derived from Figure 5.3 is related to the fact that within Rubin’s and Wertheimer’s works, the problem of object formation is approached in terms of the main conditions (principles) operating in the two dynamics (figure-ground segregation and grouping) underlying and preceding the formation of a visual object. More precisely, the closure and similarity principles can be considered as object issues before the ‘object’ meaning, where the question is ‘What is a visual object?’ In other words, they talk about the object perception but they do not say what an object is, that is, they do not define its meaning, attributes, and inner complexity. Even if the similarity principle of grouping can describe the perception of a star (e.g. Figure 5.3b), it cannot say anything about its properties and about the way these properties assign the visual meaning of being the object called ‘star’. Neither can it explain the object changes illustrated in the next sections.

Even though figure-ground segregation and grouping principles are important issues of object perception, they do not face this matter directly and, more importantly, they do not answer basic questions like:

What is a visual object?

What is its meaning?

These questions open new and interesting perspectives and levels of perceptual organization beyond the Gestalt approach. In this chapter I introduce the reader to new phenomenological and scientific paths by showing the complexity of object perception.

What is a perceptual object?

To answer this question, I am going to explore in the next sections the phenomenology of multiple possible objects within the same stimulus pattern.

Many possible objects within one stimulus pattern

In Figure 5.4a, two eight-pointed stars of different sizes, one inset in the other, are perceived. No other possible object emerges. This result is more clearly perceived in Figure 5.4b, where the opposite contrast polarity of the two stars favors their differentiated salience. In Figure 5.4c, by changing the black and white arrangement of the segments, the stars are barely perceived and two kinds of rotated and intersected square-like shapes, made up of four closed quadrangular shapes placed in their vertices, are now perceived. In Figure 5.4d, two sorts of intertwined and spiraled star-like shapes emerge. They are not one inset within the other as in Figure 5.4a, but have equal size. By playing differently with the contrast polarity of the same pattern of segments, the two stars of Figure 5.4a totally disappear becoming two wavy quadrangular shapes overlapped (Figure 5.4e), two symmetrical strange shapes (Figure 5.4f) and, finally, two irregular overlapped objects (Figure 5.4g).

Figure 5.4 The similarity principle induces many possible objects within the same stimulus pattern.

Under these conditions, the similarity of the contrast polarity plays the main role winning against other principles like good continuation or closure. Moreover, as shown in Figure 5.4g, the symmetry of the shape and the Prägnanz principle do not weaken the role of similarity and do not affect the perceived result of two irregular objects at all. Irregular shapes can also be created by playing with the achromatic colors of the overlapped stars of Figure 5.4h, as illustrated in Figure 5.4i.

The results of Figure 5.4 suggest a strong effect of the unilateral belongingness of the boundaries due to the contrast polarity. More precisely, the opposite direction of the contrast induces two strongly segregated complementary objects that mask other possible groupings. This suggests the following general statement: the perception of an object makes the perception of other possible objects invisible, which shall be the topic of the next section.

Object masking and unmasking

Figure 5.5 demonstrates that, given a pattern of stimuli, when one object emerges as such all the other potential objects disappear, i.e. the winner object takes all the phenomenal salience. This special behavior is clearly shown within the variations illustrated in Figure 5.5. In Figure 5.5a, the Gestalt closure and Prägnanz principles mostly elicit the grouping and the figure-ground segregation of regular shapes, hexagons and stars, adjoining one by one in their angles and points. Other possible results remain into the background, namely totally invisible or only partially visible. The motion of the gaze around the segments can favor the emergence as a figure of one or another grouping, which immediately disappears as soon as another grouping is favored by the direction of the gaze or by other factors.

Figure 5.5 Masking and unmasking by playing with the similarity of the contrast polarity.

By introducing the principle of similarity of the direction of the contrast, new groupings previously invisible become now the winner-takes-all objects. In Figures 5.5b–d, three sets of new possible groupings are unmistakably demonstrated, proving the strong masking or unmasking power of the grouping principle of similarity. What is done can be undone through the same principle.

In Figures 5.5e and 5.5f, it becomes ever more easy to perceive four superimposed triangles which, even under these simple conditions, can be masked or camouflaged in different ways, as shown in Figures 5.5g–h, by means of the similarity of the contrast polarity. The masking–unmasking effect is even stronger in Figures 5.5i–k, where the triangular shape arrangements of Figure 5.5i make the polygons and the stars of Figures 5.5j–k totally invisible.

The phenomenological winner-takes-all behavior is based on the unilateral border ownership that depends, in its turn, on the similarity principle elicited by the contrast polarity (achromatic color). This main and general behavior can be easily related to a similar principle applied in computational models of neural networks, according to which neurons in a layer compete with one another for activation. The basic idea is that only the neuron with the highest activation stays active while all the other neurons shut down. In terms of neural networks, output nodes in the network mutually inhibit each other, while simultaneously activating themselves through reflexive connections. After several activations, only the node, corresponding to the strongest input in the output layer, will be active due to the nonlinear inhibition. The winner-takes-all is a main computational primitive that is implemented through different types of neural network models (e.g. Grossberg, 1997; Oster et al., 2009).

The masking and unmasking effect due to similarity can be found in nature in camouflages by mimesis and crypsis, by which animals otherwise visible remain unnoticed by resembling something else (e.g. birch, willow branches, dry leaf, etc.) or by blending with their environment. Both kinds of camouflages are only at first glance in contradiction with the assumption of homogeneity previously described. The whole unique object, based on the similarity and homogeneity of its components, is not the animal but the environment. Being homogeneous with its environment, the animal becomes environment.

An interesting case of camouflage by crypsis is the disruptive patterning used by some animals with strong contrasting markings (spots or stripes) to break up their own outlines. Typically, the high-contrast light and dark patches in a non-repetitive configuration provide camouflage by disrupting the recognizable shape or the orientation of the animal. This kind of camouflage creates what I call ‘same but different object’, described in detail in the next sections.

Same but different objects

In Figures 5.5, all results, elicited by the similarity principle, contain complementary objects, whose stimulus pattern is a complex set of segments starting from juxtaposed to superimposed geometrical shapes. In order to test the role of the Gestalt principles of grouping and figure-ground segregation and to understand more deeply the meaning of object perception and the role of the assumption of homogeneity, the previous figures have been reduced to a single limiting condition to be broken down in different ways and regions. This paradigm is reminiscent of the one used in physics to explore the properties of elementary particles, but it is opposite to the holism embraced by Gestalt psychologists (Koffka, 1935), mainly based on the assumption that the whole is greater than the sum of its parts, thus implying that the whole cannot be broken down.

The questions are:

What happens if the wholeness and uniqueness of one object is weakened, discontinued, disrupted by contrast polarity?

Do the elementary properties emerge?

Are they the primitive components/properties defining and conveying the meaning of a visual object?

The answers to these questions are subsumed in the phenomenological arguments, described next, based on variations within several limiting conditions. In Figure 5.6, the black boundaries of the eight-pointed star of Figure 5.2a have been discontinued by reversing the contrast polarity of some of the black segments that now appear white. In terms of grouping, the closure principle is mainly responsible for the results perceived at first sight in the conditions illustrated in Figure 5.5a, i.e. eight-pointed stars with black and white sides. On a deeper phenomenological observation, the uniqueness and wholeness of each star appear weakened or broken down. The whole shape partially splits in two black and white adjacent components that are seen as belonging to two different objects. The most extensive black grouping reveals more easily its whole shape that, given the presence of the white sides and their ungrouping with the black sides, is not the eight-pointed star of Figure 5.2a, but the shapes with inner gaps interrupting the boundary continuation similar to those illustrated in Figure 5.6b. The removal of the white components enhances the grouping of the black segments according to the closure and good continuation principles. The effect is enhanced by virtue of the now restored assumption of chromatic homogeneity of the boundaries. The missing sides do not affect the whole shape but appear as gaps favoring the good continuation of the modal sides and the amodal wholeness (Pinna, 2010b). Given the black and white segment alternation, only the last condition is analogous to the star of Figure 5.2a.

Figure 5.6 Same but different objects by breaking the assumption of homogeneity.

These new and different phenomenal shapes are only apparently invisible in the star from which I started (Figure 5.2a).

In fact, going back to it and having the results of Figure 5.6b as a visual reference, it becomes now easy within the star of Figure 5.2a to see the subgroupings within the star of Figure 5.6b. The outcomes of Figure 5.6 clearly suggest that the same figure can be different, i.e. the same figure can manifest different possible objects, all of them implicitly included within the figure but invisible according to the winner-takes-all phenomenal principle. The emergence of other possible objects can be accomplished as soon as the homogeneity assumption is broken down again or discontinued.

To better answer the previous basic questions it is necessary to reduce the star to an even simpler and limiting condition where the good continuation and the similarity are nearly ineffective. This is the case of the polygons illustrated in Figure 5.7, where other possible groupings, due to the good continuation, except the one of the polygon, are totally neglected.

Figure 5.7 The accentuation of the ‘pointedness’ and ‘sidedness’ attributes.

In Figure 5.7a, a regular octagon is perceived with the vertices oriented along the main directions (vertical and horizontal) of space. By introducing achromatic discontinuities on the two vertices placed along the vertical axes of the polygon (Figure 5.7b), the same regular octagon is seen. However, by introducing similar discontinuities in two opposite sides placed along an oblique axes as shown in Figure 5.7c, the polygon appears now the same but, at the same time, different: its shape appears more similar to the octagon of Figure 5.7d with the sides now oriented along the main directions of space.

Unlike the stars of Figure 5.6, the wholeness of the polygons of Figures 5.7b–c is not broken down or disrupted. This effect mainly occurs due to the role played by the good continuation principle oriented only along the boundaries of the octagon and, therefore, keeping the whole shape grouped and unique in spite of the discontinuities of the achromatic colors on its boundaries. Phenomenally, by comparing Figures 5.7b and 5.7c, the white discontinuities, placed on the vertices (angles) and sides, highlight and accentuate within the shape different components (points/angles and sides) that correspond to different object attributes. These attributes are ‘pointedness’ and ‘sidedness’ (Pinna, 2012a; Pinna and Sirigu, 2011).

More specifically, the white dashes within the polygons behave phenomenally like accents inducing the popping out of one or another geometrical basic component of the polygon: angles and sides. The effect induced by the white dashes does not represent the source of the disruption of the object, as shown in the previous figures, but the accentuation of a specific attribute that is now placed in a prominent role. This change in the inner object organization of geometrical attributes determines a change also of the whole object, i.e. the accent changes the nature and the meaning of the same object. The pointedness and sidedness, accentuated respectively in Figures 5.7b–c, create two different polygons (same but different), one more pointed and the other flatter.

Object attributes and related effects

Some other related effects corroborate these outcomes. First of all, Figures 5.7b–c show a clear illusion of numerousness of the components: at a first sight, the octagon of Figure 5.7b appears as having a higher number of points than the octagon of Figure 5.7c; the opposite is true about the number of sides. Furthermore, the vertices of Figure 5.7b appear more pointed than those of Figure 5.7c, while the sides of Figure 5.7c are perceived longer than those of Figure 5.7b. Similar phenomena are perceived by comparing Figures 5.7a and 5.7d, where the source of accentuation of respectively the points and the sides are now the main direction of space, i.e. the spatial orientation of the octagons.

Another interesting effect related to the accentuation principle of object perception is illustrated in Figures 5.7e and 5.7f, where the vertical and horizontal sizes of the two pointed octagons are perceived different: the octagon of Figure 5.7e appears slightly higher and elongated, while the one of Figure 5.7f appears larger. Under these conditions, the accentuation does not work in different components of the shape, thus showing sidedness and pointedness, but along the main directions of space, therefore by enhancing a specific direction and polarizing the whole shape in that specific direction: vertical or horizontal.

Similar shape effects are shown in Figure 5.8, where the inner squared shapes, created by the white segments, appear like vertical (Figure 5.8a) or horizontal (Figure 5.8b) rectangles, although geometrically they are squares (see the illusion of rectangle, Pinna, 2010b). Oblique distortions of the whole pattern in opposite directions can instead be perceived in Figures 5.8c and 5.8d. The comparison of the four Figures 5.8a–d makes the resulting effects even stronger. A control is illustrated in Figure 5.8e. The illusion of rectangle can also be induced by the similarity principle as shown in Figures 5.8g and 5.8h. Here, the grouping in columns and rows induces a polarization of the whole shapes, made up of squares and circles, in the same direction of the grouping and, as a consequence, the illusion of rectangle (see also Pinna, 2010b, 2012a). Very compelling up/down and left/right polarizations and misalignments, rectangle illusions with apparent motion and peripheral drift effects are illustrated in Figure 5.8i.

Figure 5.8 Some illusions related to the accentuation of object attributes (see the text).

These phenomenal outcomes cannot be explained in terms of figure-ground segregation or grouping. Although the effects of Figures 5.8g–h demonstrate that some role is played by the similarity principle in polarizing the object shape, it cannot explain the whole effect due to the accentuation of specific object attributes. The accentuation principle goes beyond Gestalt principles showing the complexity of the problem of object perception due to the inner organization of opposite object attributes (e.g. points and sides, vertical–horizontal). The limited role of the grouping principles to the understanding of the meaning of visual objects is demonstrated in the next section.

One, none, and a hundred thousand objects

In Figure 5.9a, the accentuation is imparted by a black dot. Under these conditions, the grouping, as studied by Wertheimer, is totally ineffective. The only expected result is the segregation and ungrouping of each irregular quadrilateral shape from the black dot. In terms of accentuation, each dot favors the popping out of the object attributes placed in that specific location of the figure. As a consequence, the rows of irregular quadrilaterals are perceived as different irregular shapes, difficult to recognize as equivalent geometrical figures where the only variation is the location of the dot. By popping out different object attributes, the accent determines the specific shape and its meanings in terms of orientation, size, pointing and so on. These properties are immediately perceivable and change in relation to the accent position within the figure.

Figure 5.9 Shape changes due to the accentuation principle and the organic articulation.

The results of Figure 5.9a point towards the following conditions related to a new kind of perceptual organization not yet studied, although it is very common in every day visual life. This is the organic articulation driven by a meta-shape attribute called ‘headedness’ (Pinna, 2012a). This attribute is immediately perceptible within the irregular undulated objects of Figure 5.9b appearing personified, each with a head and upper and lower limbs, moving in the direction defined by what is perceived as the head of the organism due to the accent. By changing the spatial position of the dot, the whole organic articulation and, ipso facto, the direction of the perceived motion and all the static, dynamic, and expressive characteristics of the organism change accordingly. Briefly, by changing the location of the accent the organism changes its shape and renders a different living creature.

It is worth noticing that all these attributes can be spontaneously highlighted through our own gaze and by changing the focus of attention, i.e. without the need of external accents. In this way the accent is represented by attention itself. This can be easily demonstrated by watching the control shown in the last condition of Figure 5.9b. This suggests that the accentuation principle, described here, traces a bridge between perception (even auditory and musical, see Pinna and Sirigu, 2011) and attention.

Deceiving object camouflage by accentuation

Finally, I suggest some biological implications about the way the accentuation principle is used in nature by different organisms.

I previously mentioned disruptive camouflage (see Merilaita and Lind, 2006) as a way to confuse an individual organism with high contrasted colorations and markings, which disguise form and shape. These markings appear quite distinctive, therefore preventing the predator from accurately identifying shape, size, and orientation. They exploit edge detection and, more generally, object perception mechanisms of early visual processing.

There is a specific kind of disruptive camouflage more directly related to the accentuation principle and similar to the organic articulation of Figure 5.9b. In Figure 5.10, some example of organisms with false eyes (ocelli) and dots (diematic patterns) demonstrate what I call ‘deceiving camouflage by accentuation’ aimed at:

confusing the predators by changing the body organization, size, orientation, and motion direction (see the fishes illustrated here) and

at hiding the most vital and important part of the body (see the butterflies).

Figure 5.10 Examples of deceiving camouflage by accentuation.

By depending on the accentuation principle, these kinds of disrupting and deceiving effects play different biological roles at the same time: not only defense mechanisms but also to startle or frighten potential predators, to become a source of sexual attraction, to advertise the presence, and to elicit species identification/communication.

These results are far beyond the Gestalt theory of perceptual organization and contain some theoretical implications summarized and discussed in the next section.

Summary/conclusion

In the previous sections I demonstrated that both figure-ground segregation and grouping as studied by Gestalt psychologists play an important role in the understanding of object perception. The principles discovered by Rubin and Wertheimer can explain phenomenologically the multiplicity of visual objects contained implicitly within the same stimulus pattern. Nevertheless, they do not face directly the main basic questions: What is a visual object and what is its meaning?

As demonstrated by the previous examples (Figures 5.8 and 5.9), grouping per se does not make any prediction about the meaning of the object shape, about its inner attributes and the way they are organized. As a matter of fact, the role of the Gestalt principles of grouping is to define the rules of ‘what is or stay with what’, I.e. the grouping and not the shape. The notion of ‘object’ due to grouping is, in fact, phenomenally different from the one due to shape. The form of the object emerging from the grouping represents the set of elements that assume the role of ‘parts’ within a holistic percept. The form of shape is instead the outcome of a holistic perceptual process emerging parallel to or after the form of grouping and giving to the object a unitary form along the boundary contours. It follows that grouping and shape formation can be considered as two complementary integrated processes of perceptual organization.

I also suggested that the object attributes are like meaningful primitives, phenomenally relevant, of the complex language of object perception. This entails that the meaning of an object can be understood on the basis of a multiplicity of attributes. Moreover, it follows that the notion of object can be phenomenally represented like a whole visual ‘thing’ containing a set of specific phenomenal primitive attributes, i.e. like the holder of object attributes. In this view, the shape attributes are not necessarily placed all at the same phenomenal level within the gradient of visibility (Pinna, 2010a), but some emerge more strongly than others depending on the factors that can influence their strength and visibility. Among these factors, I demonstrated the role of the accentuation principle due to discontinuities of contrast polarity, dot elements placed in different shape locations, horizontal/vertical axes, and focus of attention (for other factors see Pinna, 2012a).

All the shape attributes are present at the same time, but some or only one emerges as the winner that imparts the basic meaning to the object (the winner-takes-all) It follows that the shape attributes compete and cooperate and, above all, they are all present at the same time, placed along the gradient of visibility and in a dynamic state of equilibrium that can be changed by accentuating the opposite or other competing attributes.

In conclusion, this approach, based on the idea of visual object as a holder of specific primitive attributes, suggests new possible ways to understand the uncertainty of the object multiplicity and, finally, to answer the question ‘What is a visual object?’ It can also have consequences in terms of experimental phenomenology, neural circuitry, and biology of vision. Grossberg and Pinna (2012) extend brain design principles and neural networks, as described in Grossberg’s FACADE, 3D LAMINART, ART, and ARTSCAN neural models, to explain what a visual object is in terms of three main kinds of perceptual organization: grouping, shape, and meaning.

Acknowledgments

Supported by Finanziamento della Regione Autonoma della Sardegna, ai sensi della L.R. 7 agosto 2007, n. 7, Fondo d’Ateneo (ex 60%) and Alexander von Humboldt Foundation.

References

Grossberg, S. (1997) ‘Cortical dynamics of three-dimensional figure-ground perception of two-dimensional pictures’, Psychological Review, 104: 618–658.

Grossberg, S., and Pinna B. (2012) ‘Neural dynamics of Gestalt principles of perceptual organization: From grouping to shape and meaning’, Gestalt Theory, 34: 399–482.

Katz, D. (1930) Die Erscheinungsweisen der Farben, 2nd edn. English translation: MacLeod R. B., and Fox C. W. (1935), The World of Color, London, UK: Kegan Paul.

Koffka, K. (1935) Principles of Gestalt Psychology, London, UK: Routledge and Kegan Paul.

Merilaita, S. and Lind, J. (2006) ‘Great tits (Parus major) searching for artificial prey: Implications for cryptic coloration and symmetry’, Behavioral Ecology, 17: 84–87.

Oster, M., Douglas, R., and Liu, S. C. (2009) ‘Computation with spikes in a winner-take-all network’, Neural Computation, 21(9): 2437–2465.

Palmer, S. E. (1999) Vision Science: photons to phenomenology, Cambridge, MA: MIT Press.

Pinna, B. (2010a) ‘What comes before psychophysics? The problem of “what we perceive” and the phenomenological exploration of new effects’, Seeing and Perceiving, 23: 463–481.

Pinna, B. (2010b) ‘New Gestalt principles of perceptual organization: An extension from grouping to shape and meaning’, Gestalt Theory, 32: 1–67.

Pinna, B. (2012a) ‘What is the meaning of shape?’, Gestalt Theory, 33: 383–422.

Pinna, B. (2012b) ‘Perceptual organization of shape, color, shade and lighting in visual and pictorial objects’, i-Perception, 3: 257–281.

Pinna, B. (2012c) ‘The organization of shape and color in vision and art’, Frontiers in Human Neuroscience, 5, article 104. doi: 10.3389/fnhum.2011.00104.

Pinna, B., and Reeves, A. (2006) ‘Lighting, backlighting and watercolor illusions and the laws of figurality’, Spatial Vision, 19: 341–373.

Pinna, B., and Sirigu, L. (2011) ‘The accentuation principle of visual organization and the illusion of musical upbeat suspension’, Seeing and Perceiving, 24: 595–621.

Rubin, E. (1921) Visuell wahrgenommene Figuren, Kobenhavn, Denmark: Gyldendalske Boghandel.

Spillmann, L., and Ehrenstein, W. H. (2004) ‘Gestalt factors in the visual neurosciences’, in L. Chalupa and J. S. Werner (eds), The Visual Neurosciences. Cambridge, MA: MIT Press, pp. 1573–1589.

Tanca, M., Pinna, B., and Grossberg, S. (2010) ‘Probing perceptual antinomies with the watercolor illusion and explaining how the brain resolves them’, Seeing and Perceiving, 23: 295–333.

Wertheimer, M. (1923) ‘Untersuchungen zur Lehre von der Gestalt II’, Psychologische Forschung, 4: 301–350.

6    Self-organizing properties of the visual field

Gestalt forces in action

Lewis O. Harvey, Jr. and Eric K. Schmidt

Introduction

From the beginning of the Gestalt movement in 1912 the impact of Max Werheimer’s dictum (Spillmann, 2012b; Wertheimer, 1912, 1923, 1925) and precisely articulated by Kurt Koffka as

It has been said: The whole is more than the sum of its parts. It is more correct to say that the whole is something else than the sum of its parts, because summing up is a meaningless procedure, whereas the whole-part relationship is meaningful.

(Koffka, 1935: 176)

has been profound. One area of research has aimed at understanding the basis of the Gestalt principles of figure organization and the meaning of Prägnanz, which itself has multiple meanings (Kanizsa and Luccio, 1986; Luccio, 1999). Inspired by the writings of Wertheimer (1923, 1925), Köhler (1940), Köhler and von Restorff (1933, 1937), Koffka (1935) and Lewin (1936) there have been numerous experimental attempts to measure and define figural goodness. Hubbell (1940) (Mowatt, 1940 in Beardslee and Wertheimer, 1958)¹ and Hochberg and McAlister (1953) asked subjects to rate the attributes of stimuli or had them adjust drawings to produce ‘good’ figures. They found that subjects’ versions of ‘good’ figures are fairly consistent, often leaning towards simple, symmetrical shapes.

Research on the Gestalt principles has continued to the present day as seen in recent reviews (Pinna, 2010; Spillmann, 2010, 2012a; Wagemans et al., 2012a, 2012b). The Gestalt principles have found practical applications too, ranging from graphics (Moore and Fitz, 1993a, 1993b), to video games, industrial design and systems design (Lidwell et al., 2003).

In this chapter we will focus on dynamic interactions among visual elements within the field of view. The concept of isomorphism was introduced by Köhler to provide a concrete mechanism whose actions could account for Prägnanz and the rules of perceptual organization. Köhler described isomorphism as ‘any actual consciousness is in every case not only blindly coupled to its corresponding psychophysical processes, but is akin to it in essential structural properties’ (quoted in Koffka, 1935: 62).

This idea that perceptual experience is directly related to the activity of the nervous system and therefore to the properties of internal representations constructed by the brain is a dramatic departure from the view that stimuli and their properties are perceived directly. The concept of isomorphism is ambiguous (Luccio, 2010), but one implication is that our perceptual experience is due to a dynamic interaction among stimulus elements and to the functional interactive properties of the nervous system. The nature of this interaction is the principle determinant of our perceptual experience.

Köhler and Wallach (1944) proposed that visual representations are based on point-to-point electrical field activity flowing over the surface of the visual cortex. The sources of these fields are the physical, distal stimuli and the resulting neural activity evoked by their proximal stimuli in the early stages of the visual system. One property of an electric field is that it radiates out from its source and exerts influence at a distance from the source. Multiple stimulus elements each create fields and these fields interact with each other and create a field configuration (i.e. a perception) that can be quite different from the fields created by each stimulus alone.

The cortical field hypothesis was tested by inserting gold strips (conductors) or mica plates (insulators) into the cortices of animals to disrupt these hypothesized cortical electrical fields (Lashley et al., 1951; Sperry and Miner, 1955). The predicted disruptions of perception in these animals, however, did not occur. Today, of course, we understand that the interaction among local parts of the cortex is not by means of electric fields flowing over the cortical surface, but rather by means of lateral excitatory and inhibitory neuronal interconnections. Because dendritic density falls off as the square of the distance from the neuronal cell body, the functional properties of these excitatory and inhibitory interactions are generally similar to those of interacting electric fields whose influences also weaken with the square of the distance from a source. So although the specific mechanism of electric fields was not correct, the basic idea of dynamic interaction at a distance between excitatory and inhibitory interactions is.

A system that contains multiple components interacting with each other can, under certain conditions, organize itself automatically to give rise to emergent properties not anticipated from the properties of the individual components, hinted at by Koffka who evoked ‘shape-giving forces’ (Koffka, 1935: 132) to account for the Gestalt principles. In the past 30 years, the study of self-organizing systems (Bak, 1996; Haken, 1983, 1990) has provided tools to reexamine the dynamics underlying our perceptual experiences. Many psychological phenomena can plausibly be attributed to the operation of self-organizing, dynamic systems in humans (Bak, 1996; Correll, 2008; Gilden, 2001; Van Orden et al., 2005).

Self-organizing isomorphic forces can affect the perceived and remembered spatial locations of targets presented within an enclosing frame or shape. Giovanelli (1966) showed that the perceived collinearity of a row of points was destroyed by framing the points asymmetrically within triangles or circles. The contours of the framing shapes influence the perceived position of the enclosed points, causing an illusion of non-collinearity. An example of this illusion is shown in the upper panel of Figure 6.1. Her demonstrations showed that the direction of perceptual shift within a field depends on the shape of the field, as well as on the location of points within the field in relation to the contour of the enclosing shape.

Figure 6.1 Upper panel: An example of the Giovanelli Illusion; Lower panel: Localization error vectors from within a 23 degree circular field. (Reconstructed by the authors from Attneave, 1955.)

Effect of shape contour on localization

Attneave (1955) presented a small target randomly at each of 91 locations inside a 23 degree circular field for 5 s. After a short delay, subjects indicated the location of the target. The errors between the actual location and the indicated location of each target were not random: they showed a radiating pattern relative to the edges of the field. The lower panel of Figure 6.1 summarizes Attneave’s results. The data from the complete circle have been folded into the upper right sector of the circle to make the essential findings clear. Filled circles mark the actual positions of the targets; the length of each line is the mean location error multiplied by five. These experiments show that both the perceived location (Giovanelli, 1966) and the remembered location (Attneave, 1955) of targets are strongly influenced by the proximity of nearby contours.

A striking feature of Attneave’s results is that the error vectors point away from the nearest contour and point toward a single region inside the quadrant of the circle. The closer the target was to this region, the smaller the error magnitude. It is as if the target points are being attracted to this region, an attractor basin in the language of dynamic systems theory (Haken, 1983, 1984). Targets outside this basin are drawn toward it; targets located at the basin are in a state of stable equilibrium.

Although Motokawa (1950; Motokawa et al., 1957) and Yacorzynski (1963, 1966, 1974, 1975, 1979) had developed descriptive models of isomorphic electric-like fields that surround shapes and fill their interiors, and although explicit neural models of the visual system have been developed that had properties (competitive and cooperative interactions) which could produce figural aftereffects and other related phenomena (for example, Grossberg and Mingolla, 1985; O’Reilly and Munakata, 2000), there was no formal method of analyzing the pattern of error vectors that would lead to a deeper understanding of the nature of the ‘forces’ underlying localization errors until the landmark paper of Stadler et al. (1991).

Vector field decomposition of localization errors

Stadler and colleagues (1991) applied methods of vector calculus to decompose the pattern of errors observed in a localization experiment into two underlying components: a gradient field and a circulation field (Schey, 2005). In physics, for example, an electromagnetic field can be described as the sum of an electrostatic field (gradient) and a magnetic field (circulation). When this decomposition is applied to localization errors, the resulting gradient and circulation fields are possibly the isomorphic fields hypothesized by Köhler (1940) and Koffka (1935) that represent the stimuli.

Stadler et al. (1991) applied the vector decomposition analysis to the pattern of localization errors to small dot targets presented on sheets of standard A4 paper. They concluded that the underlying gradient field accounted for these localization errors and that the circulation field played no significant role. This conclusion has been repeatedly confirmed by other experiments (Igel, 1993; Igel and Harvey, 1991; Luccio and Caporusso, 2010; Pfaff, 1991; Pfaff et al., 1996; Schmidt, 1991; Stadler et al., 1999).

Attractor basins and unstable equilibria points

The salient features of the gradient fields found by Stadler et al. (1991) are illustrated in Figure 6.2 using similar findings from Schmidt (1991), who had subjects reproduce the location of a small circular target present for 1 s within a five degree white square displayed against a blue background on a computer driven cathode ray tube (CRT) display. Subjects indicated the location of the target by positioning a mouse cursor and clicking. The patterns of errors were decomposed into gradient and circulation fields using computer routines written in C and in Mathematica (Wolfram, 1991) by the authors using appropriate vector decomposition algorithms (Schey, 2005; Stadler et al., 1991). The upper left panel of Figure 6.2 clearly shows that the localization data reveal four stable equilibrium regions located near, but not at, each corner of the square. There is also an unstable equilibrium point near the center of the square from which targets are repulsed, not attracted. Targets are repulsed from the contour of the square edge as well as from the center. These features are beautifully captured by the computed gradient potential, illustrated in the lower panels of Figure 6.2. The gradient potential field is shown as a topographical surface on the left and as a contour plot on the right. In agreement with Stadler et al. (1991) the circulation fields played no significant role in predicting the observed errors.

Figure 6.2 The gradient potential computed from target localization errors inside a 5 degree square frame. Upper left: Observed localization errors made by subjects to targets presented within a square frame. The dots mark the actual location of the target, the end of the lines mark the mean location given by 40 subjects. Upper right: The localization errors predicted by the computed gradient field. Lower left: The potential gradient plotted as a three dimensional surface. Lower right: The potential gradient field plotted as a contour map. (The figures are based on data collected by the second author and reported in his thesis Schmidt, 1991.)

Schmidt (1991) used five different frame shapes: square, circle, sector, triangle and L-shape. The gradient fields computed from the error vectors are shown in Figure 6.3 both as surface plots and as contour plots. There are three features that these five gradient fields have in common. They all have attractor basins located near, but not in, the corners (if the enclosing shape has corners) as well as peaks and saddles of unstable equilibria aligned with the main axes of symmetry (horizontal and vertical). Perceived or remembered target locations are shifted away from peaks and ridges toward basins. These features offer an explanation of the Giovanelli illusion illustrated in Figure 6.1: the dots within the circle are located on the vertical axis of symmetry which subjects them to forces that shift their perceived position both laterally to move off the saddle, and to move ‘downhill’ toward the local minimum along the ridge in the direction of the center. Thus, by virtue of their position within the circle, some dots are shifted upwards and some are shifted downwards, destroying the perception of collinearity.

Figure 6.3 Computed gradient potential fields with square, circle, sector, triangle and L-shaped frames. (The figures are based on data collected by the second author and reported in his thesis Schmidt, 1991.)

Although the circle has no corners, there are still strong attractor basins located along the 45 and 135 degree axes, as was first reported by Attneave (1955) and also by Pfaff (1991; Pfaff et al., 1996; Stadler et al., 1999). The sector figure has a gradient field that is a combination of the square and circle gradient fields: attractor basins near the three corners and a weak basin on the 45 deg diagonal. An unstable equilibrium peak occurs near the center. The L-shaped frame also shows the pattern of attractor basins and peaks rather like the square pattern but more complicated due to the more complex edges of the frame.

Drawing a second figure within an enclosing frame disrupts the nature of gradient fields, at least when using A4 paper as the frame. Pfaff found that by drawing a trapezoidal figure within the white confines of the A4 paper frame, the pattern of errors point from the paper edge toward the center and the influence of the trapezoidal figure is obscured by this main effect of the frame (Pfaff et al., 1996; Stadler et al., 1999). One dramatic finding from Pfaff’s experiments is that a subjective contour created in the Kanizsa triangle (Kanizsa, 1979) is as effective or even more effective than a triangle drawn with physical contours in producing shifts of target location (Pfaff et al., 1996; Stadler et al., 1999).

Effects of time delay on localization errors

In Attneave’s (1955) experiment, subjects were shown the target for 5.0 s within the circular field and then they performed a simple distractor task before being permitted to indicate the target location. Schmidt (1991) varied the delay between the offset of the target and the appearance of the response cursor using 2, 4, 6 and 8 s delays. He found no effect of time delay on the magnitude and pattern of the localization errors. Igel (1993; Igel and Harvey, 1991) found the same pattern of errors and gradient fields inside a square figure with 0 s delay between target and response cursor. We conclude that short delays up to 8 s between target and response do not affect the dynamics underlying the localization errors.

Effects of field size on localization errors

Schmidt (1991) also varied the size of the square and circular fields using angular widths of 2.5, 3.54, 5.0, 7.07 and 10 degree of visual angle achieved by changing the viewing distance of the subject from the computer monitor. The same pattern of localization errors and gradient fields were found at all angular sizes. Importantly, the attractor basins occurred at the same relative position within the square and circular frames, not the same absolute position on the computer monitor or on the retinal image.

Effects of number of targets on localization errors

Up to about seven, one can rapidly and accurately report the number of targets presented simultaneously; but accuracy falls off and response time increases above seven (Kaufman et al., 1949; Miller, 1956). Igel (1993; Igel and Harvey, 1991) measured localization of small targets presented inside and outside a 5.0 degree wide line square displayed on a computer monitor. The monitor was 10.0 degrees wide, the targets were presented on a 29 × 29 square grid of possible loci. Subjects were shown 1, 2, 3, 4, 5, 6, 7, 8, 9 or 10 targets presented either simultaneously or sequentially. The gradient fields computed from the localization errors are shown in Figure 6.4 for 1, 2, 4 and 8 target dots presented sequentially. The gradient field for 1 target shows the familiar attractor basin characteristic and an unstable equilibrium point near the center. With a memory load of more than one target, however, the influence of the square frame is almost entirely absent. The performance on this task is therefore not limited by the capacity of short-term working memory because even two target dots destroy the basic gradient field.

Figure 6.4 Gradient fields computed from localization errors of 1, 2, 4 and 8 sequentially presented targets. (Computed from original data in Igel and Harvey, 1991.)

Implications for artistic composition

In the growing field of neuroaesthetics (Rentschler et al., 1988; Zeki, 2009) researchers have extracted universal principles relating vision and the brain to explain the compositional choices that painters make when creating art (Pinna, 2007; Ramachandran and Hirstein, 1997, 1999; Tyler, 1999, 2007; Tyler and Likova, 2012). For example, Tyler examined 265 portraits painted over the past 600 years and noted that the artist placed one of the eyes on the vertical axis of the frame (Tyler, 1998, 1999).

In Figure 6.5 the gradient field computed from the square frame localization errors (Schmidt, 1991) have been stretched to fit over the full version of Leonardo da Vinci’s painting Mona Lisa. The portrait’s left eye is close to the vertical midline of the painting (Tyler, 1998). But not only is the eye almost centered, both eyes are placed on the local minimum of the midline ridge of the gradient field. The fingers of the left hand are place at the corresponding local minimum of the lower half of the picture. The left arm is placed parallel to the contours of the gradient field while the right arm runs down the steepest part of the gradient. Future research should examine the relationship between composition and gradient fields more thoroughly.

Figure 6.5 The gradient field computed from square frame data of the second author (Schmidt, 1991) scaled to cover the full frame of Leonardo da Vinci’s Mona Lisa. (The original photo of the Mona Lisa is from Wikipedia Commons.)

Other localization research

Following Attneave’s (1955) experiment, other studies on localization error appeared whose theoretical interest was in understanding internal psycholophysi-cal scales (Nelson and Chaiklin, 1980) rather than Gestalt forces. Huttenlocher and colleagues (1991) introduced a two-level model that combines sensory memory information with the cognitive influences of category assignments and category boundary shifts to explain localization errors. In studies that followed (Huttenlocher and Lourenco, 2007; Huttenlocher et al., 1994, 2000, 2004; Laeng et al., 1998; Lourenco and Huttenlocher, 2006) they concluded that categorization effects were largely responsible for errors. Recently, however, Friedman (2009; Friedman et al., 2012) pointed out that this conclusion is not warranted because of confounds in the experimental designs. In her experiments Friedman used outlines of provinces/states (Alberta and California) as frames within which to present target dots. To separate the contour effects from category effects she had subjects localize dots representing city locations using just the dot alone, the dot and the city name, and just the city name. Friedman concluded that dot localization error is driven mostly by the presence of framing contours and not higher level categorization (Friedman et al., 2012).

Summary/conclusion

The experiments cited above did not sample target locations in a manner that would allow gradient fields to be computed. When we examine the results of these experiments they all show patterns of errors consistent with those predicted by the computed gradient fields illustrated here, strengthening Friedman’s conclusion that contour is more important than cognitive influence. It is exciting to realize that the revolution started by the Gestalt psychologists has maintained its freshness and vigor and is still relevant today. We anticipate that future experiments will combine gradient field analysis with both contour manipulations (bottom-up) and cognitive manipulations (top-down) to give a deeper understanding of how these factors interact with each other and contribute to our perceptual experience.

Note

1. The Hubbell (1940) paper has been cited as Mowett (1940) because when it was reprinted in David Beardslee and Michael Werheimer’s 1958 book of readings, her married name, Mowatt, was substituted for Hubbell, her maiden name that she used in the original publication.

References

Attneave, F. (1955) ‘Perception of place in a circular field’, American Journal of Psychology, 68(1): 69–82.

Bak, P. (1996) How nature works: the science of self-organized criticality, New York, NY: Copernicus.

Beardslee, D. C. and Wertheimer, M. (1958) Readings in perception. Princeton, NJ: Van Nostrand.

Correll, J. (2008) ‘1/f noise and effort on implicit measures of bias’, Journal of Personality and Social Psychology, 94(1): 48–59.

Friedman, A. (2009) ‘The role of categories and spatial cuing in global-scale location estimates’, Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(1): 94–112.

Friedman, A., Montello, D. R. and Burte, H. (2012) ‘Location memory for dots in polygons versus cities in regions: evaluating the category adjustment model’, Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(5): 1336–1351.

Gilden, D. L. (2001) ‘Cognitive emissions of 1/f noise’, Psychological Review, 108(1): 3356.

Giovanelli, G. (1966) ‘Stati di tensione e di equilibrio nel campo percettivo’, Revista di Psicologia, 60(4): 327–335.

Grossberg, S. and Mingolla, E. (1985) ‘Neural dynamics of perceptual grouping: textures, boundaries, and emergent segmentations’, Perception & Psychophysics, 38(2): 141–171.

Haken, H. (1983) Synergetics: an introduction: nonequilibrium phase transitions and selforganization in physics, chemistry, and biology, Berlin, Germany: Springer.

Haken, H. (1984) The science of structure: synergetics, New York, NY: Van Nostrand Reinhold.

Haken, H. (1990) ‘Synergetics as a tool for the conceptualization and mathematization of cognition and behavior: how far can we go?’, in Haken, H. and Stadler, M. (eds) Synergetics of cognition, Berlin, Germany: Springer, pp. 2–31.

Hochberg, J. E. and McAlister, E. (1953) ‘A quantitative approach to figural “goodness”’, Journal of Experimental Psychology, 46(5): 361–364.

Hubbell, M. B. (1940) ‘Configurational properties considered “good” by naïve subjects’, The American Journal of Psychology, 53(1): 46–69.

Huttenlocher, J. and Lourenco, S. F. (2007) ‘Coding location in enclosed spaces: is geometry the principle?’, Developmental Science, 10(6): 741–746.

Huttenlocher, J., Hedges, L. V. and Duncan, S. (1991) ‘Categories and particulars: prototype effects in estimating spatial location’, Psychological Review, 98(3): 352–376.

Huttenlocher, J., Newcombe, N. and Sandberg, E. H. (1994) ‘The coding of spatial location in young children’, Cognitive Psychology, 27(2): 115–148.

Huttenlocher, J., Hedges, L. V. and Vevea, J. L. (2000) ‘Why do categories affect stimulus judgment’, Journal of Experimental Psychology, 129(2): 220–241.

Huttenlocher, J., Hedges, L. V., Corrigan, B. and Crawford, L. E. (2004) ‘Spatial categories and the estimation of location’, Cognition, 93(2): 75–97.

Igel, A. (1993) ‘Systematische Fehler bei der Ortsreproduktion: Ein Vergleich von ein- und zweidimensionalen Reizbedingungen’, unpublished diploma thesis, University of Regensburg.

Igel, A. and Harvey, L. O., Jr. (1991) ‘Spatial distortions in visual perception’, Gestalt Theory, 13(4): 210–231.

Kanizsa, G. (1979) Organization in vision, New York, NY: Praeger.

Kanizsa, G. and Luccio, R (1986) ‘Die Doppeldeutigkeiten der Prägnanz’, Gestalt Theory, 8(2): 99–135.

Kaufman, E. L., Lord, M. W., Reese, T. W. and Volkman, J. (1949) ‘The discrimination of visual number’, American Journal of Psychology, 62(4): 498–525.

Koffka, K. (1935) Principles of Gestalt psychology, New York, NY: Harcourt, Brace.

Köhler, W. (1940) Dynamics in psychology, New York, NY: Liveright.

Köhler, W. and von Restorff, H. (1933) ‘Analyse von Vorgängen im Spurenfeld. I. Über die Wirkung von Bereichsbildungen im Spurenfeld’, Psychologische Forschung, 18(1): 299–342.

Köhler, W. and von Restorff, H. (1937) ‘Analyse von Vorgängen im Spurenfeld II. Zur Theorie der Reproduktion’, Psychological Research, 21(1): 56–112.

Köhler, W. and Wallach, H. (1944) ‘Figural after-effects: An investigation of visual processes’, Proceedings of the American Philosophical Society, 88(4): 269–357.

Laeng, B., Peters, M. and McCabe, B. (1998) ‘Memory for locations within regions: spatial biases and visual hemifield differences’, Memory & Cognition, 26(1): 97–107.

Lashley, K. S., Chow, K. L. and Semmes, J. (1951) ‘An examination of the electrical field theory of cerebral organization’, Psychological Review, 58(2): 123–136.

Lewin, K (1936) Principles of topological psychology, New York, NY: McGraw-Hill.

Lidwell, W., Holden, K and Butler, J. (2003) Universal principles of design, Gloucester, MA: Rockport.

Lourenco, S. F. and Huttenlocher, J. (2006) ‘How do young children determine location? Evidence from disorientation tasks’, Cognition, 100(3): 511–529.

Luccio, R. (1999) ‘On Prägnanz’, in Albertazzi, L. (ed.) Shapes of forms: from gestalt psychology and phenomenology to ontology and mathematics, Dordrecht, The Netherlands; Boston, MA: Kluwer Academic Publishers, pp. 123–148.

Luccio, R (2010) ‘Anent isomorphism and its ambiguities: from Wertheimer to Köhler and back to Spinoza’, Gestalt Theory, 32(3): 219–262.

Luccio, R and Caporusso, G. (2010) ‘The effect of the context on the anisotropy of the visual field’, Review of Psychology, 17(1): 7–11.

Miller, G. A. (1956) ‘The magical number seven, plus or minus two: some limits on our capacity for processing information’, Psychological Review, 63(2): 81–97.

Moore, P. and Fitz, C. (1993a) ‘Gestalt theory and instructional design’, Journal of Technical Writing and Communication, 23(2): 137–157.

Moore, P. and Fitz, C. (1993b) ‘Using Gestalt theory to teach document design and graphics’, Technical Communication Quarterly, 2(4): 389–410.

Motokawa, K. (1950) ‘Field of retinal induction and optical illusion’, Journal of Neurophysiology, 13(6): 413–426.

Motokawa, K., Nakagawa, D. and Kohata, T. (1957) ‘Figural aftereffects and retinal induction’, Journal of General Psychology, 57(1): 121–135.

Nelson, T. O. and Chaiklin, S. (1980) ‘Immediate memory for spatial location’, Journal of Experimental Psychology: Human Learning and Memory, 6(5): 529–545.

O’Reilly, R C. and Munakata, Y. (2000) Computational explorations in cognitive neuroscience: Understanding the mind by simulating the brain, Cambridge, MA: MIT Press.

Pfaff, S. (1991) ‘Experimentelle Untersuchung zur Felddynamik des visuellen Systems’, unpublished diploma thesis, University of Bremen.

Pfaff, S., Kruse, P., Luccio, R and Stadler, M. (1996) ‘Field dynamics of visual perception: potential landscapes are dependent on the border conditions’, Gestalt Theory, 18(3): 163–186.

Pinna, B. (2007) ‘Art as a scientific object: toward a visual science of art’, Spatial Vision, 20(6): 493–508.

Pinna, B. (2010) ‘New Gestalt principles of perceptual organization: an extension from grouping to shape and meaning’, Gestalt Theory, 32(1): 11–78.

Ramachandran, V. S. and Hirstein, W. (1997) ‘Three laws of qualia: what neurology tells us about the biological functions of consciousness’, Journal of Consciousness Studies, 4(5–6): 429–457.

Ramachandran, V. S. and Hirstein, W. (1999) ‘The science of art: a neurological theory of aesthetic experience’, Journal of Consciousness Studies, 6(6–7): 15–51.

Rentschler, I., Herzberger, B. and Epstein, D. (eds) (1988) Beauty and the brain: biological aspects of aesthetics, Basel, Switzerland: Birkhäuser Verlag.

Schey, H. M. (2005) Div, grad, curl, and all that: an informal text on vector calculus, 4th edn, New York, NY: W.W. Norton.

Schmidt, E. K. (1991) ‘Self-organizing properties of vision: effects of field shape and field size’, unpublished thesis, University of Colorado, Boulder.

Sperry, R W. and Miner, N. (1955) ‘Pattern perception following insertion of mica plates into visual cortex’, Journal of Comparative Physiological Psychology, 48(6): 463–469.

Spillmann, L. (2010) ‘Introduction to Baingio Pinna’s “New Gestalt principles of perceptual organization: an extension from grouping to shape and meaning”’, Gestalt Theory, 32(1): 3–9.

Spillmann, L. (2012a) ‘The current status of Gestalt rules in perceptual research: psychophysics and neurophysiology’, in Spillmann, L. (ed.) Max Wertheimer: on perceived motion and figural organizaton, Cambridge, MA: MIT Press, pp. 191–252.

Spillmann, L. (ed.) (2012b) Max Wertheimer: on perceived motion and figural organization, Cambridge, MA: MIT Press.

Stadler, M., Richter, P. H., Pfaff, S. and Kruse, P. (1991) ‘Attractors and perceptual field dynamics of homogeneous stimulus areas’, Psychological Research, 53(2): 102–112.

Stadler, M., Pfaff, S. and Kruse, P. (1999) ‘Towards a theory offigural form’, in Albertazzi, L. (ed.) Shapes of forms: from Gestalt psychology and phenomenology to ontology and mathematics, Dordrecht, The Netherlands; Boston, MA: Kluwer Academic Publishers, pp. 107–122.

Tyler, C. W. (1998) ‘Painters centre one eye in portraits’, Nature, 392(6679): 877–878.

Tyler, C. W. (1999) ‘Is art lawful?’, Science, 285(5428): 673–674.

Tyler, C. W. (2007) ‘Some principles of spatial organization in art’, Spatial Vision, 20(6): 509–530.

Tyler, C. W. and Likova, L. (2012) ‘The role of the visual arts in the enhancing the learning process’, Frontiers in Human Neuroscience, 6(8): 1–7.

Van Orden, G. C., Holden, J. G. and Turvey, M. T. (2005) ‘Human cognition and 1/f scaling’, Journal of Experimental Psychology, 134(1): 117–123.

Wagemans, J., Elder, J. H., Kubovy, M., Palmer, S. E., Peterson, M. A., Singh, M. and von der Heydt, R (2012a) ‘A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure–ground organization’, Psychological Bulletin, 138(6): 1198–1218.

Wagemans, J., Feldman, J., Gepshtein, S., Kimchi, R., Pomerantz, J. R., van der Helm, P. A. and van Leeuwen, C. (2012b) ‘A century of Gestalt psychology in visual perception: II. Conceptual and theoretical foundations’, Psychological Bulletin, 138(6): 1218–1252.

Wertheimer, M. (1912) ‘Experimentelle Studien über das Gesehen von Bewegung’, Zeitschrift fur Psychololgie, 61, 301–350.

Wertheimer, M. (1923) ‘Untersuchungen zur Lehre von der Gestalt, II’, Psychologische Forschung, 4: 301–350.

Wertheimer, M. (1925) Über Gestalttheorie. Vortrag Gehalten in der Kant-Gesellschaft, Berlin, am 17. Dezember 1924, Erlangen, Germany: Verlag der Philosophischen Akademie, pp. 3–24.

Wolfram, S. (1991) Mathematica: a system for doing mathematics by computer, 2nd edn, New York, NY: Addison-Wesley.

Yacorzynski, G. K. (1963) ‘Brain dynamism as reflected in illusions’, Genetic Psychology Monographs, 68: 3–47.

Yacorzynski, G. K. (1966) ‘Brain dynamism in perception of reversible figures’, Genetic Psychology Monographs, 74(1): 37–87.

Yacorzynski, G. K (1974) ‘Brain dynamism in visual illusions of motion’, Genetic Psychology Monographs, 89(2): 169–220.

Yacorzynski, G. K. (1975) ‘Figural after-effects of Köhler and Wallach reinterpreted’, International Journal of Neuroscience, 6(5): 223–235.

Yacorzynski, G. K (1979) ‘The dynamics of the brain in perception’, Academic Psychology Bulletin, 1(2): 95–104.

Zeki, S. (2009) Splendors and miseries of the brain : love, creativity, and the quest for human happiness, Chichester, UK; Malden, MA: Wiley-Blackwell.

Part II

Attention, aftereffects and illusions

7    Attention, grouping and non-retinotopic representations

Haluk Öğmen and Michael H. Herzog

Introduction

The human visual system is faced with a staggering amount of information impinging on our retinae. Given the limited resources available at our disposal, relevant parts of this information need to be selectively filtered and organized into meaningful units. Gestaltists emphasized the roles of three major processes in achieving these tasks: attitude, attention and grouping. They defined attitude as a force that provides directedness to perception, such as the expectation of a particular organization or outcome (Koffka, 1935). Attention was defined as a special type of attitude representing an unspecific directedness, i.e. without a specific expectation of a particular organization or outcome, towards an object. Finally, grouping was defined as the organization of the input into unitary forms.

Are these three processes based on distinct mechanisms or are they simply different manifestations of a single unitary mechanism?

The Gestaltist view was that they were distinct but functionally interdependent processes (Koffka, 1922: 561). According to their hypothesis, how much these processes can influence each other depends on their strength. An ambiguous figure, such as Rubin’s vase shown in Figure 7.1a, represents a relatively weak and unstable perceptual organization in that one can switch easily between two different percepts. Shifting attention between white and dark regions of the image can demonstrate how attention can modulate perceptual organization by driving this bistable process between its two different equilibrium points. Figures 7.1b and 7.1c, on the other hand, show an example where attitude, attention or learning are unable to influence perceptual organization (Gottschaldt, 1926, 1929; Koffka, 1935). A sub-part of the geometrical shape in Figure 7.1c is the hexagon shown in Figure 7.1b. It is impossible to perceive this hexagon as a separate group inside the composite figure in Figure 7.1c, no matter how much attention or training is devoted to the task.

Figure 7.1 Various stimulus configurations used to investigate the role of attention on perceptual grouping. In d, on the first glance a square composed of dots is perceived. Closer inspection shows that vertical dot spacing is wider than horizontal one, giving rise to an alternated percept of five lines composed of dots. (From Aydin et al., 2011 with kind permission from Elsevier Ltd.)

The strength of the larger group prevents the perceptual dissociation of a smaller embedded group. The dot-square figure shown in Figure 7.1d provides an example where attention or learning can break down part-whole relations in perceptual grouping. At a large scale, the collection of dots can be perceived as a square. Focusing attention on a finer scale, based on the horizontal vs. vertical proximity of the dots, the figure can also be perceived as an array of horizontal lines. Based on these observations, as well as his animal experiments (Krechevsky, 1938), Gestalt psychologist Krech proposed a theory where perceptual organization creates a spectrum of configurations, ranging from simpler homogeneous structures (e.g. dots in Figure 7.1d grouped into a global homogeneous square pattern without column row differentiation) to more complex differentiated structures (e.g. dots grouped in an array of horizontal lines). From this spectrum of possibilities, the organism selects the configuration which suits best its goals related to attitudes and motivational factors. Furthermore, the selection process was hypothesized to progress in an orderly fashion, starting with homogeneous forms and proceeding towards more differentiated ones until the goal is satisfied.

Krech was faced with a challenge in testing his theory. In a typical experiment, subjects are given a task which becomes ‘the goal’. Thus, the goal given to the subjects should not interfere with the hypothesis under test, viz., the natural progression of configurations from homogeneous to differentiated ones. This led Krech and Calvin (1953) to devise a ‘decoy task’ approach. The stimuli consisted of brief presentations of the two patterns shown in Figures 7.1d and 7.1e. Patterns appeared either to the left or right of the center. In a given trial, the location of each pattern, left or right, was selected randomly. The task of the observer was to report which of the two is the ‘correct symbol’. Observers were not given any criterion for correctness and had to start by guessing. Feedback was given after each trial allowing the observer to discover eventually the underlying rule. The criterion for correctness was not based on shape information (square vs. X) but rather the sequence of locations to which patterns were presented (alternating left, right, left, right …). In order to test the theory, observers were also asked to reproduce each symbol as accurately as possible by placing beads on a board after each trial. The prediction of the theory was that during learning, observers’ perception of the dot stimuli will progress from a homogeneous square pattern to a differentiated line array pattern. This would imply that during initial trials observers would place beads in equidistant holes on the reproduction board. During the process of solving the problem, i.e. reaching their goal, their perception should become progressively more differentiated, leading to the placement of beads according to horizontal proximity relation. Their results provided support for their theory by showing that the percentage of observers reporting linear groupings increased from 14.2 per cent in the first trial to 46.4 per cent in the ninth trial. The decoy task approach was also used in a follow-up study by Köhler and Adams (1958). Their decoy task consisted of making aesthetic judgments about small cardboard figures that were shown on a background similar to that of Figure 7.1d. After six trials, observers were given a surprise question: they were asked to describe the background pattern. The results indicated that the proximity of dots needed for the perception of horizontal groups was much higher when attention was drawn by the decoy task compared to the case when there was no decoy task. Taken together, these studies support Krech’s theory by showing that when attitude or attention is not focused on a particular aspect of the stimulus, a condition called ‘incidental perception’ (Köhler and Adams, 1958), perceptual organization prefers simpler homogeneous configurations. Attitude or attention allows the articulation of more differentiated and more detailed perceptual organization. These techniques regained popularity more recently in the research area commonly called ‘inattentional blindness’ (Mack et al., 1992; Rock et al., 1992). For example, Mack et al. (1992) reported that texture segregation, and similarity grouping by lightness and proximity do not occur in the absence of attention. A study using a dual-task methodology also provided support for the finding that perceptual grouping by proximity and similarity requires attentional resources (Ben-Av et al., 1992). All these findings are in agreement with early Gestalt theories that attribute the lack of grouping under conditions of ‘inattention’ to the tendency of larger organizations to suppress more articulated organizations (Köhler and Adams, 1958) or to a preference for homogenous configurations over more differentiated ones (Krech and Calvin, 1953).

However, whether the decoy/cover task methods measure perception or memory remains an open question. Subjects are given surprise questions and thus need to rely on their memory for the report (Moore and Egeth, 1997; Rock et al., 1992). Moore and Egeth (1997) used an implicit-measure approach. The task was to judge the length of line segments presented in front of a background. While the background was not directly relevant to the task, it was arranged so that, if perceptually grouped, it could create length illusions (Müller-Lyer or Ponzo Illusions). Thus, observers’ length judgments would reflect whether or not unattended background elements were perceptually grouped. This and subsequent studies (Chan and Chua, 2003; Lamy et al., 2006; Russell and Driver, 2005; Shomstein et al., 2010) provided evidence that grouping occurs without attention. Grouping failures reported in earlier studies may be then attributed to failures of memory.

In considering these findings, one has to also take into consideration that explicit and implicit measures may be tapping at different levels and processes, as observed in other phenomena such as visual masking. A mask stimulus can render a target stimulus ‘invisible’ as measured by direct methods. Yet, it can be shown by indirect methods that this invisible target stimulus, which fails to reach the observer’s awareness, can nevertheless prime other stimuli (e.g. Ansorge et al., 1998; Breitmeyer et al., 2004, 2005; Klotz and Wolff, 1995; Vorberg et al., 2003). The different results obtained by different techniques can be explained within a theoretical framework consisting of parallel and interacting pathways of information processing (e.g. Breitmeyer et al., 2004, 2005). The failure to reach awareness does not imply that the stimulus has not been processed elsewhere in the visual system. In fact, in the context of grouping, it has been shown that, even when using implicit measures, not all groupings occur automatically in the absence of attention; instead the complexity of the grouping may dictate the attentional resources required (Kimchi and Razpurker-Apfeld, 2004).

Spatiotemporal grouping

Most of the grouping research is preoccupied with grouping in space. However, in a natural environment stimuli unfold in real-time, requiring grouping not only in space but also in time. Gestalt psychologist Joseph Ternus (1926) was among the first to study spatiotemporal grouping. He used a stimulus paradigm introduced by Pikler (1917). Figure 7.2 illustrates a typical ‘Ternus-Pikler display’ consisting of two frames each containing three elements. The three elements in the first frame are shifted to the right in the second frame by one inter-element distance so that the two rightmost elements of the first frame spatially overlap with the two leftmost elements of the second frame. The stimulus is designed so as to create an ambiguity: as shown in Figure 7.2 left panel, one possible grouping can be based on spatial position such that two of the elements may be grouped together by virtue of their spatial position that remains invariant over time. The remaining elements (the leftmost element in Frame 1 and the rightmost element in Frame 2) can be grouped together by apparent motion, a percept called element motion (EM). Another possible grouping can be based on a larger scale grouping of the three disks as a single Gestalt, resulting in a percept where the three disks move in group, as shown in Figure 7.2 right panel. This latter percept is called group motion (GM).

Figure 7.2 The Ternus-Pikler display. Two possible motion percepts: Element Motion (EM) at short ISIs and Group Motion (GM) at long ISIs. Motion correspondences are indicated by the dashed arrows (not shown in the actual displays).

Several studies showed that both EM and GM are possible percepts and which one will prevail depends on stimulus parameters, such as inter-element separation, element size, spatial frequency, contrast, inter-stimulus interval (ISI), luminance, frame duration and eccentricity (e.g. Alais and Lorenceau, 2002; Breitmeyer and Ritter, 1986a, 1986b; Dawson et al., 1994; Ma-Wyatt et al., 2005; Pantle and Picciano, 1976).

Non-retinotopic processing

To understand the neural foundations of these information filtering and grouping processes, one needs to put them in the context of neural representations of the visual system. The early visual system is retinotopically organized. However, under normal viewing conditions, due to both exogenous (object motion) and endogenous (head, body, eye movements) sources of motion, the retinotopic representation of stimuli is highly unstable. Given the visible persistence of the human visual system (e.g. Coltheart, 1980), our environment should appear highly blurred; yet our perception is relatively clear and stable. While Gestalt psychology provides numerous heuristics for perceptual grouping, understanding how these grouping principles apply given the transient and unstable nature of retinotopic representations remains a major challenge. We have proposed that the early retinotopic representations are transformed into non-retinotopic representations by using motion-based spatiotemporal grouping (rev. Öğmen and Herzog, 2010). Figure 7.3 illustrates the concept. At the bottom, dots move in the retinotopic space. Based on spatiotemporal grouping (common fate), dots moving in similar directions form distinct groups. These groups are then mapped into non-retinotopic representations, shown at the top of the figure. The common motion vector in retinotopic spatiotemporal grouping serves as the reference frame against which residual motions are computed in the non-retinotopic representations. According to our theory, features of elements are integrated according to motion grouping and not to retinotopic coordinates. The following experiments provide evidence for this hypothesis.

Figure 7.3 Schematic representation of how retinotopic representations are transformed into non-retinotopic representations. The bottom plane represents retinotopic areas. A simple stimulus consisting of two groups of dots is shown. One group tends to move rightwards while the second upwards. Based on motion-grouping, a common motion vector is determined. This common vector serves as the reference frame according to which the dots, constituting the group, are mapped to non-retinotopic representations with their residual motion vectors. To avoid clutter, only the non-retinotopic representation of the dots moving rightwards is shown.

Figure 7.4 shows our adaptation of the Ternus-Pikler display to study whether computation of form follows retinotopic or non-retinotopic principles (Öğmen et al., 2006). A ‘probe Vernier’ was inserted in the central element of the first frame and observers were asked to report the direction of the Vernier offset for each element in the second frame, even though these elements were not offset at all. Observers did not know where the actual Vernier offset was presented in the stimulus. According to a retinotopic processing hypothesis, observers’ response should be in agreement with the direction of the probe Vernier when they report element 1 in the second frame, and at about chance level with the probe Vernier when they report elements 2 and 3 of the second frame. According to our non-retinotopic processing hypothesis, attribution of the Vernier information from frame to frame should follow grouping relations instead of retinotopic relations. Hence, in the case of element motion, the probe Vernier is grouped with element 1 in the second frame and the predictions of the non-retinotopic hypothesis are the same as the retinotopic hypothesis. On the other hand, in group motion, the probe Vernier is grouped with element 2, and thus the accordance of responses with the offset direction of the probe Vernier should be highest for element 2 instead of element 1. The results shown in Figure 7.4a support the non-retinotopic hypothesis. The importance of the motion induced reference becomes immediately evident when the distance between the lines is increased (Figure 7.4b) or the outer lines are removed (Figure 7.4c). In the latter case, no group motion is established and Vernier offsets are reported exclusively according to their retinotopic position (Figure 7.4c). When the spacing between lines is doubled to 1600 arcseconds (”) (Figure 7.4b), performance is virtually identical as with the 800” distance even though the element 2 in the 1600” condition is presented at the same location as element 3 in the 800” condition where performance is at chance level.

Figure 7.4 Left panels show the stimuli used in the experiments and the corresponding right panels show the results for each condition. (Adapted from Öğmen et al., 2006.)

Role of attention in the transfer of information from retinotopic to non-retinotopic representations

If motion grouping establishes the reference frame for transferring information from retinotopic to non-retinotopic representations, the question that follows is whether attention can modulate this transfer. Following the Gestalt view, we propose that if the strength of motion grouping is relatively weak, then attention should be able to modulate it and thereby transform the mapping from retino-topic to non-retinotopic representations. As a first step in testing this hypothesis, we studied whether attention can modulate perceptual grouping in Ternus-Pikler displays. We chose Ternus-Pikler displays because, like Rubin’s vase, it is bi-stable and should be easily amenable to attention manipulation. Another reason is that the choice of the stable state (EM vs. GM) can be easily manipulated through a single parameter, the ISI between the two frames.

Our approach was to use a dual-task paradigm to control attentional resources devoted to the Ternus-Pikler display. We presented a Ternus-Pikler display in the upper visual field at 3 deg of eccentricity and an attention demanding counting task in the center of the screen. There were three experimental conditions. In the attention task (Attention-Only condition), 10 elements, squares or disks, were presented in rapid succession. The number of squares was randomly chosen per trial and observers reported whether this number was odd or even. Three difficulty levels were employed by varying the presentation speed by varying the ISI between elements (Lavie, 1995; Lavie and Tsal, 1994): an ISI of 265 ms for the Easy condition, 190 ms for the Medium condition, and 100 ms for the Hard condition. In the Ternus-Pikler-Only condition, the task of the observer was to report the motion percept: EM vs. GM. In the Dual-Task condition, the Attention-Only and the Ternus-Pikler-Only conditions were combined. The primary task was the attention task and the secondary task was to report GM vs. EM.

Figure 7.5 shows performance for Attention-Only and Dual-Task conditions. Performance decreased in the Dual-Task condition compared to the Attention-Only condition. As expected, performance decreased as the difficulty of attention task increased; however, there was no significant interaction between the experimental condition and the difficulty level, suggesting, that the performance drop in the Dual-Task condition with respect to the Attention-Only condition was similar for all three difficulty levels. This result suggests that the Ternus-Pikler display draws approximately the same amount of attentional resources under the three levels of difficulty of the primary task. Given the similar amount of attentional resources, we expect grouping data to be similar under these three levels of difficulty.

Figure 7.5 Performance in the Attention-Only and Dual-Task condition for the three levels of difficulty. Error bars represent ±1 SEM. (From Aydin et al., 2011 with kind permission from Elsevier Ltd.)

This is in fact what we found (Figure 7.6) confirming the prediction that attention modulates but does not determine motion grouping and that the Ternus-Pikler display draws the same amount of resource from the central attention task at all three levels of difficulty. Figure 7.7 provides a comparison of Points of Subjective Equality (PSE) and standard deviations obtained from the psychometric functions for Ternus-Pikler-Only and Dual-Task conditions. There is a significant decrease in GM responses as attention is withdrawn away from the Ternus-Pikler task. PSE value for Ternus-Pikler-Only condition (Figure 7.7 top panel) is significantly smaller than the PSE for the Dual-Task condition. The data also suggest a difference in standard deviation values (Figure 7.7 lower panel); however, the difference was not statistically significant.

Figure 7.6 Left panel: the percentage of GM responses in the Ternus-Pikler display under full attention (Ternus-Pikler-Only) and divided attention (Easy, Medium, Hard) conditions. Error bars represent ±1 SEM. Right panel: the same data with cumulative Gaussian fits. (From Aydin et al., 2011 with kind permission from Elsevier Ltd.)

Figure 7.7 PSE (top) and SD (bottom) values averaged across the three observers for Ternus-Pikler-Only and Dual-Task conditions. (From Aydin et al., 2011 with kind permission from Elsevier Ltd.)

Summary/conclusion

The enormity of information received by the visual system and the relative scarcity of resources available to process this information led researchers to investigate factors involved in the selection of information. By enhancing relevant stimuli and filtering out the distracting stimuli, attention plays a major role in this process. On the other hand, perceptual grouping allows the organization of stimuli into wholes and therefore can produce significant reduction in the way information is represented and processed. For example, instead of dealing with every pixel, edge segment, etc. in an image, a perceptual group may produce a more global unit, such as a ‘house’. Similarly in space–time, a perceptual group may produce a ‘dance pattern’ instead of unrelated sequences of complex stimuli. Gestalt psychology studied in detail these factors. To understand the neural bases of visual perception, one needs to put these operations into the context of neural representations in the visual system. Retinotopic representations do not provide an adequate basis for processing natural dynamic stimuli. We have discussed a theoretical framework where motion grouping in retinotopic representations can be used to transfer information from retinotopic to non-retinotopic representations. We have also shown that bi-stable motion grouping in Ternus-Pikler can be modulated by attention. We therefore predict that attention can modulate non-retinotopic percepts generated by Ternus-Pikler displays. Ongoing and future experiments will test this prediction.

Acknowledgement

This work was supported in part by award number R01 EY018165 from NIH and by the Swiss National Science Foundation project ‘The Dynamics of Feature Integration’.

References

Alais, D. and Lorenceau, J. (2002) ‘Perceptual grouping in the Ternus display: Evidence for an “association field” in apparent motion’, Vision Research, 42: 1005–1016.

Ansorge, U., Klotz, W. and Neumann, O. (1998) ‘Manual and verbal responses to completely masked (unreportable) stimuli: Exploring some conditions for the metacontrast dissociation’, Perception, 27: 1177–1189.

Aydin, M., Herzog, M. H. and Ogmen, H. (2011) ‘Attention modulates spatio-temporal grouping’, Vision Research, 51: 435–446.

Ben-Av, M. B., Sagi, D. and Braun, J. (1992) ‘Visual attention and perceptual grouping’, Perception & Psychophysics, 52: 277–294.

Breitmeyer, B. G. and Ritter, A. (1986a) ‘The role of visual pattern persistence in bistable stroboscopic motion’, Vision Research, 26: 1801–1806.

Breitmeyer, B. G. and Ritter, A. (1986b) ‘Visual persistence and the effect of eccentric viewing, element size, and frame duration on bistable stroboscopic motion percepts’, Perception & Psychophysics, 39: 275–280.

Breitmeyer, B. G., Öğmen, H. and Chen, J. (2004) ‘Unconscious priming by color and form: Different processes and levels’, Consciousness and Cognition, 13: 138–157.

Breitmeyer, B. G., Öğmen, H., Ramon, J. and Chen, J. (2005) ‘Unconscious priming by form and their parts’, Visual Cognition, 12: 720–736.

Chan, W. Y. and Chua, F. K. (2003) ‘Grouping with and without attention’, Psychonomic Bulletin & Review, 10: 932–938.

Coltheart, M. (1980) ‘Iconic memory and visible persistence’, Perception & Psychophysics, 27: 183–228.

Dawson, M. R. W., Nevin-Meadows, N. and Wright, R. D. (1994) ‘Polarity matching in the Ternus configuration’, Vision Research, 34: 3347–3359.

Gottschaldt, K. (1926) ‘Über den Einfluss der Erfahrung auf die Wahrnehmung von Figuren. I’, Psychologische Forschung, 8: 261–317.

Gottschaldt, K. (1929) ‘Über den Einfluss der Erfahrung auf die Wahrnehmung von Figuren. II’, Psychologische Forschung, 12: 1–87.

Kimchi, R. and Razpurker-Apfeld, I. (2004) ‘Perceptual grouping and attention: Not all groupings are equal’, Psychonomic Bulletin & Review, 11: 687–696.

Klotz, W. and Wolff, P. (1995) ‘The effect of a masked stimulus on the response to the masking stimulus’, Psychological Research, 58: 92–101.

Koffka, K. (1922) ‘Perception: An introduction to the Gestalt-theorie’, Psychological Bulletin, 19: 531–585.

Koffka, K. (1935) Principles of Gestalt Psychology, New York, NY: Harcourt, Brace, & World.

Köhler, W. and Adams, P. A. (1958) ‘Perception and attention’, American Journal of Psychology, 71: 489–503.

Krech, D. and Calvin, A. (1953) ‘Levels of perceptual organization and cognition’, Journal of Abnormal and Social Psychology, 48: 394–400.

Krechevsky, I. (1938) ‘An experimental investigation of the principle of proximity in the visual perception of the rat’, Journal of Experimental Psychology, 22: 497–523.

Lamy, D., Segal, H. and Ruderman, L. (2006) ‘Grouping does not require attention’, Perception & Psychophysics, 68: 17–31.

Lavie, N. (1995) ‘Perceptual load as a necessary condition for selective attention’, Journal of Experimental Psychology: Human Perception and Performance, 26: 1387–1400.

Lavie, N. and Tsal, Y. (1994) ‘Perceptual load as a major determinant of the locus of selection in visual attention’, Perception & Psychophysics, 56: 183–197.

Ma-Wyatt, A., Clifford, C. W. G. and Wenderoth, P. (2005) ‘Contrast configuration influences grouping in apparent motion’, Perception, 34: 669–685.

Mack, A., Tang, B., Tuma, R., Kahn, S. and Rock, I. (1992) ‘Perceptual organization and attention’, Cognitive Psychology, 24: 475–501.

Moore, C. M. and Egeth, H. (1997) ‘Perception without attention: Evidence of grouping under conditions of inattention’, Journal of Experimental Psychology: Human Perception and Performance, 23: 339–352.

Öğmen, H. and Herzog, M. H. (2010) ‘The geometry of visual perception: Retinotopic and non-retinotopic representations in the human visual system’, Proceedings of the IEEE, 98: 479–492.

Öğmen, H., Otto, T. U. and Herzog, M. H. (2006) ‘Perceptual grouping induces non-retinotopic feature attribution in human vision’, Vision Research, 46: 3234–3242.

Pantle, A. and Picciano, L. (1976) ‘A multistable movement display: Evidence for two separate motion systems in human vision’, Science, 193: 500–502.

Pikler, J. (1917) Sinnesphysiologische Untersuchungen, Leipzig, Germany: Barth.

Rock, I., Linnett, C. M., Grant, P. and Mack, A. (1992) ‘Perception without attention: Results of a new method’, Cognitive Psychology, 24: 502–534.

Russell, C. and Driver, J. (2005) ‘New indirect measures of “inattentive” visual grouping in a change-detection task’, Perception & Psychophysics, 67: 606–623.

Shomstein, S., Kimchi, R., Hammer, M. and Behrmann, M. (2010) ‘Perceptual grouping operates independently of attentional selection: Evidence from hemispatial neglect’, Attention, Perception, & Psychophysics, 72: 607–618.

Ternus, J. (1926) ‘Experimentelle Untersuchungen über phänomenale Identität’, Psychologische Forschung, 7: 81–136.

Vorberg, D., Mattler, U., Heinecke, A., Schmidt, T. and Schwarzbach, J. (2003) ‘Different time courses for visual perception and action priming’, PNAS, 100: 6275–6280.

8    Probing human vision with spatial adaptation

Mark W. Greenlee and Svein Magnussen

Introduction

Prolonged inspection of high-contrast visual displays leads to various detrimental changes in visual performance. Two highly studied effects are the impairment of contrast vision in terms of increased thresholds for detecting displays of similar spatial characteristics (Blakemore and Campbell, 1969; Pantle and Sekuler, 1968), and the perceptual distortion of supra-threshold displays of slightly different spatial characteristics (Gibson and Radner, 1937; Hofmann and Bielschowsky, 1909). In a classic monograph-length paper, Gestalt psychologists Wolfgang Köhler and Hans Wallach (1944) reported a series of experiments on perceptual distortions, termed figural aftereffects, and developed a theory in terms of electrical fields in the visual cortex to account for these adaptation effects, a theory that in principle might also account for the, then not yet discovered, contrast threshold elevation aftereffects. The pioneering studies of single cells in the visual cortex of the cat and monkey by Hubel and Wiesel (1959, 1968, 1977) buried the Gestalt theory of brain function (see also Chapter 2 by Wagemans), but spatial adaptation and aftereffects became an important research tool in the study of the functional organization of the visual system, used in countless studies in psychophysics, and human and primate electrophysiology.

The present chapter reviews studies that were conducted in Lothar Spillmann’s laboratory at the University of Freiburg over a 20-year period (1980–1999), with the focus on experiments on contrast adaptation. These studies show how psychophysical adaptation studies reveal the spatial and temporal characteristics of neural channels in the human visual system, before imaging methods were available, and perhaps in some ways still is a superior research strategy compared to brain imaging. However, recently adaptation techniques have been successfully combined with human electrophysiology to gain a more complete description of the physiological processes underlying adaptation. The combined technique of human psychophysics and functional magnetic resonance imaging (fMRI) has recently been applied to high-level vision such as that required for the identification of the gender of the person depicted. As such the method of adaptation opens new windows into brain function at different levels of visual processing.

The basic phenomenon

Figure 8.1 illustrates the basic idea behind the method of contrast adaptation (Greenlee, 2002). View the central high-contrast vertical grating for a minute by allowing your eyes to wonder along the central fixation line presented in the center of the adaptation grating. Afterwards glance up to the low contrast grating in the upper left quadrant. What you should notice is that the grating is momentarily invisible. Only after a few seconds will the grating slowly reappear. Prolonged viewing of the central high-contrast adapting grating temporarily suppresses sensitivity to low contrast gratings with similar spatial characteristics. The specificity of this effect for the orientation of the grating can be demonstrated by readapting to the central grating and then looking at the grating in the upper right quadrant. This grating shares the same spatial frequency of the adapting grating, but it now has an orthogonal orientation. Despite prolonged adaptation to the central grating, the low contrast grating with horizontal orientation remains visible. In a similar fashion the spatial frequency selectivity of contrast adaptation can be demonstrated by viewing the lower left or right grating patches. These gratings differ by one octave from that of the central grating. After prolonged adaptation to the central grating, these low contrast gratings remain visible.

Figure 8.1 Demonstration of adaptation to spatial contrast. The central stimulus is a high-contrast sinewave grating with a spatial frequency of 2 c/cm and a vertical orientation. The central horizontal grey bar should be viewed during adaptation. Let your gaze move slowly along the fixation bar. After about 60 seconds, you should now shift your gaze to the upper left quadrant. You should notice that the low contrast grating is momentarily invisible. After a few moments the grating will reappear. Repeated exposure to the high-contrast adapting grating will make the test grating in the upper left quadrant disappear again. The orientation specificity of the adaptation effect can be demonstrated by viewing the upper right quadrant after adaptation to the central pattern. Here you should note that the horizontal pattern remains visible despite prolonged adaptation to the vertical pattern in the center of the display. The spatial frequency specificity of the adaptation effect can be demonstrated by viewing the low spatial frequency pattern in the lower left quadrant or the high spatial frequency pattern in the lower right quadrant. Again, owing to the selective nature of spatial adaptation, these test patterns remain visible after adaptation to the centrally presented adapting pattern. (Reprinted from Greenlee (2002) with kind permission from Hogrefe Verlag.)

Mapping spatial channels

The selectivity of spatial adaptation to the spatial frequency and orientation of the adapting stimuli illustrated by the above exercise suggests that only a subset of neurons responds and adapt to the adapting stimulus, and that the spatial tuning of the aftereffect reflects the tuning of the underlying neural channels. It follows that contrast adaptation can be used to probe the functional organization of spatial channels in the human visual system. This is the general idea behind the classic studies of the Cambridge group in the late 1960s and early 1970s, where the spatial selectivity of orientation and spatial frequency channels were mapped with spatial adaptation techniques, measuring both contrast thresholds (Blakemore and Campbell, 1969) and supra-threshold figural aftereffects, such as size adaptation and the tilt aftereffect (Blakemore et al., 1970; Campbell and Maffei, 1971).

The seminal paper by Blakemore and Campbell (1969) established a number of the basic properties of channels underlying spatial adaptation:

First, they found that the bandwidth of spatial frequency adaptation was about ±1 octave at half amplitude, i.e when the test spatial frequency was half or double the adaptation frequency, the efficiency of the adapting stimulus fell to half its value.

Second, they found that that the range of adaptable channels in normal vision was between approximately 1.0 and 30 cycles per degree (c/deg), and that the bandwidth of adaptation was fairly stable across the range of adapting frequencies.

Tolhurst (1973) confirmed that the lowest adaptable channel in human vision peaked at 1 c/deg, as did we (Greenlee et al., 1988). In the orientation domain, the bandwidth of spatial adaptation was about ±6.5 deg at half amplitude (Blakemore and Nachmias, 1971). Similar estimates have been arrived at via a number of different psychophysical techniques (Braddick et al., 1978), and would appear to represent stable characteristics of spatial channels of the normal human visual system.

What happens to the organization of spatial channels with anomalous spatial input to the visual cortex?

We investigated this question in a series of experiments on the Norwegian researcher and famous rod monochromat Knut Nordby (KN, 1942–2005). The rod monochromat is, owing to a genetic deficiency, born with nonfunctional cone photoreceptors and with a central scotoma, is completely color blind, and has a visual acuity of approximately 0.1, corresponding to a grating acuity of 6 c/deg. Thus the spatial vision of the rod monochromat is comparable to that of the domestic cat or a six-month old human infant (Greenlee et al., 1988). The visual world as viewed by a rod-only retina depends upon a very coarse spatial input to the visual cortex.

What is the consequence of the absence of middle and high spatial frequency input to the rod-monochromat visual cortex?

Our hypothesis, derived from studies of the development of spatial selectivity of single units in the cat’s visual cortex was that the range of spatial frequency channels of the rod monochromat would be shifted downwards compared to the normal visual system. The single unit studies showed that the spatial resolution of the cortical cells is very crude at birth but that the preferred spatial frequency increased with age (Braastad and Heggelund, 1985; Derrington and Fuchs, 1981). If a similar developmental shift in the spatial resolution of cells occurred in the human visual cortex, the development of spatial resolution in the rod monochromat would be arrested at an early stage owing to the coarse spatial input to the visual cortex set by rod vision. A range shift implies a downward shift in both, the highest and the lowest adaptable channel; thus, we would expect the rod monochromat to have adaptable channels in the low-frequency range below the lowest adaptable channel in normal vision.

Using specially designed presentation techniques, we were able induce adaptation effects to adapting gratings with a spatial frequency as low as 0.09 c/deg (period length of 11.1 degrees per cycle). The results, shown in Figure 8.2, confirmed our hypothesis. Contrary to the findings in normal trichromats, the rod monochromat displayed spatial-frequency tuned adaptation effects that had a peak spatial frequency as low as 0.09 c/deg. The trichromat control subject exhibited tuned aftereffects of adaptation for gratings with a spatial frequency of 1 c/deg. When lower spatial frequencies were used, the threshold elevations measured in the trichromat control were low and skewed towards 1 c/deg, suggesting that no cortical ‘channels’ exist that can selectively respond and adapt to stationary spatial frequencies below 1 c/deg, confirming the results of Tolhurst (1973). These results indicate that the visual cortex of the rod monochromat has adjusted to the low resolution retinal input. Lacking the high spatial frequency information normally provided by the cone pathways, the rod monochromat appears to have developed spatial frequency channels that are selective to very low spatial frequencies. The results are consistent with the idea (Greenlee et al., 1988) that the development of spatial frequency channels in the rod monochromat visual brain is arrested around six months of age. Note, however, that the downward range shift in adaptable spatial frequency channels in the rod monochromat does not affect the breadth of the range of adaptable channels: both trichromats and rod monochromats exhibit a five-octave range of adaptable channels.

Figure 8.2 a) Normalized contrast threshold elevation as a function of test spatial frequency for the rod monochromat KN. Unlike the trichromat, KN shows evidence for low spatial frequency channels that are tuned to the adaptation frequency as low as 0.09 c/deg. The dashed line signifies zero contrast threshold elevation. b) Results for the trichromat control subject MWG, otherwise as in Figure 8.2a. The control subject shows no spatial frequency tuned threshold elevations below 1 c/deg. (Reprinted from Greenlee et al. (1988) with kind permission from Journal of Physiology, Wiley.)

Except for the range shift, the spatial channels of the rod monochromat were normal. The bandwidths of the spatial channels were comparable to the trichromat, the orientation selectivity of the aftereffect was comparable, as was the magnitude of the interocular transfer of the effect. Furthermore, the spatial adaptation effect exhibited normal decay function (see below). This suggests that the coarse input to the rod monochromat visual brain affects the spatial frequency resolution but not the fidelity of orientation perception. This conclusion is further supported by the fact that the orientation tuning of the suprathreshold tilt aftereffect is normal in the rod monochromat (Rosness et al., 1994).

These results point to a form of neural plasticity that partly compensates for the lack of high-resolution cone input into the visual process.

Is the visual cortex a spatial frequency analyzer?

Any visual spatial pattern may be decomposed in terms of sinusoidal components varying in spatial frequency, orientation and phase (Fourier analysis), and constructed by adding the same components. The Fourier theory of visual spatial processing assumed that spatial perception was the result of combined activity of neural channels responding to the various frequency components of the patterns (Campbell and Robson, 1968; Jaschinski-Kruza and Cavonius, 1984). Models of spatial vision usually assumed that a fairly small number of channels determined our sensitivity to different patterns (Watson and Robson, 1981; Wilson and Bergen, 1979; see also Chapter 3 by Thomas in this volume). The models were successful in predicting sensitivity to patterns containing different spatial frequency and orientation components. However, strong evidence against the Fourier theory of visual spatial processing was provided by us, showing that adaptation to sinewave gratings with spatial frequencies that corresponded to the higher harmonics of a low spatial frequency square-wave grating did not have the expected effect on contrast detection thresholds for square-wave gratings (Greenlee and Magnussen, 1987a).

A square-wave grating may be described as being composed of its fundamental frequency f adding odd frequencies of 3f, 5f, 7f 9f and so on. And it was assumed that due to independent mechanisms sensitive to the higher harmonic frequencies the contrast sensitivity to square-wave gratings does not show a fall-off at low spatial frequencies, as does the sensitivity to sinewave gratings (Campbell and Robson, 1968). It follows from the theory that if the sensitivity to the higher harmonics were reduced by adaptation, a square-wave grating should be effectively reduced to a sinewave grating. However, despite significant effects on contrast detection thresholds for sinewave gratings with a spatial frequency of 1 and 3 c/deg (corresponding to the third and ninth harmonics of a 0.33 c/deg square-wave grating) alternating adaptation to these harmonic frequencies had no effect on the ability of the observers to detect the presence of the square-wave grating (Figure 8.3). Thus, the detection of the sharp edges of the square-wave grating is not the result of activity in independent neural channels representing the higher-harmonic components, but rather some local edge-detection mechanism is involved. Such an edge-detection mechanism would not be adapted by sinewave adaptation.

Figure 8.3 Opposite: Higher harmonic adaptation and the contrast threshold elevations for different test patterns. a) Log threshold elevation is plotted as a function of test spatial frequency following adaptation to a 1 c/deg (open grey circles) and a 3 c/deg adapting gratings (filled grey circles). Sequential adaptation to these two spatial frequencies leads to the composite adaptation effects shown by open black circles. b) Log threshold elevation is shown for the detection of low spatial frequency (f = 0.33 c/deg) sinewave and square-wave gratings following sequential adaptation. Threshold elevations following sequential adaptation to sinewave gratings with spatial frequencies 1 and 3 c/deg are also presented. Despite the large and highly significant effect of adaptation at the third and ninth harmonic frequencies of a low spatial frequency square-wave grating, harmonic adaptation has no effect on the detection of a low contrast square-wave grating. (Reprinted after Greenlee and Magnussen (1987a) with kind permission from Vision Research by Elsevier.)

In a similar fashion, we showed in a subsequent study that higher-harmonic adaptation did not have the expected effect on spatial waveform discrimination (Greenlee and Magnussen, 1988). The contrast required by observers to discriminate between a sinewave and square-wave grating did not increase after adaptation to the harmonic components of the square-wave. Taken together, the results from these two studies placed the independent-channel Fourier theory of pattern detection and waveform discrimination in question. Changes in the sensitivity to harmonic components were evident for simple sinewave test gratings, but these adaptation effects did not alter detection and discrimination thresholds for complex (square-wave) gratings. These differences occurred despite randomized presentation of the waveform patterns and forced-choice techniques. We suggested at the time that these results serve a blow to theories that rely on linear combinations of activities in independent spatial-frequency channels for detection and discrimination of periodic patterns. We proposed that the local features of the patterns that result from combinations of sinewave components with different spatial phases and contrast (i.e. edge detection) needed to be taken into account to be able to predict sensitivity to square-wave patterns after harmonic adaptation.

The mechanism of spatial adaptation

What kind of physiological process underlies these short-lived plasticity effects? One prominent theory of spatial adaptation suggested that neural fatigue underlies adaptation (Barlow, 1990). Accordingly prolonged stimulation of a subset of neurons leads to neural fatigue of the activated neurons (Barlow and Hill, 1963), where the reduced firing rate in the cortical neurons post-adaptation is a consequence of depletion of neurotransmitter or ion concentrations in the postsynaptic neuron leading to an alteration in excitatory and inhibitory inputs (Ursino et al., 2008). The recovery from adaptation reflects the time required to restore baseline levels within the neural network affected by adaptation. Surprisingly at that time, Freiburg experiments on simultaneous adaptation to two orientations showed that dual orientation adaptation led to smaller tilt aftereffects (Kurtenbach and Magnussen, 1981; Magnussen and Kurtenbach, 1980) compared to adapting to one of the orientations alone, suggesting that the introduction of a second adapting orientation acts to distribute the activation over the neural network more evenly. This result was the opposite of what predicted by the fatigue theory of adaptation, but is in line with a competing theory, in terms of prolonged inhibition between channels (Burr and Morrone, 1987; Tolhurst and Thompson, 1975).

Some years later, we provided more evidence for the inhibition theory of contrast adaptation. Spatial adaptation can be interrupted for a brief period of time without loss of the magnitude of the resultant aftereffect (Magnussen and Greenlee, 1986). Since brief interruption of adaptation has little effect on the post-adaptation threshold, we could temporally interleave adaptation to two spatial frequencies or two orientations to test whether the adaptation effects interact to determine sensitivity to a 4 c/deg low contrast test grating. In this way, we could desensitize a wide range of neurons tuned to spatial frequencies or orientations without displaying more than one grating at a time. The results of these psychophysical measurements are shown for three subjects in Figure 8.4. Normalized contrast threshold elevation is plotted as a function of the spatial frequency difference between that of the test grating and the spatial frequency of the second adapting grating. When both adapting gratings have the same spatial frequency (4 c/deg) we have the most pronounced adaptation effects. Reducing the spatial frequency of the second adapter (−1.5 octaves) or increasing its spatial frequency (+2 octaves) led to a pronounced reduction in the adaptation effect at the test spatial frequency. The resultant adaptation effect was less than that found when the first adapting stimulus was interleaved with a blank field (Figure 8.4a, horizontal dashed line). This significant reduction below the baseline level strongly suggests the presence of inhibitory interactions between populations of neurons sensitive to the side-band spatial frequencies. Similar effects were found when the orientation of the second adapting grating was varied from vertical (i.e. same as the orientation of the first adapter and the test grating) and horizontal (Figure 8.4b).

Figure 8.4 Opposite: Contrast threshold elevation following sequential adaptation to two gratings. a) Normalized contrast threshold elevation is plotted as a function of the difference spatial frequency between the first (4 c/deg) adapting grating and the second (variable spatial frequency) adapting grating for a 4 c/deg test grating. Maximum threshold elevation was observed when both gratings shared the same spatial frequency (octave difference = 0). Minimal threshold elevations were noted when the two adapting gratings differed by –1.5 or 2 octaves. Dashed horizontal line signifies the threshold elevation resulting when the 4 c/deg adapting grating was interleaved with a blank screen. b) As in panel a), except now log threshold elevation is shown as a function of difference in the orientation between the first and second adaptation grating. Again, the horizontal dashed line signifies the threshold elevation resulting after adaptation to a vertical 4 c/deg grating interleaved with a blank screen. The curve shows the best-fitting difference of Gaussians (DoG) function. (Reprinted after Greenlee and Magnussen (1988) with kind permission from Vision Research by Elsevier.)

The observation that more adaptation led to less threshold elevation contradicts the idea that adaptation is a simple result of neural fatigue. The results point much more to the inhibitory nature of contrast adaptation that actively sets the contrast gain of neurons depending on the context or history of prior stimulation (Greenlee and Heitger, 1988; Wilson and Humanski, 1993). More recent results using fMRI in human visual cortex support the role of adaptation as a form of contrast gain control in the processing of stimulus contrast (Gardner et al., 2005). Spatial adaptation thus reflects the dynamic nature of contrast gain control in the visual system.

The time course of adaptation

Reflecting on short-lived plasticity of the human visual system, one could ask what is the time course of these aftereffects? Most studies of contrast adaptation used fairly short adaptation periods, in the order of 30–60 s. Blakemore and Campbell (1969) provided some evidence that longer adaptation periods produced larger and more persistent aftereffects, but the temporal limit of adaptation was not known. We therefore conducted a ‘marathon’ adaptation experiment, in which the participant adapted to a high-contrast grating for up to 180 minutes to a large adapting grating of high contrast (Magnussen and Greenlee, 1985). During this prolonged adaptation, the detection threshold for a smaller grating with the same spatial frequency and orientation, but having a low contrast, was estimated using the method of adjustment. The results indicate that the adaptation effects build up over at least a 60-minute period. After such long-lasting adaptation, the time required to regain baseline levels of contrast sensitivity also increased, apparently mirroring the build-up of the aftereffect. In a similar fashion, the build up and decay of tilt aftereffect (TAE) appears to follow a power law relating adaptation time to recovery time (Drew and Abbott, 2006; Greenlee and Magnussen, 1987b).

The results of the Magnussen and Greenlee (1985) study could be criticized since the method of adjustment is not as objective as ‘forced-choice’ techniques. With this in mind, we designed a new experiment that took advantage of the adaptive threshold estimation algorithm QUEST (Watson and Pelli, 1983) after 1, 10, 100 and 1000 s of adaptation. Typical results are shown in Figure 8.5, where contrast threshold (dB) is plotted as a function of recovery time on a logarithmic axis. The best fitting recovery functions are straight lines in this log–log plot, where prolonged adaptation leads to an upward shift in the amplitude of the threshold elevation. These parallel recovery functions collapse onto a single function when the contrast threshold (dB) is plotted as the log ratio recovery time/adapt time, suggesting that a power law can describe the relationship between adaptation duration and recovery. By varying the contrast of the adapting grating but keeping the adapting duration constant, we demonstrated that the time required to recover from adaptation was primarily determined by the adaptation duration and not by the initial threshold elevation. The results indicate that the curves for different adaptation conditions converge suggesting that recovery time is equal to or slightly greater than adaptation time.

Figure 8.5 Recovery from adaptation to spatial contrast. Contrast threshold (dB) are plotted as a function of log recovery time (sec) for two participants (panel a: MAG and panel b: MWG). The results are shown following 1 s (filled circles), 10 s (open circles), 100 s (filled squares) and 1000 s (open squares) adaptation. The regression lines are best fitting power functions constrained to have the same slope. (Reprinted from Greenlee et al. (1991) with kind permission from Vision Research by Elsevier.)

A meta-analysis of studies conducted between 1976 and 1990 suggested that the published results strongly point to the power law of adaptation to spatial contrast (Figure 8.6). Over almost four orders of magnitude, log recovery time is a linear function of log adaptation time. The slope is not significantly different from unity, suggesting a proportionality rule. Together these results suggest that contrast adaptation is like a short-term memory store that reflects the history of stimulation experienced by the underlying neurons.

Figure 8.6 Meta-analysis of studies on the recovery from adaptation to spatial contrast. The regression plot presents the results of six studies that examined the effects of spatial adaptation for different adaptation durations. Log recovery time is plotted as a function of log adaptation time. The slope of this function is not significantly different from unity with a positive offset, suggesting a power law of spatial adaptation. (Reprinted from Greenlee et al. (1992) with kind permission from Vision Research by Elsevier.)

Adaptation in the motion domain

As described above, prolonged viewing of spatial contrast alters our ability to detect similar patterns with low luminance contrast. Prolonged viewing of motion stimuli leads to the famous motion aftereffect (MAE; Anstis et al., 1998; Mather et al., 2008). Following viewing of a unidirectional moving pattern, a stationary pattern will appear to drift in the opposite direction (Barlow and Hill, 1963). The MAE is thought to reflect the antagonistic encoding of motion direction: prolonged stimulation in one direction will lead to an imbalance in the outputs of neurons encoding the adaptation direction and the opposite direction. As a consequence, a stationary pattern will appear to drift in a direction opposite that of the adapting stimulus. Even very brief adaptation can induce motion aftereffects (Kanai and Verstraten, 2005; Pavan et al., 2010). The perceived speed of a moving stimulus depends on its luminance contrast (Stone and Thompson, 1982). Adaptation to a moving stimulus can also affect the perceived speed of a subsequently viewed stimulus (Thompson, 1981). In a series of experiments conducted in the Spillmann laboratory, Müller and Greenlee (1994) showed that prolonged adaptation to a rightward drifting grating has a significant effect on the lower threshold for motion (LTM). The relationship between the LTM and stimulus contrast is nearly flat for the unadapted condition, but are significantly shifted upward following slow (1.25 deg/s) and fast (5 deg/s) adaptation. Following adaptation, a low-contrast test grating has to be moving at 0.6 to 1.3 deg/s in order to be detected as moving, suggesting that the threshold for motion detection is significantly elevated.

In a similar fashion, the Weber fraction for velocity discrimination (dV/V) varies as a function of grating contrast. These functions are shifted upwards following adaptation and the magnitude of this adaptation-induced shift is dependent on the speed of the adapting stimulus. Finally, the perceived speed of a moving grating is dependent on the contrast of the grating and on prior adaptation. Adaptation to a rightward drifting grating shifts the perceived velocity vs. contrast functions towards higher contrast values (Figure 8.7). Adaptation to a drifting grating shifts the contrast-response functions by a factor of five to seven in the two participants shown in Figure 8.7.

Figure 8.7 a) Perceived speed (V’) is shown as a function of stimulus contrast before (no adapt) and after adaptation to a drifting grating of variable speeds (temporal frequency: 2–32 Hz, see inset). The curves are best fitting hyperbolic ratios (see text). The results are shown for subject MWG. b) Results for subject RM. (Reprinted from Müller and Greenlee (1994) with kind permission from Vision Research by Elsevier.)

Higher order adaptation effects in human vision

The vast majority of spatial adaptation studies have investigated adaptation to basic features of the visual image, such as spatial frequency, orientation, and motion. However, recent developments suggest that the technique may also be applied to investigate higher-order mechanisms of visual processing, such as mechanisms involved in the perception of objects, human body parts and faces (Cziraki et al., 2010; Jiang et al., 2006; Webster and MacLin, 1999). The temporal dynamics of this form of adaptation is similar to that for low-level stimulus features (Leopold et al., 2005) and the orientation selectivity of the tilt aftereffect is comparable to the gender specificity of face adaptation (Zhao et al., 2011), suggesting similar mechanisms.

Following up in the low-level feature experiments, Kovács and colleagues (2008) showed that adaptation to female faces shifts the perception of gender-mixed facial morphs towards a male percept. This gender-specific form of adaptation is more pronounced following prolonged exposure to a face and the aftereffect is invariant for position of adapter and test stimuli. Brief exposure to face stimuli leads to smaller aftereffects but these are independent of the location of the adapting and test stimuli. Face adaptation also alters the fMRI blood oxygen level dependent (BOLD) response in the fusiform face area (FFA) in the right and left ventral visual cortex. The results of these experiments are summarized for the FFA region in ventral visual cortex in Figure 8.8 (panels c and d). Here face adaptation varied from brief (500 ms) to long (4500 ms) and occurred in the left or right visual field (Figure 8.8a). The evoked response in right and left FFA are shown in panels b and c of Figure 8.8. The findings indicate that face adaptation reduces the BOLD response to subsequently viewed test faces. Adaptation to female faces shifts the perception of a mixed gender morph toward male, as indicated by the psychometric functions for gender identity.

Figure 8.8 Spatial invariance of face adaptation. a) Schematic illustration of a typical trial with face or noise adapter and test stimulus. The adapting stimuli were presented for 500 or 4500 ms, followed by a blank period of 500 ms and a brief presentation of the test stimulus. b) Psychophysical results shown as psychometric functions, where the portion of trials endorsed as male is plotted against the percent male content of the morphed face. c) Results for the right FFA for the control condition, for short and long adaptation in the same or opposite hemifield. d) As in panel c) except now for the left FFA. The right FFA exhibits a reduced response following short and long adaptation that is invariant across locations. Similar findings are evident for the left FFA for stimuli in the contralateral visual hemifield. The adaptation effects in the left FFA for ipsilateral stimulation are not significant (asterisks). (Reprinted from Kovács et al. (2008) with kind permission from Neuro Image by Elsevier.)

The finding that adaptation to such high-level visual displays appears to behave in a very similar way to the low-order adaptation to simple visual features studied in the 1970s and 1980s proves that the technique of selective adaptation is not outdated but remains a powerful tool in vision research with strong promises for future successes.

Summary/conclusion

Adaptation to spatial displays has robust effects on the perception of visual stimuli presented after adaptation. Referred to as the ‘psychophysicist’s microelectrode’ (Frisby, 1979), spatial adaptation and aftereffects has proved to be a powerful technique in the investigation of the functional organization of the human visual system. Our 20 years of collaboration on spatial adaptation research in the hospitable atmosphere of Lothar Spillmann’s laboratory at the University of Freiburg has resulted in a number of experiments that we believe have contributed to the understanding of the mechanism of these short-lived plasticity effects, and to the understanding of the development of spatial channels in the human visual system. It was quickly realized that visual aftereffects could be explained by mechanisms at the cellular level (Barlow, 1972; Braddick et al., 1978), and our psychophysical experiments pointed to the, now accepted, explanation in terms of lateral inhibition between stimulus-specific neural channels (Ursino et al., 2008). It now appears that a similar mechanism governs higher-level adaptation to complex stimuli such as human faces.

The Gestalt psychologists described many of the figural aftereffects more than half century ago. It is appropriate that our research in this area was carried out in a German laboratory that so strongly continued the Gestalt tradition in perception research. Perception remains a mystery, but we now know a little more about the neural circuitry responsible for visual aftereffects, and we know a little more about ‘the way we see the world the way we do’ (Spillmann, 2009).

Acknowledgements

The authors thank the Alexander von Humboldt Foundation, the German Research Foundation (DFG), the German Ministry for Education and Research (BMBF), and the Norwegian Research Council for their continuing support.

References

Anstis, S., Verstraten, F. A. J. and Mather, G. (1998) ‘The motion aftereffect’, Trends in Cognitive Sciences, 2: 111–117.

Barlow, H. B. (1972) ‘Single units and sensation: a neuron doctrine for perceptual psychology?’ Perception, 1: 371–394.

Barlow, H. B. (1990) ‘A theory about the functional role and synaptic mechanism of visual after-effects’, in C. Blakemore (ed.) Vision: Coding and efficiency. Oxford, UK: Oxford University Press, pp. 363–375.

Barlow, H. B. and Hill, R M. (1963) ‘Evidence for a physiological explanation of the waterfall phenomenon and figural after-effects’, Nature, 200: 1345–1347.

Blakemore, C. and Campbell, F. W. (1969) ‘On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images’, Journal of Physiology, 203: 237–260.

Blakemore, C. and Nachmias, J. (1971) ‘The orientation specificity of two visual aftereffects’, Journal of Physiology, 213: 157–174.

Blakemore, C. Nachmias, J. and Sutton, P. (1970) ‘The perceived spatial frequency shift: evidence for frequency-selective neurones in the human brain’, Journal of Physiology, 210: 727–750.

Braastad, B. O. and Heggelund, P. (1985) ‘Development of spatial receptive-field organization and orientation selectivity in kitten striate cortex’, Journal of Neurophysiology, 53: 1158–1178.

Braddick, O., Campbell, F. W. and Atkinson, J. (1978) ‘Channels in vision: basic aspects’, in R Held, H. W. Leibowitz and H.-L. Teuber (eds), Handbook of Sensory Physiology, Vol. VIII, Perception, Berlin, Germany: Springer-Verlag, pp. 3–38.

Burr, D. C. and Morrone, M. C. (1987) ‘Inhibitory interactions in the human vision system revealed in pattern-evoked potentials’, The Journal of Physiology, 389: 1–21.

Campbell, F. W. and Robson, J. G. (1968) ‘Application of Fourier analysis to the visibility of gratings’, Journal of Physiology, 197: 551–566.

Campbell, F. W. and Maffei, L. (1971) ‘The tilt aftereffect: a fresh look’, Vision Research, 11: 833–840.

Cziraki, C., Greenlee, M. W. and Kovács, G. (2010) ‘Neural correlates of high-level adaptation-related aftereffects’, Journal of Neurophysiology, 103: 1410–1417.

Derrington, A. M. and Fuchs, A. F. (1981) ‘The development of spatial-frequency selectivity in kitten striate cortex’, Journal of Physiology, 316: 1–10.

Drew, P. J. and Abbott, L. F. (2006) ‘Models and properties of power-law adaptation in neural systems’, Journal of Neurophysiology, 96: 826–833.

Frisby, J. P. (1979) ‘Seeing: Illusion, brain and mind, Oxford, UK: Oxford University Press.

Gardner, J. L., Sun, P., Waggoner, R A., Ueno, K., Tanaka, K. and Cheng, K. (2005) ‘Contrast adaptation and representation in human early visual cortex’, Neuron, 47: 607–620.

Gibson, J. J. and Radner, M. (1937) ‘Adaptation, aftereffect and contrast in the perception of tilted lines: I. Quantitative studies’, Journal of Experimental Psychology, 20: 453–467.

Greenlee, M. W. (2002) ‘Sehen, visuelle Aufmerksamkeit’, in T. Elbert and N. Birbaumer (eds) Enzyklopädie der Psychologie. Themenbereich C, Theorie und Forschung. Serie I, Biologische Psychologie. Band 6 Biologische Grundlagen der Psychologie, Göttingen, Germany: Hogrefe Verlag, pp. 125–247.

Greenlee, M. W. and Magnussen, S. (1987a) ‘Higher-harmonic adaptation and the detection of squarewave gratings’, Vision Research, 27: 249–255.

Greenlee, M. W. and Magnussen, S. (1987b) ‘Saturation of the tilt aftereffect’, Vision Research, 27: 1041–1043.

Greenlee, M. W. and Heitger, F. (1988) ‘The functional role of contrast adaptation’, Vision Research, 28: 791–797.

Greenlee, M. W. and Magnussen, S. (1988) ‘Interactions among spatial frequency and orientation channels adapted concurrently’, Vision Research, 28: 1303–1310.

Greenlee, M. W., Magnussen, S. and Nordby, K. N. (1988) ‘Spatial vision of the achromat: spatial-frequency and orientation specific adaptation’, Journal of Physiology, 395: 661–678.

Greenlee, M. W., Georgeson, M. A., Magnussen, S. and Harris, J. P. (1991) ‘The time course of adaptation to spatial contrast’, Vision Research, 31: 223–236.

Greenlee, M. W., Georgeson, M. A., Magnussen, S. and Harris, J. P. (1992) ‘The decay of adaptation to spatial contrast: a response to David Rose’, Vision Research, 32: 1785–1788.

Hofmann, F. B. and Bielschowsky, A. (1909) ‘Über die Einstellung der scheinbaren Horizontalen und Vertikalen bei Betrachtung eines von schrägen Konturen erfallten Gesichtsfeldes’, Pflügers Archiv, 126: 453–475.

Hubel, D. H. and Wiesel, T. N. (1959) ‘Receptive fields of single neurones in the cat’s striate cortex’, The Journal of Physiology, 148: 574–591.

Hubel, D. H. and Wiesel, T. N. (1968) ‘Receptive fields and functional architecture of monkey striate cortex’, Journal of Physiology, 195: 215–243.

Hubel, D. H. and Wiesel, T. N. (1977) ‘Ferrier lecture. Functional architecture of macaque monkey visual cortex’, Proceedings of the Royal Society of London Series B, 198: 1–59.

Jaschinski-Kruza, W. and Cavonius, C. R (1984) ‘A multiple-channel model for grating detection’, Vision Research, 24: 933–941.

Jiang, F., Blanz, V. and O’Toole, A. J. (2006) ‘Probing the visual representation of faces with adaptation: a view from the other side of the mean’, Psychological Science, 17: 493–500.

Kanai, R and Verstraten, F. A. J. (2005) ‘Perceptual manifestations of fast neural plasticity: motion priming, rapid motion aftereffect and perceptual sensitization’, Vision Research, 45: 3109–3116.

Köhler, W. and Wallach, H. (1944) ‘Figural aftereffects. An investigation of visual processes’, Proceedings of the American Philosphical Society, 88: 269–357.

Kovacs, G., Cziraki, C., Vidnyanszky, Z., Schweinberger, S. R and Greenlee, M. W. (2008) ‘Position-specific and position-invariant face aftereffects reflect the adaptation of different cortical areas’, Neuro Image, 43: 156–164.

Kurtenbach, W. and Magnussen, S. (1981) ‘Inhibition, disinhibition and summation among orientation detectors in human vision’, Experimental Brain Research, 43: 183–198.

Leopold, D. A., Rhodes, G., Müller K.-M. and Jeffery, L. (2005) ‘The dynamics of visual adaptation to faces’, Proceedings Biological Sciences/The Royal Society, 272: 897–904.

Magnussen, S. and Kurtenbach, W. (1980) ‘Adapting to two orientations: disinhibition in a visual aftereffect’, Science, 207: 908–909.

Magnussen, S. and Greenlee, M. W. (1985) ‘Marathon adaptation to spatial contrast: saturation in sight’, Vision Research, 25: 1409–1411.

Magnussen, S. and Greenlee, M. W. (1986) ‘Contrast threshold elevation following continuous and interrupted adaptation’, Vision Research, 26: 673–675.

Mather, G., Pavan, A., Campana, G. and Casco, C. (2008) ‘The motion aftereffect reloaded’, Trends in Cognitive Sciences, 12: 481–487.

Muller, R and Greenlee, M. W. (1994) ‘Effect of contrast and adaptation on the perception of the direction and speed of drifting gratings’, Vision Research, 34: 2071–2092.

Pantle, A. and Sekuler, R (1968) ‘Size-detecting mechanisms in human vision’, Science, 162: 1146–1148.

Pavan, A., Campana, G., Maniglia, M. and Casco, C. (2010) ‘The role of high-level visual areas in short- and longer-lasting forms of neural plasticity’, Neuropsychologia, 48: 3069–3079.

Rosness, R., Magnussen, S. and Nordby, K. (1994) ‘Spatial vision of the achromat: the tilt after-effect’, Vision Research, 34: 2021–2022.

Spillmann, L. (2009) ‘Phenomenology and neurophysiological correlations: two approaches to perception research’, Vision Research, 49: 1507–1521.

Stone, L. S. and Thompson, P. (1992) ‘Human speed perception is contrast dependent’, Vision Research, 32: 1535–1549.

Thompson, P. (1981) ‘Velocity after-effects: the effects of adaptation to moving stimuli on the perception of subsequently seen moving stimuli’, Vision Research, 21: 337–345.

Tolhurst, D. J. (1973) ‘Separate channels for the analysis of the shape and the movement of moving visual stimulus’, Journal of Physiology, 231: 385–402.

Tolhurst, D. J. and Thompson, P. (1975) ‘Orientation illusions and aftereffects: inhibition between channels’, Vision Research, 15: 967–972.

Ursino, M., Magosso, E. and Cuppini, C. (2008) ‘Possible mechanisms underlying tilt aftereffect in the primary visual cortex: a critical analysis with the aid of simple computational models’, Vision Research, 48: 1456–1470.

Watson, A. B. and Robson, J. G. (1981) ‘Discrimination at threshold: labelled detectors in human vision’, Vision Research, 21: 1115–1122.

Watson, A. B. and Pelli, D. G. (1983) ‘QUEST: a Bayesian adaptive psychometric method’, Perception and Psychophysics, 33: 113–120.

Webster, M. A. and MacLin, O. H. (1999) ‘Figural aftereffects in the perception of faces’, Psychonomic Bulletin & Review, 6: 647–653.

Wilson, H. R and Bergen, J. R (1979) ‘A four mechanism model for threshold spatial vision’, Vision Research, 19: 19–32.

Wilson, H. R and Humanski, R (1993) ‘Spatial frequency adaptation and contrast gain control’, Vision Research, 33: 1133–1149.

Zhao, C., Seriès, P., Hancock, P. J. B. and Bednar, J. A. (2011) ‘Similar neural adaptation mechanisms underlying face gender and tilt aftereffects’, Vision Research, 51: 2021–2030.

9    From Hermann’s grid to Spillmann’s weaves

Kai Hamburger, Erica L Dixon and Arthur G. Shapiro

Introduction

In the widely known visual illusion called the Hermann grid, perceptual ‘smudges’ appear on the intersections of a white grid superimposed on a black background or on the intersections of a black grid superimposed on a white background (Brewster, 1844; Hering, 1920; Hermann, 1870). The Hermann grid is a staple for introductory psychology and perception textbooks and is one of the most frequently investigated visual illusions (e.g. Baumgartner, 1960; Geier et al., 2004, 2008; Hamburger et al., 2012; Jung and Spillmann, 1970; Levine et al., 1980; Oehler and Spillmann, 1981; Spillmann, 1971; Spillmann and Levine, 1971; for an overview see Schiller and Carvey, 2005 or Spillmann, 1994). Spillmann and Levine (1971) produced an ingenious display for investigating the contrasts of the grid’s vertical and horizontal bars with respect to the background.

Figure 9.1 shows a copy of the Spillmann and Levine display (the original Spillmann and Levine’s display was constructed with strips of gray-scale paper). The display consists of a series of horizontal bars that sit on top of a series of vertical bars; the horizontal bars shift from dark on the bottom to light on the top, and the vertical bars range from dark on the left to light on the right; as a consequence, the intersections between the horizontal and vertical bars create a gamut of varied contrast levels. The power of Spillmann and Levine’s display is that it shows at a glance the effect of contrast on the appearance at the intersections: against a dark background (Figure 9.1, left panel), smudges arise at intersections with darker bars in front of brighter bars, while against a white background (Figure 9.1, right panel), smudges arise at intersections with brighter bars in front of darker bars. One can therefore imagine that this display is an early version of what has been termed grating induction (Blakeslee and McCourt, 1999; Foley and McCourt, 1985; Frisby, 1980; McCourt, 1982), a display in which a background sinusoidally modulated grating causes an equiluminant uniform gray stripe to appear either brighter or darker in comparison to the background.

Figure 9.1 Examples of the displays used by Spillmann and Levine (1971) to investigate the effect of contrast on induced ‘smudges’ at the intersection. The display consists of horizontal bars of varying luminance placed on top of vertical bars of varying luminance. The left panel shows the bars on a dark background, the right panel shows the bars on a bright background. The perceptual smudges arise only at those intersections where the bar with the lower contrast to the background crosses in front of the bar with the higher contrast to the background (i.e. the right side of the diagonal with black background and on the left of the diagonal on the white background).

A variant of this principle can be seen in Spillmann’s weaves (Figure 9.2), a design produced by Hamburger and Shapiro (2009) to investigate various spatial and chromatic properties of grid-like displays. In Spillmann’s weaves, the vertical and horizontal bars are woven together so that the light bars cross in front of dark bars at some intersections and behind dark bars at others. As might be expected (Oehler and Spillmann, 1981; Spillmann, 1994; Spillmann and Levine, 1971), light bars in front of dark bars produce smudges at the intersections only if the grid has a white background, while dark bars in front of light bars produce smudges only if the grid has a dark background.

Figure 9.2 Example of Spillmann’s weaves (Hamburger and Shapiro, 2009). The horizontal and vertical bars are intertwined. On the left, the bars are placed on a dark background and on the right the same bars are placed on a bright background. As with Figure 9.1, the smudges arise at intersections where the bars with the lower contrast to the background cross in front of bars with the higher contrast to the background. So on the left, smudges arise when the dark horizontal bars are in front of the bright vertical bars, and on the right the smudges arise at the opposite intersections.

Hamburger and Shapiro (2009) noted that the smudges in Spillmann’s weaves differ from the smudges in the Hermann grid in a number of ways. First, the smudges in Spillmann’s weaves are far more resilient than the smudges in a typical Hermann grid because smudges in Spillmann’s weaves are not abolished by jagged, wavy, or thick bars, or orientation changes, whereas such changes affect the Hermann grid’s smudges (e.g. Geier et al., 2004, 2008). Second, differences arise with regard to eccentricity: smudges in Spillmann’s weaves can occur both foveally and peripherally, whereas smudges in the Hermann grid arise only in the periphery. Additionally, there are changes that occur when displays are filtered: low-pass versions of Hermann’s grid create the appearance of a scintillating grid (Schrauf et al., 1997), whereas low-pass versions of Spillmann’s weaves do not. Furthermore, at equiluminance, smudges are perceived at all intersections with the bars placed against a light or dark background (a pattern similar to the standard Hermann grid), but smudges do not appear when the bars are placed against a gray background.

One curious observation from Hamburger and Shapiro (2009) was that the smudges in the Hermann grid and in Spillmann’s weaves are physically present in spatially high-passed versions of the image. Shapiro and Lu (2011) showed that most brightness illusions could be accounted for simply by removing the low spatial frequency content from the image. That is, if a brightness illusion image is filtered so that low spatial frequency content is removed, then the test patches that were physically identical in the original image are physically different in the filtered image. These results may not initially seem surprising: lateral inhibition acts like a high-pass filter, and lateral inhibition has long been known to account for some brightness illusions (Ratliff, 1971). However, lateral inhibition has generally been thought to apply to edges, but Shapiro and Lu showed that a more general high-pass filtering mechanism can account for a wider range of brightness illusions since they do not need to operate strictly at thin contrast edges. They argued that the visual system has a number of functional reasons to remove low spatial frequency content from images (i.e. removal of motion blur in response to eye movements, removal of low spatial frequency content for object representation, object sharpening, and possibly others). Any visual process that removes blur from an image will create the conditions necessary for making test patches with the same pixel value to appear to be of different brightness levels. More than that, high-pass filtering based on these types of processes do not need to operate over a static range; it is likely that these processes will adapt to the content of the image so that they will remove different amounts of low spatial frequency information in different situations.

Here, we will present additional observations about the effects of high-pass filtering on a Spillmann-and-Levine-type display. Our analysis is concerned with the following three questions:

1  

Can a simple filter simultaneously account for all of the smudges found in a Spillmann-and-Levine-type display?

2  

Would such a filter be acting to enhance edges, as typically thought of in a process involving lateral inhibition, or would the filter act over a large spatial region?

3  

Since high-pass filters do not add anything to the display (they subtract low spatial frequency content), what does the success of these filters at accounting for these illusions imply about the illusory aspects of the display? That is, if the smudges appear in the high-pass filtered images, then the smudges are physically present at some portion of the spatial spectra. Would we then continue to consider the smudges in the weaves and grids to be a fiction created by the visual system?

Lastly, we will demonstrate that changes in background luminance shift the perceived grouping of the display. While this topic is slightly separate from the weaves displays, it keeps with the ‘beyond Gestalt’ theme of this book. The importance of these displays is that they suggest that perceptual organization can be based on change in contrast between the elements, not the perceived brightness or color of the objects themselves. Such organizational properties are interesting because they are cases where grouping is based upon information that cuts across the figure and ground border and therefore cannot be described by theories that concentrate solely on elements that are either figure or background.

High spatial frequency content in Spillmann’s weaves

In this section, we will show the results of high-pass filters applied to Spillmann and Levine figures. In Figure 9.3 (in the Color plate section), panels a, b, and c show the grids on white, black, and gray backgrounds; panels d, e, and f show the same grids with a portion of their low spatial frequencies removed. The filtering process used to construct images like those in panels d, e, and f of Figure 9.3 consists of three steps:

1  

blur the original image with a convolution function

2  

subtract the blurred image from the original image, and

3  

add a constant value to remove negative values from the filtered image.

The only parameter in this process is the size of the filter kernel in the convolution process in the first step. A small filter kernel removes most of the low spatial frequency content in the image and therefore results in fina l images that show only sharp edges. Large filter kernels allow more low spatial frequency content to remain in the image; these filters create final images that appear very similar to the original image. Shapiro and Lu (2011) showed that the results from this filtering process could also be created by using Adobe Photoshop’s High-Pass Filter function. The Adobe function contains a parameter (filter radius), and the procedure presumably uses a similar, but not identical, process to that described above. Panels d, e, and f of Figure 9.3 were created with Adobe Photoshop with the filter radius equal to the size of the intersection in the original image (20 pixels).

The filtered images (panels d, e, and f) show an increase or decrease in the pixel values at intersections that correspond to the locations where smudges appear in the corresponding displays. For instance, in Figure 9.3d, the inset shows three 3 × 3 grids. The values in each of these grids represent the physical values of the pixels in the highlighted region; the values were assessed as the average of a 5 × 5 pixel square placed at the centre of the location. In the original images (panels a, b, and c), the values of the horizontal bars are all equal, but in the filtered images (panels d, e, and f), the values of the centre pixels increase or decrease relative to the adjoining horizontal values. In Figure 9.3d, for example, the values of the top left square demonstrate that the centre intersection value has increased in brightness from 196–199 to 215. If we assume that the relative pixel values in the filtered image relate to the relative brightness of the original image, then one would expect that the smudges should appear.

How does the kernel size affect the ability of the filter to model the smudges at the intersections of the weaves display?

As discussed above, changing the filter size is similar to changing the cut-off frequency for a high-pass filter: a small filter kernel only allows a narrow range of high spatial frequency content to pass; a large filter kernel still works as a high-pass filter but more low spatial frequency content passes. It is therefore conceptually useful to think of small filter kernels as a type of edge extraction filter, whereas a larger filter kernel is similar to a sharpening or ‘blur removal’ filter.

In Figure 9.4 (in the Color plate section), we show the effect of changing kernel size on the Spillmann and Levine displays. The top row shows Spillmann and Levine grids on black, gray, and white backgrounds. The second row shows the pixel values of each of the horizontal bars. Each line in the plot represents the pixel values for each horizontal bar; the values of each line are at different pixel levels since each of the horizontal bars has a different luminance, and the values for each bar are constant since each bar is spatially homogeneous. The figures in each row show the values of the horizontal bars after the images are passed through a high-pass filter. In row three, the images were filtered with a small kernel (5 pixel diameter); in row four, with a medium-sized kernel (25 pixel diameter); and in row five, with a large kernel (60 pixel diameter).

Examine first the figures in the fourth row – that is, the row with a filter kernel that has a size approximately equal to the width of the intersections. The pixel values of each bar show an increase and decrease across the horizontal values; the changes arise at the intersections and correspond nicely to the perceived smudges in the Spillmann and Levine grids. That is, with the black background, the pixel fluctuations increase on the right half of the display; with a gray background, the pixel fluctuations are greatest away from the centre; and with a white background, the pixel fluctuations are greatest on the left half of the display. If the visual system has a filter that operates in this range, we would then expect that the horizontal bars would not be perceived as spatially homogeneous.

Smaller and larger kernel sizes, however, do not produce changes that correspond to the induced smudges. The effects of filtering with a small kernel are shown in the third row. A small kernel corresponds to a type of filter that would accentuate edges and is what is typically considered when discussing lateral inhibition. This size filter removes most of the mean background variation in the rows and does not produce much in the way of wavy variations that would be expected at the intersections. The effects of filtering with a large kernel, which corresponds to a type of filter used to remove shadows from a display, are shown in the bottom row. This size filter shows the general slope of luminance variation across the display, but, not surprisingly, the filter is too big to accentuate the variations that arise at the intersections.

We have demonstrated that perceptual smudges found at the intersections of Spillmann and Levine displays are physically present in the high-pass filtered version of the image. In the introduction, we set out to test three issues concerning the implications of the filtered images. The first issue was whether a single filter can account for the induction seen in the Spillmann and Levine display. Figures 9.3 and 9.4 demonstrate the success of a single filter; the filter, however, has to be adjusted to the ‘correct size’ and therefore cannot be applied arbitrarily.

The second issue was whether these filters should be thought of as an extension of retinal lateral inhibition, or whether the filter might represent some other type of visual process. As discussed in the introduction, the filter process shares some properties with lateral inhibition; lateral inhibition, however, is frequently thought of as a process for extracting or accentuating information at the edges of the display (Ratliff, 1971). We have shown that when the filter is adjusted to primarily extract edges, the resulting image does not carry enough information to produce smudges without additional processes (like ‘filling-in’ brightness between the edges). Similarly, filters that are too large will not capture variations at the intersections; the filter that creates induction would therefore have to be of the correct ‘middle’ size.

There are many possible reasons as to why only a middle-sized filter would account for the brightness variation. One possibility is that there is a multi-channel filtering process, as suggested by the Blakeslee and McCourt Oriented Difference of Gaussians (ODOG) model (Blakeslee et al., 2005). In Blakeslee and McCourt’s model, the spatial channels adaptively adjust their response so that the most active channel carries most of the information in the image. The middle-sized filter would therefore ‘select itself’, so to speak, because of the bottom-up nature of the spatial frequency channel network. Another possibility is that the size of the middle-sized filter corresponds to the relevant content in the scene. As discussed by Shapiro and Lu (2011), an object carries information over a range of spatial frequencies, but spatial frequency content that is lower than the size of the object is not necessary for the identification of the object. Carrying low spatial frequency content in the neural representation of the object would therefore be inefficient. The processes that represent the object would remove low spatial frequency content; these processes may involve bottom-up responses similar to the adaptive processes discussed by Blakeslee et al. (2005), but this does not necessarily have to be so as such filtering could occur in many different stages of visual processing. An adaptive filter response could potentially account for the absence of smudges in the Hermann grid when the display is rotated (De Lafuente and Ruiz, 2004) or manipulated (Geier et al., 2004, 2008), since such changes could also lead to adjustments in the filter-kernel size. Yet another possibility is that the visual system discards low spatial frequency information in response to eye movements, in which case a model based on multiple spatial channels does not offer a clear advantage. Indeed, this option corresponds to recent research in physiology that suggests that retinal ganglion cells (Rucci et al., 2007) and cortical cells (Frazor et al., 2004; Mihalas et al., 2011) constantly shift their spatial response to higher frequencies in direct response to eye movements and at timescales that are consistent with eye movements.

The third issue mentioned in the introduction is whether the smudges in the Spillmann and Levine display are really illusory. Richard Gregory (1997) created a taxonomy of visual illusions based on the categories of ambiguities, distortions, paradoxes, or fictions. Working within this framework prior to the current study, researchers might categorize the smudges of the Hermann grid and Spillmann’s weaves as fictions since most researchers discuss the smudges as if they were not physically present in the original image. However, the results in Figures 9.3 and 9.4 demonstrate that smudges actually correspond to energy that is physically present in some portion of the spatial spectra. This means that a photometer would indicate luminance changes at the intersection of the display if the photometer had a spatial frequency filter that removed low spatial frequency content prior to making a measurement. It would be as if we changed the spectral response of our cone photoreceptors and then found that two metameric lights were no longer metameric. Such a finding would hardly be surprising.

Figures 9.3 and 9.4 suggest that the visual system is representing real information that is not captured when discussing the display in terms of pixel values. This suggests that the visual system organizes perception based on information that may not be intuitively obvious. In keeping with the ‘beyond Gestalt’ theme of this book, we will expand on this point with a display that demonstrates that changes in background luminance can have major effects on Gestalt organization– a topic that was addressed in Shapiro and Hamburger (2007).

Figure 9.5 (in the Color plate section) shows an example of such a display; this display was presented as part of the best illusion of the year 2011 contest. Figure 9.5 consists of a 10 × 10 array of disks placed against a background filled with a diagonally oriented achromatic gradient field (top) or colored gradient field (bottom); both background gradients have the same luminance profile. The disks are always physically identical to each other, but their luminance changes as a function of time – that is, the disks change from light to dark and back again.

If the disks in Figure 9.5 were organized based on the luminance, proximity, or similarity of the disks, then one would expect that the background would not have an effect on the perceptual organization of the array; however, this is not the case. The disks follow a pattern organized diagonally, following the motion of the point of minimum luminance contrast between the disks and the background. The colored display shows the same effect as the achromatic display and confirms that the grouping is due to luminance contrast and not simply the disappearance of the disks (i.e. black disks against the black background).

The importance of displays like Figure 9.5 and the filtered weaves in panels d, e, and f in Figures 9.3 and 9.4 is that they suggest that perceptual organization can be based on changes that go beyond the description of the figures and grounds themselves but depend upon contrast between the elements. Such organizational properties suggest that Gestalt processes can group objects both by their contrast relative to the background and the luminance of the objects. In this construction, Gestalt principles take on new importance because they can be applied to different types of information in the visual scene.

Summary/conclusion

Our approach is driven in part by advances in neuroscience that indicate that the visual system is divided into multiple processing streams, with their own maps of the visual world. An unresolved problem is whether the Gestalt rules are applied only when the separate maps are combined, or whether each of these maps has its form of organizing Prägnanz, which can get relayed to a later stage of neural processing. It is our contention that systematic investigations using these principles will strengthen the connection between Gestalt theory and visual neuroscience.

We find this approach of grouping by contrast to be a response to an explicit call for a Gestalt neuroscience that was recently put forward by Tse (2004). A Gestalt neuroscience will have to address fundamental questions concerning the nature of the stimulus information that can be used by Gestalt organizing principles. Gestalt neuroscience may seem to many to be a contradiction. At the core of Gestalt theory is an abhorrence of structuralist reductionism – the idea that percepts can be understood in terms of the elementary components. Almost 120 years ago, the Austrian philosopher Chistian von Ehrenfels introduced the principle of Übersummativität (superadditivity) as one characteristic for a Gestalt (von Ehrenfels 1890). The basic premise is that the entirety of an object has properties that are different from those of its parts; therefore, an object cannot be defined strictly by its components. Over the intervening years, this perceptual property has been invoked repeatedly as a fundamental aspect of our perception (e.g. Koffka, 1935; Sharps and Wertheimer, 2000; Steinman et al., 2000; Wertheimer, 1912). Neuroscience, on the other hand, is by its very nature a reductionist enterprise: the goal of understanding perception through neuroscience is the attempt to reduce the complexities of our perceptual world into the actions of the brain. Nonetheless, a Gestalt neuroscience has become increasingly relevant today as we seek to understand Gestalt factors in terms of interactions in computational neural networks (Spillmann and Ehrenstein, 1996, 2004) and in terms of different neural pathways (e.g. Herrmann and Bosch, 2001).

References

Baumgartner, G. (1960) ‘Indirekte Grössenbestimmung der rezeptiven Felder der Retina beim Menschen mittels der Hermannschen Gittertäuschung’, Pflügers Archiv für die gesamte Physiologie, 272: 21–22.

Blakeslee, B. and McCourt, M. E. (1999) ‘A multiscale spatial filtering account of the White effect, simultaneous brightness contrast and grating induction’, Vision Research, 39: 4361–4377.

Blakeslee, B., Pasieka, W. and McCourt, M. E. (2005) ‘Oriented multiscale spatial filtering and contrast normalization: a parsimonious model of brightness induction in a continuum of stimuli including White, Howe and simultaneous brightness contrast’, Vision Research, 45: 607–615.

Brewster, D. (1844) ‘A notice explaining the cause of an optical phenomenon observed by the Rev W Selwyn’, Report of the British Association for the Advancement of Science. Transactions of the Sections, 12: 8.

De Lafuente, V. and Ruiz, O. (2004) ‘The orientation dependence of the Hermann grid illusion’, Experimental Brain Research, 154: 255–260.

Foley, J. M. and McCourt, M. E. (1985) ‘Visual grating induction’, Journal of the Optical Society of America A, 2: 1220–1230.

Frazor, R., Albrecht, D., Geisler, W. S. and Crane, A. (2004) ‘Visual cortex neurons of monkeys and cats: temporal dynamics of the spatial frequency response function’, Journal of Neurophysiology, 91: 2607–2627.

Frisby, J. P. (1980) Seeing: Illusion, Brain, and Mind, Oxford, UK: Oxford University Press.

Geier, J., Séra, L. and Bernath, L. (2004) ‘Stopping the Hermann grid illusion by simple sine distortion’, Perception, 33 supplement: 53.

Geier, J., Bernáth, L., Hudak, M. and 8éra, L. (2008) ‘Straightness as the main factor of the Hermann grid illusion’, Perception, 37: 651–665.

Gregory, R L. (1997) ‘Knowledge in perception and illusion’, Philosophical Translations of the Royal Society of London B: Biological Sciences, 352: 1121–1128.

Hamburger, K. and Shapiro, A. G. (2009) ‘Spillmann’s weaves are more resilient than Hermann’s grid’, Vision Research, 49: 2121–2130.

Hamburger, K., Baier, F. and Spillmann, L. (2012) ‘The tilted Hermann grid illusion: “illusory spots” versus “phantom bands”’, Perception, 41: 239–242.

Hering, E. (1920) Grundzüge der Lehre vom Lichtsinn, Berlin, Germany: Springer.

Hermann, L. (1870) ‘Eine Erscheinung simultanen Contrastes’, Pflügers Archiv für die gesamte Physiologie, 3: 13–15.

Herrmann, C. S. and Bosch, V. (2001) ‘Gestalt perception modulates early visual processing’, Neuro Report, 12: 901–904.

Jung, R and Spillmann, L. (1970) ‘Receptive-field estimation and perceptual intergration in human vision’, in F. A. Young and D. B. Lindsley (eds) Early Experience and Visual Information Processing in Perceptual Reading Disorders, Washington, DC: National Academy of Sciences, pp. 181–197.

Koffka, K. (1935) Principles of Gestalt Psychology, New York, NY: Hardcourt, Brace.

Levine, J., Spillmann, L. and Wolfe, E. (1980) ‘Saturation enhancement in colored Hermann grids varying only in chroma’, Vision Research, 20: 307–313.

McCourt, M. E. (1982) ‘A spatial frequency dependent grating-induction effect’, Vision Research, 22: 119–134.

Mihalas, S., Dong, Y., von der Heydt, R., and Niebur, E. (2011) Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects. PNAS, 108 (18): 7583–7588.

Oehler, R and Spillmann, L. (1981) ‘Illusory colour changes in Hermann grids varying only in hue’, Vision Research, 21: 527–541.

Ratliff, F. (1971) ‘Illusions in man and his instruments’, The Journal of Philosophy, 68: 591–597.

Rucci, M., Iovan, R., Poletti, M. and Santini, F. (2007) ‘Miniature eye movements enhance fine spatial detail’, Nature, 447: 851–854.

Schiller, P. H. and Carvey, C. E. (2005) ‘The Hermann grid illusion revisited’, Perception, 34: 1375–1397.

Schrauf, M., Lingelbach, B. and Wist, E. R (1997) ‘The scintillating grid illusion’, Vision Research, 37: 1033–1038.

Shapiro, A. G. and Hamburger, K. (2007) ‘Grouping by contrast: figure-ground segregation is not necessarily fundamental’, Perception, 36: 1104–1107.

Shapiro, A. G. and Lu, Z. L. (2011) ‘Relative brightness in natural images can be accounted for by removing blurry content’, Psychological Science, 22: 1452–1459.

Sharps, M. J. and Wertheimer, M. (2000) ‘Gestalt perspectives on cognitive science and on experimental psychology’, Review of General Psychology, 4: 315–336.

Spillmann, L. (1971) ‘Foveal perceptive fields in the human visual system measured with simultaneous contrast in grids and bars’, Pflügers Archiv für die gesamte Physiologie, 326: 281–299.

Spillmann, L. (1994) ‘The Hermann grid illusion: a tool for studying human perceptive field organization’, Perception, 23: 691–708.

Spillmann, L. and Levine, J. (1971) ‘Contrast enhancement in a Hermann grid with variable figure-ground ratio’, Experimental Brain Research, 13: 547–559.

Spillmann, L. and Ehrenstein, W. H. (1996) ‘From neuron to Gestalt – mechanisms of visual perception’, in R Greger and U. Windhorst (eds) Comprehensive Human Physiology, Heidelberg, Germany: Springer, pp. 861–893.

Spillmann, L. and Ehrenstein, W. H. (2004) ‘Gestalt factors in the visual neurosciences’, in L. M. Chalupa and J. S. Werner (eds) The Visual Neurosciences, Cambridge, MA: MIT Press, pp. 1573–1589.

Steinman, R M., Pizlo, Z. and Pizlo, F. J. (2000) ‘Phi is not beta, and why Wertheimer’s discovery launched the Gestalt revolution’, Vision Research, 40: 2257–2264.

Tse, P. U. (2004) ‘Unser Ziel muss eine Gestalt-Neurowissenschaft sein’, Gestalt Theory, 26: 287–292.

von Ehrenfels, C. (1890) ‘Über Gestaltqualkäten, Vierteljahresschrift wissenschaftliche Philosophie’, 14: 249–292. Engl. translation in Foundations of Gestalt Theory, B. Smith (ed.), Munich, Germany: Philosophia-Verlag, 1988, pp. 82–120.

Wertheimer, M. (1912) ‘Experimentelle Studien über das Sehen von Bewegung’, Zeitschrift für Psychologie, 61: 161–265.

10  Motion illusions as a psychophysical tool to investigate the visual system

Simone Gori and Alan Stubbs

Introduction

Gestalt psychology provided many contributions to our understanding of visual perception and posed the basis for several research topics that are still hotly debated. Scientific investigation regarding figure-ground segregation and perceptual grouping has been amazingly productive over the last one hundred years (among others, some of the more recent published works are: Claessens and Wagemans, 2008; Gori and Spillmann, 2010; Gori et al., 2008b; Vezzani et al., 2012; for a more complete review see Wagemans et al., 2012 and Chapter 2 by Wagemans in this volume) revealing how central these topics are in the scientific community. The Gestalt tradition, however, risked a premature ending: Wertheimer and Koffka died prematurely and in Germany Metzger, Rausch, and Gottschaldt did have many PhD students, but few of them carried on in the Gestalt tradition (Wagemans et al., 2012 and Chapter 2 by Wagemans in this volume). However, the pioneering work of Spillmann has played a prominent role in connecting the Gestalt view with modern neurophysiology and psychophysics and reinvigorating Gestalt tradition (e.g. Gori and Spillmann, 2010; Spillmann 1997, 1999, 2006, 2009; Spillmann and Dresp, 1995).

Besides grouping and figure-ground segregation, another topic that has its roots in the Gestalt tradition are motion illusions.

Motion illusions

A motion illusion refers to a perception of motion that is absent and/or different (in direction, or strength) from what it is present in the stimulus itself. Illusory motion is experienced as real motion and it should not be confused with the perception of dynamism evoked by paintings (Gori et al., 2008a). These phenomena are a peculiar case of visual illusions that, because of their striking feature, captured the attention of vision scientists but also of artists, reaching a broader audience than is normally captivated by a scientific topic.

Motion illusions were at the basis of the Gestalt revolution of perception (Steinman et al., 2000), which was born with a serendipitous observation published by Wertheimer (1912/2012) of a pure apparent movement called the φ-phenomenon to distinguish it from optimal apparent movement (P), already known at least 60 years before (Steinman et al., 2000), which resembles real movement. Wertheimer named this kind of apparent motion ‘pure’ because it was perceived in the absence of any object being seen to change its position in space. During the following one hundred years many other stimuli were discovered producing different kinds of motion perception that is not ‘veridical’, i.e. a percept that does not agree with conditions present in the physical world. There are many possible ways to classify motion illusions (see Ito, 2012; Kitaoka and Ashida, 2007 for reviews). One option, which is followed here, is to divide them in two main categories based on the type of their illusory motion:

the motion observed without any type of motion in the stimulus and

the misperception of motion, i.e. the perception of motion that differs from the physical motion of the stimulus.

Both produced excellent examples of motion illusions that provided knowledge regarding the underlying visual system mechanisms (Spillmann, 2009).

Illusory motion in stationary stimuli

Notable examples of the first category are the:

MacKay Rays

Enigma Illusion

Ouchi Illusion

Fraser-Wilcox/Rotating Snakes Illusion.

This group is particularly striking because image motion seems to appear from nowhere. It is worth noting that Ito (2012) recently showed an interesting example of the opposite situation: the perception of a static pattern in presence of a physical motion.

The MacKay Rays (MacKay, 1957) is a radial ‘ray’ pattern with high luminance contrast, which induces perceptual waver and a concentric afterimage with illusory rapid rotation (Figure 10.1a). Motion in MacKay’s Rays was attributed to a cortical antagonism by Georgeson (1976), who suggested that pattern channels selective for a specific orientation are grouped together with channels selective for movement in the orthogonal direction. Based on this interpretation, the motion signals should arise from an antagonism between these channels mediated by inhibitory processes. The restlessness (‘jazzing’) was more recently attributed to moiré effects resulting from the superposition of the afterimage onto the stimulus pattern and its displacement by fixational eye movements (Wade, 2003). MacKay’s Rays shimmer motion was also attributed to irregular fluctuations of accommodation, inducing temporal modulation in stimulus processing (Gregory, 1993; Pirenne et al., 1958).

Figure 10.1 a) The MacKay Rays induce the perception of shimmering and jazzing. b) The Enigma Illusion elicits rotary motion on the three chromatic annuli. (Reprinted with kind permission of Nature Publishing Group on behalf of Nature from MacKay, 1957.)

Another striking example is represented by the Enigma, devised by Leviant in 1981 (Figure 10.1b). This pattern elicits spontaneous perception of rotary motion in the absence of real motion. The figure consists of a black and white ray pattern with narrowly spaced radial lines, onto which three chromatic rings are superimposed. The perceived motion on the annuli is seen alternating between clockwise and counterclockwise rotation (Gori et al., 2006; Leviant, 1996). The reversal between clockwise and counterclockwise motion seems to happen (on average), every 4.7 s (Gori et al., 2006). Orthogonal junctions (T-junctions) between the radial lines and the superimposed circles are most effective in eliciting the rotatory motion (Gori et al., 2006; Hamburger, 2007; Kumar and Glaser, 2006). Interestingly, Enigma seems to be less powerful in young children (Billino et al., 2009).

Several attempts were made to understand the mechanisms underlying the motion effect induced by the Enigma. Gregory (1993) proposed an explanation in terms of fluctuations in eye accommodation (‘hunting for accomodation’) and fixational eye movements. Zeki and co-workers (1993), based on their positron emission tomography (PET) study, assumed that perceived motion of the Enigma was generated by the cortex. Other authors claimed that neither the cortical nor the accommodative hypothesis were satisfactory (Fermüler et al., 1997; Mon-Williams and Wann, 1996). According to those authors, the gaze instability provides the necessary mechanism to perceive illusory motion but a brain activity is necessary as well. On the other hand, Kumar and Glaser (2006) claimed that motion produced by the Enigma is not the result of united motion of the image on the retina due to microsaccades of the eye, but has a cortical origin. Gori et al. (2006) highlighted a clear regularity in the reversals that suggests the presence of a neural saturation mechanism at the cortical level. Moreover, they showed how real motion can influence illusory motion, suggesting that these two kinds of motion interact at some point along the motion-processing pathway. Later, Hamburger (2007) created a long-lasting positive afterimage of the Enigma using a light flash. With this setup the subjects were able to view the Enigma as a positive afterimage stabilized on the retina. Subjects reported motion in the Enigma pattern in the first seconds while Enigma, viewed in free vision, requires about two seconds to start to spin, as reported by Gori et al. (2006). Moreover, the perceived motion in the afterimage did not have the same quality of the motion experienced in the Enigma during free vision and it was described as ‘spurious and faster’. On the other hand, Troncoso et al. (2008) convincingly reported that small eye movements are a necessary condition to perceive the illusory motion. Recently, Ruzzoli and colleagues (2011) demonstrated a causal role of V5/medial temporal (MT) in the Enigma (and not of V1) using repetitive transcranial magnetic stimulation (rTMS).

In sum even though fixation eye movements can probably trigger the Enigma effect the cortical activity plays a crucial role in the perception of the illusory motion.

Another important example that evokes motion perception in absence of real motion is the so-called Ouchi Illusion (Figure 10.2). This illusion was discovered by Lothar Spillmann (Spillmann et al., 1986) looking at a pattern printed in a Japanese geometrical art book authored by Ouchi (1977). Ashida (2002), describing the Ouchi’s book, specified that it only contains figures with no text, and it is impossible to find who created this figure or if the author was aware of the effect. Kitaoka and Ashida (2007) reported later that Ouchi’s pattern was accidentally produced and no other patterns in that book provided any perceived motion. The Ouchi Illusion arises from a checkered circle superimposed on a checkered background. Both areas are filled with high-contrast check patterns of rectangular elements. The orientation of the rectangles filling the circle is orthogonal to the rectangles orientation characterizing the background. This pattern elicits a vivid sense of sliding motion between the circle and the background without physical motion. The sliding motion is enhanced if real motion is added to the configuration but the effect is present also in a static pattern. During the static view the motion necessary to trigger the illusion is probably provided by fixational eye movements.

Figure 10.2 The Ouchi Illusion produces sliding motion of the central disk.

The striking apparent motion elicited by the Ouchi’s pattern triggered several attempts to explain it. The sliding motion has been first attributed to a failure in the integration of local motion signals into a rigid percept (Hine et al., 1995). Other authors stressed the role of the orientation of the checks in the circle that was orthogonal to the ones of the background attributing the perceived motion to the integration of local motion vectors arising from the aperture problem (e.g. Mather, 2000). Ashida et al. (2005) showed that the relative amplitude of the relevant Fourier fundamentals and harmonics leads to a quantitative prediction of the Ouchi effect. However, Pinna and Spillmann (2002, 2005) showed how the orthogonal orientations of the elements filling the circle and the background cannot be the cause, as it was supposed before, but they only produce the directional bias. A definitive explanation of the phenomenon seems still absent after almost 30 years of research.

The last illusion in this family is the Fraser and Wilcox (1979) Illusion, which is characterized by repeated luminance gradations in a circular configuration that create an impression of slow rotation. The effect appears stronger in a stimulus with higher contrast and/or larger eccentricity (Naor-Raz and Sekuler, 2000). Kitaoka and Ashida (2003) optimized the Fraser and Wilcox Illusion to give a much more powerful result by proposing a rule that illusory motion is stronger in the repetition of the following luminance sequence: black-dark gray-white-light gray-black. The authors called this illusion Rotating Snakes (Figure 10.3). The mechanisms underlying this phenomenon are still highly debated. Conway et al. (2005) reported that the differences in response latency to different contrast elements are responsible for the illusion, and provided the first evidence that pairs of stimuli with the above mentioned luminance sequence can generate motion signals in V1 direction-selective neurons. Backus and Oruç (2005) introduced the effect of adaptation to model the smooth motion perception over several seconds and hypothesized that gaze instability can improve the strength of the illusion, even though it is not necessary. On the other hand, Murakami et al. (2006) found a correlation between the strength of the perceived motion and eye movements, specifically fixational drifts. Beer and colleagues (2008) also confirmed the role of fixational drifts in the illusion. Recently, Otero-Millan et al. (2012) found results supporting that transient oculomotor events such as microsaccades, saccades, and blinks, rather than continuous drift, act to trigger the Rotating Snakes. An fMRI study (Kuriki et al., 2008), reported an increase in neural activity of the motion sensitive visual cortex during the observation of Rotating Snakes. In sum, this motion illusion also seems to not be fully explained yet and both fixational eye movements and cortical processes appear to be relevant for the illusion occurrence.

Figure 10.3 The Rotating Snakes Illusion elicits a rotary motion of the disks.

Misperception of motion

The second category of motion illusions is characterized by the presence of physical motion that is incorrectly processed by the visual system ending up to a percept that differs from the real motion. Notable examples of this category are the:

Barber Pole Illusion

Revolving Wheels

Rotating Tilted Lines Illusion

Unchained Dots Illusion

Accordion Grating Illusion.

In 1929, Guilford noted the paradoxical motion inherent in the stripes on a rotating barber pole. The Barber Pole rotates around its vertical axis (horizontally), but the stripes appear to move upwards (in the direction of its vertical axis) rather than turning around the pole. The Barber Pole Illusion offers a compelling example of the importance of boundary classification (Lidén and Mingolla, 1998). Wallach (1935/1996) showed that, under normal conditions, the perceived direction of motion of a diagonal grating pattern drifting behind a rectangular aperture depends upon the direction of elongation of the aperture. The grating pattern is seen to move along the axis of the longer dimension of the rectangle. The prevailing explanation for this phenomenon suggests that the perceived direction of motion results from the integration of motion signals from the areas where the grating pattern is terminated by the edges of the aperture (Lidén and Mingolla, 1998). The local velocity signals along the grating are inherently ambiguous due to the well-known aperture problem.

The aperture problem

The classical two-dimensional aperture problem (e.g. Caplovitz, 2011; Marr and Ullman, 1981) confronted by cells with spatially limited receptive fields can be viewed as follows: each neuron in the visual system is sensitive to an input in a small portion (the aperture) of the visual field, as if each neuron is looking through a window. The motion direction of a contour is ambiguous, because the motion component parallel to the line cannot be measured based on the visual input. This means that a variety of contours with different orientations moving at different speeds can cause identical responses in a motion-sensitive neuron (Gori et al., 2011). Line ends (terminators), however, produce unambiguous motion signals. When terminators are visible inside the receptive field of a motion-processing neuron the veridical motion is signaled by that cell. On the contrary, when the line is interrupted by the spatial limit of the receptive field, the real line ends cannot be processed and the shape of the aperture determines the actual terminators. If a diagonal grating is moving from left to right under a vertically elongated aperture, for example, it will present a larger number of terminators with vertical trajectories, consequently vertical motion will be perceived (Lidén and Mingolla, 1998). Recently, a new three-dimensional aperture problem was formulated introducing the observer distance as a relevant variable to predict the final perception of motion (Yazdanbakhsh and Gori, 2011).

The Barber Pole phenomenon is, in some way, the ‘grandmother’ of almost all the phenomena of this category.

Another very popular example of misperception of motion is the Revolving Wheels Illusion (RWI) also known as the Pinna-Brelstaff Illusion (Pinna and Brelstaff, 2000). The RWI pattern appears as two concentric circles composed of microelements consisting of two black line segments and two white ones drawn on a gray background (Figure 10.4). Moving toward or backward from this stimulus it makes the circles appearently rotate. Gurnsey and colleagues (2002) showed that the microelements of the RWI have a strong orientation once they are blurred. The blurred patches are very similar with respect to how these microelements are seen in peripheral vision. When moving the head closer to the pattern the circle expands on the retina, but the interpreted motion of each microelement is strongly influenced by its local orientation, due to the aperture problem. The preferred direction of motion of each microelement will be at an angle of 45 deg to its expansion resulting in a global illusory rotary motion. The aperture problem combined with blurry peripheral vision seems able to explain the RWI.

Figure 10.4 The Revolving Wheels Illusion produces rotary motion in the circles when one approaches the stimulus pattern moving the head toward the page, maintaining central fixation. (Reprinted with kind permission of Elsevier on behalf from Vision Research from Pinna and Brelstaff, 2000.)

Another illusion of this family, the Rotating-Tilted-Lines Illusion (RTLI; Gori and Hamburger, 2006) shares important characteristics with the RWI and could be very well considered the basis of the latter and the simplest pattern able to elicit circular motion in presence of radial motion on the retina. In its simplest form (Figure 10.5a) RTLI arises in a circular pattern of 120 black, radial lines tilted to the right and presented on a white background. When one approaches the stimulus pattern, the radial lines appear to rotate in the counterclockwise direction, whereas when one recedes from it, they appear to rotate clockwise. If the lines are tilted to the left, the rotation occurs in the opposite direction. Interestingly, Billino et al. (2009) investigated an age effect on the perception of several motion illusions. The RTLI was the only one to be perceived as often (96 per cent) by children (3–6 years) as by young adults (18–39 years). Surprisingly, the RWI was perceived by only 16.7 per cent of the children suggesting the RTLI as the best pattern to study illusory motion at early age. Moreover, according to Billino et al. (2009) the illusory rotation of the RTLI is perceived more often that the one elicited by the RWI during the age period from 3 to 64 years making the RTLI not only the simplest pattern able to elicit illusory rotation due to the observer motion but also the more powerful. Recently, Hamburger (2012) showed that the RTLI breaks down (or at least is strongly reduced) with isolumi-nant colors. This result suggests an involvement of the dorsal magnocellular pathway in processing this illusion.

Figure 10.5 a) The Rotating-Tilted-Lines Illusion (RTLI) produces rotary motion in the circle of tilted lines when one approaches the stimulus pattern moving the head toward the page, maintaining central fixation. b) A more complex version of the RTLI. c) The illusory motions of the RTLI and the Enigma illusion united. d) The RTLI with blurred edges.

Orientation of the lines seems to be the main factor in eliciting perceived motion in the RTLI (Gori and Hamburger, 2006; Gori and Yazdanbakhsh, 2008). The RTLI is composed of simple lines having terminators at the line ends, which aid correct motion detection compared with the ambiguous motion signal in the middle of the lines. The origin of the RTLI may be understood as a consequence of coding ambiguous and non-ambiguous motion signals afforded by the stimulus (Gurnsey et al., 2002; Yazdanbakhsh and Gori, 2008) or, equivalently, as a result of competition between motion signals originating from two different classes of motion-processing units (Gori and Yazdanbakhsh, 2008; Gori et al., 2011; Lorenceau et al., 1993). Two different units in the early visual cortex are supposed to be involved in this misperception of direction. Some motion-processing units maximally respond when the full length of a line passes over their receptive field (called contour units hereafter). These units face the aperture problem: they can only pick up the normal component of velocity from any straight line or edge that fills their receptive field. The second type of motion-processing unit responds to a moving line best when one or other line’s ends (terminators) fall within their receptive field. Such units, symmetrical or asymmetrical end-stopped neurons, can indicate the true direction of motion of the entire line from information afforded by the line-end motion. The subjects’ misperception of line directions could happen because the overall response of the contour units overcomes the response to terminator motion. Based on that hypothesis Yazdanbakhsh and Gori (2008) were able to measure the receptive fields size psychophysically, matching the previous electrophysiological results. Gori and Yazdanbakhsh (2008) created and empirically tested the pattern shown in Figure 10.5d which generates almost no signal for the end-stopped neurons owing to the blurred edges. Only the contour units would be involved in the production of the final percept. The significant enhancement in illusory motion strength observed with this pattern supports the involvement of end-stopped neurons in the typical RTLI pattern.

In sum the RTLI is able to bridge the phenomenology and the neurophysiology associated with this motion illusion and it seems to be explained by the competition between two motion-processing units in V1, one of them facing the classical aperture problem.

Recently, another example in this category was published: the Unchained Dots Illusion (UDI) (Gori et al., 2010). This effect seems to be very peculiar because, unlike most of the illusions of this category, it cannot simply be explained by the aperture problem faced by the motion sensitive neurons in the early visual cortex. The UDI is based on the misperception of dot trajectories. This pattern is composed of a dot matrix with the shape of an isosceles trapezoid (Figure 10.6). Watching this pattern while it is moving up and down on a screen, with central fixation, observers report a diagonal expansion of the dots when the matrix stimulus is going up and a contraction when it is going down. Although the physical trajectory of each dot is purely vertical, what is perceived is a diagonal outward movement when the pattern is moving up and diagonal inward movement when it is moving down. Contrarily to the other cases of the motion misperception family the local motion is congruent with the global motion. Had this moving-dots pattern been processed only locally by small receptive fields, the resulting pattern would be perceived as dots moving up and down following only a vertical trajectory: thus, no motion misperception would occur. But this is not the result for the pattern described above. Gori et al. (2010) proposed an explanation of the effect in terms of a ‘perceptual compromise’ between a new kind of horizontal apparent motion and vertical veridical motion of the dots. This compromise cannot happen in V1, but higher visual cortical areas should be involved.

Figure 10.6 The Unchained Dots Illusion scrolled up and down, with central fixation, produces a misperception of dot trajectories (diagonal instead of vertical).

The last phenomenon of this category is the Accordion Grating Illusion (AGI) (Gori et al., 2011), which is probably the simplest pattern able to elicit a strong misperception of motion. This phenomenon is best characterized by two illusory effects that occur when viewing an elementary square-wave grating while making back-and-forth head movements along the projection line (Figure 10.7). Observers report a non-rigid distortion of the pattern, including an expansion in a direction perpendicular to the stripes, and a perceived curvature of the stripes. The first depends on the classical physiological aperture problem that confronts early cells in the visual system. A simple model able to explain that illusory effect was successfully tested by a nulling psychophysical experiment (Gori et al., 2011). Regarding the illusory curvature, that could also be experienced in a more complex pattern like a checkerboard grid (Foster and Altschuler, 2001; Gori et al., 2013), led to the introduction of a non-classical three-dimensional (3-D) aperture problem (Yazdanbakhsh and Gori, 2011) able to accurately predict the final percept. In sum the AGI is able, like the RTLI to connect the phenomenology and the neurophysiology and it challenged vision scientists to postulate a new type of aperture problem (3-D aperture problem), never theorized before, that seems to be able to explain the phenomenon of illusory curvature.

Figure 10.7 The Accordion Grating Illusion approached by moving the head towards the page with central fixation, produces a horizontal instead of radial expansion and a curvature of the stripes. (Reprinted with kind permission of Elsevier on behalf of Neural Networks from Gori et al., 2011.)

Summary/conclusion

Motion illusions act as a non-invasive window into the neurobiology of our visual system. Why are these phenomena of interest to perception research? Paraphrasing Spillmann (2009) motion illusions are nonveridical percepts that uncover the processes by which veridical perception mediates our representation of motion. These phenomena are exceptions able to reveal the ‘brain footprint’ superimposed upon the stimulus: the constraints of the visual system itself.

The psychophysical studies of the motion illusions provided a large amount of information about how the motion is processed and elaborated by the visual system during a journey that was started from the Gestalt school and arrived, reinvigorated by the connection with modern neuroscience (Ehrenstein et al., 2003; Spillmann, 1997, 1999, 2006, 2009; Spillmann and Dresp, 1995), until the present day. The Gestalt observations proved to be, once again, a priceless starting point for systematically following investigations of the brain. In particular, the family of motion illusions in stationary patterns showed us that the motion necessary to perceive the illusory motion seems to be provided by fixational eye movements. The study of this family of phenomena has helped neuroscientists to a large degree to understand the crucial problem of how the brain creates a stable world representation with such an instable sensor (the eyes are moving almost continuously). On the other hand, the second family of illusions illustrated in this chapter provided substantial insight on how our brain is able to process motion directions very accurately and how this process is hierarchically organized in the visual cortical areas.

Acknowledgements

We thank Enrico Giora, who provided useful comments on an earlier version of the manuscript.

References

Ashida, H. (2002) ‘Spatial frequency tuning of the Ouchi Illusion and its dependence on stimulus size’, Vision Research, 42: 1413–1420.

Ashida, H., Kitaoka, A. and Sakurai, K. (2005) ‘A new variant of the Ouchi Illusion reveals Fourier-component-based processing’, Perception, 34: 381–390.

Backus, B. T. and Oruç, I. (2005) ‘Illusory motion from change over time in the response to contrast and luminance’, Journal of Vision, 5(11): 1055–1069.

Beer, A. L., Heckel, A. H. and Greenlee, M. W. (2008) ‘A motion illusion reveals mechanisms of perceptual stabilization’, PLoS One, 3(7): e2741.

Billino, J., Hamburger, K. and Gegenfurtner, K. R. (2009) ‘Age effects on the perception of motion illusions’, Perception, 38: 508–521.

Caplovitz, G. P. (2011) ‘Visual form motion interactions’, in A. M. Columbus (ed.) Advances in Psychology Research, vol. 82, Hauppauge, NY: Nova Science Publishers.

Claessens, P. M. E. and Wagemans, J. (2008) ‘A Bayesian framework for cue integration in multistable grouping: Proximity, collinearity, and orientation priors in zigzag lattices’, Journal of Vision, 8(7): 1–23.

Conway, B. R., Kitaoka, A., Yazdanbakhsh, A., Pack, C. C. and Livingstone, M. S. (2005) ‘Neural basis for a powerful static motion illusion’, Journal of Neuroscience, 25: 5651–5656.

Ehrenstein, W. H., Spillmann, L. and Sarris, V. (2003) ‘Gestalt issues in modern neuroscience’, Axiomathes, 13: 433–458.

Fermüller, C., Pless, R. and Aloimonos, Y. (1997) ‘Families of stationary patterns producing illusory movement: Insights into the visual system’, Proceedings of the Royal Society of London, B, 264: 795–806.

Foster, C. and Altschuler, E. L. (2001) ‘The bulging grid’, Perception, 30: 393–395.

Fraser, A. and Wilcox, K. J. (1979) ‘Perception of illusory movement’, Nature, 281: 565–566.

Georgeson, M. A. (1976) ‘Psychophysical hallucinations of orientation and spatial frequency’, Perception, 5: 99–111.

Gori, S. and Hamburger, K. (2006) ‘A new motion illusion: The Rotating-Tilted-Lines Illusion’, Perception, 35: 853–857.

Gori, S. and Yazdanbakhsh, A. (2008) ‘The riddle of the Rotating-Tilted-Lines Illusion’, Perception, 37: 631–635.

Gori, S. and Spillmann, L. (2010) ‘Detection vs. grouping thresholds for elements differing in spacing, size and luminance. An alternative approach towards the psychophysics of Gestalten’, Vision Research, 50: 1194–1202.

Gori, S., Hamburger, K. and Spillmann, L. (2006) ‘Reversal of apparent rotation in the Enigma-figure with and without motion adaptation and the effect of T-junctions’, Vision Research, 46: 3267–3273.

Gori, S., Pedersini, R. and Giora, E. (2008a) ‘How do painters represent motion in garments? Graphic invariants across centuries’, Spatial Vision, 21: 201–227.

Gori, S., Giora, E. and Pedersini, R. (2008b) ‘Perceptual multistability in figure-ground segregation using motion stimuli’, Acta Psychologica, 129: 399–409.

Gori, S., Giora, E. and Stubbs, D. A. (2010) ‘Perceptual compromise between apparent and veridical motion indices: The Unchained-Dots Illusion’, Perception, 39: 863–866.

Gori, S., Giora, E., Yazdanbakhsh, A. and Mingolla, E. (2011) ‘A new motion illusion based on competition between two kinds of motion processing units: The Accordion Grating’, Neural Networks, 24: 1082–1092.

Gori, S., Giora, E., Yazdanbakhsh, A. and Mingolla, E. (2013) ‘The novelty of the “Accordion Grating Illusion”’, Neural Networks, 39: 52.

Gregory, R. L. (1993) ‘A comment: Mackay Rays shimmer due to accommodation changes’, Proceedings of the Royal Society of London, B, 253: 123.

Guilford, J. P. (1929) ‘Illusory movement from a rotating Barber pole’, American Journal of Psychology, 41: 686–687.

Gurnsey, R., Sally, S. L., Potechin, C. and Mancini, S. (2002) ‘Optimising the Pinna-Brelstaff Illusion’, Perception, 31: 1275–1280.

Hamburger, K. (2007) ‘Apparent rotation and jazzing in Leviant’s Enigma Illusion’, Perception, 36: 797–807.

Hamburger, K. (2012) ‘Still motion? Motion illusions and luminance contrast’, Perception, 41: 113–116.

Hine, T. J., Cook, M. and Rogers, G. T. (1995) ‘An illusion of relative motion dependent upon spatial frequency and orientation’, Vision Research, 35: 3093–3102.

Ito, H. (2012) ‘Illusory object motion in the centre of a radial pattern: The Pursuit-Pursuing Illusion’, i-Perception, 3(1): 59–87.

Kitaoka, A. and Ashida, J. (2003) ‘Phenomenal characteristics of the peripheral drift illusion’, Vision, 15: 261–262.

Kitaoka, A. and Ashida, H. (2007) ‘A variant of the anomalous motion illusion based upon contrast and visual latency’, Perception, 36: 1019–1035.

Kumar, T. and Glaser, D. A. (2006) ‘Illusory motion in Enigma: A psychophysical investigation’, Proceedings of the National Academy of Sciences of the USA, 103: 1947–1952.

Kuriki, I., Ashida, H., Murakami, I. and Kitaoka, A. (2008) ‘Functional brain imaging of the Rotating Snakes Illusion by fMRI’, Journal of Vision, 8(10): 1–10.

Leviant, I. (1996) ‘Does “brain-power” make Enigma spin?’, Proceedings of the Royal Society of London, B, 263: 997–1001.

Lidén, L. and Mingolla, E. (1998) ‘Monocular occlusion cues alter the influence of terminator motion in the barber pole phenomenon’, Vision Research, 38: 3883–3898.

Lorenceau, J., Shiffrar, M., Wells, N. and Castet, E. (1993) ‘Different motion sensitive units are involved in recovering the direction of moving lines’, Vision Research, 33: 1207–1217.

MacKay, D. M. (1957) ‘Some further visual phenomena associated with regular patterned stimulation’, Nature, 180: 1145–1146.

Marr, D. and Ullman, S. (1981) ‘Directional selectivity and its use in early processing’, Proceedings of the Royal Society of London, B, 211: 151–180.

Mather, G. (2000) ‘Integration biases in the Ouchi and other visual illusions’, Perception, 29: 721–727.

Mon-Williams, M. and Wann, J. P. (1996) ‘An illusion that avoids focus’, Proceedings of the Royal Society of London, B, 263: 573–578.

Murakami, I., Kitaoka, A., and Ashida, H. (2006) ‘A positive correlation between fixation instability and the strength of illusory motion in a static display’, Vision Research, 46: 2421–2431.

Naor-Raz, G. and Sekuler, R. (2000) ‘Perceptual dimorphism in visual motion from stationary patterns’, Perception, 29: 325–335.

Otero-Millan, J., Macknik, S. L. and Martinez-Conde, S. (2012) ‘Microsaccades and blinks trigger illusory rotation in the “Rotating Snakes” Illusion’, Journal of Neuroscience, 32: 6043–6051.

Ouchi, H. (1977) Japanese Optical and Geometrical Art, New York, NY: Dover.

Pinna, B. and Brelstaff, G. J. (2000) ‘A new visual illusion of relative motion’, Vision Research, 40: 2091–2096.

Pinna, B. and Spillmann, L. (2002) ‘New illusions of sliding motion in depth’, Perception, 31: 1501–1502.

Pinna, B. and Spillmann, L. (2005) ‘New illusions of sliding motion in depth’, Perception, 34: 1441–1458.

Pirenne, M. H., Campbell, F. W., Robson, J. G. and MacKay, D. M. (1958) ‘Moving visual images produced by regular stationary patterns’, Nature, 181: 362–363.

Ruzzoli, M., Gori, S., Pavan, A., Pirulli, C., Marzi, C. A. and Miniussi, C. (2011) ‘The neural basis of the Enigma Illusion: A transcranial magnetic stimulation study’, Neuropsychologia, 49: 3648–3655.

Spillmann, L. (1997) ‘Colour in a larger perspective: The rebirth of Gestalt psychology’, Perception, 26: 1341–1352.

Spillmann, L. (1999) ‘From elements to perception: Local and global processing in visual neurons’, Perception, 28: 1461–1492.

Spillmann, L. (2006) ‘From perceptive fields to Gestalt’, Progress in Brain Research, 155: 67–92.

Spillmann, L. (2009) ‘Phenomenology and neurophysiological correlations: Two approaches to perception research’, Vision Research, 49: 1507–1521.

Spillmann, L. and Dresp, B. (1995) ‘Phenomena of illusory form: Can we bridge the gap between levels of explanation?’, Perception, 24: 1333–1364.

Spillmann, L., Heitger, F. and Schuller, S. (1986) ‘Apparent displacement and phase unlocking in checker-board patterns’, in Proceedings of the 9th European Conference on Visual Perception, Bad Nauheim.

Steinman, R. M., Pizlo, Z. and Pizlo, F. J. (2000) ‘Phi is not beta, and why Wertheimer’s discovery launched the Gestalt revolution’, Vision Research, 40: 2257–2264.

Troncoso, X. G., Macknik, S. L., Otero-Millan, J. and Martinez-Conde, S. (2008) ‘Microsaccades drive illusory motion in the Enigma Illusion’, Proceedings of the National Academy of Sciences ofthe USA, 105: 16033–16038.

Vezzani, S., Marino, B. F., and Giora, E. (2012) ‘An early history of the Gestalt factors of organisation’, Perception, 41: 148–167.

Wade, N. J. (2003) ‘Movements in art: From Rosso to Riley’, Perception, 32: 1029–1036.

Wagemans, J., Elder, J. H., Kubovy, M., Palmer, S. E., Peterson, M. A., Singh, M. and von der Heydt, R. (2012) ‘A century of Gestalt psychology in visual perception I. Perceptual grouping and figure-ground organization’, Psychological Bulletin, 138(6) : 1172–1217.

Wallach, H. (1935) ‘Über visuell wahrgenommene Bewegungsrichtung’, Psychologische Forschung, 20: 325–380; trans. S. Würger, R. Shapley and N. Rubin (1996) ‘On the visually perceived direction of motion by Hans Wallach: 60 years later’, Perception, 25: 1317–1367.

Wertheimer, M. (1912) ‘Experimentelle Studien über das Sehen von Bewegung’, Zeitschrift für Psychologie, 61: 161–265; Engl. trans. L. Spillmann and M. Wertheimer (eds) (2012) On Perceived Motion and Figural Organization, Cambridge, MA: MIT Press.

Yazdanbakhsh, A. and Gori, S. (2008) ‘A new psychophysical estimation of the receptive field size’, Neuroscience Letters, 438: 246–251.

Yazdanbakhsh, A. and Gori, S. (2011) ‘Mathematical analysis of the accordion grating illusion: A differential geometry approach to introduce the 3D aperture problem’, Neural Networks, 24: 1093–1101.

Zeki, S., Watson, J. D. and Frackowiak, R. S. (1993) ‘Going beyond the information given: The relation of illusory visual motion to brain activity’, Proceedings of the Royal Society of London, B, 252: 215–222.

Part III

Color vision and art perception

11  In search of neurophysiological correlates to color perception

Arne Valberg

Introduction

Throughout history there have been different approaches to pursuing a better understanding of color and human color vision, depending on the phenomena one wanted to explain. For more than 100 years, up to the end of the twentieth century, the opponent-color theory and the three-color theory of human color vision challenged each other’s validity. Both were based upon observations and experiments, although of different kinds. Here, we shall see how they may be reconciled. The Young-Helmholtz three-color theory has led to many controversies and misunderstandings due to confusion of languages that describe physical, biological, and psychological phenomena. There is an increasing awareness of the complicated relationships between stimuli, neural processes, and color perception, but still one can hear teachers claim that activation of the long wavelength sensitive L-cone receptors evokes a red percept, activation of M-cones a green color, and the activation of short wavelength sensitive S-cones leads to the perception of blue. In scientific circles, however, this ancient idea gave way to a more modern form in the three-receptor theory, where the three cone types are defined as linear light detectors with different spectral sensitivities, making trichromatic color matching and color technology so successful.

Neurophysiological investigations of color vision have made great advances since De Valois’ (1965) and Wiesel and Hubel’s (1966) first recordings from single neurons in the primate visual pathway. Current neuroscience has a good understanding of the three-receptor theory and its implications for opponency (Derrington et al., 1984; Lee et al., 1987, 1988; Valberg, 2009; Valberg and Seim, 2008). The opponent-color theory, however, still lacks the proper neural correlates; i.e. the processes that underlie the perception of the elementary colors yellow, blue, red, green, white, and black. One central part of the problem is how to distinguish between cells that respond to the physical stimulus (its wavelength distribution or, rather, the relative excitation of the three cone types) and those that may respond to specific colors (if such cell assemblies exist). Cells of the first category are found in the retina and in the parvocellular and koniocellular pathways from the retina, through the lateral geniculate nucleus (LGN) to visual cortex. It is still disputed whether the visual area V1 contains color specific cells or not (see review by Shapley and Hawken, 2011), or if color as an attribute has its own brain center (e.g. visual area human V4; Zeki, 1983, 1993).

A brief history of color vision

Newton and the spectrum

Isaac Newton (1642–1727) was primarily occupied with light and color phenomena in nature, like the rainbow, and he studied refraction, reflection, transmission, and the interference of light. He sought correspondence between colors and physical processes, and his experiments with prismatic colors, where white light was dispersed into a spectrum of colors, are probably the most important experiments in the history of color science (Kaiser and Boynton, 1996). Even though it was clear to Newton that the rays reflected from the surface of an object were not colored, he regarded the spectral composition of the same rays as the proper physical stimulus for seeing color. A white surface, for instance, reflected light that was a mixture of lights from the whole spectrum. Newton arranged the appearance of the spectrum into seven regions of different hues, or in seven sectors on a color circle.

The opponent-color theory

It is usually assumed that Leonardo da Vinci (1452–1519) was the first to point out that there are six simple colors in nature (daVinci, 1906/1925; Miescher et al., 1961). Today we arrange these simple or unique colors in two chromatic and one achromatic opponent pairs: yellow/blue, red/green, and black/white. A perceived pure unique yellow color is a color that appears neither reddish nor greenish. A similar criterion can be used for the other three unique hues. Da Vinci’s original observation was developed four centuries later into a chromatic four-color theory by the German physiologists Ewald Hering (1920) and W. Trendelenburg (1943), who originally associated the perceived attributes with photochemical assimilation and dissimilation in some visual substances in the eye. Hering, whose main concern was to explain color perception, appears to have recognized the distinction between receptor excitation and the process of sensation (perception). Apparently, he was well aware that his opponent color theory and a three-receptor theory could both be valid, but at different levels (Kaiser and Boynton, 1996). This would imply that one ‘… strictly distinguished between the process of excitation and the process of sensation’ (H. Aubert, see Hering, 1920: 47 and 1964/1920: 48).

However, the apparent hostility between Ewald Hering and Hermann von Helmholtz (Figures 11.1a and 11.1b) made reconciliation impossible (Howard, 1999).

Figure 11.1 The German scientists and antagonists (left) Ewald Hering and Hermann von Helmholtz (right). (Both pictures from Wikipedia Commons.)

In 1925 Erwin Schrödinger showed that simple, linear forms of the three- and four-color theories could, in principle, be compatible as different properties of the same linear three-dimensional vector space, and, as such, applicable to different levels of the visual system (Schrödinger, 1925). A simple cone opponency was developed from three cone excitation vectors (L, M, and S in our notation) simply by taking vector differences. One fundamental vector (representing one cone type) would excite the cell while another would inhibit its activity through subtraction. Thus, both trichromacy and cone opponency became, in principle, properties of a three-dimensional vector space. Schrödinger’s work suggested that opponency is formed at a higher neural level, but it can still be questioned if Hering’s qualitative color-opponency comes into existence at this same level. This problem has not been solved yet, and the neural processes behind unique hues are still an enigma.

In the 1950s this, and similar zone theories that combined trichromacy and opponency (see below), proved useful in explaining the outcome of the original quantitative psychophysical experiments carried out by the American psychologists Dorothea Jameson and Leo Hurvich (Hurvich and Jameson, 1955). Jameson and Hurvich revived Hering’s ideas through extensive hue cancellation experiments (see Hurvich, 1981). In their view, unique yellow, for instance, can be used as an expression for the brain being in an equilibrium state between a ‘red’ process and a ‘green’ process. This idea strikes one as being fundamentally different from the notion that yellow is the result of an additive mixture of red and green lights. The same reasoning would apply to the perception and color mixtures of white.

The three-color theory and the three-receptor theory

In 1802, about three centuries after Leonardo, the British physicist Thomas Young (1773–1829) proposed a three-component theory of color vision, with the primary colors red, green, and violet. All other colors could, in principle, be matched by an additive mixture of three primary colors. In accordance with the ideas of George Palmer, a merchant of colored glass (Lee, 1991; Mollon, 1995), these three colors were closely related to the excitation of three receptors in the retina. Today, the three types of human receptors are well known (Stockmann and Sharpe, 2000), and the wavelengths of maximum sensitivity are 570 nm, 545 nm, and 445 nm.

Famous scientists like Hermann Grassmann (1853), James Clerc Maxwell 1872 (1872/1970), and Hermann von Helmholtz (1860/1911) developed this theory further and provided a basis for color measurement, not without vehement opposition from Ewald Hering and his adherents, who preferred the opponent- or four-color theory to explain the underlying physiological processes. In view of Hering’s apparent acceptance of the distinction between excitation and sensation (see above), the controversy between Hering and Helmholtz would have seemed unnecessary, provided that Helmholtz had admitted this distinction. However, it appears that both Young and Helmholtz had aimed at an explanation of the qualitative aspect of colors (Farbempfindungen). Helmholtz had tried to establish a three-component mechanism that would be responsible for color qualities (qualia). This is clear from his reference to Young in statements like: ‘In the eye there are three types of nerve fibres. Stimulation of the first one excites the sensation (Farbempfindung) of red, stimulation of the second the sensation of green, stimulation of the third the sensation of violet’ (Helmholtz, 1860/1911: 119). And later he continues: ‘[the essence of Young’s hypothesis] is that the sensations (Empfindungen) of color are imagined as composed of three mutually and completely independent processes in the neural substrate’ (Helmholtz, 1860/1911: 120).

This quotation makes it clear that the Young-Helmholtz’ theory originally was a three-color theory, dealing with qualitative perception, and not a three-receptor theory like the one we adhere to today. It is not clear to what degree Maxwell agreed with this opinion (Sherman, 1981). His experience with color matches based on additive color mixtures led him to conclude that any set of three independent colors could serve as primaries for color measurement. He wrote

… the theory which I adopt assumes the existence of three elementary sensations by combination of which all the actual sensations of color are produced. I will show that it is not necessary to specify any given color as typical for these primary sensations. Young has called them red, green, and violet; but any of the three colors might have been chosen, provided that white resulted from their combination in proper proportions.

(Maxwell, 1856/1970: 63–64)

Helmholtz, on the other hand, seemed to have accepted Young’s choice of red, green, and violet, even though he commented that Young’s ‘choice of primaries had something arbitrary about them’(Helmholtz, 1860/1911: 121).

Trivariance seemed to be a fact, but the explanation given for it in the three-color theory was too simple. Confirmation of trivariance was not a sufficient support for the Young-Helmholtz three-color theory, and the confusion was large (Le Grand, 1968). Today there is agreement that one cannot connect qualitative color perception directly to excitation of the cones (except as correlates in strictly normalized and artificial conditions, for instance in the dark laboratory of the physicist). Colors do not reside in the cones, and one cannot say, like Helmholtz did, that a color deficient person who has no L-cones (a protanope) is red-blind and that he who has no M-cones (a deuteranope) is green-blind. We need a stricter version of the theory that separates trichromatic color matches from qualitative appearance. Thus, we limit ourselves to infer that equal excitations of the three receptor types imply equal color impressions (without asking which one). What was earlier a three-color theory thus became a three-receptor theory.

This principle of color matching lies behind the international XYZ system for measuring colors that was introduced by the CIE (Commission Internationale de l’Eclairage) in 1931. CIE recently developed a new physiological system for color matching that is based on the cone spectral sensitivities of Stockman and Sharpe (2000; see Valberg, 2005).

Around 1959 Edwin H. Land (1959) gave his spectacular demonstrations of approximate color constancy, and the scientific discussion took a new course. The three-color theory of Young-Helmholtz was attacked by Land, although this time from a different angle: his two-color projections had shown that a surprising rich gamut of colors could be produced in the projections by using only two colors. Deane B. Judd (1960) set things straight by pointing out that Land’s demonstrations were vivid illustrations of well-known facts about spatial color contrast (simultaneous contrast). The similarity of Land’s later retinex theory (Land, 1983) with the von Kries cone adaptation hypothesis (Valberg, 2005; Valberg and Lange-Malecki, 1991) more or less confirmed that a three-receptor theory was restricted to explaining color matches. Neither it, nor its predecessor, the three-color theory, could, as Helmholtz had believed, account for perceptual qualities of colors. It could not predict the phenomenon of simultaneous contrast, nor the color of an illuminated surface in a complex scene.

Zone theories

As we have seen, before neuroscience had developed tools to directly measure the activity of single cells in the retina and the visual pathway, and to derive the spectral sensitivities of cones themselves, there existed more or less vague concepts about a division of color vision into a retinal three-component process and a more central opponent process. These theories were based on observations and psychophysical experiments with relatively simple stimuli. Deane B. Judd (1951) provided an excellent summary of such zone theories in Steven’s Handbook of Experimental Psychology (1951). Of the many theories that tried to combine receptor level processing and later opponent processing, the theory of Müller (1930) stands out as the most ambitious. Like many of the other theories, Müller’s concept was an initial three-receptor stage and a final neural chromatic-opponent and white-black stage (Judd, 1951). However, unlike the other zone theories, Müller introduced a second retinal chromatic and an achromatic stage. He also distinguished between Hering’s color opponency and other opponent processes that were at play in adaptation and chromatic induction.

Already in the retina, there is a complex spectral, spatial, and temporal processing of the cone signals. The transduction ofinformation from the receptors to the retinal ganglion cells is modified by the properties of a network of many other retinal cell types, like bipolar, horizontal, and amacrine cells. Of these, the influence of amacrine cells on signal transmission is the least known. Moreover, it is difficult to relate to the fact that the large individual differences in the relative number of L-, M-, and S-cones that has been discovered (Roorda and Williams, 1999) do not seem to affect color discrimination to a major degree. It is, for example, claimed that the ratio nM: nL between the number, n, of M- and L-cones can vary between 1: 3 and 17:1 without a noticeable effect on color vision (Hofer et al., 2005).

Obviously, conscious perception resides neither in the photoreceptors, nor in the opponent cells of the lateral geniculate nucleus (LGN). However, constancy of several color percepts (such as a constant hue, a constant chroma, constant chromaticity, etc.) seems to correlate well with constancy in the relative responses of opponent neurons (Valberg, 2001).

Newton said that ‘the rays are not colored’ (MacAdam, 1970). With due respect to Helmholtz we add that ‘colors neither reside in the receptors nor in the cone-opponent cells of the retina and LGN’. However, the same perceptual attribute, for example a certain orange hue independent of purity, seems to correlate with a constant ratio of response magnitudes between cone-opponent parvocellular (PC) neurons. Constant color strength (chroma) in an equiluminous chrominance diagram correlates well with a constant vector length from the white point in a vector space based on neural responses (Valberg et al., 1986a).

Stimuli that have a luminance above the magnocellular (MC) threshold, but below the chromatic threshold of PC and koniocellular (KC) cells have a colorless, achromatic appearance, and it therefore was speculated that MC cells contribute to the perception of white (Hofer et al., 2005). However, their transient firing of impulses speaks against the possibility that they alone can perform the task.

As indicated by the word ‘bistratified’, S-cone activated cells have peculiar dendritic fields, different from those of the midget ganglion cells. Their dendritic field branches in two layers, corresponding to the ON and OFF layers of other retinal cells (Dacey and Lee, 1994). Until recently there were so few findings of cells with excitatory L/M input combined with inhibitory S-cone input that there was reasonable doubt that such cells could play a role in color coding, but this doubt was removed (Dacey and Packer, 2003; Tailby et al., 2008; Valberg et al., 1986b). Cells with excitatory L/M-cone input and inhibitory S-cone input discriminate well between stimuli distributed along a white–yellow dimension in color space (but not in the white–blue direction), contrary to PC cells that cannot distinguish between mid-spectral, 570 nm stimuli and achromatic lights of the same luminance. Cells with an excitatory S-cone input discriminate well between blue and white stimuli.

Receptive fields

The primate receptive and perceptive fields (Spillmann, 1971; Spillman and Dresp, 1995) that are associated with the dendritic fields of opponent and non-opponent MC cells are still of interest. These neural fields play an important role in coding of contrasts and contours, where border processes may spread over a large area (‘filling-in’; Jung, 1973; Pinna et al., 2000; Schober and Rentschler, 1979; and Figures 1.08, 1.11, 1.13, 1.14 in Valberg, 2005; see also Chapter 4 by Breitmeyer in this volume). However, up to recently, chromatic-specific cells that fill in have not been found (Friedman et al., 1999; Spillmann, 2001). Many experiments suggest that PC and MC cells have different roles in color vision. For instance, the response of MC cells mirrors psychophysical sensitivity to luminance contrast (as in heterochromatic flicker photometry) and to borders between chromatic areas. At equal luminance, the residual (or second harmonic) opponent |L – M| response of MC cells corresponds well with minimally distinct borders (MDB; Kaiser et al., 1990; Valberg et al., 1992). Correlates of chromatic dimensions and attributes, such as hue, saturation (color strength), lightness, and chromatic discrimination in general, are allocated to PC and KC systems.

The current debate is much concerned with the organization of cone opponency in simple concentric and double opponent cells. In the retina and LGN the classical view is one of a center and surround organization that for ON-center cells (also called Increment cells; Valberg, 2005) form a ‘Mexican hat’ spatial sensitivity profile, with an excitatory cone type in the center and another, inhibitory cone type in the surrounding annulus. This spatial sensitivity profile has commonly been modeled as a Difference of Gaussian (DOG) function. It appears, however, that from this model one would expect a stimulus of constant chromaticity to change its color as stimulus size changes, and for this reason power functions would be more accurate (Seim et al., 2011, 2012). Excitatory and inhibitory power functions would be in balance as stimulus size changes, and do not fall off so rapidly to zero as do Gaussian functions. This may explain certain findings of large inhibitory areas of suppressive, so-called extra classical receptive fields.

The relative size of center and surround seems to vary among cell types, and cells with S-cone inputs tend to have coextensive or overlapping excitatory and inhibitory sub-fields. Higher up, in area V1 of the brain, one frequently finds so-called double opponent receptive fields with one cone opponency in the center (e.g. L – M) and the opposite (M – L) in the classical surround (Shapley and Hawken, 2011).

The receptive fields of cells in V1 seem to differ from those in the LGN in that most cells respond to achromatic contours and contrasts, and to movement and orientation. They are relatively insensitive to changes in color. However, there is a higher concentration of color selective cells in the so-called ‘blobs’ that form the centers of orientation selective columns in V1. In agreement with Livingston and Hubel’s (1984) earlier findings, these blob cells seem to project to the thin stripes in area V2 (Sincich et al., 2007). The color selective cells of V1 have many more preferred hue directions compared to the few cardinal directions of the six cell types in the LGN (Conway, 2009; Solomon and Lennie, 2007; Valberg et al., 1986a; Wachtler et al., 2003). These properties are not easily explained if cortical receptive fields are constructed by convergence from the LGN.

Challenges

It is probably not too surprising that one cannot account for all aspects of color vision by a mechanistic bottom-up approach of information transfer from the retina, via LGN, to areas V1, V2, V4, and later. There exists no theory of how four unique hues can be constructed from cone-opponent cells on a lower level.

Is population coding a way out?

One alternative may be to continue Hering’s approach and search for neural correlates to specific colors (i.e. qualia; see Chapter 12 by Dresp-Langley and Reeves in this volume). If this approach should prove successful, one would have obtained something that is closer to Helmholtz’ wish than is reflected in the Young-Helmholtz three-color theory. A few studies claim to have proven correspondence between, for example, the sequence of hues in a color circle and a spatial organization of color-specific neurons in the thin stripes of area V2 (Xiao et al., 2003). Semir Zeki (1983) claimed that area V4 is specialized for color perception whereas newer investigations find color specific cells in an area somewhat larger than V4. Here, the color sensitive cells are arranged in ‘globs’, i.e. areas with a high density of color specific cells that can be viewed using cytochrome oxidase marking, in analogy to the color specific cells in the blobs of V1 (Livingstone and Hubel, 1984). The high chromatic related activity in the globs can also be registered under functional magnetic resonance imaging (fMRI; Conway, 2009). These glob cells seem to project further onto inferotemporal cortex (IT), to an area that is regarded by some as pivotal in the development of perceptive color categories (Komatsu, 1997; see also Gegenfurtner and Kiper, 2003).

We earlier presented convincing evidence that, for relatively large (4 × 5 deg²) fields, the combination of the cell types in our model (see above), is capable of describing many threshold data (Valberg and Lee, 1989) and constant color attributes (Valberg et al., 1986a, 1991). The model provided correlates to color changes, such as the Bezold-Brücke effect of hue shift with increasing luminance ratio, and the Abney phenomenon where the hue changes when white light is added to a stimulus of high colorimetric purity. It remains, however, to demonstrate beyond doubt the existence of correlations in all situations. Among other things, it needs to be shown that a ‘color cell’s’ response changes in parallel with the color perceived under simultaneous and successive color contrast. When one and the same color stimulus changes its color appearance (e.g. from red to green) under simultaneous contrast, the hypothetical red-responsive cell must stop firing and a green-responsive cell must take over (Spillmann, 2001). Such ‘color cells’ must, in addition, show conventional color constancy when the color of the illumination changes without having the expected effects on surface colors (as in Edwin Land’s retinex demonstration). We have yet to reconcile these facts with color coding in cells at a lower level.

Summary/conclusion

Color matches, color thresholds, and the laws of additive color mixture are empirical facts that can be satisfactorily summarized by the three-receptor hypothesis. Beyond color matching, we need also to deal with color perception, including the ordering and scaling of colors within different categories (Hering’s approach). Attributes like hue, saturation, and lightness/brightness, chromatic strength, white and black, are the properties in focus here, as well as the scaling of these attributes in color systems (Ostwald, Natural Colour System (NCS), Munsell, etc.). Scaling requires correlations between phenomenological properties of light and color and neural processes. In our search for neural correlates to color perception, we have developed a promising physiological model of low-level color vision. This model attempts to bridge the gap between matching in the three-receptor theory and perception in the opponent colors theory. The present paper sketches a broad background for this attempt. Further details about the model can be found in Valberg et al. (1986a).

References

Conway, B. R. (2009) ‘Color vision, cones, and color-coding in the cortex’, The Neuroscientist, 15: 274–290.

Dacey, M. D. and Lee, B. B. (1994) ‘The blue-ON opponent pathway in primate retina originates from a distinct bistratified ganglion cell type’, Nature, 367: 731–735.

Dacey, M. D. and Packer, O. S. (2003) ‘Color coding in the primate retina: diverse cell types and cone-specific circuitry’, Current Opinion in Neurobiology, 13: 421–427.

da Vinci, L. (1906) A Treatise on Painting, English translation by J. F. Rigaud and G. Bell London, new edition by Hetzfeldt, M. (1925).

Derrington, A. M., Krauskopf, J. and Lennie, P. (1984) ‘Chromatic mechanisms in lateral geniculate nucleus of macaque’, Journal of Physiology, 357: 241–265.

De Valois, R. (1965) ‘Analysis and coding of color vision in the primate visual system’, Cold Spring Harbour Symposia on Quantitative Biology, 30: 567–579.

Friedman, H. S., Zhou, H. and von der Heydt, R. (1999) ‘Monkeys perceive colour filling-in under steady fixation: behavioural demonstration in monkeys and humans’, Perception, 28: 1383–1395.

Gegenfurtner, K. R and Kiper, D. C. (2003) ‘Color vision’, Annual Review of Neuroscience, 26: 181–206.

Grassmann, H. (1853) ‘Zur Theorie der Farbmischung’, Poggendorffs Annalen Physik, 89: 69–84.

Hering, E. (1920) Grundzüge der Lehre vom Lichtsinn, Berlin: Springer, English translation by L. M. Hurvich and D. Jameson (1964) Outlines of a Theory of the Light Sense, Cambridge, MA: Harvard University Press.

Hofer, H., Singer, B. and Williams, D. R. (2005) ‘Different sensations from cones with the same photopigment’, Journal of Vision, 5: 444–454.

Howard, I. P. (1999) ‘The Helmholtz-Hering debate in retrospect’, Perception, 28: 1–8.

Hurvich, L. M. (1981) Color Vision, Sunderland, MA: Sinauer.

Hurvich, L. M. and Jameson, D. (1955) ‘Some quantitative aspects of opponent-colors theory. II. Brightness, saturation and hue in normal and dichromatic vision’, Journal of the Optical Society of America, 45: 602–616.

Judd, D.B. (1951) ‘Basic correlates of the visual stimulus’, Handbook of Experimental Psychology, S. S. Stevens (ed.), New York, NY: Wiley/Chapman and Hall, pp. 811–867.

Judd, D. B. (1960) ‘Appraisal of Land’s work on two-primary color perceptions’, Journal of the Optical Society of America, 50: 254–268.

Jung, R. (1973) ‘Visual perception and neurophysiology’, in R. Jung (ed.) Handbook of Sensory Physiology Vol. VII/3. Central Processing of Visual Information. Part A, Berlin, Germany: Springer.

Kaiser, P. K and Boynton, R. M. (1996) Human Color Vision, 2nd edn, Washington DC: Optical Society of America.

Kaiser, P., Lee, B. B., Martin, P. R. and Valberg, A. (1990) ‘The physiological basis of the minimally distinct border demonstrated in the ganglion cells of the macaque retina’, Journal of Physiology, 422: 153–183.

Komatsu, H. (1997) ‘Neural representation of color in the inferior temporal cortex of the macaque monkey’, in H. Sakata, A. Mikami and J. Fuster (eds), The Associative Cortex — Structure and Function, Amsterdam, The Netherlands: Harwood Acadademic Publishers.

Land, E. H. (1959) ‘Color vision and the natural image. Partl I and II’, Proceedings of the National Academy of Sciences of the USA, 45: 115–129 and 636–644.

Land, E. H. (1983) ‘Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image’, Proceedings of the National Academy of Sciences of the USA, 80: 5163–5169.

Lee, B. B. (1991) ‘Die Universkät Göttingen und die Entstehung der Farbenlehre’, MPG Spiegel, 3(91): 11–15.

Lee, B. B., Valberg, A., Tigwell D. A. and Tryti, J. (1987) ‘An account of responses of spectrally opponent neurons in macaque lateral geniculate nucleus to successive contrast’, Proceedings of the Royal Society of London Series B, 230: 293–314.

Lee, B. B., Martin, P. R. and Valberg, A. (1988) ‘The physiological basis of heterochromatic flicker photometry demonstrated in the ganglion cells of the macaque retina’, Journal of Physiology, 404: 323–347.

Le Grand, Y. (1968) Light, Color and Vision, London, UK: Chapman and Hall, p. 430.

Livingstone, M. S. and Hubel, D. H. (1984) ‘Anatomy and physiology of a color system in the primate visual cortex’, Journal of Neuroscience, 4: 309–356.

MacAdam, D. L. (ed.) (1970) Sources of Color Science, Cambridge, MA: The MIT Press.

Maxwell, J. C. (1856) ‘Theory of the perception of colors’, Transactions ofthe Royal Scottish Society of Arts, 4: 394–400, reprinted in D. L. MacAdam (ed.) (1970) Sources of Color Science, Cambridge, MA: MIT Press, pp. 63–64.

Maxwell, J. C. (1872) ‘On color vision’, Proceedings of the Royal Institution of Great Britain, 6: 260–271. Reprinted in D. L. MacAdam (ed.) (1970) Sources of Color Science, Cambridge, MA: MIT Press, pp. 75–83.

Miescher, K., Hofman, K.-D., Weisenhorn, P. and Früh, M. (1961) ‘Ueber das natürliche Farbsystem’, Die Farbe, 10: 115–144.

Mollon, J. D. (1995) ‘George Palmer (1740–1795): glass-seller, visual theorist and draper’, in The Theory of Colors and Vision, London, UK: Drapers Hall.

Müller, G. E. (1930) Ueber die Farbempfindungen, Psychophysische Untersuchungen, Leipzig, Germany: Barth.

Pinna, B., Brelstaff, G. J. and Spillmann, L. (2001) ‘Surface color boundaries: a new “watercolour” illusion’, Vision Research, 41: 2669–2676.

Roorda, A. and Williams, D. R. (1999) ‘The arrangement of three cone classes in the living human eye’, Nature, 397. 520–522.

Schober, H. and Rentschler, I. (1979) Das Bild als Schein der Wirklichkeit, München, Germany: Moos.

Schrödinger, E. (1925) ‘Ueber das Verhältnis der Vierfarben-zur Dreifarbentheorie’, Sitzungsberichte der Akademie der Wissenschaften, Wien Iia, 134: 471–490.

Seim, T., Valberg, A. and Lee, B. B. (2011) ‘Can power functions – not DoGs – describe receptive fields of cone–opponent cells?’, Meeting of International Color Vision Society (ICVS), Buskerud College, Kongsberg, Norway. Abstract book, p. 79.

Seim, T., Valberg, A. and Lee, B. B. (2012). ‘Visual signal processing in the macaque lateral geniculate nucleus’, Visual Neuroscience, 29, 105–117.

Shapley, R. and Hawken, M. J. (2011) ‘Color in the cortex: single and double-opponent cells’, Vision Research, 51: 701–717.

Sherman, P. D. (1981). Colour Vision in the Nineteenth Century, Bristol Adam Hilger Ltd.

Sincich, L. C., Jocson, C. M. and Horton, J. C. (2007) ‘Neurons in V1 patch columns project to V2 thin stripes’, Cerebral Cortex, 17: 935–941.

Solomon, S. G. and Lennie, P. (2007) ‘The machinery of color vision’, National Review in Neuroscience, 8: 276–286.

Spillmann, L. (1971) ‘Foveal receptive fields in the human visual system measured with simultaneous contrast in grids and bars’, Pflügers Archiv für die gesamte Physiologie, 326: 281–299.

Spillmann, L. (2001) ‘Gehirn und Gestalt. II. Neuronale Mechanismen’, Kognitionswissen-schaft, 9: 122–143.

Spillmann, L. and Dresp, B. (1995) ‘Phenomena of illusory form: can we bridge the gap between levels of explanation?’, Perception, 24: 1333–1364.

Stockman, A and Sharpe, L. T. (2000) ‘The spectral sensitivities of the middle- and long wavelength sensitive cones derived from measurements in observers of known genotype’, Vision Research, 40: 1711–1737.

Tailby, C., Solomon, S. G. and Lennie, P. (2008) ‘Functional asymmetries in visual pathways carrying S-cone signals in macaque’, Journal of Neuroscience, 28: 4078–4087.

Trendelenburg, W. (1943) Der Gesichtssinn, Berlin, Germany: Springer.

Valberg, A. (2001) ‘Unique hues: an old problem for a new generation’, Vision Research, 41: 1645–1657.

Valberg, A. (2005) Light Vision Color, Chichester: Wiley and Sons.

Valberg, A. (2009) ‘Color vision: different roads from perception to neuroscience’, Dean B. Judd Lecture, Procedings of the Association Internationale de la Color (AIC), Sydney, Australia.

Valberg, A. and Lange-Malecki, B. (1990). ‘“Colour constancy” in Mondrian patterns: partial cancellation of physical chromaticity shifts by simultaneous contrast’, Vision Research, 30: 371–380.

Valberg, A. and Lee, B. B. (1989) ‘Detection and discrimination of color, a comparison of physiological and psychophysical data’, Physica Scripta, 39: 178–186.

Valberg, A. and Seim, T. (2008) ‘Neural mechanisms of chromatic and achromatic vision’, Color Research and Application, 33: 433–443.

Valberg, A., Seim, T., Lee, B. B. and Tryti, J. (1986a) ‘Reconstruction of equidistant color space from responses of visual neurons of macaques’, Journal of the Optical Society of America, A3: 1726–1734.

Valberg, A., Lee, B. B. and Tigwell, D. A. (1986b) ‘Neurons with strong inhibitory S-cone inputs in the macaque lateral geniculate nucleus’, Vision Research, 26: 1061–1064.

Valberg, A, Lange-Malecki, B. and Seim, T. (1991) ‘Color changes as a function of luminance contrast’, Perception, 20: 655–668.

Valberg, A., Lee, B. B., Kaiser, P. and Kremers, J. (1992) ‘Responses of macaque ganglion cells to movement of chromatic borders’, Journal of Physiology, 458: 579–602.

von Helmholtz, H. (1860/1911) Handbuch der Physiologischen Optik, Vol. 2, 3rd edn, Hamburg, Germany: Voss.

Wachtler, T., Sejnowski, T. J. and Albright, T. D. (2003) ‘Representation of color stimuli in awake macaque primary visual cortex’, Neuron, 37: 681–691.

Wiesel, T. N. and Hubel, D. H. (1966) ‘Spatial and chromatic interactions in the lateral geniculate body of the rhesus monkey’, Journal of Neuroscience, 29: 1115–1156.

Xiao, Y., Wang, Y. I. and Fellman, D. J. (2003) ‘A spatially organized representation of color in macaque cortical area V2’, Nature, 42: 535–539.

Zeki, S. (1983) ‘The distribution of wavelength and orientation selective cells in different areas of the monkey visual cortex’, Proceedings Royal Society, London, 217: 449–470.

Zeki, S. (1993) A Vision of the Brain, Cambridge, MA: Blackwell Scientific.

12  Color and figure-ground

From signals to qualia

Birgitta Dresp-Langley and Adam Reeves

Introduction

The laws which predict how the perceptual quality of figure-ground can be extracted from the most elementary visual signals were discovered by the Gestaltists, and form an essential part of their movement (see especially Metzger, 1930, and Wertheimer, 1923 translated and re-edited by Lothar Spillmann, 2009 and 2012, respectively). Distinguishing figure from ground is a prerequisite for perception of both form and space (the relative positions, trajectories, and distances of objects in the visual field). The human brain has an astonishing capacity for selecting and combining a few critical visual signals, through complex mechanisms, to represent form and space accurately. The phenomenon of figure-ground in its most elementary form may be seen in the famous Ehrenstein illusion (Ehrenstein, 1941, studied extensively by Spillmann and colleagues, see Dresp-Langley, 2009, for a general review). In this illusion, four thin lines form a cross configuration with an empty centre (as in Figure 12.1).

Figure 12.1 Examples of the Ehrenstein illusion, which was extensively studied by Spillmann and his colleagues. A shape percept is induced by dark or bright converging lines, where a disk lying in front of the display is perceived.

If the converging lines are dark on a lighter background, the centre appears to fill in with a disk shape that appears even lighter than the background. With white inducing lines on a darker background, the centre appears to fill in with an even darker disk. These, typically asymmetrical, perceptual sensations of brightness or darkness enhancement are deemed illusory because they have no physical origin given that the luminance at the centre of the figure is identical to the background luminance. The illusion illustrates simultaneous contrast (Spillmann, 1977; Spillmann et al., 1984), where surfaces surrounded by regions of opposite contrast polarity appear to brighten or to darken, but its most salient phenomenal attribute is the apparent figural enhancement at the centre of the configuration, induced by the four converging lines (Spillmann et al., 1976). The figural enhancement, an opaque disk placed on what could be a cross lying underneath, involves both structural depth and figure-ground (Spillmann and Dresp, 1995; Watanabe et al., 1995), as the illusory centre appears to lie in a distinct depth plane from the background. The original Ehrenstein illusion results from orientation and luminance signals. With colored inducers, the resulting figure-ground percept suggests neon-color spreading (for a review see Spillmann and Dresp, 1995). Color signals can generate strong and self-sufficient cues to figure-ground, as for example in the watercolor illusion discovered by Pinna and colleagues (2001; see also Pinna and Reeves, 2006, and von der Heydt and Pierson, 2006), and in the related van Tuijl Illusion (1975). Interestingly, color signals associated or not with a luminance contrast may either strengthen or override geometric cues to figure-ground (Dresp et al., 2002; Guibal and Dresp, 2004). In these experiments, perceptually transparent or opaque crosses were presented, and observers judged whether the horizontal or vertical component appeared nearer.

Perceptual space from planar objects: the role of luminance contrast and geometric cues

Planar geometry provides powerful cues to depth, as shown in Figure 12.2 (in the Color plate section). Such cues include the relative height of stimuli in the visual field (top), structural interposition without occlusion suggesting transparent overlapping forms (middle), and cues to partial occlusion suggesting an opaque occluding object in front of another (bottom).

In experiments by Guibal and Dresp (2004), the brighter and darker levels of the colors red and green were equiluminant with regard to each other, their luminance contrast with the grey background varied. The luminance contrast of a given color, supported by one of the three geometric cues, determines the probability of a given figural element to be perceived as nearer to the human observer, or as figure versus ground, as in Figure 12.3a. Response times for ‘near’ decrease with increasing luminance contrast (Figure 12.3b), suggesting that contrast increases salience. Interactions between luminance contrast and geometric cues, as shown here, generate variations in the relative visibility of parts, where partial occlusion is a strong pictorial cue to figure-ground and may support a weaker luminance contrast. Interposition cues lose out against spatial position and partial occlusion in determining the probability of ‘near’ at all but the highest contrast.

Figure 12.3 Response probabilities of ‘near’ (top) and response latencies (bottom) as a function of the luminance contrast (Michelson) of the selected visual part, combined with one of the three different geometric cues to figure-ground illustrated in Figure 12.2 (after Dresp et al., 2002).

Why the relative visibility of a color determines pictorial depth is explained by the fact that contrast simulates the laws of aerial perspective (Ross, 1967). Schwartz and Sperling (1983), Egusa (1983), and Rohaly and Wilson (1999) showed that the perceived depth of a given region within the visual field is determined by the brightness of that region (as illustrated here in Figure 12.4). O’Shea and co-workers (1994) demonstrated, in binocular and monocular viewing, that the higher-contrast stimulus of a pair systematically appears nearer than the lower-contrast stimulus. Moreover, when a luminance cue was tested against a size cue for linear perspective, luminance prevailed over size. Schwartz and Sperling (1967) found that brighter stimuli had a stronger tendency to stand out in front even when perspective cues suggested the opposite. Luminance contrast determines which shape of a pair in a configuration will be seen as nearer (Dresp et al., 2002), and a sufficiently strong luminance cue may prevail over structural cues to depth such as interposition and partial occlusion.

Figure 12.4 Objects with stronger luminance contrast in the plane tend to be seen as nearer to the observer than objects with weaker luminance contrast. Here, the small bright square with the highest contrast is likely to be seen as nearer than the small grey square even though the latter is supported by multiple cooperative interposition cues.

Surface effects from color

Color has a particular status in perceptual organization (Oyama and Yamamura, 1967), partly because of the well-known phenomenon of color stereopsis (Brewster, 1851; Verhoeff, 1928). Color stereopsis refers to the effect of color contrast on the perceived depth of stimuli when they are viewed binocularly, through a convex lens or with the unaided eye, and when their luminance contrast does not vary. Experiments on color stereopsis have revealed a difference between long-wavelength stimuli such as red or yellow and short-wavelength stimuli such as blue or green when participants are asked to judge which color appears closer (Brewster, 1851). When viewed binocularly, with the aided or unaided eye, reds and yellows typically appear closer than blues or greens. Whether this suggests an intrinsically competitive nature of colors, predicted by their wavelength and possibly under the influence of other cues such as luminance contrast or stimulus geometry, remains to be clarified. Observations by Dengler and Nitschke (1993) have shown that changes in contrast at the borders of isoluminant orange and blue lines or squares may induce depth reversals. When these colors were lighter than the background, the long-wavelength color orange was seen in front of the short-wavelength color blue, but when the colors were darker than the background, blue was seen in front. The relative weight of the contribution of color stereopsis to processes of perceptual organization has remained unclear. When stereopsis is not at issue, color is still a critical factor, and the most valuable observations suggesting why this may be the case have been reported almost two centuries ago by the French chemist Michel-Eugène Chevreul.

Chevreul’s laws

The idea that color is a powerful cue to figure-ground organization preceded the Gestalt movement and color psychophysics by almost half a century. In 1839, well before Katz published his observations on color experience (1911), and about 12 years before color stereopsis was discovered by Brewster (1851), Chevreul published his observations on the perceptual modifications produced by the mutual proximity of colors (De la loi du contraste simultané des couleurs et de l’assortiment des objets colorés, Chevreul, 1839). He defined what later has become known as simultaneous color contrast (e.g. Beck, 1966; De Weert and Spillmann, 1995; Gerrits and Vendrik, 1970; Heinemann, 1955; Shapley and Reid, 1985). Observing how colored surfaces placed side by side or surrounding each other change in appearance according to which color is put next to which other, Chevreul suggested how they needed to be displayed in space to produce specific effects on the perception of a human observer. His laws of color and contrast have provided valuable intuitions about the effect of color on perceptual organization, inspiring artists, architects, designers, and visual scientists ever since.

Law of ‘true’ color

Chevreul’s laws are important with regard to perceptual qualities such as color saliency and perceptual organization. One such law is concerned with the conditions necessary for the perception of unblemished, true colors, predicting that a color presented on grey will truly appear to the observer as that particular color, producing no mutual interactions with the background that alter the appearance of either. This law is based on Chevreul’s observation that all primary colors appear ‘pure’ and ‘true’ by the proximity of grey while black and white affect a color’s brightness and the perception of its tone. Psychophysical data have shown since that placing a color near other so-called inducing colors changes the appearance in either brightness or tint of the test color often dramatically (e.g. Livitz et al., 2011). This phenomenon may be reflected by a contrast effect, where the perceived brightness or tone of the test color changes away from that of the inducing color, or by an assimilation effect, where the perceived brightness or tone of the test color changes toward that of the inducing color. Interaction of a similar kind occurs between achromatic stimuli of positive and negative contrast polarities, producing either contrast, where a bright surface makes an adjacent one look darker and a dark one makes an adjacent one look brighter, or assimilation, where a bright surface makes an adjacent one look brighter and a dark surface makes an adjacent one look darker (e.g. Beck, 1966; Hamada, 1985; Heinemann, 1955). Chevreul’s law of true color has never been challenged by psychophysics, and mutual interactions where grey fields change the appearance of nearby or surrounding colors, or colors change the appearance of nearby or surrounding grey fields, have only very recently been investigated (Dresp-Langley and Reeves, 2012).

Law of contrast

A color will be seen as figure rather than as ground when the difference in luminance between the color and its background is the strongest. This was first predicted by Chevreul’s law of contrast and, more than a century later, confirmed by the psychophysical studies referred to above, showing that surfaces with the stronger luminance contrast tend to be perceived as figure rather than as ground, or as nearer to the human observer than surfaces with the weaker luminance contrast. The effect of frames on the appearance of tones and their brightness, or ‘the difference between the effect of a framed picture and the effect of that same picture when seen through an opening’ was considered by Chevreul (1839) to be of crucial importance to figure-ground relations in the visual field of the human observer. What he called the ‘contiguity of the frame’ is claimed to alter any of the perceptual effects produced by any of his laws under conditions where no frame is present. This intuition that distinct object borders influence our perception is consistent with studies showing interactions between color appearance and the spatial profile of surface contours and geometric configuration (Devinck et al., 2006; De Weert and Spillmann, 1995; Dresp and Fischer, 2001; Pinna, 2008; 2011; Pinna and Reeves, 2006).

A compelling phenomenon revealing the simultaneous contrast effects of color patterns on achromatic backgrounds is shown in Figure 12.5 (in the Color plate section). The colored squares, darker on one side of the display and brighter on the other, are placed on spatially separated grey fields of homogenous intensity and presented on a dark (black) screen. The colors’ immediate grey background is systematically seen as brighter on one side. The induction can switch from contrast, where the grey field containing the darker inducers appears brighter, to assimilation, where the grey field containing the brighter inducers appears brighter. The effects strongly depend on the geometric configuration of the backgrounds. Surprisingly little is still known about the perceptual interactions between color and achromatic backgrounds. In complex configurations of color on grey, figure-ground percepts are not influenced by lighting conditions or visual adaptation levels (Dresp-Langley and Reeves, 2012). Simonet and Campbell (1990) reported effects of illumination levels on color stereopsis suggesting a contribution to figure-ground analysis at early stages of visual processing (see also Dengler and Nitschke, 1993). However, color stereopsis does not predict how the colors will interact with geometric cues in perceptual organization. Guibal and Dresp (2004) found that partial occlusion combined with the colors green or white produced weaker depth percepts and slower perceptual judgments than partial occlusion combined with the color red, regardless of background intensity. At the same time, interposition cues combined with the color red produce stronger depth percepts and faster perceptual judgments than partial occlusion combined with the colors green or white. This is an important finding because it shows that a strong primary color such as red can override the considerable boundary advantage of configurations with partial occlusion. This effect is not predicted by any theory we know of and highlights the particular status of the color red as a cue to depth. It may be related to the finding that desaturated red targets are found more rapidly than appropriately equated targets of other colors (Lindsey et al., 2010), although such a difference is not always found (Santhi and Reeves, 2004). However, red stimuli are not inevitably seen as nearer in the visual field than stimuli with other colors. Also, the highest probabilities of near, and the fastest perceptual judgments, are produced by red figures combined with a partial occlusion cue or a positional cue presented on a light background. Though this suggests that the color red is a potentially strong candidate as a depth cue with, initially, a strong relative weight, white figures combined with a partial occlusion cue produce stronger depth percepts and faster perceptual judgments than red figures combined with an interposition cue presented on a dark background. This result clarifies that, despite an advantage due to wavelength characteristics promoting color stereopsis, red does not always benefit from this advantage, nor does it automatically attract visual attention away from other stimuli in the visual field. It may only do so when certain geometric constraints are met. Abstract expressionists intuitively exploit such color interactions to suggest shapes and figure-ground, as shown in Figure 12.6 (in the Color plate section).

How does the brain compute figure-ground?

Finding out about interactions between the different monocular cues to figure and ground, or how they may cooperate or compete in a given configuration, is critical to our understanding of how the brain extracts structural information from a complex image or visual scene. Earlier theories (e.g. Gibson, 1950) emphasized the importance of cooperative interactions, or cue combinations, claiming that information provided by multiple cues would have to be combined to generate unified percepts of shapes and their relative depth within the visual field. Cue combination, which suggests cooperation between cues, would occur when two qualitatively different depth cues contribute to the depth percept at a given location. Conflict, or competition between cues, would occur in situations where an unambiguous cue fails to strengthen an ambiguous one. Although all cues available in a given configuration cooperate in the first instance, cues with little relative weight may then be overridden by a cue with stronger relative weight. Obviously, the brain proceeds to a selection of the most likely signals, which, in a first instance, trigger what is commonly called ‘attention’.

Signal selection to trigger visual attention

It has been suggested that the probability based selection of neural signals may drive perceptual organization by telling the brain to perform the most likely structural analysis of visual input (e.g. Dresp and Langley, 2005). At some stage in this process, attention becomes important as some objects readily attract attention away from others in the visual field. In this process, objects with a stronger luminance contrast may benefit from selection for attention when presented together with objects of a lesser contrast (see again Figure 12.4). Similarly, some colors may attract attention away from others in the visual field (e.g. Dresp and Grossberg, 1999; Yantis and Jones, 1991) and, as explained above, the color red is often readily seen as nearer than others, not only on the basis of color stereopsis, but in many complex visual scenes where multiple factors determine perceptual organization.

Nakayama and Silverman (1986) showed that the visual detection of targets defined by combinations of color and depth in a visual search task is faster than the detection of targets defined by a single attribute. The implications of a correlation between perceived depth in geometric configurations and neural selection for attention were discussed further by Nakayama et al. (1990). If such a correlation exists, the probability that a given object is seen to be nearer than another object in the visual field should indeed correlate with the response latencies of near percepts, as suggested by the data summarized here in Figures 12.3a and b. In particular, under normal viewing conditions where luminance contrast, color contrast, stimulus geometry, and visual attention interact, the cue combinations that yield the most likely depth percept should be more salient, thereby promoting selective visual attention. Then, a strong luminance or color cue combined with a strong geometric cue should produce shorter response latencies than a weaker luminance or color cue combined with a weaker geometric cue, as indeed confirmed by the data of Guibal and Dresp (2004).

From contours in the plane to surfaces in depth: Grossberg’s FACADE theory

Grossberg’s FACADE (Feature-And-Contour-And-DEpth) model (1997) is a neural model of perceptual organization, with two subsystems (see also Chapter 4 by Breitmeyer in this volume). The Boundary Contour System (BCS) is orientation selective and detects boundaries. The BCS interacts with the Feature Contour System (FCS) of the model, which detects luminance contrasts within image regions defined by boundaries. The combined monocular outputs of BCS and FCS cooperatively feed into a selective BCS–FCS mechanism that groups input from a given location in the visual field. Visual configurations generating a so-called relative boundary advantage yield stronger figure-ground percepts because they generate comparatively stronger signals in the BCS. Functional asymmetries in the processing of brighter versus darker contrasts (Dengler and Nitschke, 1993; De Weert and Spillmann, 1995; Dresp and Langley, 2005; Dresp-Langley and Reeves, 2012; Magnussen and Glad, 1975) are accounted for by the model. The output of BCS–FCS interaction determines whether a stimulus in a given location survives against stimuli in other locations. Bottom-up representations surviving this monocular, competitive stage of the FACADE model are selected and integrated into a binocular form representation. Figure-ground representations are generated by non-linear, long-range interactions (Spillmann and Werner, 1996), which ensure that changes in lighting conditions or visual adaptation will not hinder object perception. The FACADE predictions are consistent with observations that strong achromatic luminance contrasts, although they may cooperate with any given geometric cue at weaker intensities, generate competitive interactions with other cues, especially with interposition. Partial occlusion survives the competition against a strong luminance cue significantly better than an interposition cue, although both resolve ambiguities about which object is likely to lie in front of which other in the plane (Dresp et al., 2002; Guibal and Dresp, 2004). Why partial occlusion wins against interposition in competition with a strong luminance cue is explained by the relative boundary advantage of partial occlusion cue, with a larger amount of visible boundary information provided by the figure lying in front. This pictorial depth cue was first mentioned by Leonardo da Vinci in his Trattato della Pittura (1651). In scenes representing objects of the same size that do not overlap, the sole cue to depth is relative location. FACADE theory predicts that neural expectation signals generated by information of relative spatial location are powerful attractors of visual attention (Grossberg, 1997), and that the perceptual system readily expects objects presented at lower locations in the visual field to be nearer to the human eye because they yield higher probabilities to be immediately relevant. Unlike partial occlusion, a positional cue does not generate a relative boundary advantage, but the figure-ground solution is computed faster than through other geometric cues to depth, which require further structural analyses. Also, several cues combined in a given configuration do not produce simple summation of figure-ground probabilities. This is consistent with cue-mixture models where different probabilistic weights are given to different depth cues, and where the weight of a given cue is conditional upon the relative weight of others (Knill, 2003). Although all cues available in a given configuration cooperate in the first instance, cues with little relative weight may subsequently be overridden by a cue with a stronger relative weight. Unlike the FACADE theory, cue mixture models do not attempt to link figure-ground perception to cortical mechanisms. They use Bayesian principles for computing conditional probability functions, which are the product of the likelihood functions for each given cue. FACADE proposes a biologically more plausible solution where figure-ground representation results from dynamic processes of cooperation and competition between neural (‘expectation’) signals in cortical pathways. The major postulates from this theory have proven to be consistent with neurophysiological data (e.g. Spillmann and Werner, 1996; von der Heydt and Peterhans, 1989).

Summary/conclusion

The pioneering work by Lothar Spillmann and his colleagues has inspired original scientific research into visual phenomena such as the Ehrenstein illusion, the watercolor illusion, and similar configurations. This has deepened our understanding of the processes of perceptual organization which allow the human brain to extract qualia such as brilliancy, figure-ground, and visual object integrity from the most elementary signals of contrast and shape displayed in two-dimensional space. This has opened doors to new and exciting investigations into the effects of color, and their interaction with geometric properties, on our perception of a visual world that is rapidly evolving and increasing in complexity. Modern art, architecture, and conceptual design, where the idea of a colorful world with a freeform geometry now largely dominates over the sober traditional approaches (see also Pinna, 2011), is about to challenge our perceptual system in ways we have not even begun to imagine. Understanding how we adapt to such constantly changing visual environments will no doubt motivate new lines of research aimed at linking perception to the workings of the human brain.

Acknowledgement

Financial support from Centre National de la Recherche Scientifique (CNRS-PICS05971) is gratefully acknowledged.

References

Beck, J. (1966) ‘Contrast and assimilation in lightness judgments’, Perception & Psychophysics, 1: 342–344.

Brewster, D. (1851) ‘Notice of a chromatic stereoscope’, Philosophical Magazine, 4th Series, 3: 31.

Chevreul, M. E. (1839) De la loi du contraste simultané des couleurs et de l’assortiment des objets colorés, English trans. J. Spanton (1857) Laws of Contrast of Color, London: Routledge.

Da Vinci, L. (1651) Trattato della Pittura di Leonardo da Vinci. Scritta da Raffaelle du Fresne, Paris, France: Langlois.

Dengler, M. and Nitschke, W. (1993) ‘Color stereopsis: A model for depth reversals based on border contrast’, Perception & Psychophysics, 53: 150–156.

Devinck, F., Spillmann, L. and Werner, J. S. (2006) ‘Spatial profile of contours inducing long-range color assimilation’, Visual Neuroscience, 23: 573–577.

De Weert, C. M. M. and Spillmann, L. (1995) ‘Assimilation: Asymmetry between brightness and darkness’, Vision Research, 35: 1413–1419.

Dresp, B. and Grossberg, S. (1999) ‘Spatial facilitation by color and luminance edges: Boundary, surface, and attention factors’, Vision Research, 39: 3431–3443.

Dresp, B. and Fischer, S. (2001) ‘Asymmetrical contrast effects induced by luminance and color configurations’, Perception & Psychophysics, 63: 1262–1270.

Dresp, B. and Langley, O. K. (2005) ‘Long-range spatial integration across contrast signs: A probabilistic mechanism’, Vision Research, 45: 275–284.

Dresp, B., Durand, S. and Grossberg, S. (2002) ‘Depth perception from pairs of overlapping cues in pictorial displays’, Spatial Vision, 15: 255–276.

Dresp-Langley, B. (2009) ‘The Ehrenstein Illusion’, Scholarpedia, 4: 53–64.

Dresp-Langley, B. and Reeves, A. (2012) ‘Simultaneous contrast and apparent depth from true colors on grey: Chevreul revisited’, Seeing & Perceiving, 25(6): 597–618.

Ehrenstein, W. (1941) ‘Über Abwandlungen der L. Herrmannschen Helligkeitserscheinung’, Zeitschrift für Psychologies 150: 83–91.

Egusa, H. (1983) ‘Effects of brightness, hue, and saturation on perceived depth between adjacent regions in the visual field’, Perception, 12: 167–175.

Gerrits, H. J. M. and Vendrik, A. J. (1970) ‘Simultaneous contrast, filling-in process, and information processing in mans visual system’, Experimental Brain Research, 11: 411–430.

Gibson, J. J. (1950) The Perception of the Visual World, Boston, MA: Houghton Mifflin.

Grossberg, S. (1997) ‘Cortical dynamics of three-dimensional figure-ground perception of two-dimensional figures’, Psychological Review, 104: 618–658.

Guibal, C. R. C. and Dresp, B. (2004) ‘Interaction of color and geometric cues in depth perception: When does red mean near’, Psychological Research, 10: 167–178.

Hamada, J. (1985) ‘Asymmetric lightness cancellation in Craik-O’Brien patterns of negative and positive contrast’, Biological Cybernetics, 52: 117–122.

Heinemann, E. G. (1955) ‘Simultaneous brightness induction as a function of inducing and test-field luminance’, Journal of Experimental Psychology, 50: 89–96.

Katz, D. (1911) ‘Die Erscheinungsweisen der Farben und ihre Beeinflussung durch die individuelle Erfahrung’, Zeitschrift für Psychologie, 7: 6–31.

Knill, D. (2003) ‘Mixture models and the probabilistic structure of depth cues’, Vision Research, 43, 831–854.

Lindsey, D. T., Brown, A. M., Reijnen, E., Rich, A. N., Kuzmova, Y. I. and Wolfe, J. M. (2010) ‘Color channels, not color appearance or color categories, guide visual search for desaturated color targets’, Psychological Science, 21: 1–21.

Livitz, G., Yazdanbakhsh, A., Eskew, R. T. Jr. and Mingolla, E. (2011) ‘Perceiving opponent hues in color induction displays’, Seeing & Perceiving, 24: 1–17.

Magnussen, S. and Glad, A. (1975) ‘Brightness and darkness enhancement during flicker: Perceptual correlates of neuronal B- and D-systems in human vision’, Experimental Brain Research, 22: 399–413.

Metzger, W. (1930) Gesetze des Sehens, English trans. L. Spillmann (2009) Laws of Seeing Cambridge, MA: MIT Press.

Nakayama, K. and Silverman, G. H. (1986) ‘Serial and parallel processing of visual feature conjunctions’, Nature, 320: 264–265.

Nakayama, K., Shimojo, S. and Ramachandran, V. S. (1990) ‘Transparency, relation to depth, subjective contours, luminance and neon color spreading’, Perception, 19: 497–513.

O’Shea, R. P., Blackburn, S. G. and Ono, H. (1994) ‘Contrast as a depth cue’, Vision Research, 34: 1595–1604.

Oyama, T. and Yamamura, T. (1967) ‘The effects of hue and brightness on depth perception in normal and color-blind subjects’, Psychologia, 3: 191–194.

Pinna, B. (ed.) (2008) Color, Line and Space: The Neuroscience of Spatio-Chromatic Vision, Leiden, The Netherlands: Brill.

Pinna, B. (2011) ‘The organization of shape and color in vision and art’, Frontiers in Human Neuroscience, 5: A 104.

Pinna, B. and Reeves, A. (2006) ‘Lighting, backlighting and the laws of figurality in the watercolor illusion’, Spatial Vision, 19: 341–373.

Pinna, B., Brelstaff, G. and Spillmann, L. (2001) ‘Surface color from boundaries: A new “watercolor” illusion’, Vision Research, 41: 2669–2676.

Rohaly, A. M. and Wilson, H. R. (1999) ‘The effects of contrast on perceived depth and depth discrimination’, Vision Research, 39: 9–18.

Ross, H. E. (1967) ‘Water, fog, and the size distance invariance hypotheses’, British Journal of Psychology, 58: 301–313.

Santhi, N. and Reeves, A. (2004) ‘The roles of distractor noise and target certainty in search: A signal detection model’, Vision Research, 44: 1235–1256.

Schwartz, B. J. and Sperling, G. (1983) ‘Luminance controls the perceived 3D structure of dynamic 2D displays’, Bulletin of the Psychonomic Society, 17: 456–458.

Shapley, R. and Reid, R. C. (1985) ‘Contrast and assimilation in the perception of brightness’, Proceedings of the National Academy of Science, 82: 5983–5986.

Simonet, P. and Campbell, M. C. W. (1990) ‘Effect of luminance on the directions of chromatostereopsis and transverse chromatic aberration observed with natural pupils’, Ophthalmic & Physiological Optics, 10: 271–279.

Spillmann, L. (1977) ‘Contrast and brightness illusions’, in H. Spekreijse and L. H. van der Tweel (eds) Spatial Contrast, Amsterdam, The Netherlands: North-Holland, pp. 45–49.

Spillmann, L. and Dresp, B. (1995) ‘Phenomena of illusory form: Can we bridge between levels of explanation?’, Perception, 24: 1333–1364.

Spillmann, L. and Werner, J. S. (1996) ‘Long-range interactions in visual perception’, Trends in Neurosciences, 19: 428–434.

Spillmann, L., Fuld, K. and Gerrits, H. J. M. (1976) ‘Brightness contrast in the Ehrenstein illusion’, Vision Research, 16: 713–719.

Spillmann, L., Fuld, K. and Neumeyer, C. (1984) ‘Brightness matching, brightness cancellation, and increment threshold in the Ehrenstein illusion’, Perception, 13: 513–520.

Van Tuijl, H. F. J. M. (1975) ‘A new visual illusion: Neonlike color spreading and complementary color induction between subjective contours’, Acta Psychologica, 39: 441–445.

Verhoeff, F. H. (1928) ‘An optical illusion due to chromatic aberration’, American Journal of Ophthalmology, 11: 898–900.

Von der Heydt, R. and Peterhans, E. (1989) ‘Ehrenstein and Zöllner illusions in a neuronal theory of contour processing’, in J. J. Kulikowski, C. M. Dickinson and I. J. Murray (eds) Seeing contour and color, Oxford, UK: Pergamon Press, pp. 729–734.

Von der Heydt, R. and Pierson, R. (2006) ‘Dissociation of color and figure-ground effects in the watercolor illusion’, Spatial Vision, 19: 323–340.

Watanabe, T., Nanez, J. F. and Moreno, M. A. (1995) ‘Depth release of illusory contour shape in the Ehrenstein grid’, Vision Research, 35: 2845–2851.

Wertheimer, M. (1923) Perceived Motion and Figural Organization, English trans. L. Spillmann, M. Wertheimer, K. W. Watkins, S. Lehar and V. Sarris (2012) Cambridge, MA: MIT Press.

Yantis, S. and Jones, E. (1991) ‘Mechanisms of attentional selection: Temporally modulated priority tags’, Perception & Psychophysics, 50: 166–178.

13  Chromatic assimilation in visual art and perception

Frédéric Devinck, Baingio Pinna and John S. Werner

Introduction

In natural viewing, nearly all colors that we experience are seen within a context of surrounding light and nearby objects, and this context affects color appearance. The change in color appearance caused by surrounding light is called ‘chromatic induction’. There are two types of induction: chromatic contrast and chromatic assimilation. Chromatic contrast is the shift in appearance of the test field away from the chromaticity of the inducing (surrounding) color. For example, a light that appears yellow in isolation will appear greenish-yellow when surrounded by a light that appears red. Chromatic assimilation is the opposite phenomenon, in which the appearance of a test field shifts toward the chromaticity of the inducing color. For example, a grid that appears blue may shift the appearance of an orange background toward the appearance of the grid (e.g. bluish). Chromatic contrast has dominated the vision literature on induction, while chromatic assimilation has received less attention. However, assimilation probably occurs at least as frequently as contrast. De Valois and De Valois (1988) stated that assimilation is by far the more important phenomenon because assimilation is the rule in color interactions while chromatic contrast is relatively rare. De Weert (1991) made clear how assimilation can influence perception in our natural environment such as when the perceived color of bricks depends on whether they are observed with or without mortar.

This review begins with a discussion of how assimilation has played multiple roles in art. Particular attention is directed to paintings in which assimilation is exploited. The second part reviews psychophysical studies of short-range assimilation. Finally, the last section describes long-range assimilation as exemplified by the watercolor effect.

Assimilation in art

The appearance of a color is influenced by other colors in the visual scene. Most artists have understood this and have exploited chromatic induction to obtain different visual effects in their painting. Claude Monet once said:

For me, a landscape does not exist in its own right, since its appearance changes at every moment; but its surroundings bring it to life – the air and the light which vary continually … For me, it is only the surrounding atmosphere which gives objects their true value.

(Barnes, 1990: 36)

The systematic use of chromatic induction shows that visual artists are often aware of coloration effects based upon phenomenological observations, but in some cases based on explicit rules similar to those of vision scientists.

Studies by Chevreul (1839) showed how color induces its complement in adjacent areas (see also the preceding chapter by Dresp-Langley and Reeves). These ideas were used by Eugène Delacroix, who influenced the works of the Impressionists. In his painting Femmes d’Alger dans leur appartement, shown in Figure 13.1 (in the Color plate section), the light passing through the window (on the left side) is made vivid by the use of saturated pigments. Delacroix also used hatching to produce assimilation, wherein the colors tended to blend together with surrounding colors.

In the 8th Impressionists Exhibition in 1866, Georges Seurat formalized the approaches of Chevreul, Delacroix and others through the technique of Pointillism. Small dots of pure color were applied systematically to the canvas in order to exploit numerous effects of light and color (Ratliff, 1992). The results depended on the size of the dots and the distance of the observer. When the dots were too small to be resolved at the viewing distance, additive color mixture (mélange optique) would occur. The result of this technique is displayed in the painting La Parade, 1889, shown in Figures 13.2a and 13.2b (in the Color plate section). A section of this painting is enlarged below to show the colors used by Seurat. At closer distances, a contrast effect may occur, especially if the dots are of complementary color. Assimilation effects will dominate at intermediate distance. Clearly the assimilation effect is dependent upon spatial scale, and presumed to be dependent on the size of the underlying receptive fields (see the next section of this chapter for more details).

Richard Anuszkiewicz, a contemporary painter, used assimilation to generate some colors from a limited palette. Figure 13.3a (in the Color plate section) shows an example that he based on a geometric composition. In this example entitled Convexity (top), the geometric form looks like a grid alternating in different colors presented on a red background. Thus, the red background seems to blend with the colors of the stripes. Note that the color assimilation persists at different viewing distances. Roy Lichtenstein, one of the most important contributors to American Pop Art, also used this coloration effect. Figure 13.3b (bottom) presents one of his paintings named Explosion (1965–1966). In this example, the white background takes on the faint coloration of the surrounding dots (e.g. red or blue). Assimilation effects are also evident in the Op Art of Bridget Riley. An example is displayed in Figure 13.4 (Hesitate, 1964). Here, the white background shows an achromatic shift toward the dots and disappears on close viewing.

Figure 13.4 Bridget Riley Hesitate, 1964. Put your book at farther distance. What do you see? Most observers report that the white background with white dots (along the diagonal on the left side) appears lighter that the white background with dark dots. ((c) Tate, London, 2012 and (c) Bridget Riley. All right reserved; courtesy Karsten Shubert, London.)

What do we know about chromatic assimilation?

Assimilation effects have been obtained with different kinds of patterns since the pioneering studies of von Bezold (1876) and subsequent vision scientists (e.g. De Valois and De Valois, 1988; de Weert and Spillmann, 1995; Fach and Sharpe, 1986; Helson, 1963; Monnier and Shevell, 2003; Pinna et al., 2001). Quantitative data were obtained at least 50 years ago for brightness assimilation, while data in the color domain were reported only recently. All these works indicate that the geometry of the stimulus display is the most important factor in determining the induction direction. While contrast occurs for simple displays (e.g. test stimuli in larger surrounds), assimilation is found for complex displays, especially those with repetitive patterns (Wyszecki, 1986). These results demonstrate that assimilation is strongly dependent on spatial parameters. Consequently, one may expect different amounts of assimilation for each. Here, we will discuss some common parameters that have been quantified.

Contour parameters producing coloration effects have been recognized from some previous work and the importance of these changes has been emphasized. Helson (1963) reported quantitative measurements of assimilation for an achromatic square-wave grating. His results showed that assimilation and contrast depend on stripe width. For narrow gratings, white or black bars were assimilated onto a grey background, so that the background looks lighter with the white grating and vice versa with the dark grating. With increasing stripe width, the assimilation effect changes into contrast. Similar results were found by Burnham (1953) by varying the viewing distance. Helson and Rohles (1959) studied the effects of varying the distance between lines inducing a lightness assimilation phenomenon. Observers reported that as the width of the gray background between the white or dark lines increases, assimilation decreases. Data from Reid and Shapley (1988) showed also that with increasing stimulus width, the strength of assimilation decreased (see also De Weert and Spillmann, 1995). Thus, the effect in the achromatic domain is strongest with narrow inducers as well as induction fields.

Assimilation shifts can also be found in chromatic gratings. Fach and Sharpe (1986) measured hue shifts for an isoluminant grating whose width varied from 2 to 20 arcmin. A color-matching task was used with Munsell papers. Similar results to those shown above were obtained. Greater hue shifts occurred for the narrower stripe widths and disappeared between 4.5 and 7.5 arcmin. These shifts were dependent on the color grating combination with small hue changes for yellow in all combinations. Smith et al. (2001) measured color induction for test and inducing chromaticities presented in spatial square-wave alternation, with four spatial frequencies specified in a cone chromaticity space. Assimilation was found at the highest spatial frequency (9.0 cycles per degree (cpd)) while contrast occurred at the lowest spatial frequency (0.7 cpd). Spatial frequency is an important factor affecting the transition from assimilation to contrast in grating stimuli. Data from Smith et al. (2001) indicate that the transition point from assimilation to contrast should be around 4 cpd (around 7.5 arcmin bar width). In using stimuli displayed along the l and s cone axes, these authors showed that the chromaticity shifts in assimilation at 9 cpd were similar for both axes.

These results led to the view that contrast and assimilation are the extremes of a continuum of chromatic induction and depend on the spatial frequencies in the visual scene. Thus, simultaneous contrast arises at low spatial frequencies whereas assimilation arises at high spatial frequency. According to this view, both effects could not occur simultaneously because they are antagonistic.

The fact that assimilation is strongest with narrow inducers can be seen in paintings presented previously by Anuszkiewicz and Lichtenstein (see Figures 13.3a and 13.3b, in the Color plate section). In these examples, the background seems to blend with the colors of narrow stripes or dots.

Some other studies demonstrated the importance of the spatial structure of the inducing patterns. Patterns of concentric circles alternating in two inducing chromaticities generate large chromaticity shifts with patterned backgrounds and weak chromaticity shifts with a uniform background (Monnier and Shevell, 2003, 2004). Moreover, two different color shifts were shown by changing the contiguity of the rings in the achromatic domain. Thus, assimilation was obtained when the inducing ring was contiguous to the test ring, whereas the chromaticity shift was consistent with simultaneous contrast when the inducing ring was noncontiguous with the test ring. These results indicate that the appearance of the test field is highly dependent on the relative distance of the inducing light (see also Hong and Shevell, 2004). The influences of both regions of the inducing patterns are synergistic and produce color shifts when a test ring is presented in alternating purple and lime inducing rings. In sum, chromatic induction occurs in specific stimulus configurations depending on contiguous and remote light in view. This effect is seen also with the paintings of Anuszkiewicz and Lichtenstein (see Figure 13.3, in the Color plate section) and also by Riley (Figure 13.4). Thus, the background seems to blend with the colors of stripes and/or dots. Here, the assimilation effect is obtained because the inducing stripes and/or dots are contiguous with the background.

Other factors such as the luminance of the inducing stimuli can also influence the strength of assimilation (Beck, 1966; de Weert and Spillmann, 1995; Helson, 1963; Shapley and Reid, 1985). Beck (1966) demonstrated that assimilation or contrast depended on whether stripes of varying reflectance were darker or lighter than a gray background. A rating scale for brightness judgments indicated that contrast occurred when the reflectance of a test spot was above the reflectance of the background and that assimilation occurred when the reflectance was below the reflectance of the background. This result was confirmed by de Weert and Spillmann (1995). They used a pincushion pattern to report an assimilation effect. In their experiments, observers adjusted the luminance of a matching field displayed on a uniform background until it appeared identical to the test field of the pincushion. They found that a stimulus decrement relative to the background is required to obtain assimilation with their spatial configuration.

Non-neural theories developed to explain assimilation include wavelength independent light spread due to optical imperfection, and also wavelength dependent chromatic aberration. Spread light causes different chromaticity shifts by changing the light reaching the retina from the test stimulus. Bradley et al. (1992) found chromatic aberration to be more important at higher spatial frequencies, than at lower spatial frequencies. Thus, the aberrations were not different, but the impact would be more important at higher spatial frequencies. For example, if a very thin blue bar were next to a very thin yellow bar (high spatial frequency), then a small offset due to chromatic aberration could cause them to overlap on the retina. This might result in the perception of a light bar and a dark bar rather than of blue and yellow bars. For wider contours, the impact would not be as great because only the edges would overlap. Smith et al. (2001) suggested that chromatic aberration contributes to assimilation, but that there must be additional factors.

Others have argued against a substantial contribution of light scatter (Fach and Sharpe, 1986; Moulden et al., 1993). Notwithstanding the effect of chromatic aberration, different investigators concluded that a neural process contributes to assimilation (Cao and Shevell, 2005; de Weert and van Kruysbergen, 1997; Helson, 1963; Hurvich and Jameson, 1974; Shevell and Cao, 2003; Sugita, 1995), although the neural locus is still unknown (Shevell and Cao, 2004).

Neural theories explain assimilation in terms of receptive field organization. Jameson and Hurvich (1975, 1989) proposed that color assimilation depends upon the diameters of both center and surround regions of the underlying neural receptive fields. Accordingly, if the stimulus elements are small relative to the receptive field centers, assimilation occurs due to color mixing; and if the stimulus elements are larger, then assimilation gives way to spatial contrast. Thus, the chromaticity shift should vary with the width of the stimulus. Indeed, a larger assimilation shift should be obtained with the small widths because the inducing contour adjacent to the induced area will be weighted strongly in the center of the receptive field. Moreover, assimilation should decrease then lead to a contrast effect with larger stimuli because of the inhibitory surround of the receptive field.

But which receptive fields set the limit, retinal or cortical?

Monnier and Shevell (2003, 2004) showed that a patterned background, composed of concentric inducing circles alternating between two chromaticities that stimulate the S-cones, produced a large shift in color assimilation. This result is consistent with S-cone antagonistic center-surround (+S/−S) receptive fields (Monnier and Shevell, 2004). Such neurons have not been found in the retina (Dacey, 2000), but have been reported for the visual cortex (Conway, 2001; Solomon et al., 2004). The transition from assimilation to contrast suggested by Jameson and Hurvich (1975, 1989) might also occur in cortical S-cone spatial antagonism.

Long-range color assimilation: the watercolor illusion

The watercolor effect (WCE) is a phenomenon of long-range color assimilation arising from thin chromatic contours. For example, when a dark purple, closed contour flanks a light-orange, closed contour on the inside, the orange color will spread evenly across the entire enclosed surface area resembling the faint coloration of a watercolor painting (Pinna, 1987; Pinna et al., 2001). An example is illustrated by Figure 13.5 (in the Color plate section). Thus, the interior of the figure is perceived as filled in with a uniform color. This coloration effect is distinguished from other assimilation patterns due to its spatial extent. Indeed, the interior surface is seen with the hue of the lighter colored contour over distances of up to 45 deg (Pinna et al., 2001). Both inducing contours are critical to obtaining a long-range coloration effect. As shown above, a short-range assimilation effect is found for complex displays with a narrow inducing contour. In comparison with these previous patterns, the WCE is distinguished by introducing another inducing contour to obtain a long-range assimilation effect. Finally, the WCE is not purely chromatic and can also be seen with achromatic stimuli (Cao et al., 2011; Pinna et al., 2001).

Characteristics of the WCE

Different features (e.g. size, luminance and chromaticity) of the inducers have been shown to influence the strength of the coloration effect. This illusion has generally been demonstrated using a background higher in luminance (white) than the two contours (Pinna et al., 2001). Devinck and colleagues (2005) quantified the effect of color spreading using both color-matching and hue-cancellation tasks to test the influence of luminance contrast borders. Although the phenomenon still occurs for equiluminant contours, its strength increased with increasing luminance contrast between inner and outer contours. These investigators noted that observers did not need to change the luminance of the enclosed area to make the match. Others studies have reported that the contrast difference between both inducing contours affects the WCE when measured using paired comparisons (Cao et al., 2011), and also with a recently developed procedure, ‘Maximum Likelihood Difference Scaling’ (Devinck and Knoblauch, 2012).

Pinna et al. (2001) determined the optimal contour configurations for eliciting color assimilation. Thus, the optimal contour thickness is approximately 6 arcmin. Although the coloration effect is stronger with wiggly contours, it also occurs with straight contours and with chains of dots (see also Devinck and Spillmann, 2009). A study from Devinck et al. (2006) demonstrates the importance of stimulus width for the WCE. Whereas the wider stimuli (≥ 30.6 arcmin) exhibited small shifts in color appearance, the narrower ones (≥ 9.3 arcmin) produced larger shifts. Using an achromatizing lens to correct for longitudinal chromatic aberration shows that chromatic aberration explains the stronger assimilation effect obtained for narrow stimuli, but not for wide stimuli. Figure 13.6 shows the strength of the WCE as a function for the stimulus width.

Figure 13.6 WCE quantified by color-cancellation and expressed by the shift size vector divided by the inducing contour vector as a function of stimulus width. Results of hue-cancellation are displayed for three viewing conditions: binocular (top panel), monocular (middle panel), and monocular with an achromatizing lens (bottom panel). Different symbols represent different observers. Error bars are (±1 SEM. (From Devinck et al., 2006.)

Pinna et al. (2001) tested different combinations of inducing colors. Different pairs of colors generated the coloration effect. Similar results were found by Devinck et al. (2005) in using a wide range of stimulus chromaticities; data showed a reliable shift in color appearance that closely followed the direction of the inducing contour. Specifically, Devinck and colleagues showed that the chromatic coordinates of the inner and outer contours strongly affect the WCE. Using a hue-cancelation task, the same authors demonstrated that a strong WCE is obtained only when the chromatic coordinates of the inner and outer contours were approximately on opposite sides of the color diagram (i.e. the chromatic contrast in the contour pair was relatively large). Contour manipulation of the kind used here would therefore be expected to affect the spread of watercolor even more. It has also been reported than the illusory coloration occurs on colored and black backgrounds (Pinna et al., 2001). The WCE is most pronounced when a number of intersecting constraints are satisfied in the chromatic and luminance contrast relations. It is well known that color and luminance information are multiplexed in the visual system (De Valois, 2004; De Valois and De Valois, 1975). This suggests that the WCE is a phenomenon that depends upon mechanisms tuned selectively to particular luminance-color conjunctions.

Another phenomenon of importance in understanding the watercolor illusion is that the coloration areas tend to be perceived as the figure and the surrounding areas as the ground (Pinna et al., 2001; Pinna et al., 2003). By reversing the colors of both inducing contours, the figure-ground segregation is reversed, as shown by Figure 13.5. Moreover, Pinna and Reeves (2006) reported that the figure-ground effect was reversed when the luminance contrast of the background was inverted while the overall luminance contrast of the contours was kept constant. One remaining problem is to determine if the color spreading can be dissociated from the figure-ground effect. In a previous study, von der Heydt and Pierson (2006) tested whether the figure-ground effect is a consequence of color assimilation or due to a separate mechanism. To that end, the authors determined the color equivalent to the coloration effect, and then applied it to the complementary area. Using this method, they could cancel the perceived contrast between both regions, and measure the figure-ground segregation. The results showed that nulling the contrast did not neutralize the figure-ground effect in the WCE. Thus, these authors concluded that the figure-ground effect in the WCE is due to mechanisms that are independent from those generating the color illusion.

Neural mechanisms underlying the WCE

The WCE presents a significant challenge to any complete model of chromatic assimilation. The strength of the apparent color spreading seen in this effect depends on a conjunction of chromatic and luminance mechanisms, which may not be sufficient to account for the figure-ground effect.

The computational model developed by Grossberg and Mingolla (1985) involves mechanisms of boundary grouping and surface filling-in that may explain the coloration effect in the WCE (Pinna and Grossberg, 2005). The theory distinguishes between two parallel processing streams, the Boundary Contour System (BCS), which generates perceptual boundaries, and the Feature Contour System (FCS) which creates features (e.g. color or texture) that fill in the regions defined by the BCS (see also Chapter 4 by Breitmeyer). Only the BCS is sensitive to the contrast polarity. Both systems act synergistically and simultaneously such that the FCS will induce the orange color within the inner contour while the BCS will inhibit the boundary between the orange contour and the background. Consequently, the weakened boundary will allow orange color to flow out and spread, whereas the presence of a strong boundary (e.g. the purple contour) will confine the color spreading to within the closed contour.

The color of a surface depends on its contrast with nearby surfaces. Here, the coloration effect is characterized by a spread of color from the inner contour onto the enclosed surface area, suggesting a global effect from local stimulation. One possible explanation is that it involves filling-in, a process in which a neuronal mechanism detects the contour and generalizes it beyond the confines of the immediate stimulus. In neurophysiological terms, the question is how the color diffuses out of the boundary to fill the adjacent area. Two possible explanations can be distinguished: ‘isomorphic’ and ‘symbolic’ color representations.

The first theory, based on ‘isomorphic’ color representation, assumes that color signals spread from the borders into uniform regions at the same cortical level. Thus, it could be argued that long-range interaction via horizontal cortical axons in area V1 provide for the large-scale spreading necessary to perceive the WCE in extended areas, much as Spillmann and Werner (1996) proposed to account for the Craik-O’Brien-Cornsweet Illusion. In addition, Hung et al. (2007) in cat demonstrated border-to-surface propagation from area V1 to V2 with both real and illusory (Cornsweet effect) brightness contrast stimuli. These results suggested that oriented color-contrast cells feed-forward signals to cells with receptive fields located in the interior of the stimulus area. This correlation might play a role in the explanation of color filling-in from inducing contours as it would provide a neurophysiological basis for the transformation of local signals to global signals.

Color spreading observed in the WCE can, in principle, be explained by an alternative theory based on a ‘symbolic’ color representation. It assumes that the border signals are integrated at a higher level to produce uniform color appearance. This hypothesis is supported by the work of Friedman et al. (2003) showing that color-coded cells in layers 2 and 3 of cortical areas V1 and V2 are orientation and edge selective. These authors suggested that the visual system derives surface color from oriented edge signals. A study from von der Heydt et al. (2003) argued also for a ‘symbolic’ color representation. They recorded the neural activity using stabilized images (Krauskopf, 1963). Under this condition, the perceived color of a stimulus patch fills-in with the color of the surrounding patch (e.g. from red to green). Their results indicated that the color surface signals of V1 and V2 remained unaltered by stabilization even though the behavioral response signaled a perceptual change (filling-in). The responses of polarity-selective edge cells representing the color at the boundary of the stimulus showed a decline correlated with the time course of the perceived filling-in.

Most experiments suggest that the WCE is a phenomenon that depends upon mechanisms tuned selectively to particular conjunctions of luminance and color contrast. Thus, the coloration effect in the WCE is initiated at the level of local feature representation. Cortical cells sensitive to border ownership are candidates for explaining this characteristic of the WCE. Such edge-selective neurons of V2 signal the color on the figure side of the edge (Qiu and von der Heydt, 2007; Zhou et al., 2000). These cells respond, for example, to a white–orange border but not to an orange–white border. This response asymmetry may lead to spreading via ‘isomorphic’ or ‘symbolic’ color representation to impart the assimilation effect, but also associated properties defining figure and ground.

Summary/conclusion

Generations of scientists and artists have studied short- and long-range assimilation in the laboratory and on the canvas. This has provided an important empirical basis for distinguishing among several types of contrast mechanism. A unifying neural and computational account of assimilation is still needed. In the future, we look forward to an advance in our understanding of contrast and assimilation, not as separate phenomena, but a consequence of more general aspects of color and spatial processing.

References

Barnes, R. (1990) Monet by Monet, New York, NY: Alfred A. Knopf.

Beck, J. (1966) ‘Contrast and assimilation in lightness judgements’, Perception & Psychophysics, 1: 342–344.

Bradley, A., Zhang, X. and Thibos, L. (1992) ‘Failures of isoluminance caused by ocular chromatic aberrations’, Applied Optics, 31: 3657–3667.

Burnham, R. W. (1953) ‘Bezold’s color mixture effect’, American Journal of Psychology, 66: 378–385.

Cao, D. and Shevell, S. K. (2005) ‘Chromatic assimilation:spread light or neural mechanisms?’, Vision Research, 45: 1031–1045.

Cao, B., Yazdanbakhsh, A. and Mingolla, E. (2011) ‘The effect of contrast intensity and polarity in the achromatic watercolour effect’, Journal of Vision, 11: 1–8.

Chevreul, M.-E. (1839) De la loi du contraste simultané des couleurs, Paris, France: Pitois-Levrault.

Conway, B. R. (2001) ‘Spatial structure of cone inputs to color cells in alert macaque primary visual cortex’, The Journal of Neuroscience, 21: 2768–2783.

Dacey, D. M. (2000) ‘Parallel pathways for spectral coding in primate retina’, Annual Review of Neuroscience, 23: 743–775.

De Valois, R. L. (2004) ‘Neural coding of color’, in L. M. Chalupa and J. S.Werner (eds) The visual neurosciences, Cambridge, MA: MIT Press.

De Valois, R L. and De Valois, K. K. (1975) ‘Neural coding of color’, in E. C. Carterette and M. P. Friedman (eds), Handbook of perception: Vol. 5, New York, NY: Academic Press.

De Valois, R and De Valois, K. K. (1988) Spatial vision, New York, NY: Oxford University Press.

Devinck, F. and Spillmann, L. (2009) ‘The watercolor effect: spacing constraints’, Vision Research, 49: 2911–2917.

Devinck, F. and Knoblauch, K. (2012) ‘A common signal detection model for the perception and discrimination of the watercolor effect’, Journal of Vision, 12: 1–14.

Devinck, F., Delahunt, P. B., Hardy, J. L., Spillmann, L. and Werner, J. S. (2005) ‘The watercolor effect: quantitative evidence for luminance-dependent mechanisms of long-range color assimilation’, Vision Research, 45: 1413–1424.

Devinck, F., Delahunt, P. B., Hardy, J. L., Spillmann, L. and Werner, J. S. (2006) ‘Spatial dependence of color assimilation by the watercolor effect’, Perception, 35: 461–468.

de Weert, C. M. M. (1991) ‘Assimilation versus contrast’, in A. Varlberg and B.B. Lee (eds) From pigments to perception, New York, NY: Plenum Press.

de Weert, C. M. M. and Spillmann, L. (1995) ‘Assimilation: asymmetry between brightness and darkness?’, Vision Research, 35: 1413–1419.

de Weert, C. M. M. and van Kruysbergen, N. A. W. H. (1997) ‘Assimilation: central and peripheral effect’, Perception, 26: 1217–1224.

Fach, C. and Sharpe, L. (1986) ‘Assimilative hue shifts in color gratings depend on bar width’, Perception & Psychophysics, 40: 412–418.

Friedman, H. S., Zhou, H. and von der Heydt, R. (2003) ‘The coding of uniform color figures in monkey visual cortex’, Journal of Physiology, London, 548: 593–613.

Grossberg, S. and Mingolla, E. (1985) ‘Neural dynamics of form perception. Boundary completion, illusory figures and neon color spreading’, Psychological Review, 92: 173–211.

Helson, H. (1963) ‘Studies of anomalous contrast and assimilation’, Journal of the Optical Society of America, 53: 179–184.

Helson, H. and Rohles, F. G. (1959) ‘A quantitative study of reversal of classical lightness-contrast’, American Journal of Psychology, 72: 530–538.

Hong, S. W. and Shevell, S. K. (2004) ‘Brightness contrast and assimilation from patterned inducing backgrounds’, Vision Research, 44: 35–43.

Hung, C. P., Ramsden, R. M. and Roe, A. W. (2007) ‘A functional circuitry for edge-induced brightness perception’, Nature Neuroscience, 10: 1185–1190.

Hurvich, L. M. and Jameson, D. (1974) ‘Opponent processes as a model of neural organization’, American Psychologist, 40: 1–22.

Jameson, D. and Hurvich, L. M. (1975) ‘From contrast to assimilation: in art and in the eye’, Leonardo, 8: 125–131.

Jameson, D. and Hurvich, L. M. (1989) ‘Essay concerning color constancy’, Annual Review of Psychology, 40: 1–22.

Krauskopf, J. (1963) ‘Effect of retinal image stabilization on the appearance’, Journal of Optical Society of America, 53: 741–744.

Monnier, P. and Shevell, S. K. (2003) ‘Large shifts in color appearance from patterned chromatic backgrounds’, Nature Neuroscience, 6: 801–802.

Monnier, P. and Shevell, S. K. (2004) ‘Chromatic induction from S-cone patterns’, Vision Research, 44: 849–856.

Moulden, B., Kingdom, F. and Wink, B. (1993) ‘Colour pools, brightness pools, assimilation, and the spatial resolving power of the human colour-vision sytem’, Perception, 22: 343–351.

Pinna, B. (1987) ‘Un effeto di colorazione’, in V. Majer and M. Santinello (eds) Il laboratorio e la città. XXI Congresso degli Psicologi Italiani, Edizioni SIPs, Milan, Italy: Societa Italiana di Psicologia.

Pinna, B. and Grossberg, S. (2005) ‘The watercolor illusion and neon color spreading: a unified analysis of new cases and neural mechanisms’, Journal of the Optical Society of America A, 22: 2207–2221.

Pinna, B. and Reeves, A. (2006) ‘Lighting, backlighting and watercolor illusions and the laws of figurality’, Spatial Vision, 19: 341–373.

Pinna, B., Brelstaff, G. and Spillmann, L. (2001) ‘Surface color from boundaries: a new “watercolor” illusion’, Vision Research, 41: 2669–2676.

Pinna, B., Werner, J. S. and Spillmann, L. (2003) ‘The watercolour effect: a new principle of grouping and figure-ground organization’, Vision Research, 43: 43–52.

Qiu, F. T. and von der Heydt, R. (2007) ‘Neural representation of transparent overlay’, Nature Neuroscience, 10: 283–284.

Ratliff, F. (1992) Paul Signac and color in neo-impresionism, New York, NY: Rockefeller University Press.

Reid, R. C and Shapley, R. (1988) ‘Brightness induction by local contrast and the spatial dependence of assimilation’, Vision Research, 28: 115–132.

Shapley, R. and Reid, R. C (1985) ‘Contrast and assimilation in the perception of brightness’, Proceedings of the National Academy of Science of the United States of America, 82: 5983–5986.

Shevell, S. K. and Cao, D. (2003) ‘Chromatic assimilation: evidence for a neural mechanism’, in J. D. Mollon, J. Pokorny and K. Knoblauch (eds), Normal and defective colour vision, New York, NY: Oxford University Press.

Shevell, S. K. and Cao, D. (2004) ‘Chromatic assimilation unaffected by perceived depth of inducing light’, Visual Neuroscience, 21: 373–376.

Smith, V. C., Jin, P. Q. and Pokorny J. (2001) ‘The role of spatial frequency in color induction’, Vision Research, 41: 1007–1021.

Solomon, S. G., Peirce, J. W. and Lennie, P. (2004) ‘The impact of suppressive surrounds on chromatic properties of cortical neurons’, The Journal of Neuroscience, 24: 148–160.

Spillmann, L. and Werner, J. S. (1996) ‘Long-range interactions in visual perception’, Trends in Neurosciences, 19: 428–434.

Sugita, Y. (1995) ‘Contrast and assimilation on different depth planes’, Vision Research, 35: 881–884.

von Bezold W. (1876) The theory of color and its relation with art and art-industry, Boston, MA: Prang.

von der Heydt, R. and Pierson, R (2006) ‘Dissociation of color and figure-ground effects in the watercolor illusion’, Spatial Vision, 19: 323–340.

von der Heydt, R., Friedman, H. S. and Zhou, H. (2003) ‘Searching for the neural mechanism of color filling-in’, in L. Pessoa and P. DeWeerd (eds) Filling-in: from perceptual completion to cortical reorganization, New York, NY: Oxford University Press.

Wyszecki, G. (1986) ‘Color appearance’, in L. K. K. R. Boff and J. P. Thomas (eds) Handbook of perception and human performance, Chichester, UK: Wiley.

Zhou, H., Friedman, H. S. and von der Heydt, R. (2000) ‘Coding of border ownership in monkey visual cortex’, Journal of Neuroscience, 20: 6594–6611.

14  The phenomenon of colored shadows

Julian Kallmann, Jürgen Schramme and Christa Neumeyer

Introduction

On a journey through the Harz Mountains near Göttingen in winter 1777, Goethe observed the colors of the snow-covered landscape and described them as yellowish during daylight changing into a wonderful purple near sunset. The shadows cast on the snow appeared to him bluish to violet at first, turning into ‘green comparable in its clarity to green of the sea (Meergrün), but in its beauty to emerald green’ when the sun disappeared behind the horizon (Goethe, 1810, part I, § 75). The phenomenon of colored shadows must have fascinated Goethe for a long time, maybe since 1770, when he noted a book by Nikolaus von Béguelin on ‘Mémoire sur les ombres colorees’, published in Berlin 1767 (Ott, 1979). According to Béguelin (1767) and Monge (Kuehni, 1997) it seems that the first scientific description of the colored shadows goes back to an announcement by de Buffon in 1743, taking up an earlier observation by Abbé de Sauvages. In a preliminary study to his Farbenlehre Goethe described the colored shadows for the first time in 1792. Probably the first detailed and in many respects interesting paper was published by Gaspard Monge in 1789, drawn to our attention by John Mollon (1995). Finally, there is a contribution to the topic by Benjamin Thompson on ‘An account of some experiments upon coloured shadows’ in the Philosophical Transactions of the Royal Society in 1794. According to all three contributions of the late eighteenth century the most frequent and convenient way to observe colored shadows is described as follows: a white piece of paper or another white surface is illuminated by a candle and at the same time by dim daylight at dusk or dawn. An object will then cast two shadows, one by the candlelight, and one by daylight. The shadow cast by daylight is illuminated by the yellow candlelight and appears yellow. The shadow cast by the candlelight is illuminated by daylight, and although we expect it to appear grey or white this shadow is perceived as blue or blue–green.

This classical situation is shown in a painting in Figures 14.1a and 14.1b (in the Color plate section). For the painter it was quite a challenge to depict the scene ‘live’, illuminated by candlelight and simultaneously by the dim daylight of an overcast sky at dusk. It was difficult not only because the ambient light level for painting was rather low, but also because the vividness of the shadow colors was changing with fading daylight. At the beginning the shadow cast by the candle and illuminated by the white daylight through a window on the right appeared in a wonderful, slightly unsaturated turquois. With vanishing daylight this shadow became darker blue and finally black. Accordingly, the daylight cast a weaker and weaker shadow and the yellow color faded away.

Goethe (1810) discussed the possibility that the shadow illuminated by daylight is blue because of the reflected blue skylight. However, as the shadow is seen blue in the light of an overcast sky as well and most vivid when illuminated by the light of the full moon, he concluded that the shadow area is actually illuminated by white light and the effect must be due to an influence of the surrounding slightly colored area. In the earlier report Goethe (1792) summarized his observations of 18 different experiments performed obviously as a response to Béguelin (1767). Involved were always two light sources: on the one hand, candlelight or light reflected by a colored surface, or rising or falling sun, on the other hand, dim daylight, sky light, moonlight or light reflected from white objects. Thus, Goethe’s experiments demonstrate that colored shadows can be observed under many different light conditions, and if one has seen the phenomenon once one finds it almost everywhere. This experience was also reported in 1793 by Georg Christoph Lichtenberg, physicist in Göttingen and author of many sharp aphorisms. After having received a letter by Goethe describing his experiments on colored shadows he wrote: ‘Since the arrival of your letter, I have pursued coloured shadows like a small boy chasing butterflies …’ (Joost and Schöne, 1992: 160).

To demonstrate colored shadows for an audience it is convenient to use two slide projectors and to project the two light beams on a large white screen on top of each other. One projector throws just white light, the other one contains a colored filter. An object between lamps and screen will cast two shadows, a shadow in the same color as the colored filter and a second one tinged in a color about complementary to the filter color. This latter shadow is the interesting one, as the light reflected by this part of the screen is physically just the white projector light, as can easily be seen by occluding the white light beam. Obviously, the color of this shadow is caused by influences of its surround as already supposed by our predecessors. However, the surprising thing is that this surround is not really very colored but appears almost white with only a slight tinge of color, in contrast to the colored shadow, which is sometimes highly saturated. The fact that rather unsaturated surround colors may cause very vivid colored shadows is puzzling taking into account the effects of simultaneous color contrast for which the phenomenon of colored shadows is thought to be an example.

Why are the colored shadows so vivid? In conventional demonstrations of simultaneous color contrast a colored surround of high saturation causes on a small grey test field a slight tinge of color only, which is not at all impressive.

Is a surround of low saturation more appropriate than a highly saturated one?

Is it possible that colored shadows demonstrate the mechanisms underlying color constancy rather than the effects of chromatic induction?

Quantitative measurements, which could give an answer to these questions, are rare. The most careful investigation was performed by Arne Valberg (1974; see Chapter 11 by Valberg). Here we present more recent measurements of colored shadows with stimuli corresponding to the classical case of two superimposed light sources by Kallmann (2003).

Measuring the shadow colors

Using a successive asymmetric color matching technique (Brainard and Wandell, 1992), Kallmann (2003) determined the CIE chromaticity coordinates (CIE 1931 color space defined by the Commission Internationale de l‘Éclairage) of colors perceived on a white test field surrounded by colored light. The surround consisted of an annulus (outer diameter: 22 deg, inner diameter: 9.7 deg visual angle) illuminated with colored light and simultaneously with the same white light as the test field (diameter: 9.7 deg) using two slide projectors. This stimulus corresponds to a shadow illuminated with white light and a surround illuminated by the same white light and colored light in addition. The perceived test field color was matched on a computer monitor in an adjacent black box using the hue, saturation, brightness (HSV) and red, green, blue (RGB) values of the graphic software. After each match, the spectral radiant flux between 380 nm and 780 nm, and the CIE x, y and Y values were determined for the white test field, the colored surround and the matched field on the monitor using a spectroradiometer. One of six broad band filters in the colors dark blue, blue–green, green, yellow, red and pink was inserted in the slide projector which created the annulus, while the second projector, casting white light on annulus and test field as well, contained a day light filter to make the white tungsten light less yellowish. In the first experiment the influence of surround saturation was tested with three subjects by increasing the intensity of the white light in three steps whereas the colored light was kept constant. Luminance of test field and surround were about the same (3–23 cd/cm² depending on color) and increased with decreasing saturation up to 9–80 cd/cm². The CIE-tristimulus values of the surround colors and the white test fields are shown in Figure 14.2 for dark blue, yellow, green and red as examples.

Figure 14.2 CIE-diagram with the color loci of the surround (squares) in three steps of increasing saturation 1–3, the white test field (circle) and the matched test field colors (triangles). The test field was illuminated with white light, the surround with the same white light and colored light in addition. The test field stimulus and the surround stimulus corresponded to the colored (blue) shadow and the yellowish table, respectively, in Figure 14.1a. (From Kallmann, 2003.) The diagrams indicate that the central test field area illuminated with white light is perceived approximately in the complementary color of the surround.

Triangles in Figure 14.2 indicate the tristimulus values of the matched test field colors. In comparison to the loci of the white test fields, the CIE-values of the perceived color are shifted approximately towards the complementary color of the surround. Increasing saturation of the surround color in three steps 1–3 (squares in Figure 14.2) did not change the color loci of the matched test field colors remarkably. To quantify saturation Kallmann used a graphical method (Richter, 1976). He determined at first the dominant wavelengths of the surround color and the matched test field color and calculated then the distance CW between the color locus and the white locus and the distance DW between the locus of the dominant wavelength and the white locus. The saturation is given by the ratio CW/DW.

As shown in Figure 14.3 the saturation of the matched test field color remained almost constant when the saturation of the surround color was increased in three steps. This was the case for green, red and pink. For blue–green, dark blue and yellow a slight increase was found. The results infer that a small saturation of a surround color is sufficient to render about the same effect as a larger one.

Figure 14.3 Saturation of the surround color was increased in three steps and the induced test field colors were matched on a color monitor. The saturation values were determined by the graphical method (Richter, 1976). The result indicates that surrounds of lowest saturation achieve remarkable saturation of the test field (= colored shadow), which does not further increase for red, green and pink, and only slightly does so for blue, blue-green and yellow. (From Kallmann, 2003.)

In a second experiment, Kallmann (2003) investigated the influence of luminosity contrast between surround and test field. For this purpose he used a third slide projector to illuminate the test field additionally and selectively with white light. As shown in Figure 14.4, for most colors the saturation of the matched test field color was stronger when the test field was darker than the surround. In the original demonstrations of colored shadows (see Figure 14.1b) the shadow area is also darker than the surround per definition. This is different in conventional demonstrations of simultaneous color contrast. Here, a grey test field surrounded by a saturated color is in general of higher luminance than the surround. This may be the reason why the effect is rather weak in this case.

Figure 14.4 Saturation of the matched test field color as a function of luminosity contrast. Negative values on the abscissa: test field (t) darker than surround (s) (Kallmann, 2003). Highest saturation and most vivid shadow colors were found whenever the test field was darker than its surround.

Using in principle the same stimulus as Kallmann (2003), Arne Valberg has investigated colored shadows thoroughly (Valberg 1974, 2005; Valberg and Seim, 1983). He quantified the shadow color by haploscopic matching. In his experiments the white test field was kept constant whereas the surround color varied in hue, saturation and luminance. Valberg (1974) found that the purity of the test field increased with decreasing test field luminance, a result similar to that shown in Figure 14.4. In contrast to Kallmann (Figure 14.3) he showed that the purity of the test field color increased with increasing surround purity. This is in line with earlier measurements of chromatic induction in simultaneous color contrast, but not with the intuitive experience in observing rather saturated colored shadows in an almost white and highly unsaturated surround.

Is it possible that the surround in general is seen rather unsaturated because it is already subject to chromatic adaptation shifting the surround color in direction of white?

In a third experiment (not shown here) Kallmann (2003) matched the surround color as well. He found that for some colors, especially dark blue, blue-green, yellow and pink the color loci of the matched surround were slightly shifted towards white in comparison to the physical stimulus. This indicates that the surround appeared less saturated than to be expected. Almost no effect was found for red and green. This result and the finding that colored shadows are almost independent of surround saturation indicate that the phenomenon has to be seen not only in the context of color induction due to lateral interactions but also in the context of more global color constancy mechanisms.

Chromatic induction and colored shadows

Simultaneous color contrast is the consequence of lateral neuronal interactions between adjacent areas on the retina. In the case of colored shadows the area stimulated by white light is influenced by the surrounding area stimulated simultaneously by white and colored light. The interactions responsible for chromatic induction can be inhibitory or excitatory. Assuming lateral inhibition as a principle widely used in visual systems, a shift in color of the white area towards the complementary color of the surround would result whenever cones of equal spectral sensitivity inhibit each other more strongly than unequal ones (Cornsweet, 1970). As the strength of inhibition is proportional to the strength of excitation, a yellow surround exciting the L- and M-cone types more strongly than the S-cone type will inhibit L- and M-cones in an adjacent white area stronger than the S-cones. The equal excitation ratio of all three cone types due to stimulation by the white area will consequently be altered to a relatively stronger excitation of the S-cone type. Thus, the perceived hue of this area will be shifted towards blue, the complementary color of yellow. Lateral interactions are certainly limited to a rather narrow spatial range and one would assume that the induced color is restricted to the border. This, however, is not the case: the contrast color covers the entire area homogeneously. It is very likely that filling-in mechanisms are responsible for such wide field effects, found also in the phenomenon of colored shadows (Spillmann and Werner, 1996).

The neuronal basis of lateral interactions explaining the effects of chromatic induction is not entirely clear. In general, to assign a certain neuron type to ‘color’ on the basis of its wavelength-specific response behavior has to be handled with care because the photoreceptors as the input side of the visual system respond in a wavelength-specific way, and consequently the neurons connected with them as well. Thus, whether a neuron plays a role in color vision at all has to be shown experimentally and is not even entirely clear on the level of the visual cortex (Lennie, 1999; Zeki, 1980, 1995). In primates and other vertebrates excitatory and inhibitory interactions are spectrally and spatially organized in receptive fields of neurons in retina and brain. The most frequent neurons intuitively associated with color vision are the color opponent neurons (for example R+/G−). These neurons, most common in the retina of primates, are certainly not responsible for simultaneous color contrast as they do not respond at all to a red or grey patch in green surround. However, there is another cell type that was found first in the retina of goldfish and carp (Daw, 1967; Kaneko and Tachibana, 1983) and then later in the visual cortex of rhesus monkeys (Michael, 1978) that could account for color contrast. These double-opponent cells with their, for example, R+G−/R−G+ center-surround organization respond optimally to red light in the center and green light in the surround. Furthermore, the inhibition of equal cone types suggested above is realized. In a study of colored shadows Valberg and Seim (1983) successfully tested a quantitative model based on such units.

Color contrast and color constancy

Double opponent neurons are assumed to play a role not only in color contrast or chromatic induction, but also in color constancy. Lateral inhibitory interactions between cones of the same spectral type in general will contribute to chromatic adaptation by selectively reducing the level of excitation stimulated by a homogeneously colored surface. Color constancy is the capacity of the visual system to recognize a surface color widely independent of the color of the illumination. Several mechanisms at different levels of the visual system are involved and it is still not clear which are the most important ones (for reviews: Maloney, 1999; Valberg, 2005; Wandell, 1995). Most peripheral, the molecular feedback loops within the phototransduction cascade regulate sensitivity within each single cone. In a first approximation, selective chromatic adaptation can be described by the von Kries coefficient law stating that sensitivity of each cone type is adjusted by a factor inversely proportional to the level of excitation. As a result the excitation ratio due to stimulation by a white surface under a slightly colored illumination would be adjusted in such a way that it is the same as under white light. Assuming that this is an overall effect, an adjacent surface reflecting white light would appear in the complementary color. By calculating the coefficients for the three cone types for blue illumination, Kallmann (2003) found a good agreement with the observed colored shadows. However, this was not the case for red and green illumination colors.

The close interconnection between color constancy and color contrast became obvious in an experiment performed with human subjects and goldfish. In goldfish, an animal with tetrachromatic color vision, simultaneous color contrast and color constancy have been shown (Dörr and Neumeyer, 1997, 2000). Furthermore, it was possible to investigate the role of the spatial conditions and the lightness relationships for color constancy using red and green color stimuli and illuminations (Neumeyer et al., 2002). Color constancy was perfect whenever the test fields were presented in grey surround, and suboptimal in a black surround. Surprisingly, a white annulus of 4–11 mm width on black background provided perfect color constancy in the same way as a large white surround for green illumination. For red illumination, however, with a large white surround or an 11 mm wide annulus, color constancy was overcompensated and evoked color contrast: all test field colors appeared shifted towards green. This was found not only in goldfish, but also in human subjects who performed the same task. The results of this study indicate that lateral interactions are involved not only in color contrast but also in color constancy, and they show that the mechanisms underlying both phenomena are very similar in goldfish and humans.

The biological significance of color constancy

Taking into account the variations in the spectral composition of natural daylight (Henderson, 1977), color constancy seems to be a necessary precondition for color vision. Biologically relevant are the colors of objects. The light reflected by an object and impinging the eye is determined by the absorption characteristics of its surface and by the spectral composition of the incident light. Without a mechanism that compensates for the spectral changes of illumination, an object, for example a ripe fruit, could not reliably be recognized on the basis of color. Furthermore, not even the lightness of an object could be used for recognition and discrimination without color constancy. The requirement of constant lightness ratios may have even been the selective pressure for the evolution of color vision in general, because more than one photoreceptor type is necessary to detect changes in the spectral composition of daylight (von Campenhausen, 1986).

The biological relevance of color constancy can also be inferred by the fact that it is found in animals as different as honeybee and goldfish, and in all other animals tested for this ability (Dörr and Neumeyer, 2000; Neumeyer, 1980, 1981, 1998). The results of quantitative measurements in honeybee and goldfish indicate that the overall effects can be described by the von Kries coefficient law. In goldfish the underlying mechanisms may be located at a rather peripheral level. Their retina is similar, but in terms of wavelength-specific receptive field organization more complicated than the retina of primates. As mentioned before, the so-called double opponent cells have been found on the level of bipolar (Kaneko and Tachibana, 1983) and ganglion cells (Daw, 1967). Studying the complicated inhibitory and excitatory interactions within cone pedicles, Kamermans et al. (2001) came to the conclusion that they provide a basis for color constancy. They may be relevant not only for goldfish and other lower vertebrates but also for primates. In any case, honeybee and goldfish with their highly effective color vision systems, developed widely independently of each other during evolution, show that color constancy must be the result of rather basic mechanisms functioning probably without ‘higher’ cognitive brain functions.

Colored shadows and color constancy

The fact that the surface surrounding the colored shadow appears highly unsaturated and almost white suggests that the phenomenon reflects primarily the mechanisms underlying color constancy. The color of the shadow may then be regarded as a byproduct of the color constancy mechanism. The black and white version of the painting (Figure 14.1b, in the Color plate section) shows the distribution of lightness in this scene. Whereas the area of the blue shadow is rather dark, highest lightness is found on the surrounding surface. It seems that the visual system interprets this area of highest lightness as the color of the illumination and shifts its color towards neutral by adding the complementary color (D’Zmura and Lennie, 1986).

The connection between the phenomenon of colored shadows and color constancy was probably first formulated by Gaspard Monge in 1789 (Mollon, 1995). According to Monge the following observation is related to the phenomenon of colored shadows: looking through a red glass in front of one eye at red, white and differently colored objects one expects that the red object appears highly saturated red, whereas the other objects take on a color of darker red color or appear black. Surprisingly, the red object appears almost white, and is very similar to a white object. This indicates that the area of highest lightness in the visual field is perceived as white or neutral, even if this light stimulates the L-cone type almost exclusively. The effect is very obvious with red and yellow but less so with other colors. High overall illumination is necessary and many different surfaces with some of them being white. According to Monge small patches of highest lightness are found in each natural scene reflecting the illumination directly and provide information about the position of the object’s surface in space. Such specular highlights in a natural scene reflect the illumination directly and could theoretically be used to determine the spectral properties of a surface by discounting the illumination color and, thus, to achieve color constancy (Lee, 1986; D’Zmura and Lennie, 1986; for review: Hurlbert, 1998).

Colored shadows in art

The phenomenon of colored shadows in the history of painting became evident in the early fifteenth century (Renaissance in Italy), for example in portraits by Leonardo da Vinci, inventor of the famous sfumato technique in oil painting, and other masterpieces of that time. It can also be seen in greenish shadows in the incarnate of nudes, figure compositions and portraits by Peter Paul Rubens, and in the work of other masters of the baroque epoch. The climax is certainly achieved by the Impressionists of France, mainly by Claude Monet in the nineteenth century. Wonderful blue shadows can be seen for example in his painting The Magpie (1868–1869), and in another winter landscape by Alfred Sisley Snow in Argenteuil (1874). Both painters obviously observed the colored shadows directly as they translated their sensations in view of nature immediately into their paintings. However, neo-impressionistic painters like Seurat worked analytically by studying the scientific literature by Chevreul (Bomford, 1995) or by Ogden Rood (Kemp, 2008) on color contrast and complementary colors. Their divisionistic style in which small patches of primary colors were set side by side was the result of this approach. With progressing abstraction in the art of painting since the twentieth century, color has become more and more independent from objects and represents itself, for example in the work of Op Art, by artists like Bridget Riley and others. This included the phenomena of afterimages (described by Goethe, 1810) giving rise to motion in colored structures. However, the phenomenon of colored shadows continues to fascinate in contemporary art (Figure 14.5, in the Color plate section).

Summary/conclusion

The fact that rather vivid shadow colors are induced by rather unsaturated surround colors appearing almost white indicates that the illusion reflects mainly color constancy. The effect is stronger, the darker the shadow area is in comparison to the surround. The surround color, which is the brightest surface in the scene, appears almost white due to chromatic adaptation. In this way the visual system compensates the color of the surround towards neutral. The complementary shadow color can then be regarded as an area in the visual field in which the compensating mechanism underlying color constancy is visible. Thus, colored shadows demonstrate primarily color constancy, accompanied by color contrast. The results of behavioral experiments in goldfish and honeybee suggest that color constancy is based on widely used peripheral mechanisms.

Acknowledgements

We thank Hennes Ruißing for painting for us the colored shadows, for permission to publish his winter landscape and for many discussions on light and shadow in nature and art. We are grateful to Anastasios Nesis and Tom Reuter for critically reading the text.

References

Béguelin, N. von (1767) ‘Mémoire sur les ombres colorées’, Histoire de l’Académie et des Belles-Lettres de Berlin, 27–40.

Bomford, D. (1995) ‘The history of colour in art’, in T. Lamb and J. Bourriau (eds) Colour: Art & science, Cambridge, New York, Melbourne: Cambridge University Press, pp. 7–30.

Brainard, D. H. and Wandell, B. A. (1992) ‘Asymmetric color matching: How color appearance depends on the illuminant’, Journal of the Optical Society of America A, 9: 1433–1448.

Cornsweet, T. (1970) Visual perception, New York, NY: Academic Press, pp. 380–383.

Daw, N. W. (1967) ‘Goldfish retina: Organization for simultaneous color contrast’, Science, 158: 942–944.

Dörr, S. and Neumeyer, C. (1997) ‘Simultaneous color contrast in goldfish – a quantitative study’, Vision Research, 37: 1581–1593.

Dörr, S. and Neumeyer, C. (2000) ‘Color constancy in goldfish: The limits’, Journal of Comparative Physiology A, 186: 885–896.

D’Zmura, M. and Lennie, P. (1986) ‘Mechanisms of color constancy’, Journal of the Optical Society of America A, 3: 1662–1672.

Goethe, J. W. von (1792) ‘Von den farbigen Schatten’, in G. Ott and H. O. Proskauer (eds) Farbenlehre vol. 2 (1979), Stuttgart: Verlag Freies Geistesleben, pp. 97–112.

Goethe, J. W. von (1810) Zur Farbenlehre, Stuttgart: Cotta’sche Verlagsbuchhandlung. Theory of colours (1970) transl. C. L. Eastlake, Cambridge MA: MIT Press.

Henderson, S. T. (1977) Daylight and its spectrum, 2nd edn, Bristol, UK: Hilger.

Hurlbert, A. (1998) ‘Computational models of color constancy’, in V. Walsh and J. Kulikowski (eds) Perceptual constancy. Why things look as they do. Cambridge, MA: Cambridge University Press, pp. 283–322.

Joost, U. and Schöne, A. (1992) Georg Christoph Lichtenberg. Briefwechsel, Bd. IV, München, Germany: C.H. Beck, pp. 160–167.

Kallmann, A. J. (2003) ‘Farbkonstanz und Farbkontrast – eine Untersuchung mit Hilfe der farbigen Schatten’, unpublished thesis, University of Mainz, Germany.

Kamermans, M., Kraaij, D. and Spekreijse, H. (2001) ‘The dynamic characteristics of the feedback signal from horizontal cells to cones in the goldfish’, The Journal of Physiology, 534: 489–500.

Kaneko, A. and Tachibana, M. (1983) ‘Double-opponent receptive fields of carp bipolar cells’, Vision Research, 23: 381–388.

Kemp, M. (2008) ‘The impressionists’ bible’, Nature, 453: 37.

Kuehni, R G. (1997) ‘Memoir concerning certain phenomena of vision’, Commentary and translation by M. Monge. COLOR research and application, 22: 199–203.

Lee, H.-C. (1986) ‘Method for computing the scene-illuminant chromaticity from specular highlights’, Journal of the Optical Society of America A, 3: 1694–1699.

Lennie, P. (1999) ‘Color coding in the cortex’, in K. R Gegenfurtner and L. T. Sharpe (eds) Color vision, from genes to perception, Cambridge, UK: Cambridge University Press, pp. 235–247.

Maloney, L. T. (1999) ‘Physics-based approaches to modeling surface color perception’, in K. R Gegenfurtner and L. T. Sharpe (eds) Color vision, from genes to perception, Cambridge, UK: Cambridge University Press, pp. 387–416.

Michael, C. R (1978) ‘Color vision mechanisms in monkey striate cortex: Dual opponent cells with concentric receptive fields’, Journal Neurophysiology, 41: 572–588.

Mollon, J. (1995) ‘Seeing colour’, in T. Lamb and J. Bourriau (eds) Colour: Art & science, Cambridge, UK: Cambridge University Press, pp. 127–150.

Monge, G. (1789) ‘Memoire sur quelques phénomènes de la vision’, Annales de Chimie, 3: 131–147.

Neumeyer, C. (1980) ‘Simultaneous color contrast in the honeybee’, Journal of Comparative Physiology, 139: 165–176.

Neumeyer, C. (1981) ‘Chromatic adaptation in the honeybee: Successive color contrast and color constancy’, Journal of Comparative Physiology, 144: 543–553.

Neumeyer, C. (1998) ‘Comparative aspects of color constancy’, in V. Walsh and J. Kulikowski (eds) Perceptual constancy. Why things look as they do. Cambridge, UK: Cambridge University Press, pp. 323–351.

Neumeyer, C., D0rr, S., Fritsch, J., and Kardelky, C. (2002) ‘Colour constancy in goldfish and man: Influence of surround size and lightness’, Perception, 31: 171–187.

Ott, G. (1979) ‘Vorbemerkung zum Aufsatz “Von den farbigen Schatten”’, in G. Ott and H.O. Proskauer (eds), Farbenlehre vol. 2. Stuttgart: Verlag Freies Geistesleben, pp. 95–96.

Richter, M. (1976) Einführung in die Farbmetrik, Berlin, Germany; New York, NY: de Gruyter, p. 120.

Spillmann, L. and Werner, J. S. (1996) ‘Long-range interactions in visual perception’, Trends in Neuroscience, 19: 428–434.

Thompson, B., Count of Rumford. (1794) ‘An account of some experiments upon coloured shadows’, Philosophical Transactions of the Royal Society, 84: 107–118.

Valberg, A. (1974) ‘Color induction: Dependence on luminance, purity, and dominant or complementary wavelength of inducing stimuli’, Journal of the Optical Society of America, 64: 1531–1540.

Valberg, A. (2005) Light vision color, Chichester, UK: John Wiley & Sons, pp. 261–274.

Valberg, A. and Seim, T. (1983) ‘Chromatic induction: Responses of neurophysiological double opponent units?’, Biological Cybernetics, 46: 149–158.

von Campenhausen, C. (1986) ‘Photoreceptors, lightness constancy and color vision’, Naturwissenschaften, 73: 674–675.

Wandell, B. A. (1995) Foundations of vision, Sunderland, MA: Sinauer, pp. 287–339.

Zeki, S. (1980) ‘The representation of colours in the cerebral cortex of the monkey’, Nature, 284: 412–418.

Zeki, S. (1985) ‘Colour pathways and hierarchies in the cerebral cortex’, in D. Ottoson and S. Zeki (eds) Ventral and peripheral mechanisms of colour vision, Houndsmills, UK: The Macmillan Press, pp. 19–44.

Colour plate section

Figure 9.3 Spillmann and Levine figures (light-to-dark horizontal and vertical bars) placed against a gray a), black b), and white c) background. The bars on the rows are always in front of the bars on the columns. Panels d), e), and f) show high-pass filtered versions of a), b), and c). The insets represent the average pixel values in the corresponding regions of a particular intersection. At some intersections the value of the centre pixel increases or decreases relative to the horizontal bars.

Figure 9.4 High-pass filtered Spillmann and Levine displays at three different filter sizes – 5 pixel diameter, 25 pixel diameter, and 60 pixel diameter. The filter that best replicates the percept of smudges in the grid is the one closest to the diameter of the intersections in the grid (25 pixels); a smaller filter creates a mid-level gray value out of all the lines, while a larger filter creates sloped lines that do not show the smudges in the intersections. Only the middle filter shows the dynamic luminance changes perceived by observers.

Figure 9.5 Examples of grouping by contrast. When the luminance of the dot lattice is modulated (from light-to-dark and dark-to-light) apparent motion (grouping) occurs on the gradient background along the diagonal. This effect occurs with a luminance background as well as with a colored background.

Figure 12.2 The luminance contrast of a given color interacts with geometric cues of relative position (top), partial occlusion (bottom), and interposition (middle) to determine which part of the configuration stands out as figure against the ground (after Guibal and Dresp, 2004).

Figure 12.5 True colors (Chevreul, 1839) on grey induce unexpected paradoxical changes in the apparent brightness of their backgrounds, which appear brighter on one side. The induction can switch from contrast, where the grey field containing the darker inducers appears brighter, to assimilation, where the grey field containing the brighter inducers appears brighter (after Dresp and Reeves, 2012).

Figure 12.6 Abstract expressionist painting (White Christmas, Byrg Langley, 2008); shades of white, red, and gold suggest Christmas decoration behind a transparent window of frozen glass.

Figure 13.1 Eugène Delacroix Femmes d’Alger dans leur appartement, 1834. (Image © RMN-Grand Palais/Art Resource, NY.)

Figure 13.2 Georges Seurat La Parade, 1889. Bottom panel displays one part of the canvas showing the technique of Pointillism, based upon placement of small dots of pure color according to defined principles. Here, a contrast effect may occur at closer distance and assimilation may dominate at intermediate distance. (Image © The Metropolitan Museum of Art.)

Figure 13.3 Richard Anuszkiewicz Convexity, 1966 (top). In this painting, the red background seems to blend with the colors of stripes. (Art © Richard Anuszkiewicz/Licensed by VAGA, New-York, NY.) Roy Lichtenstein Explosion, 1965–1966 (bottom). Here, the white background shifts toward the coloration of the dots when it is surrounded by the red or blue dots. (© Estate of Roy Lichtenstein and Tate London, 2012.)

Figure 13.5 An example of the watercolor effect. When a light orange contour is surrounded by a dark purple contour, the enclosed area acquires the tint of the orange border. This illusion also produces a figure-ground effect. Here, the color assimilation occurs in the background (displayed on the top) or the figure (displayed on the bottom) by reversing the colors and the luminance of the contours. In this second example, the name Lothar is labelled as foreground figures (e.g. salient shapes) and others as background.

Figure 14.1a Colored shadows as they are observed on a white surface illuminated by two light sources: candlelight from the left and dim daylight through a window (not visible) to the right. The object casts two shadows: a faint yellow one to the left, and a bluish one to the right. The latter is illuminated by the white daylight only, but is perceived as vividly blue. (Still life Colored Shadows, painting by Hennes Ruißing. Oil on canvas, 19.8 x 28.8 cm. Painted on July 3, 2012, between 8 and 10 p.m.)

Figure 14.1b Black and white version of the painting showing the lightness distribution of the scene with the surround of the shadows as the area of highest luminosity.

Figure 14.5 An example of colored shadows in art. In this painting, as in winter landscapes of Impressionist artists, there are pronounced blue shadows cast on snow. (Landscape Hohenaltheim in Winter, 1990, painting by Hennes Ruißing. Oil on canvas, 50 x 60 cm, private collection.)

Index

Accordion Grating Illusion (ADI) 138, 139, 139f

Adams, P. A. 87

aerial perspective 162, 162f

aftereffects see figural aftereffects

Ahissar, M. 17

Allman, J. et al. 15

Anuszkiewicz, R. 173, 175, 176, Plate 9

aperture problems 134

Arnheim, R. 5

art and visual perception 5; chromatic assimilation 172–173, 174f, Plates 7–9; colored shadows 185–186, 195, Plates 11–12; localization errors 75–76, 77f; watercolor effect 177–178, Plate 10; see also color and figure-ground; color perception

Ashida, H. et al. 131, 132

assimilation see chromatic assimilation attention: decoy task approach 86–87; definition 85; and perceptual grouping 85–95, 86f, 93f 94f; role in retinotopic/non-retinotopic representations 92–94, 93f 94f

attitude: definition 85

Attneave, F. 69f, 70, 71, 74

Aubert, H. 148

Aydin, M. et al. 86f, 93f 94f

Backus, B. T. 132–133

Bar, M. et al. 17

Barber Pole Illusion 134–135

Barlow, H. 14

Barnes, R. 173

Beer, A. L. et al. 133

Béguelin, N. von 185, 186

Billino, J. et al. 136

bistable percepts 1; see also Rubin’s vase; Ternus-Pikler displays

Blake, R. 40–41

Blakemore, C. 28, 99, 108

Blakeslee, B. et al. 123

Boynton, G. M. 44

Bradley, A. et al. 176

Breitmeyer, B. G. et al. 35, 42–44, 42f, 43f

brightness illusions 1, 118–125; grating induction 118; Hermann grid 26, 118, 119, 120; high spatial frequency content 121–125, Plates 1–3; Spillmann and Levine displays 118, 119f 120–123, Plates 1–2; Spillmann’s weaves 118, 119–120, 120

Britten, K. H. et al. 30, 31

Burnham, R. W. 175

Byrg Langley (2008) Plate 6

Calvin, A. 86–87

Campbell, F. W. 28, 99, 108

Campbell, M. C. W. 165

Cao, B. et al. 178

Chevreul, M.-E. 173, 195; law of contrast 164–166, Plate 6; law of ‘true’ color 164, Plate 5; simultaneous color contrast 163–164

chromatic assimilation 172–182; in art 172–173, 174f, Plates 7–9; contour parameters 175; long-range assimilation 177–181, 179f Plate 10; luminance effects 176; neural theories 177; non-neural theories 176–177; short-range assimilation 174–177; spatial structure of patterns 176

chromatic contrast 159, 163–166, 172, 192, 193, Plate 6

chromatic induction 172; and colored shadows 192

CIE (Commission Internationalie de l’Eclairage) 151, 187

classification images 29

color and figure-ground 159–168; Chevreul’s laws 163–166, Plates 5–6; cue combination 166; Grossberg’s FACADE theory 167–168; luminance contrast and geometric cues 160–163, 161f, 162f, Plate 4; signal selection to trigger visual attention 166–167; surface effects from color 163–166

color constancy 193; biological significance of 193–194; and color contrast 193; and colored shadows 194–195

color perception 147–155; challenges 154–155; filter model 24; history 148–154; opponent-color theory 147, 148–149; receptive fields 153–154; spectrum 148; three-color theory 147, 150–151; three-receptor theory 147, 150, 151; zone theories 151–153

color spectrum 148

color stereopsis 163, 165

colored shadows 185–196; in art 185–186, 195, Plates 11–12; and chromatic induction 192; color constancy 193–195; color contrast 193; measuring shadow colors 187–191, 188–189f, 190f, 191f

common fate principle 2f

cone adaptation hypothesis 151

contrast: and aerial perspective 162, 162f; Chevreul’s laws 164–166, Plates 5–6

contrast adaptation 98–99, 100f, 106–107f, 106–108

contrast sensitivity function (CSF) 27, 28

Conway, B. R. et al. 132

Craik-O’Brien-Cornsweet Illusion 181

Critchley, M. 40–41

De Buffon, J.-L. Leclerc, Comte 185

De Valois, K. K. 30, 31, 172

De Valois, R. 30, 31, 147, 172

De Weert, C. M. M. 172, 176

Delacroix, E. 173, Plate 7

Dengler, M. 163

depth cues 167–168

Devinck, F. et al. 178, 179f

Difference of Gaussians (DOG) 26, 153

Dresp, B. et al. 160–162, 161f, 165

Duncan, R. O. 44

Egeth, H. 87–88

Egusa, H. 162

Ehrenstein Illusion 159, 160f

electrical field theory 11, 12–14, 68

Enigma Illusion 130–131, 130f

Enroth-Cugell, C. 26

FACADE (Feature-And-Contour-And DEpth) model 36, 37–38, 38f, 41, 167–168, 180; Boundary Contour System (BCS) 36, 37–38, 167; Feature Contour System (FCS) 37, 38, 167

face adaptation 113, 114f

Fach, C. 175

Felleman, D. J. et al. 39

figural aftereffects 98, 99, 101; motion aftereffect (MAE) 109, 111, 112f; tilt aftereffects (TAE) 99, 106, 108; time course 108

figural enhancement 159

figural goodness 67

figure-ground segmentation 1, 48–50, 49f, 159; and grouping 53–54; see also color and figure-ground

filter model 24

Foley, J. M. 29

Fourier theory of vision 102, 104, 104–105f

Fraser Wilcox Illusion 132

Friedman, A. 77, 78

Friedman, H. S. et al. 181

Frisby, J. P. 113

ganglion cell model 25–26

Georgeson, M. A. 129

Gestalt theory 1–3, 2f, 9–19; electrical field theory 11, 12–14, 68; figure-ground segmentation 48–50, 49f, 53–54; law of Prägnanz 12; laws of perceptual organization 11–12; Neo-Gestaltist views 17; perceptual grouping 50–54, 51f, 52f; phi motion 9–10; physical Gestalten 11; pure simultaneity 10; pure succession 10

Giovanelli, G. 68

Giovanelli Illusion 68–69, 69f, 71

Girard, P. et al. 39, 40

Glaser, D. A. 130

Goethe, J. W. von 185, 186

Gombrich, E. H. 5

Gori, S. et al. 130–131, 134, 136, 137, 138

Grassman, H. 150

grating induction 118

Greenlee, M. W. et al. 100f, 108, 109, 110f, 111f, 112f

Gregory, R. L. 124, 130

Grossberg, S. 180; see also FACADE (Feature-And-Contour-And DEpth) model

grouping see perceptual grouping

Guibal, C. R. C. 160–162, 165

Guildford, J. P. 134

Gurnsey, R. et al. 135

Hamburger, K. 118, 119–120, 120f, 131, 136

Hartline, H. K. 24–25

Harvey, L. I., Jr. 76f

Hawken, M. 30–31

Held, R. 13

Helmholtz, H. V. 24

Helson, H. 175

Hering, E. 148, 149, 149f, 150

Hermann grid illusion 26, 118, 119, 120

Heywood, C. A. et al. 40

Hochberg, J. E. 67

Hochstein, S. 17

Hong, S. W. 40–41

Hubbell, M. B. 67, 78n1

Hubel, D. H. 14, 18, 27, 98, 147, 154

Hung, C. P. et al. 181

Hurvich, L. M. 149, 177

Huttenlocher, J. et al. 76, 77

Igel, A. 74–75, 76f

Impressionism 173, 195

inattentional blindness 87

isomorphism 11, 13, 45, 67–68

Ito, H. 129

Jacob, J. 42–44, 42f, 43f

Jameson, D. 149, 177

Joost, U. 186

Judd, D. B. 151–152

Kallman, A. J. 187–190, 188–189f, 190f, 191, 191f, 193

Kamermans, M. et al. 194

Kanizsa, G. 74

Kitaoka, A. 131, 132

Klee, P. 5

Knoblauch, K. 178

Koffka, K. 1, 10, 12, 67, 68, 70

Köhler, W. 1, 10, 11, 12–13, 14, 18, 67–68, 70, 87, 98

Kovács, G. et al. 113

Krech, D. 86–87

Kumar, T. 130

Kuriki, I. et al. 133

Lamme, V. A. F. et al. 16, 36

Land, E. H. 151

Lashley, K. S. et al. 1, 3, 13, 18

law of Prägnanz 12

Lee, B. B. 154

Legge, G. E. 29

length illusions 88

Leonardo da Vinci 75, 77f, 148, 167, 195

Levi, D. M. 30

Leviant, I. 130

Lewin, K. 1

Lichtenberg, G. C. 186

Lichtenstein, R. 173, 175, 176, Plate 9

Livingstone, M. S. 154

localization: effect of shape contour on 69f, 70

localization errors: attractor basins and unstable equilibria points 71–74, 72f, 73–74f effects of field size on 75; effects of number of targets on 75, 76f; effects of time delay on 74–75; implications for artistic composition 75–76, 77f; internal psycholophysical scales 76–77; vector field decomposition of 70–71

Lu, Z. L. 120, 121, 123

MacAdam, D. L. 152

McAlister, E. 67

McCourt, M. E. 123

Mack, C. M. et al. 87

MacKay Rays 129, 130f

Magnusson, S. 108

masking 88; metacontrast masking 35–36, 37f; object masking and unmasking 54, 55–57, 56f

Maxwell, J. C. 150

Metzger, W. 1, 12

microgenesis of object perception 35

Mingolla, E. 180

Mollon, J. 185

Mona Lisa 75, 77f

Monet, C. 172–173, 195

Monge, G. 185, 194–195

Monnier, P. 176, 177

Moore, C. M. 87–88

motion aftereffect (MAE) 109, 111, 112f

motion illusions 128–140; illusory motion in stationary stimuli 129–133; misperception of motion 133–139

Motokawa, K. 70

Müller, G. E. 152

Müller, R. 109, 112f

Murakami, I. et al. 133

Nakayama, K. et al. 166

Neri, P. 30

neural network approaches to object vision 36–41; form- and surface-processing in human vision 40–41; form- and surface-processing in primate visual cortex 39–40, 39f; Grossberg’s model 36, 37–38, 38f

neuroaesthetics 75

Newton, I. 148, 152

Nitschke, W. 163

non-retinotopic processing 89–91, 91f, role of attention 92–94, 93f, 94f

Nordby, K. 101–102, 103f

object perception 48–65; Bar’s general theory 17; figure-ground segmentation and grouping 53–54; Gestalt approach 48–53; perceptual objects 54–63

Ögmen, H. et al. 36, 90, 91f

Op Art 173, 174f, 195

opponent-color theory 147, 148–149

Oriented Difference of Gaussians (ODOG) model 123

Oruç, I. 132–133

O’Shea, R P. et al. 162, 163

Otero-Millan, J. et al. 133

Ouchi Illusion 131–132, 132f

Palmer, G. 150

Parker, A. 30–31

perceptive and receptive fields 22–31; classification images 29; comparisons 29–31; conceptual architectures 23–24; general schema 23; historical developments 24–27; multiple pathways 27–28; Reverse Correlation 29; terminology 22

perceptual grouping 50–53, 51f, 52f; and attention 85–95, 86f, 93f, 94f, definition 85; and figure-ground segmentation 53–54; and non-retinotopic processing 89–91, 91f, 93f, 94f; spatiotemporal grouping 88–89

perceptual objects 54–63; attributes and related effects 60–61f, 60–62; deceiving object camouflage by accentuation 63, 64f; many objects within one pattern 54, 55f; object masking and unmasking 54, 55–57, 56f; one, none, a hundred thousand objects 62–63, 62f; same but different objects 57–59, 58f, 59f

Pfaff, S. 71, 72, 74

phi-motion 3, 9–10

pictorial depth cue 167–168

Pierson, R 180

Pikler, J. 88

Pinna, B. et al. 131–132, 177, 178, 180

Pinna-Brelstaff Illusion see Revolving Wheels Illusion (RWI)

Pointillism 173, Plate 8

Pop Art Plate 9

pure simultaneity 10

pure succession 10

receptive fields 22, 23; in color vision 153–154; see also perceptive and receptive fields

Reeves, A. 180

Reid, R. C. 175

retinex theory 151

Reverse Correlation 29

Reverse Hierarchy Theory 17

Revolving Wheels Illusion (RWI) 135–136, 135f

Ricco’s law 24

Riley, B. 173, 174f, 176, 195

Ringach, D. 29

Robson, J. G. 26, 28

rod monochromat KN 101–102, 103f

Rodieck, R. W. 24, 25–26

Rohaly, A. M. 162

Rohles, F. G. 175

Rood, O. N. 195

Rotating Snakes Illusion 132–133, 133f

Rotating-Tilted-Lines Illusion (RTLI) 136–138, 137f

Rubens, P. P. 195

Rubin, E. 48

Rubin’s vase 85, 86f

Ruissing, H. 185–186, Plates 11–12

Ruzzoli, M. et al. 131

RWI see Revolving Wheels Illusion

Sachs, M. B. et al. 28

Schmidt, E. K. 71, 72f, 73–74f, 74, 75

Scholte, H. S. et al. 36

Schöne, A. 186

Schrödinger, E. 148, 149

Schwartz, B. J. 162, 163

Seim, T. 192

self-organizing properties of the visual field 11, 67–78; effect of shape contour on localization 69f, 70; Giovanelli Illusion 68–69, 69f, 71; localization errors 70–77, 72f, 73–74f, 76f, 77f

Seurat, G. 173, 195, Plate 8

Shadlen, M. N. et al. 31

Shapiro, A. G. 118, 119–120, 120f, 121, 123

Shapley, R. 29, 175

Sharpe, L. 175

Sharpe, L. T. 151

Sherrington, C. 22

Shevell, S. K. 176, 177

Silverman, G. H. 166

Simonet, P. 165

simultaneous contrast 159, 192

single-neuron doctrine 14, 15–16

Sisley, A. 195

Smith, V. C. et al. 175, 176–177

spatial adaptation in human vision 98–114; adaptation in motion domain 109, 111, 112f; contrast adaptation 98–99, 100f, 106–107f, 106–108; face adaptation 113, 114f; Fourier theory 102, 104, 104–105f; higher order effects 111, 113; mapping spatial channels 99, 101–102, 103f; mechanism 106–109; neural fatigue 106; time course of adaptation 108–109, 110f; 111f

spatiotemporal dynamics of form and surface processing: conscious level 41; nonconscious level 41; surface filling-out 42–44, 42f, 43f

spatiotemporal grouping 88–89

spatiotemporal unity of perception 35–45; metacontrast masking 35–36, 37f; neural network approaches to object vision 36–41; spatiotemporal dynamics of form and surface processing 41–44, 42f, 43f

Sperling, G. 162, 163

Sperry, R. W. et al. 13, 18

Spillmann and Levine displays 118, 119f, 120–123, Plates 1–2

Spillmann, L. et al. 12, 26, 29–30, 31, 98, 113, 131–132, 140, 176, 181

Spillmann’s weaves 118, 119–120, 120f

Stadler, M. et al. 70–71

Stockman, A. 151

Stumpf, C. 1

Ternus, J. 88

Ternus-Pikler displays 88, 89f, 90, 92–94, 93f, 94f, 95

Thomas, J. P. 28

Thompson, B. 185

three-color theory 147, 150–151

three-dimensional aperture problem 134

three-receptor theory 147, 150, 151

tilt aftereffect (TAE) 99, 106, 108

Tolhurst, D. J. 101

Trendelenburg, W. 148

Troncoso, X. G. et al. 131

two-dimensional aperture problem 134

Tyler, C. W. 75

Unchained Dots Illusion (UDI) 138, 138f

Valberg, A. et al. 154–155, 187, 190, 191, 192

visual illusions: Gregory’s taxonomy 124; see also brightness illusions; Ehrenstein Illusion; Giovanelli Illusion; length illusions; motion illusions; watercolor illusion

visual neuroscience: standard view 14–15

visual space: definition 22

Von Bekesy, G. 26

Von der Heydt, R. et al. 15–16, 180, 181

Von Ehrenfels, C. 1, 10, 125

Von Helmholtz, H. 148, 149f, 150–151

Wallach, H. 13, 68, 98, 134

Wang, Y. et al. 40

watercolor effect (WCE) 160, 177–181, Plate 10; characteristics 178–180, 179f; underlying neural mechanisms 180–181

Werner, J. S. 181

Wertheimer, M. 1, 3, 5, 9–12, 17–18, 50, 67, 128

Westheimer, G. 26

Wiesel, T. N. 14, 18, 27, 98, 147

Wilson, H. R. 162

Xiao, Y. et al. 39, 40, 154

Yacorzynski, G. K. 70

Yazdanbakhsh, A. 134, 136, 137

Yeh, C. I. et al. 22, 24

Young, T. 150–151

Zeki, S. et al. 40, 130, 154

Zhou, H. et al. 16

zone theories 151–153