ZZH coupling : A probe to the origin of EWSB ?
Choudhury, Debajyoti
2003-02-17
Creator
Publisher
Contributor
Date
2010-01-19
Description
When people think about highly abstract concepts, they draw upon concrete experiences to structure their thoughts. For example, black knights in fairytales are evil, and knights in shining armor are good. The sensory experiences black and white are used to represent the abstract concepts of good and evil. These and similar metaphors are not only used to intentionally communicate the meaning of abstract concepts, but also underlie abstract conceptual thought. Several views on the representation of concepts have emerged over the last decennium which all share the idea that conceptual processing is perceptual in nature. According to these grounded approaches to cognition, conceptual thought consists of representations built on concrete sensory information. Even though people cannot directly see, hear or touch abstract concepts, perceptual information is theorized to play an essential role in abstract conceptual thought. The current thesis investigates to which extent people use perceptual information to think about abstract concepts. The studies reveal that sensory experiences structure abstract thoughts. Chapter 2 proposes that many of our concrete experiences with moral situations are directly related to symmetrical or balanced distributions of rewards and other outcomes, and aims to shows that thinking about moral or immoral words activates associations with perceptual symmetry. People were asked to guess which of two Chinese ideographs correctly translated moral and immoral Dutch words. As expected, they chose more symmetric ideographs for moral words, and more asymmetric ideographs for immoral words. Chapter 3 focuses on the concept of time, which has been shown to be represented with the past at our left, and the future at our right. Since the auditory modality shows a similar sensitivity to spatial information as the visual modality, the previously observed structuring of time in space was assumed to extend to the auditory modality. Past and future related words were presented to participants over headphones. Words were louder in the left or right ear, and people were asked to indicate in which ear they judged the words to be louder. Critical experimental trials were presented equally loud in both ear. Participants thought future related words in these critical trials were louder in the right ear more often than past related words. Chapter 4 shows that sensory dimensions structure abstract dimensions. Black and white represent bad and good. White in isolation, however, is rather neutral. It seems that perceptual information such as the color white does is not intrinsically positive, but can be used in opposition to black to represent the opposition between good and evil. Together, these chapters show that perceptual information influences abstract conceptual processing, even for highly abstract concepts that lack perceptual characteristics. Future research could focus on how the learning of abstract concepts by children can be facilitated by perceptually representing their meaning. Furthermore, metaphors cannot only facilitate, but also constrain moral reasoning, an aspect which has not been investigated. Abstract thought is one of the most sophisticated abilities of human beings, and acknowledging the importance of perceptual representations will substantially improve our understanding of abstract reasoning.
Subject
time;
Type
Identifier
Dissertatie reeks Kurt Lewis Instituut 2010-3 (2010)
Database
Language
Show preview
Hide preview
Abstract Concepts in Grounded Cognition
Daniël Lakens
Abstract concepts in grounded cognition
Abstracte concepten in gegronde cognitie
(met een samenvatting in het Nederlands)
Proefschrift
ter verkrijging van de graad van doctor
aan de Universiteit Utrecht
op gezag van de rector magnificus, prof. dr. J. C. Stoof,
ingevolge het besluit van het college van promoties,
in het openbaar te verdedigen op
19 januari 2010 des middags te 2.30 uur
door
Daniël Lakens
geboren op 18 juli 1980 te Rotterdam
Promotor: Prof. dr. G. R. Semin
Contents
Introduction 5
Chapter 1 Grounded Cognition 11
Chapter 2 On the Grounding of Morality in
Perceptual Symmetry 55
Chapter 3 The Sound of Time 71
Chapter 4 But for the Bad, There Would not
be Good 84
Footnotes 112
Appendices 113
References 115
Summary in Dutch 133
Curriculum Vitae 139
5 Introduction
Morality, time, and valence are highly abstract concepts. Exactly how people are able to think and talk about abstract concepts is a question that has intrigued philosophers and scientists for centuries. In this thesis I focus on the question how people represent the meaning of abstract concepts, by investigating whether abstract concepts are understood through the use of metaphors. Metaphors allow people to think about abstract concepts in terms of concrete experiences. For example, the scales of Lady Justice are a metaphor for justice. People might not be able to see the future, but it can be represented as being ahead of us. Black knights in fairytales are usually evil, and the color of their armor is a metaphoric representation of the relation they have with the knights in shining armor. These and similar metaphors are not only used to intentionally communicate the meaning of abstract concepts, but these metaphoric representations also underlie abstract conceptual thought. How the meaning of abstract concepts is grounded in concrete experiences is the topic of this dissertation.
Broadly speaking, two views exist on how abstract concepts are represented in our mind. On the one hand, amodal views share the common theme that people think in a purely symbolic language, where concepts
6 are represented as artificial symbols, and computational operations are performed upon these symbols during conceptual processing (Dennett, 1969; Fodor, 1975; Pylyshyn, 1984; Jackendoff, 2002). These artificial symbols are inherently non-perceptual; none of the original sensory or motor information from any modality (e.g., vision, touch, sound, smell, etc) which created these symbols is present when the final amodal symbols are used to represent concepts. On the other hand, several views on the representation of concepts have emerged over the last decennium which all share the idea that conceptual processing is perceptual in nature (e.g., Barsalou, 1999; Gallese & Lakoff, 2005; Glenberg & Kaschak, 2002; Jeannerod, 2001; Pulvermüller, 1999; Zwaan, 2004). According to these grounded approaches to cognition, conceptual thought consists of representations built on concrete sensorimotor information. For example, understanding the word ‘kick’ is assumed to depend at least in part on the activation of the muscles used to perform a kicking motion.
As a result of the growing number of empirical studies that have investigated the role of sensorimotor activation in language comprehension, a consensus is emerging that sensorimotor areas in the brain are activated when people read words related to concrete concepts (Zwaan, 2009; Mahon & Caramazza, 2009; Vigliocco & Meteyard, 2008). For example, thinking about the color of a banana activates brain areas in the visual cortex (Simmons, Ramjee, Beauchamp, McRae, Martin, & Barsalou, 2007). Thinking about the color of a banana leads to remarkably similar patterns of brain activation as actually seeing a banana.
The extent to which abstract conceptual thought is grounded in perceptual information is an important theoretical question for grounded approaches to cognition due to the lack of perceptual characteristics of abstract concepts. Which visual, motor or other sensory information could in any way facilitate the understanding of abstract concepts such as ‘morality’? Several possible explanations have been put forward to accommodate the perceptual representation of abstract concepts in grounded cognition (Barsalou, 1999; Boroditsky, 2000; Glenberg & Robertson, 2000; Lakoff & Johnson, 1980, 1999), but whether abstract
7 conceptual processing can be accounted for by grounded approaches to cognition remains highly disputed (Arbib, 2008; Barsalou; 2008; Boroditsky & Prinz, 2008; Lakoff, 2008; Mahon & Caramazza, 2008).
Abstract concepts play important roles in human cognition as well as in social interaction. The empirical investigation of conceptual thought has focused primarily on concrete concepts (for reviews, see Fischer & Zwaan, 2008; Vigliocco & Meteyard, 2008), and the question how abstract concepts are grounded and especially which process underlies the perceptual representation of abstract concepts remains debated. Given that many abstract concepts constitute those things in life that people care greatly about (e.g., morality, compassion, power, knowledge) it is worthwhile to investigate how people think about these concepts. How people represent abstract concepts will both structure and constrain how these concepts are understood (Lakoff & Johnson, 1999), and as such, different representations might have profound consequences for peoples’ beliefs and decisions. For example, Lakoff (1996) describes the different metaphors used by liberals and conservatives to think about morality, and highlights the consequences of these different representations of morality in politics.
Overview
In the present dissertation, I examine the role of sensory information for abstract conceptual thought. In Chapter 1, the central assumptions in grounded approaches to cognition are introduced and contrasted with more traditional views on the representation of concepts expressed by cognitive researchers since the cognitive revolution in psychology. After detailing the different theoretical views on cognition, I present an overview of studies investigating whether and how abstract concepts are represented perceptually (including the studies reported in the three empirical chapters of this thesis). This review is concluded by a discussion of what these empirical findings reveal about how the meaning of abstract concepts is grounded in concrete experiences. Alternative explanations
8 for the research findings, based on structural and linguistic features of the paradigms, are discussed. Finally, I provide my own interpretation of what the findings on the relationship between perception and cognition reveal about abstract conceptual thought.
The empirical investigation of how abstract concepts are grounded in perceptual information is gaining popularity, but constitutes a relatively new field of interest. Many studies have aimed to demonstrate that perceptual information plays a role in conceptual processing, but there are only a few theories to explain these findings, and an integrative framework to account for the empirical results is currently not available. Researchers have noted that grounded theories of cognition need further development to account more precisely for the effects observed in the literature (Zwaan, 2009). In addition, the mechanisms which are assumed to underlie perceptual representations should be specified in more detail to be able to empirically test the predictions made by these theories (Boroditsky, 2000). Investigating the possible limitations of grounded approaches to cognition, such as whether abstract concepts can be represented perceptually, in which modalities people can represent abstract concepts, as well as the influence of the context in which perceptual information is presented, are all important steps to achieve a more detailed understanding of whether and how perceptual information structures abstract concepts. The goal of the present dissertation is to investigate these questions empirically, thereby contributing to the development of an integrative theoretical framework of the role of sensorimotor information in the grounding of abstract concepts.
The three empirical chapters of this thesis focus on how morality, time, and valence are grounded in perceptual symmetry, left-right auditory space, and brightness, respectively. The aim of Chapter 2 is to show that morality, one of the hallmark abstract concepts assumed to lack the perceptual characteristics to be grounded in sensorimotor information, can be represented perceptually. Based on the assumption that equality is an important feature of morality, and the equality component of morality is argued to be metaphorically represented in balance and symmetry, I
9 predicted and found that moral concepts show a stronger association with perceptually symmetrical stimuli compared to immoral concepts.
Chapter 3 increases the scope of perceptual grounding by providing support for the theoretical prediction that abstract concepts can be structured in the auditory modality. Earlier research into the grounding of abstract concepts has focused predominantly on the visual modality. Since the auditory modality shows a similar sensitivity to spatial information as the visual modality, the previously observed structuring of time in space should extend to the auditory domain. Time is represented from left to right visual space, with the past to the left and the future to the right, and the same structuring was expected to be observed in auditory space. Participants were asked to indicate whether words were presented louder in the left or the right ear while passively listening to time related words. Critical experimental trials were presented equally loud to both ears. Future words were more often judged to be louder in the right ear than past related words were, supporting the left to right representation of time in auditory space. Furthermore, the visual representation of time and the auditory representation of time were shown to be significantly related, providing additional support for the multimodal grounding of abstract concepts.
In Chapter 4, two process explanations concerning the metaphorical representation of abstract concepts were tested against each other. These studies provide a rare theoretical comparison of two processes that have been suggested to underlie the grounding of abstract concepts. Whereas previous researchers (Meier, Robinson, & Clore, 2004; Sherman & Clore, 2009) have argued that visual information (e.g., the color white) will automatically and obligatorily activate an abstract meaning (e.g., positivity), these studies support a more active metaphoric structuring process. The metaphoric representation of positivity as white depends on whether the conceptual opposition between good and bad and the perceptual opposition between white and black are activated. Thus, the shared relational structure between the concrete dimension and the abstract dimension is crucial for the metaphoric representation of
10
valence in brightness to emerge. Together, these chapters show that perceptual information
influences abstract conceptual processing, even for highly abstract concepts that lack perceptual characteristics. Some abstract concepts, such as time, seem to consist of multimodal representations, and judgments in the visual and the auditory domain are influenced by these metaphoric representations in a similar way. Nevertheless, shared relational structures (such as polar oppositions) between the abstract and concrete dimension can be an essential pre-requisite for the metaphoric representation of abstract concepts to become activated. Further research should explore the circumstances under which the activation of shared relational structures is necessary for metaphoric mappings to emerge, and these attentional processes should be taken into account in future studies that aim to investigate how abstract concepts are grounded.
Chapter 1
Grounded Cognition
13
Wenn wir der Körperwelt, welche unmittelbar nur in unserer Vorstellung dasteht, die größte uns bekannte Realität beilegen wollen; so geben wir ihr die Realität, welche für Jeden sein eigener Leib hat: denn der ist Jedem das Realste.
Schopenhauer, WWR 1 105.
When people think about highly abstract concepts, they draw upon concrete experiences to structure their thoughts. Due to the increasing popularity of grounded approaches to cognition, the question how concrete experiences shape abstract thoughts has received renewed attention over the last decennium. Grounded approaches to cognition argue that sensorimotor experiences play an important role when people think about concepts. Sensorimotor information stored in visual, auditory, and motor areas in the brain when people directly experience the world is activated when they think about the world at a later time. For example, when you think about what the earth looks like from outer space, visual areas become activated in a similar way as when you would see a blue and green ball covered with white dots of clouds. Even though people cannot directly see, hear or touch abstract concepts, and therefore have not directly experienced sensory information that is related to the meaning of abstract concepts such as morality, perceptual information is theorized to play an essential role in abstract conceptual thought (Barsalou, 1999; Glenberg & Kaschak, 2002; Gallese & Lakoff, 2005).
Such a grounded approach to cognition stands in stark contrast to the dominant views expressed by researchers since the cognitive revolution in psychology. Traditional theories of conceptual processing generally do not assume that conceptual thought is perceptual in nature, but argue for (or implicitly assume) a purely symbolic language of thought (Anderson, 1990; Dennett, 1969; Fodor, 1975; Pylyshyn, 1984; Jackendoff, 2002), which consists of artificial symbols. These symbolic (or amodal)
14
theories draw a sharp distinction between perception and cognition, and posit that perceptual information processing and conceptual thought are fundamentally different processes. Essentially, thoughts are independent from our experiences with the world. A symbolic representation of the color red is not the equivalent of the sensory experience of the color red (Pylyshyn, 1984).
Symbolic representations are theorized to exist independently from sensory experiences. Concrete sensory information is encoded into semantically interpretable syntactic structures that do not longer retain the perceptual information from which they originate. This encoding is not unlike what happens when I press the ‘d’ key on my keyboard. Although the letter that appears on the screen resembles the letter ‘d’, this letter is represented in a completely artificial and symbolic logical code by the computer. This becomes apparent when this file is opened in a simple text editor, and the letter ‘d’ is represented somewhere in this string of symbols: ¿òmËÊÄÍ?áÈ6·U©ÏñK©æ²8ø. Not only is the internal representation of the input completely symbolic, but the relation of the perceptual input (‘d’) and the internal representation is arbitrary. Any symbol can be used to represent the letter ‘d’. This becomes apparent when the font is changed from Times to Wingdings, where the input of the letter ‘d’ is mapped to the symbol ‘d’.
According to proponents of symbolic symbol systems, computations performed upon symbolic representations are the essence of mental processes (Fodor, 1980; Pylyshyn, 1984). An important advantage of amodal theories is that they allow different modal perceptual processes to be integrated at the symbolic level of representation. The term modal is used to indicate that these perceptual processes are domain specific: for example, visual, auditory, and haptic perceptual information is processed in different brain areas. Modal perceptual processes are fast, bottom-up and isolated from much of our knowledge about the world (Fodor, 1985). Higher cognitive processes, on the other hand, are assumed to be slow, neither bottom-up nor top-down (information can flow either way) and well-connected, so these processes can integrate sensory information
15
from different modal domains. One of the most important (and most difficult to answer)
challenges for symbolic symbol systems is how symbolic representations are connected to their referents, objects in the world. This challenge is known as the symbol grounding problem (Harnad, 1990; Searle, 1980). Where do amodal symbols get their meaning from, if their only sources of meaning are other meaningless amodal symbols? The meaning of symbolic representations has to be grounded in the perceptual experiences they refer to (e.g., the symbolic representation of the color red has to be, in some way, related to the perceptual experience of seeing the color red), but so far, no fully satisfactory solution to the symbol grounding problem has been provided. A second open question for symbolic symbol systems is a detailed process explanation that describes the mechanisms through which symbolic representations are created from perceptual information. How exactly is the sensory experience of the color red transduced, interpreted and encoded into an amodal symbol? Finally, the absence of empirical support for the presence of amodal symbols in the brain has been mentioned as a problematic issue for symbolic symbol systems (Barsalou, 1999). Together, these three open questions warrant the careful consideration of alternative approaches to cognition.
Harnad (1990) proposes a hybrid account of iconic representations of entities (which consist of analogue transformations of the sensory- motor experiences when encountering the entity) complemented by categorical representations (which consist of symbolic abstractions from all iconic representations people have). Importantly, both iconic and categorical representations are non-symbolic. The original sensory-motor information that was activated when a concrete entity was encountered is preserved due to analogue transformations. Although such a hybrid system, consisting of both perceptual memories and amodal symbols, provides a solution to the symbol grounding problem by proposing that amodal symbols activate perceptual memories during comprehension, one might wonder whether there is still anything left to do for amodal symbols in such a model.
16
Barsalou (1999, 2008a) proposes an exclusively perceptual symbol system where conceptual knowledge consists of modal representations. This grounded approach to the representation of concepts assumes that when concrete objects are perceived, the sensory-motor activation in the brain is captured in local association areas, which are subsequently integrated into multi-modal representations by higher-order cross- modal associations (or convergence zones, Damasio, 1989). Convergence zones can integrate information from different modalities. Contrary to symbolic representations, the information in convergence zones has retained the perceptual information of the concrete experiences the modal representations refer to, thus performing the integrative function of symbolic representations while retaining the connection to the perceptual input. The multimodal associations in convergence zones (or simulators) can be partially activated during language comprehension, leading to neural reenactments (or simulations) that activate the same sensory and motor areas in our brain as were active when we directly experienced its referent. For example, the representation of the color red has recently been shown to activate the same visual areas in the brain when we directly perceive the color red, as when we think about it (Simmons et al., 2007). Studies like these provide support for the a-priori predictions derived from grounded theories of cognition. If a computer would work as our brain works, the letter ‘d’ on a computer screen would be stored as a picture and a vinyl record, and the computer could play the sound recording on a record player or take the picture out of a drawer when it is needed.
Similar theoretical views have appeared in recent years (e.g., Gallese & Lakoff, 2005; Glenberg & Kaschak, 2002; Jeannerod, 2001; Pulvermüller, 1999; Zwaan, 2004) which all share the idea that conceptual processing is built upon and activates sensory-motor processes. According to these approaches to cognition, conceptual thought is fundamentally perceptual. More traditional approaches to cognition have acknowledged the possibility that people use imagery to facilitate conceptual processes (e.g., Fodor, 1975), but these theories share the basic assumption that
17
conceptual thought is not perceptual in nature. Perceptual imagery can help people to think about concepts, but perceptual information is not necessary. Therefore, the real debate between modal and amodal symbol systems is not about whether perception and cognition are related, but how they are related. As Fodor (1985) summarizes: “No one in his right mind doubts that perception interacts with cognition somewhere. What’s at issue […] is the locus of this interaction.” According to amodal symbol systems, perceptual information might play a marginal role in conceptual thought, but these images are generated based on symbolic representations (Kosslyn, 1980). According to modal symbol systems, conceptual thought consists of the reactivation of multimodal representations stored during direct experiences with its referents (Barsalou, 1999; Glenberg & Kaschak, 2002).
With the emerging popularity of grounded approaches to cognition, the question where and how perception and cognition interact has received an increasing amount of attention from a variety of scientific disciplines such as linguistics, cognitive science, and social psychology. The main goal of these investigations is to show that conceptual processing activates experientially consistent sensorimotor areas. For example, Hauk, Johnsrude and Pulvermüller (2004) show that reading the words ‘pick’, ‘kick’ and ‘lick’ do not only activate areas in the brain related to language processing, but that these words also activate motor areas in the brain related to hand movements, leg movements, and mouth movements, respectively. The empirical support for intimate relationship between sensorimotor information and the processing of concrete concepts is mounting (for a recent review, see Meteyard & Vigliocco, 2008).
Research based on a grounded approach to cognition has provided alternative ways to think about conceptual thought. First of all, our minds and bodies are not fundamentally distinct, as symbolic theories of cognition presuppose. The widespread bodily influences on social cognitive processes (for reviews, see Niedenthal, Barsalou, Winkielman, Krauth-Gruber, & Ric, 2005; Semin & Smith, 2008) is directly at odds with the claim that “an examination of bodies and their workings will
18
never bring us to the subject matter of the mind at all” (Dennett, 1969, p. 189). Instead, the insights from grounded cognition are in line with the theoretical view that bodies shape our minds (e.g., Lakoff & Johnson, 1980; Varela, Thompson, & Rosch, 1991). Second, these studies reveal that symbolic computations are influenced by perceptual information. Whereas in computational models of cognition “the symbolic codes themselves do not specify their intended interpretation (and the model’s behavior is not influenced by such an interpretation)” (Pylyshyn, 1984, p. 42), research has shown that perceptual information does influence cognitive processes such as language comprehension (for a review, see below).
Despite the fact that many a-priori predictions from grounded approaches to cognition have received empirical support, grounded cognition has not been without its criticisms. Some researchers have argued that the activation of sensorimotor information does occur during conceptual processing, but only to contribute to the ‘full’ representation of a concept (see Mahon & Caramazza, 2008). As such, sensorimotor information is theorized to complement amodal symbolic representations. The current empirical investigations into the relationship between perceptual information and conceptual processing do not provide a univocal answer to question whether conceptual processing necessarily relies on perceptual information. Nevertheless, the causal influence of manipulated bodily states and sensorimotor activation on language processing (e.g., Glenberg & Kaschak, 2002; Pulvermüller, Hauk, Nikulin, & Ilmoniemi, 2005) supports the idea that the activation of perceptual information has a more important function than simply enriching amodal representations (see Fischer & Zwaan, 2008; Zwaan, 2009). Concluding, neither the presence of modal symbols, nor the necessary role of sensorimotor activation in language comprehensions has received undisputed empirical support. Although these questions lie outside of the scope of the current thesis, the exact nature of mental representations and the possible limitations of grounded approaches to cognition remain important questions for further research.
19
A second source of criticism on grounded approaches to cognition has been the apparent impossibility to represent abstract concepts perceptually – which brings us to the topic of the current dissertation. Abstract concepts (e.g., virtue, morality, or time) have no perceptual characteristics, since they cannot be seen, heard, touched or directly perceived in any other sense. Amodal representations have therefore been argued to be the only logical answer to the question how abstract concepts are represented (Dove, 2009; Fodor, 1975; Mahon & Caramazza, 2008). Two possible approaches to accommodate perceptual representations of abstract concepts in grounded theories of cognition have been put forward in the literature. The first approach consists of metaphorically mapping abstract target concepts on perceptual source dimensions (Boroditsky, 2000; Lakoff & Johnson, 1980, 1999). The second approach suggests that abstract concepts can be represented in simulations of situations or actions (Barsalou, 2008b; Barsalou & Wiemer- Hastings, 2005; Glenberg & Robertson, 1999). Both these theoretical approaches propose that much of the conceptual content of abstract concepts is provided by either metaphoric representations or perceptual simulations, and is thus inherently perceptual. The possible objections against symbolic approaches to cognition such as the symbol grounding problem and the lack of empirical support for the existence of amodal symbols, together with the increasing amount of empirical support for the perceptual representation of concrete concepts, warrants the careful examination of possible ways in which abstract conceptual thought can be accommodated from a grounded cognition perspective.
Grounding Concrete Concepts
The literature regarding the grounding of concrete concepts is too broad to discuss in much detail in this dissertation, but in order to be able to appreciate the novel insights that grounded approaches to cognition have provided, I will describe several studies that have furthered our understanding of the nature of conceptual representations. In addition,
20
I will focus on several studies that have provided relevant insights for the question how abstract concepts are grounded, and on studies that have used paradigms that have subsequently been adapted to investigate the grounding of abstract concepts.
Recent well-designed studies provide a good example of the empirical support for the theoretical assumption that language comprehension automatically activates motor processes (Zwaan & Taylor, 2006; Taylor & Zwaan, 2008; for a review, see Fischer & Zwaan, 2008). In these studies, participants read sentences such as: ‘Because the music was too loud, he turned down the volume.’ Only one to three words of the complete sentence were presented on the screen at one time. Participants could read the entire sentence by rotating a knob, with each 5 degrees of rotation resulting in the presentation of the next segment. The reading-rotation direction either matched or mismatched the rotation direction implied by the sentence (e.g., turning down the volume consists of a counter-clockwise movement). When participants had to read the sentences describing counter-clockwise movement by moving the knob clockwise, they rotated to knob more slowly compared to participants who could read the sentence by rotating the knob counter-clockwise (Zwaan & Taylor, 2006), but only on the segment of the sentence that contained the verb. Apparently, reading a description of a movement incongruent to the movement people are currently performing slows down reading times. A subsequent study revealed this effect occurs as long as the action is within linguistic focus, and extends to the subsequent adjectives such as ‘opened slowly’ (Taylor & Zwaan, 2008).
There have been many more studies showing that semantic processing accesses the motor system. For example, participants were faster to categorize sentences toward the body (such as ‘open the drawer’) or away from the body (such as ‘close the drawer’) as sensible (vs. not sensible) when the button participants had to press required a congruent movement toward or away from the body (Glenberg & Kaschak, 2002). Pulvermüller (2005) showed that people were faster to identify leg-related words (compared to arm-related words) when then left-hemisphere leg
21
motor areas were stimulated using transcranial magnetic stimulation. When arm motor areas were activated, participants were faster to identify arm related words. These and similar studies all support the idea that motor areas in the brain are activated when people process words related to body movements, and words describing objects that can be touched and manipulated.
Beyond the motor domain, several studies provide support for the activation of modality specific brain areas when words or sentences related to visual or auditory domains are processed. Pecher, Zeelenberg and Barsalou (2003, 2004) investigated the assumption that perceptual properties of concrete objects are represented in modality specific areas in the brain. In a series of studies, they show that switching from a representation in one modality to a representation in a different modality takes time. Using a property verification task (“Is a lemon yellow, yes or no?”), they show that people are faster to make judgments about objects when the modality that is referred to in the question (e.g., the visual modality) is the same as in the previous question, than when the question refers to a different modality (e.g., auditory: “Do microwaves beep, yes or no?”). People were faster to indicate a lemon was yellow after judging an apple is green, compared to judging whether a lemon is yellow after judging that leaves are rustling. These findings are taken as support for the assumption that verifying the properties of concrete objects does not rely on amodal symbolic representations, but requires the activation of perceptual simulations. Although property verification tasks require explicit judgments about stimuli (which make them less conclusive about the spontaneous recruitment of modality specific representations) they support the assumption that different sensory domains are activated when answering questions about concrete concepts (see also Van Dantzig, Pecher, Zeelenberg, & Barsalou, 2008).
Processing concrete and abstract verbs has been shown to drive attention to specific spatial locations. In a norming study, Richardson, Spivey, Edelman, and Naples (2001) showed that people share an agreement about how verbs should be represented in space. For example,
22
‘give’ is represented in horizontal space, whereas ‘respect’ is represented in vertical space. Building on predictions from linguistics (Lakoff, 1987; Langacker, 1987; Talmy, 1983) and grounded cognition (Barsalou, 1999; Glenberg & Robertson, 2000), the spatial representations associated with these words were expected to be automatically activated during language processing in a set of studies by Richardson, Spivey, Barsalou, and McRae (2003). Processing a sentence containing a horizontally represented verb (e.g., give) was expected to activate visual representations in the left-right dimension, interfering with the recognition of a stimulus presented on the left or right side of the screen. Sentences containing vertically represented verbs (e.g., respect) were expected to interfere with the recognition of a subsequently presented stimulus up or down on the screen. The predicted findings were observed, and taken as support for the assumption that the same visual resources are used to represent the meaning of the sentence and to perceive the subsequent stimulus (for similar findings, see Estes, Verges, & Barsalou, 2008).
These results have subsequently been replicated for concrete verbs, but not for abstract verbs (Bergen, Lindsay, Matlock, & Narayanan, 2007). Furthermore, this last set of studies revealed that words associated with vertical or horizontal space were not enough to lead to interference effects, but that the results were caused by mental imagery. Importantly, building a mental simulation can also facilitate the processing of visually congruent information (Glenberg & Kaschak, 2002; Kaschak et al., 2005; Stanfield & Zwaan, 2001; Zwaan, Stanfield, & Yaxley, 2002). The critical difference between interference and facilitation seems to be the length of time between the presentation of the linguistic stimulus and the perceptual stimulus. If the visual brain areas are still busy representing the linguistic stimulus (e.g., between 50 to 200 milliseconds) then interference is more likely, but after the visual brain areas are no longer active, the previous visual representation of vertical or horizontal locations might facilitate recognition (for a discussion, see Bergen et al., 2007).
In addition to the effects of language processing driving visual attention, theoretically related studies have investigated the opposite
23
relationship between visual perception and language processing. When participants were asked to judge whether two words were semantically related, their relative positions on the screen influenced the speed with which these judgments were performed (Zwaan & Yaxley, 2003). More precisely, when the relative position of these words was congruent with their typical locations in the real words, judgments were faster compared to when the words were presented in relative spatial positions incongruent with their typical locations. When the words branch and root were presented on the screen, participants were faster to judge the words were related when branch was presented vertically above root, compared to when root was presented above branch. Similar results have been shown when only one word (e.g., eagle) was being presented either on the top or the bottom of the screen (Šetić & Domijan, 2007). In these studies, words are assumed to activate perceptual representations, with attention being focused on the respective regions. The relative spatial position of the words is either congruent or incongruent with the presentation of the stimulus words, thus resulting in a spatial Stroop effect (Lu & Proctor, 1995). In further support of this reasoning, similar effects are not observed when pictorial (compared to verbal) stimuli of objects typically seen up or down in the world are processed, since these pictorial stimuli do not have to be visualized to be represented (Verges & Duffy, 2009). A similar reasoning and paradigm has been used extensively in abstract domains (e.g., Meier & Robinson, 2004; Schubert, 2005).
Taken together, the many empirical demonstrations of sensorimotor involvement in thoughts about concrete concepts reveal that conceptual processing and sensorimotor information are intimately related. Important questions still remain, such as the question whether motor activation is necessary for language understanding, or whether the activation of sensorimotor systems during semantic processing is optional (Fischer & Zwaan, 2008; Meteyard & Vilgliocco, 2008; Zwaan, 2009). In addition, people can sometimes rely on purely linguistic information to understand the meaning of words, without drawing on sensorimotor brain areas (see for example Andrews, Vigliocco, & Vinson,
24
2009; Boroditsky & Prinz, 2008; Simmons, Hamann, Harenski, Hu, & Barsalou, 2008). Nevertheless, the grounded cognition approach seems to be a fruitful theoretical framework to investigate conceptual processing – both for concrete as abstract concepts.
Grounding Abstract Concepts
Highly abstract concepts such as morality, truth, and beauty cannot be seen or heard. Because such abstract concepts lack perceptual characteristics, some researchers have argued that “perceptual representations seem ill-suited for representing abstract concepts” (Dove, 2009). The much debated question of whether and how abstract concepts are represented perceptually remains a challenge for grounded approaches to cognition (Arbib, 2008; Barsalou; 2008; Boroditsky & Prinz, 2008; Dove, 2009; Lakoff, 2008; Mahon & Caramazza, 2008). If grounded theories of cognition fail to account for the perceptual representation of abstract concepts, grounded cognition would be a rather limited framework for conceptual thought, applying exclusively to concrete concepts (Mahon & Caramazza, 2008). Therefore, providing a theoretical account of how abstract concepts are grounded, as well as empirical support for the perceptual representation of abstract concepts are both essential for a more mature grounded cognition framework.
Several theoretical proposals have been made to accommodate abstract concepts in grounded approaches to cognition (Barsalou, 1999, 2008; Barsalou & Wiemer-Hastings, 2005; Boroditsky, 2000; Glenberg & Robertson, 1999; Gallese & Lakoff, 2005; Lakoff & Johnson, 1980, 1999; Meier & Robinson, 2004; Richardson, et al., 2003). Although the perceptual representation of abstract concepts has received less empirical attention than the perceptual representation of concrete concepts, the predictions drawn from these theories have been tested in a decent number of studies. These theoretical frameworks are often complementary (Glenberg, Sato, Cattaneo, Riggio et al., 2008), and focus on the simulation of perceptual information, relating abstract events to
25
actions, and metaphoric representation. According to the simulation account (Barsalou, 1999, 2008a;
Richardson et al., 2003) people create images of situations, objects and introspections to perceptually simulate specific instances of abstract concepts. Through selective attention, an unlimited number of simulators can develop for objects (e.g., “car”), properties (e.g., “red”), relations (e.g., “above”), actions (e.g., “driving”) or introspective states (e.g., “fear”). These simulators makes the implementation of symbolic operations possible (for an overview, see Barsalou, 2008b). Even abstract concepts such as ‘truth’, which seems to lack any perceptual characteristics to build a simulation of, can be perceptually simulated in such a way (Barsalou, 1999). To verify the truth of a proposition (e.g., “the balloon is above the cloud”) people create and image of a balloon above a cloud and subsequently compare whether this image corresponds to their experience. After repeated experiences of this comparison process, a simulator develops which grounds the concept of truth.
Another proposal aimed at explaining how abstract concepts are grounded focuses on how certain abstract events can be grounded in movements. The indexical hypothesis (Glenberg & Robertson, 1999) proposes that sentences are understood by creating a simulation of the actions that underlie them. It seems to matter little whether these actions are concrete (e.g., ‘giving a book’) or abstract (e.g., ‘giving information’). Glenberg and Kaschak (2002) asked people to judge the sensibility of sentences describing the transfer of concrete objects such as “Andy delivered the pizza to you/You delivered the pizza to Andy” and abstract information transfer, such as “Liz told you the story/You told Liz the story”. Participants responded by moving their hand from a middle button to either a button farther away or closer by. Judgments were faster when the action implied by the sentence matched the action required to make the response, and this was true for both the concrete and the abstract transfer sentences. A similar study showed that moving 600 beans from one container to the other (either away or towards the body) influenced sentence comprehension in a subsequent task of sentences describing
26
concrete and abstract transfer away from or towards the body (Glenberg, Sato, & Cattaneo, 2008).
A final approach to grounding abstract concepts draws upon the seminal work by Lakoff and Johnson (1980, 1999), who have proposed that abstract concepts are grounded in concrete experiences through metaphoric representations. People use metaphors abundantly when they talk about abstract concepts. The Conceptual Metaphor Theory (CMT) introduced by Lakoff and Johnson (1980, 1999) holds that metaphors are not merely conventional ways to talk about abstract concepts, but that abstract conceptual thought is inherently metaphorical (cf. Richards, 1936). According to CMT, people use concrete experiences (or sources) to think about more abstract concepts (or targets). When concrete experiences in the source domain repeatedly co-occur with more abstract experiences in the target domain, the two brain areas that are activated together will over time form neural mappings (Lakoff, 2008), which constitute so-called ‘primary metaphors’ (Grady, 1997).
An example of such a primary metaphor is ‘happy is up’. Lakoff and Johnson (1980) argue that positive states are usually accompanied by erect postures, whereas negative states coincide with a drooping posture, thereby experientially grounded the relationship between the concrete domain of vertical space and the abstract domain of valence. The assumption of CMT that abstract thought is built on concrete experiences is relatively undisputed (for similar reasoning, see Clark, 1973; James, 1890; Piaget, 1954; Richards, 1936; Whitney, 1875). CMT has inspired many researchers to investigate whether abstract concepts are represented perceptually. Although CMT provides compelling theoretical arguments for a strong relation between concrete experiences and abstract concepts, even the updated neural theory of metaphor (Feldman, 2006; Lakoff, 2008) fails to clearly specify how basic image schemas, personal experiences, contextual variations, and overall knowledge collectively determine which neural mappings are activated in which situation. This makes it difficult to predict how abstract concepts will be grounded.
Conceptual Metaphor Theory has previously been criticized for
27
lacking a more detailed model of how the relationship between concrete concepts and abstract concepts is established (Murphy, 1996; Boroditsky, 2000; McGlone, 2007). The recent neural theory of metaphor put forward by Lakoff (2008) is the most explicit account of the processes underlying CMT so far, and proposes that neural mappings between concrete experiences and abstract concepts emerge through peoples’ experiences, and these neural mappings are combined into larger neural circuits that allow us to ground metaphoric language. However, two criticisms that have been put forward previously still apply (e.g., Murphy, 1996), namely the strong focus on linguistic evidence (see also Casasanto, in press; Crawford, 2009) and the lack of detail of the theory, which makes it difficult for psychological researchers to derive clear testable predictions (Boroditsky, 2000). Two interpretations of CMT have been put forward that allow the underlying assumptions to be tested empirically. Although both views are derived from CMT, they highlight different aspects of the processes assumed to underlie the metaphoric representation of abstract concepts. Below, I will introduce these two views in greater detail. Chapter 4 provides a set of studies that aim to test the assumptions of these different views on the metaphorical representation of abstract concepts against each other.
Metaphoric Structuring View
The Metaphoric Structuring View was developed by Boroditsky (2000) to provide an empirically testable theory derived from the work on CMT by Lakoff and Johnson (1999). The main assumption of the Metaphoric Structuring View is that metaphors ‘provide relational structure to those domains where the structure may not be obvious from world experience’ (Boroditsky, 2008, p. 3). Abstract domains are argued to lack a clear relational structure. For example, whereas moments in time seem to appear, disappear and never return, time does not contain concrete information about where moments in time come from and where they go to. People can move through time themselves, or time can approach
28
them. Time can flow from the left to the right, or from our front to back. Metaphors allow us to structure, among others, the abstract concept of time by mapping time onto concrete dimensions such as space. According to the Metaphoric Structuring View, metaphors in language such as ‘looking forward to next week’ influence how we structure time in space.
Boroditsky (2000) suggests that metaphors have shaped the way we think about time, but after frequent use, metaphors can lead to independent representations that no longer rely on spatial schemas. Which specific metaphoric representation of time is used depends upon the on- line mapping of spatial schemas onto the domain of time. Priming people with an ego-moving spatial scenario (where objects move through space) facilitates responses for time questions framed in an ego-moving schema (compared to a time-moving scenario, where time approaches objects). Similar flexibility in the spatial representation of time was shown in a cross-cultural study comparing Mandarin speakers with English speakers (Boroditsky, 2001). In Manderin there is a vertical metaphor for time (comparable to the uncommon English metaphor of a meeting ‘coming up’ and ‘handing down’ knowledge from generation to generation, see also Lindner, 1983), and thus Chinese speakers tend to represent time as moving from the top to the bottom. Importantly, English speakers trained in the vertical representation of time showed a similar influence of spatial primes on time questions as Manderin speakers, revealing that metaphoric representations can be learned.
The Metaphoric Structuring View highlights the importance of the spatial schemas used to think about time, and shows that metaphoric representations of abstract concepts are highly contextualized. Different representations of time can be primed (Boroditsky, 2000), trained (Boroditsky, 2001) or changed depending on peoples’ mindset (Boroditsky & Ramscar, 2002), and subsequently influence how time is structured in space. This suggests that different representations of abstract concepts exist and can be used to structure the abstract domain. Such flexibility in the spatial representation of time is difficult to account for by more automatic views on metaphoric representation such as the neural theory
29
of metaphor, which largely rely on stable neural connections between concrete experiences and abstract concepts (Feldman, 2006; Lakoff, 2008).
Further support for the flexibility of spatial representations comes from a recent set of studies revealing that whereas the front-back representation of time is the default representation, introducing left and right response keys activates the left-right representation of time (Torralbo, Santiago, & Lupiáñez, 2006). These authors argue that attention plays an important role in the flexible on-line representation of abstract concepts such as time. After abstract concepts and concrete dimensions have repeatedly been mapped onto each other, these mappings are stored in long-term memory (cf. Boroditsky, 2000). These mappings are subsequently used when people try to construct the most parsimonious mental model (cf. Johnson-Laird, 1983) to support conceptual processing.
Stroop-like Interference Effects
A more recent translation of Conceptual Metaphor Theory to an empirically testable hypothesis was initiated by Meier, Robinson and Clore (2004). They reason that if the neural networks that are active when people represent abstract concepts are intimately bound with the neural networks that are active when people represent sensory experiences (Lakoff, 2008; Lakoff & Johnson, 1999), there should be an obligatory and automatic relationship between perceptual information and abstract concepts. Lakoff (1993) appears to endorse this view when he suggests that the system of conceptual metaphors ‘‘is used constantly and automatically, with neither effort nor awareness’’ (pp. 227–228).
Meier and colleagues (2004) investigated whether the abstract concepts of positive and negative valence are metaphorically represented in the concrete experiences of light and dark. Their results reveal that positive words presented in a white font and negative words presented in a black font word were evaluated faster and more accurately compared to positive words presented in a black font and negative words presented in a white font. In other words, metaphor congruent valence-brightness
30
combinations were processed more efficiently than metaphor incongruent valence-brightness combinations.
Metaphor congruency effects are argued to be the result of a Stroop-like interference effect. Meier et al., (2004, p. 86) argue that: “a negative word presented in white would give rise to two response tendencies, one to respond ‘negative’ (on the basis of stimulus valence) and one to respond ‘positive’ (on the basis of stimulus color).” A similar reasoning is provided by Schubert (2005), who investigated the vertical representation of power. He asked participants to categorize words referring to powerful and powerless groups on power, and found that words referring to powerful groups presented on the top of the screen were categorized faster compared to when these words were presented on the bottom of the screen. According to Schubert (2005, p. 18), “The response that a presented group is powerful or powerless has a certain threshold, and evidence is accumulated until that threshold is reached. The evidence comes from multiple dimensions, and vertical position of the group label and vertical motor images are among these dimensions.”
According to this Stroop-like interference explanation for metaphor congruency effects, which I shall refer to as the one-to-one grounding explanation, the abstract concept of positivity is grounded in the sensory experience of seeing a light stimulus, and negativity is grounded in the sensory experience of darkness. As such, seeing the color white is assumed to automatically activate its associated valence (e.g., positivity), which in turn can facilitate or interfere with the categorization of the conceptual meaning of the word (see also Schubert, 2005; Meier & Robinson, 2004, Sherman & Clore, 2009). These brightness-valence associations are argued to be unavoidable at the stage of word encoding, and point to “the obligatory nature of affective inferences based on stimulus brightness” (Meier et al., 2004, p. 85; see also Meier & Robinson, 2004).
The one-to-one grounding explanation does not yet provide a detailed explanation of when and how perceptual information influences abstract conceptual processing, and cannot account for all observed
31
empirical results. For example, Schubert (2005) shows that words such as ‘dictator’ or ‘tyrant’, which are powerful and negative, are processed faster on the top of the screen when participants have to categorize these word on power, but are processed faster on the bottom of the screen when these words are categorized on the basis of their valence. Metaphor congruency effects seem to depend upon the current categorization goals, speaking against a purely automatic activation of word meaning. Relating metaphor congruency effects to Stroop-like interference effects is fitting in this respect, because the assumption that Stroop-like effects are completely automatic does not seem to be the most parsimonious explanation for the effects observed in the literature (see MacLeod, 1991). For example, adding additional irrelevant words to the task reduces Stroop interference effects (Kahneman & Chajczyk, 1983), which would not be expected if the process is completely automatic.
Although the processes underlying the metaphoric representation of abstract concepts remain debated (but see Chapter 4 for a set of studies which experimentally tests the one-to-one grounding approach against the metaphoric structuring view), there is a growing amount of studies that have investigated the relationship between perceptual information and abstract conceptual processing. Below, I will review these studies, whereby I will focus primarily on the three abstract concepts under investigation in the three chapters of this thesis, namely morality, time, and valence.
Morality
Morality is one of the hallmark abstract concepts for which it is argued that there is “no sensory or motor information that could correspond in any reliable or direct way to their meaning” (Mahon & Caramazza, 2008). The assumed lack of perceptual characteristics of abstract concepts is one of the most prevalent criticisms on perceptual theories of cognition. After all, how can people perceptually represent abstract concepts which they have never seen or heard?
32
Some researchers have proposed that moral concepts cannot be grounded perceptually (Andrews, et al., 2009; Dove, 2009; Mahon & Caramazza, 2008). Some of these researchers have proposed that moral concepts can only be grounded in the emotional responses they cause in us (Boroditsky & Prinz, 2008; Prinz, 2005). Indeed, many studies have revealed that immorality is strongly associated with feelings of disgust. For example, the olfactory experience of bad smell influences moral judgments (Schnall, Haidt, Clore, & Jordan, 2008). Other studies have revealed that morality and cleanliness are strongly related. Thinking about an unethical deed makes cleaning related words more accessible and reading an unethical story increases the desire for cleaning related products (Zhong & Liljenquist, 2006). Activating the concept of cleanliness before reading moral dilemmas, as well as washing ones hands after seeing a disgusting movie clip, reduces the severity of moral judgments (Schnall, Benton, & Harvey, 2008). Taken together, there is abundant support for a strong relationship between experienced disgust and moral judgments.
Intuitive approaches to morality (Haidt, 2001; 2008) have highlighted the importance of perceptual representations of morality together with affective responses to moral situations. Building on the work by Lakoff & Johnson (1999), Haidt (2001, p. 285) argues that moral concepts are “built up largely by metaphorical extensions from physical experience.” Several other researchers have argued in favor of the perceptual representation of moral concepts (Barsalou, 2008b; Gibbs, 1996; Haidt, 2001; Johnson, 1993; Meier, Robinson, & Wygant, 2007; Lakoff & Johnson, 1980, 1999; Sherman & Clore, 2009). Both the simulation approach (Barsalou, 1999; Barsalou & Wiemer-Hastings, 2005) as the metaphoric representation approach (Lakoff & Johnson, 1999) provide possible suggestions how moral concepts might be represented perceptually. According to the metaphoric representation approach, there are multiple concrete sources which are commonly used to represent moral concepts metaphorically (for an overview, see Lakoff & Johnson, 1999, chapter 14). Morality is seen as pure, straight, upright, balanced, and clean, for example, whereas immorality is seen as contaminated,
33
crooked, low, unbalanced and dirty. Two sets of studies have provided some initial empirical support for these assumptions.
Meier, Sellbom, and Wygant (2007) investigated whether morality is represented on a vertical dimension. They presented both moral and immoral words on the top or the bottom of the screen (cf. Meier and Robinson, 2004; Schubert, 2005) and found that responses were faster in the metaphor congruent conditions (moral-up and immoral-down) compared to the metaphor incongruent conditions (moral-down and immoral up). The results are not exactly in line with the predictions, in that immoral words on the bottom of the screen were responded to slower than all other conditions, for example. Furthermore, it is unclear to what extent these effects differ from the vertical representation of valence (Meier & Robinson, 2004; Crawford, Margolies, Drake, & Murphy, 2006).
Sherman and Clore (2009) adapted the Stroop paradigm, and found that participants were faster to indicate the font color of a word was white when the word had a moral (compared to immoral) meaning. Their first study is a re-analysis of Meier and colleagues’ (2004) Study 4, which originally was designed to test the a-priori prediction that brightness categorizations would not be influenced by the valence of the stimuli. Meier, Robinson, and Clore (2004, p. 83) assumed that the interaction between valence and brightness would be asymmetric, in that “color would interact with valence judgments (Studies 1, 2, and 3), whereas valence would not interact with color judgments (Study 4)”. Such a prediction is in line with both theoretical views on the metaphoric structuring of abstract concepts based on concrete concepts (Lakoff & Johnson, 1980) and empirical findings revealing an asymmetry between the concrete source and abstract target in metaphoric representations (Boot & Pecher, in press; Boroditsky, 2000; Casasanto & Boroditsky, 2008). Nevertheless, in the studies by Sherman and Clore (2009), the abstract meaning of the words did influence categorizations of the color in which the words were presented. The authors offer no theoretical explanation to reconcile the difference between their findings and previous empirical observations. In addition, it is not specified in which way these results go beyond
34
the previously reported more general valence-brightness relationship (Meier et al., 2004). These studies show the necessity of investigating the relationship between perceptual information and abstract conceptual processing from an integrative theoretical framework. The question when and why metaphoric representations are asymmetric is surely an interesting one, and further insights into this question will improve our theoretical understanding of the processes underlying metaphoric representations.
In Chapter 2, I take a new approach to the perceptual representation of morality by investigating how the equality component of morality is grounded in the ‘balance of justice’ metaphor. Equality is a central feature of morality in philosophy (e.g., Rawls, 1971), anthropology (e.g., Fiske, 1992), folk-theories (e.g., ‘an eye for an eye’) and classical justice theories in psychology such as equity theory (Adams, 1965). The equality component of morality is theorized to be grounded in the balance image schema (Gibbs, 2005; Johnson, 1987, Lakoff, 1987) and conceptualized metaphorically as balance or symmetry (Lakoff, 2005). Many of our experiences with fairness or justice are directly related to balanced and symmetrical distributions, such as when parents cut a birthday cake into equal pieces. Given this strong experiential association between equal divisions and justice, moral concepts should be associated with perceptual symmetry. If peoples’ understanding of morality and justice is experientially grounded in symmetrical distributions, then thinking about morality should activate non-verbal associations with perceptual symmetry.
The studies in Chapter 2 investigate whether moral words are associated with symmetrical stimuli, and immoral words are associated with asymmetric stimuli. Participants were asked to indicate which of two Chinese ideographs presented on the screen correctly translated the Dutch stimulus word. After this translation task, participants rated all ideographs on symmetry. The results revealed that participants had chosen more symmetric ideographs over more asymmetric ideographs when translating moral words. The reverse was true for immoral words.
35
A second study extended these results to judgments where the moral or immoral meaning of the word was unrelated to the judgments participants had to perform. Participants were asked to indicate how many of the letters of previously presented moral and immoral words had been blue. In line with the assumption that morality is associated with symmetry, participants displayed biased recall of the number of blue and green letters of previously presented words, indicating that the number of blue letters had been more equally distributed for moral words compared to immoral words.
Time
The perceptual representation of time has been investigated more thoroughly than any other abstract concept (Boroditsky, 2000; Boroditsky, 2001; Boroditsky & Ramscar, 2002; Casasanto & Boroditsky, 2008; Santiago, Lupiáñez, Pérez, & Funes, 2007; Torralbo et al., 2006; Tversky, et al., 1991; Vallesi, Binns, & Shallice, 2008; Weger & Pratt, 2008; Zacks & Tversky, 1999). These studies show that time can be represented in the back- front, up-down and left-right spatial dimensions. The overlap between spatial and temporal representations is also present at a neural level, with common brain areas being active during the processing of spatial and temporal information (Basso, Nichelli, Frassinetti, & Di Pellegrino, 1996; Bjoertomt, Cowey, & Walsh, 2002; Rao, Mayer & Harrington, 200People often use space to talk about time (e.g. ‘The worst is behind us’ and ‘We are approaching the holidays’). Boroditsky (2000) reasoned that if people use space to talk about time, they might also use space to think about time. If this reasoning holds, then influencing the way people think about space should influence how they think about time. The literature differentiates between two reference frames to think about time in terms of space. One the one hand, people can move through time (the ego- moving frame of reference), on the other hand, time can move toward us (the time-moving frame of reference). Participants were presented with an ambiguous temporal statement (e.g. `Next Wednesday’s meeting has
36
been moved forward two days’). In English, this statement is just as likely to mean that the meeting has been moved to Monday, as that it has been moved to Friday. From an ego-moving frame of reference, where people move through time, moving forward means the meeting will now be on Friday. From an object-moving perspective, where things move through time towards us, the meeting should now be on Monday (McGlone & Harding, 1998).
Activating these respective temporal frameworks by presenting pictures of objects moving through space toward the individual (time moving perspective), or the individual moving through space (ego- moving perspective) makes people respond to this temporal statement in a metaphor consistent manner (Boroditsky, 2000). A subsequent set of studies (Boroditsky & Ramscar, 2002) found that the answer people gave to the previously mentioned ambiguous question depended on whether people had moved through space themselves (e.g., when flying in an airplane, moving through the lunch line, or on a train), with the more they moved themselves, the more they answered the question using an ego-moving frame of reference.
Spatial primes influence thoughts about time differently for Manderin speakers than for English speakers. In Chinese, time can be represented as moving downward through space, in addition to the left- to-right representation of time in space prevalent in Western culture. Due to this cultural difference, priming participants with either horizontal spatial differences (e.g., a picture where a black worm is positioned to the left of a white worm) or vertical spatial differences (e.g., a picture where a black ball is placed above a white ball) influences temporal judgments (e.g., ‘March comes earlier than May’) in a different manner for English speakers than for Mandarin speakers (Boroditsky, 2001).
Time can also be represented from left (past) to right (future). Santiago, Lupiáñez, Pérez, and Funes (2007) manipulated both the position (left vs. right) on the screen of past and future related words, as the key assignment with which these words had to be categorized. Participants had to press the left key for past words and the right key for
37
future words, or they were instructed to use the reversed key assignment. Both spatial position and key assignment influenced reaction times, but these effects were notably stronger for the latter manipulation. The fact that participants were faster to correctly categorize future related words by pressing the right key, and past related words by pressing the left key, than the reverse, is a strong indication of the representation of the past and future from the left to the right. The importance of task instructions in categorization tasks such as these was further exemplified in two studies where participants responded either verbally (categorizing past and future related words through vocal responses) or manually (categorizing past and future related words by pressing a left or right response key). When answering verbally, the back (past) to front (future) representation of time was most active, but when answering manually, the left-right representation of time was most salient (Torralbo, Santiago, & Lupiáñez, 2006).
The left-to-right representation of time has been investigated more closely in two sets of studies (Vallesi, et al., 2008; Weger & Pratt, 2008), in which the left-to-right representation of time is approached from a spatial stimulus-response compatibility view (for reviews, see Hommel & Prinz, 1997; Umiltà & Nicoletti, 1990). Weger and Pratt (2008) replicate earlier findings supporting the left-to-right representation of time, while using less explicitly time related stimuli (namely names of actors such as Charlie Chaplin or Brad Pitt). In addition, they show that past related words presented in the center of the screen facilitate recognition of target on the left of the screen, whereas future related words facilitate the recognition of targets on the right side of the screen. This effect was stronger when participants had to respond to the left target with their left hand, and to the right target with their right hand (Study 2A), than when left or right targets had to be responded to by pressing the space bar, and filler trials had to be ignored (Study 2B). They conclude from this comparison of a bimanual categorization task versus a go-no-go paradigm, that the results are “primarily - although not necessarily exclusively - due to a facilitation of response codes, rather than to an impact on stimulus processing”
38
(Weger & Pratt, p. 429). Vallesi and colleagues (2008) asked participants to indicate how
long a fixation cross appeared on the screen (which was presented for one vs. three seconds). Inspired by the spatial representation of numbers, with small numbers being represented on the left and larger numbers on the right, which is also known as the SNARC effect (Dehaene, Bossini, & Giraux, 1993), these researchers investigated whether short time periods (one second) would shift attention to left-hand responses and longer time periods (three seconds) would shift attention to right-hand responses. Indeed, left-hand responses were slower when participants had to indicate the fixation cross had been presented for three seconds (vs. one second), whereas right-hand responses were slower when indicating the fixation cross had been presented for one second (vs. three seconds). The authors conclude that this interference emerges somewhere after response selection and during response preparation. Furthermore, the authors argue that the spatial representation of time is not easily activated automatically, but requires the awareness of the passage of time (for example, by making time task-relevant).
The relationship between time and space has been shown to be asymmetric, in that spatial primes influence judgments about time, but temporal primes do not influence judgments about space (Boroditsky, 2000, Study 2 and 3). Spatial primes which activate the ego-moving or object-moving frames of reference influence thoughts about time, but temporal primes did not influence thoughts about space. This asymmetry was investigated in more detail in a paradigm where participants were presented with growing lines on a computer screen, and were asked to judge the length of these lines or the duration the lines were presented. The length of the lines influenced judgments about the time the lines had been presented on the screen, but the time the stimuli were presented on the screen, did not influence judgments about the length of the lines (Casasanto & Boroditsky, 2008). Participants were unable to ignore irrelevant spatial information when they performed judgments about duration, but not the other way around.
39
Research on the metaphoric representation of abstract concepts has almost exclusively focused on the visual modality, and when people talk about the perceptual grounding of abstract concepts, they implicitly seem to equate this to visual grounding (e.g., Crawford, 2009). However, there is no reason to assume that the spatial structuring of abstract concepts is limited to the visual modality. In principle, any modality that is sensitive to spatial information should be able to structure abstract concepts. Chapter 3 provides empirical support for this assumption by revealing how time is metaphorically structured in the left-to-right auditory dimension. Participants listened to auditory stimulus consisting of past and future related words over headphones. Their task was simply to indicate in which ear they judged the stimulus to be presented loudest. In most of the trials, there was an objective difference in loudness, but in the critical experimental trials, the words were presented equally loud over both channels of the headphone.
As predicted, participants chose the right channel more often for future words than for past related words, when the auditory stimuli were presented equally loud to both ears. Furthermore, the average number of right ear judgments for each stimulus overlapped significantly with the spatial position of the items on a horizontal line as measured during a visual pilot study. This is strong support for the multi-modal representation of time in space. Note that the temporal meaning of words influenced loudness judgments even when their meaning was unrelated to the task participants had to perform, speaking against the view that the temporal meaning of the words should be task relevant for the spatial representation of time to be activated (Vallesi et al., 2008).
Valence
The metaphorical representation of positivity and negativity has revealed that the same abstract concept can be structured in many different concrete dimensions (for reviews, see Crawford, 2009; Meier & Robinson, 2005). The first demonstration of congruency effects in the abstract domain of
40
valence was provided by Meier, Robinson and Clore (2004) who asked participants to evaluate positive and negative words, while presenting these words in either a black or a white font. Although the color of the words was irrelevant to perform the task correctly, Meier and colleagues reasoned that based on the widespread use of black and white as a metaphor for evil and good the irrelevant perceptual characteristics of the stimuli would influence the speed and accuracy with which participants could evaluate the words. Their results confirmed this hypothesis: participants were faster and/or more accurate to categorize positive words written in a white font and negative words in a black font, than to categorize negative words in a white font and positive words in a black font.
In further support of the relationship between valence and brightness, priming participants with positive or negatively valenced words has been shown to bias subsequent brightness judgments (Meier, Robinson, Crawford, & Ahlvers, 2007). Identical grey cubes were judged to be lighter when judged after a positive (vs. negative) word had briefly been presented on the screen. This result was replicated using a response- deadline procedure, where participants were forced to judge whether a grey cube presented on the screen was the lighter or darker cube of two previously presented stimuli. The fact that a similar pattern of results is observed when using a response deadline procedure is commonly interpreted in support of the assumption that valence and brightness are automatically associated. Of interest is that in none of these studies participants were actually asked to respond to the positive or negative stimuli presented before the brightness judgments were made.
Other studies have shown congruency effects for positive and negative words in different visual domains. In addition to the association between valence and brightness, Meier and Robinson (2004) found that the categorization of valenced words was influenced by the vertical position of the stimuli. In line with the metaphoric representation of ‘happy’ being up, and ‘sad’ being down (Lakoff & Johnson, 1980), positive words were classified faster when presented on the top of the screen than on the bottom, and negative words were classified faster when presented
41
on the bottom of the screen than on the top. In subsequent studies Meier & Robinson (2004) showed that
positive words direct attention upwards and negative words direct attention downward (similar to related effects in concrete domains, e.g., Bergen et al., 2007; Estes et al., 2008; Richardson et al., 2003). People were faster to indicate whether a letter presented on the top of the screen was a p or a q after evaluating a positive word, than after evaluating a negative word. The reverse was not true: after indicating whether a stimulus (+++) appeared on the top or the bottom of the screen, there were no differences in the speed with which positive or negative words are classified. Although the absence of differences is troublesome to interpret, the authors argue that the relationship between verticality and valence is asymmetrical: evaluations activate spatial positions, but spatial positions do not activate evaluations. These attentional effects were subsequently shown to be moderated by individual differences in affective states, with people higher in neuroticism being faster in detecting stimuli in the lower (versus higher) half of the screen (Meier & Robinson, 2006).
Further support for the relationship between verticality and valence was observed in studies which investigated whether the valence of pictures would influence judgments about the location where these pictures were previously observed (Crawford, et al., 2006). Positive pictures showed an upward bias when participants tried to indicate where these stimuli were presented previously. The authors suggest that the relationship between valence and vertical position might reflect the usefulness of the vertical spatial domain to organize and structure more abstract domains (cf. Gattis, 2001), without necessarily being the result of a direct experiential associations between positive mood and an upright position (Lakoff & Johnson, 1980).
Valence can also be represented in size, following the “more is better” metaphor (Lakoff & Johnson, 1999). Meier, Robinson and Caven (2008) presented words in a large or small font, and asked participants to evaluate the words as fast and accurately as possible. Participants were slower to evaluate negative words written in a big font compared
42
to negative words in a small font. In another study the authors show that words presented in a large font are rated more positively than words presented in a small font. The authors conclude that “stimulus size leads to relatively automatic inferences concerning stimulus valence”.
A last set of studies to investigate how abstract concepts are grounded in the vertical dimension looked at representations of affectively charged divine concepts (Meier, Hauser, Robinson, Friesen, & Schjeldahl, 2007). These studies showed the concept of God is strongly associated with up vertical locations, but gives less handholds to assume the vertical representation of God is grounded in sensory-motor experience (e.g., looking up when praying). If the vertical representation of God is grounded in sensory-motor experiences, the effects should be stronger for people who look up when praying (e.g., Christians) compared to people who never pray. However, the religion of the participants did not moderate the observed effects.
Chapter 4 focuses on the brightness-valence metaphor. Instead of merely demonstrating this metaphoric mapping, the studies in this chapter aim to further our understanding of the processes through which the brightness-valence mapping emerges. Two process explanations, the metaphoric structuring view and the one-to-one grounding explanation (described earlier in this chapter) are tested against each other. According to the one-to-one grounding account for metaphor congruency effect, brightness automatically activates positivity, and darkness automatically activates negativity. According to the metaphoric structuring view, the perceptual opposition between white and black structure the conceptual opposition between good and bad. Whereas according to the metaphoric structuring view shared relational structures are an essential prerequisite for the brightness-valence mapping to emerge, the polar oppositions in the brightness and valence dimensions are irrelevant for the one-to- one grounding account. These two process explanations were directly compared by manipulating the endpoints of the valence and brightness dimensions within or between participants, thereby activating the shared polar oppositions in the perceptual and conceptual dimensions when
43
people performed the study, or breaking them apart. Previous studies in the literature, as well as two pilot studies, reveal
that white is rated as a neutral color. How can a neutral color metaphorically represent the meaning of positivity? Six studies investigated whether the positivity of white was dependent upon the presence of the negativity of black, and that the opposition between good and bad is metaphorically structured in the opposition between light and dark. A Chinese translation task, where participants were asked to indicate whether a white or black Chinese ideograph correctly translated the meaning of the word, revealed that whereas black is strongly associated with negativity, white in isolation is not used to represent positivity. The positivity of white emerges only in opposition to the negativity of black. Whereas negative words are always translated by black ideographs above guessing average, positive words are only translated above guessing average by white ideographs when both the valence as the brightness dimensions are manipulated within subjects. Furthermore, two subsequent studies show that the positivity of white emerges when activating white-black positive-negative opposition perceptually during the translation task or conceptually before participants performed the translation task. Together, these studies reveal that the shared relational structure in the concrete and abstract domains is an essential feature of the brightness-valence mapping, supporting the metaphoric structuring view. Metaphoric representations are at best conditionally automatic, and require knowledge of the relational structures in abstract domains.
Linguistic Explanations
As the above review reveals, there is no shortage of empirical studies that provide support for the relationship between perceptual information and abstract conceptual processing. Studies that have examined the exact nature of this relationship, however, are substantially scarcer. This criticism on the lack of process explanations within embodied cognition is not limited to studies investigating abstract concepts (e.g., Barsalou,
44
2008b; Zwaan, 2009), but is less pressing in the case of concrete concepts, where the automatic activation of sensorimotor information in conceptual processing has received behavioral (e.g., Zwaan & Taylor, 2006, Glenberg, Sato, Cattaneo, et al., 2008) and neurophysiological (e.g., Hauk et al., 2004) support. The question whether abstract conceptual processing necessarily activates sensorimotor areas in the brain or whether sensorimotor activation largely plays a peripheral role in conceptual processing remains unanswered. The relationship between perceptual information and abstract concepts might even be epiphenomenal, in that people perceptually represent concepts when possible, but that the sensorimotor activation is unrelated to conceptual processing.
How conclusive are the empirical results that have been observed so far in the domains of morality, time and valence with regard to the perceptual nature of abstract thought? Could these findings be explained based on associations between purely amodal symbols? The short answer is yes. There is no empirical support for the idea that sensorimotor activation is absolutely necessary for abstract conceptual thought, or even that thoughts about abstract concepts are limited without relying on perceptual representations. Therefore, it is impossible to confirm the hypothesis that abstract concepts are automatically structured in concrete dimensions, and reject alternative explanations. Beside this obvious limitation of the current empirical data, there are several other reasons to take a cautionary approach when interpreting the results of studies that have investigated the perceptual representation of abstract concepts.
First of all, the support for metaphoric representations is almost exclusively linguistic in nature (but see Boot & Pecher, in press; Casasanto & Boroditsky, 2008; Crawford et al., 2006). This poses a problem, because many conceptual metaphors are derived from language use, and subsequently tested by asking people to process abstract language (Casasanto, in press). This circularity problem (Crawford, 2009) is even more problematic because language encodes embodied relationships (Louwerse, 2008). For example, remember the findings by Zwaan & Yaxley (2003), revealing that participants were faster to confirm that the
45
words branch and root were related when branch was presented above root, compared to when these words were presented in the reverse spatial layout. Louwerse (2008) shows that the words branch and root occur more frequently in the order branch-root than in the root-branch order in human language. Furthermore, the more frequently these combinations occur in language, the higher participants judge the likelihood of encountering the first word (branch) above the other (root), when explicitly asked. Crucially, word order frequencies predicted reaction times in the semantic judgment task better than spatial iconicity did. These studies show that findings in reaction time studies that are currently explained from a grounded cognition perspective can be explained by linguistic statistical information, because pre-linguistic conceptualizations such as spatial iconicity are reflected in language (see also Cooper & Ross, 1975, Greenberg, 1963).
A highly related linguistic characteristic that has been argued to play a role in studies investigating the perceptual representation of abstract concepts is the notion of linguistic markedness. People have preferred ways to talk and think about dimensions. For example, the question “how long will it take?” is used more frequently than “how short will it take?” which carries additional meaning (in this case, the expectancy that something will not take long). Therefore “long” is called the linguistically unmarked category, and “short” is called the marked category (Greenberg, 1963). Typically the category that gives a dimension its name is unmarked (Clark, 1973), such as long (length), bright (brightness) or happy (happiness). In addition to being used more frequently, unmarked categories are processed faster than marked categories (Clark, 1969).
These differences in markedness are not restricted to linguistic stimuli but extend to non-verbal stimuli such as arrows or left-right responses (for an overview, see Proctor & Cho, 2006). Finally, differences in markedness have also been observed for abstract relationships between people. De Soto, London and Handel (1965) were the first to show that the amount of errors people make when answering syllogisms about the relationship between two individuals depends on whether the words
46
‘better’ or ‘worse’ are used. Although ‘A is better than B’ and ‘B is better than C’ contains the same information as ‘B is worse than A’ and ‘C is worse than B’, the ‘better’ statements led to more accurate judgments about the relationship between A and C than the ‘worse’ statements. To acknowledge this more general response facilitation for the default endpoint of a category, unmarked categories (and their non-verbal counterparts) are referred to as + polarity, while marked categories are referred to as – polarity (see also Gattis, 2001).
Some differences in markedness have been attributed to the nature of the physical body (Benor & Levy, 2006; Clark, 1973; Krzeszowski 1997; Mayerthaler, 1980). Because humans have eyes in their heads, look to the front, and stand upright, up, front and verticality are unmarked. Differences in polarity have been argued to be essential aspects of image schema’s (Hampe, 2005; Krzeszowski, 1997), and most image schema’s (e.g., up- down, front-back, right-left) are indeed all +polar and –polar (Grady, 1997; Lakoff & Johnson, 1999). Furthermore, conceptual metaphors always map +polar source domains (e.g., up) onto +polar target domains (e.g., happy). Because the +polar endpoints are categorized faster than – polar endpoints (Clark, 1969), differences in linguistic markedness have often been discussed as an alternative explanation for response facilitation effects of metaphor congruent stimuli (e.g., Schubert, 2005; Vallesi et al., 2008; Van Dantzig, Zeelenberg, & Pecher, 2009; Weger & Pratt, 2008).
In addition, many studies have noted the difficulties of interpreting observed interactions between perceptual and abstract domains, given the frequent observations of two main effects, with the unmarked endpoints of both dimensions being categorized faster than the unmarked endpoints (Boot & Pecher, in press; Meier, Hauser, et al., 2007; Meier, Sellbom, et al., 2007). These two main effects could account for the observed interactions through the polarity correspondence principle (Proctor & Cho, 2006). The relative contribution of differences in markedness in studies investigating the representation of abstract concepts has not been examined in detail, but distinguishing linguistic markedness effects from true perceptual representations should be possible (cf. Louwerse, 2008) and deserves
47
greater attention.
Structural Explanations
Whereas differences in linguistic markedness might not directly reflect perceptual influences on abstract conceptual processing, linguistic markedness is at least argued to reflect the systematic relationship between language and the way people experience the world. Recently, researchers have highlighted several structural factors which might account for the metaphor congruency effects in the paradigms used to investigate the perceptual representation of abstract concepts. These structural factors include the level of task-representation, stimulus-response mappings, the salience of abstract and concrete dimension due to specific parameters in the paradigm, attentional factors, etc. All these factors can influence response times, but their influence on processing times of the stimuli is not indicative of the relationship between the intrinsic meaning of stimuli and their perceptual representation. Therefore, it is essential that future studies developed with the aim to investigate how abstract concepts are grounded take these alternative structural explanations into account and try to control for them.
Eder and Rothermund (2008) have recently highlighted the possible influence of structural factors in response time differences in some of the studies reviewed earlier in this chapter. The authors argue against the view that the mapping of affect onto sensorimotor information reflects an automatic relationship between sensorimotor experiences and conceptual representations (e.g., Lakoff, 2008, Niedenthal, et al., 2005; Zajonc & Marcus, 1985). They show that simply reversing the response labels ‘towards’ and ‘away’ while performing identical approach-avoidance lever movements influenced the speed with which valenced stimuli were categorized. Since previously assumed automatic bodily responses are shown to depend strongly on how these responses are labeled, the authors conclude that “affective-mapping effects are located in flexible specifications of evaluative stimulus and response properties that take task demands
48
and situational constraints into account” (Eder & Rothermund, 2008, p. 278). These findings are argued to show that perceptual and conceptual information is related on a cognitive, representational level, and not through ‘hard-wired’ automatic reactions upon perceiving sensorimotor information (see also Weger & Pratt, 2008). Scherer & Lambert (2009) have similarly warned against interpreting automatic priming effects in light of any pre-existing associations, because “automatic priming effects can arise for different reasons, and sometimes these reasons have little to do with the intrinsic features of the priming stimulus itself ” (p. 400).
A second structural factor which is argued to be important for the mapping of abstract concepts (e.g., valence) on perceptual domains is the amount of attention that participants assign to the affective stimulus dimension. The Stroop-like metaphor congruency effects reviewed earlier (and especially those using a response-deadline procedure) are interpreted in support of the idea that perceptual information automatically and obligatory influences abstract conceptual processing at the level of encoding (e.g., Meier & Robinson, 2004). However, the Stroop paradigm is goal-dependent, which speaks against the idea that unrelated dimensions (e.g., vertical spatial position) interfere with categorizations because of their automatically activated intrinsic meaning. As Neumann (1984, p. 269) summarizes: “to a large degree, a distracter causes interference not because of its intrinsic properties but because it is related to the intended action” (see also Klauer and Musch, 2002). In other words, the interference effects observed in these studies might not be the results of the automatic activation of the intrinsic value of the stimuli, but a result of the categorization goals participants have. The finding that a focus on the evaluative dimension is essential for affective priming to occur is likely to be just as important in studies that investigate how abstract concepts are grounded in perceptual dimensions, and where perceptual characteristics of stimuli are expected to automatically activate their associated valence. Since “the likelihood or degree of automatic affective stimulus processing depends on the extent to which the experimental context, as a whole, encourages participants to assign attention to the
49
affective stimulus dimension” (Spruyt, De Houwer, Hermans, & Eelen, 2007) it seems attentional processes are an integrative aspect of perceptual representations.
From Experience to Culture
Grounded theories of cognition differ in the function they ascribe to sensorimotor information in conceptual processing. On the most basic level, embodied cognition suggests that concrete experiences play a role in language processing, which is a relatively undisputed claim. A more debated question is to which extent conceptual processing necessarily and directly recruits sensory and motor systems. Meteyard and Vigliocco (2008) organize different theoretical proposals ranging from complete dependence of conceptual processing on sensorimotor information (e.g., Gallese & Lakoff, 2005), to stable associations between sensorimotor information and semantic representations which nevertheless do not necessarily activate sensorimotor content during conceptual processing (e.g., Boroditsky, 2000; Vigliocco, Vinson, Lewis, & Garrett, 2004), to complete independence between perceptual information and conceptual processing (Landauer & Dumais, 1997; Pylyshyn, 1984).
Instead of adopting a single view on the interplay between perception and cognition to explain all conceptual processing, a more fruitful approach might consist of looking carefully at how specific abstract and concrete concepts are understood and grounded. It should not be surprising that people rely on different mechanisms to understand complex and abstract sentences (such as the previous sentence in this paragraph) than more concrete sentences such as “Hitting gray walls is painful”. The assumption that there are multiple ways to process the meaning of concepts might not be the most parsimonious approach to conceptual processing, but it might be the most realistic (for a similar reasoning, see Machery, 2009). Some researchers have indicated that relying on perceptual grounding in some cases but not in others is a reason to do away with embodied approaches to cognition completely. For
50
example, Mahon and Caramazza (2008, p. 60) have argued that abstract concepts cannot be represented perceptually, and thus conclude: “Given that an embodied theory of cognition would have to admit ‘disembodied’ cognitive processes in order to account for the representation of abstract concepts, why have a special theory just for concepts of concrete objects and actions?” However, as the results in the current thesis as well as many recent findings in the literature show, perceptual information also play a role when processing the meaning of abstract concepts.
Is it necessary to admit ‘disembodied’ cognitive processes to account for the representation of abstract concepts, as Mahon & Caramazza (2008) have argued? The answer to this question will most likely depend on the definition of ‘representation’, which is why some researchers prefer not to use that terminology (e.g., Jackendoff, 2002). Some abstract concepts seem to be related rather directly to sensorimotor experiences, and sensorimotor information can represent the meaning of these abstract concepts without any need for additional cognitive processes. For example, both the grounding of interpersonal closeness in warmth (Bargh & Williams, 2008; IJzerman & Semin, 2009) and the grounding or morality in disgust (Chapman, Kim, Susskind, & Anderson, 2009; Schnall, Haidt, et al., 2008; Zhong & Liljenquist, 2006) seem to relate the full abstract meaning of these concepts to concrete experiences.
Other metaphoric representations influence unrelated judgments, but do so in a more cognitively moderated fashion, depending on attentional processes, task representations, and flexible stimulus-response mappings. A wide range of abstract concepts exists, ranging from relatively universal concepts such as positivity and negativity, through ontogenetic factors such as interpersonal affection, and social and cultural concepts such as morality, and democracy. Abstract concepts differ in the extent to which they are grounded in concrete human experiences or cultural influences. On the more experiential end of the scale, the relationship between sensorimotor information and abstract concepts might be automatic, non-verbal, based on shared neural activation, inflexible, and universal. On the more cultural end of the scale, abstract concepts might
51
be related to sensorimotor information indirectly, specified in language, cognitively mediated, and flexibly mapped onto concrete domains depending on cultural factors.
As mentioned previously, several studies on the more experiential end of the experience-culture dimension have shown that concrete perceptual experiences directly affect abstract processing. For example, Williams & Bargh (2008) investigated the experiential relationship between warmth and affection, and showed that simply holding a warm or cold cup will influence ratings of affection (for similar findings, see IJzerman & Semin, 2009). Another set of studies on the relationship between weight and importance has revealed that holding either a heavy or light clipboard influences how important participants judge the topic of a questionnaire to be, resulting in more invested effort in answering the questions, which leads to more elaboration in the heavy clipboard condition (Jostmann, Lakens & Schubert, 2009). Furthermore, experiencing a bad smell influences subsequent moral judgments (Schnall, Haidt, et al., 2008). It seems to be difficult to account for these findings by the previously discussed linguistic and structural explanations.
On the more culturally determined end of the scale are concepts which are grounded in concrete experiences, but these concepts are grounded in different ways in different cultures, or even within cultures. It is also possible that certain aspects of abstract concepts are culturally determined (and therefore flexibly mapped on concrete dimensions) whereas other aspects of abstract concepts are more experientially determined (and show more direct perceptual-conceptual relationships). For example, Boroditsky (2000) notes that people experience time as unidirectional change, marked by the appearance and disappearance of events, and this aspect of time should be universal across cultures. However, which way time moves (up-down, left-right, front-back, or the reverse) is not observable, and these aspects are specified in language, often through spatial metaphors. Indeed, which spatial metaphor is applied to time is flexible (Boroditsky, 2001; Torralbo et al., 2006). Given that this flexibility in how abstract concepts are metaphorically structured exists
52
for many (aspects of) abstract concepts, the strong claim that all abstract processing relies completely and exclusively on perceptual information (e.g., Lakof & Johnson, 1980) seems untenable (e.g., Boroditsky, 2000; Zwaan, 2009). Nevertheless, abstract concepts fall well within the scope of grounded cognition, and acknowledging the importance of perceptual representations of abstract concepts will substantially improve our understanding of abstract conceptual processing.
Practical Implications
Embodied cognition research has highlighted the many ways in which subtle perceptual cues influence affective reactions, judgments under uncertainty, person perception, social interaction, and information processing (for an overview, see Pecher & Zwaan, 2005; Niedenthal et al., 2005; Semin & Smith, 2008). Regardless of how sensorimotor information influences social psychological processes, research findings over the last decennium make it abundantly clear that sensorimotor information influences conceptual thought in more ways than researchers were previously aware of.
The way people metaphorically structure abstract concepts has been argued to constrain reasoning about these abstract concepts (Lakoff, 1996; Lakoff & Johnson, 1999). Several recent findings reveal this is indeed the case. Judgments about an ambiguous question concerning the time at which events take place (“The meeting originally scheduled for next Wednesday has been moved forward two days”) depends on whether people have just moved through time themselves, or are sitting still (Boroditsky & Ramscar, 2002). Animals presented high up in the visual field were judged to be more respectable than animals lower in the visual field (Schubert, 2005). Pictures of people presented in the top of the screen were rated as more religious than pictures of people presented at the bottom of the screen (Meier, Hauser, et al., 2007). And finally, words presented in a big font were judged to be more positive than words presented in a small font (Meier et al., 2008). In all these instances,
53
events, objects or people were seen differently depending on sensorimotor experiences of the perceiver.
The metaphorical representation of morality has been theorized to influence and constrain moral judgments in a similar manner (Lakoff & Johnson, 1999). Graham, Haidt, and Nosek (in press) collected data that supports this assumption by comparing the way liberals and conservatives represent morality. Whereas liberals frame moral issues predominantly in terms of fairness and harm, conservative speakers also use metaphors related to authority and purity when talking about moral issues. These different representations of what morality is might have profound consequences in domains such as politics (Lakoff, 1996). It therefore seems crucial to increase our understanding of how different metaphoric representations of morality shape the way people think about moral issues (see Chapter 2).
Additionally, the perceptual representation of abstract concepts might be important for the way children learn concepts (e.g., Smith, 2005). The conceptual system has been argued to develop simultaneously and in parallel with the perceptual and motor systems (Mandler, 1992, 2004), and many global and abstract concepts are built on image-schema’s that are derived from concrete experiences. As such, it will be interesting to investigate to what degree infants’ abstract conceptual development is based on sensory and motor experiences (e.g., Gattis, 2002; Piaget, 1954). For example, concrete objects and spatial relations have been argued to play an important role in the mathematical thinking of children (Bryant & Squire, 2001). Investigating the role metaphoric representations play when children develop an understanding of abstract concepts seems to be a promising avenue for further research.
Conclusions
People draw on concrete experiences when they think about abstract concepts such as morality, time and valence. Even though perceptual information is intimately related to conceptual processing, perceptual
54
representations of abstract concepts are not expected to account for all abstract conceptual thought (see Boroditsky & Prinz, 2008). Luckily, perceptual information can be easily integrated with other sources of information, such as linguistic information based on the statistical distribution of words in spoken and written language. A growing body of work exists that compares the explanatory power of sensorimotor information in conceptual processing with linguistic information (e.g., Louwerse, 2008) and theoretical frameworks with the aim to integrate these two sources of information have been developed (e.g., Andrews et al., 2009). Future studies that aim to distinguish between the contributions of perceptual and linguistic sources of information while controlling for structural explanations for the relation between perceptual information and abstract concepts will open the way for a more informed approach to the grounding of abstract concepts.
People rely on concrete perceptual information to structure abstract concepts (see Chapters 2 and 3). Under certain circumstances, the partial re-activation of the same neural mechanisms that process concrete information might be activated when abstract concepts are processed. For example, researchers have argued such re-activation exists for the morality-disgust relationship (Chapman et al., 2009) and the relationship between affection and warmth (Williams & Bargh, 2008; IJzerman & Semin, 2009). Other perceptual representations might be more cognitively mediated and be a more activate structuring of abstract concepts in terms of concrete experiences (see Chapter 4), but this doesn’t make the role of perceptual information in abstract conceptual processing trivial. Abstract thought is possibly one of the most sophisticated abilities of human beings, and regardless of whether it is necessarily perceptual in nature, it is worthwhile to improve our understanding of how people perform such a remarkable feat.
Chapter 2
On the Grounding of Morality in Perceptual Symmetry
This chapter is based on: Lakens, D., van den Bos, K., & Semin, G. R. (in preparation). On the grounding of morality in perceptual symmetry.
57
Δίκης ὄνομα οὐκ ἂν ᾔδεσαν, εἰ ταῦτα μὴ ἦν
Men would not have known the name of justice if these things were not
Heraclitus, fr. 23 DK
Although the exact relationship between sensorimotor information and abstract conceptual processing remains debated (Arbib, 2008; Barsalou; 2008; Boroditsky & Prinz, 2008; Lakoff, 2008; Mahon & Caramazza, 2008), many researchers agree that concrete perceptual information plays a role in the processing of abstract concepts. According to Lakoff & Johnson (1980, 1999) concrete information is used to structure abstract concepts through metaphorical representations. Expressions such as “the mighty above the weak”, “looking forward to next week” or “things are looking up” are not merely linguistic expressions, but reflect the way people think about abstract concepts (Boroditsky, 2000; Meier & Robinson, 2004; Schubert, 2005). Abstract concepts are metaphorically structured in concrete experiential domains.
Despite such empirical demonstrations of the grounding of abstract concepts, some researchers argue that “perceptual representations seem ill-suited for representing abstract concepts” (Dove, 2009). Grounded theories of cognition would be sharply limited up front if they fail to accommodate highly abstract conceptual thought, and morality is one of the hallmark abstract concepts for which is argued that there is “no sensory or motor information that could correspond in any reliable or direct way to their meaning” (Mahon & Caramazza, 2008). In the current chapter I adopt a different perspective and argue that morality is metaphorically represented in perceptual symmetry. Studying this hypothesis would both contribute to the morality domain as well as further the investigation of the perceptual representation of abstract concepts.
Traditionally, moral judgments were assumed to result purely from deliberative reasoning processes (Kohlberg, 1969). However, more recently,
58
a more intuitive view on moral judgment has emerged (Haidt, 2001), which proposes that moral judgments are often based on both affective experiences (such as disgust; Chapman, Kim, Susskind, & Anderson, 2009; Schnall, Haidt, Clore, & Jordan, 2008) and bodily experiences (such as cleanliness and purity; Schnall, Benton, & Harvey, 2008; Zhong & Liljenquist, 2006) which precede conscious verbal reasoning (Haidt, 2008). Several physical experiences have been shown to influence moral and justice judgments, such as arm flexion and extension (Van Prooijen, Van Beest, & Karremans, 2006), the weight of the questionnaire on which moral scenarios are presented (Jostmann, Lakens, & Schubert, 2009), the bodily position of participants (Broeders, Van den Bos, & Müller, 2009), perceptual fluency (Laham, Alter, & Goodwin, in press), or the use of personal force (Greene, Cushman, Stewart, Lowenberg, Nystrom, & Cohen, in press).
Together, these studies indicate that moral judgments are influenced by sensorimotor information (see also Greene et al., in press; Laham et al., in press). Experiential factors seem to influence peoples’ decisions about whether something is moral or immoral. Despite these new insights into the role of experiential factors in moral judgments, the role of sensorimotor information in how people understand and think about abstract moral concepts has received relatively little attention. Morality is an important concept in social interaction (Folger, 1984), and thus in human communication (Wijn & Van den Bos, in press), and people often think and talk about moral concepts without performing moral judgments (as throughout this chapter, for example). Does perceptual information play a role when people process moral concepts?
Some researchers argue that only the affective consequences of moral judgments can ground morality, but that the concept of morality itself cannot be grounded perceptually (Andrews, Vigliocco, & Vinson, 2009; Boroditsky & Prinz, 2008; Dove; 2009; Mahon & Caramazza, 2008; Prinz, 2005). As Boroditsky and Prinz (2008, p. 103) argue: “The range of things we call morally bad have little in common perceptually (stealing, cheating, hitting, and so on), so there cannot be a single image of badness.
59
But all of these things are united by the family of emotions they cause in us.” Other researchers have argued in favor of the perceptual grounding of moral concepts (Barsalou, 2008a; Gibbs, 1996; Haidt, 2001; Johnson, 1993; Meier, Robinson, & Wygant, 2007; Lakoff & Johnson, 1980, 1999; Sherman & Clore, 2009). This relationship between sensorimotor information and moral concepts has already been put forward by Whitney, when he noted the importance of the widespread “application of terms having a physical, sensible meaning, to the designation of intellectual and moral conceptions and their relations” (Whitney, 1875, p. 88).
Previous studies that have investigated the perceptual grounding of moral concepts primarily focused on the affective components of morality. Comparable to how positivity is up, and negativity is down (Lakoff & Johnson, 1999, Meier & Robinson, 2004), or positivity is white, and negativity is black (Meier, Robinson, & Clore, 2004), moral and immoral words have been shown to be associated with up and down, and white and black, respectively (Meier et al., 2007; Sherman & Clore, 2009). The present research takes a new approach to the grounding of morality by focusing on the equality component of morality, and showing how perceptual symmetry is used to metaphorically structure moral concepts. Equality is considered a constitutive feature of justice in the writing of philosophers like Aristotle, Hobbes, Locke, Rousseau, Kant and Rawls. Equality underlies common folk theories about morality (e.g., ‘an eye for an eye’) as well as classical theories about distributive justice such as equity theory (Adams, 1965), where the basic underlying justice principle is a balance between the contributions and rewards of two people.
The equality component of justice is theorized to be grounded in the balance image schema (Gibbs, 2005; Johnson, 1987, Lakoff, 1987) and conceptualized metaphorically as balance or symmetry (Lakoff, 2005), at least in Western cultures (Lakoff & Johnson, 1999). The balance image schema is the basis of a fundamental metaphor for morality, called moral accounting (Lakoff & Johnson, 1999), where harms and benefits are weighed against each other until the moral books are balanced (Lakoff & Johnson, 1980, cf. Adams, 1965). Messick (1993) proposes that the
60
symmetric nature of equality may be used as an efficient decision heuristic when judging how fair goods are distributed. Indeed, many of our concrete experiences with moral concepts are directly related to symmetrical or balanced distributions of rewards, goods and affect (for example, seeing parents divide a birthday cake into equal pieces). Through metaphorical extension, such sensory experiences can structure our understanding of moral concepts (Lakoff, 1996).
In the present chapter I reason that given this strong experiential association between equal divisions and justice, moral concepts should be associated with perceptual symmetry. If our understanding of abstract concepts builds on perceptual information (Lakoff & Johnson, 1980, 1999), then thinking about abstract concepts should activate the associated sensorimotor information. If people learn about morality and justice through experiential information related to equality then thinking about morality should activate non-verbal associations with perceptual symmetry. In Study 2.1, I investigated whether moral words were associated with symmetrical stimuli, and immoral words were associated with asymmetric stimuli. In Study 2.2, moral words were expected to show a stronger association with perceptually equal distributions of colored letters than immoral words.
Study 2.1: Morality is symmetrical
If morality is grounded in perceptual symmetry then moral concepts should activate non-verbal associations with balance and symmetry, whereas immoral concepts should be associated with imbalance and asymmetry. In the current study, I tested the hypothesis that when performing judgments under uncertainty, participants would be drawn towards symmetrical choice alternatives for moral words, whereas participants would be drawn towards asymmetrical choice alternatives when performing judgments for immoral words.
61
Method
Participants. Thirty-seven students at Utrecht University (26 females, mean age 21) participated in this study for partial course credit or payment.
Procedure. Participants completed a Chinese translation task where they were asked to translate 20 Dutch words into Chinese by choosing one of two Chinese ideographs presented on the screen. Participants were told that previous research had shown that people could translate Dutch words into Hebrew characters correctly above guessing average, even though people did not speak Hebrew. The current study would test a similar hypothesis with Chinese ideographs. The Dutch words consisted of 10 moral words and 10 immoral words (see Appendix 1) and all ideographs were presented in black on a white background, side by side underneath the morality-related word. The presentation of all words, ideographs and their combinations was completely randomized. After choosing the ideograph that participants thought was the best translation for each word, participants rated all 40 ideographs on symmetry by crossing a line ranging from 0 (not at all symmetrical) to 100 (completely symmetrical). None of the participants knew the real meaning of the Chinese ideographs, which were in reality all unrelated to morality.
Results
To test the hypothesis I compared the absolute average symmetry ratings for the 10 ideographs participants chose to translate moral words with the 10 symmetry ratings for ideographs participants chose to translate immoral words. In addition, I compared the relative difference in symmetry between the ideograph participants chose and the ideograph participants did not choose for both moral and immoral words. Both analyses should show that symmetrical choice alternatives are preferred for moral words, compared to immoral words.
Absolute symmetry ratings. The symmetry ratings for the
62
ideographs participants chose to translate the Dutch words were averaged for both moral and immoral words. In line with the predictions about the association between morality and symmetry, a repeated measures ANOVA revealed that the average symmetry of the ideographs chosen to translate moral words was higher (M = 46.92, SD = 11.66) than the average symmetry for ideographs chosen to translate immoral words (M = 41.31, SD = 9.47), F(1, 36) = 7.68, p = .009, ηp² = .18.
Relative symmetry ratings. For each word, the difference between the symmetry rating of the ideograph that participants chose minus the symmetry rating for the ideograph that participants didn’t choose was calculated. Positive difference scores indicate that the participant chose the more symmetric ideograph to translate the stimulus word, whereas negative difference ratings indicate participants on average chose the more asymmetric ideograph of the two choice alternatives. These difference ratings were averaged over the 10 moral and the 10 immoral words. As predicted, a repeated measures ANOVA revealed that participants on average chose more symmetric ideographs to translate moral words (M = 7.53, SD = 11.46) than immoral words (M = -3.37, SD = 12.72), F(1, 36) = 14.06, p = .001, ηp² = .28.
Discussion
When choosing between two unknown Chinese ideographs to translate a Dutch word, participants were more drawn towards symmetrical choice alternatives when translating moral words, whereas participants preferred the more asymmetric choice alternatives for immoral words. These results confirm the prediction that morality is associated with perceptual symmetry. When participants had to judge which of the two ideographs they thought correctly translated morality related words, the amount of symmetry in the ideographs was used as a source of information. The more perceptually symmetrical ideograph was more likely to be seen as the correct translation for moral words, whereas the more asymmetric ideograph was preferred for immoral words.
63
These results support the hypothesis that there is a strong association between morality and perceptual symmetry, in line with the assumption that morality is metaphorically grounded in non-verbal representations of balance and symmetry (Lakoff & Johnson, 1999; Lakoff, 2005). It seems that previous criticisms about the lack of perceptual characteristics of moral concepts (Boroditsky & Prinz, 2008; Dove; 2009; Mahon & Caramazza, 2008) are unwarranted. The “balance of justice” seems to be more than just a figure of speech; it serves as an experiential source which provides structure to moral concepts. In the second study, I aimed to replicate and extend these findings with a new paradigm.
Study 2.2: Morality is fifty-fifty
The current study investigated the relationship between morality and perceptual symmetry more indirectly. Whereas Study 2.1 showed that symmetric stimuli are judged to be a better translation of words with a moral meaning, the current study investigated whether moral words would influence judgments that were unrelated to the meaning of the stimulus words. Under the pretext of a memory task, 40 words were presented (half with a moral meaning, half with an immoral meaning) to participants, and asked them to remember the words. The first few letters of these words were presented in a blue font, the remaining letters of the words were presented in a green font. After all words were presented, participants were asked to indicate how many letters of each word had been blue. I hypothesized, based on the “balance of justice” metaphor, that participants would display a biased recall of the distribution of blue and green letters. Judgments for the number of blue letters in moral words were expected to be closer to half the number of letters in the word compared to judgments for immoral words, indicating a more perceptually symmetrical color distribution.
64
Method
Participants. Thirty-one students at Utrecht University (19 females, mean age 20) participated in this study for partial course credit or payment.
Stimuli. Half of the 40 morality related words (see appendix 1) had an even number of letters and half had an uneven number of letters. The word length did not differ for moral (M = 8.00, SD = 2.53) and immoral stimuli (M = 8.30, SD = 2.11), F < 1. Half of the stimuli were presented with an equal distribution of blue and green letters, whereas for the other half of the stimuli the number of blue and green letters was distributed unequally. More specifically, for the stimulus words with an equal distribution of blue and green letters, 50% of the word (rounded up or down for words with an uneven number of letters) were presented in a blue font (for example, a 6 letter word would have 3 blue letters, the absolute average deviation from half being 0, and a 7 letter word would have 3 or 4 blue letters, the absolute average deviation from half being 0.5). For the other half of the stimuli, 50% of the word (rounded up or down for words with an uneven number of letters) plus or minus one letter was blue (for example, a 6 letter word would have either 2 or 4 blue letters, the absolute average deviation from half being 1, a 7 letter word would have either 2 or 5 blue letters, the absolute average deviation from half being 1.5). For both moral and immoral words, the true average absolute deviation of the blue letters from half the number of letters in the word was 0.75.
Procedure. Under the cover story of a memory task, a series of 40 words were randomly presented on the screen. Participants were instructed to try to remember each word. The stimuli consisted of 20 moral words, and 20 immoral words. Part of each word was written in a blue courier font, part was written in a green courier font. During the memory phase, all words were presented randomly for 1.5 seconds. Unbeknownst to the participants, their real task consisted of answering how many letters of each word had been colored blue. After the memory
65
phase, all words appeared randomly for a second time, but now in a grey font. Participants could type in a number to indicate how many letters of the word they thought had been blue when the word was presented during the memory phase. After the recognition task, people were asked to rate all words on morality, after which they were thanked and debriefed.
Results
Manipulation check. The manipulation check confirmed that moral words were rated as more moral (M = 5.94, SD = 0.47) than immoral words (M = 2.06, SD = 1.07), F(1, 30) = 18.29, p < .001, ηp²= .38.
Letter judgments. I calculated the average absolute deviation from half the number of letters in each word for all the blue letter judgments. Half of the total number of letters of each stimulus word was subtracted from the judgments participants made. For example, if a participant thought the word ‘unfair’ originally had 2 blue letters, and half the number of letters in the word is 3, the deviation would be -1. Subsequently, the absolute deviations were averaged for the 20 moral and 20 immoral words. As noted in the procedure section, the correct absolute average deviation of blue letters from half the number of letters in both moral and immoral words was 0.75. In line with the hypothesis, the judged number of blue letters deviated less from half the word for moral words (M = 0.97, SD = 0.27) than for immoral words (M = 1.14, SD = 0.37), F(1, 30) = 18.29, p < .001, ηp² = .38, indicating that participants judged the distribution of blue and green letters in moral words to have been closer to a perceptually symmetrical fifty-fifty distribution compared to immoral words.
Discussion
When participants were asked to remember how many letters of a previously presented word had been blue, the moral or immoral meaning of the words influenced participants’ judgments. For moral words, judgments were closer to an equal number of blue and green letters,
66
compared to the judgments made for immoral words. This result provides further support to the hypothesis that moral concepts are associated with visually equal distributions. When guessing the number of blue letters for moral words, choice alternatives closer to a perceptually symmetrical color distribution had a stronger likelihood of being chosen than choice alternatives further from a perceptually symmetrical color distribution. Whereas in Study 2.1, participants were asked to judge which ideograph best represented the meaning of the Dutch word to be translated, Study 2.2 extended these findings to a paradigm where the meaning of the stimuli was irrelevant for the judgments participants made.
General Discussion
In two studies I found support for the hypothesis that morality is associated with perceptual symmetry. In the first study, participants preferred the more symmetric Chinese ideograph to translate moral words, but preferred the more asymmetric ideograph to translate immoral words. In the second study, participants indicated in a memory task that the number of blue and green letters of previously presented moral words had been more equally distributed than the number of blue and green letters of immoral words. These studies support the prediction that moral concepts activate non-verbal representations of visual equality and symmetry.
The implications of these findings are twofold. On a theoretical level, morality has often been used as an example of an abstract concept that lacks clear perceptual characteristics, and is used to argue against the perceptual representation of abstract concepts (e.g., Dove, 2009; Mahon & Caramazza, 2008; Prinz, 2005). The current studies reveal that even highly abstract concepts as morality are metaphorically structured in concrete experiences. Although the extent to which sensorimotor information is necessarily activated to process concepts remains to be determined (Mahon & Caramazza, 2009; Meteyard & Vigliocco, 2008; Zwaan, 2009), sensory information seems to be an important source of structure and meaning for abstract concepts. Given this strong relationship
67
between sensory information and abstract concepts, embodied theories of cognition can improve our understanding of how people think and talk about abstract concepts such as morality.
On a practical level, the finding that morality is metaphorically structured in perceptual symmetry might have consequences for how people interpret moral situations. Previous studies in the abstract domains of time, power and divinity have shown that judgments about the time at which events take place, how powerful animals are and how religious people are can all be influenced by sensory information related to the metaphoric representation of these abstract concepts (Boroditsky & Ramscar, 2002; Meier, Hauser, Robinson, Friesen, & Schjeldahl, 2007; Schubert, 2005). The metaphorical representation of morality has been theorized to influence and constrain moral judgments in a similar manner (Lakoff & Johnson, 1999; Graham, Haidt, & Nosek, in press). For example, does the fact that the US Supreme court sits in a perfectly symmetrical court, inside a symmetrical building, influence our judgments about how just their decisions are? Does perceptual inequality activate the concept of immorality, possibly leading to subsequent moral reasoning? The current research suggests this might be the case, and given the importance of moral concepts for social interaction (Folger, 1984), such empirical questions are worthwhile to pursue.
Valence is an important component of morality, with moral concepts being more positive than immoral concepts. One might argue that the results in Study 2.1 can be explained by the possibility that symmetrical ideographs were chosen more often for moral words because symmetric ideographs are more positive than asymmetric ideographs. Symmetry and positivity have been shown to be related in situations where the symmetrical nature of stimuli leads to a more fluent processing of the stimulus (for a review, see Reber, Schwarz, & Winkielman, 2004). This reasoning is based on the assumption that perfectly symmetrical stimuli contain less information (with one half being identical to the other half) and are therefore easier to process. None of the ideographs used in the current study were perfectly symmetrical, and the ideographs
68
were on average low in symmetry, making fluency related differences in the processing of the ideographs unlikely. Furthermore, it seems difficult to account for the findings in Study 2.2 purely based on differences in positivity of the choice alternatives. Nevertheless, both the valence and the equality component of morality can be metaphorically represented, and their interplay deserves further attention.
It has rightly been noted that even when abstract concepts are experientially grounded, concepts such as morality cannot be fully understood without complementing the meaning of these abstract concepts by additional sources of knowledge, such as contextual information, linguistic data and cultural background (Andrews, Vigliocco, & Vinson, in press; Boroditsky & Prinz, 2008). Understanding moral concepts is certainly not limited to perceptual processes, but the sensorimotor information which is used to metaphorically structure moral concepts is argued to be an essential part of how morality is conceptualized (Lakoff, 1996; Lakoff & Johnson, 1999). As such, experiential grounding provides a basis to structure moral concepts, as well as a constraint on peoples’ idea of what morality is (Haidt, 2001; Graham et al., in press; Lakoff, 1996). A promising avenue for further research would be the investigation of the role metaphoric representations play when children develop an understanding of abstract concepts, and whether metaphoric representations play the facilitatory role in moral development often ascribed to them (e.g., Lakoff & Johnson, 1980, 1999).
The present studies support the hypothesis that people’s understanding of morality and immorality is intimately related to sensory information about perceptual symmetry. These results complement the emerging view that experientially grounded moral concepts underlie moral intuition (Haidt, 2008). Where previous work has focused on an affective basis for moral judgments (Haidt, 2001, Prinz, 2007), the currents results reveal that moral concepts are metaphorically structured based on sensorimotor information. These findings show that sometimes, the theorized lack of perceptual characteristics of highly abstract concepts is simply a matter of finding the right perceptual representation.
69
Moral concepts fall well within the scope of grounded cognition, and acknowledging the importance of perceptual representations of moral concepts will substantially improve our understanding of moral reasoning.
Chapter 3
The Sound of Time
This chapter is based on: Lakens, D., Semin, G. R., & Garrido, M. (under review). The sound of time: Locating time in auditory space.
73
Alice sighed wearily. ‘I think you might do something better with the time,’ she said, ‘than waste it in asking riddles that have no answers.’
Lewis Carroll
How people represent, think, and communicate about abstract concepts that do not afford immediate sensorimotor experiences remains a challenge for embodied approaches to cognition (cf. Barsalou, 2008a; Boroditsky, 2000; Lakoff & Johnson, 1999). As the introduction of this thesis as well as the previous chapter have shown, perceptual experiences can serve as the source for metaphorical representations of abstract target concepts. Indeed, there is abundant empirical evidence showing that concrete spatial dimensions provide a way to structure abstract concepts, such as time (Boroditsky, 2000; Boroditsky, 2001; Boroditsky & Ramscar, 2002; Casasanto & Boroditsky, 2008; Núñez, & Sweetser, 2006; Santiago, Lupiáñez, Pérez, & Funes, 2007; Torralbo, Santiago, & Lupiáñez 2006; Tversky, Kugelmass, & Winter, 1991; Vallesi, Binns, & Shallice, 2008; Weger & Pratt, 2008; Zacks & Tversky, 1999).
The three studies reported in this chapter introduce a novel perspective and method to examine the embodied grounding of time. This perspective relies on the argument that if time is spatially structured, then the abstract concept of time could in principle be structured in any modality that is sensitive to spatial information. Our auditory senses have developed a great sensitivity in deriving spatial information from sound (Blauert, 1997). Consequently, the auditory system should map time concepts in a manner comparable to the visual modality. The current research focuses on the multimodal grounding of time, and is the very first examination of the role that the auditory system plays in grounding abstract concepts.
These studies investigate the left-right representation of time in space, where the past is located on the left and the future on the right (Santiago et al., 2006; Maass & Russo, 2003; Tversky et al., 1991; Weger
74
& Pratt, 2008). Study 3.1, in which the systematic visual mapping of time in space was established, lays the groundwork for the subsequent two auditory studies. In these two studies, participants are asked to place words referring to the past and the future on a horizontal line, and a clear left-right ordering of past and future-related words, respectively, was expected.
The second and third study introduced a novel method, namely an auditory loudness judgment task. Participants received past and future-related words via headphones. On critical trials these words were presented equally loud on both the right and left auditory channels. These critical trials were distributed among a number of randomly presented filler trials varying systematically in their degrees of loudness in the right or left auditory channels. The participants’ task was to judge in which auditory channel the word was presented louder, whereby judgments on critical trials constituted the chief dependent variable. Participants were expected to more often judge future related words to be louder in the right ear, compared to past related words. Furthermore, comparisons of the visual positioning of the time related words in the first study and the loudness judgments in the second and third study were expected to reveal substantial overlap, indicating that the further words were visually positioned to the right, the more often they were judged to be louder in the right ear.
Study 3.1
In this study, participants were asked to place past and future-related words on a horizontal line. The expectation was that future-related words would be placed to the right of the midpoint of the line, whereas past- related words were expected to be placed to the left of the midpoint.
Participants. Fifty-six students (38 females, mean age 20) participated in this study.
Procedure. Participants were asked to place 8 past and 8 future- related words on a horizontal line, with an indicator at the midpoint of the
75
line anchoring the present. The ends of the line were not marked. Eight future (later, future, day after tomorrow, coming, tomorrow, immediately, soon, shortly) and eight past-related words (past, day before yesterday, earlier, been, before, yesterday, recently, a short while ago) were used. All stimuli consisted of single words in Dutch. Participants indicated the position they thought was best suited for the stimulus by placing the cursor on the line followed by a left mouse click, which was scored from 0 (completely left) to 100 (completely right).
Results
For the analyses I averaged the spatial position scores of future and past- related words. As expected, future-related words were placed further to the right from the midpoint (M = 66.42, SD = 3.99) than past-related words (M = 29.71, SD = 4.31), t(55) = 37.43, p = .001, prep = .99. Both differed significantly from the scale midpoint, t(7) = 3.66, p = .005, prep = .97 and t(7) = -4.02, p = .008, prep = .98, respectively. The absolute distances of past and future words from the scale midpoint did not differ, t(14) = .57, p = .58, prep = .65.
Discussion
Along with earlier findings (e.g., Tversky et al., 1991), the positioning of past and future-related words uniformly showed a respective left to right ordering on a horizontal dimension. I subsequently set out to test whether the left-right representation of time extends to the auditory modality, and to examine whether the visual spatial position of the past and future words is related to auditory left vs. right loudness judgments.
Study 3.2
The current study was designed to investigate whether the pattern of spatial distributions obtained for time-related words in the first study is
76
reproduced in the auditory modality. A new paradigm was developed, where participants had to judge whether words presented over headphones were louder in the left or right auditory channel. For the critical experimental trials, the future and past related words were presented equally loud on both channels. I expected the temporal meaning of the words to bias participants’ loudness judgments. On trials where time- related words are presented equally loud on both channels, participants were expected to judge future related words to have been louder in the right ear, and past related words to have been louder in the left ear.
This judgment bias was not expected to be completely symmetrical. Verbal information is processed more efficiently when presented to the right than the left ear (Belin et al., 1998; Kimura, 1961) due to the hemispheric asymmetry in language processing, with Broca’s speech area located in the left hemisphere, (Belin et al., 1998; Pujol, Deus, Losilla, & Capdevila, 1999; Toga & Thompson, 2003). The literature suggests that there should be an overall shift towards a right channel bias for loudness judgments for all time-related words. Despite this possible main effect, the average number of right ear loudness judgments was still expected to differ for past and future related words. In addition, the comparison of the spatial positioning obtained in Study 3.1 and the loudness judgments in this study were expected to show a systematic overlap, since this comparison involves relative differences between the different time- related words.
Method
Participants. Twenty-eight students (18 females, mean age 20) participated in this study.
Stimuli. The temporal meaning (past vs. future) of the stimuli was manipulated within participants. The 8 past and 8 future-related words from Study 3.1 were transformed into audio files using a text-to-speech program. Two sets of auditory stimuli each with five sound volumes (50%, 40%, 30%, 20%, or 10% louder on the left or right channel) were
77
created using Audacity for Windows. Additionally, there was a version with identical volume levels in both auditory channels. These auditory stimuli constituted the critical experimental trials.
Procedure. Participants were seated in individual cubicles and wore headphones. They had to listen to computer generated words and asked to indicate for each word whether the word they heard was louder in the left (by pressing the “A” key) or the right channel (by pressing the “L” key). Participants were told that the meaning of the words was irrelevant for the task. They were also informed that the differences in volume between the left and right ear could be very subtle, and that they should try their best to answer accurately. The 10 audio files with unequal volume in the left and right audio channels were presented once for each of the 16 words. The audio files used in the experimental trials, with equal volume in both auditory channels, were presented three times, resulting in 48 critical trials, of which half consisted of future related words, and half consisted of past related words. The order in which the 208 trials were presented was completely randomized.
Results
Analyses by participants. Since left and right judgments are mutually dependent, the average number of times the 24 past and 24 future related critical trials were judged to be louder in the right channel was calculated. As expected, these averages differed based on the temporal meaning of the stimuli. A repeated measures analysis revealed that perfectly balanced future related words were judged to be louder in the right ear more often (M = 14.79, SD = 5.27) than perfectly balanced past related words (M = 13.38, SD = 4.89), F(1, 27) = 4.61, p = .04, ηp² = .15 (see figure 1, left pane). This bias in loudness judgments is exactly what one would predict given the left to right representation of time in space: participants judged future related words to be louder in the right channel more often than past related words.
Analyses by items. For the analysis by items, the average number
78
of times the responses for the 8 past and 8 future-related words in the critical trials were judged to be louder in the right channel was calculated. This gives a range between a value of 0 and 3, with 1.5 indicating an equal number of left and right channel judgments. As expected, the judgments differed as a function of the temporal meaning of the words. A repeated measures analysis revealed that future-related words were judged to be louder in the right ear more often (M = 1.85, SD = 0.17) than past-related words (M = 1.67, SD = 0.13), F(1, 14) = 5.49, p = .03, prep = .93, ηp² = .28 (see Figure 1, left pane).
Figure 1. Average number of right ear loudness judgments for words presented equally loud on both auditory channels in Study 3.2 (left pane) and Study 3.3 (right pane) as a function of their temporal meaning.
1,3
1,4
1,5
1,6
1,7
1,8
1,9
2
2,1
Past Future Past Neutral Future
Study 3,2 Study 3,3
R ig
ht E
ar D
is am
bi gu
at io
ns
79
These results support the expected judgment bias asymmetry for past and future words. Moreover, as expected, there was an overall right- bias in loudness judgments. One-sample t-tests revealed that the mean of the past words was significantly to the right of the midpoint, t(7) = 3.72, p = .01, prep = .97, Cohen’s d = 1.31, as was the mean for the future words, t(7) = 5.71, p = .001, prep = .99, Cohen’s d = 2.01.
The cross-modal overlap was tested by comparing the relative visual positioning scores obtained for the 16 items in Study 3.1 with the average auditory right channel loudness judgments for the 16 items in this study. The data points for this analysis were the average values for each of the time-related stimuli in Study 3.1 and 3.2. A regression analysis revealed that the spatial ordering of past and future-related words on the horizontal position in Study 3.1 overlapped significantly with their relative right channel judgment bias in Study 3.2, β = .55, r2 = .30; t = 2.46, p =.03, prep = .99.
Discussion
These results reveal that time is represented in left-right auditory space. Furthermore, these findings confirm the hypothesis that there is a systematic overlap between the representation of time in the visual and auditory modality. The right channel judgments for past and future- related words in the auditory loudness judgments task reproduced the relative positions that had been obtained in the visual positioning task. It is noteworthy that whereas participants were explicitly asked to place time- related words on a horizontal line in Study 3.1, the loudness judgments task only required participants to decide in which ear they thought the word was presented louder, and the time related words were not classified on a temporal dimension.
Study 3.3
The third study was a replication of Study 3.2, but with the following
80
modifications. First, 8 neutral words were added that were unrelated to time (e.g., table). The neutral words provided a baseline and were expected to fall between the average past and future loudness judgments. Second, in order to control for a possible unequal hearing strength between the left and right ears, the loudness was calibrated for each individual separately. Finally, the response key assignment was changed from a left-right alignment (keys “A” and “L”), which may activate the left- right representation of time in space (Torralbo et al., 2006) to an up-down alignment (the “T” and “V” keys). The hypotheses remained the same.
Method
Participants. Thirty-two students (22 females, mean age 22) participated in this experiment.
Stimuli. The stimuli were identical to Study 3.1, except for the inclusion of eight temporally neutral words (identical, closet, even, sandal, paper, glass, table, and triangle).
Procedure. The study started with a hearing test to reduce individual differences in right-left hearing imbalances. Participants received a 440 Hz tone that was less loud on one of the channels of the headphones. Their task was to adjust the sound until the tone was equally loud on both channels. This was repeated until participants reported that the tone was perceived as equally loud in both ears. The auditory stimuli were presented with the corresponding individual adjustment.
The auditory loudness judgments task was identical to that in Study 3.2, except that the response keys to indicate the left or right channel were vertically aligned (the “T” and “V” keys on the keyboard), and key assignment was counterbalanced between participants. The classification task consisted of three blocks, where all stimuli where presented once with the volume equally loud in both ears, and once while the sound was louder in one ear than the other. The presentation of the words within each block was completely randomized. Thus, the total number of trials was 144, with 72 experimental trials (24 past, 24 neutral, and 24 future
81
words) and 72 filler trials to create a believable auditory classification task with real differences in the loudness between the ears. In each block, the filler trials were randomly distributed over 10 differences in volume between the ears, being 10%, 20%, 30%, 40%, or 50% louder in the left or the right ear, with each position occurring twice, except for the 40% and 50% volume settings, which occurred 3 times.
Results
Analyses by participants. The average number of times the 72 perfectly balanced past, future and temporally neutral words were judged to be louder in the right ear was calculated. As predicted, these judgments differed based on the temporal meaning of the stimuli. A repeated measures analysis on the average right ear loudness judgments for the 24 past, neutral and future words revealed an effect of words meaning F(2, 62) = 2.77, p = .07, ηp² = .08. A linear trend best fitted the data, with the number of right judgments being highest for future words (M = 13.72, SD = 5.43) and lowest for past related words (M = 12.41, SD = 4.82), with the neutral words falling in the middle (M = 13.38, SD = 4.62), F(2,62) = 4.87, p = .035, ηp² = .14.
Analyses by items. The average number of times the responses for the 8 past, 8 neutral, and 8 future-related stimuli in the critical trials were judged to be louder in the right ear were calculated similar to Study 3.2. Response key assignment did not affect the results (all prep < .58). A repeated measures analysis revealed that the predicted linear trend of a graded right-ear bias for future, to neutral to past words was significant, F(1, 21) = 4.72; p = .04, prep = .93, ηp² = .26, see Figure 1, right pane. Future related words were judged to be louder in the right ear most often (M = 1.71, SD = 0.14), with neutral words occupying an intermediary position (M = 1.67, SD = 0.16) followed by past related words (M = 1.55, SD = 0.16). One-sample t-tests revealed that the right bias was significant in the case of future words, t(7) = 4.34, p = .003, prep = .98, Cohen’s d = 1.55, and neutral words, t(7) = 3.12; p = .02, prep = .95, Cohen’s d = 1.10, but not
82
in the case of past words, t(7) = .92, p = .39, prep = .73, Cohen’s d = 0.33. As in Study 3.2, a regression analysis testing the cross-modal
overlap revealed that the horizontal position of the time-related words on the visual positioning task (Study 3.1) overlapped significantly with the auditory judgments of the very same stimuli in the loudness judgments task in this study, β = .55, r2 = .30, t = 2.59, p = .03, prep = .93.
Discussion
The results of Study 3.3 reproduce the cross-modal findings of Study 3.2. As predicted, participants chose the right channel more often for future words than for past related words when the words were presented equally loud in both channels. Time is represented in left-to-right auditory space. This conclusion is strengthened in Study 3.3 by controlling for individual differences in left-right hearing, counterbalancing vertically aligned response keys, and adding a set of neutral words, which occupied a middle position between past and future-related words in the average number of right channel judgments. More importantly, the third study replicated the overlap between the auditory and visual mapping of time in space. This significant cross-modal overlap underlines the argument under examination, namely that time is mapped in the same relative order in the visual as well as the auditory modality.
General Conclusions
The unique contribution of these studies is the demonstration that the abstract concept of time is structured in auditory space and the remarkable overlap between the visual and auditory representation of time. The data analyses of Study 3.2 and 3.3 revealed that these findings can be generalized not only across participants, but also across items, whereby participants and words serve as random factors respectively.
Although the current studies are the first to show cross-modal overlap between the visual and auditory representation of the abstract
83
dimension of time, previous studies have revealed cross-modal equivalences between many concrete dimensions. For example, in studies using explicit cross-modality matching tasks, participants match bright lights with loud sounds, and dim lights with soft sounds (Marks, 1978). As Rudel and Teuber (1964, p. 6) summarize: “There is some common aspect of perceptual activity which permits one to utilize information from within a sensory channel or from several channels in such a way that invariant properties of objects are extracted.” Earlier findings have indicated a cross-modal overlap in meaning between auditory stimuli and visual representations of these stimuli (Karwoski, Odbert, & Osgood, 1942). For example, participants in this study showed a general agreement to represent loudness by heavier, darker or thicker lines. Osgood (1952) argues that a similar process underlies metaphor in language as visual- auditory synesthesia, namely the parallel alignment of two or more dimensions of experience, relating the corresponding extremes of these dimensions.
Marks (1978), on the other hand, has argued that the equivalences in cross-modal perception reflect intrinsic properties of the sensory systems and depend upon the neurophysiological mechanisms that code perceptual qualities. In a set of reaction time studies (Marks, 1987), participants received compound (sound-light) stimuli, and were asked to categorize the stimuli based on one dimension (e.g., sound) while ignoring the other component (e.g., light) by pressing one of two keys. The results revealed that responses were faster (and the accuracy higher) on those trials where the critical stimulus (e.g., high pitch) is accompanied by “matching” (e.g., bright light) rather than “mismatching” (e.g., dim light) stimuli from another modality. Although these results do not exclude a semantic mediation of the findings, Marks (1987, p. 393) argues they ‘do point to a possible non-semantic basis’.
How cross-modal structural alignment occurs remains an open question, but the finding itself has been observed in many studies and over a multitude of concrete domains (Kornblum, Hasbroucq, & Osman, 1990). The current studies take cross-modal matching to an altogether new
84
level, by showing that the left-right spatial mapping of time in the visual modality is structurally aligned with the left-right auditory mapping of time. Apparently, people are able to utilize concrete information in the visual and the auditory dimensions to represent a common aspect of the abstract concept of time.
The findings in the current chapter introduce a novel multimodal perspective into the embodied grounding of time. Loudness judgments are likely to prove to be a fertile method in investigations across a variety of different abstract constructs that are grounded spatially such as valence (e.g., Meier & Robinson, 2004), or power (e.g., Schubert, 2005). The main contribution of the current set of studies is however the demonstration of a structural alignment between both an auditory and visual source in how the abstract concept of time is grounded. These findings open up a more informed approach to the multimodal grounding of abstract concepts.
Chapter 4
But for the Bad,
There Would not be Good
This chapter is based on: Lakens, D., Semin, G. R., & Foroni, F. (submitted). But for the bad, there would not be good: Grounding valence in brightness through shared relational structures.
87
Mr. Pink: Why can’t we pick out our own colors?
Joe: I tried that once, it don’t work. You get four guys fighting over who’s gonna be Mr. Black.
Reservoir Dogs, Tarantino (1992).
In Plato’s chariot allegory of the journey of the soul, the charioteer (or reason) has to control two horses: one is white, representing the moral impulse, and the other is black, representing the immoral passions. The opposition between black and white is used pervasively to represent the opposition between the abstract concepts of bad and good. According to Lakoff & Johnson’s Conceptual Metaphor Theory (CMT; 1980, 1999) people use metaphors to ground the meaning of abstract concepts (e.g., positivity and negativity) in concrete experiences (e.g., light and dark). CMT and other grounded approaches to cognition (e.g., Barsalou, 1999; Gallese & Lakoff, 2005; Glenberg & Kaschak, 2002; Jeannerod, 2001; Pulvermüller, 1999; Zwaan, 2004) are a response to the dominant view on conceptual processing in psychology that perception and cognition are functionally independent (e.g., Fodor, 1975; Pylyshyn, 1984; Jackendoff, 2002). Instead, grounded approaches to cognition propose that conceptual processing is intimately related with the activation of sensorimotor information. Several studies have found support for this assumption for both concrete and abstract concepts (for reviews, see Crawford, 2009; Meteyard & Vigliocco, 2008). Recently, researchers have stressed the necessity to go beyond studies demonstrating the influence of perceptual information on conceptual processing and to investigate the processes that underlie this influence (e.g., Barsalou, 2008a; Zwaan, 2009). In the current studies, I compare two process explanations in the literature against each other with the aim to gain a better understanding of how metaphoric representations relate perceptual information to abstract
88
concepts. Meier, Robinson, and Clore (2004) set out to test the hypothesis
that people rely on perceptual information when they process abstract concepts by investigating whether the speed with which positive and negative words were categorized would be influenced by their brightness. Given that good and bad are metaphorically represented in light and dark (Adams & Osgood, 1973), positive words in white and negative words in black were expected to be categorized more efficiently than positive words written in black and negative words written in white. These predictions were confirmed, supporting the assumption that conceptual processing is influenced by seemingly unrelated perceptual information. Meier and colleagues (2004, p. 85) conclude that their results point to ‘the obligatory nature of affective inferences based on stimulus brightness’. Black and white stimuli are thus hypothesized to automatically activate negativity and positivity, respectively.
Other researchers have argued that the function of concrete source domains is to provide structure to abstract target domains (cf. Boroditsky, 2000; Gentner & Bowdle, 2008; Murphy, 1996)1. Through metaphoric representation, the relationship between concepts in the abstract domain can be structured by importing the relational structure from concrete domains. According to this view on metaphoric representations, the valence-brightness metaphor does not make us believe positivity is white, but the metaphor uses the perceptual opposition between black and white to structure the conceptual opposition between good and bad. Thus, the mapping of the abstract concepts ‘bad and good’ on the concrete experiences of ‘dark and light’ represents a shared relational structure, and not necessarily a direct and automatic association between darkness and negativity or brightness and positivity. In a set of six studies I pit a shared relational structure view against an automatic association account, with the objective to experimentally investigate how the brightness-valence metaphor influences conceptual processing.
In the remainder of this chapter, the assumptions underlying the automatic association account and the shared relational structure view
89
will be discussed in greater detail. Subsequently, studies on the explicit evaluation of the colors black and white will be summarized, which together with data from pilot studies reveal that while black is generally evaluated negatively, white is judged to be a neutral color. If white is a neutral color, then it is unlikely that white will automatically activate positivity. Instead, the positivity of white seems to emerge only in opposition to the negativity of the color black. Finally, a series of studies will be presented that were designed to test whether shared relational structures between the perceptual black-white opposition and the conceptual bad-good opposition are an essential pre-requisite for the metaphoric mapping of the valence dimension on the brightness dimension.
One-to-One Grounding
The question how the abstract domain of valence is grounded has been the subject of considerable research over the last few years (for a review, see Crawford, 2009). Meier, Robinson and Clore (2004) investigated whether the processing of positive and negative concepts is built on perceptual experiences. Focusing on the brightness-valence metaphor, they set out to test whether task-irrelevant perceptual characteristics of stimuli would automatically influence speeded evaluations of stimulus words. Participants were asked to evaluate positive and negative words, written in either a white or a black font. Negative words presented in black and positive words presented in white (so-called metaphor congruent mappings) were categorized faster and more accurately compared to positive words presented in black and negative words presented in white (or metaphor incongruent mappings). The authors conclude such metaphor congruency effects point ‘to the obligatory nature of affective inferences based on stimulus brightness’ (Meier et al., p. 85).
Meier and colleagues (2004) propose that metaphor congruency effects are the result of Stroop-like interference (for a similar reasoning, see Meier & Robinson, 2004, Schubert, 2005, Sherman & Clore, 2009). Due to the theorized obligatory nature of perceptual-conceptual links
90
(Lakoff & Johnson, 1999), the color white is hypothesized to automatically active positive valence. During a speeded evaluation task, the automatic activation of positivity when presented with a bright stimulus should facilitate positive evaluations, but interfere with negative evaluations. As Meier and colleagues explain (2004, p. 86): “a negative word presented in white would give rise to two response tendencies, one to respond ‘‘negative’’ (on the basis of stimulus valence) and one to respond ‘‘positive’’ (on the basis of stimulus color).”
According to the one-to-one grounding account, relationships between concrete sources (e.g., white) and abstract targets (e.g., positivity) are fixed neural connections, resulting from the co-activation of the two domains (Lakoff, 2008). The translation of perceptual information into conceptual information is argued to be “used constantly and automatically, with neither effort nor awareness’’ (Lakoff, 1993, p. 227– 228), and is thus assumed to be obligatory at the stage of word encoding (Meier & Robinson, 2004). Thus, perceiving the color white on its own is sufficient to activate its associated valence (e.g., positivity). I call this one- to-one grounding. Following the predictions from a one-to-one grounding perspective, the shared relational structure of the perceptual opposition between white and black and the conceptual opposition between good and bad is irrelevant for the brightness-valance mapping to emerge.
Shared Relational Structures
An alternative approach to the metaphoric mapping of valence and brightness presupposes that the main function of metaphors is to import a relational structure from concrete dimensions into an abstract domain (Murphy, 1996; Boroditsky, 2000; Gattis, 2001; Gentner & Bowdle, 2008). It has been noted that many image schema’s upon which conceptual metaphors are built (e.g., up-down, balance-imbalance, front-back, in- out) consist of polar opposites (Grady, 1997; Hampe, 2005; Krzeszowski, 1997). Therefore, the relational structure that is imported from concrete domains is often that of an opposition. With respect to the brightness-
91
valence metaphor, the opposition in the concrete brightness dimension is deployed to express the opposition in the abstract valence dimension. The opposition between white and black is imported in the valence domain through the brightness-valence metaphor to structure the relationship between good and bad.
If this shared relational structure view is correct, then the activation of the polar oppositions in the concrete dimension and the abstract domain is critical in grounding valence in brightness. Therefore, the main difference between the one-to-one grounding account and the shared relational structure view is that according to the latter the mere perceptual experience of ‘white’ will not automatically activate the meaning ‘positive’. Instead, the relational structure of the perceptual opposition in the brightness dimension (black vs. white) grounds the opposition in the abstract valence dimension (negative vs. positive). By implication, if a relational structure is not activated (e.g., only one endpoint of the brightness continuum is available), then the metaphoric mapping between brightness and valence is unlikely to emerge. Since the studies by Meier and colleagues (2004) and Sherman & Clore (2009) rely on within participant manipulation of the brightness and valence dimensions (each participants has to categorize both positive and negative words, presented both in a black as in a white font), these studies inevitably activate the good-bad and light-dark oppositions. Thus, the question whether black or white stimuli in isolation activate negativity and positivity remains open, making it impossible to distinguish between the one-to-one grounding account and the shared relational structures view.
Explicit Evaluations of Black and White
Color research investigating the explicit evaluation of black and white generally shows that people rate black as negative, while white is rated neutrally (e.g., Burkitt, Barrett & Davis, 2003; Götz & Götz, 1974; McManus, Jones, & Cotrell, 1981) or even slightly disliked, albeit to a lesser extent than the color black (Meerum Terwogt & Hoeksma, 1995).
92
The notion that white is a positive color comes not from color research, but from the highly influential cross-cultural studies on the measurement of meaning by Osgood and colleagues (Adams & Osgood, 1973; Osgood, 1960; Osgood, Suci, & Tanenbaum, 1957). However, it is important to note that in these studies the meaning of concepts is defined as a location in a multi-dimensional semantic space consisting of pairs of polar (opposite-in-meaning) adjectives. Therefore, the semantic relations in these studies should be interpreted as “the parallel alignment of two or more dimensions of experience, definable verbally by pairs of polar adjectives, with translations occurring between equivalent portions of the continua.” (Osgood et al., 1957, p. 23). In other words, white and black are functionally treated as opposites in these cross-cultural studies. As Osgood (1960, p. 155) summarizes: “If one visual alternative is chosen as appropriate to HAPPY (say, the light circle), we can predict that the other alternative will be chosen as appropriate to its opposite, SAD (the dark circle).” Also note that the Dutch sample in the cross-cultural study by Adams & Osgood (1973), rated white neutrally, as did many other samples from western countriesPsychological research suggests a clear anchoring of black as ‘bad’. For instance, referees have been shown to judge sports teams wearing black outfits as more aggressive, and consequently penalize these teams more often (Frank & Gilovich, 1988). In relation to the neutrality of white discussed above, it is noteworthy that no special positivity benefit was observed for teams wearing light outfits. The negativity of black and darkness has been argued to results from the fact that humans are diurnal animals (Williams & Morland, 1976). This evolutionary explanation is supported by research that shows people have a general tendency to feel less at ease in the dark, as signified by an increase in the human startle reflex compared to when the startle reflex is measured under normal lighting circumstances (Grillon, Pellowski, Merikangas, & Davis, 1997). Concluding, there is a strong association between darkness and negativity, but no strong clear association between white and positivity. The association between white and positivity does emerge when white is evaluated in opposition to black (Adams & Osgood,
93
1973; Osgood, 1960). Together, this pattern of evaluations seems to highlight
the importance of shared relational structures for the metaphoric representation of good and bad in light and dark, and to contradict the one-to-one grounding account, which assumes that the brightness of a stimulus automatically activates positive valence. The empirical studies in this chapter were developed with the aim of testing the one-to-one grounding account against the shared relational structure approach, in order to gain a better understanding of the process underlying the grounding of valence in brightness. If the shared relational structure between two dimensions is a prerequisite for metaphor congruency effects to occur, then breaking up the opposition in the valence (good vs. bad) and brightness (light vs. dark) dimensions by manipulating these polar endpoints between (instead of within) participants should influence whether the brightness-valance mapping emerges, especially for the white-positivity association.
Overview of the studies
According to the shared relational structure view the metaphoric representation of good and bad in light and dark hinges upon the opposition between the endpoints of the valence and brightness dimensions. According to the one-to-one grounding account, white should always activate associations with positivity, and black should always activate associations with negativity. In the remainder of this chapter, I will present data in line with the hypothesis that white in isolation is evaluated neutrally, and that the positivity of white emerges in conceptual or perceptual opposition with the negativity of the color black. The color black is negative in isolation, and is argued to be the relational tie between the valence and brightness dimensions. I will first discuss the results from three pilot studies confirming the negativity of black, and the neutrality of white, in line with the findings in the literature discussed previously.
Subsequently I introduce a new paradigm which does not rely on
94
response interference and allows us to manipulate dimensions between participants. As noted by Meier and Robinson (2004) choice reaction time paradigms based on speeded bimanual categorizations is a way to address the automaticity of the brightness-valence mapping. However, these bimanual categorization paradigms necessarily manipulate both the valence and the brightness of the stimuli within participants, thereby inherently activating the good-bad and white-black oppositions. Such response interference paradigms do not allow a direct comparison of within versus between participant manipulation of the white-black and good-bad oppositions. Furthermore, recent articles have revealed that response interference in paradigms with stimuli and responses of opposite evaluations are often not indicative of the intrinsic valence of a stimulus (e.g., the positivity of white) but the result of structural factors such as polarity correspondence (Proctor & Cho, 2006) or the top-down specification of stimulus-response codes (Eder & Rothermund, 2008).
In first four studies, I employed a novel Chinese Translation Paradigm, where participants are asked to indicate whether Chinese ideographs are the correct translation for a word presented on the screen. The idea of this paradigm is that irrelevant perceptual characteristics (in the current set of studies, their brightness, which was either white or black) will influence translation judgments for positive and negative words because the perceptual attributes are either congruent or incongruent with the brightness-valence metaphor. Given that the participants do not know the meaning of Chinese ideographs, this paradigm investigates judgments under uncertainty (cf. Crawford, Margolies, Drake, & Murphy, 2006, Meier, Hauser, Robinson, Kelland, Friesen, & Schjeldahl, 2007). As such, it does not allow us to test whether the brightness-valence metaphor is automatic. However, my main interest lies in determining whether the valence-brightness metaphor is conditional upon the presence of the shared structural relationship between the opposition in the brightness dimension and the opposition in the valence dimension.
By manipulating the brightness and valence dimensions within or between participants, this paradigm allows us to compare whether black
95
or white ideographs will be chosen above guessing average to represent the meaning of negative or positive words as a function of whether the opposition between black and white or positivity and negativity is present in the translation task or not. If the one-to-one grounding account is correct, participants should always judge white ideographs to correctly translate positive words above guessing average. However, if the shared relational structure of the opposition between black and white and the opposition between negativity and positivity is an essential pre-requisite, then positive words should be translated above guessing average by white ideographs only if the dimensions are manipulated within participants (activating the perceptual and conceptual oppositions), but not when the dimensions are manipulated between participants. Based on the fact that black is a negative color independent of the context, I assume that participants will always judge the black ideograph to be a correct translation of negative words above guessing average.
The final two studies aimed to provide direct support for the hypothesis that the positivity of white emerges in opposition to the negativity of black. By presenting only positive words and white ideographs these studies tested whether participants judged the white ideographs to correctly translate the positive stimulus words above guessing average in perceptual opposition to the negativity of black (by presenting the white ideographs on a gray vs. black background), or in conceptual opposition to the negativity of black (by priming participants with conceptual similarity or conceptual opposition). Finding support for the hypothesis that the positivity of white emerges in opposition to the negativity of black shows that certain metaphors (e.g., white is good) depend on the activation of the abstract and perceptual dimensions. This would support the assumption that the metaphors provide structure, but not necessarily meaning, to abstract concepts.
Color Evaluation Pilots
To complement the previously summarized literature on explicit color
96
ratings for black and white, I performed three pilot studies in empirical support for the neutral evaluation of white and the negative evaluation of black.
In a first pilot, I collected color ratings for 12 colors by asking 31 participants to indicate how positive each randomly presented color was on a scale from 1 to 9. Black was the most negative color (M = 4.13, SD = 2.68), white was rated around the midpoint of the scale (M = 5.13, SD = 2.31) with blue being the most positive color (M = 7.23, SD = 1.45). This pattern of color evaluations is identical to that found in previous studies (e.g., Eysenck, 1941; Frank, 1976; Meerum Terwocht & Hoeksma, 1995) where blue is the most preferred color (Crozier, 1999). In a second pilot, 90 participants evaluated only one color on a scale from -4 to + 4. Half of the participants reported the valence of black, and evaluated it as negative (M = -2.22, SD = 1.66) and the other half of the participants reported the valence of white, and evaluated it as neutral (M = 0.07, SD = 1.67).
A final pilot was performed to provide more conclusive support for the neutrality of the color white. If white has a neutral valence, then participants’ affective state should influence how this neutral stimulus is evaluated (Murphy & Zajonc, 1993; Cacioppo, Priester, & Berntson, 1993). Therefore, it was predicted that participants’ self-reported mood would correlate with their evaluations of the color white, but not with their evaluations of the color black, which is negatively valenced. Two colors were evaluated on a scale from -4 to +4 by 66 participants. The results revealed that white was evaluated around the midpoint of the scale (M = 0.53, SD = 1.89) whereas black was evaluated negatively (M = -2.52, SD = 2.12). More importantly, participants’ self-reported mood significantly correlated with participants’ evaluation of the color white (r = .30, p = .01) but not with their evaluation of the color black (r = -0.01, p = .95). Together, these results provide support for the assumption that white is a neutral color, and black is negative.
97
Pitting one-to-one grounding against shared relational structures
The current studies were developed with the aim to compare the shared relational structure view and the one-to-one grounding account by manipulating the valence and brightness dimension within or between participants in the next four studies. In Study 4.1 I implement the Chinese translation task in a conceptual replication of the studies by Meier and colleagues (2004) by manipulating both dimensions within participants. Participants are presented with positive and negative words and black and white Chinese ideographs. Positive words should be translated above guessing average by white ideographs, and negative words should be translated above guessing average by black ideographs. This outcome would be expected from both a one-to-one grounding account as well as from the shared relational structure view. In Study 4.2, brightness is manipulated within participants, but valence is manipulated between participants. While a one-to-one grounding account of valence in brightness would predict similar results as in Study 4.1, the shared relational structures view predict positive words will no longer be translated by white ideographs above guessing average. Because the polar opposition in the perceptual and conceptual dimensions is not activated, the positivity of white should not emerge.
In Study 4.3 brightness is manipulated between participants while valence is manipulated within participants. In order to manipulate brightness between participants, the Chinese translation paradigm had to be modified. Only one Chinese ideograph was presented, and participants had to judge whether the Chinese ideograph correctly translated the stimulus words by clicking on a ‘yes’ or ‘no’ button. A one-to-one grounding account would predict a replication of Study 4.1, whereas the shared relational structures view predicts no differences from guessing average for the translation judgments in the white ideograph condition, according to a similar reasoning as in Study 4.2. In Study 4.4 both brightness and valence are manipulated between participants. The
98
one-to-one grounding account would again predict an identical pattern of results as in Study 4.1. The shared relational structures view predicts no differences from guessing average when translating positive words by white ideographs. However, due to the intrinsic negativity of black, negative words are expected to be translated by black ideographs above guessing average in all four studies.
The final two studies aimed to provide direct support for the hypothesis that the positivity of white emerges in opposition to the negativity of black. Study 4.5 tested whether the positivity of white would emerge by perceptually priming the opposition between black and white. Participants were presented with positive words and white ideographs, either presented against a grey background (identical to Study 4.3) or against a black background. According to the shared relational structures view, presenting white ideographs against a black background (activating negativity) should induce the opposition between black and white in terms of the negativity of the background and the positivity of the words to be translated, and the positivity of white should emerge. The one-to- one grounding account would not predict differences in the translation judgments between the two conditions.
Study 4.6, goes one step further by conceptually priming participants to process the positive words and white ideographs in opposition to negativity and black. The conceptual opposition prime consists of an antonym generation task, whereas participants in the control condition generate synonyms. Since generating antonyms activates an opposition mindset in participants, I predicted the positive stimulus words and white ideographs would be processed more in relation to their opposites, even though these are not present in the task, compared to participants who performed a synonym generation task. As in Study 4.5, the hypothesis in Study 4.6 follows only from the shared relational structures view, and cannot be explained by a one-to-one grounding account.
99
Study 4.1: Valence and Brightness Within
To establish a paradigmatic baseline in the Chinese translation paradigm, I aimed to replicate the pattern of results as observed by Meier et al., (2004) by manipulating valence (positive vs. negative) and brightness (white vs. black) in a within participants design. Positive words were expected to be translated above chance by white ideographs, and negative words were expected to be translated above chance by black ideographs.
Method
Participants. Thirty-four paid voluntary students (23 females, mean age 20) at a Dutch university took part in this study for a monetary compensation.
Procedure. All instructions were presented in dark-grey letters on a perfectly gray background (RGB: 128,128,128). Participants were informed that recent studies had revealed that people who did not speak Hebrew nevertheless had a 60% chance of correctly translating Hebrew words. The current study would ostensibly investigate if people were similarly able to correctly translate Dutch words into Chinese ideographs above chance. Before starting the translation task, participants saw a Chinese ideograph (not used in the translation task) and were asked if they could read Chinese ideographs similar to the displayed example: if they indicated yes, participants were removed from the analysis. This general instruction and procedure was identical across the next six studies.
Participants received 12 stimulus words (six positive, six negative, see Appendix 2) presented in random order on a perfect grey background. Underneath each word two Chinese ideographs were displayed, one white, one black. Participants were asked to judge which of the two ideographs they thought correctly represented the meaning of the Dutch stimulus word. Twenty-four ideographs were randomly displayed in either white or a black2. The location of the black and white ideographs (left or right) was randomized between participants. There was no time limit for the
100
translation judgments. After performing the 12 translation judgments, participants were asked to rate the valence of the 12 words on a scale from 1 (very negative) to 7 (very positive).
Results
Manipulation check. Positive words were judged as more positive (M = 6.51) than negative words (M = 1.80), and a paired-samples t-test indicated this difference was significant, t(32) = 29,06, p < .001.
Translation judgments. Since the choices for white or black ideographs are mutually dependent, I only calculated the average number of times positive and negative words were translated by a white ideograph. Participants translated the six positive words above guessing average by white ideographs (61.67%, M = 3.70, SD = 1.22), as indicated by a test against chance, t(33) = 3.24, p < .01, Cohen’s d = .57. The six negative words were translates below guessing average by white ideographs, (40.34%, M = 2.42, SD = 1.08), t(33) = -3.03, p < .01, Cohen’s d = .54.
Discussion
Even though the brightness of the ideographs was irrelevant for the translation task, differences in the brightness of the ideographs influenced the likelihood with which they were judged to correctly translate positive or negative words. As such, this study replicates previous findings (Meier et al., 2004; Sherman & Clore, 2009) in a paradigm based on performing judgments under uncertainty.
Studies 2 to 4 were performed to investigate whether white ideographs would still be judged to correctly represent the meaning of positive words above guessing average when the polar oppositions in the valence and brightness dimensions were no longer present in the translation task. Either valence (Study 4.2), brightness (Study 4.3), or valence and brightness (Study 4.4) were manipulated between participants. If white activates positivity, then these studies should show an identical pattern of
101
results as Study 4.1. If the predictions of the shared relational structures view hold then negative words should be translated above chance level by black ideographs, but positive words should no longer be translated above guessing average by white ideographs.
Study 4.2: Valence Between, Brightness Within
Participants and procedure. Fifty-three students (31 females, mean age 20) at a Dutch university participated and were randomly assigned to the positive or negative word condition. The procedure was identical to Study 4.1, with the exception that valence of the 12 words were either all positive or all negative, depending on condition (see Appendix 2).
Results
Manipulation check. Positive words were judged as more positive (M = 6.33) than negative words (M = 2.08) as indicated by an independent- samples t-test, t(51) = 27.48, p < .001.
Translation judgments. The average number of times participants in the negative word condition chose the white ideograph to translate the stimulus word was significantly lower than guessing average, (39.17%, M = 4.70, SD = 2.84), t(26) = 2.37, p = .03, Cohen’s d = .46, indicating they preferred the black ideograph to translate negative words. In the positive word condition the likelihood with which participants translated positive words by white ideographs did not differ from guessing average, (51.92%, M = 6.23, SD = 1.89), t(25) = 0.63, p = .54, Cohen’s d = .01.
Discussion
If the valence dimension is manipulated between participants, white ideographs are no longer chosen to translate positive words above guessing average. Black ideographs are still seen as correctly translating negative words above chance. These results are in line with the shared relational
102
structures view, according to which the presence of the opposition in the valence and brightness dimensions is an essential pre-requisite for the positivity of white to emerge.
Study 4.3: Valence Within, Brightness Between
Where valence was manipulated between participants in Study 4.2, the current study compared the one-to-one grounding account with the shared relational structures view by manipulating brightness between participants. The translation task was adapted to enable the manipulation of ideograph brightness between participants. Instead of choosing between two Chinese ideographs (one black, one white), only one ideograph was displayed underneath each stimulus word, and participants were asked to judge whether the ideograph correctly translated the stimulus word or not by clicking YES or NO buttons. I predicted participants would judge black ideographs correctly translated negative words above chance. Based on the fact that the opposition in the brightness dimension is not present in the translation task, no differences from guessing average were expected in the white ideograph condition.
Method
Participants and procedure. Thirty-nine students (30 females, mean age 21) at a Dutch university participated in return for a monetary reward, and were randomly assigned to the white or black ideograph condition. Participants were asked to indicate if they felt the displayed Chinese ideograph was the correct translation of the stimulus word or not by clicking on the YES or the NO button on the left and right side of the ideograph. The location of the YES and NO buttons on the screen was randomized in each of the 12 trials. The same stimulus words were used as in Study 4.1.
103
Results
The average number of YES responses was calculated for positive and negative words in the black and white ideograph conditions. Since responses were no longer mutually exclusive as in Studies 4.1 and 4.2, where choosing the white ideograph meant participants did not choose the black ideograph, all four averages were tested against chance. In the white ideograph condition, participants’ average number of YES judgments did not differ from guessing average for positive (47.34%, M = 2.84, SD = 1.74) or negative (42.17%, M = 2.53, SD = 1.39) stimulus words, t(18) = 0.40, p = .70, Cohen’s d = .09 and t(18) = 1.49, p = .16, Cohen’s d = .34, respectively. In the black ideograph condition both averages for the positive words (39.17%, M = 2.35, SD = 0.99) and the negative (63.34%, M = 3.80, SD = 1.24) stimulus words deviated significantly from guessing average in the expected directions, t(19) = 2.94, p = .01, Cohen’s d = .66 and t(19) = 2.89, p = .01, Cohen’s d = .65, respectively.
Discussion
White ideographs did not influence translation judgments for positive or negative words. Black ideographs presented in isolation did influence translation judgments for positive and negative words. While the association between black and negativity was clearly hypothesized, the non-association between negativity and the opposite color (‘negative is not white’) is not a necessity, but is in line with the expectations. These findings are in line with the importance of the activation of shared relational structures in the brightness and valence domains, which predicts the positivity of white emerges only in opposition with the negativity of black. The differences between Study 4.3 and Study 4.1 are not a-priori predicted by a one-to-one grounding account.
104
Study 4.4: Valence and Brightness Between
I proceeded with a final comparison of the predictions made by the shared relational structures view and the one-to-one grounding account by manipulating both the valence and brightness dimensions between participants. Only translation judgments for negative words by black ideographs were expected to show a deviation from guessing average.
Method
Participants and procedure. A total of 81 students (54 females, mean age 20) at a Dutch university participated and were randomly assigned to one of the four conditions of the 2 (word valence: positive vs. negative) x 2 (ideograph brightness: white vs. black) between participants design. The procedure was identical to Study 4.3 with the exception that the valence of the 12 stimulus words was manipulated between participants as in Study 4.2.
Results
The average number of YES judgments in the negative word condition deviated significantly from chance, (57.50%, M = 6.90, SD = 1.14), t(20) = 3,67, p < .01, Cohen’s d =.80. No significant deviations from chance were observed in the positive word white ideograph condition (M = 6.05), t(20) = .15, p = .89, Cohen’s d = .03, negative word white ideograph condition, (55.67%, M = 6.68, SD = 1.70), t(18) = 1.75, p = .10, Cohen’s d = .40, and positive word black ideograph condition (55.67%, M = 6.45, SD = 1.91), t(19) = 1.06, p = .30, Cohen’s d = .24.
Discussion
The results confirm that even outside of any perceptual or conceptual opposition, black ideographs are still judged to correctly translate the
105
meaning of negative words above guessing average. This result is also important from a methodological perspective. The Chinese translation paradigm can be used to show the influence of perceptual characteristics of the ideographs on the likelihood with which they are seen to correctly translate valenced words, even without activating opposite endpoints of the conceptual or perceptual dimensions. This aspect of the Chinese translation paradigm will be built upon in the following two studies.
Study 4.5: Perceptual Opposition
After conceptually replicating earlier findings by Meier et al., (2004) in Study 4.1, the subsequent three studies supported the shared relational structures view by showing no deviations from guessing average for the average number of translations of positive words by white ideographs. In the current study, I aimed to activate the positivity of white not by manipulating both endpoints of the valence and brightness dimensions within participants as in Study 4.1, but by presenting positive words and white ideographs in perceptual opposition to the negativity of black. As the pilot studies have shown, a black background screen is evaluated negatively. Presenting positive words and white ideographs on a black background (which was negatively evaluated in the pilot studies reported earlier) should activate the opposition in the valence and brightness dimensions. Thus, translation judgments for positive words by white ideographs presented on a grey background should not differ from chance (as in Study 4.4), but when these white ideographs are presented on a black background, positive words would be judged to be correctly translated by white ideographs above guessing average.
Method
Participants and procedure. Forty-eight students (32 females, mean age 21) at a Dutch university participated in this experiment for a monetary reward and were randomly assigned to either the black or
106
grey background condition. The procedure was identical to the positive words white ideographs condition in Study 4.4. However, whereas for half the participants the white ideographs were presented on a perfect grey background (identical to studies 4.1 to 4.4), the white ideographs were presented on a black background for the other half of the participants.
Results
The average number of times participants answered YES to the question whether the white ideograph correctly translated the positive words was calculated. Whereas this average (49.33%, M = 5.92, SD = 1.19) did not differ from chance in the grey background condition, t(24) = -0.34, p = .74, Cohen’s d = .07, the average number of Yes judgments was higher than average when participants were asked to indicate whether the white ideograph correctly translated the positive word when it was presented against a black background (54.33%, M = 6.52, SD = 0.90), t(22) = 2.79, p = .01, Cohen’s d = .58.
Discussion
Presenting white ideographs and positive words against a black background increases the likelihood that participants will judge the ideograph to correctly translate the positive word. As predicted from the shared relational structures view, the positivity of white emerges after introducing the perceptual opposition between the negativity of black and the positive words and white ideographs. These effects cannot be explained by the one-to-one grounding account.
Study 4.6: Conceptual Opposition
After activating the shared relational structures between the brightness and valence dimensions through perceptual opposition in Study 4.5, the current study goes one step further by conceptually activating the
107
black-white and bad-good oppositions. I tested whether participants in a conceptual opposition mindset (activated by generating antonyms before performing the Chinese translation task) would process white and positivity in relation to their opposites, thus activating the shared relational structures between the valence and brightness dimensions, and allowing the positivity of white to emerge. The other half of the participants performed a synonym generation task (activating a similarity mindset) before performing the Chinese Translation Task. All participants judged whether white ideographs presented on a grey background correctly translated positive words. I expected participants to translate positive words above guessing average by white ideographs when participants first performed the antonym generation task, but not when participants first performed the synonym generation task.
Method
Participants and procedure. A total of 101 students (51 females, mean age 20) at a Dutch university took part in the current study and were randomly assigned to either the synonym or the antonym condition. Before performing the translation task, participants performed either the synonym or the antonym generation task, which would ostensibly measure language skills. Participants were presented with 20 words (see Appendix 3), one after another, and were instructed to generate and write down (depending on the experimental condition) an antonym or a synonym for each word. After the antonym or synonym generation task, participants performed the Chinese translation task where translation judgments had to be made for positive words and white ideographs presented on a grey background.
Results
Two participants were excluded from the analysis because they made more than 50% errors in the synonym generation task. The average
108
number of YES judgments in both conditions was calculated and tested these against guessing average. After the antonym generation task, white ideographs were judged to correctly represent the meaning of positive words above guessing average (54,08%, M = 6.49, SD = 1.26), t(50) = 2.79, p = .01, Cohen’s d = .39, whereas in the synonym condition the translation judgments did not differ from guessing average (51,92%, M = 6.23, SD = 1.50), t(47) = 1.08, p = .29, Cohen’s d = .16.
Discussion
After priming participants with a conceptual opposition mindset through an antonym generation task, participants were more likely to see white ideographs as correctly translating positive words. These results indicate that seeing white and positivity in relation to their polar opposites activates the positivity of white. These results support the shared relational structures view, and cannot be explained by the one-to-one grounding account.
General Discussion
The last years have showed a continued growth of studies from a grounded cognition perspective which together have revealed the influence of perceptual information on abstract conceptual processing (Boroditsky, 2000; Crawford et al., 2006; Meier et al., 2004; Meier et al., 2007; Santiago, Lupiáñez, Pérez, & Funes, 2007; Schubert, 2005). Recently, researchers have argued to move beyond demonstration studies, and investigate how perceptual information is related to conceptual processing (e.g., Barsalou, 2008a; Zwaan, 2009). In this set of studies, I compared two process explanations against each other by investigating whether the shared opposition in the brightness and valence dimensions is an essential pre- requisite for the brightness-valance mapping to occur. Over six studies, these results reveal that whereas black is indeed associated with negativity, the positivity of white emerges only in opposition to the negativity of black.
109
As such, the brightness-valence metaphor is dependent upon the within participant manipulation of both endpoints of the brightness and valence dimensions (Study 4.1) or upon the perceptual (Study 4.5) or conceptual (Study 4.6) activation of the polar oppositions in both dimensions.
These studies compared an one-to-one grounding account for metaphor congruency effects with a shared relational structures view. According to the one-to-one grounding account, the perceptual characteristics of stimuli (e.g., their brightness) automatically activate their associated evaluative meaning (e.g., Meier et al., 2004; Meier & Robinson, 2004; Sherman & Clore, 2009). According to the shared relational structures view, metaphors map a relational structure from a concrete domain (e.g., the opposition between black and white) unto an abstract domain (e.g., the opposition between bad and good). On the basis of the results of the current studies, I conclude that a one-to- one grounding account cannot fully explain the metaphoric grounding of valence in brightness, and that the activation of the shared relational structures in the valence and brightness domains is essential for the brightness-valance mapping to occur.
The metaphoric representation of abstract concepts has been widely investigated using judgments under uncertainty, such as asking ambiguous time questions (Boroditsky, 2000; Boroditsky & Ramscar, 2002), judgments about the number of calories in food (Williams & Bargh, 2008), estimations about the room temperature (Zhong & Leonardelli, 2008), or the vertical locations of affective (Crawford, et al., 2006) and divine pictures (Meier et al., 2007). A benefit of using the Chinese translation paradigm is the possibility to compare the effect of manipulating dimensions within or between participants, which is not possible in bimanual categorization tasks. As a consequence however, these findings are less conclusive on whether the relationship between brightness and valence is automatic or not.
The importance of activating the polar oppositions in the brightness and valence dimensions is not at odds with the assumption that the brightness valence relationship is automatic (Meier et al., 2004; Sherman
110
& Clore, 2009). The all-or-none conception of automaticity has been challenged in recent years (for a review, see Moors & De Houwer, 2006), and many processes which were previously assumed to be fully automatic, such as the Stroop effect argued to underlie metaphor congruency effects (Meier et al., 2004; Meier & Robinson, 2004; Schubert, 2005; Sherman & Clore, 2009), turn out to be more conditional and less automatic than assumed (e.g., Kahneman & Chajczyk, 1983; Klauer & Musch, 2002; Neumann, 1984). The attention assigned to stimulus dimensions has been shown to matter for automatic affective stimuli processing (e.g., Spruyt, De Houwer, Hermans, & Eelen, 2007). Therefore, the brightness- valence relation could still be conditionally automatic, depending on the activation of the shared opposition between white-black and good-bad. Given that the polar opposites in these dimensions are salient enough (for example, by manipulating the endpoints of these dimensions within subjects), the brightness-valence association could influence abstract conceptual processing automatically.
In addition to providing new insights in underlying process of valence-brightness metaphor, the current set of studies has important methodological implications. Especially when investigating the meaning of perceptual characteristics of stimuli, shared relational structures in within-participant designs can be an important source of emergent meaning. For example, two measures which are often used to measure a pro-white/anti-black bias in children are the Color Meaning Test II (CMT II; Williams, Boswell, & Best, 1975) and the Preschool Racial Attitude Measure II (PRAM II; Williams, Best, Boswell, Mattson, & Graves, 1975). Children read a sentence with a positive or negative adjective, and are asked to choose between a white or black animal or human (e.g., Which is the bad man?). Although the preference to choose white figures for positive behaviors and black figures for negative is often given a racial interpretation, these results show it is likely that these studies do not tap into the intrinsic meaning assigned to the colors black and white, given that children, just like adults, rate the color white as neutral (Burkitt et al., 2003; Meerum Terwogt & Hoeksma, 1995).
111
Based on these studies, I argue that these results should be interpreted with great caution, since the preference to link positive actions to white figures and negative actions to the black figures does not necessarily reflect underlying stereotypes. Instead, it could reflect a brightness-valence mapping due to the shared relational structures of the dimensions. To control for effects due to the shared relational structures between the valence and brightness dimensions, researchers should consider adapting the paradigm by presenting figures of only one color, and asking children if this figure performed a positive or negative action, yes or no. Such a reasoning could be extended to include many different paradigms which rely on the within-participant manipulation of two dimensions consisting of polar opposites.
The interplay between metaphoric grounding of abstract concepts and attentional processes such as the salience of the structural similarity between the polar opposites of the brightness and valence dimensions suggests an important direction that research from a grounded cognition perspective should take. Instead of merely demonstrating patterns of relationships between stimuli and responses, researchers should aim to investigate the underlying processes at both an embodied and cognitive level. The demonstration of the prerequisite of the polar opposition in the perceptual and conceptual dimensions highlights the psychological interface by means of which the grounding of valence is driven. I see these results as an important step in developing a more mature understanding of how abstract concepts are grounded.
112
Footnotes
1. Lakoff & Johnson (1980) acknowledge that some metaphors structure not one concept in terms of another, but a whole system of concepts with respect to another domain (so-called orientational metaphors). These metaphors, such as the vertical representation of affect, often structure abstract concepts in spatial dimensions, such as the polar- oppositions up-down or in-out. However, throughout their work, such orientiational metaphors are argued to be based on two complementary metaphors (e.g., GOOD IS UP, BAD IS DOWN), and it is unclear whether the relational structures in these polar oppositions have a special status beyond the sum of their parts.
2. In a pilot study, sixty-five students rated the valence of 10 black or white Chinese ideograms on a 7 point scale from -3 (very negative) through 0 (neutral) to +3 (very positive), with the brightness of the ideograms manipulated between participants. The average rating was calculated for the ideograms and a two-sample t-test was used to compare the differences between ratings of the black and white ideograms. The valence ratings for the two ideograms did not differ for the white (M = 0.06, SD = 0.48) and black (M = 0.09, SD = 0.57) ideograms, t(63) = .17, p = .87, n.s..
113
Appendix 1
Moral Words: Charity*, Honest*, Equal*, Truth*, Good*, Honorable*, Integrity*, Righteous*, Rightful*, Moral*, Trust, Donate, Ethical, Volunteer, Just, Decent, Virtuous, Lawful, Assist, Noble. Immoral Words: Corrupt*, Lie*, Molest*, Deceive*, Injustice*, Immoral*, Unfair*, Evil*, Theft*, Traitor*, Adultery, Bribe, Self-centered, Rob, Sinfulness, Untrustworthy, Villainous, Selfish, Fraud, Favor. *Stimuli used in Study 2.1.
Appendix 2
Positive Words: Baby*, Brave*, Hero*, Love*, Sincere*, Loyal*, Party, Humour, Friend, Nice, Shining, Pleasurable. Negative Words: Death*, Cruel*, Poison*, Sick*, Enemy*, Hate*, Criminal, Cancer, Disgust, Unfair, Mean, Filthy. *Stimulus words used in Experiment 4.1 and 4.3
Appendix 3
Words presented during the antonym and synonym task: Beginning, Brave, Broken, Calm, Closed, Correct, Dirty, Drunk, Expensive, Fail, Get, Handsome, Naked, Nice, Order, Simple, Special, Talk, Thin, Weak.
115
References Adams, J. S. (1965). Inequity in social exchange. In L. Berkowitz (Ed.),
Advances in experimental social psychology, (Vol. 2, pp. 267-299). New York: Academic Press.
Adams, F. M., & Osgood, C. E. (1973). A cross-cultural study of the affective meanings of color. Journal of Cross-Cultural Psychology, 4, 135-156.
Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Lawrence Erlbaum.
Andrews, M., Vigliocco, G., & Vinson, D. (2009). Integrating experiential and distributional data to learn semantic representations. Psychological Review, 116, 463-498.
Arbib, M. A. (2008). From grasp to language: Embodied concepts and the challenge of abstraction. Journal of Physiology-Paris, 102, 4-20.
Barsalou, L.W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577-660.
Barsalou, L. W. (2008a). Grounded cognition. Annual Review of Psychology, 59, 617-645.
Barsalou, L.W. (2008b). Grounding symbolic operations in the brain’s modal systems. In G. R. Semin & E. R. Smith (Eds.), Embodied grounding: Social, cognitive, affective, and neuroscientific approaches (pp. 9-42). New York: Cambridge University Press.
Barsalou, L.W., & Wiemer-Hastings, K. (2005). Situating abstract concepts. In D. Pecher and R. Zwaan (Eds.), Grounding cognition: The role of perception and action in memory, language, and thought (pp. 129-163). New York: Cambridge University Press.
Basso, G., Nichelli, P., Frassinetti, F., & Di Pellegrino, G. (1996). Time perception in a neglected space. Neuroreport, 7, 2111–2114.
Belin, P., Zilbovicius, M., Crozier, S., Thivard, L., Fontaine, A., Masure, M. C., et al. (1998). Lateralization of speech and auditory temporal
116
processing. Journal of Cognitive Neuroscience, 10, 536-540. Benor, S. B., & Levy, R. (2006). The chicken or the egg? A probabilistic
analysis of English binomials. Language, 82, 233-278. Bergen, B., Lindsay, S., Matlock, T., & Narayan, S. (2007). Spatial and
linguistic aspects of visual imagery in sentence comprehension. Cognitive Science, 31, 733–764.
Bjoertomt, O., Cowey, A., & Walsh, V. (2002). Spatial neglect in near and far space investigated by repetitive transcranial magnetic stimulation. Brain, 125, 2012–2022.
Blauert, J. (1997). Spatial hearing: The psychophysics of human sound localization. Revised edition. Cambridge, MA: MIT Press.
Boot, I., & Pecher, D (in press). Similarity is closeness: metaphorical mapping in a conceptual task. Quarterly Journal of Experimental Psychology.
Boroditsky, L. (2000). Metaphoric structuring: Understanding time through spatial metaphors. Cognition, 75, 1-28.
Boroditsky, L. (2001). Does language shape thought? Mandarin and English speakers’ conceptions of time. Cognitive Psychology, 43, 1-22.
Boroditsky, L., & Prinz, J. J. (2008). What thoughts are made of. In G. R. Semin and E. R. Smith (Eds.), Embodied Grounding: Social, Cognitive, Affective, and Neuroscientific Approaches, (pp. 98 - 115). Cambridge: Cambridge University Press.
Boroditsky, L., & Ramscar, M. (2002). The roles of body and mind in abstract thought. Psychological Science, 13, 185-189.
Broeders, R., Van den Bos, K., & Müller, P. A. (submitted). Embodied power to the people: Powerful postures decrease the inhibition to intervene in moral and bystander dilemmas.
Bryant, P., & Squire, S. (2001). Children’s mathematics: Lost and found in space. In M. Gattis, (Ed.), Spatial schemas and abstract thought (pp. 175–201). MIT Press.
Burkitt, E., Barrett, M., & Davis, A. (2003). Children’s colour choices for completing drawings of affectively characterised topics. Journal of
117
Child Psychology and Psychiatry, 44, 445-455. Cacioppo, J. T., Priester, J. R., & Berntson, G. G. (1993). Rudimentary
determinants of attitudes: II. Arm flexion and extension have differential effects on attitudes. Journal of Personality and Social Psychology, 65, 5-17.
Casasanto, D. (in press). When is a linguistic metaphor a conceptual metaphor? In V. Evans & S. Pourcel (Eds.), New Directions in Cognitive Linguistics. Amsterdam: John Benjamins.
Casasanto, D., & Boroditsky, L. (2008). Time in the mind: Using space to think about time. Cognition, 106, 579-593.
Chapman, H. A., Kim, D.A., Susskind, J.M., & Anderson, A.K. (2009). In bad taste: Evidence for the oral origins of moral disgust. Science, 323, 1222-1226.
Clark, H. H. (1969). Linguistic processes in deductive reasoning. Psychological Review, 76, 387-404.
Clark, H. H. (1973). Space, time, semantics and the child. In T. E. Moore (Ed.), Cognitive development and the acquisition of language (pp. 27–63). New York: Academic Press.
Cooper, W. E., & Ross, J. R. (1975). World order. In R. E. Grossman, L. J. San, & T. J. Vance (Eds.), Papers from the parasession on functionalism (pp. 63-111). Chicago: Chicago Linguistic Society.
Crawford, L. E., (2009). Conceptual metaphors of affect. Emotion Review, 1, 129-139.
Crawford, L. E., Margolies, S. M., Drake, J. T., & Murphy, M. E. (2006). Affect biases memory of location: Evidence for the spatial representation of affect. Cognition and Emotion, 20, 1153–1169.
Crozier, W. R. (1999). The meanings of colour: Preferences among hues. Pigment & Resin Technology, 28, 6-14.
Damasio, A. R. (1989). Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition. Cognition, 33, 25-62.
Dennett, D. C. (1969). Content and Consciousness. London: Routledge & Kegan Paul.
118
De Soto, C. B., London, M., and Handel, S. (1965). Social reasoning and spatial paralogic. Journal of Personality and Social Psychology, 2, 513-521.
Dehaene, S., Bossini, S., & Giraux, P. (1993). The mental representation of parity and number magnitude. Journal of Experimental Psychology: General, 122, 371–396.
Dove, G. (2009). Beyond perceptual symbols: A call for representational pluralism. Cognition, 110, 412-431.
Eder, A. B., & Rothermind, K. (2008). When do motor behaviors (mis) match affective stimuli? An evaluative coding view of approach and avoidance reactions. Journal of Experimental Psychology: General, 137, 262-281.
Estes, Z., Verges, M., & Barsalou, L. W. (2008). Head up, foot down: Object words orient attention to the objects’ typical location. Psychological Science, 19, 93–97.
Eysenck, H. J. (1941). A critical and experimental study of color preferences. American Journal of Psychology, 54, 385-391.
Feldman, J. (2006). From molecules to metaphor: a neural theory of language. Cambridge: MIT Press.
Fischer, M. H., & Zwaan, R. A. (2008). Embodied language: A review of the role of the motor system in language comprehension. Quarterly Journal of Experimental Psychology, 61, 825-850.
Fiske, A. (1992). The four elementary forms of sociality: Framework for a unified theory of social relations. Psychological Review, 99, 689-723.
Fodor, J. A. (1975). The language of thought. Cambridge, MA: Harvard University Press.
Fodor, J. A. (1980). Methodological solipsism considered as a research strategy in cognitive psychology. Behavioral and Brain Sciences, 3, 63 - 109.
Fodor, J. A. (1983). The Modularity of Mind. Cambridge, MA: MIT Press. Fodor, J. A. (1985). Précis of The Modularity of Mind. Behavioral and
Brain Sciences, 8, 1 - 42. Folger, R. (1984). Emerging issues in the social psychology of injustice. In
119
R. Folger (Ed.), The sense of injustice: Social psychological perspectives (pp. 3 - 24). New York: Plenum.
Frank, G. (1976). On the validity of a hypothesis derived from the Rorschach: The relationship between color and affect. Perception and Motor Skills, 43, 411-427.
Frank, M. G., & Gilovich, T. (1988). The dark side of self- and social perception: Black uniforms and aggression in professional sports. Journal of Personality and Social Psychology, 54, 74-85.
Gallese V., & Lakoff, G. (2005). The brain’s concepts: the role of the sensory- motor system in reason and language. Cognitive Neuropsychology, 22, 455-479.
Gattis, M. (2001). Spatial schemas and abstract thought. Cambridge, MA: MIT Press.
Gattis, M. (2002). Structure mapping in spatial reasoning. Cognitive Development, 17, 1157-1183.
Gentner, D, & Bowdle, B. (2008). Metaphor as structure mapping. In R. W. Gibbs (Ed.). Cambridge Handbook of Metaphor and Thought (pp. 109 – 128). Cambridge, MA, Cambridge University Press.
Gibbs, R. W. (1996). Why many concepts are metaphorical. Cognition, 61, 309-319.
Gibbs, R. W. (2005). Embodiment in metaphorical imagination. In R. Zwaan and D. Pecher (Eds.), The grounding of cognition: The role of perception and action in memory, language, and thinking (pp. 65- 92). Cambridge: Cambridge University Press.
Glenberg, A., & Kaschak, M. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9, 558–565.
Glenberg, A. M., & Robertson, D. A. (1999). Indexical understand-ing of instructions. Discourse Processes, 28, 1-26.
Glenberg, A. M., & Robertson, D. A. (2000). Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning. Journal of Memory and Language, 43, 379-401.
Glenberg, A. M., Sato, M., Cattaneo, L. (2008). Use-induced motor plasticity affects the processing of abstract and concrete language.
120
Current Biology, 18, R290-R291. Glenberg, A. M., Sato, M., Cattaneo, L., Riggio, L., Palumbo, D., Buccino,
G. (2008). Processing abstract language modulates motor system activity. Quarterly Journal of Experimental Psychology, 61, 905-919.
Grady, J. (1997). Foundations of meaning: Primary metaphors and primary scenes. Unpublished doctoral dissertation, University of California, Berkeley.
Graham, J., Haidt, J., & Nosek, B. A. (in press). Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology.
Greenberg, J. H. (1963). Some universals of grammar with particular reference to the order of meaningful elements. In J. H. Greenberg (Ed.), Universals of grammar (pp. 73-113). Cambridge, MA: MIT Press.
Greene, J. D., Cushman, F. A., Stewart, L. E., Lowenberg, K., Nystrom, L. E., & Cohen, J. D. (in press). Pushing moral buttons: The interaction between personal force and intention in moral judgment. Cognition, doi:10.1016/j.cognition.2009.02.001
Grillon, C., Pellowski, M., Merikangas, K. R., & Davis, M. (1997). Darkness facilitates acoustic startle reflex in humans. Biological Psychiatry, 42, 453-460.
Götz, K.O., Götz, K. (1974) Color attitudes of art students and university students: I. Imagined colors. Perceptual and Motor Skills, 1974, 38, 63-70.
Haidt, J. (2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review, 108, 814–834.
Haidt, J. (2008). Morality. Perspectives on Psychological Science, 3, 65-72. Hampe, B. (2005). When down is not bad, and up not good enough: A
corpus-based assessment of the plus-minus parameter in image schema theory. Cognitive Linguistics, 16, 81-112.
Harnad, S. (1990). The symbol grounding problem. Physica D, 42, 335- 346.
Hauk, O., Johnsrude, I., & Pulvermuller, F. (2004). Somatotopic
121
representation of action words in human motor and premotor cortex. Neuron, 41, 301–307.
Hommel, B., & Prinz, W. (Eds.). (1997). Theoretical issues in stimulus– response compatibility. Amsterdam: North-Holland.
IJzerman, H. & Semin, G. R. (2009). The Thermometer of Social Relations: Mapping Social Proximity on Temperature. Psychological Science, 20, 1214-1220.
Jackendoff, R. (2002). Foundations of language: Brain, meaning, grammar, evolution. Oxford University Press.
Jeannerod, M. (2001). Neural simulation of action: A unifying mechanism for motor cognition. NeuroImage, 14, 103–109.
James, W. (1890). Principles of psychology. New York: Holt. Johnson, M. (1987). The body in the mind: The bodily basis of meaning,
imagination, and reason. Chicago: University of Chicago Press. Johnson, M. (1993). Moral imagination. Chicago: University of Chicago
Press. Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science
of language, inference, and consciousness. Harvard University Press.
Jostmann, N. B., Lakens, D., & Schubert, T. W. (2009). Weight as an embodiment of importance. Psychological Science, 20, 1169-1174.
Kahneman, D., & Chajczyk, D. (1983). Test of the automaticity of reading: Dilution of Stroop effects by color-irrelevant stimuli. Journal of Experimental Psychology: Human Perception and Performance, 9, 497–509.
Kan, I. P., Barsalou, L. W., Solomon, K. O., Minor, J. K., & Thompson- Schill, S. L. (2003). Role of mental imagery in a property verification task: fMRI evidence for perceptual representations of conceptual knowledge. Cognitive Neuropsychology, 20, 525-540.
Karwoski, T. F., Odbert, H. S., & Osgood, C. E. (1942). Studies in synesthetic thinking: II. The role of form in visual responses to music. Journal of General Psychology, 26, 199-222.
Kaschak, M., Madden, C., Therriault, D., Yaxley, R., Aveyard, M.,
122
Blanchard, A., & Zwaan, R. A. (2005). Perception of motion affects language processing. Cognition, 94, B79–B89.
Kimura, D. (1961). Cerebral dominance and the perception of verbal stimuli. Canadian Journal of Psychology, 15, 156-165.
Klauer, K. C., & Musch, J. (2002). Goal-dependent and goal-in-dependent effects of irrelevant evaluations. Personality and Social Psychology Bulletin, 28, 802–814.
Kohlberg, L. (1969). Stage and sequence: The cognitive-developmental approach to socialization. In D. A. Goslin (Ed.), Handbook of socialization theory and research. Chicago: Rand McNally.
Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: Cognitive basis of stimulus-response compatibility - A model and taxonomy. Psychological Review, 97, 253-170.
Kosslyn S. M. (1980). Image and Mind. Cambridge, MA: Harvard University Press.
Krzeszowski, T. P. (1997). Angels and devils in hell: Elements of axiology in semantics. Warsaw: Wydawn.
Laham, S. M., Alter, A. L., & Goodwin, G. P. (in press). Easy on the mind, easy on the wrongdoer: Discrepantly fluent violations are deemed less morally wrong. Cognition, doi:10.1016/j.cognition.2009.06.001
Lakens, D., Semin, G. R., & Foroni, F. (in preparation). But for the bad, there would not be good: Grounding valence in brightness through shared relational structures.
Lakens, D., Semin, G. R., & Garrido, M. (under review). The sound of time: Locating time in auditory space.
Lakens, D., Van den Bos, K., & Semin, G. R. (in preparation). On the grounding of morality in perceptual symmetry.
Lakoff, G. (1987). Women, fire, and dangerous things: What categories reveal about the mind. Chicago: University of Chicago Press.
Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortony (Ed.), In Metaphor and Thought, (pp. 202-251). Cambridge: Cambridge University Press.
Lakoff, G. (1996). Moral politics: What conservatives know that liberals
123
don’t. Chicago: University of Chicago Press. Lakoff, G. (2005). A cognitive scientist looks at Daubert. American Journal
of Public Health, 95, S114–S120. Lakoff, G. (2008). The neural theory of metaphor. In: R. W. Gibbs (Ed.),
Cambridge handbook of metaphor and thought (pp. 17 – 38). Cambridge, MA, Cambridge University Press.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press.
Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. Chicago: University of Chicago Press.
Landauer, T. K. & Dumais, S. T. (1997) A solution to Plato’s problem: The Latent Semantic Analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–40.
Langacker, R.W. (1987). Foundations of cognitive grammar: Vol. 1. Theoretical prerequisites. Stanford, CA: Stanford University Press.
Lindner, S. (1983). A lexic-semantic analysis of verb-particle constructions with up and out. Doctoral dissertation. University of California, San Diego. Reproduced by Linguistic Agency University Trier.
Louwerse, M. M. (2008). Embodied representations are encoded in language. Psychonomic Bulletin and Review, 15, 838-844.
Lu, C. H., & Proctor, R. W. (1995). The influence of irrelevant location information on performance: A review of the Simon and spatial Stroop effects. Psychonomic Bulletin & Review, 5, 174-207.
Maass, A., & Russo, A. (2003). Directional bias in the mental representation of spatial events: Nature or culture? Psychological Science, 14, 296– 301.
Machery, E. (2009). Doing Without Concepts. New York: Oxford University Press.
Mahon, B. Z., & Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. Journal of Physiology-Paris, 102, 59–70.
Mahon, B. Z., & Caramazza, A. (2009). Concepts and categories: A
124
cognitive neuropsychological perspective. Annual Review of Psychology, 60, 27-51.
MacLeod, C. (1991). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109, 163–203.
Mandler, J. (2004). The foundation of mind: Origins of conceptual thought. New York: Oxford University Press.
Mandler M. (1992). How to build a baby: II. Conceptual primitives. Psychological Review, 99, 587–604
Marks, L. E. (1978). The unity of the senses: Interrelations among the modalities. New York: Academic Press.
Marks, L. E. (1987). On cross-modal similarity: Auditory-visual interactions in speeded discrimination. Journal of Experimental Psychology: Human Perception and Performance, 13, 384-394.
Mayerthaler, W. (1980). Aspekte der Analogietheorie. In H. Lüdtke (Ed.), Kommunikationstheoretische Grundlagen des Sprachwandels (pp. 80-130). Berlin: de Gruyter.
McManus, I.C., Jones, A.L., Cottrell, J. (1981). The aesthetics of colour. Perception, 10, 651-666.
McGlone, M.S. (2007). What is the explanatory value of a conceptual metaphor? Language & Communication, 27, 109-126.
McGlone, M. S., & Harding, J. L. (1998). Back (or forward?) to the future: the role of perspective in temporal language comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 1211-1223.
Meerum Terwogt, M., & Hoeksma, J. B. (1995). Colors and emotions: Preferences and combinations. Journal of General Psychology, 122, 5-13.
Meier, B. P., Hauser, D. J., Robinson, M. D., Kelland Friesen, C., & Schjeldahl, K. (2007). What’s “up” with God? Vertical space as a representation of the divine. Journal of Personality and Social Psychology, 93, 699 – 710.
Meier, B. P., & Robinson, M. D. (2004). Why the sunny side is up: Associations between affect and vertical position. Psychological
125
Science, 15, 243-247. Meier, B. P., & Robinson, M. D. (2005). The metaphorical representation
of affect. Metaphor and Symbol, 20, 239-257. Meier, B. P., & Robinson, M. D. (2006). Does “feeling down” mean seeing
down? Depressive symptoms and vertical selective attention. Journal of Research in Personality, 40, 451–461.
Meier, B. P., Robinson, M. D., & Caven, A. J. (2008). Why a big mac is a good mac: Associations between affect and size. Basic and Applied Social Psychology, 30, 46-55.
Meier, B. P., Robinson, M. D., & Clore, G. L. (2004). Why good guys wear white: Automatic inferences about stimulus valence based on brightness. Psychological Science, 15, 82-87.
Meier, B. P., Robinson, M. D., Crawford, L. E., & Ahlvers, W. J. (2007). When “light” and “dark” thoughts become light and dark responses: Affect biases brightness judgments. Emotion, 7, 366–376.
Meier, B. P., Sellbom, M., & Wygant, D. B. (2007). Failing to take the moral high ground: Psychopathy and the vertical representation of morality. Personality and Individual Differences, 43, 757–767.
Messick, D. M. (1993). Equality as a decision heuristic. In B. A. Mellers & J. Baron (Eds.), Psychological perspectives on justice: Theory and applications (pp. 11-31). Cambridge, MA: Cambridge University Press.
Meteyard, L. & Vigliocco, G. (2008). The role of sensory and motor information in semantic representation: A review. In P. Calvo and A. Gomila (Eds.), Handbook of cognitive science: An embodied approach (pp. 293 - 312). Academic Press, London.
Murphy, G. (1996). On metaphoric representation. Cognition, 60, 173– 204.
Neumann, O. (1984). Automatic processing: A review of recent findings and a plea for an old theory. In W. Prinz & A. F. Sanders (Eds.), Cognition and motor processes (pp. 255-293). Berlin, Germany: Springer.
Niedenthal, P. M., Barsalou, L., Winkielman, P., Krauth-Gruber, S., & Ric,
126
F. (2005). Embodiment in attitudes, social perception, and emotion. Personality and Social Psychology Review, 9, 184-211.
Núñez, R., & Sweetser, E. (2006). Looking ahead to the past: Convergent evidence from aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30, 401- 450.
Osgood, C. E. (1952). The nature and measurement of meaning. Psychological Bulletin, 49, 197-237.
Osgood, C. E. (1960). The cross-cultural generality of visual-verbal synesthetic tendencies. Behavioral Science, 5, 146-169.
Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). The measurement of meaning. Urbana, IL: University of Illinois Press.
Pecher, D., Zeelenberg, R., & Barsalou, L. W. (2003). Verifying different- modality properties for concepts produces switching costs. Psychological Science, 14, 119 – 124.
Pecher, D., Zeelenberg, R., & Barsalou, L. W. (2004). Sensorimotor simulations underlie conceptual representations: Modality-specific effects of prior activation. Psychonomic Bulletin and Review, 11, 164–167.
Pecher, D. & Zwaan, R.A. (Eds.). (2005). Grounding cognition: The role of perception and action in memory, language, and thinking. Cambridge, UK: Cambridge University.
Piaget, J. (1954). The construction of reality in the child. New York: Basic Books.
Prinz, J. J. (2005). Passionate Thoughts. In R. Zwaan and D. Pecher (Eds.), The grounding of cognition: The role of perception and action in memory, language, and thinking (pp. 93-114). Cambridge: Cambridge University Press.
Prinz, J. J. (2007). The emotional construction of morals. Oxford: Oxford University Press.
Proctor, R. W., & Cho, Y. S. (2006). Polarity correspondence: A general principle for performance of speeded binary classification tasks. Psychological Bulletin, 132, 416-442.
127
Pujol, J, Deus J, Losilla, J. M., & Capdevila, A. (1999) Cerebral lateralization of language in normal left-handed people studied by functional MRI. Neurology, 52, 1038-1043.
Pulvermüller, F. (1999). Words in the brain’s language. Behavioral and Brain Sciences, 22, 253–336.
Pulvermüller, F. (2005). Brain mechanisms linking language and action. Nature Reviews Neuroscience, 6, 576–582.
Pulvermüller, F., Hauk, O., Nikulin, V. & Ilmoniemi, R. J. (2005). Functional links between motor and language systems. European Journal of Neuroscience, 21, 793-797.
Pylyshyn, Z. W. (1984). Computation and cognition. MIT Press. Rao, S. M., Mayer, A. R., & Harrington, D. L. (2001). The evolution of
brain activation during temporal processing. Nature Neuroscience, 4, 317–323.
Rawls, J. (1971). A theory of justice. Cambridge, MA: Harvard University Press.
Reber, R., Schwarz, N., & Winkielman, P. (2004). Processing fluency and aesthetic pleasure: Is beauty in the perceiver’s processing experience? Personality and Social Psychology Review, 8, 364–382.
Richards, I. A. (1936). The Philosophy of Rhetoric. Oxford, Oxford University Press.
Richardson, D. C., Spivey, M. J., & Cheung, J. (2001). Motor representations in memory and mental models: Embodiment in cognition. In Proceedings of the 23rd Annual Meeting of the Cognitive Science Society (pp. 867–872). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Richardson, D. C., Spivey, M. J., McRae, K., & Barsalou, L. W. (2003). Spatial representations activated during real-time comprehension of verbs. Cognitive Science, 27, 767–780.
Rudel, R. G., & Teuber, H.-L. (1964). Cross modal transfer of shape discrimination by children. Neuropsychologia, 2, 1-8.
Santiago, J., Lupiáñez, J., Pérez, E., & Funes, M. J. (2007). Time (also) flies from left to right. Psychonomic Bulletin & Review, 14, 512-516.
128
Scherer, L. D., & Lambert, A. J. (2009). Contrast effects in priming paradigms: Implications for theory and research on implicit attitudes. Journal of Personality and Social Psychology, 97, 383–403.
Schnall, S., Benton, J., & Harvey, S. (2008). With a clean conscience: Cleanliness reduces the severity of moral judgments. Psychological Science, 19, 1219-1222.
Schnall, S., Haidt, J., Clore, G., & Jordan, A. H. (2008). Disgust as embodied moral judgment. Personality & Social Psychology Bulletin, 34, 1096- 1109.
Schubert, T. W. (2005). Your highness: Vertical positions as perceptual symbols of power. Journal of Personality and Social Psychology, 89, 1-21.
Searle, J. (1980). Minds, brains, and programs, Behavioral and Brain Sciences, 3, 417-58.
Semin, G. R., & Smith, E. R. (Eds.). (2008). Embodied grounding: Social, cognitive, affective, and neuroscientific approaches. New York: Cambridge University Press.
Šetić, M., & Domijan, D. (2007). The influence of vertical spatial orientation on property verification. Language & Cognitive Processes, 22, 297- 312.
Sherman, G. D., & Clore, G. L. (2009). The color of sin: White and black are perceptual symbols of moral purity and pollution. Psychological Science, 20, 1019-1025.
Simmons, W.K., Hamann, S.B., Harenski, C.N., Hu, X.P., & Barsalou, L.W. (2008). fMRI evidence for word association and situated simulation in conceptual processing. Journal of Physiology–Paris, 102, 106-119.
Simmons, W. K., Ramjee, V., Beauchamp, M. S., McRae, K., Martin, A., & Barsalou, L. W. (2007). A common neural substrate for perceiving and knowing about color. Neuropsychologia, 45, 2802–2810.
Smith, L. B. (2005). Action alters shape categories. Cognitive Science, 29, 665–679.
Spruyt, A., De Houwer, J., Hermans, D., & Eelen, P. (2007). Affective priming of non-affective semantic categorization responses.
129
Experimental Psychology, 54, 44-53. Stanfield, R., & Zwaan, R. (2001). The effect of implied orientation derived
from verbal context on picture recognition. Psychological Science, 12, 153–156.
Talmy, L. (1983). How language structures space. In H. Pick & L. Acredelo (Eds.), Spatial orientation: Theory, research, and application (pp. 225-282). New York: Plenum Press.
Taylor, L. J., & Zwaan, R. A. (2008). Motor resonance and linguistic focus. Quarterly Journal of Experimental Psychology, 61, 896-904.
Toga, A. W., & Thompson, P. M. (2003). Mapping brain asymmetry. Nature Reviews Neuroscience, 4, 37-48.
Torralbo, A., Santiago, J., & Lupiáñez, J. (2006). Flexible conceptual projection of time onto spatial frames of reference. Cognitive Science, 30, 745-757.
Tversky, B., Kugelmass, S., & Winter, A. (1991). Cross-cultural and developmental trends in graphic productions. Cognitive Psychology, 23, 515-557.
Umiltà, C., & Nicoletti, R. (1990). Spatial S–R compatibility. In R. W. Proctor & T. G. Reeve (Eds.), Stimulus–response compatibility: An integrated perspective (pp. 89–116). Amsterdam: North-Holland.
Vallesi, A., Binns, M. A., & Shallice, T. (2008). An effect of spatial-temporal association of response codes: Understanding the cognitive representations of time. Cognition, 107, 501-527.
Van Dantzig, S., Zeelenberg, R., & Pecher, D. (2009). Unconstraining theories of embodied cognition. Journal of Experimental Social Psychology, 45, 345-351.
Van Dantzig, S., Pecher, D., & Zwaan, R. A. (2008). Approach and avoidance as action effects. Quarterly Journal of Experimental Psychology, 61, 1298-1306.
Varela, F., Thompson, E., & Rosch, E. (1991). The embodied mind. Cambridge, MA: MIT Press.
Verges, M., & Duffy, S. (2009). Spatial representations elicit dual-coding effects in mental imagery. Cognitive Science, 33, 1157-1172.
130
Vigliocco, G., Vinson, D. P., Lewis, W., & Garrett, M. F. (2004). Representing the meaning of object and action words: The featural and unitary semantic space hypothesis. Cognitive Psychology, 48, 422-488.
Weger, U. W., & Pratt, J. (2008). Time flies like an arrow: Space- time compatibility effects suggest the use of a mental time line. Psychonomic Bulletin & Review, 15, 426-430.
Whitney, W. D. (1875). The life and growth of language: An outline of linguistic science. New York: Appleton.
Wijn, R., & Van den Bos, K. (in press). On the social-communicative function of justice: the influence of communication goals and personal involvement on the use of justice assertions. Personality and Psychology Bulletin.
Williams, J. E., Boswell, D. A., Best, D. L. (1975). Evaluative responses of preschool children to the colors white and black. Child Development, 46, 501-508.
Williams, J. E, Best, D. L., Boswell, D. A., Mattson, L. A., & Graves, D. J. (1975). Preschool Racial Attitude Measure II. Educational and Psychological Measurement, 35, 3-18.
Williams, J. E., & Morland, J. K. (1976). Race, color, and the young child. Chapel Hill: University of North Carolina Press.
Williams, L. E., & Bargh, J. A. (2008a). Experiencing physical warmth promotes interpersonal warmth. Science, 322, 606 - 607.
Williams, L. E., & Bargh, J. A. (2008b). Keeping one’s distance: The influence of spatial distance cues on affect and evaluation. Psychological Science, 19, 302 – 308.
Zacks, J., & Tversky, B. (1999). Bars and lines: A study of graphic communication. Memory & Cognition, 27, 1073–1079.
Zajonc, R. B., & Markus, H. (1985). Affect and cognition: The hard interface. In C. E. Izard, J. Kagan, & R. B. Zajonc (Eds.), Emotions, cognition, and behavior (pp. 73–102). New York: Cambridge University Press.
Zhong, C. B., & Liljenquist, K. A. (2006). Washing away your sins: Threatened morality and physical cleansing. Science, 313, 1451–
131
1452. Zhong, C. B., & Leonardelli, G. J. (2008). Cold and lonely: Does social
exclusion literally feel cold? Psychological Science, 19, 838-842. Zwaan, R. (2004). The immersed experiencer: Toward an embodied theory
of language comprehension. In B. H. Ross (Ed.), The Psychology of Learning and Motivation 44 (p. 35). San Diego, CA: Academic Press.
Zwaan, R. A. (2009). Mental simulation in language comprehension and social cognition. European Journal of Social Psychology, 37, 1142- 1150.
Zwaan, R., Stanfield, R., & Yaxley, R. (2002). Do language comprehenders routinely represent the shapes of objects? Psychological Science, 13, 168–171.
Zwaan, R. A., & Taylor, L. J. (2006). Seeing, acting, understanding: motor resonance in language comprehension. Journal of Experimental Psychology: General, 135, 1-11.
Zwaan, R. A., & Yaxley, R. H. (2003). Spatial iconicity affects semantic- relatedness judgments. Psychonomic Bulletin & Review, 10, 954-958.
133
Samenvatting
Veel van de belangrijkste concepten in ons leven zijn zeer abstract. In dit proefschrift beschrijf ik hoe mensen over abstracte concepten zoals moraliteit, tijd, en valentie nadenken. Specifiek richt ik me op de vraag of mensen over abstracte concepten nadenken door de betekenis van deze concepten middels metaforen in concrete dimensies te representeren. Het gebruik van metaforen om abstracte concepten uit te beelden is alomtegenwoordig in onze samenleving. Zo is de zwarte ridder in sprookjes meestal de slechterik, en is de ridder op het witte paard de held, waarbij de kleuren zwart en wit worden gebruikt als een metafoor voor goed en slecht. Vrouwe Justitia heeft een weegschaal in haar hand, waarbij balans een metafoor is voor gerechtigheid, en hoewel we de toekomst niet kunnen zien, zeggen we dat de toekomst voor ons ligt. Deze metaforische representaties van abstracte concepten in termen van concrete ervaringen als kleur, balans of ruimte worden niet alleen intentioneel gebruikt in menselijke communicatie, maar spelen ook een rol als mensen zelf over abstracte concepten nadenken. De vraag hoe mensen de betekenis van abstracte concepten gronden in concrete ervaringen is het onderwerp van dit proefschrift.
De afgelopen tien jaar zijn steeds meer onderzoekers geïnteresseerd geraakt in de vraag hoe de betekenis van abstracte concepten
134
gerepresenteerd wordt in onze hersenen. Deze toegenomen interesse is veroorzaakt door nieuwe theorieën over menselijke cognitie, waarin gesteld wordt dat conceptuele gedachten perceptueel gerepresenteerd worden in onze hersenen. De assumptie is dat de betekenis van abstracte concepten is gegrond in concrete ervaringen. In tegenstelling tot amodale theorieën over conceptuele gedachten, waarbij de grote gemene deler is dat aangenomen wordt dat conceptuele representaties bestaan uit een puur symbolische taal, wordt in modale theorieën over conceptuele gedachten gesuggereerd dat concepten opgeslagen zijn als perceptuele informatie. Onze gedachten bestaan niet uit artificiële symbolen (zoals een computertaal) maar uit modaliteitspecifieke informatie (visueel, auditief, haptisch, etc). Ons begrip van het woord ‘schoppen’ is gebaseerd op de activatie van de spieren die we gebruiken om zelf te schoppen.
De afgelopen jaren zijn in toenemende mate studies verricht die de rol van zintuiglijke informatie in het begrijpen van concrete woorden hebben onderzocht. De studies laten zien dat het nadenken over de kleur van een banaan hersengebieden in de visuele cortex activeert (Simmons, Ramjee, Beauchamp, McRae, Martin, & Barsalou, 2007), vergelijkbaar met de hersenactivatie die waar te nemen is als mensen daadwerkelijk naar een banaan zouden kijken. Een belangrijke vraag met betrekking tot de perceptuele representatie van concepten is hoe abstracte concepten gerepresenteerd worden. Welke visuele, haptische of andere zintuiglijke informatie zou geactiveerd kunnen worden om de betekenis van een abstract concept als ‘moraliteit’ te begrijpen? Er zijn verschillende theoretische raamwerken voorgesteld welke een verklaring proberen te geven voor de perceptueel representatie van abstracte concepten (Barsalou, 1999; Boroditsky, 2000; Glenberg & Robertson, 2000; Lakoff & Johnson, 1980, 1999). Er woedt echter nog een verhitte discussie over de vraag of een perceptuele benadering van cognitie abstracte conceptuele gedachtes kan verklaren (Arbib, 2008; Barsalou; 2008; Boroditsky & Prinz, 2008; Lakoff, 2008; Mahon & Caramazza, 2008). In dit proefschrift onderzoek ik de rol van zintuiglijke informatie binnen abstracte gedachteprocessen. Het eerste hoofdstuk geeft een overzicht van
135
de huidige theorievorming binnen gegronde benaderingen van cognitie, en vergelijkt de assumpties van deze theorieën met meer traditionele benaderingen van cognitie. Vervolgens beschrijf ik empirische studies die hebben onderzocht of abstracte concepten perceptueel kunnen worden gerepresenteerd. Mogelijke alternatieve verklaringen voor de resultaten van deze onderzoeken worden besproken, op basis van structurele en taalkundige eigenschappen van de gebruikte onderzoeksparadigma’s. Ook geef ik een korte interpretatie van de inzichten die deze studies ons geven over hoe mensen over abstracte concepten nadenken.
De drie empirische hoofdstukken van dit proefschrift richten zich op de vraag hoe moraliteit, tijd en valentie gegrond worden in perceptuele symmetrie, auditieve ruimte en helderheid, respectievelijk. Het doel van hoofdstuk 2 is om te laten zien dat moraliteit, een concept waarvan vaak wordt beweerd dat het geen perceptuele eigenschappen heeft waarmee de betekenis gegrond zou kunnen worden, in concrete een concrete dimensie gerepresenteerd wordt. Gebaseerd op de assumptie dat gelijkheid een belangrijke eigenschap is van moraliteit, en dat de gelijkheidscomponent van moraliteit metaforisch gerepresenteerd wordt als balans of symmetrie, voorspelde en vond ik dat morele concepten sterker geassocieerd worden met perceptuele symmetrie dan immorele concepten.
In hoofdstuk 3 wordt de breedte van de perceptuele gronding van abstracte concepten uitgebreid naar de auditieve modaliteit. Hoewel voorgaand onderzoek zich voornamelijk heeft gericht op de visuele modaliteit, zouden abstracte concepten theoretisch in elke modaliteit gegrond moeten kunnen worden. Omdat het menselijk gehoor heel goed is in het lokaliseren van geluid, voorspelde ik dat tijd niet alleen in visuele ruimte gestructureerd zou worden, maar ook in de auditieve ruimte. Tijd wordt visueel gerepresenteerd van links (het verleden) naar rechts (de toekomst), en eenzelfde representatie werd in auditieve ruimte verwacht. Deelnemers kregen woorden te horen over een koptelefoon, en werden gevraagd om aan te geven of woorden harder in het linker of het rechter oor werden aangeboden. Sommige woorden werden even hard in beide kanalen aangeboden. Wanneer deze woorden toekomstgerelateerd waren,
136
gaven deelnemers vaker aan dat het woord harder in het rechteroor klonk, in vergelijking met verleden gerelateerde woorden. Daarbij was er een systematische overlap tussen de visuele representatie van tijd (gemeten in een eerdere studie) en oordelen van deelnemers tijdens de auditieve taak, welke verdere steun geeft aan het idee dat abstracte concepten in multimodale dimensies gestructureerd worden.
In Hoofdstuk 4 worden twee procesverklaringen met betrekking tot de metaforische representatie van abstracte concepten met elkaar vergeleken. Deze set studies is een zeldzaam voorbeeld van een directe vergelijking van twee theoretische verklaringen voor de processen die achter het gronden van abstracte concepten liggen. Waar sommige onderzoekers (Meier, Robinson, & Clore, 2004; Sherman & Clore, 2009) hebben gesuggereerd dat visuele informatie (bijvoorbeeld de kleur wit) automatisch en verplicht leidt tot de activatie van een abstracte betekenis (bijvoorbeeld positiviteit), ondersteunen de resultaten van deze studies een meer actief proces. De metaforische representatie van positiviteit als de kleur wit hangt af van de activatie van de conceptuele tegenstelling tussen goed en slecht en de perceptuele tegenstelling van wit en zwart. Het is dus de gedeelde relatiestructuur tussen de concrete dimensie en de abstractie dimensie die cruciaal is voor de metafoor van valentie in helderheid.
Tezamen laten deze hoofdstukken zien dat perceptuele informatie geassocieerd wordt met abstracte concepten. Zelfs zeer abstracte concepten welke geen perceptuele eigenschappen lijken te bezitten, ontlenen bij nader inzien hun structuur toch aan concrete dimensies. Sommige abstracte concepten, zoals tijd, lijken te worden gestructureerd in meerdere modaliteiten, en er is sprake van een sterke overlap tussen de visuele en auditieve structurering van tijd. Gedeelde relationele structuren (zoals tegenoverliggende polen) in de abstracte en concrete dimensie lijken in sommige situaties een essentiële voorwaarde te zijn voor de activatie van metaforische representaties. Verder onderzoek zou zich kunnen richten op de omstandigheden waarbinnen de activatie van gedeelde structuren noodzakelijk is, en wanneer de relatie tussen
137
abstracte concepten en concrete dimensies onafhankelijk is van de context. De mogelijkheid om over abstracte concepten na te denken is een van de meest verfijnde eigenschappen van mensen, en ongeacht of deze cognities noodzakelijkerwijs perceptueel van aard zijn, is het waardevol om een beter inzicht te verwerven in hoe mensen tot zo’n opmerkelijke vaardigheid in staat zijn.
139
Curriculum Vitae
Daniël Lakens was born and raised in Rotterdam, the only true city in the Netherlands. He studied psychology at Leiden University and philosophy at the Erasmus University, and received his master’s degree in social psychology in 2002. After working in the elderly care for a year, he pursued a career in science, first working as a research assistant for Wilco van Dijk at the VU University Amsterdam, before embarking on a PhD project under the supervision of Gün Semin in 2005. His research first focused on psychological consequences of movement synchrony, but he soon developed in interest in the grounding of abstract concepts. Early 2008 he moved to Utrecht University, and continued to investigate how people use concrete experiences to think about abstract concepts, which has resulted in the current thesis. Since January 2010, Daniel works as a post-doctoral researcher at the Eindhoven University of Technology.
The “Kurt Lewin Institute Dissertation Series” started in 1997. Since 2008 the following dissertations have been published:
2008-1: Marijke van Putten: Dealing with missed opportunities. The causes and boundary conditions of inaction inertia
2008-2: Marjolein Maas: Experiential Social Justice Judgment Processes 2008-3: Lonneke de Meijer: Ethnicity effects in police officer selection: Applicant, assessor,
and selection-method factors 2008-4: Frederike Zwenk: Voice by Representation 2008-5: Margreet Reitsma: The Impact of Linguistically Biased Messages on Involved
Receivers 2008-6: Marcus Maringer: Feeling one thing, seeing another: Emotion comparison effects
in person judgments 2008-7: Hanneke Heinsman: The competency concept revealed: Its nature, relevance, and
practice 2008-8: Joris Lammers: Toward a more social social psychology of power 2008-9: Daniël Fockenberg: Between Good and Evil: Affective Priming in Dynamic
Context 2008-10: Arne van den Bos: Why we stereotype influences how we stereotype: self-
enhancement and comprehension effects on social perception 2008-11: Lidewij Niezink: Considering Others in Need: On Altruism, Empathy and
Perspective Taking 2008-12: Aad Oosterhof: Better together: Antecedents and consequences of perceived
expertise dissimilarity and perceived expertise complementarity in teams 2008-13: Femke ten Velden: Negotiation in dyads and groups: The effects of social and
epistemic motives 2008-14: Maike Wehrens: How did YOU do? Social comparison in secondary education 2008-15: Kyra Luijters: Making Diversity Bloom: Coping Effectively with Cultural
Differences at Work 2008-16: Ilona de Hooge: Moral emotions in decision making: Towards a better
understanding of shame and guilt 2008-17: Lindred L. Greer: Team Composition and Conflict: The Role of Individual
Differences
2008-18: Sezgin Cihangir: The Dark Side of Subtle Discrimination: How targets respond to different forms of discrimination
2008-19: Giel Dik: On the contagiousness of others’ goals: The role of perceiving effort 2008-20: Lotte van Dillen: Dealing with negative feelings: The role of working memory in
emotion regulation 2008-21: Marijn Poortvliet: Information exchange examined: An interpersonal account of
achievement goals 2008-22: Sjoerd Pennekamp: Dynamics of disadvantage: Uncovering the role of group-
based anger 2008-23: Chris Reinders Folmer: Cooperation and communication: Plastic goals and
social roles 2009-1: Marijke Leliveld: Ethics in Economic Decision-Making 2009-2: Monique Pollmann: Accuracy and Bias in Person Perception 2009-3: Krispijn Faddegon: Regulatory Focus in Group Contexts 2009-4: Lieven Brebels: Mirror, mirror on the wall… Procedural fairness as an evaluative
and regulatory looking-glass self 2009-5: Daphne Wiersema: Taking it personally: Self-esteem and the protection of self-
related attitudes 2009-6: Judith D.M. Grob: Dial E for Emotion: Context and Consequences of Emotion
Regulation 2009-7: Katherine Stroebe: Is this about me? Responding to subtle discrimination - beyond
an individual versus group perspective 2009-8: Menno Vos: Identity patterns in diverse work groups: Improving social integration
outcomes through relational identities 2009-9: Lennart Renkema: Facing Death Together: Understanding The Consequences of
Mortality Threats 2009-10: Michael Vliek: Group-based social comparison processes: An intragroup level of
analysis 2009-11: Karlijn Massar: Unconscious rivals: The automatic evaluation of rivals in
jealousy-evoking situations 2009-12: Bart Terwel: Origins and consequences of public trust: Towards an understanding
of public acceptance of carbon dioxide capture and storage 2009-13: Emma ter Mors: Dealing with information about complex issues: The role of
source perceptions 2009-14: Martijn Veltkamp: On the Instigation of Implicit Motivation: How Deprivation
and Positive Affect Cause Motivated Behavior 2009-15: Marret K. Noordewier: Consistency and the unexpected 2009-16: Sytske van der Velde: Imitation of Emotion: How meaning affects the link
between imitation and liking 2009-17: Jacomijn Hofstra: Attaching Cultures: The role of attachment styles in explaining
majority members’ acculturation attitudes 2009-18: Jacqueline Tanghe: Affect in Groups: Convergence, Conditions and Consequences 2009-19: Anne Marike Lokhorst: Using Commitment to Improve Environmental Quality 2009-20: Jonathan van ‘t Riet: Framing Health Communication Messages 2009-21: Suzanne Pietersma: Persuasive Health Communication: A Self-Perspective 2009-22: Remco Wijn: A functional perspective on the justice judgment process and its
consequences 2009-23: Niels van de Ven: The bright side of a deadly sin: The psychology of envy 2009-24: Anthon Klapwijk: The Power of Interpersonal Generosity 2010-1: Maarten Wubben: Social Functions of Emotions in Social Dilemmas 2010-2: Joyce Rupert: Diversity faultlines and team learning 2010-3: Daniel Lakens: Abstract Concepts in Grounded Cognition
k u r t l e
w i n i n s
t i t u u t
Dissertatiereeks Kurt Lewin Instituut 2010-3
Daniël Lakens
Abstract concepts in grounded cognition
Abstracte concepten in gegronde cognitie
(met een samenvatting in het Nederlands)
Proefschrift
ter verkrijging van de graad van doctor
aan de Universiteit Utrecht
op gezag van de rector magnificus, prof. dr. J. C. Stoof,
ingevolge het besluit van het college van promoties,
in het openbaar te verdedigen op
19 januari 2010 des middags te 2.30 uur
door
Daniël Lakens
geboren op 18 juli 1980 te Rotterdam
Promotor: Prof. dr. G. R. Semin
Contents
Introduction 5
Chapter 1 Grounded Cognition 11
Chapter 2 On the Grounding of Morality in
Perceptual Symmetry 55
Chapter 3 The Sound of Time 71
Chapter 4 But for the Bad, There Would not
be Good 84
Footnotes 112
Appendices 113
References 115
Summary in Dutch 133
Curriculum Vitae 139
5 Introduction
Morality, time, and valence are highly abstract concepts. Exactly how people are able to think and talk about abstract concepts is a question that has intrigued philosophers and scientists for centuries. In this thesis I focus on the question how people represent the meaning of abstract concepts, by investigating whether abstract concepts are understood through the use of metaphors. Metaphors allow people to think about abstract concepts in terms of concrete experiences. For example, the scales of Lady Justice are a metaphor for justice. People might not be able to see the future, but it can be represented as being ahead of us. Black knights in fairytales are usually evil, and the color of their armor is a metaphoric representation of the relation they have with the knights in shining armor. These and similar metaphors are not only used to intentionally communicate the meaning of abstract concepts, but these metaphoric representations also underlie abstract conceptual thought. How the meaning of abstract concepts is grounded in concrete experiences is the topic of this dissertation.
Broadly speaking, two views exist on how abstract concepts are represented in our mind. On the one hand, amodal views share the common theme that people think in a purely symbolic language, where concepts
6 are represented as artificial symbols, and computational operations are performed upon these symbols during conceptual processing (Dennett, 1969; Fodor, 1975; Pylyshyn, 1984; Jackendoff, 2002). These artificial symbols are inherently non-perceptual; none of the original sensory or motor information from any modality (e.g., vision, touch, sound, smell, etc) which created these symbols is present when the final amodal symbols are used to represent concepts. On the other hand, several views on the representation of concepts have emerged over the last decennium which all share the idea that conceptual processing is perceptual in nature (e.g., Barsalou, 1999; Gallese & Lakoff, 2005; Glenberg & Kaschak, 2002; Jeannerod, 2001; Pulvermüller, 1999; Zwaan, 2004). According to these grounded approaches to cognition, conceptual thought consists of representations built on concrete sensorimotor information. For example, understanding the word ‘kick’ is assumed to depend at least in part on the activation of the muscles used to perform a kicking motion.
As a result of the growing number of empirical studies that have investigated the role of sensorimotor activation in language comprehension, a consensus is emerging that sensorimotor areas in the brain are activated when people read words related to concrete concepts (Zwaan, 2009; Mahon & Caramazza, 2009; Vigliocco & Meteyard, 2008). For example, thinking about the color of a banana activates brain areas in the visual cortex (Simmons, Ramjee, Beauchamp, McRae, Martin, & Barsalou, 2007). Thinking about the color of a banana leads to remarkably similar patterns of brain activation as actually seeing a banana.
The extent to which abstract conceptual thought is grounded in perceptual information is an important theoretical question for grounded approaches to cognition due to the lack of perceptual characteristics of abstract concepts. Which visual, motor or other sensory information could in any way facilitate the understanding of abstract concepts such as ‘morality’? Several possible explanations have been put forward to accommodate the perceptual representation of abstract concepts in grounded cognition (Barsalou, 1999; Boroditsky, 2000; Glenberg & Robertson, 2000; Lakoff & Johnson, 1980, 1999), but whether abstract
7 conceptual processing can be accounted for by grounded approaches to cognition remains highly disputed (Arbib, 2008; Barsalou; 2008; Boroditsky & Prinz, 2008; Lakoff, 2008; Mahon & Caramazza, 2008).
Abstract concepts play important roles in human cognition as well as in social interaction. The empirical investigation of conceptual thought has focused primarily on concrete concepts (for reviews, see Fischer & Zwaan, 2008; Vigliocco & Meteyard, 2008), and the question how abstract concepts are grounded and especially which process underlies the perceptual representation of abstract concepts remains debated. Given that many abstract concepts constitute those things in life that people care greatly about (e.g., morality, compassion, power, knowledge) it is worthwhile to investigate how people think about these concepts. How people represent abstract concepts will both structure and constrain how these concepts are understood (Lakoff & Johnson, 1999), and as such, different representations might have profound consequences for peoples’ beliefs and decisions. For example, Lakoff (1996) describes the different metaphors used by liberals and conservatives to think about morality, and highlights the consequences of these different representations of morality in politics.
Overview
In the present dissertation, I examine the role of sensory information for abstract conceptual thought. In Chapter 1, the central assumptions in grounded approaches to cognition are introduced and contrasted with more traditional views on the representation of concepts expressed by cognitive researchers since the cognitive revolution in psychology. After detailing the different theoretical views on cognition, I present an overview of studies investigating whether and how abstract concepts are represented perceptually (including the studies reported in the three empirical chapters of this thesis). This review is concluded by a discussion of what these empirical findings reveal about how the meaning of abstract concepts is grounded in concrete experiences. Alternative explanations
8 for the research findings, based on structural and linguistic features of the paradigms, are discussed. Finally, I provide my own interpretation of what the findings on the relationship between perception and cognition reveal about abstract conceptual thought.
The empirical investigation of how abstract concepts are grounded in perceptual information is gaining popularity, but constitutes a relatively new field of interest. Many studies have aimed to demonstrate that perceptual information plays a role in conceptual processing, but there are only a few theories to explain these findings, and an integrative framework to account for the empirical results is currently not available. Researchers have noted that grounded theories of cognition need further development to account more precisely for the effects observed in the literature (Zwaan, 2009). In addition, the mechanisms which are assumed to underlie perceptual representations should be specified in more detail to be able to empirically test the predictions made by these theories (Boroditsky, 2000). Investigating the possible limitations of grounded approaches to cognition, such as whether abstract concepts can be represented perceptually, in which modalities people can represent abstract concepts, as well as the influence of the context in which perceptual information is presented, are all important steps to achieve a more detailed understanding of whether and how perceptual information structures abstract concepts. The goal of the present dissertation is to investigate these questions empirically, thereby contributing to the development of an integrative theoretical framework of the role of sensorimotor information in the grounding of abstract concepts.
The three empirical chapters of this thesis focus on how morality, time, and valence are grounded in perceptual symmetry, left-right auditory space, and brightness, respectively. The aim of Chapter 2 is to show that morality, one of the hallmark abstract concepts assumed to lack the perceptual characteristics to be grounded in sensorimotor information, can be represented perceptually. Based on the assumption that equality is an important feature of morality, and the equality component of morality is argued to be metaphorically represented in balance and symmetry, I
9 predicted and found that moral concepts show a stronger association with perceptually symmetrical stimuli compared to immoral concepts.
Chapter 3 increases the scope of perceptual grounding by providing support for the theoretical prediction that abstract concepts can be structured in the auditory modality. Earlier research into the grounding of abstract concepts has focused predominantly on the visual modality. Since the auditory modality shows a similar sensitivity to spatial information as the visual modality, the previously observed structuring of time in space should extend to the auditory domain. Time is represented from left to right visual space, with the past to the left and the future to the right, and the same structuring was expected to be observed in auditory space. Participants were asked to indicate whether words were presented louder in the left or the right ear while passively listening to time related words. Critical experimental trials were presented equally loud to both ears. Future words were more often judged to be louder in the right ear than past related words were, supporting the left to right representation of time in auditory space. Furthermore, the visual representation of time and the auditory representation of time were shown to be significantly related, providing additional support for the multimodal grounding of abstract concepts.
In Chapter 4, two process explanations concerning the metaphorical representation of abstract concepts were tested against each other. These studies provide a rare theoretical comparison of two processes that have been suggested to underlie the grounding of abstract concepts. Whereas previous researchers (Meier, Robinson, & Clore, 2004; Sherman & Clore, 2009) have argued that visual information (e.g., the color white) will automatically and obligatorily activate an abstract meaning (e.g., positivity), these studies support a more active metaphoric structuring process. The metaphoric representation of positivity as white depends on whether the conceptual opposition between good and bad and the perceptual opposition between white and black are activated. Thus, the shared relational structure between the concrete dimension and the abstract dimension is crucial for the metaphoric representation of
10
valence in brightness to emerge. Together, these chapters show that perceptual information
influences abstract conceptual processing, even for highly abstract concepts that lack perceptual characteristics. Some abstract concepts, such as time, seem to consist of multimodal representations, and judgments in the visual and the auditory domain are influenced by these metaphoric representations in a similar way. Nevertheless, shared relational structures (such as polar oppositions) between the abstract and concrete dimension can be an essential pre-requisite for the metaphoric representation of abstract concepts to become activated. Further research should explore the circumstances under which the activation of shared relational structures is necessary for metaphoric mappings to emerge, and these attentional processes should be taken into account in future studies that aim to investigate how abstract concepts are grounded.
Chapter 1
Grounded Cognition
13
Wenn wir der Körperwelt, welche unmittelbar nur in unserer Vorstellung dasteht, die größte uns bekannte Realität beilegen wollen; so geben wir ihr die Realität, welche für Jeden sein eigener Leib hat: denn der ist Jedem das Realste.
Schopenhauer, WWR 1 105.
When people think about highly abstract concepts, they draw upon concrete experiences to structure their thoughts. Due to the increasing popularity of grounded approaches to cognition, the question how concrete experiences shape abstract thoughts has received renewed attention over the last decennium. Grounded approaches to cognition argue that sensorimotor experiences play an important role when people think about concepts. Sensorimotor information stored in visual, auditory, and motor areas in the brain when people directly experience the world is activated when they think about the world at a later time. For example, when you think about what the earth looks like from outer space, visual areas become activated in a similar way as when you would see a blue and green ball covered with white dots of clouds. Even though people cannot directly see, hear or touch abstract concepts, and therefore have not directly experienced sensory information that is related to the meaning of abstract concepts such as morality, perceptual information is theorized to play an essential role in abstract conceptual thought (Barsalou, 1999; Glenberg & Kaschak, 2002; Gallese & Lakoff, 2005).
Such a grounded approach to cognition stands in stark contrast to the dominant views expressed by researchers since the cognitive revolution in psychology. Traditional theories of conceptual processing generally do not assume that conceptual thought is perceptual in nature, but argue for (or implicitly assume) a purely symbolic language of thought (Anderson, 1990; Dennett, 1969; Fodor, 1975; Pylyshyn, 1984; Jackendoff, 2002), which consists of artificial symbols. These symbolic (or amodal)
14
theories draw a sharp distinction between perception and cognition, and posit that perceptual information processing and conceptual thought are fundamentally different processes. Essentially, thoughts are independent from our experiences with the world. A symbolic representation of the color red is not the equivalent of the sensory experience of the color red (Pylyshyn, 1984).
Symbolic representations are theorized to exist independently from sensory experiences. Concrete sensory information is encoded into semantically interpretable syntactic structures that do not longer retain the perceptual information from which they originate. This encoding is not unlike what happens when I press the ‘d’ key on my keyboard. Although the letter that appears on the screen resembles the letter ‘d’, this letter is represented in a completely artificial and symbolic logical code by the computer. This becomes apparent when this file is opened in a simple text editor, and the letter ‘d’ is represented somewhere in this string of symbols: ¿òmËÊÄÍ?áÈ6·U©ÏñK©æ²8ø. Not only is the internal representation of the input completely symbolic, but the relation of the perceptual input (‘d’) and the internal representation is arbitrary. Any symbol can be used to represent the letter ‘d’. This becomes apparent when the font is changed from Times to Wingdings, where the input of the letter ‘d’ is mapped to the symbol ‘d’.
According to proponents of symbolic symbol systems, computations performed upon symbolic representations are the essence of mental processes (Fodor, 1980; Pylyshyn, 1984). An important advantage of amodal theories is that they allow different modal perceptual processes to be integrated at the symbolic level of representation. The term modal is used to indicate that these perceptual processes are domain specific: for example, visual, auditory, and haptic perceptual information is processed in different brain areas. Modal perceptual processes are fast, bottom-up and isolated from much of our knowledge about the world (Fodor, 1985). Higher cognitive processes, on the other hand, are assumed to be slow, neither bottom-up nor top-down (information can flow either way) and well-connected, so these processes can integrate sensory information
15
from different modal domains. One of the most important (and most difficult to answer)
challenges for symbolic symbol systems is how symbolic representations are connected to their referents, objects in the world. This challenge is known as the symbol grounding problem (Harnad, 1990; Searle, 1980). Where do amodal symbols get their meaning from, if their only sources of meaning are other meaningless amodal symbols? The meaning of symbolic representations has to be grounded in the perceptual experiences they refer to (e.g., the symbolic representation of the color red has to be, in some way, related to the perceptual experience of seeing the color red), but so far, no fully satisfactory solution to the symbol grounding problem has been provided. A second open question for symbolic symbol systems is a detailed process explanation that describes the mechanisms through which symbolic representations are created from perceptual information. How exactly is the sensory experience of the color red transduced, interpreted and encoded into an amodal symbol? Finally, the absence of empirical support for the presence of amodal symbols in the brain has been mentioned as a problematic issue for symbolic symbol systems (Barsalou, 1999). Together, these three open questions warrant the careful consideration of alternative approaches to cognition.
Harnad (1990) proposes a hybrid account of iconic representations of entities (which consist of analogue transformations of the sensory- motor experiences when encountering the entity) complemented by categorical representations (which consist of symbolic abstractions from all iconic representations people have). Importantly, both iconic and categorical representations are non-symbolic. The original sensory-motor information that was activated when a concrete entity was encountered is preserved due to analogue transformations. Although such a hybrid system, consisting of both perceptual memories and amodal symbols, provides a solution to the symbol grounding problem by proposing that amodal symbols activate perceptual memories during comprehension, one might wonder whether there is still anything left to do for amodal symbols in such a model.
16
Barsalou (1999, 2008a) proposes an exclusively perceptual symbol system where conceptual knowledge consists of modal representations. This grounded approach to the representation of concepts assumes that when concrete objects are perceived, the sensory-motor activation in the brain is captured in local association areas, which are subsequently integrated into multi-modal representations by higher-order cross- modal associations (or convergence zones, Damasio, 1989). Convergence zones can integrate information from different modalities. Contrary to symbolic representations, the information in convergence zones has retained the perceptual information of the concrete experiences the modal representations refer to, thus performing the integrative function of symbolic representations while retaining the connection to the perceptual input. The multimodal associations in convergence zones (or simulators) can be partially activated during language comprehension, leading to neural reenactments (or simulations) that activate the same sensory and motor areas in our brain as were active when we directly experienced its referent. For example, the representation of the color red has recently been shown to activate the same visual areas in the brain when we directly perceive the color red, as when we think about it (Simmons et al., 2007). Studies like these provide support for the a-priori predictions derived from grounded theories of cognition. If a computer would work as our brain works, the letter ‘d’ on a computer screen would be stored as a picture and a vinyl record, and the computer could play the sound recording on a record player or take the picture out of a drawer when it is needed.
Similar theoretical views have appeared in recent years (e.g., Gallese & Lakoff, 2005; Glenberg & Kaschak, 2002; Jeannerod, 2001; Pulvermüller, 1999; Zwaan, 2004) which all share the idea that conceptual processing is built upon and activates sensory-motor processes. According to these approaches to cognition, conceptual thought is fundamentally perceptual. More traditional approaches to cognition have acknowledged the possibility that people use imagery to facilitate conceptual processes (e.g., Fodor, 1975), but these theories share the basic assumption that
17
conceptual thought is not perceptual in nature. Perceptual imagery can help people to think about concepts, but perceptual information is not necessary. Therefore, the real debate between modal and amodal symbol systems is not about whether perception and cognition are related, but how they are related. As Fodor (1985) summarizes: “No one in his right mind doubts that perception interacts with cognition somewhere. What’s at issue […] is the locus of this interaction.” According to amodal symbol systems, perceptual information might play a marginal role in conceptual thought, but these images are generated based on symbolic representations (Kosslyn, 1980). According to modal symbol systems, conceptual thought consists of the reactivation of multimodal representations stored during direct experiences with its referents (Barsalou, 1999; Glenberg & Kaschak, 2002).
With the emerging popularity of grounded approaches to cognition, the question where and how perception and cognition interact has received an increasing amount of attention from a variety of scientific disciplines such as linguistics, cognitive science, and social psychology. The main goal of these investigations is to show that conceptual processing activates experientially consistent sensorimotor areas. For example, Hauk, Johnsrude and Pulvermüller (2004) show that reading the words ‘pick’, ‘kick’ and ‘lick’ do not only activate areas in the brain related to language processing, but that these words also activate motor areas in the brain related to hand movements, leg movements, and mouth movements, respectively. The empirical support for intimate relationship between sensorimotor information and the processing of concrete concepts is mounting (for a recent review, see Meteyard & Vigliocco, 2008).
Research based on a grounded approach to cognition has provided alternative ways to think about conceptual thought. First of all, our minds and bodies are not fundamentally distinct, as symbolic theories of cognition presuppose. The widespread bodily influences on social cognitive processes (for reviews, see Niedenthal, Barsalou, Winkielman, Krauth-Gruber, & Ric, 2005; Semin & Smith, 2008) is directly at odds with the claim that “an examination of bodies and their workings will
18
never bring us to the subject matter of the mind at all” (Dennett, 1969, p. 189). Instead, the insights from grounded cognition are in line with the theoretical view that bodies shape our minds (e.g., Lakoff & Johnson, 1980; Varela, Thompson, & Rosch, 1991). Second, these studies reveal that symbolic computations are influenced by perceptual information. Whereas in computational models of cognition “the symbolic codes themselves do not specify their intended interpretation (and the model’s behavior is not influenced by such an interpretation)” (Pylyshyn, 1984, p. 42), research has shown that perceptual information does influence cognitive processes such as language comprehension (for a review, see below).
Despite the fact that many a-priori predictions from grounded approaches to cognition have received empirical support, grounded cognition has not been without its criticisms. Some researchers have argued that the activation of sensorimotor information does occur during conceptual processing, but only to contribute to the ‘full’ representation of a concept (see Mahon & Caramazza, 2008). As such, sensorimotor information is theorized to complement amodal symbolic representations. The current empirical investigations into the relationship between perceptual information and conceptual processing do not provide a univocal answer to question whether conceptual processing necessarily relies on perceptual information. Nevertheless, the causal influence of manipulated bodily states and sensorimotor activation on language processing (e.g., Glenberg & Kaschak, 2002; Pulvermüller, Hauk, Nikulin, & Ilmoniemi, 2005) supports the idea that the activation of perceptual information has a more important function than simply enriching amodal representations (see Fischer & Zwaan, 2008; Zwaan, 2009). Concluding, neither the presence of modal symbols, nor the necessary role of sensorimotor activation in language comprehensions has received undisputed empirical support. Although these questions lie outside of the scope of the current thesis, the exact nature of mental representations and the possible limitations of grounded approaches to cognition remain important questions for further research.
19
A second source of criticism on grounded approaches to cognition has been the apparent impossibility to represent abstract concepts perceptually – which brings us to the topic of the current dissertation. Abstract concepts (e.g., virtue, morality, or time) have no perceptual characteristics, since they cannot be seen, heard, touched or directly perceived in any other sense. Amodal representations have therefore been argued to be the only logical answer to the question how abstract concepts are represented (Dove, 2009; Fodor, 1975; Mahon & Caramazza, 2008). Two possible approaches to accommodate perceptual representations of abstract concepts in grounded theories of cognition have been put forward in the literature. The first approach consists of metaphorically mapping abstract target concepts on perceptual source dimensions (Boroditsky, 2000; Lakoff & Johnson, 1980, 1999). The second approach suggests that abstract concepts can be represented in simulations of situations or actions (Barsalou, 2008b; Barsalou & Wiemer- Hastings, 2005; Glenberg & Robertson, 1999). Both these theoretical approaches propose that much of the conceptual content of abstract concepts is provided by either metaphoric representations or perceptual simulations, and is thus inherently perceptual. The possible objections against symbolic approaches to cognition such as the symbol grounding problem and the lack of empirical support for the existence of amodal symbols, together with the increasing amount of empirical support for the perceptual representation of concrete concepts, warrants the careful examination of possible ways in which abstract conceptual thought can be accommodated from a grounded cognition perspective.
Grounding Concrete Concepts
The literature regarding the grounding of concrete concepts is too broad to discuss in much detail in this dissertation, but in order to be able to appreciate the novel insights that grounded approaches to cognition have provided, I will describe several studies that have furthered our understanding of the nature of conceptual representations. In addition,
20
I will focus on several studies that have provided relevant insights for the question how abstract concepts are grounded, and on studies that have used paradigms that have subsequently been adapted to investigate the grounding of abstract concepts.
Recent well-designed studies provide a good example of the empirical support for the theoretical assumption that language comprehension automatically activates motor processes (Zwaan & Taylor, 2006; Taylor & Zwaan, 2008; for a review, see Fischer & Zwaan, 2008). In these studies, participants read sentences such as: ‘Because the music was too loud, he turned down the volume.’ Only one to three words of the complete sentence were presented on the screen at one time. Participants could read the entire sentence by rotating a knob, with each 5 degrees of rotation resulting in the presentation of the next segment. The reading-rotation direction either matched or mismatched the rotation direction implied by the sentence (e.g., turning down the volume consists of a counter-clockwise movement). When participants had to read the sentences describing counter-clockwise movement by moving the knob clockwise, they rotated to knob more slowly compared to participants who could read the sentence by rotating the knob counter-clockwise (Zwaan & Taylor, 2006), but only on the segment of the sentence that contained the verb. Apparently, reading a description of a movement incongruent to the movement people are currently performing slows down reading times. A subsequent study revealed this effect occurs as long as the action is within linguistic focus, and extends to the subsequent adjectives such as ‘opened slowly’ (Taylor & Zwaan, 2008).
There have been many more studies showing that semantic processing accesses the motor system. For example, participants were faster to categorize sentences toward the body (such as ‘open the drawer’) or away from the body (such as ‘close the drawer’) as sensible (vs. not sensible) when the button participants had to press required a congruent movement toward or away from the body (Glenberg & Kaschak, 2002). Pulvermüller (2005) showed that people were faster to identify leg-related words (compared to arm-related words) when then left-hemisphere leg
21
motor areas were stimulated using transcranial magnetic stimulation. When arm motor areas were activated, participants were faster to identify arm related words. These and similar studies all support the idea that motor areas in the brain are activated when people process words related to body movements, and words describing objects that can be touched and manipulated.
Beyond the motor domain, several studies provide support for the activation of modality specific brain areas when words or sentences related to visual or auditory domains are processed. Pecher, Zeelenberg and Barsalou (2003, 2004) investigated the assumption that perceptual properties of concrete objects are represented in modality specific areas in the brain. In a series of studies, they show that switching from a representation in one modality to a representation in a different modality takes time. Using a property verification task (“Is a lemon yellow, yes or no?”), they show that people are faster to make judgments about objects when the modality that is referred to in the question (e.g., the visual modality) is the same as in the previous question, than when the question refers to a different modality (e.g., auditory: “Do microwaves beep, yes or no?”). People were faster to indicate a lemon was yellow after judging an apple is green, compared to judging whether a lemon is yellow after judging that leaves are rustling. These findings are taken as support for the assumption that verifying the properties of concrete objects does not rely on amodal symbolic representations, but requires the activation of perceptual simulations. Although property verification tasks require explicit judgments about stimuli (which make them less conclusive about the spontaneous recruitment of modality specific representations) they support the assumption that different sensory domains are activated when answering questions about concrete concepts (see also Van Dantzig, Pecher, Zeelenberg, & Barsalou, 2008).
Processing concrete and abstract verbs has been shown to drive attention to specific spatial locations. In a norming study, Richardson, Spivey, Edelman, and Naples (2001) showed that people share an agreement about how verbs should be represented in space. For example,
22
‘give’ is represented in horizontal space, whereas ‘respect’ is represented in vertical space. Building on predictions from linguistics (Lakoff, 1987; Langacker, 1987; Talmy, 1983) and grounded cognition (Barsalou, 1999; Glenberg & Robertson, 2000), the spatial representations associated with these words were expected to be automatically activated during language processing in a set of studies by Richardson, Spivey, Barsalou, and McRae (2003). Processing a sentence containing a horizontally represented verb (e.g., give) was expected to activate visual representations in the left-right dimension, interfering with the recognition of a stimulus presented on the left or right side of the screen. Sentences containing vertically represented verbs (e.g., respect) were expected to interfere with the recognition of a subsequently presented stimulus up or down on the screen. The predicted findings were observed, and taken as support for the assumption that the same visual resources are used to represent the meaning of the sentence and to perceive the subsequent stimulus (for similar findings, see Estes, Verges, & Barsalou, 2008).
These results have subsequently been replicated for concrete verbs, but not for abstract verbs (Bergen, Lindsay, Matlock, & Narayanan, 2007). Furthermore, this last set of studies revealed that words associated with vertical or horizontal space were not enough to lead to interference effects, but that the results were caused by mental imagery. Importantly, building a mental simulation can also facilitate the processing of visually congruent information (Glenberg & Kaschak, 2002; Kaschak et al., 2005; Stanfield & Zwaan, 2001; Zwaan, Stanfield, & Yaxley, 2002). The critical difference between interference and facilitation seems to be the length of time between the presentation of the linguistic stimulus and the perceptual stimulus. If the visual brain areas are still busy representing the linguistic stimulus (e.g., between 50 to 200 milliseconds) then interference is more likely, but after the visual brain areas are no longer active, the previous visual representation of vertical or horizontal locations might facilitate recognition (for a discussion, see Bergen et al., 2007).
In addition to the effects of language processing driving visual attention, theoretically related studies have investigated the opposite
23
relationship between visual perception and language processing. When participants were asked to judge whether two words were semantically related, their relative positions on the screen influenced the speed with which these judgments were performed (Zwaan & Yaxley, 2003). More precisely, when the relative position of these words was congruent with their typical locations in the real words, judgments were faster compared to when the words were presented in relative spatial positions incongruent with their typical locations. When the words branch and root were presented on the screen, participants were faster to judge the words were related when branch was presented vertically above root, compared to when root was presented above branch. Similar results have been shown when only one word (e.g., eagle) was being presented either on the top or the bottom of the screen (Šetić & Domijan, 2007). In these studies, words are assumed to activate perceptual representations, with attention being focused on the respective regions. The relative spatial position of the words is either congruent or incongruent with the presentation of the stimulus words, thus resulting in a spatial Stroop effect (Lu & Proctor, 1995). In further support of this reasoning, similar effects are not observed when pictorial (compared to verbal) stimuli of objects typically seen up or down in the world are processed, since these pictorial stimuli do not have to be visualized to be represented (Verges & Duffy, 2009). A similar reasoning and paradigm has been used extensively in abstract domains (e.g., Meier & Robinson, 2004; Schubert, 2005).
Taken together, the many empirical demonstrations of sensorimotor involvement in thoughts about concrete concepts reveal that conceptual processing and sensorimotor information are intimately related. Important questions still remain, such as the question whether motor activation is necessary for language understanding, or whether the activation of sensorimotor systems during semantic processing is optional (Fischer & Zwaan, 2008; Meteyard & Vilgliocco, 2008; Zwaan, 2009). In addition, people can sometimes rely on purely linguistic information to understand the meaning of words, without drawing on sensorimotor brain areas (see for example Andrews, Vigliocco, & Vinson,
24
2009; Boroditsky & Prinz, 2008; Simmons, Hamann, Harenski, Hu, & Barsalou, 2008). Nevertheless, the grounded cognition approach seems to be a fruitful theoretical framework to investigate conceptual processing – both for concrete as abstract concepts.
Grounding Abstract Concepts
Highly abstract concepts such as morality, truth, and beauty cannot be seen or heard. Because such abstract concepts lack perceptual characteristics, some researchers have argued that “perceptual representations seem ill-suited for representing abstract concepts” (Dove, 2009). The much debated question of whether and how abstract concepts are represented perceptually remains a challenge for grounded approaches to cognition (Arbib, 2008; Barsalou; 2008; Boroditsky & Prinz, 2008; Dove, 2009; Lakoff, 2008; Mahon & Caramazza, 2008). If grounded theories of cognition fail to account for the perceptual representation of abstract concepts, grounded cognition would be a rather limited framework for conceptual thought, applying exclusively to concrete concepts (Mahon & Caramazza, 2008). Therefore, providing a theoretical account of how abstract concepts are grounded, as well as empirical support for the perceptual representation of abstract concepts are both essential for a more mature grounded cognition framework.
Several theoretical proposals have been made to accommodate abstract concepts in grounded approaches to cognition (Barsalou, 1999, 2008; Barsalou & Wiemer-Hastings, 2005; Boroditsky, 2000; Glenberg & Robertson, 1999; Gallese & Lakoff, 2005; Lakoff & Johnson, 1980, 1999; Meier & Robinson, 2004; Richardson, et al., 2003). Although the perceptual representation of abstract concepts has received less empirical attention than the perceptual representation of concrete concepts, the predictions drawn from these theories have been tested in a decent number of studies. These theoretical frameworks are often complementary (Glenberg, Sato, Cattaneo, Riggio et al., 2008), and focus on the simulation of perceptual information, relating abstract events to
25
actions, and metaphoric representation. According to the simulation account (Barsalou, 1999, 2008a;
Richardson et al., 2003) people create images of situations, objects and introspections to perceptually simulate specific instances of abstract concepts. Through selective attention, an unlimited number of simulators can develop for objects (e.g., “car”), properties (e.g., “red”), relations (e.g., “above”), actions (e.g., “driving”) or introspective states (e.g., “fear”). These simulators makes the implementation of symbolic operations possible (for an overview, see Barsalou, 2008b). Even abstract concepts such as ‘truth’, which seems to lack any perceptual characteristics to build a simulation of, can be perceptually simulated in such a way (Barsalou, 1999). To verify the truth of a proposition (e.g., “the balloon is above the cloud”) people create and image of a balloon above a cloud and subsequently compare whether this image corresponds to their experience. After repeated experiences of this comparison process, a simulator develops which grounds the concept of truth.
Another proposal aimed at explaining how abstract concepts are grounded focuses on how certain abstract events can be grounded in movements. The indexical hypothesis (Glenberg & Robertson, 1999) proposes that sentences are understood by creating a simulation of the actions that underlie them. It seems to matter little whether these actions are concrete (e.g., ‘giving a book’) or abstract (e.g., ‘giving information’). Glenberg and Kaschak (2002) asked people to judge the sensibility of sentences describing the transfer of concrete objects such as “Andy delivered the pizza to you/You delivered the pizza to Andy” and abstract information transfer, such as “Liz told you the story/You told Liz the story”. Participants responded by moving their hand from a middle button to either a button farther away or closer by. Judgments were faster when the action implied by the sentence matched the action required to make the response, and this was true for both the concrete and the abstract transfer sentences. A similar study showed that moving 600 beans from one container to the other (either away or towards the body) influenced sentence comprehension in a subsequent task of sentences describing
26
concrete and abstract transfer away from or towards the body (Glenberg, Sato, & Cattaneo, 2008).
A final approach to grounding abstract concepts draws upon the seminal work by Lakoff and Johnson (1980, 1999), who have proposed that abstract concepts are grounded in concrete experiences through metaphoric representations. People use metaphors abundantly when they talk about abstract concepts. The Conceptual Metaphor Theory (CMT) introduced by Lakoff and Johnson (1980, 1999) holds that metaphors are not merely conventional ways to talk about abstract concepts, but that abstract conceptual thought is inherently metaphorical (cf. Richards, 1936). According to CMT, people use concrete experiences (or sources) to think about more abstract concepts (or targets). When concrete experiences in the source domain repeatedly co-occur with more abstract experiences in the target domain, the two brain areas that are activated together will over time form neural mappings (Lakoff, 2008), which constitute so-called ‘primary metaphors’ (Grady, 1997).
An example of such a primary metaphor is ‘happy is up’. Lakoff and Johnson (1980) argue that positive states are usually accompanied by erect postures, whereas negative states coincide with a drooping posture, thereby experientially grounded the relationship between the concrete domain of vertical space and the abstract domain of valence. The assumption of CMT that abstract thought is built on concrete experiences is relatively undisputed (for similar reasoning, see Clark, 1973; James, 1890; Piaget, 1954; Richards, 1936; Whitney, 1875). CMT has inspired many researchers to investigate whether abstract concepts are represented perceptually. Although CMT provides compelling theoretical arguments for a strong relation between concrete experiences and abstract concepts, even the updated neural theory of metaphor (Feldman, 2006; Lakoff, 2008) fails to clearly specify how basic image schemas, personal experiences, contextual variations, and overall knowledge collectively determine which neural mappings are activated in which situation. This makes it difficult to predict how abstract concepts will be grounded.
Conceptual Metaphor Theory has previously been criticized for
27
lacking a more detailed model of how the relationship between concrete concepts and abstract concepts is established (Murphy, 1996; Boroditsky, 2000; McGlone, 2007). The recent neural theory of metaphor put forward by Lakoff (2008) is the most explicit account of the processes underlying CMT so far, and proposes that neural mappings between concrete experiences and abstract concepts emerge through peoples’ experiences, and these neural mappings are combined into larger neural circuits that allow us to ground metaphoric language. However, two criticisms that have been put forward previously still apply (e.g., Murphy, 1996), namely the strong focus on linguistic evidence (see also Casasanto, in press; Crawford, 2009) and the lack of detail of the theory, which makes it difficult for psychological researchers to derive clear testable predictions (Boroditsky, 2000). Two interpretations of CMT have been put forward that allow the underlying assumptions to be tested empirically. Although both views are derived from CMT, they highlight different aspects of the processes assumed to underlie the metaphoric representation of abstract concepts. Below, I will introduce these two views in greater detail. Chapter 4 provides a set of studies that aim to test the assumptions of these different views on the metaphorical representation of abstract concepts against each other.
Metaphoric Structuring View
The Metaphoric Structuring View was developed by Boroditsky (2000) to provide an empirically testable theory derived from the work on CMT by Lakoff and Johnson (1999). The main assumption of the Metaphoric Structuring View is that metaphors ‘provide relational structure to those domains where the structure may not be obvious from world experience’ (Boroditsky, 2008, p. 3). Abstract domains are argued to lack a clear relational structure. For example, whereas moments in time seem to appear, disappear and never return, time does not contain concrete information about where moments in time come from and where they go to. People can move through time themselves, or time can approach
28
them. Time can flow from the left to the right, or from our front to back. Metaphors allow us to structure, among others, the abstract concept of time by mapping time onto concrete dimensions such as space. According to the Metaphoric Structuring View, metaphors in language such as ‘looking forward to next week’ influence how we structure time in space.
Boroditsky (2000) suggests that metaphors have shaped the way we think about time, but after frequent use, metaphors can lead to independent representations that no longer rely on spatial schemas. Which specific metaphoric representation of time is used depends upon the on- line mapping of spatial schemas onto the domain of time. Priming people with an ego-moving spatial scenario (where objects move through space) facilitates responses for time questions framed in an ego-moving schema (compared to a time-moving scenario, where time approaches objects). Similar flexibility in the spatial representation of time was shown in a cross-cultural study comparing Mandarin speakers with English speakers (Boroditsky, 2001). In Manderin there is a vertical metaphor for time (comparable to the uncommon English metaphor of a meeting ‘coming up’ and ‘handing down’ knowledge from generation to generation, see also Lindner, 1983), and thus Chinese speakers tend to represent time as moving from the top to the bottom. Importantly, English speakers trained in the vertical representation of time showed a similar influence of spatial primes on time questions as Manderin speakers, revealing that metaphoric representations can be learned.
The Metaphoric Structuring View highlights the importance of the spatial schemas used to think about time, and shows that metaphoric representations of abstract concepts are highly contextualized. Different representations of time can be primed (Boroditsky, 2000), trained (Boroditsky, 2001) or changed depending on peoples’ mindset (Boroditsky & Ramscar, 2002), and subsequently influence how time is structured in space. This suggests that different representations of abstract concepts exist and can be used to structure the abstract domain. Such flexibility in the spatial representation of time is difficult to account for by more automatic views on metaphoric representation such as the neural theory
29
of metaphor, which largely rely on stable neural connections between concrete experiences and abstract concepts (Feldman, 2006; Lakoff, 2008).
Further support for the flexibility of spatial representations comes from a recent set of studies revealing that whereas the front-back representation of time is the default representation, introducing left and right response keys activates the left-right representation of time (Torralbo, Santiago, & Lupiáñez, 2006). These authors argue that attention plays an important role in the flexible on-line representation of abstract concepts such as time. After abstract concepts and concrete dimensions have repeatedly been mapped onto each other, these mappings are stored in long-term memory (cf. Boroditsky, 2000). These mappings are subsequently used when people try to construct the most parsimonious mental model (cf. Johnson-Laird, 1983) to support conceptual processing.
Stroop-like Interference Effects
A more recent translation of Conceptual Metaphor Theory to an empirically testable hypothesis was initiated by Meier, Robinson and Clore (2004). They reason that if the neural networks that are active when people represent abstract concepts are intimately bound with the neural networks that are active when people represent sensory experiences (Lakoff, 2008; Lakoff & Johnson, 1999), there should be an obligatory and automatic relationship between perceptual information and abstract concepts. Lakoff (1993) appears to endorse this view when he suggests that the system of conceptual metaphors ‘‘is used constantly and automatically, with neither effort nor awareness’’ (pp. 227–228).
Meier and colleagues (2004) investigated whether the abstract concepts of positive and negative valence are metaphorically represented in the concrete experiences of light and dark. Their results reveal that positive words presented in a white font and negative words presented in a black font word were evaluated faster and more accurately compared to positive words presented in a black font and negative words presented in a white font. In other words, metaphor congruent valence-brightness
30
combinations were processed more efficiently than metaphor incongruent valence-brightness combinations.
Metaphor congruency effects are argued to be the result of a Stroop-like interference effect. Meier et al., (2004, p. 86) argue that: “a negative word presented in white would give rise to two response tendencies, one to respond ‘negative’ (on the basis of stimulus valence) and one to respond ‘positive’ (on the basis of stimulus color).” A similar reasoning is provided by Schubert (2005), who investigated the vertical representation of power. He asked participants to categorize words referring to powerful and powerless groups on power, and found that words referring to powerful groups presented on the top of the screen were categorized faster compared to when these words were presented on the bottom of the screen. According to Schubert (2005, p. 18), “The response that a presented group is powerful or powerless has a certain threshold, and evidence is accumulated until that threshold is reached. The evidence comes from multiple dimensions, and vertical position of the group label and vertical motor images are among these dimensions.”
According to this Stroop-like interference explanation for metaphor congruency effects, which I shall refer to as the one-to-one grounding explanation, the abstract concept of positivity is grounded in the sensory experience of seeing a light stimulus, and negativity is grounded in the sensory experience of darkness. As such, seeing the color white is assumed to automatically activate its associated valence (e.g., positivity), which in turn can facilitate or interfere with the categorization of the conceptual meaning of the word (see also Schubert, 2005; Meier & Robinson, 2004, Sherman & Clore, 2009). These brightness-valence associations are argued to be unavoidable at the stage of word encoding, and point to “the obligatory nature of affective inferences based on stimulus brightness” (Meier et al., 2004, p. 85; see also Meier & Robinson, 2004).
The one-to-one grounding explanation does not yet provide a detailed explanation of when and how perceptual information influences abstract conceptual processing, and cannot account for all observed
31
empirical results. For example, Schubert (2005) shows that words such as ‘dictator’ or ‘tyrant’, which are powerful and negative, are processed faster on the top of the screen when participants have to categorize these word on power, but are processed faster on the bottom of the screen when these words are categorized on the basis of their valence. Metaphor congruency effects seem to depend upon the current categorization goals, speaking against a purely automatic activation of word meaning. Relating metaphor congruency effects to Stroop-like interference effects is fitting in this respect, because the assumption that Stroop-like effects are completely automatic does not seem to be the most parsimonious explanation for the effects observed in the literature (see MacLeod, 1991). For example, adding additional irrelevant words to the task reduces Stroop interference effects (Kahneman & Chajczyk, 1983), which would not be expected if the process is completely automatic.
Although the processes underlying the metaphoric representation of abstract concepts remain debated (but see Chapter 4 for a set of studies which experimentally tests the one-to-one grounding approach against the metaphoric structuring view), there is a growing amount of studies that have investigated the relationship between perceptual information and abstract conceptual processing. Below, I will review these studies, whereby I will focus primarily on the three abstract concepts under investigation in the three chapters of this thesis, namely morality, time, and valence.
Morality
Morality is one of the hallmark abstract concepts for which it is argued that there is “no sensory or motor information that could correspond in any reliable or direct way to their meaning” (Mahon & Caramazza, 2008). The assumed lack of perceptual characteristics of abstract concepts is one of the most prevalent criticisms on perceptual theories of cognition. After all, how can people perceptually represent abstract concepts which they have never seen or heard?
32
Some researchers have proposed that moral concepts cannot be grounded perceptually (Andrews, et al., 2009; Dove, 2009; Mahon & Caramazza, 2008). Some of these researchers have proposed that moral concepts can only be grounded in the emotional responses they cause in us (Boroditsky & Prinz, 2008; Prinz, 2005). Indeed, many studies have revealed that immorality is strongly associated with feelings of disgust. For example, the olfactory experience of bad smell influences moral judgments (Schnall, Haidt, Clore, & Jordan, 2008). Other studies have revealed that morality and cleanliness are strongly related. Thinking about an unethical deed makes cleaning related words more accessible and reading an unethical story increases the desire for cleaning related products (Zhong & Liljenquist, 2006). Activating the concept of cleanliness before reading moral dilemmas, as well as washing ones hands after seeing a disgusting movie clip, reduces the severity of moral judgments (Schnall, Benton, & Harvey, 2008). Taken together, there is abundant support for a strong relationship between experienced disgust and moral judgments.
Intuitive approaches to morality (Haidt, 2001; 2008) have highlighted the importance of perceptual representations of morality together with affective responses to moral situations. Building on the work by Lakoff & Johnson (1999), Haidt (2001, p. 285) argues that moral concepts are “built up largely by metaphorical extensions from physical experience.” Several other researchers have argued in favor of the perceptual representation of moral concepts (Barsalou, 2008b; Gibbs, 1996; Haidt, 2001; Johnson, 1993; Meier, Robinson, & Wygant, 2007; Lakoff & Johnson, 1980, 1999; Sherman & Clore, 2009). Both the simulation approach (Barsalou, 1999; Barsalou & Wiemer-Hastings, 2005) as the metaphoric representation approach (Lakoff & Johnson, 1999) provide possible suggestions how moral concepts might be represented perceptually. According to the metaphoric representation approach, there are multiple concrete sources which are commonly used to represent moral concepts metaphorically (for an overview, see Lakoff & Johnson, 1999, chapter 14). Morality is seen as pure, straight, upright, balanced, and clean, for example, whereas immorality is seen as contaminated,
33
crooked, low, unbalanced and dirty. Two sets of studies have provided some initial empirical support for these assumptions.
Meier, Sellbom, and Wygant (2007) investigated whether morality is represented on a vertical dimension. They presented both moral and immoral words on the top or the bottom of the screen (cf. Meier and Robinson, 2004; Schubert, 2005) and found that responses were faster in the metaphor congruent conditions (moral-up and immoral-down) compared to the metaphor incongruent conditions (moral-down and immoral up). The results are not exactly in line with the predictions, in that immoral words on the bottom of the screen were responded to slower than all other conditions, for example. Furthermore, it is unclear to what extent these effects differ from the vertical representation of valence (Meier & Robinson, 2004; Crawford, Margolies, Drake, & Murphy, 2006).
Sherman and Clore (2009) adapted the Stroop paradigm, and found that participants were faster to indicate the font color of a word was white when the word had a moral (compared to immoral) meaning. Their first study is a re-analysis of Meier and colleagues’ (2004) Study 4, which originally was designed to test the a-priori prediction that brightness categorizations would not be influenced by the valence of the stimuli. Meier, Robinson, and Clore (2004, p. 83) assumed that the interaction between valence and brightness would be asymmetric, in that “color would interact with valence judgments (Studies 1, 2, and 3), whereas valence would not interact with color judgments (Study 4)”. Such a prediction is in line with both theoretical views on the metaphoric structuring of abstract concepts based on concrete concepts (Lakoff & Johnson, 1980) and empirical findings revealing an asymmetry between the concrete source and abstract target in metaphoric representations (Boot & Pecher, in press; Boroditsky, 2000; Casasanto & Boroditsky, 2008). Nevertheless, in the studies by Sherman and Clore (2009), the abstract meaning of the words did influence categorizations of the color in which the words were presented. The authors offer no theoretical explanation to reconcile the difference between their findings and previous empirical observations. In addition, it is not specified in which way these results go beyond
34
the previously reported more general valence-brightness relationship (Meier et al., 2004). These studies show the necessity of investigating the relationship between perceptual information and abstract conceptual processing from an integrative theoretical framework. The question when and why metaphoric representations are asymmetric is surely an interesting one, and further insights into this question will improve our theoretical understanding of the processes underlying metaphoric representations.
In Chapter 2, I take a new approach to the perceptual representation of morality by investigating how the equality component of morality is grounded in the ‘balance of justice’ metaphor. Equality is a central feature of morality in philosophy (e.g., Rawls, 1971), anthropology (e.g., Fiske, 1992), folk-theories (e.g., ‘an eye for an eye’) and classical justice theories in psychology such as equity theory (Adams, 1965). The equality component of morality is theorized to be grounded in the balance image schema (Gibbs, 2005; Johnson, 1987, Lakoff, 1987) and conceptualized metaphorically as balance or symmetry (Lakoff, 2005). Many of our experiences with fairness or justice are directly related to balanced and symmetrical distributions, such as when parents cut a birthday cake into equal pieces. Given this strong experiential association between equal divisions and justice, moral concepts should be associated with perceptual symmetry. If peoples’ understanding of morality and justice is experientially grounded in symmetrical distributions, then thinking about morality should activate non-verbal associations with perceptual symmetry.
The studies in Chapter 2 investigate whether moral words are associated with symmetrical stimuli, and immoral words are associated with asymmetric stimuli. Participants were asked to indicate which of two Chinese ideographs presented on the screen correctly translated the Dutch stimulus word. After this translation task, participants rated all ideographs on symmetry. The results revealed that participants had chosen more symmetric ideographs over more asymmetric ideographs when translating moral words. The reverse was true for immoral words.
35
A second study extended these results to judgments where the moral or immoral meaning of the word was unrelated to the judgments participants had to perform. Participants were asked to indicate how many of the letters of previously presented moral and immoral words had been blue. In line with the assumption that morality is associated with symmetry, participants displayed biased recall of the number of blue and green letters of previously presented words, indicating that the number of blue letters had been more equally distributed for moral words compared to immoral words.
Time
The perceptual representation of time has been investigated more thoroughly than any other abstract concept (Boroditsky, 2000; Boroditsky, 2001; Boroditsky & Ramscar, 2002; Casasanto & Boroditsky, 2008; Santiago, Lupiáñez, Pérez, & Funes, 2007; Torralbo et al., 2006; Tversky, et al., 1991; Vallesi, Binns, & Shallice, 2008; Weger & Pratt, 2008; Zacks & Tversky, 1999). These studies show that time can be represented in the back- front, up-down and left-right spatial dimensions. The overlap between spatial and temporal representations is also present at a neural level, with common brain areas being active during the processing of spatial and temporal information (Basso, Nichelli, Frassinetti, & Di Pellegrino, 1996; Bjoertomt, Cowey, & Walsh, 2002; Rao, Mayer & Harrington, 200People often use space to talk about time (e.g. ‘The worst is behind us’ and ‘We are approaching the holidays’). Boroditsky (2000) reasoned that if people use space to talk about time, they might also use space to think about time. If this reasoning holds, then influencing the way people think about space should influence how they think about time. The literature differentiates between two reference frames to think about time in terms of space. One the one hand, people can move through time (the ego- moving frame of reference), on the other hand, time can move toward us (the time-moving frame of reference). Participants were presented with an ambiguous temporal statement (e.g. `Next Wednesday’s meeting has
36
been moved forward two days’). In English, this statement is just as likely to mean that the meeting has been moved to Monday, as that it has been moved to Friday. From an ego-moving frame of reference, where people move through time, moving forward means the meeting will now be on Friday. From an object-moving perspective, where things move through time towards us, the meeting should now be on Monday (McGlone & Harding, 1998).
Activating these respective temporal frameworks by presenting pictures of objects moving through space toward the individual (time moving perspective), or the individual moving through space (ego- moving perspective) makes people respond to this temporal statement in a metaphor consistent manner (Boroditsky, 2000). A subsequent set of studies (Boroditsky & Ramscar, 2002) found that the answer people gave to the previously mentioned ambiguous question depended on whether people had moved through space themselves (e.g., when flying in an airplane, moving through the lunch line, or on a train), with the more they moved themselves, the more they answered the question using an ego-moving frame of reference.
Spatial primes influence thoughts about time differently for Manderin speakers than for English speakers. In Chinese, time can be represented as moving downward through space, in addition to the left- to-right representation of time in space prevalent in Western culture. Due to this cultural difference, priming participants with either horizontal spatial differences (e.g., a picture where a black worm is positioned to the left of a white worm) or vertical spatial differences (e.g., a picture where a black ball is placed above a white ball) influences temporal judgments (e.g., ‘March comes earlier than May’) in a different manner for English speakers than for Mandarin speakers (Boroditsky, 2001).
Time can also be represented from left (past) to right (future). Santiago, Lupiáñez, Pérez, and Funes (2007) manipulated both the position (left vs. right) on the screen of past and future related words, as the key assignment with which these words had to be categorized. Participants had to press the left key for past words and the right key for
37
future words, or they were instructed to use the reversed key assignment. Both spatial position and key assignment influenced reaction times, but these effects were notably stronger for the latter manipulation. The fact that participants were faster to correctly categorize future related words by pressing the right key, and past related words by pressing the left key, than the reverse, is a strong indication of the representation of the past and future from the left to the right. The importance of task instructions in categorization tasks such as these was further exemplified in two studies where participants responded either verbally (categorizing past and future related words through vocal responses) or manually (categorizing past and future related words by pressing a left or right response key). When answering verbally, the back (past) to front (future) representation of time was most active, but when answering manually, the left-right representation of time was most salient (Torralbo, Santiago, & Lupiáñez, 2006).
The left-to-right representation of time has been investigated more closely in two sets of studies (Vallesi, et al., 2008; Weger & Pratt, 2008), in which the left-to-right representation of time is approached from a spatial stimulus-response compatibility view (for reviews, see Hommel & Prinz, 1997; Umiltà & Nicoletti, 1990). Weger and Pratt (2008) replicate earlier findings supporting the left-to-right representation of time, while using less explicitly time related stimuli (namely names of actors such as Charlie Chaplin or Brad Pitt). In addition, they show that past related words presented in the center of the screen facilitate recognition of target on the left of the screen, whereas future related words facilitate the recognition of targets on the right side of the screen. This effect was stronger when participants had to respond to the left target with their left hand, and to the right target with their right hand (Study 2A), than when left or right targets had to be responded to by pressing the space bar, and filler trials had to be ignored (Study 2B). They conclude from this comparison of a bimanual categorization task versus a go-no-go paradigm, that the results are “primarily - although not necessarily exclusively - due to a facilitation of response codes, rather than to an impact on stimulus processing”
38
(Weger & Pratt, p. 429). Vallesi and colleagues (2008) asked participants to indicate how
long a fixation cross appeared on the screen (which was presented for one vs. three seconds). Inspired by the spatial representation of numbers, with small numbers being represented on the left and larger numbers on the right, which is also known as the SNARC effect (Dehaene, Bossini, & Giraux, 1993), these researchers investigated whether short time periods (one second) would shift attention to left-hand responses and longer time periods (three seconds) would shift attention to right-hand responses. Indeed, left-hand responses were slower when participants had to indicate the fixation cross had been presented for three seconds (vs. one second), whereas right-hand responses were slower when indicating the fixation cross had been presented for one second (vs. three seconds). The authors conclude that this interference emerges somewhere after response selection and during response preparation. Furthermore, the authors argue that the spatial representation of time is not easily activated automatically, but requires the awareness of the passage of time (for example, by making time task-relevant).
The relationship between time and space has been shown to be asymmetric, in that spatial primes influence judgments about time, but temporal primes do not influence judgments about space (Boroditsky, 2000, Study 2 and 3). Spatial primes which activate the ego-moving or object-moving frames of reference influence thoughts about time, but temporal primes did not influence thoughts about space. This asymmetry was investigated in more detail in a paradigm where participants were presented with growing lines on a computer screen, and were asked to judge the length of these lines or the duration the lines were presented. The length of the lines influenced judgments about the time the lines had been presented on the screen, but the time the stimuli were presented on the screen, did not influence judgments about the length of the lines (Casasanto & Boroditsky, 2008). Participants were unable to ignore irrelevant spatial information when they performed judgments about duration, but not the other way around.
39
Research on the metaphoric representation of abstract concepts has almost exclusively focused on the visual modality, and when people talk about the perceptual grounding of abstract concepts, they implicitly seem to equate this to visual grounding (e.g., Crawford, 2009). However, there is no reason to assume that the spatial structuring of abstract concepts is limited to the visual modality. In principle, any modality that is sensitive to spatial information should be able to structure abstract concepts. Chapter 3 provides empirical support for this assumption by revealing how time is metaphorically structured in the left-to-right auditory dimension. Participants listened to auditory stimulus consisting of past and future related words over headphones. Their task was simply to indicate in which ear they judged the stimulus to be presented loudest. In most of the trials, there was an objective difference in loudness, but in the critical experimental trials, the words were presented equally loud over both channels of the headphone.
As predicted, participants chose the right channel more often for future words than for past related words, when the auditory stimuli were presented equally loud to both ears. Furthermore, the average number of right ear judgments for each stimulus overlapped significantly with the spatial position of the items on a horizontal line as measured during a visual pilot study. This is strong support for the multi-modal representation of time in space. Note that the temporal meaning of words influenced loudness judgments even when their meaning was unrelated to the task participants had to perform, speaking against the view that the temporal meaning of the words should be task relevant for the spatial representation of time to be activated (Vallesi et al., 2008).
Valence
The metaphorical representation of positivity and negativity has revealed that the same abstract concept can be structured in many different concrete dimensions (for reviews, see Crawford, 2009; Meier & Robinson, 2005). The first demonstration of congruency effects in the abstract domain of
40
valence was provided by Meier, Robinson and Clore (2004) who asked participants to evaluate positive and negative words, while presenting these words in either a black or a white font. Although the color of the words was irrelevant to perform the task correctly, Meier and colleagues reasoned that based on the widespread use of black and white as a metaphor for evil and good the irrelevant perceptual characteristics of the stimuli would influence the speed and accuracy with which participants could evaluate the words. Their results confirmed this hypothesis: participants were faster and/or more accurate to categorize positive words written in a white font and negative words in a black font, than to categorize negative words in a white font and positive words in a black font.
In further support of the relationship between valence and brightness, priming participants with positive or negatively valenced words has been shown to bias subsequent brightness judgments (Meier, Robinson, Crawford, & Ahlvers, 2007). Identical grey cubes were judged to be lighter when judged after a positive (vs. negative) word had briefly been presented on the screen. This result was replicated using a response- deadline procedure, where participants were forced to judge whether a grey cube presented on the screen was the lighter or darker cube of two previously presented stimuli. The fact that a similar pattern of results is observed when using a response deadline procedure is commonly interpreted in support of the assumption that valence and brightness are automatically associated. Of interest is that in none of these studies participants were actually asked to respond to the positive or negative stimuli presented before the brightness judgments were made.
Other studies have shown congruency effects for positive and negative words in different visual domains. In addition to the association between valence and brightness, Meier and Robinson (2004) found that the categorization of valenced words was influenced by the vertical position of the stimuli. In line with the metaphoric representation of ‘happy’ being up, and ‘sad’ being down (Lakoff & Johnson, 1980), positive words were classified faster when presented on the top of the screen than on the bottom, and negative words were classified faster when presented
41
on the bottom of the screen than on the top. In subsequent studies Meier & Robinson (2004) showed that
positive words direct attention upwards and negative words direct attention downward (similar to related effects in concrete domains, e.g., Bergen et al., 2007; Estes et al., 2008; Richardson et al., 2003). People were faster to indicate whether a letter presented on the top of the screen was a p or a q after evaluating a positive word, than after evaluating a negative word. The reverse was not true: after indicating whether a stimulus (+++) appeared on the top or the bottom of the screen, there were no differences in the speed with which positive or negative words are classified. Although the absence of differences is troublesome to interpret, the authors argue that the relationship between verticality and valence is asymmetrical: evaluations activate spatial positions, but spatial positions do not activate evaluations. These attentional effects were subsequently shown to be moderated by individual differences in affective states, with people higher in neuroticism being faster in detecting stimuli in the lower (versus higher) half of the screen (Meier & Robinson, 2006).
Further support for the relationship between verticality and valence was observed in studies which investigated whether the valence of pictures would influence judgments about the location where these pictures were previously observed (Crawford, et al., 2006). Positive pictures showed an upward bias when participants tried to indicate where these stimuli were presented previously. The authors suggest that the relationship between valence and vertical position might reflect the usefulness of the vertical spatial domain to organize and structure more abstract domains (cf. Gattis, 2001), without necessarily being the result of a direct experiential associations between positive mood and an upright position (Lakoff & Johnson, 1980).
Valence can also be represented in size, following the “more is better” metaphor (Lakoff & Johnson, 1999). Meier, Robinson and Caven (2008) presented words in a large or small font, and asked participants to evaluate the words as fast and accurately as possible. Participants were slower to evaluate negative words written in a big font compared
42
to negative words in a small font. In another study the authors show that words presented in a large font are rated more positively than words presented in a small font. The authors conclude that “stimulus size leads to relatively automatic inferences concerning stimulus valence”.
A last set of studies to investigate how abstract concepts are grounded in the vertical dimension looked at representations of affectively charged divine concepts (Meier, Hauser, Robinson, Friesen, & Schjeldahl, 2007). These studies showed the concept of God is strongly associated with up vertical locations, but gives less handholds to assume the vertical representation of God is grounded in sensory-motor experience (e.g., looking up when praying). If the vertical representation of God is grounded in sensory-motor experiences, the effects should be stronger for people who look up when praying (e.g., Christians) compared to people who never pray. However, the religion of the participants did not moderate the observed effects.
Chapter 4 focuses on the brightness-valence metaphor. Instead of merely demonstrating this metaphoric mapping, the studies in this chapter aim to further our understanding of the processes through which the brightness-valence mapping emerges. Two process explanations, the metaphoric structuring view and the one-to-one grounding explanation (described earlier in this chapter) are tested against each other. According to the one-to-one grounding account for metaphor congruency effect, brightness automatically activates positivity, and darkness automatically activates negativity. According to the metaphoric structuring view, the perceptual opposition between white and black structure the conceptual opposition between good and bad. Whereas according to the metaphoric structuring view shared relational structures are an essential prerequisite for the brightness-valence mapping to emerge, the polar oppositions in the brightness and valence dimensions are irrelevant for the one-to- one grounding account. These two process explanations were directly compared by manipulating the endpoints of the valence and brightness dimensions within or between participants, thereby activating the shared polar oppositions in the perceptual and conceptual dimensions when
43
people performed the study, or breaking them apart. Previous studies in the literature, as well as two pilot studies, reveal
that white is rated as a neutral color. How can a neutral color metaphorically represent the meaning of positivity? Six studies investigated whether the positivity of white was dependent upon the presence of the negativity of black, and that the opposition between good and bad is metaphorically structured in the opposition between light and dark. A Chinese translation task, where participants were asked to indicate whether a white or black Chinese ideograph correctly translated the meaning of the word, revealed that whereas black is strongly associated with negativity, white in isolation is not used to represent positivity. The positivity of white emerges only in opposition to the negativity of black. Whereas negative words are always translated by black ideographs above guessing average, positive words are only translated above guessing average by white ideographs when both the valence as the brightness dimensions are manipulated within subjects. Furthermore, two subsequent studies show that the positivity of white emerges when activating white-black positive-negative opposition perceptually during the translation task or conceptually before participants performed the translation task. Together, these studies reveal that the shared relational structure in the concrete and abstract domains is an essential feature of the brightness-valence mapping, supporting the metaphoric structuring view. Metaphoric representations are at best conditionally automatic, and require knowledge of the relational structures in abstract domains.
Linguistic Explanations
As the above review reveals, there is no shortage of empirical studies that provide support for the relationship between perceptual information and abstract conceptual processing. Studies that have examined the exact nature of this relationship, however, are substantially scarcer. This criticism on the lack of process explanations within embodied cognition is not limited to studies investigating abstract concepts (e.g., Barsalou,
44
2008b; Zwaan, 2009), but is less pressing in the case of concrete concepts, where the automatic activation of sensorimotor information in conceptual processing has received behavioral (e.g., Zwaan & Taylor, 2006, Glenberg, Sato, Cattaneo, et al., 2008) and neurophysiological (e.g., Hauk et al., 2004) support. The question whether abstract conceptual processing necessarily activates sensorimotor areas in the brain or whether sensorimotor activation largely plays a peripheral role in conceptual processing remains unanswered. The relationship between perceptual information and abstract concepts might even be epiphenomenal, in that people perceptually represent concepts when possible, but that the sensorimotor activation is unrelated to conceptual processing.
How conclusive are the empirical results that have been observed so far in the domains of morality, time and valence with regard to the perceptual nature of abstract thought? Could these findings be explained based on associations between purely amodal symbols? The short answer is yes. There is no empirical support for the idea that sensorimotor activation is absolutely necessary for abstract conceptual thought, or even that thoughts about abstract concepts are limited without relying on perceptual representations. Therefore, it is impossible to confirm the hypothesis that abstract concepts are automatically structured in concrete dimensions, and reject alternative explanations. Beside this obvious limitation of the current empirical data, there are several other reasons to take a cautionary approach when interpreting the results of studies that have investigated the perceptual representation of abstract concepts.
First of all, the support for metaphoric representations is almost exclusively linguistic in nature (but see Boot & Pecher, in press; Casasanto & Boroditsky, 2008; Crawford et al., 2006). This poses a problem, because many conceptual metaphors are derived from language use, and subsequently tested by asking people to process abstract language (Casasanto, in press). This circularity problem (Crawford, 2009) is even more problematic because language encodes embodied relationships (Louwerse, 2008). For example, remember the findings by Zwaan & Yaxley (2003), revealing that participants were faster to confirm that the
45
words branch and root were related when branch was presented above root, compared to when these words were presented in the reverse spatial layout. Louwerse (2008) shows that the words branch and root occur more frequently in the order branch-root than in the root-branch order in human language. Furthermore, the more frequently these combinations occur in language, the higher participants judge the likelihood of encountering the first word (branch) above the other (root), when explicitly asked. Crucially, word order frequencies predicted reaction times in the semantic judgment task better than spatial iconicity did. These studies show that findings in reaction time studies that are currently explained from a grounded cognition perspective can be explained by linguistic statistical information, because pre-linguistic conceptualizations such as spatial iconicity are reflected in language (see also Cooper & Ross, 1975, Greenberg, 1963).
A highly related linguistic characteristic that has been argued to play a role in studies investigating the perceptual representation of abstract concepts is the notion of linguistic markedness. People have preferred ways to talk and think about dimensions. For example, the question “how long will it take?” is used more frequently than “how short will it take?” which carries additional meaning (in this case, the expectancy that something will not take long). Therefore “long” is called the linguistically unmarked category, and “short” is called the marked category (Greenberg, 1963). Typically the category that gives a dimension its name is unmarked (Clark, 1973), such as long (length), bright (brightness) or happy (happiness). In addition to being used more frequently, unmarked categories are processed faster than marked categories (Clark, 1969).
These differences in markedness are not restricted to linguistic stimuli but extend to non-verbal stimuli such as arrows or left-right responses (for an overview, see Proctor & Cho, 2006). Finally, differences in markedness have also been observed for abstract relationships between people. De Soto, London and Handel (1965) were the first to show that the amount of errors people make when answering syllogisms about the relationship between two individuals depends on whether the words
46
‘better’ or ‘worse’ are used. Although ‘A is better than B’ and ‘B is better than C’ contains the same information as ‘B is worse than A’ and ‘C is worse than B’, the ‘better’ statements led to more accurate judgments about the relationship between A and C than the ‘worse’ statements. To acknowledge this more general response facilitation for the default endpoint of a category, unmarked categories (and their non-verbal counterparts) are referred to as + polarity, while marked categories are referred to as – polarity (see also Gattis, 2001).
Some differences in markedness have been attributed to the nature of the physical body (Benor & Levy, 2006; Clark, 1973; Krzeszowski 1997; Mayerthaler, 1980). Because humans have eyes in their heads, look to the front, and stand upright, up, front and verticality are unmarked. Differences in polarity have been argued to be essential aspects of image schema’s (Hampe, 2005; Krzeszowski, 1997), and most image schema’s (e.g., up- down, front-back, right-left) are indeed all +polar and –polar (Grady, 1997; Lakoff & Johnson, 1999). Furthermore, conceptual metaphors always map +polar source domains (e.g., up) onto +polar target domains (e.g., happy). Because the +polar endpoints are categorized faster than – polar endpoints (Clark, 1969), differences in linguistic markedness have often been discussed as an alternative explanation for response facilitation effects of metaphor congruent stimuli (e.g., Schubert, 2005; Vallesi et al., 2008; Van Dantzig, Zeelenberg, & Pecher, 2009; Weger & Pratt, 2008).
In addition, many studies have noted the difficulties of interpreting observed interactions between perceptual and abstract domains, given the frequent observations of two main effects, with the unmarked endpoints of both dimensions being categorized faster than the unmarked endpoints (Boot & Pecher, in press; Meier, Hauser, et al., 2007; Meier, Sellbom, et al., 2007). These two main effects could account for the observed interactions through the polarity correspondence principle (Proctor & Cho, 2006). The relative contribution of differences in markedness in studies investigating the representation of abstract concepts has not been examined in detail, but distinguishing linguistic markedness effects from true perceptual representations should be possible (cf. Louwerse, 2008) and deserves
47
greater attention.
Structural Explanations
Whereas differences in linguistic markedness might not directly reflect perceptual influences on abstract conceptual processing, linguistic markedness is at least argued to reflect the systematic relationship between language and the way people experience the world. Recently, researchers have highlighted several structural factors which might account for the metaphor congruency effects in the paradigms used to investigate the perceptual representation of abstract concepts. These structural factors include the level of task-representation, stimulus-response mappings, the salience of abstract and concrete dimension due to specific parameters in the paradigm, attentional factors, etc. All these factors can influence response times, but their influence on processing times of the stimuli is not indicative of the relationship between the intrinsic meaning of stimuli and their perceptual representation. Therefore, it is essential that future studies developed with the aim to investigate how abstract concepts are grounded take these alternative structural explanations into account and try to control for them.
Eder and Rothermund (2008) have recently highlighted the possible influence of structural factors in response time differences in some of the studies reviewed earlier in this chapter. The authors argue against the view that the mapping of affect onto sensorimotor information reflects an automatic relationship between sensorimotor experiences and conceptual representations (e.g., Lakoff, 2008, Niedenthal, et al., 2005; Zajonc & Marcus, 1985). They show that simply reversing the response labels ‘towards’ and ‘away’ while performing identical approach-avoidance lever movements influenced the speed with which valenced stimuli were categorized. Since previously assumed automatic bodily responses are shown to depend strongly on how these responses are labeled, the authors conclude that “affective-mapping effects are located in flexible specifications of evaluative stimulus and response properties that take task demands
48
and situational constraints into account” (Eder & Rothermund, 2008, p. 278). These findings are argued to show that perceptual and conceptual information is related on a cognitive, representational level, and not through ‘hard-wired’ automatic reactions upon perceiving sensorimotor information (see also Weger & Pratt, 2008). Scherer & Lambert (2009) have similarly warned against interpreting automatic priming effects in light of any pre-existing associations, because “automatic priming effects can arise for different reasons, and sometimes these reasons have little to do with the intrinsic features of the priming stimulus itself ” (p. 400).
A second structural factor which is argued to be important for the mapping of abstract concepts (e.g., valence) on perceptual domains is the amount of attention that participants assign to the affective stimulus dimension. The Stroop-like metaphor congruency effects reviewed earlier (and especially those using a response-deadline procedure) are interpreted in support of the idea that perceptual information automatically and obligatory influences abstract conceptual processing at the level of encoding (e.g., Meier & Robinson, 2004). However, the Stroop paradigm is goal-dependent, which speaks against the idea that unrelated dimensions (e.g., vertical spatial position) interfere with categorizations because of their automatically activated intrinsic meaning. As Neumann (1984, p. 269) summarizes: “to a large degree, a distracter causes interference not because of its intrinsic properties but because it is related to the intended action” (see also Klauer and Musch, 2002). In other words, the interference effects observed in these studies might not be the results of the automatic activation of the intrinsic value of the stimuli, but a result of the categorization goals participants have. The finding that a focus on the evaluative dimension is essential for affective priming to occur is likely to be just as important in studies that investigate how abstract concepts are grounded in perceptual dimensions, and where perceptual characteristics of stimuli are expected to automatically activate their associated valence. Since “the likelihood or degree of automatic affective stimulus processing depends on the extent to which the experimental context, as a whole, encourages participants to assign attention to the
49
affective stimulus dimension” (Spruyt, De Houwer, Hermans, & Eelen, 2007) it seems attentional processes are an integrative aspect of perceptual representations.
From Experience to Culture
Grounded theories of cognition differ in the function they ascribe to sensorimotor information in conceptual processing. On the most basic level, embodied cognition suggests that concrete experiences play a role in language processing, which is a relatively undisputed claim. A more debated question is to which extent conceptual processing necessarily and directly recruits sensory and motor systems. Meteyard and Vigliocco (2008) organize different theoretical proposals ranging from complete dependence of conceptual processing on sensorimotor information (e.g., Gallese & Lakoff, 2005), to stable associations between sensorimotor information and semantic representations which nevertheless do not necessarily activate sensorimotor content during conceptual processing (e.g., Boroditsky, 2000; Vigliocco, Vinson, Lewis, & Garrett, 2004), to complete independence between perceptual information and conceptual processing (Landauer & Dumais, 1997; Pylyshyn, 1984).
Instead of adopting a single view on the interplay between perception and cognition to explain all conceptual processing, a more fruitful approach might consist of looking carefully at how specific abstract and concrete concepts are understood and grounded. It should not be surprising that people rely on different mechanisms to understand complex and abstract sentences (such as the previous sentence in this paragraph) than more concrete sentences such as “Hitting gray walls is painful”. The assumption that there are multiple ways to process the meaning of concepts might not be the most parsimonious approach to conceptual processing, but it might be the most realistic (for a similar reasoning, see Machery, 2009). Some researchers have indicated that relying on perceptual grounding in some cases but not in others is a reason to do away with embodied approaches to cognition completely. For
50
example, Mahon and Caramazza (2008, p. 60) have argued that abstract concepts cannot be represented perceptually, and thus conclude: “Given that an embodied theory of cognition would have to admit ‘disembodied’ cognitive processes in order to account for the representation of abstract concepts, why have a special theory just for concepts of concrete objects and actions?” However, as the results in the current thesis as well as many recent findings in the literature show, perceptual information also play a role when processing the meaning of abstract concepts.
Is it necessary to admit ‘disembodied’ cognitive processes to account for the representation of abstract concepts, as Mahon & Caramazza (2008) have argued? The answer to this question will most likely depend on the definition of ‘representation’, which is why some researchers prefer not to use that terminology (e.g., Jackendoff, 2002). Some abstract concepts seem to be related rather directly to sensorimotor experiences, and sensorimotor information can represent the meaning of these abstract concepts without any need for additional cognitive processes. For example, both the grounding of interpersonal closeness in warmth (Bargh & Williams, 2008; IJzerman & Semin, 2009) and the grounding or morality in disgust (Chapman, Kim, Susskind, & Anderson, 2009; Schnall, Haidt, et al., 2008; Zhong & Liljenquist, 2006) seem to relate the full abstract meaning of these concepts to concrete experiences.
Other metaphoric representations influence unrelated judgments, but do so in a more cognitively moderated fashion, depending on attentional processes, task representations, and flexible stimulus-response mappings. A wide range of abstract concepts exists, ranging from relatively universal concepts such as positivity and negativity, through ontogenetic factors such as interpersonal affection, and social and cultural concepts such as morality, and democracy. Abstract concepts differ in the extent to which they are grounded in concrete human experiences or cultural influences. On the more experiential end of the scale, the relationship between sensorimotor information and abstract concepts might be automatic, non-verbal, based on shared neural activation, inflexible, and universal. On the more cultural end of the scale, abstract concepts might
51
be related to sensorimotor information indirectly, specified in language, cognitively mediated, and flexibly mapped onto concrete domains depending on cultural factors.
As mentioned previously, several studies on the more experiential end of the experience-culture dimension have shown that concrete perceptual experiences directly affect abstract processing. For example, Williams & Bargh (2008) investigated the experiential relationship between warmth and affection, and showed that simply holding a warm or cold cup will influence ratings of affection (for similar findings, see IJzerman & Semin, 2009). Another set of studies on the relationship between weight and importance has revealed that holding either a heavy or light clipboard influences how important participants judge the topic of a questionnaire to be, resulting in more invested effort in answering the questions, which leads to more elaboration in the heavy clipboard condition (Jostmann, Lakens & Schubert, 2009). Furthermore, experiencing a bad smell influences subsequent moral judgments (Schnall, Haidt, et al., 2008). It seems to be difficult to account for these findings by the previously discussed linguistic and structural explanations.
On the more culturally determined end of the scale are concepts which are grounded in concrete experiences, but these concepts are grounded in different ways in different cultures, or even within cultures. It is also possible that certain aspects of abstract concepts are culturally determined (and therefore flexibly mapped on concrete dimensions) whereas other aspects of abstract concepts are more experientially determined (and show more direct perceptual-conceptual relationships). For example, Boroditsky (2000) notes that people experience time as unidirectional change, marked by the appearance and disappearance of events, and this aspect of time should be universal across cultures. However, which way time moves (up-down, left-right, front-back, or the reverse) is not observable, and these aspects are specified in language, often through spatial metaphors. Indeed, which spatial metaphor is applied to time is flexible (Boroditsky, 2001; Torralbo et al., 2006). Given that this flexibility in how abstract concepts are metaphorically structured exists
52
for many (aspects of) abstract concepts, the strong claim that all abstract processing relies completely and exclusively on perceptual information (e.g., Lakof & Johnson, 1980) seems untenable (e.g., Boroditsky, 2000; Zwaan, 2009). Nevertheless, abstract concepts fall well within the scope of grounded cognition, and acknowledging the importance of perceptual representations of abstract concepts will substantially improve our understanding of abstract conceptual processing.
Practical Implications
Embodied cognition research has highlighted the many ways in which subtle perceptual cues influence affective reactions, judgments under uncertainty, person perception, social interaction, and information processing (for an overview, see Pecher & Zwaan, 2005; Niedenthal et al., 2005; Semin & Smith, 2008). Regardless of how sensorimotor information influences social psychological processes, research findings over the last decennium make it abundantly clear that sensorimotor information influences conceptual thought in more ways than researchers were previously aware of.
The way people metaphorically structure abstract concepts has been argued to constrain reasoning about these abstract concepts (Lakoff, 1996; Lakoff & Johnson, 1999). Several recent findings reveal this is indeed the case. Judgments about an ambiguous question concerning the time at which events take place (“The meeting originally scheduled for next Wednesday has been moved forward two days”) depends on whether people have just moved through time themselves, or are sitting still (Boroditsky & Ramscar, 2002). Animals presented high up in the visual field were judged to be more respectable than animals lower in the visual field (Schubert, 2005). Pictures of people presented in the top of the screen were rated as more religious than pictures of people presented at the bottom of the screen (Meier, Hauser, et al., 2007). And finally, words presented in a big font were judged to be more positive than words presented in a small font (Meier et al., 2008). In all these instances,
53
events, objects or people were seen differently depending on sensorimotor experiences of the perceiver.
The metaphorical representation of morality has been theorized to influence and constrain moral judgments in a similar manner (Lakoff & Johnson, 1999). Graham, Haidt, and Nosek (in press) collected data that supports this assumption by comparing the way liberals and conservatives represent morality. Whereas liberals frame moral issues predominantly in terms of fairness and harm, conservative speakers also use metaphors related to authority and purity when talking about moral issues. These different representations of what morality is might have profound consequences in domains such as politics (Lakoff, 1996). It therefore seems crucial to increase our understanding of how different metaphoric representations of morality shape the way people think about moral issues (see Chapter 2).
Additionally, the perceptual representation of abstract concepts might be important for the way children learn concepts (e.g., Smith, 2005). The conceptual system has been argued to develop simultaneously and in parallel with the perceptual and motor systems (Mandler, 1992, 2004), and many global and abstract concepts are built on image-schema’s that are derived from concrete experiences. As such, it will be interesting to investigate to what degree infants’ abstract conceptual development is based on sensory and motor experiences (e.g., Gattis, 2002; Piaget, 1954). For example, concrete objects and spatial relations have been argued to play an important role in the mathematical thinking of children (Bryant & Squire, 2001). Investigating the role metaphoric representations play when children develop an understanding of abstract concepts seems to be a promising avenue for further research.
Conclusions
People draw on concrete experiences when they think about abstract concepts such as morality, time and valence. Even though perceptual information is intimately related to conceptual processing, perceptual
54
representations of abstract concepts are not expected to account for all abstract conceptual thought (see Boroditsky & Prinz, 2008). Luckily, perceptual information can be easily integrated with other sources of information, such as linguistic information based on the statistical distribution of words in spoken and written language. A growing body of work exists that compares the explanatory power of sensorimotor information in conceptual processing with linguistic information (e.g., Louwerse, 2008) and theoretical frameworks with the aim to integrate these two sources of information have been developed (e.g., Andrews et al., 2009). Future studies that aim to distinguish between the contributions of perceptual and linguistic sources of information while controlling for structural explanations for the relation between perceptual information and abstract concepts will open the way for a more informed approach to the grounding of abstract concepts.
People rely on concrete perceptual information to structure abstract concepts (see Chapters 2 and 3). Under certain circumstances, the partial re-activation of the same neural mechanisms that process concrete information might be activated when abstract concepts are processed. For example, researchers have argued such re-activation exists for the morality-disgust relationship (Chapman et al., 2009) and the relationship between affection and warmth (Williams & Bargh, 2008; IJzerman & Semin, 2009). Other perceptual representations might be more cognitively mediated and be a more activate structuring of abstract concepts in terms of concrete experiences (see Chapter 4), but this doesn’t make the role of perceptual information in abstract conceptual processing trivial. Abstract thought is possibly one of the most sophisticated abilities of human beings, and regardless of whether it is necessarily perceptual in nature, it is worthwhile to improve our understanding of how people perform such a remarkable feat.
Chapter 2
On the Grounding of Morality in Perceptual Symmetry
This chapter is based on: Lakens, D., van den Bos, K., & Semin, G. R. (in preparation). On the grounding of morality in perceptual symmetry.
57
Δίκης ὄνομα οὐκ ἂν ᾔδεσαν, εἰ ταῦτα μὴ ἦν
Men would not have known the name of justice if these things were not
Heraclitus, fr. 23 DK
Although the exact relationship between sensorimotor information and abstract conceptual processing remains debated (Arbib, 2008; Barsalou; 2008; Boroditsky & Prinz, 2008; Lakoff, 2008; Mahon & Caramazza, 2008), many researchers agree that concrete perceptual information plays a role in the processing of abstract concepts. According to Lakoff & Johnson (1980, 1999) concrete information is used to structure abstract concepts through metaphorical representations. Expressions such as “the mighty above the weak”, “looking forward to next week” or “things are looking up” are not merely linguistic expressions, but reflect the way people think about abstract concepts (Boroditsky, 2000; Meier & Robinson, 2004; Schubert, 2005). Abstract concepts are metaphorically structured in concrete experiential domains.
Despite such empirical demonstrations of the grounding of abstract concepts, some researchers argue that “perceptual representations seem ill-suited for representing abstract concepts” (Dove, 2009). Grounded theories of cognition would be sharply limited up front if they fail to accommodate highly abstract conceptual thought, and morality is one of the hallmark abstract concepts for which is argued that there is “no sensory or motor information that could correspond in any reliable or direct way to their meaning” (Mahon & Caramazza, 2008). In the current chapter I adopt a different perspective and argue that morality is metaphorically represented in perceptual symmetry. Studying this hypothesis would both contribute to the morality domain as well as further the investigation of the perceptual representation of abstract concepts.
Traditionally, moral judgments were assumed to result purely from deliberative reasoning processes (Kohlberg, 1969). However, more recently,
58
a more intuitive view on moral judgment has emerged (Haidt, 2001), which proposes that moral judgments are often based on both affective experiences (such as disgust; Chapman, Kim, Susskind, & Anderson, 2009; Schnall, Haidt, Clore, & Jordan, 2008) and bodily experiences (such as cleanliness and purity; Schnall, Benton, & Harvey, 2008; Zhong & Liljenquist, 2006) which precede conscious verbal reasoning (Haidt, 2008). Several physical experiences have been shown to influence moral and justice judgments, such as arm flexion and extension (Van Prooijen, Van Beest, & Karremans, 2006), the weight of the questionnaire on which moral scenarios are presented (Jostmann, Lakens, & Schubert, 2009), the bodily position of participants (Broeders, Van den Bos, & Müller, 2009), perceptual fluency (Laham, Alter, & Goodwin, in press), or the use of personal force (Greene, Cushman, Stewart, Lowenberg, Nystrom, & Cohen, in press).
Together, these studies indicate that moral judgments are influenced by sensorimotor information (see also Greene et al., in press; Laham et al., in press). Experiential factors seem to influence peoples’ decisions about whether something is moral or immoral. Despite these new insights into the role of experiential factors in moral judgments, the role of sensorimotor information in how people understand and think about abstract moral concepts has received relatively little attention. Morality is an important concept in social interaction (Folger, 1984), and thus in human communication (Wijn & Van den Bos, in press), and people often think and talk about moral concepts without performing moral judgments (as throughout this chapter, for example). Does perceptual information play a role when people process moral concepts?
Some researchers argue that only the affective consequences of moral judgments can ground morality, but that the concept of morality itself cannot be grounded perceptually (Andrews, Vigliocco, & Vinson, 2009; Boroditsky & Prinz, 2008; Dove; 2009; Mahon & Caramazza, 2008; Prinz, 2005). As Boroditsky and Prinz (2008, p. 103) argue: “The range of things we call morally bad have little in common perceptually (stealing, cheating, hitting, and so on), so there cannot be a single image of badness.
59
But all of these things are united by the family of emotions they cause in us.” Other researchers have argued in favor of the perceptual grounding of moral concepts (Barsalou, 2008a; Gibbs, 1996; Haidt, 2001; Johnson, 1993; Meier, Robinson, & Wygant, 2007; Lakoff & Johnson, 1980, 1999; Sherman & Clore, 2009). This relationship between sensorimotor information and moral concepts has already been put forward by Whitney, when he noted the importance of the widespread “application of terms having a physical, sensible meaning, to the designation of intellectual and moral conceptions and their relations” (Whitney, 1875, p. 88).
Previous studies that have investigated the perceptual grounding of moral concepts primarily focused on the affective components of morality. Comparable to how positivity is up, and negativity is down (Lakoff & Johnson, 1999, Meier & Robinson, 2004), or positivity is white, and negativity is black (Meier, Robinson, & Clore, 2004), moral and immoral words have been shown to be associated with up and down, and white and black, respectively (Meier et al., 2007; Sherman & Clore, 2009). The present research takes a new approach to the grounding of morality by focusing on the equality component of morality, and showing how perceptual symmetry is used to metaphorically structure moral concepts. Equality is considered a constitutive feature of justice in the writing of philosophers like Aristotle, Hobbes, Locke, Rousseau, Kant and Rawls. Equality underlies common folk theories about morality (e.g., ‘an eye for an eye’) as well as classical theories about distributive justice such as equity theory (Adams, 1965), where the basic underlying justice principle is a balance between the contributions and rewards of two people.
The equality component of justice is theorized to be grounded in the balance image schema (Gibbs, 2005; Johnson, 1987, Lakoff, 1987) and conceptualized metaphorically as balance or symmetry (Lakoff, 2005), at least in Western cultures (Lakoff & Johnson, 1999). The balance image schema is the basis of a fundamental metaphor for morality, called moral accounting (Lakoff & Johnson, 1999), where harms and benefits are weighed against each other until the moral books are balanced (Lakoff & Johnson, 1980, cf. Adams, 1965). Messick (1993) proposes that the
60
symmetric nature of equality may be used as an efficient decision heuristic when judging how fair goods are distributed. Indeed, many of our concrete experiences with moral concepts are directly related to symmetrical or balanced distributions of rewards, goods and affect (for example, seeing parents divide a birthday cake into equal pieces). Through metaphorical extension, such sensory experiences can structure our understanding of moral concepts (Lakoff, 1996).
In the present chapter I reason that given this strong experiential association between equal divisions and justice, moral concepts should be associated with perceptual symmetry. If our understanding of abstract concepts builds on perceptual information (Lakoff & Johnson, 1980, 1999), then thinking about abstract concepts should activate the associated sensorimotor information. If people learn about morality and justice through experiential information related to equality then thinking about morality should activate non-verbal associations with perceptual symmetry. In Study 2.1, I investigated whether moral words were associated with symmetrical stimuli, and immoral words were associated with asymmetric stimuli. In Study 2.2, moral words were expected to show a stronger association with perceptually equal distributions of colored letters than immoral words.
Study 2.1: Morality is symmetrical
If morality is grounded in perceptual symmetry then moral concepts should activate non-verbal associations with balance and symmetry, whereas immoral concepts should be associated with imbalance and asymmetry. In the current study, I tested the hypothesis that when performing judgments under uncertainty, participants would be drawn towards symmetrical choice alternatives for moral words, whereas participants would be drawn towards asymmetrical choice alternatives when performing judgments for immoral words.
61
Method
Participants. Thirty-seven students at Utrecht University (26 females, mean age 21) participated in this study for partial course credit or payment.
Procedure. Participants completed a Chinese translation task where they were asked to translate 20 Dutch words into Chinese by choosing one of two Chinese ideographs presented on the screen. Participants were told that previous research had shown that people could translate Dutch words into Hebrew characters correctly above guessing average, even though people did not speak Hebrew. The current study would test a similar hypothesis with Chinese ideographs. The Dutch words consisted of 10 moral words and 10 immoral words (see Appendix 1) and all ideographs were presented in black on a white background, side by side underneath the morality-related word. The presentation of all words, ideographs and their combinations was completely randomized. After choosing the ideograph that participants thought was the best translation for each word, participants rated all 40 ideographs on symmetry by crossing a line ranging from 0 (not at all symmetrical) to 100 (completely symmetrical). None of the participants knew the real meaning of the Chinese ideographs, which were in reality all unrelated to morality.
Results
To test the hypothesis I compared the absolute average symmetry ratings for the 10 ideographs participants chose to translate moral words with the 10 symmetry ratings for ideographs participants chose to translate immoral words. In addition, I compared the relative difference in symmetry between the ideograph participants chose and the ideograph participants did not choose for both moral and immoral words. Both analyses should show that symmetrical choice alternatives are preferred for moral words, compared to immoral words.
Absolute symmetry ratings. The symmetry ratings for the
62
ideographs participants chose to translate the Dutch words were averaged for both moral and immoral words. In line with the predictions about the association between morality and symmetry, a repeated measures ANOVA revealed that the average symmetry of the ideographs chosen to translate moral words was higher (M = 46.92, SD = 11.66) than the average symmetry for ideographs chosen to translate immoral words (M = 41.31, SD = 9.47), F(1, 36) = 7.68, p = .009, ηp² = .18.
Relative symmetry ratings. For each word, the difference between the symmetry rating of the ideograph that participants chose minus the symmetry rating for the ideograph that participants didn’t choose was calculated. Positive difference scores indicate that the participant chose the more symmetric ideograph to translate the stimulus word, whereas negative difference ratings indicate participants on average chose the more asymmetric ideograph of the two choice alternatives. These difference ratings were averaged over the 10 moral and the 10 immoral words. As predicted, a repeated measures ANOVA revealed that participants on average chose more symmetric ideographs to translate moral words (M = 7.53, SD = 11.46) than immoral words (M = -3.37, SD = 12.72), F(1, 36) = 14.06, p = .001, ηp² = .28.
Discussion
When choosing between two unknown Chinese ideographs to translate a Dutch word, participants were more drawn towards symmetrical choice alternatives when translating moral words, whereas participants preferred the more asymmetric choice alternatives for immoral words. These results confirm the prediction that morality is associated with perceptual symmetry. When participants had to judge which of the two ideographs they thought correctly translated morality related words, the amount of symmetry in the ideographs was used as a source of information. The more perceptually symmetrical ideograph was more likely to be seen as the correct translation for moral words, whereas the more asymmetric ideograph was preferred for immoral words.
63
These results support the hypothesis that there is a strong association between morality and perceptual symmetry, in line with the assumption that morality is metaphorically grounded in non-verbal representations of balance and symmetry (Lakoff & Johnson, 1999; Lakoff, 2005). It seems that previous criticisms about the lack of perceptual characteristics of moral concepts (Boroditsky & Prinz, 2008; Dove; 2009; Mahon & Caramazza, 2008) are unwarranted. The “balance of justice” seems to be more than just a figure of speech; it serves as an experiential source which provides structure to moral concepts. In the second study, I aimed to replicate and extend these findings with a new paradigm.
Study 2.2: Morality is fifty-fifty
The current study investigated the relationship between morality and perceptual symmetry more indirectly. Whereas Study 2.1 showed that symmetric stimuli are judged to be a better translation of words with a moral meaning, the current study investigated whether moral words would influence judgments that were unrelated to the meaning of the stimulus words. Under the pretext of a memory task, 40 words were presented (half with a moral meaning, half with an immoral meaning) to participants, and asked them to remember the words. The first few letters of these words were presented in a blue font, the remaining letters of the words were presented in a green font. After all words were presented, participants were asked to indicate how many letters of each word had been blue. I hypothesized, based on the “balance of justice” metaphor, that participants would display a biased recall of the distribution of blue and green letters. Judgments for the number of blue letters in moral words were expected to be closer to half the number of letters in the word compared to judgments for immoral words, indicating a more perceptually symmetrical color distribution.
64
Method
Participants. Thirty-one students at Utrecht University (19 females, mean age 20) participated in this study for partial course credit or payment.
Stimuli. Half of the 40 morality related words (see appendix 1) had an even number of letters and half had an uneven number of letters. The word length did not differ for moral (M = 8.00, SD = 2.53) and immoral stimuli (M = 8.30, SD = 2.11), F < 1. Half of the stimuli were presented with an equal distribution of blue and green letters, whereas for the other half of the stimuli the number of blue and green letters was distributed unequally. More specifically, for the stimulus words with an equal distribution of blue and green letters, 50% of the word (rounded up or down for words with an uneven number of letters) were presented in a blue font (for example, a 6 letter word would have 3 blue letters, the absolute average deviation from half being 0, and a 7 letter word would have 3 or 4 blue letters, the absolute average deviation from half being 0.5). For the other half of the stimuli, 50% of the word (rounded up or down for words with an uneven number of letters) plus or minus one letter was blue (for example, a 6 letter word would have either 2 or 4 blue letters, the absolute average deviation from half being 1, a 7 letter word would have either 2 or 5 blue letters, the absolute average deviation from half being 1.5). For both moral and immoral words, the true average absolute deviation of the blue letters from half the number of letters in the word was 0.75.
Procedure. Under the cover story of a memory task, a series of 40 words were randomly presented on the screen. Participants were instructed to try to remember each word. The stimuli consisted of 20 moral words, and 20 immoral words. Part of each word was written in a blue courier font, part was written in a green courier font. During the memory phase, all words were presented randomly for 1.5 seconds. Unbeknownst to the participants, their real task consisted of answering how many letters of each word had been colored blue. After the memory
65
phase, all words appeared randomly for a second time, but now in a grey font. Participants could type in a number to indicate how many letters of the word they thought had been blue when the word was presented during the memory phase. After the recognition task, people were asked to rate all words on morality, after which they were thanked and debriefed.
Results
Manipulation check. The manipulation check confirmed that moral words were rated as more moral (M = 5.94, SD = 0.47) than immoral words (M = 2.06, SD = 1.07), F(1, 30) = 18.29, p < .001, ηp²= .38.
Letter judgments. I calculated the average absolute deviation from half the number of letters in each word for all the blue letter judgments. Half of the total number of letters of each stimulus word was subtracted from the judgments participants made. For example, if a participant thought the word ‘unfair’ originally had 2 blue letters, and half the number of letters in the word is 3, the deviation would be -1. Subsequently, the absolute deviations were averaged for the 20 moral and 20 immoral words. As noted in the procedure section, the correct absolute average deviation of blue letters from half the number of letters in both moral and immoral words was 0.75. In line with the hypothesis, the judged number of blue letters deviated less from half the word for moral words (M = 0.97, SD = 0.27) than for immoral words (M = 1.14, SD = 0.37), F(1, 30) = 18.29, p < .001, ηp² = .38, indicating that participants judged the distribution of blue and green letters in moral words to have been closer to a perceptually symmetrical fifty-fifty distribution compared to immoral words.
Discussion
When participants were asked to remember how many letters of a previously presented word had been blue, the moral or immoral meaning of the words influenced participants’ judgments. For moral words, judgments were closer to an equal number of blue and green letters,
66
compared to the judgments made for immoral words. This result provides further support to the hypothesis that moral concepts are associated with visually equal distributions. When guessing the number of blue letters for moral words, choice alternatives closer to a perceptually symmetrical color distribution had a stronger likelihood of being chosen than choice alternatives further from a perceptually symmetrical color distribution. Whereas in Study 2.1, participants were asked to judge which ideograph best represented the meaning of the Dutch word to be translated, Study 2.2 extended these findings to a paradigm where the meaning of the stimuli was irrelevant for the judgments participants made.
General Discussion
In two studies I found support for the hypothesis that morality is associated with perceptual symmetry. In the first study, participants preferred the more symmetric Chinese ideograph to translate moral words, but preferred the more asymmetric ideograph to translate immoral words. In the second study, participants indicated in a memory task that the number of blue and green letters of previously presented moral words had been more equally distributed than the number of blue and green letters of immoral words. These studies support the prediction that moral concepts activate non-verbal representations of visual equality and symmetry.
The implications of these findings are twofold. On a theoretical level, morality has often been used as an example of an abstract concept that lacks clear perceptual characteristics, and is used to argue against the perceptual representation of abstract concepts (e.g., Dove, 2009; Mahon & Caramazza, 2008; Prinz, 2005). The current studies reveal that even highly abstract concepts as morality are metaphorically structured in concrete experiences. Although the extent to which sensorimotor information is necessarily activated to process concepts remains to be determined (Mahon & Caramazza, 2009; Meteyard & Vigliocco, 2008; Zwaan, 2009), sensory information seems to be an important source of structure and meaning for abstract concepts. Given this strong relationship
67
between sensory information and abstract concepts, embodied theories of cognition can improve our understanding of how people think and talk about abstract concepts such as morality.
On a practical level, the finding that morality is metaphorically structured in perceptual symmetry might have consequences for how people interpret moral situations. Previous studies in the abstract domains of time, power and divinity have shown that judgments about the time at which events take place, how powerful animals are and how religious people are can all be influenced by sensory information related to the metaphoric representation of these abstract concepts (Boroditsky & Ramscar, 2002; Meier, Hauser, Robinson, Friesen, & Schjeldahl, 2007; Schubert, 2005). The metaphorical representation of morality has been theorized to influence and constrain moral judgments in a similar manner (Lakoff & Johnson, 1999; Graham, Haidt, & Nosek, in press). For example, does the fact that the US Supreme court sits in a perfectly symmetrical court, inside a symmetrical building, influence our judgments about how just their decisions are? Does perceptual inequality activate the concept of immorality, possibly leading to subsequent moral reasoning? The current research suggests this might be the case, and given the importance of moral concepts for social interaction (Folger, 1984), such empirical questions are worthwhile to pursue.
Valence is an important component of morality, with moral concepts being more positive than immoral concepts. One might argue that the results in Study 2.1 can be explained by the possibility that symmetrical ideographs were chosen more often for moral words because symmetric ideographs are more positive than asymmetric ideographs. Symmetry and positivity have been shown to be related in situations where the symmetrical nature of stimuli leads to a more fluent processing of the stimulus (for a review, see Reber, Schwarz, & Winkielman, 2004). This reasoning is based on the assumption that perfectly symmetrical stimuli contain less information (with one half being identical to the other half) and are therefore easier to process. None of the ideographs used in the current study were perfectly symmetrical, and the ideographs
68
were on average low in symmetry, making fluency related differences in the processing of the ideographs unlikely. Furthermore, it seems difficult to account for the findings in Study 2.2 purely based on differences in positivity of the choice alternatives. Nevertheless, both the valence and the equality component of morality can be metaphorically represented, and their interplay deserves further attention.
It has rightly been noted that even when abstract concepts are experientially grounded, concepts such as morality cannot be fully understood without complementing the meaning of these abstract concepts by additional sources of knowledge, such as contextual information, linguistic data and cultural background (Andrews, Vigliocco, & Vinson, in press; Boroditsky & Prinz, 2008). Understanding moral concepts is certainly not limited to perceptual processes, but the sensorimotor information which is used to metaphorically structure moral concepts is argued to be an essential part of how morality is conceptualized (Lakoff, 1996; Lakoff & Johnson, 1999). As such, experiential grounding provides a basis to structure moral concepts, as well as a constraint on peoples’ idea of what morality is (Haidt, 2001; Graham et al., in press; Lakoff, 1996). A promising avenue for further research would be the investigation of the role metaphoric representations play when children develop an understanding of abstract concepts, and whether metaphoric representations play the facilitatory role in moral development often ascribed to them (e.g., Lakoff & Johnson, 1980, 1999).
The present studies support the hypothesis that people’s understanding of morality and immorality is intimately related to sensory information about perceptual symmetry. These results complement the emerging view that experientially grounded moral concepts underlie moral intuition (Haidt, 2008). Where previous work has focused on an affective basis for moral judgments (Haidt, 2001, Prinz, 2007), the currents results reveal that moral concepts are metaphorically structured based on sensorimotor information. These findings show that sometimes, the theorized lack of perceptual characteristics of highly abstract concepts is simply a matter of finding the right perceptual representation.
69
Moral concepts fall well within the scope of grounded cognition, and acknowledging the importance of perceptual representations of moral concepts will substantially improve our understanding of moral reasoning.
Chapter 3
The Sound of Time
This chapter is based on: Lakens, D., Semin, G. R., & Garrido, M. (under review). The sound of time: Locating time in auditory space.
73
Alice sighed wearily. ‘I think you might do something better with the time,’ she said, ‘than waste it in asking riddles that have no answers.’
Lewis Carroll
How people represent, think, and communicate about abstract concepts that do not afford immediate sensorimotor experiences remains a challenge for embodied approaches to cognition (cf. Barsalou, 2008a; Boroditsky, 2000; Lakoff & Johnson, 1999). As the introduction of this thesis as well as the previous chapter have shown, perceptual experiences can serve as the source for metaphorical representations of abstract target concepts. Indeed, there is abundant empirical evidence showing that concrete spatial dimensions provide a way to structure abstract concepts, such as time (Boroditsky, 2000; Boroditsky, 2001; Boroditsky & Ramscar, 2002; Casasanto & Boroditsky, 2008; Núñez, & Sweetser, 2006; Santiago, Lupiáñez, Pérez, & Funes, 2007; Torralbo, Santiago, & Lupiáñez 2006; Tversky, Kugelmass, & Winter, 1991; Vallesi, Binns, & Shallice, 2008; Weger & Pratt, 2008; Zacks & Tversky, 1999).
The three studies reported in this chapter introduce a novel perspective and method to examine the embodied grounding of time. This perspective relies on the argument that if time is spatially structured, then the abstract concept of time could in principle be structured in any modality that is sensitive to spatial information. Our auditory senses have developed a great sensitivity in deriving spatial information from sound (Blauert, 1997). Consequently, the auditory system should map time concepts in a manner comparable to the visual modality. The current research focuses on the multimodal grounding of time, and is the very first examination of the role that the auditory system plays in grounding abstract concepts.
These studies investigate the left-right representation of time in space, where the past is located on the left and the future on the right (Santiago et al., 2006; Maass & Russo, 2003; Tversky et al., 1991; Weger
74
& Pratt, 2008). Study 3.1, in which the systematic visual mapping of time in space was established, lays the groundwork for the subsequent two auditory studies. In these two studies, participants are asked to place words referring to the past and the future on a horizontal line, and a clear left-right ordering of past and future-related words, respectively, was expected.
The second and third study introduced a novel method, namely an auditory loudness judgment task. Participants received past and future-related words via headphones. On critical trials these words were presented equally loud on both the right and left auditory channels. These critical trials were distributed among a number of randomly presented filler trials varying systematically in their degrees of loudness in the right or left auditory channels. The participants’ task was to judge in which auditory channel the word was presented louder, whereby judgments on critical trials constituted the chief dependent variable. Participants were expected to more often judge future related words to be louder in the right ear, compared to past related words. Furthermore, comparisons of the visual positioning of the time related words in the first study and the loudness judgments in the second and third study were expected to reveal substantial overlap, indicating that the further words were visually positioned to the right, the more often they were judged to be louder in the right ear.
Study 3.1
In this study, participants were asked to place past and future-related words on a horizontal line. The expectation was that future-related words would be placed to the right of the midpoint of the line, whereas past- related words were expected to be placed to the left of the midpoint.
Participants. Fifty-six students (38 females, mean age 20) participated in this study.
Procedure. Participants were asked to place 8 past and 8 future- related words on a horizontal line, with an indicator at the midpoint of the
75
line anchoring the present. The ends of the line were not marked. Eight future (later, future, day after tomorrow, coming, tomorrow, immediately, soon, shortly) and eight past-related words (past, day before yesterday, earlier, been, before, yesterday, recently, a short while ago) were used. All stimuli consisted of single words in Dutch. Participants indicated the position they thought was best suited for the stimulus by placing the cursor on the line followed by a left mouse click, which was scored from 0 (completely left) to 100 (completely right).
Results
For the analyses I averaged the spatial position scores of future and past- related words. As expected, future-related words were placed further to the right from the midpoint (M = 66.42, SD = 3.99) than past-related words (M = 29.71, SD = 4.31), t(55) = 37.43, p = .001, prep = .99. Both differed significantly from the scale midpoint, t(7) = 3.66, p = .005, prep = .97 and t(7) = -4.02, p = .008, prep = .98, respectively. The absolute distances of past and future words from the scale midpoint did not differ, t(14) = .57, p = .58, prep = .65.
Discussion
Along with earlier findings (e.g., Tversky et al., 1991), the positioning of past and future-related words uniformly showed a respective left to right ordering on a horizontal dimension. I subsequently set out to test whether the left-right representation of time extends to the auditory modality, and to examine whether the visual spatial position of the past and future words is related to auditory left vs. right loudness judgments.
Study 3.2
The current study was designed to investigate whether the pattern of spatial distributions obtained for time-related words in the first study is
76
reproduced in the auditory modality. A new paradigm was developed, where participants had to judge whether words presented over headphones were louder in the left or right auditory channel. For the critical experimental trials, the future and past related words were presented equally loud on both channels. I expected the temporal meaning of the words to bias participants’ loudness judgments. On trials where time- related words are presented equally loud on both channels, participants were expected to judge future related words to have been louder in the right ear, and past related words to have been louder in the left ear.
This judgment bias was not expected to be completely symmetrical. Verbal information is processed more efficiently when presented to the right than the left ear (Belin et al., 1998; Kimura, 1961) due to the hemispheric asymmetry in language processing, with Broca’s speech area located in the left hemisphere, (Belin et al., 1998; Pujol, Deus, Losilla, & Capdevila, 1999; Toga & Thompson, 2003). The literature suggests that there should be an overall shift towards a right channel bias for loudness judgments for all time-related words. Despite this possible main effect, the average number of right ear loudness judgments was still expected to differ for past and future related words. In addition, the comparison of the spatial positioning obtained in Study 3.1 and the loudness judgments in this study were expected to show a systematic overlap, since this comparison involves relative differences between the different time- related words.
Method
Participants. Twenty-eight students (18 females, mean age 20) participated in this study.
Stimuli. The temporal meaning (past vs. future) of the stimuli was manipulated within participants. The 8 past and 8 future-related words from Study 3.1 were transformed into audio files using a text-to-speech program. Two sets of auditory stimuli each with five sound volumes (50%, 40%, 30%, 20%, or 10% louder on the left or right channel) were
77
created using Audacity for Windows. Additionally, there was a version with identical volume levels in both auditory channels. These auditory stimuli constituted the critical experimental trials.
Procedure. Participants were seated in individual cubicles and wore headphones. They had to listen to computer generated words and asked to indicate for each word whether the word they heard was louder in the left (by pressing the “A” key) or the right channel (by pressing the “L” key). Participants were told that the meaning of the words was irrelevant for the task. They were also informed that the differences in volume between the left and right ear could be very subtle, and that they should try their best to answer accurately. The 10 audio files with unequal volume in the left and right audio channels were presented once for each of the 16 words. The audio files used in the experimental trials, with equal volume in both auditory channels, were presented three times, resulting in 48 critical trials, of which half consisted of future related words, and half consisted of past related words. The order in which the 208 trials were presented was completely randomized.
Results
Analyses by participants. Since left and right judgments are mutually dependent, the average number of times the 24 past and 24 future related critical trials were judged to be louder in the right channel was calculated. As expected, these averages differed based on the temporal meaning of the stimuli. A repeated measures analysis revealed that perfectly balanced future related words were judged to be louder in the right ear more often (M = 14.79, SD = 5.27) than perfectly balanced past related words (M = 13.38, SD = 4.89), F(1, 27) = 4.61, p = .04, ηp² = .15 (see figure 1, left pane). This bias in loudness judgments is exactly what one would predict given the left to right representation of time in space: participants judged future related words to be louder in the right channel more often than past related words.
Analyses by items. For the analysis by items, the average number
78
of times the responses for the 8 past and 8 future-related words in the critical trials were judged to be louder in the right channel was calculated. This gives a range between a value of 0 and 3, with 1.5 indicating an equal number of left and right channel judgments. As expected, the judgments differed as a function of the temporal meaning of the words. A repeated measures analysis revealed that future-related words were judged to be louder in the right ear more often (M = 1.85, SD = 0.17) than past-related words (M = 1.67, SD = 0.13), F(1, 14) = 5.49, p = .03, prep = .93, ηp² = .28 (see Figure 1, left pane).
Figure 1. Average number of right ear loudness judgments for words presented equally loud on both auditory channels in Study 3.2 (left pane) and Study 3.3 (right pane) as a function of their temporal meaning.
1,3
1,4
1,5
1,6
1,7
1,8
1,9
2
2,1
Past Future Past Neutral Future
Study 3,2 Study 3,3
R ig
ht E
ar D
is am
bi gu
at io
ns
79
These results support the expected judgment bias asymmetry for past and future words. Moreover, as expected, there was an overall right- bias in loudness judgments. One-sample t-tests revealed that the mean of the past words was significantly to the right of the midpoint, t(7) = 3.72, p = .01, prep = .97, Cohen’s d = 1.31, as was the mean for the future words, t(7) = 5.71, p = .001, prep = .99, Cohen’s d = 2.01.
The cross-modal overlap was tested by comparing the relative visual positioning scores obtained for the 16 items in Study 3.1 with the average auditory right channel loudness judgments for the 16 items in this study. The data points for this analysis were the average values for each of the time-related stimuli in Study 3.1 and 3.2. A regression analysis revealed that the spatial ordering of past and future-related words on the horizontal position in Study 3.1 overlapped significantly with their relative right channel judgment bias in Study 3.2, β = .55, r2 = .30; t = 2.46, p =.03, prep = .99.
Discussion
These results reveal that time is represented in left-right auditory space. Furthermore, these findings confirm the hypothesis that there is a systematic overlap between the representation of time in the visual and auditory modality. The right channel judgments for past and future- related words in the auditory loudness judgments task reproduced the relative positions that had been obtained in the visual positioning task. It is noteworthy that whereas participants were explicitly asked to place time- related words on a horizontal line in Study 3.1, the loudness judgments task only required participants to decide in which ear they thought the word was presented louder, and the time related words were not classified on a temporal dimension.
Study 3.3
The third study was a replication of Study 3.2, but with the following
80
modifications. First, 8 neutral words were added that were unrelated to time (e.g., table). The neutral words provided a baseline and were expected to fall between the average past and future loudness judgments. Second, in order to control for a possible unequal hearing strength between the left and right ears, the loudness was calibrated for each individual separately. Finally, the response key assignment was changed from a left-right alignment (keys “A” and “L”), which may activate the left- right representation of time in space (Torralbo et al., 2006) to an up-down alignment (the “T” and “V” keys). The hypotheses remained the same.
Method
Participants. Thirty-two students (22 females, mean age 22) participated in this experiment.
Stimuli. The stimuli were identical to Study 3.1, except for the inclusion of eight temporally neutral words (identical, closet, even, sandal, paper, glass, table, and triangle).
Procedure. The study started with a hearing test to reduce individual differences in right-left hearing imbalances. Participants received a 440 Hz tone that was less loud on one of the channels of the headphones. Their task was to adjust the sound until the tone was equally loud on both channels. This was repeated until participants reported that the tone was perceived as equally loud in both ears. The auditory stimuli were presented with the corresponding individual adjustment.
The auditory loudness judgments task was identical to that in Study 3.2, except that the response keys to indicate the left or right channel were vertically aligned (the “T” and “V” keys on the keyboard), and key assignment was counterbalanced between participants. The classification task consisted of three blocks, where all stimuli where presented once with the volume equally loud in both ears, and once while the sound was louder in one ear than the other. The presentation of the words within each block was completely randomized. Thus, the total number of trials was 144, with 72 experimental trials (24 past, 24 neutral, and 24 future
81
words) and 72 filler trials to create a believable auditory classification task with real differences in the loudness between the ears. In each block, the filler trials were randomly distributed over 10 differences in volume between the ears, being 10%, 20%, 30%, 40%, or 50% louder in the left or the right ear, with each position occurring twice, except for the 40% and 50% volume settings, which occurred 3 times.
Results
Analyses by participants. The average number of times the 72 perfectly balanced past, future and temporally neutral words were judged to be louder in the right ear was calculated. As predicted, these judgments differed based on the temporal meaning of the stimuli. A repeated measures analysis on the average right ear loudness judgments for the 24 past, neutral and future words revealed an effect of words meaning F(2, 62) = 2.77, p = .07, ηp² = .08. A linear trend best fitted the data, with the number of right judgments being highest for future words (M = 13.72, SD = 5.43) and lowest for past related words (M = 12.41, SD = 4.82), with the neutral words falling in the middle (M = 13.38, SD = 4.62), F(2,62) = 4.87, p = .035, ηp² = .14.
Analyses by items. The average number of times the responses for the 8 past, 8 neutral, and 8 future-related stimuli in the critical trials were judged to be louder in the right ear were calculated similar to Study 3.2. Response key assignment did not affect the results (all prep < .58). A repeated measures analysis revealed that the predicted linear trend of a graded right-ear bias for future, to neutral to past words was significant, F(1, 21) = 4.72; p = .04, prep = .93, ηp² = .26, see Figure 1, right pane. Future related words were judged to be louder in the right ear most often (M = 1.71, SD = 0.14), with neutral words occupying an intermediary position (M = 1.67, SD = 0.16) followed by past related words (M = 1.55, SD = 0.16). One-sample t-tests revealed that the right bias was significant in the case of future words, t(7) = 4.34, p = .003, prep = .98, Cohen’s d = 1.55, and neutral words, t(7) = 3.12; p = .02, prep = .95, Cohen’s d = 1.10, but not
82
in the case of past words, t(7) = .92, p = .39, prep = .73, Cohen’s d = 0.33. As in Study 3.2, a regression analysis testing the cross-modal
overlap revealed that the horizontal position of the time-related words on the visual positioning task (Study 3.1) overlapped significantly with the auditory judgments of the very same stimuli in the loudness judgments task in this study, β = .55, r2 = .30, t = 2.59, p = .03, prep = .93.
Discussion
The results of Study 3.3 reproduce the cross-modal findings of Study 3.2. As predicted, participants chose the right channel more often for future words than for past related words when the words were presented equally loud in both channels. Time is represented in left-to-right auditory space. This conclusion is strengthened in Study 3.3 by controlling for individual differences in left-right hearing, counterbalancing vertically aligned response keys, and adding a set of neutral words, which occupied a middle position between past and future-related words in the average number of right channel judgments. More importantly, the third study replicated the overlap between the auditory and visual mapping of time in space. This significant cross-modal overlap underlines the argument under examination, namely that time is mapped in the same relative order in the visual as well as the auditory modality.
General Conclusions
The unique contribution of these studies is the demonstration that the abstract concept of time is structured in auditory space and the remarkable overlap between the visual and auditory representation of time. The data analyses of Study 3.2 and 3.3 revealed that these findings can be generalized not only across participants, but also across items, whereby participants and words serve as random factors respectively.
Although the current studies are the first to show cross-modal overlap between the visual and auditory representation of the abstract
83
dimension of time, previous studies have revealed cross-modal equivalences between many concrete dimensions. For example, in studies using explicit cross-modality matching tasks, participants match bright lights with loud sounds, and dim lights with soft sounds (Marks, 1978). As Rudel and Teuber (1964, p. 6) summarize: “There is some common aspect of perceptual activity which permits one to utilize information from within a sensory channel or from several channels in such a way that invariant properties of objects are extracted.” Earlier findings have indicated a cross-modal overlap in meaning between auditory stimuli and visual representations of these stimuli (Karwoski, Odbert, & Osgood, 1942). For example, participants in this study showed a general agreement to represent loudness by heavier, darker or thicker lines. Osgood (1952) argues that a similar process underlies metaphor in language as visual- auditory synesthesia, namely the parallel alignment of two or more dimensions of experience, relating the corresponding extremes of these dimensions.
Marks (1978), on the other hand, has argued that the equivalences in cross-modal perception reflect intrinsic properties of the sensory systems and depend upon the neurophysiological mechanisms that code perceptual qualities. In a set of reaction time studies (Marks, 1987), participants received compound (sound-light) stimuli, and were asked to categorize the stimuli based on one dimension (e.g., sound) while ignoring the other component (e.g., light) by pressing one of two keys. The results revealed that responses were faster (and the accuracy higher) on those trials where the critical stimulus (e.g., high pitch) is accompanied by “matching” (e.g., bright light) rather than “mismatching” (e.g., dim light) stimuli from another modality. Although these results do not exclude a semantic mediation of the findings, Marks (1987, p. 393) argues they ‘do point to a possible non-semantic basis’.
How cross-modal structural alignment occurs remains an open question, but the finding itself has been observed in many studies and over a multitude of concrete domains (Kornblum, Hasbroucq, & Osman, 1990). The current studies take cross-modal matching to an altogether new
84
level, by showing that the left-right spatial mapping of time in the visual modality is structurally aligned with the left-right auditory mapping of time. Apparently, people are able to utilize concrete information in the visual and the auditory dimensions to represent a common aspect of the abstract concept of time.
The findings in the current chapter introduce a novel multimodal perspective into the embodied grounding of time. Loudness judgments are likely to prove to be a fertile method in investigations across a variety of different abstract constructs that are grounded spatially such as valence (e.g., Meier & Robinson, 2004), or power (e.g., Schubert, 2005). The main contribution of the current set of studies is however the demonstration of a structural alignment between both an auditory and visual source in how the abstract concept of time is grounded. These findings open up a more informed approach to the multimodal grounding of abstract concepts.
Chapter 4
But for the Bad,
There Would not be Good
This chapter is based on: Lakens, D., Semin, G. R., & Foroni, F. (submitted). But for the bad, there would not be good: Grounding valence in brightness through shared relational structures.
87
Mr. Pink: Why can’t we pick out our own colors?
Joe: I tried that once, it don’t work. You get four guys fighting over who’s gonna be Mr. Black.
Reservoir Dogs, Tarantino (1992).
In Plato’s chariot allegory of the journey of the soul, the charioteer (or reason) has to control two horses: one is white, representing the moral impulse, and the other is black, representing the immoral passions. The opposition between black and white is used pervasively to represent the opposition between the abstract concepts of bad and good. According to Lakoff & Johnson’s Conceptual Metaphor Theory (CMT; 1980, 1999) people use metaphors to ground the meaning of abstract concepts (e.g., positivity and negativity) in concrete experiences (e.g., light and dark). CMT and other grounded approaches to cognition (e.g., Barsalou, 1999; Gallese & Lakoff, 2005; Glenberg & Kaschak, 2002; Jeannerod, 2001; Pulvermüller, 1999; Zwaan, 2004) are a response to the dominant view on conceptual processing in psychology that perception and cognition are functionally independent (e.g., Fodor, 1975; Pylyshyn, 1984; Jackendoff, 2002). Instead, grounded approaches to cognition propose that conceptual processing is intimately related with the activation of sensorimotor information. Several studies have found support for this assumption for both concrete and abstract concepts (for reviews, see Crawford, 2009; Meteyard & Vigliocco, 2008). Recently, researchers have stressed the necessity to go beyond studies demonstrating the influence of perceptual information on conceptual processing and to investigate the processes that underlie this influence (e.g., Barsalou, 2008a; Zwaan, 2009). In the current studies, I compare two process explanations in the literature against each other with the aim to gain a better understanding of how metaphoric representations relate perceptual information to abstract
88
concepts. Meier, Robinson, and Clore (2004) set out to test the hypothesis
that people rely on perceptual information when they process abstract concepts by investigating whether the speed with which positive and negative words were categorized would be influenced by their brightness. Given that good and bad are metaphorically represented in light and dark (Adams & Osgood, 1973), positive words in white and negative words in black were expected to be categorized more efficiently than positive words written in black and negative words written in white. These predictions were confirmed, supporting the assumption that conceptual processing is influenced by seemingly unrelated perceptual information. Meier and colleagues (2004, p. 85) conclude that their results point to ‘the obligatory nature of affective inferences based on stimulus brightness’. Black and white stimuli are thus hypothesized to automatically activate negativity and positivity, respectively.
Other researchers have argued that the function of concrete source domains is to provide structure to abstract target domains (cf. Boroditsky, 2000; Gentner & Bowdle, 2008; Murphy, 1996)1. Through metaphoric representation, the relationship between concepts in the abstract domain can be structured by importing the relational structure from concrete domains. According to this view on metaphoric representations, the valence-brightness metaphor does not make us believe positivity is white, but the metaphor uses the perceptual opposition between black and white to structure the conceptual opposition between good and bad. Thus, the mapping of the abstract concepts ‘bad and good’ on the concrete experiences of ‘dark and light’ represents a shared relational structure, and not necessarily a direct and automatic association between darkness and negativity or brightness and positivity. In a set of six studies I pit a shared relational structure view against an automatic association account, with the objective to experimentally investigate how the brightness-valence metaphor influences conceptual processing.
In the remainder of this chapter, the assumptions underlying the automatic association account and the shared relational structure view
89
will be discussed in greater detail. Subsequently, studies on the explicit evaluation of the colors black and white will be summarized, which together with data from pilot studies reveal that while black is generally evaluated negatively, white is judged to be a neutral color. If white is a neutral color, then it is unlikely that white will automatically activate positivity. Instead, the positivity of white seems to emerge only in opposition to the negativity of the color black. Finally, a series of studies will be presented that were designed to test whether shared relational structures between the perceptual black-white opposition and the conceptual bad-good opposition are an essential pre-requisite for the metaphoric mapping of the valence dimension on the brightness dimension.
One-to-One Grounding
The question how the abstract domain of valence is grounded has been the subject of considerable research over the last few years (for a review, see Crawford, 2009). Meier, Robinson and Clore (2004) investigated whether the processing of positive and negative concepts is built on perceptual experiences. Focusing on the brightness-valence metaphor, they set out to test whether task-irrelevant perceptual characteristics of stimuli would automatically influence speeded evaluations of stimulus words. Participants were asked to evaluate positive and negative words, written in either a white or a black font. Negative words presented in black and positive words presented in white (so-called metaphor congruent mappings) were categorized faster and more accurately compared to positive words presented in black and negative words presented in white (or metaphor incongruent mappings). The authors conclude such metaphor congruency effects point ‘to the obligatory nature of affective inferences based on stimulus brightness’ (Meier et al., p. 85).
Meier and colleagues (2004) propose that metaphor congruency effects are the result of Stroop-like interference (for a similar reasoning, see Meier & Robinson, 2004, Schubert, 2005, Sherman & Clore, 2009). Due to the theorized obligatory nature of perceptual-conceptual links
90
(Lakoff & Johnson, 1999), the color white is hypothesized to automatically active positive valence. During a speeded evaluation task, the automatic activation of positivity when presented with a bright stimulus should facilitate positive evaluations, but interfere with negative evaluations. As Meier and colleagues explain (2004, p. 86): “a negative word presented in white would give rise to two response tendencies, one to respond ‘‘negative’’ (on the basis of stimulus valence) and one to respond ‘‘positive’’ (on the basis of stimulus color).”
According to the one-to-one grounding account, relationships between concrete sources (e.g., white) and abstract targets (e.g., positivity) are fixed neural connections, resulting from the co-activation of the two domains (Lakoff, 2008). The translation of perceptual information into conceptual information is argued to be “used constantly and automatically, with neither effort nor awareness’’ (Lakoff, 1993, p. 227– 228), and is thus assumed to be obligatory at the stage of word encoding (Meier & Robinson, 2004). Thus, perceiving the color white on its own is sufficient to activate its associated valence (e.g., positivity). I call this one- to-one grounding. Following the predictions from a one-to-one grounding perspective, the shared relational structure of the perceptual opposition between white and black and the conceptual opposition between good and bad is irrelevant for the brightness-valance mapping to emerge.
Shared Relational Structures
An alternative approach to the metaphoric mapping of valence and brightness presupposes that the main function of metaphors is to import a relational structure from concrete dimensions into an abstract domain (Murphy, 1996; Boroditsky, 2000; Gattis, 2001; Gentner & Bowdle, 2008). It has been noted that many image schema’s upon which conceptual metaphors are built (e.g., up-down, balance-imbalance, front-back, in- out) consist of polar opposites (Grady, 1997; Hampe, 2005; Krzeszowski, 1997). Therefore, the relational structure that is imported from concrete domains is often that of an opposition. With respect to the brightness-
91
valence metaphor, the opposition in the concrete brightness dimension is deployed to express the opposition in the abstract valence dimension. The opposition between white and black is imported in the valence domain through the brightness-valence metaphor to structure the relationship between good and bad.
If this shared relational structure view is correct, then the activation of the polar oppositions in the concrete dimension and the abstract domain is critical in grounding valence in brightness. Therefore, the main difference between the one-to-one grounding account and the shared relational structure view is that according to the latter the mere perceptual experience of ‘white’ will not automatically activate the meaning ‘positive’. Instead, the relational structure of the perceptual opposition in the brightness dimension (black vs. white) grounds the opposition in the abstract valence dimension (negative vs. positive). By implication, if a relational structure is not activated (e.g., only one endpoint of the brightness continuum is available), then the metaphoric mapping between brightness and valence is unlikely to emerge. Since the studies by Meier and colleagues (2004) and Sherman & Clore (2009) rely on within participant manipulation of the brightness and valence dimensions (each participants has to categorize both positive and negative words, presented both in a black as in a white font), these studies inevitably activate the good-bad and light-dark oppositions. Thus, the question whether black or white stimuli in isolation activate negativity and positivity remains open, making it impossible to distinguish between the one-to-one grounding account and the shared relational structures view.
Explicit Evaluations of Black and White
Color research investigating the explicit evaluation of black and white generally shows that people rate black as negative, while white is rated neutrally (e.g., Burkitt, Barrett & Davis, 2003; Götz & Götz, 1974; McManus, Jones, & Cotrell, 1981) or even slightly disliked, albeit to a lesser extent than the color black (Meerum Terwogt & Hoeksma, 1995).
92
The notion that white is a positive color comes not from color research, but from the highly influential cross-cultural studies on the measurement of meaning by Osgood and colleagues (Adams & Osgood, 1973; Osgood, 1960; Osgood, Suci, & Tanenbaum, 1957). However, it is important to note that in these studies the meaning of concepts is defined as a location in a multi-dimensional semantic space consisting of pairs of polar (opposite-in-meaning) adjectives. Therefore, the semantic relations in these studies should be interpreted as “the parallel alignment of two or more dimensions of experience, definable verbally by pairs of polar adjectives, with translations occurring between equivalent portions of the continua.” (Osgood et al., 1957, p. 23). In other words, white and black are functionally treated as opposites in these cross-cultural studies. As Osgood (1960, p. 155) summarizes: “If one visual alternative is chosen as appropriate to HAPPY (say, the light circle), we can predict that the other alternative will be chosen as appropriate to its opposite, SAD (the dark circle).” Also note that the Dutch sample in the cross-cultural study by Adams & Osgood (1973), rated white neutrally, as did many other samples from western countriesPsychological research suggests a clear anchoring of black as ‘bad’. For instance, referees have been shown to judge sports teams wearing black outfits as more aggressive, and consequently penalize these teams more often (Frank & Gilovich, 1988). In relation to the neutrality of white discussed above, it is noteworthy that no special positivity benefit was observed for teams wearing light outfits. The negativity of black and darkness has been argued to results from the fact that humans are diurnal animals (Williams & Morland, 1976). This evolutionary explanation is supported by research that shows people have a general tendency to feel less at ease in the dark, as signified by an increase in the human startle reflex compared to when the startle reflex is measured under normal lighting circumstances (Grillon, Pellowski, Merikangas, & Davis, 1997). Concluding, there is a strong association between darkness and negativity, but no strong clear association between white and positivity. The association between white and positivity does emerge when white is evaluated in opposition to black (Adams & Osgood,
93
1973; Osgood, 1960). Together, this pattern of evaluations seems to highlight
the importance of shared relational structures for the metaphoric representation of good and bad in light and dark, and to contradict the one-to-one grounding account, which assumes that the brightness of a stimulus automatically activates positive valence. The empirical studies in this chapter were developed with the aim of testing the one-to-one grounding account against the shared relational structure approach, in order to gain a better understanding of the process underlying the grounding of valence in brightness. If the shared relational structure between two dimensions is a prerequisite for metaphor congruency effects to occur, then breaking up the opposition in the valence (good vs. bad) and brightness (light vs. dark) dimensions by manipulating these polar endpoints between (instead of within) participants should influence whether the brightness-valance mapping emerges, especially for the white-positivity association.
Overview of the studies
According to the shared relational structure view the metaphoric representation of good and bad in light and dark hinges upon the opposition between the endpoints of the valence and brightness dimensions. According to the one-to-one grounding account, white should always activate associations with positivity, and black should always activate associations with negativity. In the remainder of this chapter, I will present data in line with the hypothesis that white in isolation is evaluated neutrally, and that the positivity of white emerges in conceptual or perceptual opposition with the negativity of the color black. The color black is negative in isolation, and is argued to be the relational tie between the valence and brightness dimensions. I will first discuss the results from three pilot studies confirming the negativity of black, and the neutrality of white, in line with the findings in the literature discussed previously.
Subsequently I introduce a new paradigm which does not rely on
94
response interference and allows us to manipulate dimensions between participants. As noted by Meier and Robinson (2004) choice reaction time paradigms based on speeded bimanual categorizations is a way to address the automaticity of the brightness-valence mapping. However, these bimanual categorization paradigms necessarily manipulate both the valence and the brightness of the stimuli within participants, thereby inherently activating the good-bad and white-black oppositions. Such response interference paradigms do not allow a direct comparison of within versus between participant manipulation of the white-black and good-bad oppositions. Furthermore, recent articles have revealed that response interference in paradigms with stimuli and responses of opposite evaluations are often not indicative of the intrinsic valence of a stimulus (e.g., the positivity of white) but the result of structural factors such as polarity correspondence (Proctor & Cho, 2006) or the top-down specification of stimulus-response codes (Eder & Rothermund, 2008).
In first four studies, I employed a novel Chinese Translation Paradigm, where participants are asked to indicate whether Chinese ideographs are the correct translation for a word presented on the screen. The idea of this paradigm is that irrelevant perceptual characteristics (in the current set of studies, their brightness, which was either white or black) will influence translation judgments for positive and negative words because the perceptual attributes are either congruent or incongruent with the brightness-valence metaphor. Given that the participants do not know the meaning of Chinese ideographs, this paradigm investigates judgments under uncertainty (cf. Crawford, Margolies, Drake, & Murphy, 2006, Meier, Hauser, Robinson, Kelland, Friesen, & Schjeldahl, 2007). As such, it does not allow us to test whether the brightness-valence metaphor is automatic. However, my main interest lies in determining whether the valence-brightness metaphor is conditional upon the presence of the shared structural relationship between the opposition in the brightness dimension and the opposition in the valence dimension.
By manipulating the brightness and valence dimensions within or between participants, this paradigm allows us to compare whether black
95
or white ideographs will be chosen above guessing average to represent the meaning of negative or positive words as a function of whether the opposition between black and white or positivity and negativity is present in the translation task or not. If the one-to-one grounding account is correct, participants should always judge white ideographs to correctly translate positive words above guessing average. However, if the shared relational structure of the opposition between black and white and the opposition between negativity and positivity is an essential pre-requisite, then positive words should be translated above guessing average by white ideographs only if the dimensions are manipulated within participants (activating the perceptual and conceptual oppositions), but not when the dimensions are manipulated between participants. Based on the fact that black is a negative color independent of the context, I assume that participants will always judge the black ideograph to be a correct translation of negative words above guessing average.
The final two studies aimed to provide direct support for the hypothesis that the positivity of white emerges in opposition to the negativity of black. By presenting only positive words and white ideographs these studies tested whether participants judged the white ideographs to correctly translate the positive stimulus words above guessing average in perceptual opposition to the negativity of black (by presenting the white ideographs on a gray vs. black background), or in conceptual opposition to the negativity of black (by priming participants with conceptual similarity or conceptual opposition). Finding support for the hypothesis that the positivity of white emerges in opposition to the negativity of black shows that certain metaphors (e.g., white is good) depend on the activation of the abstract and perceptual dimensions. This would support the assumption that the metaphors provide structure, but not necessarily meaning, to abstract concepts.
Color Evaluation Pilots
To complement the previously summarized literature on explicit color
96
ratings for black and white, I performed three pilot studies in empirical support for the neutral evaluation of white and the negative evaluation of black.
In a first pilot, I collected color ratings for 12 colors by asking 31 participants to indicate how positive each randomly presented color was on a scale from 1 to 9. Black was the most negative color (M = 4.13, SD = 2.68), white was rated around the midpoint of the scale (M = 5.13, SD = 2.31) with blue being the most positive color (M = 7.23, SD = 1.45). This pattern of color evaluations is identical to that found in previous studies (e.g., Eysenck, 1941; Frank, 1976; Meerum Terwocht & Hoeksma, 1995) where blue is the most preferred color (Crozier, 1999). In a second pilot, 90 participants evaluated only one color on a scale from -4 to + 4. Half of the participants reported the valence of black, and evaluated it as negative (M = -2.22, SD = 1.66) and the other half of the participants reported the valence of white, and evaluated it as neutral (M = 0.07, SD = 1.67).
A final pilot was performed to provide more conclusive support for the neutrality of the color white. If white has a neutral valence, then participants’ affective state should influence how this neutral stimulus is evaluated (Murphy & Zajonc, 1993; Cacioppo, Priester, & Berntson, 1993). Therefore, it was predicted that participants’ self-reported mood would correlate with their evaluations of the color white, but not with their evaluations of the color black, which is negatively valenced. Two colors were evaluated on a scale from -4 to +4 by 66 participants. The results revealed that white was evaluated around the midpoint of the scale (M = 0.53, SD = 1.89) whereas black was evaluated negatively (M = -2.52, SD = 2.12). More importantly, participants’ self-reported mood significantly correlated with participants’ evaluation of the color white (r = .30, p = .01) but not with their evaluation of the color black (r = -0.01, p = .95). Together, these results provide support for the assumption that white is a neutral color, and black is negative.
97
Pitting one-to-one grounding against shared relational structures
The current studies were developed with the aim to compare the shared relational structure view and the one-to-one grounding account by manipulating the valence and brightness dimension within or between participants in the next four studies. In Study 4.1 I implement the Chinese translation task in a conceptual replication of the studies by Meier and colleagues (2004) by manipulating both dimensions within participants. Participants are presented with positive and negative words and black and white Chinese ideographs. Positive words should be translated above guessing average by white ideographs, and negative words should be translated above guessing average by black ideographs. This outcome would be expected from both a one-to-one grounding account as well as from the shared relational structure view. In Study 4.2, brightness is manipulated within participants, but valence is manipulated between participants. While a one-to-one grounding account of valence in brightness would predict similar results as in Study 4.1, the shared relational structures view predict positive words will no longer be translated by white ideographs above guessing average. Because the polar opposition in the perceptual and conceptual dimensions is not activated, the positivity of white should not emerge.
In Study 4.3 brightness is manipulated between participants while valence is manipulated within participants. In order to manipulate brightness between participants, the Chinese translation paradigm had to be modified. Only one Chinese ideograph was presented, and participants had to judge whether the Chinese ideograph correctly translated the stimulus words by clicking on a ‘yes’ or ‘no’ button. A one-to-one grounding account would predict a replication of Study 4.1, whereas the shared relational structures view predicts no differences from guessing average for the translation judgments in the white ideograph condition, according to a similar reasoning as in Study 4.2. In Study 4.4 both brightness and valence are manipulated between participants. The
98
one-to-one grounding account would again predict an identical pattern of results as in Study 4.1. The shared relational structures view predicts no differences from guessing average when translating positive words by white ideographs. However, due to the intrinsic negativity of black, negative words are expected to be translated by black ideographs above guessing average in all four studies.
The final two studies aimed to provide direct support for the hypothesis that the positivity of white emerges in opposition to the negativity of black. Study 4.5 tested whether the positivity of white would emerge by perceptually priming the opposition between black and white. Participants were presented with positive words and white ideographs, either presented against a grey background (identical to Study 4.3) or against a black background. According to the shared relational structures view, presenting white ideographs against a black background (activating negativity) should induce the opposition between black and white in terms of the negativity of the background and the positivity of the words to be translated, and the positivity of white should emerge. The one-to- one grounding account would not predict differences in the translation judgments between the two conditions.
Study 4.6, goes one step further by conceptually priming participants to process the positive words and white ideographs in opposition to negativity and black. The conceptual opposition prime consists of an antonym generation task, whereas participants in the control condition generate synonyms. Since generating antonyms activates an opposition mindset in participants, I predicted the positive stimulus words and white ideographs would be processed more in relation to their opposites, even though these are not present in the task, compared to participants who performed a synonym generation task. As in Study 4.5, the hypothesis in Study 4.6 follows only from the shared relational structures view, and cannot be explained by a one-to-one grounding account.
99
Study 4.1: Valence and Brightness Within
To establish a paradigmatic baseline in the Chinese translation paradigm, I aimed to replicate the pattern of results as observed by Meier et al., (2004) by manipulating valence (positive vs. negative) and brightness (white vs. black) in a within participants design. Positive words were expected to be translated above chance by white ideographs, and negative words were expected to be translated above chance by black ideographs.
Method
Participants. Thirty-four paid voluntary students (23 females, mean age 20) at a Dutch university took part in this study for a monetary compensation.
Procedure. All instructions were presented in dark-grey letters on a perfectly gray background (RGB: 128,128,128). Participants were informed that recent studies had revealed that people who did not speak Hebrew nevertheless had a 60% chance of correctly translating Hebrew words. The current study would ostensibly investigate if people were similarly able to correctly translate Dutch words into Chinese ideographs above chance. Before starting the translation task, participants saw a Chinese ideograph (not used in the translation task) and were asked if they could read Chinese ideographs similar to the displayed example: if they indicated yes, participants were removed from the analysis. This general instruction and procedure was identical across the next six studies.
Participants received 12 stimulus words (six positive, six negative, see Appendix 2) presented in random order on a perfect grey background. Underneath each word two Chinese ideographs were displayed, one white, one black. Participants were asked to judge which of the two ideographs they thought correctly represented the meaning of the Dutch stimulus word. Twenty-four ideographs were randomly displayed in either white or a black2. The location of the black and white ideographs (left or right) was randomized between participants. There was no time limit for the
100
translation judgments. After performing the 12 translation judgments, participants were asked to rate the valence of the 12 words on a scale from 1 (very negative) to 7 (very positive).
Results
Manipulation check. Positive words were judged as more positive (M = 6.51) than negative words (M = 1.80), and a paired-samples t-test indicated this difference was significant, t(32) = 29,06, p < .001.
Translation judgments. Since the choices for white or black ideographs are mutually dependent, I only calculated the average number of times positive and negative words were translated by a white ideograph. Participants translated the six positive words above guessing average by white ideographs (61.67%, M = 3.70, SD = 1.22), as indicated by a test against chance, t(33) = 3.24, p < .01, Cohen’s d = .57. The six negative words were translates below guessing average by white ideographs, (40.34%, M = 2.42, SD = 1.08), t(33) = -3.03, p < .01, Cohen’s d = .54.
Discussion
Even though the brightness of the ideographs was irrelevant for the translation task, differences in the brightness of the ideographs influenced the likelihood with which they were judged to correctly translate positive or negative words. As such, this study replicates previous findings (Meier et al., 2004; Sherman & Clore, 2009) in a paradigm based on performing judgments under uncertainty.
Studies 2 to 4 were performed to investigate whether white ideographs would still be judged to correctly represent the meaning of positive words above guessing average when the polar oppositions in the valence and brightness dimensions were no longer present in the translation task. Either valence (Study 4.2), brightness (Study 4.3), or valence and brightness (Study 4.4) were manipulated between participants. If white activates positivity, then these studies should show an identical pattern of
101
results as Study 4.1. If the predictions of the shared relational structures view hold then negative words should be translated above chance level by black ideographs, but positive words should no longer be translated above guessing average by white ideographs.
Study 4.2: Valence Between, Brightness Within
Participants and procedure. Fifty-three students (31 females, mean age 20) at a Dutch university participated and were randomly assigned to the positive or negative word condition. The procedure was identical to Study 4.1, with the exception that valence of the 12 words were either all positive or all negative, depending on condition (see Appendix 2).
Results
Manipulation check. Positive words were judged as more positive (M = 6.33) than negative words (M = 2.08) as indicated by an independent- samples t-test, t(51) = 27.48, p < .001.
Translation judgments. The average number of times participants in the negative word condition chose the white ideograph to translate the stimulus word was significantly lower than guessing average, (39.17%, M = 4.70, SD = 2.84), t(26) = 2.37, p = .03, Cohen’s d = .46, indicating they preferred the black ideograph to translate negative words. In the positive word condition the likelihood with which participants translated positive words by white ideographs did not differ from guessing average, (51.92%, M = 6.23, SD = 1.89), t(25) = 0.63, p = .54, Cohen’s d = .01.
Discussion
If the valence dimension is manipulated between participants, white ideographs are no longer chosen to translate positive words above guessing average. Black ideographs are still seen as correctly translating negative words above chance. These results are in line with the shared relational
102
structures view, according to which the presence of the opposition in the valence and brightness dimensions is an essential pre-requisite for the positivity of white to emerge.
Study 4.3: Valence Within, Brightness Between
Where valence was manipulated between participants in Study 4.2, the current study compared the one-to-one grounding account with the shared relational structures view by manipulating brightness between participants. The translation task was adapted to enable the manipulation of ideograph brightness between participants. Instead of choosing between two Chinese ideographs (one black, one white), only one ideograph was displayed underneath each stimulus word, and participants were asked to judge whether the ideograph correctly translated the stimulus word or not by clicking YES or NO buttons. I predicted participants would judge black ideographs correctly translated negative words above chance. Based on the fact that the opposition in the brightness dimension is not present in the translation task, no differences from guessing average were expected in the white ideograph condition.
Method
Participants and procedure. Thirty-nine students (30 females, mean age 21) at a Dutch university participated in return for a monetary reward, and were randomly assigned to the white or black ideograph condition. Participants were asked to indicate if they felt the displayed Chinese ideograph was the correct translation of the stimulus word or not by clicking on the YES or the NO button on the left and right side of the ideograph. The location of the YES and NO buttons on the screen was randomized in each of the 12 trials. The same stimulus words were used as in Study 4.1.
103
Results
The average number of YES responses was calculated for positive and negative words in the black and white ideograph conditions. Since responses were no longer mutually exclusive as in Studies 4.1 and 4.2, where choosing the white ideograph meant participants did not choose the black ideograph, all four averages were tested against chance. In the white ideograph condition, participants’ average number of YES judgments did not differ from guessing average for positive (47.34%, M = 2.84, SD = 1.74) or negative (42.17%, M = 2.53, SD = 1.39) stimulus words, t(18) = 0.40, p = .70, Cohen’s d = .09 and t(18) = 1.49, p = .16, Cohen’s d = .34, respectively. In the black ideograph condition both averages for the positive words (39.17%, M = 2.35, SD = 0.99) and the negative (63.34%, M = 3.80, SD = 1.24) stimulus words deviated significantly from guessing average in the expected directions, t(19) = 2.94, p = .01, Cohen’s d = .66 and t(19) = 2.89, p = .01, Cohen’s d = .65, respectively.
Discussion
White ideographs did not influence translation judgments for positive or negative words. Black ideographs presented in isolation did influence translation judgments for positive and negative words. While the association between black and negativity was clearly hypothesized, the non-association between negativity and the opposite color (‘negative is not white’) is not a necessity, but is in line with the expectations. These findings are in line with the importance of the activation of shared relational structures in the brightness and valence domains, which predicts the positivity of white emerges only in opposition with the negativity of black. The differences between Study 4.3 and Study 4.1 are not a-priori predicted by a one-to-one grounding account.
104
Study 4.4: Valence and Brightness Between
I proceeded with a final comparison of the predictions made by the shared relational structures view and the one-to-one grounding account by manipulating both the valence and brightness dimensions between participants. Only translation judgments for negative words by black ideographs were expected to show a deviation from guessing average.
Method
Participants and procedure. A total of 81 students (54 females, mean age 20) at a Dutch university participated and were randomly assigned to one of the four conditions of the 2 (word valence: positive vs. negative) x 2 (ideograph brightness: white vs. black) between participants design. The procedure was identical to Study 4.3 with the exception that the valence of the 12 stimulus words was manipulated between participants as in Study 4.2.
Results
The average number of YES judgments in the negative word condition deviated significantly from chance, (57.50%, M = 6.90, SD = 1.14), t(20) = 3,67, p < .01, Cohen’s d =.80. No significant deviations from chance were observed in the positive word white ideograph condition (M = 6.05), t(20) = .15, p = .89, Cohen’s d = .03, negative word white ideograph condition, (55.67%, M = 6.68, SD = 1.70), t(18) = 1.75, p = .10, Cohen’s d = .40, and positive word black ideograph condition (55.67%, M = 6.45, SD = 1.91), t(19) = 1.06, p = .30, Cohen’s d = .24.
Discussion
The results confirm that even outside of any perceptual or conceptual opposition, black ideographs are still judged to correctly translate the
105
meaning of negative words above guessing average. This result is also important from a methodological perspective. The Chinese translation paradigm can be used to show the influence of perceptual characteristics of the ideographs on the likelihood with which they are seen to correctly translate valenced words, even without activating opposite endpoints of the conceptual or perceptual dimensions. This aspect of the Chinese translation paradigm will be built upon in the following two studies.
Study 4.5: Perceptual Opposition
After conceptually replicating earlier findings by Meier et al., (2004) in Study 4.1, the subsequent three studies supported the shared relational structures view by showing no deviations from guessing average for the average number of translations of positive words by white ideographs. In the current study, I aimed to activate the positivity of white not by manipulating both endpoints of the valence and brightness dimensions within participants as in Study 4.1, but by presenting positive words and white ideographs in perceptual opposition to the negativity of black. As the pilot studies have shown, a black background screen is evaluated negatively. Presenting positive words and white ideographs on a black background (which was negatively evaluated in the pilot studies reported earlier) should activate the opposition in the valence and brightness dimensions. Thus, translation judgments for positive words by white ideographs presented on a grey background should not differ from chance (as in Study 4.4), but when these white ideographs are presented on a black background, positive words would be judged to be correctly translated by white ideographs above guessing average.
Method
Participants and procedure. Forty-eight students (32 females, mean age 21) at a Dutch university participated in this experiment for a monetary reward and were randomly assigned to either the black or
106
grey background condition. The procedure was identical to the positive words white ideographs condition in Study 4.4. However, whereas for half the participants the white ideographs were presented on a perfect grey background (identical to studies 4.1 to 4.4), the white ideographs were presented on a black background for the other half of the participants.
Results
The average number of times participants answered YES to the question whether the white ideograph correctly translated the positive words was calculated. Whereas this average (49.33%, M = 5.92, SD = 1.19) did not differ from chance in the grey background condition, t(24) = -0.34, p = .74, Cohen’s d = .07, the average number of Yes judgments was higher than average when participants were asked to indicate whether the white ideograph correctly translated the positive word when it was presented against a black background (54.33%, M = 6.52, SD = 0.90), t(22) = 2.79, p = .01, Cohen’s d = .58.
Discussion
Presenting white ideographs and positive words against a black background increases the likelihood that participants will judge the ideograph to correctly translate the positive word. As predicted from the shared relational structures view, the positivity of white emerges after introducing the perceptual opposition between the negativity of black and the positive words and white ideographs. These effects cannot be explained by the one-to-one grounding account.
Study 4.6: Conceptual Opposition
After activating the shared relational structures between the brightness and valence dimensions through perceptual opposition in Study 4.5, the current study goes one step further by conceptually activating the
107
black-white and bad-good oppositions. I tested whether participants in a conceptual opposition mindset (activated by generating antonyms before performing the Chinese translation task) would process white and positivity in relation to their opposites, thus activating the shared relational structures between the valence and brightness dimensions, and allowing the positivity of white to emerge. The other half of the participants performed a synonym generation task (activating a similarity mindset) before performing the Chinese Translation Task. All participants judged whether white ideographs presented on a grey background correctly translated positive words. I expected participants to translate positive words above guessing average by white ideographs when participants first performed the antonym generation task, but not when participants first performed the synonym generation task.
Method
Participants and procedure. A total of 101 students (51 females, mean age 20) at a Dutch university took part in the current study and were randomly assigned to either the synonym or the antonym condition. Before performing the translation task, participants performed either the synonym or the antonym generation task, which would ostensibly measure language skills. Participants were presented with 20 words (see Appendix 3), one after another, and were instructed to generate and write down (depending on the experimental condition) an antonym or a synonym for each word. After the antonym or synonym generation task, participants performed the Chinese translation task where translation judgments had to be made for positive words and white ideographs presented on a grey background.
Results
Two participants were excluded from the analysis because they made more than 50% errors in the synonym generation task. The average
108
number of YES judgments in both conditions was calculated and tested these against guessing average. After the antonym generation task, white ideographs were judged to correctly represent the meaning of positive words above guessing average (54,08%, M = 6.49, SD = 1.26), t(50) = 2.79, p = .01, Cohen’s d = .39, whereas in the synonym condition the translation judgments did not differ from guessing average (51,92%, M = 6.23, SD = 1.50), t(47) = 1.08, p = .29, Cohen’s d = .16.
Discussion
After priming participants with a conceptual opposition mindset through an antonym generation task, participants were more likely to see white ideographs as correctly translating positive words. These results indicate that seeing white and positivity in relation to their polar opposites activates the positivity of white. These results support the shared relational structures view, and cannot be explained by the one-to-one grounding account.
General Discussion
The last years have showed a continued growth of studies from a grounded cognition perspective which together have revealed the influence of perceptual information on abstract conceptual processing (Boroditsky, 2000; Crawford et al., 2006; Meier et al., 2004; Meier et al., 2007; Santiago, Lupiáñez, Pérez, & Funes, 2007; Schubert, 2005). Recently, researchers have argued to move beyond demonstration studies, and investigate how perceptual information is related to conceptual processing (e.g., Barsalou, 2008a; Zwaan, 2009). In this set of studies, I compared two process explanations against each other by investigating whether the shared opposition in the brightness and valence dimensions is an essential pre- requisite for the brightness-valance mapping to occur. Over six studies, these results reveal that whereas black is indeed associated with negativity, the positivity of white emerges only in opposition to the negativity of black.
109
As such, the brightness-valence metaphor is dependent upon the within participant manipulation of both endpoints of the brightness and valence dimensions (Study 4.1) or upon the perceptual (Study 4.5) or conceptual (Study 4.6) activation of the polar oppositions in both dimensions.
These studies compared an one-to-one grounding account for metaphor congruency effects with a shared relational structures view. According to the one-to-one grounding account, the perceptual characteristics of stimuli (e.g., their brightness) automatically activate their associated evaluative meaning (e.g., Meier et al., 2004; Meier & Robinson, 2004; Sherman & Clore, 2009). According to the shared relational structures view, metaphors map a relational structure from a concrete domain (e.g., the opposition between black and white) unto an abstract domain (e.g., the opposition between bad and good). On the basis of the results of the current studies, I conclude that a one-to- one grounding account cannot fully explain the metaphoric grounding of valence in brightness, and that the activation of the shared relational structures in the valence and brightness domains is essential for the brightness-valance mapping to occur.
The metaphoric representation of abstract concepts has been widely investigated using judgments under uncertainty, such as asking ambiguous time questions (Boroditsky, 2000; Boroditsky & Ramscar, 2002), judgments about the number of calories in food (Williams & Bargh, 2008), estimations about the room temperature (Zhong & Leonardelli, 2008), or the vertical locations of affective (Crawford, et al., 2006) and divine pictures (Meier et al., 2007). A benefit of using the Chinese translation paradigm is the possibility to compare the effect of manipulating dimensions within or between participants, which is not possible in bimanual categorization tasks. As a consequence however, these findings are less conclusive on whether the relationship between brightness and valence is automatic or not.
The importance of activating the polar oppositions in the brightness and valence dimensions is not at odds with the assumption that the brightness valence relationship is automatic (Meier et al., 2004; Sherman
110
& Clore, 2009). The all-or-none conception of automaticity has been challenged in recent years (for a review, see Moors & De Houwer, 2006), and many processes which were previously assumed to be fully automatic, such as the Stroop effect argued to underlie metaphor congruency effects (Meier et al., 2004; Meier & Robinson, 2004; Schubert, 2005; Sherman & Clore, 2009), turn out to be more conditional and less automatic than assumed (e.g., Kahneman & Chajczyk, 1983; Klauer & Musch, 2002; Neumann, 1984). The attention assigned to stimulus dimensions has been shown to matter for automatic affective stimuli processing (e.g., Spruyt, De Houwer, Hermans, & Eelen, 2007). Therefore, the brightness- valence relation could still be conditionally automatic, depending on the activation of the shared opposition between white-black and good-bad. Given that the polar opposites in these dimensions are salient enough (for example, by manipulating the endpoints of these dimensions within subjects), the brightness-valence association could influence abstract conceptual processing automatically.
In addition to providing new insights in underlying process of valence-brightness metaphor, the current set of studies has important methodological implications. Especially when investigating the meaning of perceptual characteristics of stimuli, shared relational structures in within-participant designs can be an important source of emergent meaning. For example, two measures which are often used to measure a pro-white/anti-black bias in children are the Color Meaning Test II (CMT II; Williams, Boswell, & Best, 1975) and the Preschool Racial Attitude Measure II (PRAM II; Williams, Best, Boswell, Mattson, & Graves, 1975). Children read a sentence with a positive or negative adjective, and are asked to choose between a white or black animal or human (e.g., Which is the bad man?). Although the preference to choose white figures for positive behaviors and black figures for negative is often given a racial interpretation, these results show it is likely that these studies do not tap into the intrinsic meaning assigned to the colors black and white, given that children, just like adults, rate the color white as neutral (Burkitt et al., 2003; Meerum Terwogt & Hoeksma, 1995).
111
Based on these studies, I argue that these results should be interpreted with great caution, since the preference to link positive actions to white figures and negative actions to the black figures does not necessarily reflect underlying stereotypes. Instead, it could reflect a brightness-valence mapping due to the shared relational structures of the dimensions. To control for effects due to the shared relational structures between the valence and brightness dimensions, researchers should consider adapting the paradigm by presenting figures of only one color, and asking children if this figure performed a positive or negative action, yes or no. Such a reasoning could be extended to include many different paradigms which rely on the within-participant manipulation of two dimensions consisting of polar opposites.
The interplay between metaphoric grounding of abstract concepts and attentional processes such as the salience of the structural similarity between the polar opposites of the brightness and valence dimensions suggests an important direction that research from a grounded cognition perspective should take. Instead of merely demonstrating patterns of relationships between stimuli and responses, researchers should aim to investigate the underlying processes at both an embodied and cognitive level. The demonstration of the prerequisite of the polar opposition in the perceptual and conceptual dimensions highlights the psychological interface by means of which the grounding of valence is driven. I see these results as an important step in developing a more mature understanding of how abstract concepts are grounded.
112
Footnotes
1. Lakoff & Johnson (1980) acknowledge that some metaphors structure not one concept in terms of another, but a whole system of concepts with respect to another domain (so-called orientational metaphors). These metaphors, such as the vertical representation of affect, often structure abstract concepts in spatial dimensions, such as the polar- oppositions up-down or in-out. However, throughout their work, such orientiational metaphors are argued to be based on two complementary metaphors (e.g., GOOD IS UP, BAD IS DOWN), and it is unclear whether the relational structures in these polar oppositions have a special status beyond the sum of their parts.
2. In a pilot study, sixty-five students rated the valence of 10 black or white Chinese ideograms on a 7 point scale from -3 (very negative) through 0 (neutral) to +3 (very positive), with the brightness of the ideograms manipulated between participants. The average rating was calculated for the ideograms and a two-sample t-test was used to compare the differences between ratings of the black and white ideograms. The valence ratings for the two ideograms did not differ for the white (M = 0.06, SD = 0.48) and black (M = 0.09, SD = 0.57) ideograms, t(63) = .17, p = .87, n.s..
113
Appendix 1
Moral Words: Charity*, Honest*, Equal*, Truth*, Good*, Honorable*, Integrity*, Righteous*, Rightful*, Moral*, Trust, Donate, Ethical, Volunteer, Just, Decent, Virtuous, Lawful, Assist, Noble. Immoral Words: Corrupt*, Lie*, Molest*, Deceive*, Injustice*, Immoral*, Unfair*, Evil*, Theft*, Traitor*, Adultery, Bribe, Self-centered, Rob, Sinfulness, Untrustworthy, Villainous, Selfish, Fraud, Favor. *Stimuli used in Study 2.1.
Appendix 2
Positive Words: Baby*, Brave*, Hero*, Love*, Sincere*, Loyal*, Party, Humour, Friend, Nice, Shining, Pleasurable. Negative Words: Death*, Cruel*, Poison*, Sick*, Enemy*, Hate*, Criminal, Cancer, Disgust, Unfair, Mean, Filthy. *Stimulus words used in Experiment 4.1 and 4.3
Appendix 3
Words presented during the antonym and synonym task: Beginning, Brave, Broken, Calm, Closed, Correct, Dirty, Drunk, Expensive, Fail, Get, Handsome, Naked, Nice, Order, Simple, Special, Talk, Thin, Weak.
115
References Adams, J. S. (1965). Inequity in social exchange. In L. Berkowitz (Ed.),
Advances in experimental social psychology, (Vol. 2, pp. 267-299). New York: Academic Press.
Adams, F. M., & Osgood, C. E. (1973). A cross-cultural study of the affective meanings of color. Journal of Cross-Cultural Psychology, 4, 135-156.
Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Lawrence Erlbaum.
Andrews, M., Vigliocco, G., & Vinson, D. (2009). Integrating experiential and distributional data to learn semantic representations. Psychological Review, 116, 463-498.
Arbib, M. A. (2008). From grasp to language: Embodied concepts and the challenge of abstraction. Journal of Physiology-Paris, 102, 4-20.
Barsalou, L.W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577-660.
Barsalou, L. W. (2008a). Grounded cognition. Annual Review of Psychology, 59, 617-645.
Barsalou, L.W. (2008b). Grounding symbolic operations in the brain’s modal systems. In G. R. Semin & E. R. Smith (Eds.), Embodied grounding: Social, cognitive, affective, and neuroscientific approaches (pp. 9-42). New York: Cambridge University Press.
Barsalou, L.W., & Wiemer-Hastings, K. (2005). Situating abstract concepts. In D. Pecher and R. Zwaan (Eds.), Grounding cognition: The role of perception and action in memory, language, and thought (pp. 129-163). New York: Cambridge University Press.
Basso, G., Nichelli, P., Frassinetti, F., & Di Pellegrino, G. (1996). Time perception in a neglected space. Neuroreport, 7, 2111–2114.
Belin, P., Zilbovicius, M., Crozier, S., Thivard, L., Fontaine, A., Masure, M. C., et al. (1998). Lateralization of speech and auditory temporal
116
processing. Journal of Cognitive Neuroscience, 10, 536-540. Benor, S. B., & Levy, R. (2006). The chicken or the egg? A probabilistic
analysis of English binomials. Language, 82, 233-278. Bergen, B., Lindsay, S., Matlock, T., & Narayan, S. (2007). Spatial and
linguistic aspects of visual imagery in sentence comprehension. Cognitive Science, 31, 733–764.
Bjoertomt, O., Cowey, A., & Walsh, V. (2002). Spatial neglect in near and far space investigated by repetitive transcranial magnetic stimulation. Brain, 125, 2012–2022.
Blauert, J. (1997). Spatial hearing: The psychophysics of human sound localization. Revised edition. Cambridge, MA: MIT Press.
Boot, I., & Pecher, D (in press). Similarity is closeness: metaphorical mapping in a conceptual task. Quarterly Journal of Experimental Psychology.
Boroditsky, L. (2000). Metaphoric structuring: Understanding time through spatial metaphors. Cognition, 75, 1-28.
Boroditsky, L. (2001). Does language shape thought? Mandarin and English speakers’ conceptions of time. Cognitive Psychology, 43, 1-22.
Boroditsky, L., & Prinz, J. J. (2008). What thoughts are made of. In G. R. Semin and E. R. Smith (Eds.), Embodied Grounding: Social, Cognitive, Affective, and Neuroscientific Approaches, (pp. 98 - 115). Cambridge: Cambridge University Press.
Boroditsky, L., & Ramscar, M. (2002). The roles of body and mind in abstract thought. Psychological Science, 13, 185-189.
Broeders, R., Van den Bos, K., & Müller, P. A. (submitted). Embodied power to the people: Powerful postures decrease the inhibition to intervene in moral and bystander dilemmas.
Bryant, P., & Squire, S. (2001). Children’s mathematics: Lost and found in space. In M. Gattis, (Ed.), Spatial schemas and abstract thought (pp. 175–201). MIT Press.
Burkitt, E., Barrett, M., & Davis, A. (2003). Children’s colour choices for completing drawings of affectively characterised topics. Journal of
117
Child Psychology and Psychiatry, 44, 445-455. Cacioppo, J. T., Priester, J. R., & Berntson, G. G. (1993). Rudimentary
determinants of attitudes: II. Arm flexion and extension have differential effects on attitudes. Journal of Personality and Social Psychology, 65, 5-17.
Casasanto, D. (in press). When is a linguistic metaphor a conceptual metaphor? In V. Evans & S. Pourcel (Eds.), New Directions in Cognitive Linguistics. Amsterdam: John Benjamins.
Casasanto, D., & Boroditsky, L. (2008). Time in the mind: Using space to think about time. Cognition, 106, 579-593.
Chapman, H. A., Kim, D.A., Susskind, J.M., & Anderson, A.K. (2009). In bad taste: Evidence for the oral origins of moral disgust. Science, 323, 1222-1226.
Clark, H. H. (1969). Linguistic processes in deductive reasoning. Psychological Review, 76, 387-404.
Clark, H. H. (1973). Space, time, semantics and the child. In T. E. Moore (Ed.), Cognitive development and the acquisition of language (pp. 27–63). New York: Academic Press.
Cooper, W. E., & Ross, J. R. (1975). World order. In R. E. Grossman, L. J. San, & T. J. Vance (Eds.), Papers from the parasession on functionalism (pp. 63-111). Chicago: Chicago Linguistic Society.
Crawford, L. E., (2009). Conceptual metaphors of affect. Emotion Review, 1, 129-139.
Crawford, L. E., Margolies, S. M., Drake, J. T., & Murphy, M. E. (2006). Affect biases memory of location: Evidence for the spatial representation of affect. Cognition and Emotion, 20, 1153–1169.
Crozier, W. R. (1999). The meanings of colour: Preferences among hues. Pigment & Resin Technology, 28, 6-14.
Damasio, A. R. (1989). Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition. Cognition, 33, 25-62.
Dennett, D. C. (1969). Content and Consciousness. London: Routledge & Kegan Paul.
118
De Soto, C. B., London, M., and Handel, S. (1965). Social reasoning and spatial paralogic. Journal of Personality and Social Psychology, 2, 513-521.
Dehaene, S., Bossini, S., & Giraux, P. (1993). The mental representation of parity and number magnitude. Journal of Experimental Psychology: General, 122, 371–396.
Dove, G. (2009). Beyond perceptual symbols: A call for representational pluralism. Cognition, 110, 412-431.
Eder, A. B., & Rothermind, K. (2008). When do motor behaviors (mis) match affective stimuli? An evaluative coding view of approach and avoidance reactions. Journal of Experimental Psychology: General, 137, 262-281.
Estes, Z., Verges, M., & Barsalou, L. W. (2008). Head up, foot down: Object words orient attention to the objects’ typical location. Psychological Science, 19, 93–97.
Eysenck, H. J. (1941). A critical and experimental study of color preferences. American Journal of Psychology, 54, 385-391.
Feldman, J. (2006). From molecules to metaphor: a neural theory of language. Cambridge: MIT Press.
Fischer, M. H., & Zwaan, R. A. (2008). Embodied language: A review of the role of the motor system in language comprehension. Quarterly Journal of Experimental Psychology, 61, 825-850.
Fiske, A. (1992). The four elementary forms of sociality: Framework for a unified theory of social relations. Psychological Review, 99, 689-723.
Fodor, J. A. (1975). The language of thought. Cambridge, MA: Harvard University Press.
Fodor, J. A. (1980). Methodological solipsism considered as a research strategy in cognitive psychology. Behavioral and Brain Sciences, 3, 63 - 109.
Fodor, J. A. (1983). The Modularity of Mind. Cambridge, MA: MIT Press. Fodor, J. A. (1985). Précis of The Modularity of Mind. Behavioral and
Brain Sciences, 8, 1 - 42. Folger, R. (1984). Emerging issues in the social psychology of injustice. In
119
R. Folger (Ed.), The sense of injustice: Social psychological perspectives (pp. 3 - 24). New York: Plenum.
Frank, G. (1976). On the validity of a hypothesis derived from the Rorschach: The relationship between color and affect. Perception and Motor Skills, 43, 411-427.
Frank, M. G., & Gilovich, T. (1988). The dark side of self- and social perception: Black uniforms and aggression in professional sports. Journal of Personality and Social Psychology, 54, 74-85.
Gallese V., & Lakoff, G. (2005). The brain’s concepts: the role of the sensory- motor system in reason and language. Cognitive Neuropsychology, 22, 455-479.
Gattis, M. (2001). Spatial schemas and abstract thought. Cambridge, MA: MIT Press.
Gattis, M. (2002). Structure mapping in spatial reasoning. Cognitive Development, 17, 1157-1183.
Gentner, D, & Bowdle, B. (2008). Metaphor as structure mapping. In R. W. Gibbs (Ed.). Cambridge Handbook of Metaphor and Thought (pp. 109 – 128). Cambridge, MA, Cambridge University Press.
Gibbs, R. W. (1996). Why many concepts are metaphorical. Cognition, 61, 309-319.
Gibbs, R. W. (2005). Embodiment in metaphorical imagination. In R. Zwaan and D. Pecher (Eds.), The grounding of cognition: The role of perception and action in memory, language, and thinking (pp. 65- 92). Cambridge: Cambridge University Press.
Glenberg, A., & Kaschak, M. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9, 558–565.
Glenberg, A. M., & Robertson, D. A. (1999). Indexical understand-ing of instructions. Discourse Processes, 28, 1-26.
Glenberg, A. M., & Robertson, D. A. (2000). Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning. Journal of Memory and Language, 43, 379-401.
Glenberg, A. M., Sato, M., Cattaneo, L. (2008). Use-induced motor plasticity affects the processing of abstract and concrete language.
120
Current Biology, 18, R290-R291. Glenberg, A. M., Sato, M., Cattaneo, L., Riggio, L., Palumbo, D., Buccino,
G. (2008). Processing abstract language modulates motor system activity. Quarterly Journal of Experimental Psychology, 61, 905-919.
Grady, J. (1997). Foundations of meaning: Primary metaphors and primary scenes. Unpublished doctoral dissertation, University of California, Berkeley.
Graham, J., Haidt, J., & Nosek, B. A. (in press). Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology.
Greenberg, J. H. (1963). Some universals of grammar with particular reference to the order of meaningful elements. In J. H. Greenberg (Ed.), Universals of grammar (pp. 73-113). Cambridge, MA: MIT Press.
Greene, J. D., Cushman, F. A., Stewart, L. E., Lowenberg, K., Nystrom, L. E., & Cohen, J. D. (in press). Pushing moral buttons: The interaction between personal force and intention in moral judgment. Cognition, doi:10.1016/j.cognition.2009.02.001
Grillon, C., Pellowski, M., Merikangas, K. R., & Davis, M. (1997). Darkness facilitates acoustic startle reflex in humans. Biological Psychiatry, 42, 453-460.
Götz, K.O., Götz, K. (1974) Color attitudes of art students and university students: I. Imagined colors. Perceptual and Motor Skills, 1974, 38, 63-70.
Haidt, J. (2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review, 108, 814–834.
Haidt, J. (2008). Morality. Perspectives on Psychological Science, 3, 65-72. Hampe, B. (2005). When down is not bad, and up not good enough: A
corpus-based assessment of the plus-minus parameter in image schema theory. Cognitive Linguistics, 16, 81-112.
Harnad, S. (1990). The symbol grounding problem. Physica D, 42, 335- 346.
Hauk, O., Johnsrude, I., & Pulvermuller, F. (2004). Somatotopic
121
representation of action words in human motor and premotor cortex. Neuron, 41, 301–307.
Hommel, B., & Prinz, W. (Eds.). (1997). Theoretical issues in stimulus– response compatibility. Amsterdam: North-Holland.
IJzerman, H. & Semin, G. R. (2009). The Thermometer of Social Relations: Mapping Social Proximity on Temperature. Psychological Science, 20, 1214-1220.
Jackendoff, R. (2002). Foundations of language: Brain, meaning, grammar, evolution. Oxford University Press.
Jeannerod, M. (2001). Neural simulation of action: A unifying mechanism for motor cognition. NeuroImage, 14, 103–109.
James, W. (1890). Principles of psychology. New York: Holt. Johnson, M. (1987). The body in the mind: The bodily basis of meaning,
imagination, and reason. Chicago: University of Chicago Press. Johnson, M. (1993). Moral imagination. Chicago: University of Chicago
Press. Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science
of language, inference, and consciousness. Harvard University Press.
Jostmann, N. B., Lakens, D., & Schubert, T. W. (2009). Weight as an embodiment of importance. Psychological Science, 20, 1169-1174.
Kahneman, D., & Chajczyk, D. (1983). Test of the automaticity of reading: Dilution of Stroop effects by color-irrelevant stimuli. Journal of Experimental Psychology: Human Perception and Performance, 9, 497–509.
Kan, I. P., Barsalou, L. W., Solomon, K. O., Minor, J. K., & Thompson- Schill, S. L. (2003). Role of mental imagery in a property verification task: fMRI evidence for perceptual representations of conceptual knowledge. Cognitive Neuropsychology, 20, 525-540.
Karwoski, T. F., Odbert, H. S., & Osgood, C. E. (1942). Studies in synesthetic thinking: II. The role of form in visual responses to music. Journal of General Psychology, 26, 199-222.
Kaschak, M., Madden, C., Therriault, D., Yaxley, R., Aveyard, M.,
122
Blanchard, A., & Zwaan, R. A. (2005). Perception of motion affects language processing. Cognition, 94, B79–B89.
Kimura, D. (1961). Cerebral dominance and the perception of verbal stimuli. Canadian Journal of Psychology, 15, 156-165.
Klauer, K. C., & Musch, J. (2002). Goal-dependent and goal-in-dependent effects of irrelevant evaluations. Personality and Social Psychology Bulletin, 28, 802–814.
Kohlberg, L. (1969). Stage and sequence: The cognitive-developmental approach to socialization. In D. A. Goslin (Ed.), Handbook of socialization theory and research. Chicago: Rand McNally.
Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: Cognitive basis of stimulus-response compatibility - A model and taxonomy. Psychological Review, 97, 253-170.
Kosslyn S. M. (1980). Image and Mind. Cambridge, MA: Harvard University Press.
Krzeszowski, T. P. (1997). Angels and devils in hell: Elements of axiology in semantics. Warsaw: Wydawn.
Laham, S. M., Alter, A. L., & Goodwin, G. P. (in press). Easy on the mind, easy on the wrongdoer: Discrepantly fluent violations are deemed less morally wrong. Cognition, doi:10.1016/j.cognition.2009.06.001
Lakens, D., Semin, G. R., & Foroni, F. (in preparation). But for the bad, there would not be good: Grounding valence in brightness through shared relational structures.
Lakens, D., Semin, G. R., & Garrido, M. (under review). The sound of time: Locating time in auditory space.
Lakens, D., Van den Bos, K., & Semin, G. R. (in preparation). On the grounding of morality in perceptual symmetry.
Lakoff, G. (1987). Women, fire, and dangerous things: What categories reveal about the mind. Chicago: University of Chicago Press.
Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortony (Ed.), In Metaphor and Thought, (pp. 202-251). Cambridge: Cambridge University Press.
Lakoff, G. (1996). Moral politics: What conservatives know that liberals
123
don’t. Chicago: University of Chicago Press. Lakoff, G. (2005). A cognitive scientist looks at Daubert. American Journal
of Public Health, 95, S114–S120. Lakoff, G. (2008). The neural theory of metaphor. In: R. W. Gibbs (Ed.),
Cambridge handbook of metaphor and thought (pp. 17 – 38). Cambridge, MA, Cambridge University Press.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press.
Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. Chicago: University of Chicago Press.
Landauer, T. K. & Dumais, S. T. (1997) A solution to Plato’s problem: The Latent Semantic Analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–40.
Langacker, R.W. (1987). Foundations of cognitive grammar: Vol. 1. Theoretical prerequisites. Stanford, CA: Stanford University Press.
Lindner, S. (1983). A lexic-semantic analysis of verb-particle constructions with up and out. Doctoral dissertation. University of California, San Diego. Reproduced by Linguistic Agency University Trier.
Louwerse, M. M. (2008). Embodied representations are encoded in language. Psychonomic Bulletin and Review, 15, 838-844.
Lu, C. H., & Proctor, R. W. (1995). The influence of irrelevant location information on performance: A review of the Simon and spatial Stroop effects. Psychonomic Bulletin & Review, 5, 174-207.
Maass, A., & Russo, A. (2003). Directional bias in the mental representation of spatial events: Nature or culture? Psychological Science, 14, 296– 301.
Machery, E. (2009). Doing Without Concepts. New York: Oxford University Press.
Mahon, B. Z., & Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. Journal of Physiology-Paris, 102, 59–70.
Mahon, B. Z., & Caramazza, A. (2009). Concepts and categories: A
124
cognitive neuropsychological perspective. Annual Review of Psychology, 60, 27-51.
MacLeod, C. (1991). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109, 163–203.
Mandler, J. (2004). The foundation of mind: Origins of conceptual thought. New York: Oxford University Press.
Mandler M. (1992). How to build a baby: II. Conceptual primitives. Psychological Review, 99, 587–604
Marks, L. E. (1978). The unity of the senses: Interrelations among the modalities. New York: Academic Press.
Marks, L. E. (1987). On cross-modal similarity: Auditory-visual interactions in speeded discrimination. Journal of Experimental Psychology: Human Perception and Performance, 13, 384-394.
Mayerthaler, W. (1980). Aspekte der Analogietheorie. In H. Lüdtke (Ed.), Kommunikationstheoretische Grundlagen des Sprachwandels (pp. 80-130). Berlin: de Gruyter.
McManus, I.C., Jones, A.L., Cottrell, J. (1981). The aesthetics of colour. Perception, 10, 651-666.
McGlone, M.S. (2007). What is the explanatory value of a conceptual metaphor? Language & Communication, 27, 109-126.
McGlone, M. S., & Harding, J. L. (1998). Back (or forward?) to the future: the role of perspective in temporal language comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 1211-1223.
Meerum Terwogt, M., & Hoeksma, J. B. (1995). Colors and emotions: Preferences and combinations. Journal of General Psychology, 122, 5-13.
Meier, B. P., Hauser, D. J., Robinson, M. D., Kelland Friesen, C., & Schjeldahl, K. (2007). What’s “up” with God? Vertical space as a representation of the divine. Journal of Personality and Social Psychology, 93, 699 – 710.
Meier, B. P., & Robinson, M. D. (2004). Why the sunny side is up: Associations between affect and vertical position. Psychological
125
Science, 15, 243-247. Meier, B. P., & Robinson, M. D. (2005). The metaphorical representation
of affect. Metaphor and Symbol, 20, 239-257. Meier, B. P., & Robinson, M. D. (2006). Does “feeling down” mean seeing
down? Depressive symptoms and vertical selective attention. Journal of Research in Personality, 40, 451–461.
Meier, B. P., Robinson, M. D., & Caven, A. J. (2008). Why a big mac is a good mac: Associations between affect and size. Basic and Applied Social Psychology, 30, 46-55.
Meier, B. P., Robinson, M. D., & Clore, G. L. (2004). Why good guys wear white: Automatic inferences about stimulus valence based on brightness. Psychological Science, 15, 82-87.
Meier, B. P., Robinson, M. D., Crawford, L. E., & Ahlvers, W. J. (2007). When “light” and “dark” thoughts become light and dark responses: Affect biases brightness judgments. Emotion, 7, 366–376.
Meier, B. P., Sellbom, M., & Wygant, D. B. (2007). Failing to take the moral high ground: Psychopathy and the vertical representation of morality. Personality and Individual Differences, 43, 757–767.
Messick, D. M. (1993). Equality as a decision heuristic. In B. A. Mellers & J. Baron (Eds.), Psychological perspectives on justice: Theory and applications (pp. 11-31). Cambridge, MA: Cambridge University Press.
Meteyard, L. & Vigliocco, G. (2008). The role of sensory and motor information in semantic representation: A review. In P. Calvo and A. Gomila (Eds.), Handbook of cognitive science: An embodied approach (pp. 293 - 312). Academic Press, London.
Murphy, G. (1996). On metaphoric representation. Cognition, 60, 173– 204.
Neumann, O. (1984). Automatic processing: A review of recent findings and a plea for an old theory. In W. Prinz & A. F. Sanders (Eds.), Cognition and motor processes (pp. 255-293). Berlin, Germany: Springer.
Niedenthal, P. M., Barsalou, L., Winkielman, P., Krauth-Gruber, S., & Ric,
126
F. (2005). Embodiment in attitudes, social perception, and emotion. Personality and Social Psychology Review, 9, 184-211.
Núñez, R., & Sweetser, E. (2006). Looking ahead to the past: Convergent evidence from aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30, 401- 450.
Osgood, C. E. (1952). The nature and measurement of meaning. Psychological Bulletin, 49, 197-237.
Osgood, C. E. (1960). The cross-cultural generality of visual-verbal synesthetic tendencies. Behavioral Science, 5, 146-169.
Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). The measurement of meaning. Urbana, IL: University of Illinois Press.
Pecher, D., Zeelenberg, R., & Barsalou, L. W. (2003). Verifying different- modality properties for concepts produces switching costs. Psychological Science, 14, 119 – 124.
Pecher, D., Zeelenberg, R., & Barsalou, L. W. (2004). Sensorimotor simulations underlie conceptual representations: Modality-specific effects of prior activation. Psychonomic Bulletin and Review, 11, 164–167.
Pecher, D. & Zwaan, R.A. (Eds.). (2005). Grounding cognition: The role of perception and action in memory, language, and thinking. Cambridge, UK: Cambridge University.
Piaget, J. (1954). The construction of reality in the child. New York: Basic Books.
Prinz, J. J. (2005). Passionate Thoughts. In R. Zwaan and D. Pecher (Eds.), The grounding of cognition: The role of perception and action in memory, language, and thinking (pp. 93-114). Cambridge: Cambridge University Press.
Prinz, J. J. (2007). The emotional construction of morals. Oxford: Oxford University Press.
Proctor, R. W., & Cho, Y. S. (2006). Polarity correspondence: A general principle for performance of speeded binary classification tasks. Psychological Bulletin, 132, 416-442.
127
Pujol, J, Deus J, Losilla, J. M., & Capdevila, A. (1999) Cerebral lateralization of language in normal left-handed people studied by functional MRI. Neurology, 52, 1038-1043.
Pulvermüller, F. (1999). Words in the brain’s language. Behavioral and Brain Sciences, 22, 253–336.
Pulvermüller, F. (2005). Brain mechanisms linking language and action. Nature Reviews Neuroscience, 6, 576–582.
Pulvermüller, F., Hauk, O., Nikulin, V. & Ilmoniemi, R. J. (2005). Functional links between motor and language systems. European Journal of Neuroscience, 21, 793-797.
Pylyshyn, Z. W. (1984). Computation and cognition. MIT Press. Rao, S. M., Mayer, A. R., & Harrington, D. L. (2001). The evolution of
brain activation during temporal processing. Nature Neuroscience, 4, 317–323.
Rawls, J. (1971). A theory of justice. Cambridge, MA: Harvard University Press.
Reber, R., Schwarz, N., & Winkielman, P. (2004). Processing fluency and aesthetic pleasure: Is beauty in the perceiver’s processing experience? Personality and Social Psychology Review, 8, 364–382.
Richards, I. A. (1936). The Philosophy of Rhetoric. Oxford, Oxford University Press.
Richardson, D. C., Spivey, M. J., & Cheung, J. (2001). Motor representations in memory and mental models: Embodiment in cognition. In Proceedings of the 23rd Annual Meeting of the Cognitive Science Society (pp. 867–872). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Richardson, D. C., Spivey, M. J., McRae, K., & Barsalou, L. W. (2003). Spatial representations activated during real-time comprehension of verbs. Cognitive Science, 27, 767–780.
Rudel, R. G., & Teuber, H.-L. (1964). Cross modal transfer of shape discrimination by children. Neuropsychologia, 2, 1-8.
Santiago, J., Lupiáñez, J., Pérez, E., & Funes, M. J. (2007). Time (also) flies from left to right. Psychonomic Bulletin & Review, 14, 512-516.
128
Scherer, L. D., & Lambert, A. J. (2009). Contrast effects in priming paradigms: Implications for theory and research on implicit attitudes. Journal of Personality and Social Psychology, 97, 383–403.
Schnall, S., Benton, J., & Harvey, S. (2008). With a clean conscience: Cleanliness reduces the severity of moral judgments. Psychological Science, 19, 1219-1222.
Schnall, S., Haidt, J., Clore, G., & Jordan, A. H. (2008). Disgust as embodied moral judgment. Personality & Social Psychology Bulletin, 34, 1096- 1109.
Schubert, T. W. (2005). Your highness: Vertical positions as perceptual symbols of power. Journal of Personality and Social Psychology, 89, 1-21.
Searle, J. (1980). Minds, brains, and programs, Behavioral and Brain Sciences, 3, 417-58.
Semin, G. R., & Smith, E. R. (Eds.). (2008). Embodied grounding: Social, cognitive, affective, and neuroscientific approaches. New York: Cambridge University Press.
Šetić, M., & Domijan, D. (2007). The influence of vertical spatial orientation on property verification. Language & Cognitive Processes, 22, 297- 312.
Sherman, G. D., & Clore, G. L. (2009). The color of sin: White and black are perceptual symbols of moral purity and pollution. Psychological Science, 20, 1019-1025.
Simmons, W.K., Hamann, S.B., Harenski, C.N., Hu, X.P., & Barsalou, L.W. (2008). fMRI evidence for word association and situated simulation in conceptual processing. Journal of Physiology–Paris, 102, 106-119.
Simmons, W. K., Ramjee, V., Beauchamp, M. S., McRae, K., Martin, A., & Barsalou, L. W. (2007). A common neural substrate for perceiving and knowing about color. Neuropsychologia, 45, 2802–2810.
Smith, L. B. (2005). Action alters shape categories. Cognitive Science, 29, 665–679.
Spruyt, A., De Houwer, J., Hermans, D., & Eelen, P. (2007). Affective priming of non-affective semantic categorization responses.
129
Experimental Psychology, 54, 44-53. Stanfield, R., & Zwaan, R. (2001). The effect of implied orientation derived
from verbal context on picture recognition. Psychological Science, 12, 153–156.
Talmy, L. (1983). How language structures space. In H. Pick & L. Acredelo (Eds.), Spatial orientation: Theory, research, and application (pp. 225-282). New York: Plenum Press.
Taylor, L. J., & Zwaan, R. A. (2008). Motor resonance and linguistic focus. Quarterly Journal of Experimental Psychology, 61, 896-904.
Toga, A. W., & Thompson, P. M. (2003). Mapping brain asymmetry. Nature Reviews Neuroscience, 4, 37-48.
Torralbo, A., Santiago, J., & Lupiáñez, J. (2006). Flexible conceptual projection of time onto spatial frames of reference. Cognitive Science, 30, 745-757.
Tversky, B., Kugelmass, S., & Winter, A. (1991). Cross-cultural and developmental trends in graphic productions. Cognitive Psychology, 23, 515-557.
Umiltà, C., & Nicoletti, R. (1990). Spatial S–R compatibility. In R. W. Proctor & T. G. Reeve (Eds.), Stimulus–response compatibility: An integrated perspective (pp. 89–116). Amsterdam: North-Holland.
Vallesi, A., Binns, M. A., & Shallice, T. (2008). An effect of spatial-temporal association of response codes: Understanding the cognitive representations of time. Cognition, 107, 501-527.
Van Dantzig, S., Zeelenberg, R., & Pecher, D. (2009). Unconstraining theories of embodied cognition. Journal of Experimental Social Psychology, 45, 345-351.
Van Dantzig, S., Pecher, D., & Zwaan, R. A. (2008). Approach and avoidance as action effects. Quarterly Journal of Experimental Psychology, 61, 1298-1306.
Varela, F., Thompson, E., & Rosch, E. (1991). The embodied mind. Cambridge, MA: MIT Press.
Verges, M., & Duffy, S. (2009). Spatial representations elicit dual-coding effects in mental imagery. Cognitive Science, 33, 1157-1172.
130
Vigliocco, G., Vinson, D. P., Lewis, W., & Garrett, M. F. (2004). Representing the meaning of object and action words: The featural and unitary semantic space hypothesis. Cognitive Psychology, 48, 422-488.
Weger, U. W., & Pratt, J. (2008). Time flies like an arrow: Space- time compatibility effects suggest the use of a mental time line. Psychonomic Bulletin & Review, 15, 426-430.
Whitney, W. D. (1875). The life and growth of language: An outline of linguistic science. New York: Appleton.
Wijn, R., & Van den Bos, K. (in press). On the social-communicative function of justice: the influence of communication goals and personal involvement on the use of justice assertions. Personality and Psychology Bulletin.
Williams, J. E., Boswell, D. A., Best, D. L. (1975). Evaluative responses of preschool children to the colors white and black. Child Development, 46, 501-508.
Williams, J. E, Best, D. L., Boswell, D. A., Mattson, L. A., & Graves, D. J. (1975). Preschool Racial Attitude Measure II. Educational and Psychological Measurement, 35, 3-18.
Williams, J. E., & Morland, J. K. (1976). Race, color, and the young child. Chapel Hill: University of North Carolina Press.
Williams, L. E., & Bargh, J. A. (2008a). Experiencing physical warmth promotes interpersonal warmth. Science, 322, 606 - 607.
Williams, L. E., & Bargh, J. A. (2008b). Keeping one’s distance: The influence of spatial distance cues on affect and evaluation. Psychological Science, 19, 302 – 308.
Zacks, J., & Tversky, B. (1999). Bars and lines: A study of graphic communication. Memory & Cognition, 27, 1073–1079.
Zajonc, R. B., & Markus, H. (1985). Affect and cognition: The hard interface. In C. E. Izard, J. Kagan, & R. B. Zajonc (Eds.), Emotions, cognition, and behavior (pp. 73–102). New York: Cambridge University Press.
Zhong, C. B., & Liljenquist, K. A. (2006). Washing away your sins: Threatened morality and physical cleansing. Science, 313, 1451–
131
1452. Zhong, C. B., & Leonardelli, G. J. (2008). Cold and lonely: Does social
exclusion literally feel cold? Psychological Science, 19, 838-842. Zwaan, R. (2004). The immersed experiencer: Toward an embodied theory
of language comprehension. In B. H. Ross (Ed.), The Psychology of Learning and Motivation 44 (p. 35). San Diego, CA: Academic Press.
Zwaan, R. A. (2009). Mental simulation in language comprehension and social cognition. European Journal of Social Psychology, 37, 1142- 1150.
Zwaan, R., Stanfield, R., & Yaxley, R. (2002). Do language comprehenders routinely represent the shapes of objects? Psychological Science, 13, 168–171.
Zwaan, R. A., & Taylor, L. J. (2006). Seeing, acting, understanding: motor resonance in language comprehension. Journal of Experimental Psychology: General, 135, 1-11.
Zwaan, R. A., & Yaxley, R. H. (2003). Spatial iconicity affects semantic- relatedness judgments. Psychonomic Bulletin & Review, 10, 954-958.
133
Samenvatting
Veel van de belangrijkste concepten in ons leven zijn zeer abstract. In dit proefschrift beschrijf ik hoe mensen over abstracte concepten zoals moraliteit, tijd, en valentie nadenken. Specifiek richt ik me op de vraag of mensen over abstracte concepten nadenken door de betekenis van deze concepten middels metaforen in concrete dimensies te representeren. Het gebruik van metaforen om abstracte concepten uit te beelden is alomtegenwoordig in onze samenleving. Zo is de zwarte ridder in sprookjes meestal de slechterik, en is de ridder op het witte paard de held, waarbij de kleuren zwart en wit worden gebruikt als een metafoor voor goed en slecht. Vrouwe Justitia heeft een weegschaal in haar hand, waarbij balans een metafoor is voor gerechtigheid, en hoewel we de toekomst niet kunnen zien, zeggen we dat de toekomst voor ons ligt. Deze metaforische representaties van abstracte concepten in termen van concrete ervaringen als kleur, balans of ruimte worden niet alleen intentioneel gebruikt in menselijke communicatie, maar spelen ook een rol als mensen zelf over abstracte concepten nadenken. De vraag hoe mensen de betekenis van abstracte concepten gronden in concrete ervaringen is het onderwerp van dit proefschrift.
De afgelopen tien jaar zijn steeds meer onderzoekers geïnteresseerd geraakt in de vraag hoe de betekenis van abstracte concepten
134
gerepresenteerd wordt in onze hersenen. Deze toegenomen interesse is veroorzaakt door nieuwe theorieën over menselijke cognitie, waarin gesteld wordt dat conceptuele gedachten perceptueel gerepresenteerd worden in onze hersenen. De assumptie is dat de betekenis van abstracte concepten is gegrond in concrete ervaringen. In tegenstelling tot amodale theorieën over conceptuele gedachten, waarbij de grote gemene deler is dat aangenomen wordt dat conceptuele representaties bestaan uit een puur symbolische taal, wordt in modale theorieën over conceptuele gedachten gesuggereerd dat concepten opgeslagen zijn als perceptuele informatie. Onze gedachten bestaan niet uit artificiële symbolen (zoals een computertaal) maar uit modaliteitspecifieke informatie (visueel, auditief, haptisch, etc). Ons begrip van het woord ‘schoppen’ is gebaseerd op de activatie van de spieren die we gebruiken om zelf te schoppen.
De afgelopen jaren zijn in toenemende mate studies verricht die de rol van zintuiglijke informatie in het begrijpen van concrete woorden hebben onderzocht. De studies laten zien dat het nadenken over de kleur van een banaan hersengebieden in de visuele cortex activeert (Simmons, Ramjee, Beauchamp, McRae, Martin, & Barsalou, 2007), vergelijkbaar met de hersenactivatie die waar te nemen is als mensen daadwerkelijk naar een banaan zouden kijken. Een belangrijke vraag met betrekking tot de perceptuele representatie van concepten is hoe abstracte concepten gerepresenteerd worden. Welke visuele, haptische of andere zintuiglijke informatie zou geactiveerd kunnen worden om de betekenis van een abstract concept als ‘moraliteit’ te begrijpen? Er zijn verschillende theoretische raamwerken voorgesteld welke een verklaring proberen te geven voor de perceptueel representatie van abstracte concepten (Barsalou, 1999; Boroditsky, 2000; Glenberg & Robertson, 2000; Lakoff & Johnson, 1980, 1999). Er woedt echter nog een verhitte discussie over de vraag of een perceptuele benadering van cognitie abstracte conceptuele gedachtes kan verklaren (Arbib, 2008; Barsalou; 2008; Boroditsky & Prinz, 2008; Lakoff, 2008; Mahon & Caramazza, 2008). In dit proefschrift onderzoek ik de rol van zintuiglijke informatie binnen abstracte gedachteprocessen. Het eerste hoofdstuk geeft een overzicht van
135
de huidige theorievorming binnen gegronde benaderingen van cognitie, en vergelijkt de assumpties van deze theorieën met meer traditionele benaderingen van cognitie. Vervolgens beschrijf ik empirische studies die hebben onderzocht of abstracte concepten perceptueel kunnen worden gerepresenteerd. Mogelijke alternatieve verklaringen voor de resultaten van deze onderzoeken worden besproken, op basis van structurele en taalkundige eigenschappen van de gebruikte onderzoeksparadigma’s. Ook geef ik een korte interpretatie van de inzichten die deze studies ons geven over hoe mensen over abstracte concepten nadenken.
De drie empirische hoofdstukken van dit proefschrift richten zich op de vraag hoe moraliteit, tijd en valentie gegrond worden in perceptuele symmetrie, auditieve ruimte en helderheid, respectievelijk. Het doel van hoofdstuk 2 is om te laten zien dat moraliteit, een concept waarvan vaak wordt beweerd dat het geen perceptuele eigenschappen heeft waarmee de betekenis gegrond zou kunnen worden, in concrete een concrete dimensie gerepresenteerd wordt. Gebaseerd op de assumptie dat gelijkheid een belangrijke eigenschap is van moraliteit, en dat de gelijkheidscomponent van moraliteit metaforisch gerepresenteerd wordt als balans of symmetrie, voorspelde en vond ik dat morele concepten sterker geassocieerd worden met perceptuele symmetrie dan immorele concepten.
In hoofdstuk 3 wordt de breedte van de perceptuele gronding van abstracte concepten uitgebreid naar de auditieve modaliteit. Hoewel voorgaand onderzoek zich voornamelijk heeft gericht op de visuele modaliteit, zouden abstracte concepten theoretisch in elke modaliteit gegrond moeten kunnen worden. Omdat het menselijk gehoor heel goed is in het lokaliseren van geluid, voorspelde ik dat tijd niet alleen in visuele ruimte gestructureerd zou worden, maar ook in de auditieve ruimte. Tijd wordt visueel gerepresenteerd van links (het verleden) naar rechts (de toekomst), en eenzelfde representatie werd in auditieve ruimte verwacht. Deelnemers kregen woorden te horen over een koptelefoon, en werden gevraagd om aan te geven of woorden harder in het linker of het rechter oor werden aangeboden. Sommige woorden werden even hard in beide kanalen aangeboden. Wanneer deze woorden toekomstgerelateerd waren,
136
gaven deelnemers vaker aan dat het woord harder in het rechteroor klonk, in vergelijking met verleden gerelateerde woorden. Daarbij was er een systematische overlap tussen de visuele representatie van tijd (gemeten in een eerdere studie) en oordelen van deelnemers tijdens de auditieve taak, welke verdere steun geeft aan het idee dat abstracte concepten in multimodale dimensies gestructureerd worden.
In Hoofdstuk 4 worden twee procesverklaringen met betrekking tot de metaforische representatie van abstracte concepten met elkaar vergeleken. Deze set studies is een zeldzaam voorbeeld van een directe vergelijking van twee theoretische verklaringen voor de processen die achter het gronden van abstracte concepten liggen. Waar sommige onderzoekers (Meier, Robinson, & Clore, 2004; Sherman & Clore, 2009) hebben gesuggereerd dat visuele informatie (bijvoorbeeld de kleur wit) automatisch en verplicht leidt tot de activatie van een abstracte betekenis (bijvoorbeeld positiviteit), ondersteunen de resultaten van deze studies een meer actief proces. De metaforische representatie van positiviteit als de kleur wit hangt af van de activatie van de conceptuele tegenstelling tussen goed en slecht en de perceptuele tegenstelling van wit en zwart. Het is dus de gedeelde relatiestructuur tussen de concrete dimensie en de abstractie dimensie die cruciaal is voor de metafoor van valentie in helderheid.
Tezamen laten deze hoofdstukken zien dat perceptuele informatie geassocieerd wordt met abstracte concepten. Zelfs zeer abstracte concepten welke geen perceptuele eigenschappen lijken te bezitten, ontlenen bij nader inzien hun structuur toch aan concrete dimensies. Sommige abstracte concepten, zoals tijd, lijken te worden gestructureerd in meerdere modaliteiten, en er is sprake van een sterke overlap tussen de visuele en auditieve structurering van tijd. Gedeelde relationele structuren (zoals tegenoverliggende polen) in de abstracte en concrete dimensie lijken in sommige situaties een essentiële voorwaarde te zijn voor de activatie van metaforische representaties. Verder onderzoek zou zich kunnen richten op de omstandigheden waarbinnen de activatie van gedeelde structuren noodzakelijk is, en wanneer de relatie tussen
137
abstracte concepten en concrete dimensies onafhankelijk is van de context. De mogelijkheid om over abstracte concepten na te denken is een van de meest verfijnde eigenschappen van mensen, en ongeacht of deze cognities noodzakelijkerwijs perceptueel van aard zijn, is het waardevol om een beter inzicht te verwerven in hoe mensen tot zo’n opmerkelijke vaardigheid in staat zijn.
139
Curriculum Vitae
Daniël Lakens was born and raised in Rotterdam, the only true city in the Netherlands. He studied psychology at Leiden University and philosophy at the Erasmus University, and received his master’s degree in social psychology in 2002. After working in the elderly care for a year, he pursued a career in science, first working as a research assistant for Wilco van Dijk at the VU University Amsterdam, before embarking on a PhD project under the supervision of Gün Semin in 2005. His research first focused on psychological consequences of movement synchrony, but he soon developed in interest in the grounding of abstract concepts. Early 2008 he moved to Utrecht University, and continued to investigate how people use concrete experiences to think about abstract concepts, which has resulted in the current thesis. Since January 2010, Daniel works as a post-doctoral researcher at the Eindhoven University of Technology.
The “Kurt Lewin Institute Dissertation Series” started in 1997. Since 2008 the following dissertations have been published:
2008-1: Marijke van Putten: Dealing with missed opportunities. The causes and boundary conditions of inaction inertia
2008-2: Marjolein Maas: Experiential Social Justice Judgment Processes 2008-3: Lonneke de Meijer: Ethnicity effects in police officer selection: Applicant, assessor,
and selection-method factors 2008-4: Frederike Zwenk: Voice by Representation 2008-5: Margreet Reitsma: The Impact of Linguistically Biased Messages on Involved
Receivers 2008-6: Marcus Maringer: Feeling one thing, seeing another: Emotion comparison effects
in person judgments 2008-7: Hanneke Heinsman: The competency concept revealed: Its nature, relevance, and
practice 2008-8: Joris Lammers: Toward a more social social psychology of power 2008-9: Daniël Fockenberg: Between Good and Evil: Affective Priming in Dynamic
Context 2008-10: Arne van den Bos: Why we stereotype influences how we stereotype: self-
enhancement and comprehension effects on social perception 2008-11: Lidewij Niezink: Considering Others in Need: On Altruism, Empathy and
Perspective Taking 2008-12: Aad Oosterhof: Better together: Antecedents and consequences of perceived
expertise dissimilarity and perceived expertise complementarity in teams 2008-13: Femke ten Velden: Negotiation in dyads and groups: The effects of social and
epistemic motives 2008-14: Maike Wehrens: How did YOU do? Social comparison in secondary education 2008-15: Kyra Luijters: Making Diversity Bloom: Coping Effectively with Cultural
Differences at Work 2008-16: Ilona de Hooge: Moral emotions in decision making: Towards a better
understanding of shame and guilt 2008-17: Lindred L. Greer: Team Composition and Conflict: The Role of Individual
Differences
2008-18: Sezgin Cihangir: The Dark Side of Subtle Discrimination: How targets respond to different forms of discrimination
2008-19: Giel Dik: On the contagiousness of others’ goals: The role of perceiving effort 2008-20: Lotte van Dillen: Dealing with negative feelings: The role of working memory in
emotion regulation 2008-21: Marijn Poortvliet: Information exchange examined: An interpersonal account of
achievement goals 2008-22: Sjoerd Pennekamp: Dynamics of disadvantage: Uncovering the role of group-
based anger 2008-23: Chris Reinders Folmer: Cooperation and communication: Plastic goals and
social roles 2009-1: Marijke Leliveld: Ethics in Economic Decision-Making 2009-2: Monique Pollmann: Accuracy and Bias in Person Perception 2009-3: Krispijn Faddegon: Regulatory Focus in Group Contexts 2009-4: Lieven Brebels: Mirror, mirror on the wall… Procedural fairness as an evaluative
and regulatory looking-glass self 2009-5: Daphne Wiersema: Taking it personally: Self-esteem and the protection of self-
related attitudes 2009-6: Judith D.M. Grob: Dial E for Emotion: Context and Consequences of Emotion
Regulation 2009-7: Katherine Stroebe: Is this about me? Responding to subtle discrimination - beyond
an individual versus group perspective 2009-8: Menno Vos: Identity patterns in diverse work groups: Improving social integration
outcomes through relational identities 2009-9: Lennart Renkema: Facing Death Together: Understanding The Consequences of
Mortality Threats 2009-10: Michael Vliek: Group-based social comparison processes: An intragroup level of
analysis 2009-11: Karlijn Massar: Unconscious rivals: The automatic evaluation of rivals in
jealousy-evoking situations 2009-12: Bart Terwel: Origins and consequences of public trust: Towards an understanding
of public acceptance of carbon dioxide capture and storage 2009-13: Emma ter Mors: Dealing with information about complex issues: The role of
source perceptions 2009-14: Martijn Veltkamp: On the Instigation of Implicit Motivation: How Deprivation
and Positive Affect Cause Motivated Behavior 2009-15: Marret K. Noordewier: Consistency and the unexpected 2009-16: Sytske van der Velde: Imitation of Emotion: How meaning affects the link
between imitation and liking 2009-17: Jacomijn Hofstra: Attaching Cultures: The role of attachment styles in explaining
majority members’ acculturation attitudes 2009-18: Jacqueline Tanghe: Affect in Groups: Convergence, Conditions and Consequences 2009-19: Anne Marike Lokhorst: Using Commitment to Improve Environmental Quality 2009-20: Jonathan van ‘t Riet: Framing Health Communication Messages 2009-21: Suzanne Pietersma: Persuasive Health Communication: A Self-Perspective 2009-22: Remco Wijn: A functional perspective on the justice judgment process and its
consequences 2009-23: Niels van de Ven: The bright side of a deadly sin: The psychology of envy 2009-24: Anthon Klapwijk: The Power of Interpersonal Generosity 2010-1: Maarten Wubben: Social Functions of Emotions in Social Dilemmas 2010-2: Joyce Rupert: Diversity faultlines and team learning 2010-3: Daniel Lakens: Abstract Concepts in Grounded Cognition
k u r t l e
w i n i n s
t i t u u t
Dissertatiereeks Kurt Lewin Instituut 2010-3
Comments







