What is the study of signs and symbols called?

Semiotics is a branch of the study of culture that explores signs and sign systems as modes of communication and how the meaning of signs and symbols are encoded and decoded. In structural theory, semiotics provides a means for conceptualizing and analyzing how individuals and collectivities communicate and interact in both denotative and connotative terms. In postmodernist theory, semiotics provides tools for deconstructing communicational behavior to reveal relationships of power and genealogies of authority and social control. Social semiotics focuses on semiotic procedures that underlie signification in specific social contexts and in particular communities. Semiotics has been employed in geography, either explicitly or implicitly, in a number of areas focusing on architecture and the social use of objects; the city as symbol and text; reading ordinary, elite and symbolic landscapes; popular culture; cartographic representation and meaning; toponymy; and public memory and commemoration. Despite these overlapping themes, the diverging theoretical interests of geographers and semioticians mean that contacts between the two fields are likely to remain modest.

Abstract

Semiotics is the interdisciplinary study of signs, particularly their social origins, uses, and consequences. Rooted in philosophy, logic, and linguistics, the field has developed since the 1960s as an internationally influential approach to research on contemporary culture and communication. Revisiting classic epistemological issues, semiotics further has contributed to the theory of science by clarifying the relationship of a third form of inference – abduction – to deduction and induction. In the context of digital networks, semiotics helps to specify the several different categories of signs that enter into computer-mediated social interaction.

Classifying Signs

The first fundamental task of any science is the classification of its objects of investigation. This is also the case with semiotics. The first classification system of signs goes right back to the ancient world, starting with the definition of sign by Hippocrates (c. 460–370 BCE), the founder of Western medicine, as a symptom. Hippocrates argued that the particular physical form that a symptom takes – a semeion (mark) – constitutes a vital clue for finding its etiological source. Shortly thereafter, philosophers started referring to signs as being either natural (produced by the body or nature) or conventional. Among the first to tackle this basic typology was St. Augustine (354–430 CE) in his De Doctrina Christiana, in which he describes natural signs (signa naturalia) as forms lacking intentionality and conventional ones (signa data) as forms produced by human intentions. The former include not only symptoms, but also such phenomena as plant coloration, animal signals, and the like; the latter include not only words, but also gestures and the various symbols that humans invent to serve their psychological, social, and communicative needs (Deely, 2001: 24–56). It was the British philosopher John Locke (1632–1704) who first put forward the proposal of incorporating the study of conventional signs into philosophy in his Essay Concerning Human Understanding (Locke, 1690). However, the idea of fashioning an autonomous discipline of sign study did not emerge until the late nineteenth century, when the Swiss philologist Ferdinand de Saussure proposed such an idea in his Cours de linguistique générale (de Saussure, 1916). Saussure suggested that the main goal of a science of signs was to understand how signs stand conventionally for things in specific social environments and how they permit people to interact meaningfully.

The first sophisticated classification of signs is due to the American pragmatist philosopher Charles S. Peirce (1839–1914), who identified 66 species of signs in total. Some of these are simply terminological strategies that provide detail as to how signs function cognitively. For example, as its name implies, a qualisign is a form that draws attention to some quality of its referent (the object it represents). In language, an adjective is a qualisign since it draws attention to the qualities (color, shape, size, etc.) of nouns. In other sign systems, qualisigns include colors (painting), harmonies and tones (music), etc. A sinsign is a form used to single out a particular object – a pointing finger and the words here and there are examples of sinsigns. A legisign is a form that designates something by convention (literally by law). Legisigns include symbols, emblems, such as those used on flags, and various other forms that are used according to specific social rules or conventions. However, there are three signs identified by Pierce that are now part and parcel of every semiotician's methodological toolkit – icons, indexes, and symbols. Signs originating in the perception of some property in a referent are icons. Iconic signs stand for their referents by resemblance, imitation, simulation, or emulation. Portraits are icons of human faces (and what they mean); onomatopoeic words are icons of sounds made by certain objects or actions (drip, bang, etc.); and so on. The meanings of icons can generally be figured out even by those who are not a part of the culture that use them. Signs that relate referents, sign users, and even other sign forms in some way are known as indexes. The pointing finger is a perfect example of an index. When we point to something, we are in fact relating it to our location as pointers. If it is close by, we refer to it as near or here. If not, we refer to it as far or there. Finally, signs that stand for referents in historically based or conventional ways are symbols. The V-figure standing for peace has symbolic valence, that is, it is interpreted conventionally for peace as a result of some historically or culturally based event.

Semiotic Analysis

Semiotic analysis is grounded on the work of the Swiss linguist Ferdinand de Saussure. His theory sheds light to the process of signification, whereby certain sounds are typically related to ideas as a result of the structure of language. Saussure put forward that sounds of speech or signs have no intrinsic meaning. The meaning of a sign is relational; it derives from its difference to other signs. Linguistic signs, he explained, are made up of the signified and the signifier. The signifier is the spoken sound or written word that conventionally expresses an idea. The signified is the idea that is conventionally expressed by the signifier. For example, the sound or word ‘house’ conventionally refers to the idea of a structure that serves as a dwelling for one or more persons. Saussure emphasised that the sign ‘house’ is not intrinsically tied to this idea because the connection between the signifier and the signified is arbitrary. Poststructural theories challenged Saussure’s claim that once a signifier and a signified are brought together to form a sign, this sign has a fixed meaning. Poststructuralist semiotics argues that because signifiers are separable from the things that are being signified, meanings are never fixed but are constantly open to interpretation.

Roland Barthes developed Saussure’s theory further, arguing that beyond their literal meaning (denotation) signs may have other meanings attached (connotations). From this, Barthes concluded that there are two levels of signification, denotation and connotation. At the level of denotation is a sign which is made up by a signifier and a signified; for example, the signifier ‘house’ and the signified of a structure that serves as a dwelling for one or more persons. At a second level this sign becomes a signifier to which another signified is attached. For example, at this second level the sign ‘house’ is normally attached to notions of domesticity and family life. Barthes extended the ideas of Saussure further by applying semiotic analysis beyond language to a variety of objects. He investigated how a variety of objects, such as housing, can be thought of as signs in that they not only fulfil a particular function but are also imbued with meaning.

19.3.3.2 Semiotic levels and multimodal analysis

From a Semiotics perspective, we can define a Cybercartographic Atlas as multimodal text or discourse, precisely for its multisensory character. A multimodal text or discourse combines two or more semiotic systems, for example: linguistic, visual, verbal or spatial systems. In the Pa Ipai Atlas oral language converges with songs, narratives, spatial and astronomical systems and images.

In a Cybercartographic Map a Semiotic syncretism also occurs. Here is the explanation: When in the interactive platform the recordings of indigenous names of places, songs, narratives and drawings, as attributes of the landscape, are linked to the map, they pass from a linguistic and musical language into a spatial language giving us a mix between different semiotic systems (Rosales, 2012; Zilberberg, 2001).

Simultaneously, different narratives are superimposed in the Atlas. ‘Cybercartography allows the presentation of different ontologies or narratives on the same topic without privileging one over another. The user can consider the various narratives presented and have a greater understanding of the complexities and uncertainties surrounding many topics’ (Taylor, 2014, 3).

The coexistence of different narratives poses a big challenge for semiotic analyses. Traditional semiotics was used to study isolated narratives. From a semiotics perspective, Cinema represents a breaking point because of the need to take into account the simultaneity of visual and musical narratives (Caquard and Naud, 2014). Cybercartographic Atlases represent a step forward in the understanding of the relationship among systems of spatial, audio visual and linguistic, and other kind of narratives. Multimodality, semiotic syncretism and coexistence of different narratives present the possibility of creating new geographical, collaborative and non-hegemonic discourses. A semiotic approach to these processes offers theoretical tools to understand why and how new meanings emerge through Cybercartography and the use of its technological frameworks.

In the following section, we analyse the Atlas using semiotic tools. We will use two different methodological strategies to analyse the multimedia content and consider how these semiotic methodologies help to develop the new Pa Ipai Atlas.

The first methodology is based on ‘the Semiotic pertinence levels of the discourse’ as named by the French semiotician Jacques Fontanille. These seven semiotic levels are: Signs, Texts, Objects, Practices, Strategies, Ways of Life and finally Culture (Fig.19.2).

Signs correspond to the most elementary structures of the discourse, units that take the place of something else; the union of signs creates statements. Texts are signs and statements organized and structured within a code. Objects are material structures that have a morphology and a functionality, and are destined to specific uses or practices; texts are integrated into objects. The morphology and shape of the object determine the semiotic practices and effects related to it. So the object has two dimensions: it is a support of the text and it is an actor of the semiotic scene practices, which produce meaning using the object for those purposes. Practices are composed of two levels: semiotic situations and semiotic scenes. The first consist of all the elements that permit the production and interpretation of the meaning during a communicative interaction. Scenes are open processes that circumscribe semiotic practices. There can be no practices without scenes. Strategies mean that each practical scene needs to be connected in time and space with other scenes and practices. Each scene and each practice is part of a bigger sequence. Strategies give us an idea about these bigger sequences. Ways of life are styles of behaviours, in other words meaningful behaviours. They are constituted by each one of the following levels; in them we can identify rhythms and attitudes. The attitudes, values and principles are expressed in stylistic rhythms. Finally we have the culture level. The combination of the previous levels constitutes what semiotics defines as culture. By identifying and describing the previous levels it is possible to obtain and infer the characteristics of a culture.

1.1 Systems of Objects

In the 1960s, semiotic approaches (see Semiotics) described systems of objects that constitute codes, that is systems of signs, comparable to language. These codes were stipulated to be autonomous from the social and economic systems. Roland Barthes (1964) analyzed the system of clothing fashion. He differentiated three structural levels of fashion: technological (textile structure), iconological (photographic presentation), and verbal (writing about fashion). Other semiotic analyses concerned everyday classifications of objects in museums or department stores (see Semiotics). In a related vein, structural anthropologist Lévy-Strauss analyzed systems of objects, for example food or clothing, as binary codes that establish basic cultural categories and determine cultural practices (see Structuralism).

Semiotic and structural approaches describe systems of material signs or categories synchronically. Systems of objects may also be described in the same formal manner diachronically. Richardson and Kroeber (1940), for example, identified a cyclical pattern in the change of formal properties of female dress across centuries. Barthes concluded from this that semiotic systems have a temporal dimension, but not a historical dimension. However, in 1982 Lowe and Lowe continued Kroeber's analysis in time and demonstrated that historical changes were not cyclical and were influenced by particular historical and social changes.

Others constructed a diachronic perspective on systems of objects by using the metaphor of evolution. George Kubler (1972) constructed the history of style in art as that of successive solutions to formal problems, an idea that he extended to instruments such as ploughs. Sigfried Giedion (1948) described the history of instruments in terms of constitutive and transitory facts, alluding to the evolutionary dichotomy of geno- and phenotypes. The evolution of Mickey Mouse and of teddy bears during the twentieth century have been analyzed in terms of maximizing congruence with the innate preference for infantile forms. In the quasi-evolution of forms of cultural artifacts, however, environmental demands such as aesthetic preferences and utility compete with each other, as demonstrated by David Norman (1988) with regard to the evolution of the telephone.

4 Conclusion

Gesture is a semiotic system distinct from language. At the same time, it is intimately linked to various aspects of linguistic performance. Gesture is an integral part of the organization of face-to-face conversation. Pointing gesture is an essential part of referential acts by means of spatial deixis. The production of intonation and ideophones are tightly coupled with the production of a gesture that is co-expressive. The speaker's imagistic thinking during verbalization of thought is revealed by gestures that spontaneously accompany speech. Because of the interactional, referential, and psychological unity of gesture and language, the understanding of language is not complete unless gesture is taken into account.

Marxism

Whereas semiotics and hermeneutics are mostly used to analyze visual representations and spaces already produced, Marxist methodologies are concerned with the production as well as the consumption of images and narratives. Of particular interest are questions about who owns and controls the production of movies, and who stands to gain most from their distribution. How are the directors, writers, actors, artists, and other creators of movies, and how are cinemas and the consumers of movies affected by the patterns of ownership and control of the movie industry?

In classical Marxism, there are assumptions of underlying economic structures with their own inner logics, and an understanding of these logics enables researchers to analyze everyday cultural elements. In the 1980s, concern was raised that the complexity of culture was lost to an endless recoding of property relations that are reduced to an explanation based upon class conflict. Concerned about classical Marxism’s inability to interpret small variations in cultural meaning, Fredric Jameson used literature and film studies to argue that there are important underlying esthetic and psychic processes – which he called the ‘political unconscious’ – that are co-created along with larger economic forces. This consciousness is socially produced, and is filtered through the minds of men and women to the extent that dominant ideologies – and particularly capitalism – can be contested. In The Geopolitical Aesthetic (1992), Jameson uses the works of French filmmaker Jean-Luc Godard and Filipino filmmaker Kidlat Tahimik, among others, to argue that film analysis is one of the primary methodologies for exposing the importance of space in the construction of social and economic relations in late capitalism. Jameson’s Marxist inspired approach to film reinvigorated studies of visual culture through the 1980s and 1990s, and paved the way for psychoanalytic and post-structural frameworks.

Thick Description and Reflexive Ethnography

The semiotic conceptualization of culture expounded by symbolic and interpretive anthropology revolutionized ethnography during the 1970s. Following the intellectual footsteps of German sociologist Max Weber, American anthropologist Clifford Geertz (1973) conceptualized culture as a web of meanings spun by the people themselves. Ethnography moved from its empiricist tendency toward hermeneutic interpretation. Fieldwork became like reading an old, partially faded manuscript with all the accompanying misreadings and mistranslations. Drawing on French philosopher Paul Ricoeur (1971), Geertz resembled cultures to documents as systems of meaning embodied in symbolic forms because of four similarities between cultural practices and texts: they are both meaningful; have observable objectifying consequences; supersede the original conditions, relevance, and authorship of their production; and address different audiences.

Geertz defined ethnography as thick description. Thick description tried to unravel the conceptual structures and frames of local interpretation by studying people's conflicting social discourses in order to arrive at cultural systems of meaning. These systems are not emic analyses or descriptions of the actor's point of view but anthropological constructions of the actor's interpretations of his or her actions, emotions, beliefs, and experiences. The test of validity is ultimately the ethnography's imaginative quality. Textual representations take the form of detailed analyses of specific cultural phenomena as interpretations of larger sociocultural realities. Geertz's (1973) interpretation of cockfights in Bali is a typical study. He describes these events as social gatherings at which men gamble and play putting their prestige, masculinity, and social statuses at risk. The cockfight reveals the multiple fixed social hierarchies of Balinese society without ever changing them. It is therefore a metasocial commentary, a text about Balinese experience informing people about the lives they live.

The attention of interpretive anthropology to local meaning prompted the critique that the intersubjective construction of field data remained untouched. Reflexive anthropology arose in the 1980s to examine the narrative genres, forms, styles, structure, and rhetoric of ethnographic texts (Clifford and Marcus, 1986). This approach was strongly influenced by the practice theory of French sociologist Pierre Bourdieu, the study of discourses of power by French social philosopher Michel Foucault, and the dialogic theory of Russian philosopher Mikhail Bakhtin. Ethnographic research became interpreted as a practice of domination because of the unequal power relation between the authoritative field-worker and the local informant.

Reflexive ethnography implies the study of the field-worker's interpretive presuppositions and ethnographic authority, the interaction with research participants, the intercultural construction of ethnographic knowledge, and the narrative style of research findings. Ethnographers became conscious of how knowledge was produced in research and reporting, as exemplified by Paul Rabinow's (1977) analysis of his fieldwork in a Moroccan town. Intercultural translation by key informants required that they first understand their own culture before they could describe it to a foreign researcher who lacked any experiential knowledge of the culture under study. Moroccan interlocutors who were situated at the social periphery proved to be excellent interpreters because they could see across social boundaries.

In an era in which researched ‘others’ increasingly ‘spoke back’ to what was written about them, and with a heightened awareness of the relationship between power and the construction of knowledge, ethnographers sought to increase the voice of the ‘other’ through more active involvement of research participants. This was achieved by extensive use of direct quotes, coauthorship, reciprocity, collaboration, and partnership. Vincent Crapanzano (1980) addressed the intersubjective construction of cultural meaning in an exegesis of his research relation with the illiterate Moroccan tilemaker Tuhami. He reproduced the dialogic quality of fieldwork by presenting a literal transcription of distinct voices, questions and answers of Tuhami and himself, intermingled with anthropological commentary. Crapanzano chose an experimental writing style to convey this field encounter; other field-workers took an additional step toward ethnographic fiction and poetry.

Envisioning the Linguistic Turn

The intersection between semiotics, linguistics, and visual studies goes beyond a mere borrowing of theoretical jargon, for it is embedded in the very conceptualization of the turn. In fact, the phrase ‘visual turn’ has gained popularity by its assonance to other postmodern turns within the humanities and social sciences, most notably the ‘linguistic turn’. The linguistic turn, which in the humanities and social sciences is often associated with a shift from structuralist to poststructuralist semiotics, bears the proposition that language constitutes reality and that the linguistic model ought to be the model for all communications. While metaphysical thinking takes language as a label that is tagged to concepts and objects, thinkers like Jacques Derrida and Michel Foucault borrowed from Ferdinand de Saussure's structuralist linguistics the idea that the relationship between signified and signifier is arbitrary. Thus, the linguistic turn is particularly interested in the metaanalysis of the linguistic tropes and narratives that convey truth and believability as such.

There are two key issues that arguably made the linguistic turn attractive for what here is being termed as the visual turn. The first has to do with the fact that, by shifting our attention from an abstract conception of text to a material source of textualities, the linguistic turn emphasized the unacknowledged role of writing as discourses in which the subject is inscribed. The emphasis on discourse is grounded on the idea that the speaker is not the subject of language, but the speaker exists as long as he or she is subjected to language. Thus, language and symbolic systems in general are not an expression of subjectivity, but rather they represent the agency that produces subjectivity by projecting around the speaker the space of the subject (Benveniste, 1971[1958]). Michel Foucault, who referred to this process as assujettissement, defined discourse as a body of knowledge that both defines and delimits what can be said about something. Discourses obviously change over time and are specific to certain social and historical contexts. As discourses regulate the administration of power and knowledge between the sayable and the seeable, there is no other way to visualize the subject but in intersection with discursive formations of identity and identification. Hence, visuality could be considered as the multifarious and fragmented discourse about vision, power, and knowledge.

Therefore, the second attractive aspect it follows is that the linguistic turn represents a break with the humanist conception of the subject, and challenged the idea that meaning should be traced back to the intentions of an agent or maker. Once it has been acknowledged that the meaning of a text cannot be tied up to a univocal source, but it has a life of its own, so to speak, it becomes easier to understand that the same might apply to images and visual culture that have in language their model of manifestation. For Derrida, the interpretation of a text or an image is enfolded within an irreducible and generative multiplicity of contexts and readings that he compared to ‘dissemination’. Derrida proposes that meaning is a function of context, and that there are no limits or barriers to an endless sequence of recontextualizations. Hence, it follows that what dissemination displaces is the concept of an ‘exhaustively determinable’ context of meaning, that is a context originating in conscious intention (Derrida, 1982).