By Elisa Roßberger and Patrizia Heindl
Introduction
We can trace the use of multimodal communication strategies in Egypt and Mesopotamia back to the early days of writing. An inscribed and/or additionally relief decorated statue, a vessel, a weapon, a make-up palette or a seal enrolled on a clay tablet could speak for and to a certain person or deity, invite the viewer to speak, touch or act, and visualize past (or future) actions/events — obviously in a more effective way than monomodal, i.e. purely figuratively or purely textually rendered artifacts. Often, surprising, art- or playful combinations of verbal and non-verbal elements were used. The structural composition of the text-image connections, their framing, scaling, place of application and rendering, could all contribute to the communicative effectiveness of the artifacts, as could the materiality and physical placement of the artefact itself.
Thus, ancient and contemporary communication processes that use multiple channels of sensory experience and semiotic modes to convey their message(s) do not differ fundamentally, but in their resource-related creative ressources. The growing importance of visual and auditive forms of expression in combination with text-based information which is typical for digital media has led in recent years to an intensified scientific examination of the mechanisms of multimodality and their cognitive preconditions, particularly in fields like advertising, visual design, infographics or social media.[1] Based on linguistically inspired methods and concepts of sociosemiotics, research in ancient studies can profit from these insights and at the same time shift attention to cultural specifics and long-term developments in these processes.[2]
The conference focuses on forms of interaction between verbal and non-verbal elements in the graphic design in/on artefacts from ancient Egypt, the Near East and beyond. The term “multimodality”, commonly used in communication and media studies, serves as a conceptual anchor for thinking about the interface between semiotic codes, sensory modalities and cognitive processes. The comparative view should help us to approach the similarities and differences of the material and mental sign ideologies underlying the production and use of artefacts.[3]
The conference brings together scholars from the fields of pictorial philosophy, visual linguistics, media and religious studies, with scholars studying the textual and archaeological remains of ancient cultures from two neighbouring regions.
The “Münchner Zentrum für Antike Welten” (MZAW), as a working group of academics from all disciplines dealing with ancient cultures at LMU Munich, offers an ideal platform for the conference to which both Munich experts and international colleagues will contribute.
Terminology and methodology of multimodal artifact analysis
A comparative examination of multimodal forms of communication in antiquity and the present requires a reflection on terminological and theoretical-methodological issues. We consider semiotics as a useful and coherent conceptual and analytical tool that can help us to grasp relationships between different sign systems and compare them across cultures.
For a long time, semiotics was considered too language-centered to be useful for the analysis of imagery and text-image correlates; furthermore, its focus on communication endpoints (sign users: senders/receivers) and messaging function was criticized for its neglect of the material and visual features of the media and channels of communication themselves. This concentration on the communicative potential of sign phenomena supposedly lead to a blindness regarding aesthetic and atmospheric experiences (“forgetfulness of perception” according to Horst Bredekamp), to a “forgetfulness of the body” (Hans Belting) and to a lack of “presence orientation” (Hans Ulrich Gumbrecht).[4] Further developments in the fields of cultural and socio-semiotics took up on these points of criticism aiming at a better understanding of the social dimensions and signification processes that lie behind the combined experience of different sign systems of a visual, verbal, auditory or tactile nature.[5]
Forms of multimodality in Egypt and the Near East
Thus, discussions of time- and context-specific manifestations of socially constituted systems of meaning-making (semiotic practices or ideologies) become relevant. These need to be worked out against the background of concrete historical constellations and circumstances in case studies from Ancient Egypt and the Near East.
We will focus on the overall meaning/message that results from a specific combination of text and image in/at an artifact (semantics). The forms and design of verbal and non-verbal elements applied to the artefact, their proportionate size, prioritization, arrangement and framing should receive special attention (materiality; syntax), as should the gestural, tactile or tonal actions suggested to the user/viewer by certain constellations of images and text (performativity; pragmatics). The spatial placement of an artefact provides important hints in this regard (contextualization).
For ancient Egypt, we can observe an intensive entanglement of writing, image and language. This led to a ‘translatability’ of entire pictorial works as rebus and forms of emblematics (including ‘cryptography’) on the one hand, and to an ambiguous legibility of individual pictorial signs, on the other.
For the ancient Near East, such phenomena are far less obvious and much less investigated. However, we can equally observe the targeted use and the great social significance of text-image-artifact constellations at a very early point and throughout history.
Multimodality and cultural order: Ontological consequences
Can we arrive at conclusions concerning the constitutiveness of reality, i.e. the ontological foundations of a social group, when we study the culture- and context-specific forms and functions of multimodal artefacts? Here, we enter the discursive fields of cultural semiotics[6] and “semiospheres”,[7] which deal with connections between social culture (society as a set of sign users), material culture (the totality of artefacts as a set of texts) and mental culture (the mentality of a society as a set of conventional codes) (cf. Bal/Bryson 1991; Keane 2018).
For example, inscriptions on Greek vase paintings usually evoke narratives in the heads of their viewers through their text content, while inscriptions on ancient Near Eastern cylinder seals and their rollouts on clay tablets referred to the personal and social identity of their owners and thus referenced physical presence and responsibility. Egyptian artworks, on the other hand, generated at least two levels of reality through allusions to mythical traditions: one for the deceased himself, who was cared for in the afterlife, and one for the ancient and recent observer, who knows the deceased to be taken care of. Through the tonal reproduction of ‘readable’ images of gods or cryptographically written names, it was also possible to create a further level of reality, which could also influence the overall significance of the pictorial-inscribed-artwork.
In this regard, we may make use of incentives from cognitive sciences. These may contribute substantially to our current understanding of the added value resulting from the combination of sensory experiences and cognitive knowledge in various processes of social communication and self-understanding (see e.g. Levinson/Holler 2014). While we should assume that certain connections between sign modalities and channels of sensory perception are and have been used in ancient as well as contemporary cultures for specific media and situational configurations (cf. Stöckl 2016), we should also cautiously consider the diversity of their manifestations before arriving at universal assumptions.
The conference aims to test the possibilities and benefits of multimodal analysis strategies for research in ancient studies. We look forward to stimulating lectures and discussions that will bring us closer to a better understanding of the diversity of human communication and its graphical implementation.
Literature cited
Bal, M./Bryson, N. (1991): Semiotics and Art History. The Art Bulletin 73 (2), 174–208.
Baines, J. (2007): Visual and Written Culture in Ancient Egypt. Oxford.
Bateman, J./Wildfeuer, J./Hiippala, T. (2017): Multimodality. Foundations, Research and Analysis. A Problem-Oriented Introduction (Mouton Textbook). Berlin, Boston.
Cassirer, E. (2010 [1923-1929]): Philosophie der symbolischen Formen. Hamburg.
Diekmannshenke, H.-J./Klemm, M./Stöckl, H. (Hrsg.) (2011): Bildlinguistik. Theorien – Methoden – Fallbeispiele (Philologische Studien und Quellen 228). Berlin.
Große, F. (2011): Bild-Linguistik. Grundbegriffe und Methoden der linguistischen Bildanalyse in Text- und Diskursumgebungen (Germanistische Arbeiten zu Sprache und Kulturgeschichte 50). Frankfurt a.M.
Halawa, M. A. (2009): Widerständigkeit als Quellpunkt der Semiose Materialität, Präsenz und Ereignis in der Semiotik von C.S. Peirce. KODIKAS/CODE. Ars Semeiotica 32 (1-2), 11–24.
Keane, W. (2018), On Semiotic Ideology. Signs and Society 6 (1), 64–87.
Kress, G. (2010): Multimodality. A Social Semiotic Approach to Contemporary Communication. London.
Levinson, S. C./Holler, J. (2014): The origin of human multi-modal communication. Philosophical transactions of the Royal Society of London. Series B, Biological sciences 369 (1651), 1–9.
Lotman, J. M. (1990): „Über die Semiosphäre“. Zeitschrift für Semiotik 12 (4), 287–305.
Sachs-Hombach, K. (1999): Bildgrammatik. Interdisziplinäre Forschungen zur Syntax bildlicher Darstellungsformen. Magdeburg.
Siefkes, M. (2015): Sturm auf die Zeichen: Was die Semiotik von ihren Kritikern lernen kann. Schriften zur Kultur- und Mediensemiotik Online (1), 7–42.
Stöckl, H. (2004): Die Sprache im Bild – das Bild in der Sprache: zur Verknüpfung von Sprache und Bild im massenmedialen Text. Konzepte, Theorien, Analysemethoden (Linguistik-Impulse & Tendenzen). Berlin.
Stöckl, H. (2014): Semiotic Paradigms and Multimodality. In: Carey Jewitt (Hrsg.): The Routledge Handbook of Multimodal Analysis. Abingdon, New York, 274–386.
Stöckl, H. (2016): Multimodalität. Semiotische und textlinguistische Grundlagen. In: Hartmut Stöckl, Nina-Maria Klug (Hrsg.): Handbuch Sprache im multimodalen Kontext (Handbücher Sprachwissen 7). Berlin, Boston, 3–35.
Thibault, P. J. (1991): Social Semiotics as Praxis: Text, Social Meaning Making, and Nabokov’s Ada. Minneapolis, Oxford.
Wildfeuer, J./Bateman, J. A. (2018): Theoretische und methodologische Perspektiven des Multimodalitätskonzepts aus linguistischer Sicht (IMAGE 28), Minneapolis, Oxford.
[1] See for instance Kress and van Leeuwen 1996, 2001; Stöckl 2016.
[2] See Bateman/Wildfeuer/Hiippala 2017; Wildfeuer/Bateman 2018; Kress 2010; Diekmannshenke/Klemm/Stöckl 2011.
The Berlin Topoi conference “Pictures and Texts—Pictures as Text. Iconicity and Indexicality in Graphic Communication” (07/2017) was a first step into this direction with a stronger focus on text-based artefacts than is intended here. The organizers of this conference, Silvia Kutscher and Aleksandra Lapčić) will join us at Munich.
[3] For the understanding of “semiotic ideologies” used here, see Keane 2018.
[4] For semiotics-critical positions of the academic disciplines dealing with “aesthetic objects” see Halawa 2009 in detail and, in summary, Siefkes 2015, 7-16.
[5] See already Thibault 1991.
[6] Based on Ernst Cassirer’s conception of the culture of a society as a sign system of “symbolic forms” (Cassirer 1923-29).
[7] Jurij Lotman (et al. 1990) developed the concept of the “semiosphere” based on the idea of culture as a self-spun web of meaning advocated by anthropologists such as Clifford Geertz and sociologists such as Max Weber.