Abstracts appear in alphabetical order (speakers’ surnames).

Valérie Angenot

Eye-tracking Egypt. A semio-cognitive approach to reading Egyptian art

A few years ago, I established models of reading patterns used in the funerary decoration of tombs from the Old and the New Kingdom. I called these patterns “vectorialities”, which I defined as topological layouts of scenes translating spatio-temporal data on a two-dimensional surface, and inducing a specific gaze trail on the wall by an observer.

Until now, the semiotic analysis of the corpus has allowed the elaboration of theoretical models addressing the terms of conception of these funerary decorations. I would now like to build upon these results, by adopting an experimental approach aimed at analysing the terms of reception of the same set of images.
The project is called Eye-tracking Egypt. It is the first empirical endeavour aimed at studying the way the Egyptian paintings and reliefs were perceived and visualized in antiquity. It gathers a multidisciplinary team of specialists in art history, Egyptology, arts practice and 3D-modelling (Hexagram), visual anthropology, oculometry (NeuroLab), polysensoriality and cognitive psychology from UQAM, Freie Universität Berlin and Case Western Reserve University, Cleveland.
Leading such an experiment on a lost civilisation certainly is a great challenge, of which countering the lack of native “readers” is the most important aspect. It is therefore necessary to determine the constituent features of the “period eye” of an Egyptian living during Pharaonic times; in other words, to reconstruct the underlying mental patterns which give meaning and structure to visual practices. With the help of cognitive psychologists, we will thereupon be able to transfer what we know of the Egyptians’ reading habits into the mental models of our modern experimentors.  The physiological parameters underlying their act of reading will then be captured, recorded and measured using a portable eye-tracking device. The results will be confronted to the cognitive treatment of the scenes by untrained “naive” viewers, in order to determine where the differences lie in their respective mental models. Each of these data sets will be treated using multilevel models, which constitute the most recent statistical methods recommended in experimental psychology.

John Baines

Is all art multimodal? (With examples from ancient Egypt)

Human beings are (among other things) aesthetic animals, and they construct aesthetic environments. Within this overall context, they create genres of aesthetic objects and performances that are widely termed “art”—a much-contested designation. In most societies numerous such genres coexist. Art is made and received both between and within genres. Creators of art and their creations constitute the irreducible discourse of art, and an audience, even if small or imaginary, is comparably essential. All these three protagonists are aesthetic entities; hence, all art can be seen as multimodal. Even the most unimodal artistic products require multimodal points of reference to be meaningful. At the same time, products in many traditions incorporate multimodality within single genres, forming a second level of multimodality. Western genres, especially since the revival of perspective in the renaissance, are exceptionally unimodal, an unusual development that has clouded some art-historical approaches.
As an introduction to the conference, this paper is intended to outline the fundamental multimodality of art and to apply that point of departure to case studies from ancient Egypt.

Dominik Bonatz

Multimodal monsters in Assyria

In ancient Mesopotamia, creatures that combine human and animal or different animal features were depicted through the ages. Most of them were perceived as primordial or antediluvian beings that, through a complex process of mythological transformation and ritual incantation, became apotropaic and benevolent for human beings. The actual invisibility of these monsters was overcome by a fantastic imagery that covered all categories of visual art.
The paper raises the question what the visualization of the invisible means in terms of the multimodal function of an image. The difference to other images and subjects in art, which have ‘real’ points of references, is important. Taking the imagery of monsters in ancient Assyria as case study, it can be demonstrated how theses depictions indeed stimulated senses and emotions, which go far beyond the modern judgement of Assyrian art as being narrative, propagandistic or apotropaic. Especially in the context of the Neo-Assyrian palaces, monster figures fabricate a false reality that was perceived as real. The different semiotic processes in the same spatial context are extremely meaningful, they explain how the multimodality of images can help to transform reality into hyperreality.

Nikolaus Dietrich

Griechische Bronzespiegel als multimodale Artefakte

Bronzene Spiegel gehören zu den frühesten Objektgattungen, welche in der klassischen Archäologie systematisch publiziert wurden, und dies aus einem klar benennbaren Grund: Viele dieser luxuriösen Gegenstände wurden reich mit Bildern geschmückt, was sie neben anderen oft bildgeschmückten Gebrauchsgegenständen wie griechischen Vasen zu Quellen für die Kunstgeschichte der klassischen Antike werden ließ. Bereits Eduard Gerhard initiierte die illustrierte Vorlage aller bildtragenden etruskischen Bronzespiegel in einem umfassenden Corpus, dessen erster Band 1843 erschien. Während derartiges systematisches Sammeln, Ordnen und Publizieren von archäologischem Material einerseits die Grundlage für jede wissenschaftliche Beschäftigung bildet, so wohnt dieser Praxis andererseits eine Dialektik inne, die gerade am Artefakt ‚Spiegel‘ deutlich zutage tritt.
Die Aufbereitung des Materials zu Quellen der Kunstgeschichte reduzierte Bronzespiegel schließlich auf ihre Eigenschaft, Bildträger zu sein. In zweierlei Hinsicht wird man so dem Artefakt ‚Spiegel‘ nicht gerecht:
(1) Als bloße Bildträger macht man ihn zum passiven Gegenstand des Blicks, während die besondere Leistung des Spiegels ja darin besteht, Bilder aktiv produzieren zu können.
(2) Man reduziert den Bezug zum Gegenstand auf seine visuelle Dimension.
Wie es im Vortrag an einigen ausgewählten Beispielen aus der griechischen Antike dargelegt werden soll, erweisen sich Bronzespiegel im Kontext ihres Gebrauchs jedoch als multimodale Artefakte, welche in der visuell-haptischen Interaktion mit ihnen ein weitaus breiteres Spektrum an Sinnen involvieren. Nicht zuletzt der Bildschmuck von griechischen Spiegeln appelliert dabei an multisensorische Erfahrung. Nachdem griechische Bronzespiegel nach allem, was wir wissen, sowohl in ihrer kulturellen Semantik wie in ihrer lebensweltlichen Einbindung betont ‚weibliche‘ Gegenstände waren, soll diesen mit der (bildgeschmückten) Weinschale ein ‚männliches‘ Äquivalent eines ebenso multimodalen Artefakts gegenübergestellt werden, welches in vieler Hinsicht dem Charakter eines Spiegels nahekam.

Patrizia Heindl

The multimodality of Egyptian statues: Vivid memory?

“Give me your arm with libation and with incense when you see my image!”
“Put ointment on my head and beer in my mouth!”
“Do not hurry past me! Water and the breath of your words are more useful to me than millions of things!”
“Remember my name on the statue, when it is erected in the temple.”
Inscriptions on Egyptian statues give instructions for a variety of interactions. This is often associated with the wish for the name to be remembered or the hope that the statue will last. These interactions consist of various creations of scent, haptic impressions, auditory stimuli, and a combined perception of image, writing, and space. 
The overall concept of “statue” was employed to interact with diverse forms of existence, such as gods, the deceased, or living beings not otherwise present. By means of the multimodal interactions, these entities could thus be intensely perceived by the beholder(s), and their experience could be made a reality. The memory became literally vivid.
This presentation will address the question of whether, and to what extent, multimodal art was actively used to ground abstract ideas in experience. What effect would multimodality have on the evocation of memory?

Anna-Katharina Höpflinger

Luxury in modesty. A multimodal perspective on the early Neoclassical Berlin Iron jewellery

Detailed, exclusive and made of iron: In the late 18th and early 19th century, the so-called Berlin Iron jewellery, which was industrially produced in the area of Berlin, became fashionable. Elegant ladies wore these pieces from North America to Russia as part of their mourning attire. The material, unusual for jewellery of this time, was combined with neoclassical motifs showing gods, goddesses or mythological scenes. Through the extraordinary material, these references to antiquity were connected with concepts of modesty and humility. The paper will focus on this neoclassical iron jewellery and explore the multimodal communication processes of these pieces.

Kate M. Justement

Power through image in Neo-Assyrian officials’ cylinder seals

Most often analyzed through an historical lens as tools of administration or through an art historical lens with an eye to iconographic meaning, cylinder seals are a rich resource for studying society in the ancient Near East. This paper will consider a small group of Neo-Assyrian period (ca. 1000 – 612 BCE) cylinder seals that utilize a similar iconographic scene – that of the warrior god Ninurta shooting the mythical Anzu bird. In an attempt to more deeply contextualize these seals, some of which are directly attested as the seals of Neo-Assyrian officials, this paper will explore potential connections to various other media within the provincial administrative sphere. The myth of Anzu and the warrior aspect of the god Ninurta will be tentatively shown to be an often utilized allusion among the administration of the Neo-Assyrian empire through various avenues and to various effects.

Frank Kammerzell

Egyptian art and intersemiotic strategies for creating meaning in Picasso’s œuvre

The paper addresses the significance of Pharaonic Egyptian Art for the development of Western Modern Art — and is, I must confess, more about interpictorial than intermodal relationships.
In addition to the application of motifs in (decorative) Egyptian Revival Art and Orientalist painting, there are a few well-known examples of adaptations and transformations of Egyptian representational conventions (especially in the works of Paul Gauguin), but in general the direct impact of Egyptian Art on the outstanding movements in early 20th century Western painting is considered anything but substantial. Unlike African and Iberian sculpture, Egyptian Art has not been in the focus of studies investigating the sources of inspiration which played a role in the emergence of Early Cubism. Using a number of works by Pablo Ruiz Picasso, I will try to show that this view deserves re-examination, as there is ample evidence that he studied the principles of Egyptian multiperspectivity intensively.

Silvia Kutscher

Multimodale Analyse altägyptischer 
Artefakte: Entwurf einer Methode

Der Vortrag stellt eine an der Multimodalitätsforschung orientierte
Methode zur Analyse hieroglyphen-ägyptischer Kommunikate
vor. Ziel der Methode ist es, durch die Verschränkung der multimodalitätsorientierten semiotischen mit der ägyptologischen Perspektive neue Wege der Erforschung hieroglyphen-ägyptischer Artefakte zu präsentieren, die für beide Forschungsbereiche und den interdisziplinären Austausch fruchtbar gemacht werden können.

Aleksandra Lapčić

Ambimodalität als Kohäsionsmittel in ägyptischen Bild-Text-Komplexen

Dass die Ikonizität der ägyptischen Hieroglyphenschrift das Zusammenwirken zwischen Text und Bild in pharaonischen Hinterlassenschaften besonders begünstigt, ist ein (nicht nur) den Ägyptologen gut bekanntes Phänomen. In diesem Beitrag sollen Strategien der modalen Verflechtung zwischen dem Schrift- und dem Bildmodus an ausgewählten Beispielen aufgezeigt werden. Im Fokus stehen Zeichen an der Schnittstelle zwischen der graphemsprachlichen und der bildlichen Modalität, die ich als ambimodal definiert habe (Lapčić 2014). Diese Zeichen sind insofern in beiden Modi eingebunden, als sie – qua Bildzeichen – mit anderen Bildern interagieren, zugleich aber als Teil einer sprachlichen Äußerung mitgelesen werden müssen, und deshalb die eindeutige Zuordnung zu der einen oder anderen Modalität nicht möglich ist. Die Offenheit des hieroglyphischen Schriftsystems spielt dabei eine Rolle, da potentiell jedes Bild in das Zeicheninventar integriert werden konnte und dies in der Schreibpraxis nicht selten genutzt wurde. Auf die Wirkung solcher kohäsiven Praktiken auf den Lese- und Rezeptionsvorgang der Bild-Text-Komplexe soll eingegangen werden.

Lapčić, Aleksandra (2014): “Bild-Schrift-Gestalten des Göttlichen: multimodale Informationsverarbeitung im Amduat Thutmosis’ III.”, in: Neunert, Gregor, Katrin Gabler & Alexandra Verbovsek (Hgg.), Bild: Ästhetik – Medium – Kommunikation. Beiträge des dritten Münchner Arbeitskreises Junge Ägyptologie (MAJA 3), 7. bis 9.12.2012, Göttinger Orientforschungen, IV. Reihe: Ägypten 58, Wiesbaden: Harrassowitz, 169–192.

Nathan Morello

Multimodality of inscriptions in Assyrian monuments

This paper investigates the inscriptions in Assyrian monumental art from a multimodal perspective. Especially in the Neo-Assyrian period, the literary content of monumental inscriptions was not the only mean by which messages of royal propaganda could be conveyed. A whole range of visual characteristics of the inscriptions can be as well considered. The various ways in which an inscription was engraved on a monument had an impact that could affect audiences of different nature and levels of literacy. From the analysis of this phenomenon a sort of fascinating multimodality game emerges, which involved, for example, the choice of an inscription’s ductus, and the different ways of organizing the space between inscriptions, images and monument’s surface. Furthermore, one notices different degrees of interactions (from very simple to highly complex) existing between the inscriptions and the images which decorated the monuments, sometimes in apparent relation with the space in which the monuments were erected.

Daria Pezzoli-Olgiati

Places of visual performance. A reflection on “image”, “thing” and “place” in the study of religion

This contribution focuses on the general question about the spatial dimension of visual performance. Therefore, it contributes to the debate about “Multimodal Artefacts Analysis in Ancient Studies” with a theoretical reflection. Selected key concepts that are crucial for understanding an image as a socio-cultural practice are firstly introduced. We will discuss the concept of “image”, of the materiality of a medium and of place. In a second step these concepts are related to each other in order to offer a horizon to approach visual communication as spatial performance and its relevance for the study of religion. 

Beate Pongratz-Leisten

The spatio-social syntax of the object and multimodal literacy in the Assyrian city

In the vein of what has been coined either the iconic turn or pictorial turn on either side of the Atlantic and of subsequent intermediality and multimodality studies, during the last decade, big strides have been made also in ancient Near Eastern studies in going beyond the mere iconographic interpretation of the image. In combining the analysis of text and image on the object on the one hand and giving due attention to the original context where the object was found on the other a dialogue between art history and archaeology emerged that promoted the study of the multi-modal effects of the object linked to its placement or location within the urban landscape or a building.
Moreover, focusing on the action, procedures, rituals, and routines revolving around objects and monuments, cultural studies, art history, and ancient near Eastern studies have come to emphasize their performative quality in constituting the relationship between urban form and the actions and interactions of groups of people and institutions and shaping the way space is socially produced as these agents operate in multiple spheres. The Assyrian obelisks will serve as a case study to explore these various aspects.

Elisa Roßberger

The In-betweens. Tracking multimodality in human-divine encounters on Old Babylonian cylinder seals

Meeting your god/dess or king in early Babylonia was an honourable but decisive event since the divine or royal decisions made upon such an occasion could have repercussions on the rest of your life. Well-chosen mediators using multimodal channels of communication may have enhanced chances for a successful outcome.
The visual programme of cylinder seals, which were used as means of (multimodal) communication in ancient Iraq from the late fourth to the first millennium BCE, frequently refers to these pivotal moments. While the visual setting of encounter scenes appears relatively austere in the Ur III period (late third millennium BCE), it becomes increasingly populated by “in-betweens” during the Old Babylonian period (early second millennium). These miniature-sized, interspersed elements between the main figures are conventionally called “filling motifs” and considered decorative features to avoid horror vacui. I argue that they served as indices of the multimodal experience which characterized human-royal-divine encounters on a conceptual as well as on a ritual level. Differing in their degree of iconicity, “in-betweens” evoked acts of cleansing with pure water, anointment with fragrant oils, the shrieking sounds of animals and frenetic movement of dancers. But they could also take written form as short cuneiform inserts among the pictorial elements invoking the names of gods who served a similar intermediary function between human/divine spheres.

Marina Sartori

Ancient Egyptian tomb paintings as semasiographies

‘Multimodality needs to be seen as always having been the norm’ (Bateman et al. 2017, 15). This could not be truer for ancient Egypt, where script and figure always feature together on artefacts and especially on wall paintings. In fact, these two semiotic modes even share the same pictorial vocabulary and constantly play with each other, challenging the observer to distinguish them. Faced with such “fuzzy boundaries”, an approach that could prove successful in the analysis of ancient Egyptian wall paintings is that of semasiography.
This concept, as newly elaborated by Jackson (in Englehart 2013, 21–43), allows more flexibility, bringing together image, script, and notation, without however letting a purely linguistic approach dominate onto a visual one, and vice versa.
The paper will present case studies of tomb painting scenes to show how operational semasiographies, narrative semasiographies, as well as performative semasiographies coexist in the microcosm of the tomb decoration. The analysed material comes from selected New Kingdom tombs from the Theban Necropolis, personally investigated for my doctoral research during three archaeological campaigns with the Swiss Mission in Sheikh Abd el-Qurna. In particular, the paper will show how the use of emblems (signs recognizable as hieroglyphs but used in a non-textual context) is representative of narrative and performance-based semasiographies, whereas the offering lists offer examples of operational semasiographies. Visualizations with Adobe Illustrator will also make evident how the different semiotic elements interact with each other on the wall to create such complex systems – and how they enhance each other.

Bateman, John, Janina Wildfeuer & Tuomo Hiippala (2017): Multimodality: Foundations, Research and Analysis. A problem-Oriented Introduction, Berlin/Boston: De Gruyter. (As already reported also by Ms. Wildfeuer).

Jackson, Margaret A. (2013): “The Mediated Image: Reflections on Semasiographic Notation in the Ancient Americas”, in: Englehardt, Joshua (ed.), Agency in Ancient Writing, University Press of Colorado: Boulder, 21–43.

Ilona Schulze

Text-Bild-Kommunikate zwischen Aggregation und Integration. Eine Annäherung an die Bildlinguistik

Kombinationen aus Text/Sprache und bildlichen/grafischen Elementen (Kommunikate) sind der zentrale Untersuchungsgegenstand der Bildlinguistik. Im Zentrum des Interesses steht dabei die Frage nach dem Beitrag des jeweiligen Modus zur Gesamtaussage, die zwischen Aggregation und Integration variieren kann (um eine Prägung von Wolfgang Raible zu übernehmen), in der Regel aber einen informationellen Mehrwert im Vergleich zu den Einzelinformationen der jeweiligen Modi aufweist. Wesentlich für das Verständnis der Funktionsweise solcher Kommunikate ist das Wissen um die unterschiedlichen Verarbeitungsweisen von Sprache und Bild sowie um die Stärken und Schwächen beider Modalitäten in der Informationsvermittlung. Während die Verwendung von Sprache und Bild sowohl einzeln als auch in Kombination epochen- und kulturspezifischen Einflüssen unterliegt, sind die grundlegenden kognitiven Verarbeitungswege zeitstabil, universell und besonders nach gestaltpsychologischen Parametern organisiert. Diese Parameter können sich auch in strukturellen Merkmalen der Text-Bild-Beziehung äußern, die gegebenenfalls auch durch entsprechende ‚Lesehilfen‘ kodiert sind. Von Interesse sind auch unmittelbare Verschränkungen von Schrift und Bild (etwa das Rebus-Prinzip), die wie andere Optionen der Bildlinguistik auch auf dem gemeinsamen Modus der Sinnesmodalität ‚Visualität‘ beruhen. Der Vortrag wird sich den genannten Aspekten widmen und diese an Beispielen illustrieren. Fallweise wird auf historische Dimensionen eingegangen und so die Anschlussfähigkeit des Forschungsbereichs Bildlinguistik an außersprachwissenschaftliche Disziplinen aufgezeigt.

Gebhard Selz

Multimodalities and signification in an ancient Near Eastern perspective. Incongruent corpora or non-discrete texts?

My starting point is the hypothesis that our senses – their aesthetic properties – provide the basics of the visual and aural codes and representations; other senses are ignored here. I would like to exemplify then how these two largely independent aesthetic systems interact with each other and influenced the modes of discrete representations – the codes – in early Mesopotamia, specifically in art and writing. Finally, I would like to exemplify the salient role of analogies and metaphors in early history showing that transmedial perspectives frame the emic background of all Mesopotamian hermeneutics, reflecting non-discrete culturally determined (hyper-)texts.

Martin Siefkes

Cultural semiotics and meaning attribution to artefacts

Semiotics of culture (Kultursemiotik) investigates the role of signs and sign systems in human culture. Influential theories in this area have been developed by Roland Barthes, Juri Lotman and Roland Posner, among others. While traditionally meaning had primarily been located in signs and sign systems such as language, images, or religious symbols, cultural semiotics has developed various approaches that connect the material aspects of cultures with the layers of meaning and interpretation that permeate our cultures. They complement the more specific approaches to artefact semantics developed in archaeology, anthropology, and art history, by outlining the general principles explaining how meanings are attributed to human-made (or even natural) objects, making them into signs.
The contribution first presents Roland Posner’s semiotic theory of culture. While this theory offers a precise terminological approach to cultural processes, it offers only limited insight into artefact semantics. This gap is filled by another approach that postulates a range of different but interacting principles for meaning attribution. The semiotic theory of artefact semantics considers both processes on the social level (e.g. artefact meanings based on function or defined by convention) and individual meaning attribution (e.g. objects acquiring meaning through our lived experience).

Christoffer Theis

Der multimodale Bes – Kommunikationsstrategien eines persönlichen Schutzgottes

Im Fokus des Vortrags soll eine bisher kaum untersuchte spezielle Ausformung von Bes stehen, die in der Literatur u. A. als der mehrköpfige oder der polymorphe Bes bekannt ist. Es handelt sich hierbei um ein schutzmächtiges Wesen, welches als Hauptkopf den eines Bes trägt und aus dessen Kopfseiten mehrere Tierprotomen austreten. Aus Ägypten sind derzeit etwa 300 Statuetten, Reliefs, Papyri und Gemmen bekannt, welche diese spezielle Ausformung darstellen.

Im Vortrag sollen mögliche Ritualzurüstungen für und Umgangsformen mit dem mehrköpfigen Bes behandelt und diskutiert werden. Wie könnte mit den verschiedenen Objekten, die einen Bes mit drei, vier, fünf, sechs, sieben, acht, neun, zwölf oder vierzehn Köpfen darstellen, umgegangen worden sein? Was ist anhand weiterer Funde rekonstruierbar? Hierzu werden spezifische Kombinationen von Text, Bild und weiteren Artefakten kombiniert um die (mögliche) Multimodalität der Bes-Gestalt(en) in einem Ritual deutlich werden zu lassen. Die Kommunikation von und mit dem Wesen kann hierbei auf textlicher, sprachlicher, räumlicher und visueller Weise erfolgen.

Lorenzo Verderame

Texts on statues and statues in texts: Evidence of multimodality in third millennium Mesopotamian statues

In this paper I investigate the relationship between text and statues in third millennium Mesopotamia from a multimodal perspective. The paper is composed of two parts.In the first one, I deal with texts mentioning the statues. I discuss the rich and multifaceted information provided by the written sources that contrast the scanty archaeological evidence. I analyse different aspects of statues such as materials and fashioning, and their interaction with space and sensory world.In the second part, I consider writing on statues, from short epigraphs to long narratives. I analyse the relationship among the donor, the text, and the statue and I advance some hypothesis on the disposal of inscribed statues according to their inscriptions.

Janina Wildfeuer

Multimodality’s terminological jungle – and a way through

Almost 30 years of interdisciplinary research on multimodality have
brought about a very diverse terminology and a variety of concepts and definitions for key terms such as semiotic modes, texts and media. While all of them certainly have their raison d’être, the
terminological jungle often leaves us with many open questions,
especially when several disciplines meet and borrow from each other.
In our textbook (Bateman et al. 2017/Wildfeuer et al. 2020), we
therefore provide a more robust foundation for multimodality as an
interdisciplinary and ideally empirical endeavour and give clear
definitions for the most important concepts. In my talk, I will
summarize some of these and particularly focus on the often
problematic distinction between modes and media as well as between ‘multimodality and ‘multimediality’. With a critical look at and description of some examples from the conference programme, I hope to show the applicability and usefulness of this approach for a broad range of disciplines and research contexts and to clarify some misunderstandings and uncertainties.

Bateman, John; Wildfeuer, Janina; Hiippala, Tuomo (2017): Multimodality. Foundations, Research, Analysis. A Problem-Oriented Introduction. Berlin, New York: de Gruyter.
Wildfeuer, Janina; Bateman, John; Hiippala, Tuomo (2020): Multimodalität. Grundlagen, Forschung, Analysis. Eine problemorientierte Einführung. Berlin, New York: de Gruyter.