Multimodal discourse analysis is an approach to discourse that takes into account multiple modes of
communication, such as text, color, images, sound, and physical gestures, and how they interact with
one another to create semiotic meaning]. It is a method of discursive analysis that looks at not just how
individual modes communicate, but how they interact with one another to create meaning. Multimodal
discourse analysis is an important tool for understanding the complexities of communication and can be
applied to a variety of contexts, from small group conversations to large-scale media productions. The
first step in conducting a multimodal discourse analysis is to identify the modes of communication that
are being used, including the written language, visual images, sound, and physical gestures that are
being used in the context. By examining these elements, researchers can gain insight into how meaning
is created and interpreted, which can help them better understand how people interact and how they
interpret and construct meaning. Some examples of multimodal discourse analysis in practice include:
1. Analyzing social semiotic situations such as films, images, TV programs, and videos
2. Introducing a multimodal perspective to emotional variables in second language acquisition
education, where systemic functional multimodal discourse analysis is used to address the
multimodality of discourse
3. Analyzing how verbal and visual signs relate within a text to create a meaningful message, using
three interrelated systems: informational value, salience, and framing
4. These examples demonstrate the diverse applications of multimodal discourse analysis, from
analyzing visual media to studying language acquisition.
The key concepts in multimodal discourse analysis include:
1. Multimodality: This refers to the use of multiple modes of communication, such as text, color,
images, sound, and physical gestures, and how they interact with one another to create semiotic
meaning
2. Semiotics: This is the study of signs and symbols and how they are used to create meaning
3. Discourse: This refers to the use of language in social contexts and how it is used to create
meaning
4. Social semiotics: This is the study of how meaning is created through the use of signs and
symbols in social contexts
5. Informational value: This refers to the way in which information is conveyed through different
modes of communication
6. Salience: This refers to the way in which different modes of communication are used to highlight
certain aspects of a message
7. Framing: This refers to the way in which different modes of communication are used to structure
a message and create meaning
8. These concepts are central to understanding how multimodal discourse analysis works and how
it can be applied to different contexts.
Here are some examples of multimodal analysis in various contexts:
1. Video analysis: Multimodal analysis can be used to analyze video content by considering the
different modes of communication, such as text, images, facial expressions, and gestures, within
the video.This approach helps researchers to better understand the meaning-making process in
video and can be particularly useful in educational contexts, where instructors can analyze how
students engage with multimodal forms of expression
2. Conceptual art: Conceptual art is a form of artistic expression that often involves the use of
multiple modes of communication, such as text, images, and sound. By examining these different
modes, researchers can gain insights into the artist's intentions and the audience's
interpretations of the artwork
3. Online environments: Multimodal discourse analysis can be applied to online environments,
where text, images, color, and other visual elements interact to create meaning
4. This approach can help researchers understand how meaning is constructed and interpreted in
digital contexts, such as social media platforms, websites, and online games
5. Teaching methods: Multimodal teaching methods involve using different sensory modes to cater
to various learning styles. By incorporating multiple modes of communication, such as visual,
auditory, kinesthetic, and reading/writing, teachers can create a more inclusive and engaging
learning environment for students.
Multimodal discourse analysis can be applied to teaching in various ways. For example, it can be
used to analyze how different modes of communication, such as language, images, and gestures,
are used in teaching materials and how they interact to create meaning. Multimodal discourse
analysis can also be used to analyze pedagogic discourse in the classroom, including how
teachers use different modes of communication to engage students and how students respond
to these modes. By using multimodal discourse analysis in teaching, educators can gain insights
into how students learn and how to create more effective teaching materials and strategies that
cater to different learning styles
6. Multimodal texts: Multimodal texts combine two or more modes of communication, such as
print, spoken word, sound, and images, to create meaning. Examples of multimodal texts include
street signs, advertisements, and instructional materials that use a combination of linguistic,
visual, gestural, spatial, and aural elements to convey meaning
Citations:
https://www.diggitmagazine.com/wiki/multimodal-discourse-analysis
https://onlinelibrary.wiley.com/doi/abs/10.1002/9781405198431.wbeal0813.pub2
https://www.frontiersin.org/articles/10.3389/fpsyg.2022.1016441
https://www.amazon.com/Multimodal-Discourse-Hodder-Arnold-Publication/dp/0340608773
https://speakai.co/how-to-conduct-multimodal-discourse-analysis/
