Multimodal analysis within an interactive software environment: critical discourse perspectives

Critical discourse analysts are increasingly required to account for multimodal phenomena constructed through language and other resources (e.g. images, sound and music) and to relate high-level critical insights on the social motivations of these texts to their realizations in low-level expressive phenomena, and vice versa. In this paper, we use interactive software resources for critical multimodal discourse analysis (O'Halloran, K.L. (2011b). Multimodal analysis and digital technology. In A. Baldry & E. Montagna (Eds.), Interdisciplinary approaches to multimodality: Theory and practice. Readings in intersemiosis and multimedia (pp. 21–34). Campobasso: Palladino; O'Halloran, K. L., Tan, S., Smith, B. A., & Podlasov, A. (2010). Challenges in designing digital interfaces for the study of multimodal phenomena. Information Design Journal, 18(1), 2–12; Smith, B. A., Tan, S., Podlasov, A., & O'Halloran, K.L. (in press). Analyzing multimodality in an interactive digital environment: Software as metasemiotic tool. Social Semiotics) to help in achieving these aims. The field of critical discourse studies, itself being interdisciplinary, along with its holistic approach, is well placed to take advantage of interactive software, as these resources encourage the application, exploration and correlation of various analytical perspectives at different levels of description. Drawing on the analysis of a short video advertisement, we demonstrate how the interactive (multimodal) digital environment is one in which the discourse analyst can effectively draw upon different traditions of analysis, including ‘mainstream’ and social semiotic traditions, as well as other traditions such as media studies, to interpret dynamic audiovisual media texts in a critically self-reflexive manner.