On July 9-10, the Martin Centre for Appliable Linguistics (hereafter “Martin Centre”) successfully held the 2021 Annual Seminar on Multimodal Analysis. The seminar was conducted by Prof. Kay O'Halloran (University of Liverpool, UK), an internationally renowned multimodal expert and founder of the MMAV multimodal software, Dr. Yang Xiran and Dr. Qu Tao, young scholars in domestic multimodal research. The seminar was held both offline and online, and a dozen of teachers, master students, and doctoral students from universities in China and abroad attended.

Professor Wang Zhenhua, Director of Martin Centre of Shanghai Jiao Tong University, delivered the opening speech. Based on an informative summary of the contents discussed in the three-day seminar on appliable linguistics, Prof. Wang provoked our thinking about the knowledge and values revealed by multimodality through the light show on the Bund in Shanghai for centennial of the founding of the CPC. Before closing the speech, Prof. Wang wished the Seminar on Multimodal Analysis a great success.

Professor Kay O'Halloran gave two presentations in the afternoon of July 9-10, introducing related concepts of multimodality, research approaches, analytic software, and multimodal analysis in the era of Big Data. When introducing the concepts relating to multimodality, she compared the focus of different branches of multimodal research by analyzing a Guardian report of the Vietnam epidemic. In the practice session, she demonstrated the operating steps to use Multimodal Analysis Video (MMAV) and Multimodal Analysis Image (MMAI), and introduced the methods to interpret the data. In the section of multimodal big data analysis, she presented the automatic identification and analysis of multimodal content supported by natural language processing and image processing models such as DenseCap, and demonstrated the content retrieval and sentiment analysis of social and print media on Multimodal Analysis Platform (MAP). The technical guidance provided by Prof. Kay O’Halloran has significant theoretical and practical implications for future multimodal research.  

Prof. Kay O’Halloran

The morning session on July 9-10 was presented by Dr. Yang and Dr. Qu, covering static image-text multimodal analysis and dynamic video multimodal analysis.

Dr. Yang gave a keynote speech on "Multimodality: From Concepts to Applications". Dr. Yang introduced the theoretical frameworks of multimodal analysis, focusing on the objectives, terminology, systems and research design of multimodal research under the paradigm of Systemic Functional Linguistics. She discussed the major points and methods of analyzing the multimodal meaning of images in the frameworks of FOCALIZATION, AMBIENCE, and FRAMING, and pointed out the possibility and necessity of modifying and improving the existing analytical systems.

Dr.Yang Xiran, Martin Centre

Dr. Qu Tao gave a keynote presentation on "Multimodal Discourse Analysis: A Case Study". Following the four steps of determining the research objective, proposing research questions, selecting theoretical framework, and analyzing and interpreting data, Dr. Qu Tao analyzed the multimodal interaction of conflicts in TV dramas, introduced the multimodal construction of conflict processes from the perspective of cognitive appraisal, and elaborated the dynamic construction of conflict in civil court trials. The analytic steps introduced by Dr. Qu are instructive for conducting multimodal discourse studies.

Dr. Qu Tao, Martin CentreDr. Yang Xiaoran and Dr. Qu Tao also conducted Q&A sessions to help solve problems encountered by the participants while using MMAV and MMAI.

The two-day seminar came to an end in the afternoon of July 10th. Prof. Zhenhua Wang, Director of Martin Centre, delivered a closing speech and awarded the certificates of participation to the offline participants.