Dalarna University's logo and link to the university's website

In contemporary society, important activities of our daily lives are increasingly audiovisually mediated such as in film, television, games, advertising etc. In these activities, principles for storytelling and communication are used, building on socially developed conventions (Wingstedt, 2008). The voice (dialogue or voice over) in audiovisual modes of communication is often used as a resource for contributing a sense of presence, engagement, identification and immersion. This paper presents the design and preliminary results of a pilot study, where the purpose was to investigate how narrative principles for audiovisual storytelling can be applied to online learning environments. Focus is primarily on how resources for mediating the human voice can contribute to a sense of presence and immersion in the learning situation. Theories of Multimodality (Kress, 2010) and Social semiotics (van Leeuwen, 1999, 2005, and others) are used as a point of departure. In using Halliday’s (1978) three metafunctions of communication as a model – describing communicational meaning as being ideational (content), interpersonal (interaction) and textual (structure) – this study focuses on how interpersonal meaning is realized through the mode of sound.  Empirical data was collected in two kinds of online learning situations: 1) Japanese language lessons, performed in real time, and; 2) recorded lectures in the course Analysis of music productions (in the Music- and sound design programme). As one important purpose of this pilot study was to evaluate the validity of the research method, it was decided to include these two disparate but common forms of online teaching/learning. In all, 16 lessons were evaluated. Half of the lessons were carried on “as usual”, using standard technical solutions for sound production and transmission. The other half of the lessons were “produced”, where aspects such as frequency range, sound level, noise, pitch and proximity were treated using professional audio technology and methods. The preferred sound ideal was not necessarily ”naturalism” but rather the heightened sense of ”hyper realism” often heard in film, where the voice sound is used to achieve a sense of immersion, continuity and presence. Each of the 16 lessons was followed by an online survey, where the students rated different communicational aspects of the session. In all, 147 individual surveys were collected. The preliminary results of these surveys will be presented, and an evaluation of different aspects of the research methodology will be discussed.