Dalarna University's logo and link to the university's website

Narrative media music, music used for narrative purposes in multimedia such as film, television or computer games, is becoming one of the largest sources of musical experience in our daily lives. Though typically experienced on an unconscious and unreflected level, this kind of music actively contributes narrative meaning in multimodal interplay with image, speech and sound effects. Often, what we (think we) see is to a large degree determined by what we hear. Using Halliday’s (1978) metafunctions of communication as a starting point, two short film scenes (from Jaws and The Secret of My Success) are examined, with a focus on the intermodal relationships of music and image. The examples illustrate how musical and visual expressions combine to form multimodal statements where the whole is certainly different than the sum of the parts.

This study investigates usage and knowledge of musical narrative functions in contemporary multimedia. A group of young adolescents were given the task of adapting musical expression, using the non-verbal research tool REMUPP, to fit different visual scenes shown on a computer screen. This was accomplished by manipulating seven musical parameters: instrumentation, tempo, harmonic complexity, rhythmic complexity, register, articulation and reverb. They also answered a questionnaire giving information about their musical training and media habits. Numerical data from the manipulation of the musical parameters were analysed to search for tendencies within the group with regard to the musical expression in relation to the different visual scenes shown. The results showed a large degree of in-group consensus regarding narrative functions of music, indicating knowledge about musical narrative codes and conventions. Also, the results were clearly influenced by factors such as the participants' musical training, gender and habits of music listening, playing computer games and watching movies — highlighting the complexity of learning and pointing to the impact of the increasing availability of narrative media on our attitudes and knowledge.