主页 International Journal of Multilingualism Movie got your tongue? Effects of language switching on film reception
报告问题This book has a different problem? Report it to us
Check Yes if Check Yes if Check Yes if Check Yes if
you were able to open the file
the file contains a book (comics are also acceptable)
the content of the book is acceptable
Title, Author and Language of the file match the book description. Ignore other fields as they are secondary!
Check No if Check No if Check No if Check No if
- the file is damaged
- the file is DRM protected
- the file is not a book (e.g. executable, xls, html, xml)
- the file is an article
- the file is a book excerpt
- the file is a magazine
- the file is a test blank
- the file is a spam
you believe the content of the book is unacceptable and should be blocked
Title, Author or Language of the file do not match the book description. Ignore other fields.
Change your answer
International Journal of Multilingualism ISSN: 1479-0718 (Print) 1747-7530 (Online) Journal homepage: http://www.tandfonline.com/loi/rmjm20 Movie got your tongue? Effects of language switching on film reception Xavier Aparicio & Dominique Bairstow To cite this article: Xavier Aparicio & Dominique Bairstow (2016) Movie got your tongue? Effects of language switching on film reception, International Journal of Multilingualism, 13:1, 106-119, DOI: 10.1080/14790718.2015.1037306 To link to this article: http://dx.doi.org/10.1080/14790718.2015.1037306 Published online: 08 May 2015. Submit your article to this journal Article views: 52 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=rmjm20 Download by: [University of California, San Diego] Date: 26 February 2016, At: 08:18 International Journal of Multilingualism, 2016 Vol. 13, No. 1, 106–119, http://dx.doi.org/10.1080/14790718.2015.1037306 Movie got your tongue? Effects of language switching on film reception Xavier Aparicioa,b* and Dominique Bairstowa,b Downloaded by [University of California, San Diego] at 08:18 26 February 2016 a Laboratory Epsylon EA 4556, University of Montpellier 3, Place Albert 1er, 34000 Montpellier, France; bDepartment of Psychology, University of Paul Valery Montpellier, Route de Mende, 34090 Montpellier, France (Received 17 December 2014; accepted 23 March 2015) Cinema is in part a reflection of our society and, in these times of cultural mix, it is more and more common to find different language communities appearing on-screen together. Thus, it is not unusual to have to process (voluntarily or not) more than one language throughout the day. From a cognitive point of view, language switching is widely studied as it enables to observe interferences or facilitations across languages, modulated by an individual’s knowledge of those languages as well as by the orthographic proximity between them. F; ilm comprehension is also studied, particularly depending on the languages used on-screen and those known by the viewer. This study proposes to unite both paradigms by using a multilingual film (English and Spanish dialogues), either with subtitles (English or French) or without, to analyse the effects of language switching on visual and dialogue processing, depending on the viewers’ knowledge of the languages provided. The results obtained in this study should increase our understanding of languages through an original approach. Keywords: foreign language acquisition; bilingualism; code switching; multilingualism; speech perception; reading comprehension 1. Introduction 1.1. Films and subtitles Films provide a very interesting and ecological context for studying languages. Indeed, there is often a close parallel between cinema and everyday situations, which are more and more multicultural, and therefore increasingly multilingual. As has often been noted, movies were silent to begin with and, with the arrival of sound and dialogues, their dissemination was made difficult by language barriers. In order to overcome this issue, various audiovisual translation techniques were established, either by replacing the original soundtrack with a new one in the targeted audience’s language (dubbing) or by adding a translation of the dialogues, orally (voice-over) or in a written form (subtitles). This additional linguistic information can have consequences on the viewer’s experience of the film (Lavaur & Bairstow, 2011). Indeed, a movie is made up of many different sources of information, audio and visual, that can convey either linguistic (respectively, dialogues and written information) or non-linguistic information (respectively, music or noises and images). Furthermore, this data will be in constant interaction *Corresponding author. Email: firstname.lastname@example.org © 2015 Taylor & Francis Downloaded by [University of California, San Diego] at 08:18 26 February 2016 International Journal of Multilingualism 107 with the viewer’s own knowledge concerning films in general and, more particularly, the specific situation shown on-screen. Therefore, the viewing of a movie involving different language communities should impact its comprehension, with regards to the language knowledge/experience of the viewer. The assumption is that comprehension difficulty originates in the presence of language switching within the dialogues of the movie. In many studies, language switching issues are investigated in an artificial way, with single word recognition tasks in different languages (see Aparicio & Lavaur, 2013) or within sentence reading (Bultena, Dijkstra, & Van Hell, in press) or hearing (Costa & Santesteban, 2004). Here, the use of a multilingual movie provides a more ecological approach for understanding language switching mechanisms. In the case of a multilingual film, the presence of more than one language in the dialogues can give indications as to the situation in which the movie is set or the ethnic/ social position of the characters, amongst other things. When this type of film is presented to another language community to the ones on-screen, comprehension may be affected and different subtitling solutions are possible. On the one hand, if the viewers do not understand any of the languages used in the dialogues, the whole movie (independently from the language spoken) can be subtitled in the audience’s own language, implying the presence of three different languages on the screen (two spoken, one written). On the other hand, if one of the spoken languages is known to the viewers, then only the other will need subtitling. In this case, two languages will be presented onscreen, one spoken, the other alternatively spoken (dialogues) or written (subtitles). This can make for very interesting language switching situations and provide important data concerning the effects of these switches in non-laboratory contexts. 1.2. Multilingual films The complexity of the situation in which a viewer is placed when seeing a multilingual film is self-evident when all the different types of information provided by such a movie are taken into consideration. Indeed, not only does visual and linguistic data need to be simultaneously processed and linked to previous knowledge, but different languages must also be understood. When studying multilingualism and multilingual information processing, this is very interesting as it offers a fairly ecological setting, resembling real-life situations (Brunick, Cutting, & Delong, 2013; Carroll, 1996; Merleau-Ponty, 1947; Mitry, 2001; Zacks & Magliano, 2011), in which various languages are used. What is more, the number of films requiring the viewer to process more than one language has been steadily increasing since the 1980s (Heiss, 2004, with examples for Germany). This can easily be explained by the fact that movies have a long history of representing the way in which our societies evolve, in this case in relation to mass immigration, cultural mix and the side-by-side coexistence of various language communities in one same country (Baldo, 2009). These factors, linked to the strong increase in people speaking more than one language (Lemhöfer, Dijkstra, & Michel, 2004), imply that this line of research is particularly relevant, especially in the field of cognitive psychology. 1.3. Subtitled film comprehension and language switching Understanding a film is an amazing feat of cognitive processing (Zacks & Magliano, 2011), implying that the additional processing of different languages and linguistic sources (spoken and written, in the case of subtitling) is even more exceptional. Downloaded by [University of California, San Diego] at 08:18 26 February 2016 108 X. Aparicio and D. Bairstow Indeed, studies have shown that the very presence of subtitles can lead to an automatic reading behaviour, even when they are unnecessary to comprehension (d’Ydewalle, Praet, Verfaillie, & Van Rensbergen, 1991), which can induce, in certain cases, a partial loss of information (Lavaur & Nava, 2008). On the other hand, this visual input of the linguistic information (subtitles) can also prove to be a benefit as it can add sense to the spoken input (Marian, 2009). Moreover, these contrasting effects of subtitles on comprehension have been shown to be strongly dependent on the viewers’ fluency in the languages onscreen, both spoken and written (Lavaur & Bairstow, 2011). Another line of research in cognitive psychology investigates language switching which can happen in many different everyday situations. Language switching is a very common phenomenon in populations speaking different languages, as is the case in Luxembourg for instance, where at least three different languages are used (Luxembourgish, French and German). From a psycholinguistics point of view, it is widely admitted that reading or hearing a word initiates a matching with the abstract representations of the words known by a person. These representations are stored inside the brain in a vast database called the mental lexicon (van Heuven, Dijkstra, & Grainger, 1998). There, we store not only the words, but also all the information we possess about their characteristics (orthographic, phonologic, semantic, syntax), as well as the language they belong to in the case of multilinguals (Dijkstra, 2005; Grainger & Beauvillain, 1987). The most widely accepted hypothesis in psycholinguistics is that the access to the multilingual lexicon is non-selective, or language independent, i.e. an incoming word will first activate all the lexical candidates from the different languages before being identified (Chauncey, Grainger, & Holcomb, 2008; Christoffels, Firk, & Schiller, 2007; Dijkstra & Van Hell, 2003; Lemhöfer et al., 2004; Thomas & Allport, 2000; Van Heuven et al., 1998). According to the Bilingual Interactive Activation model (Grainger & Dijkstra, 1992; van Heuven et al., 1998), this implies that two or more languages could be active at the same time, leading to possible interferences between them (see Figure 1). These interferences could induce the intrusion of a word in a different language from the one spoken (unexpected switches), during language production for instance. This raises the question of language control in multilinguals, which has a direct impact on language switching. The term code switching is intended to designate the process whereby speakers move from one language to another either within a single utterance or between one utterance and the next in the same interaction (Bentahila & Davies, 1995; Cantone, 2007). It is different to code-mixing, because it refers to all cases where lexical items and grammatical features from two languages appear in one sentence (Muysken, 2000). To be efficient in the relevant language, it is necessary for multilingual people to reduce the interferences from the words of the other languages (Dijkstra & Van Hell, 2003). Multilinguals mostly succeed in this task with a relative ease. As mentioned before, it is not unusual to notice the intrusion of words from a different language to the one currently spoken within a conversation, intentionally or not (Meuter, 2005; Moreno, Federmeier, & Kutas, 2002). It is important to notice that the intrusion of words could occur in a totally unexpected way, eliciting a greater difficulty of comprehension for the speaker that have to process the switch utterance. According to Moreno et al. (2002), the intrusion of a word from a different language originates in a retrieval problem of a word in the language currently spoken, leading to a mix-up with a translation equivalent, used in order to preserve the meaning of what is being said. For the speaker, language switching seems easy to operate so as to maintain the intended meaning. For the person at the receiving end of this switch, however, it is more difficult to process the word in the Downloaded by [University of California, San Diego] at 08:18 26 February 2016 International Journal of Multilingualism 109 Figure 1. The Bilingual Interactive Activation Model, (BIA, adapted from Grainger, Midgley, & Holcomb, 2010). For languages sharing the same alphabet (here, English and French), a printed word stimulus can activate whole-word orthographic representations from both languages as a function of their orthographic overlap with the stimulus. All activated representations enter in the competition for word identiﬁcation, but the probability that a given word is in fact the stimulus is regulated by the probability that the stimulus is a word in one or the other language (as indexed by the activation of language nodes). new language than the word expected in the original language. Indeed, language switching is often unexpected or unintentional and not necessarily relevant in the context of the conversation. Nevertheless, it does frequently occur, and not only during verbal exchanges, but also in reading (Altarriba & Basnight-Brown, 2007), making multilingual films with subtitles a very interesting and ecological tool for studying language switching aspects. 1.4. Aims The main aim of this work was to investigate film comprehension in a multilingual context, while evaluating the effects of language switching on the processing of movie information. Indeed, multilingualism is a very up-to-date issue as an increasing number of people speak, or at least have some understanding, of more than one language, which increases the occurrence of language switching situations. In addition to this, studies looking into the mechanisms of language switching have generally measured effects in highly specific situations using visual presentation of single word lists and rarely with auditory presentation. Although the data obtained in this way is very interesting since it gives insight into very precise cognitive processes, whether or not these effects can be generalised to everyday situations remains an issue. Bearing this in mind, we carried out an exploratory study in order to examine the effects of language switching on movie comprehension, depending on different factors. To begin, the number of languages on the screen was manipulated by adding (or not, depending on the condition) subtitles to a film extract in which two languages were used in the spoken dialogues (US-English and Mexican Spanish). In this way, either the film was presented without subtitles or with either French or English subtitles. Concerning the Downloaded by [University of California, San Diego] at 08:18 26 February 2016 110 X. Aparicio and D. Bairstow participants, different profiles were taken into account, depending on their knowledge of the three languages (French–English–Spanish trilinguals, English monolinguals and French monolinguals). When the non-subtitled version of the film is shown to trilingual participants, their knowledge of both languages used in the dialogues should enable a good global comprehension of the movie. However, this comprehension may be somewhat affected when language switches occur, since these switches will take place between their two non-dominant languages (L2 and L3, see Aparicio & Lavaur, 2013). On the other hand, when this trilingual population is shown the French-subtitled version, all three of their languages will be on-screen, presented two at a time (English dialogues with French subtitles, alternated with Spanish dialogues with French subtitles). In this case, the viewers can use different viewing strategies: these strategies can include either choosing to ignore the subtitles as they understand the languages used in the dialogues (although a certain amount of automatic reading is unavoidable); ignoring the dialogues in order to concentrate only on the subtitles which are always in the same language; or use the subtitles selectively when they come across spoken words they do not understand. Either way, it is in this situation that language switches can be expected to be the most costly for comprehension since three different languages will be presented on-screen, requiring the use of successive inhibition/activation mechanisms. When examining monolinguals’1 French speakers watching the French-subtitled movie should achieve a good level of comprehension by reading the subtitles and watching the images. Therefore, language switching should have little effect on their understanding of the sequence, even though a certain amount of audio information processing implies that these participants should notice when these language switches occur. Given that they are reading the subtitles in order to understand the dialogues, we can also make the hypothesis that their reading activity could lead them to pay less attention to visual information (especially background visual data). Finally, with regards to English monolinguals seeing the English-subtitled version of the movie, language switches in the dialogues (from English over to Spanish, and vice versa) should compel the viewers to refer to the subtitles when the spoken language is Spanish. It can therefore be predicted that this condition will provide the biggest effect on overall film comprehension since the viewers must continuously adapt their viewing strategy throughout the movie (alternatively watching and listening or watching and reading). 2. Experimental study 2.2. Method 2.2.1. Materials An extract from the movie ‘Real women have curves’ (P. Cardoso, 2002) lasting 5 minutes and 10 seconds was selected for this experiment on the basis of strict criteria (such as languages used in the dialogues, speech intelligibility, time spent speaking each language, number of switches, number of characters on-screen and action/dialogue ratio, amongst others). Three versions of the extract were prepared, the non-subtitled version with English (US) and Spanish (Mexican) dialogues, the French-subtitled version and the English-subtitled version. In the case of these last two versions, it was made sure that the subtitles were an exact translation of the spoken dialogues. International Journal of Multilingualism 111 Downloaded by [University of California, San Diego] at 08:18 26 February 2016 A multiple-choice questionnaire concerning the film was created, including 32 items referring to the images (16 visual questions) and to the dialogues (16 questions). Each question was provided with one correct answer, three distractors and one possibility of answering ‘I don’t know’. A second questionnaire concerning personal data was also presented to the participants, including questions about language use and proficiency, as well as auto-evaluation on a five points Likert scale (self-rating). The self-rating includes questions about reading comprehension, reading frequency, oral production and oral comprehension for the three languages (French, English and Spanish). 2.2.2. Participants About 48 participants took part in this study, divided into groups depending on their language knowledge and the version of the film they were shown (see Table 1). In this way, two groups of 12 trilinguals (L1 French, L2 English and L3 Spanish; seeing either the non-subtitled movie or the French-subtitled version) and two groups of monolinguals (12 French monolinguals seeing the French-subtitled version and 12 English monolinguals seeing the English-subtitled version) were used in this study. 2.2.3. Procedure To begin, participants were informed that they would see a short film extract (with subtitles, when it was the case). They were asked to watch the film attentively as they would have to answer questions about it when it was finished. It was also demanded that they do not pause the film or go back to re-watch. Following the viewing, the participants had to fill in the comprehension questionnaire, presented on the same computer. To finish, they were asked to complete the personal data and the self-rating questionnaires. All together, the experiment lasted approximately 20 minutes. Table 1. Main characteristics of all participants depending on the group they were assigned to, including mean age, mean percentage of time spent using each of the three languages weekly, and mean self-ratings (on a 5-point Likert scale, standard deviation in parentheses). Group/ Version Mean% of language use Language self-rating Participants Mean age French English Spanish French English Spanish a TriOV 12 TriFRa 12 MonoFRa 12 MonoENa 12 27.3 (4.1) 33.3 (10.4) 32.1 (7.7) 35 (7.3) 76.7 (23.9) 75.2 (20.5) 86 (14.6) 7.1 (4.1) 12.3 (6.4) 13.1 (7.4) 7 (3.6) 88 (15.6) 11.0 (7.8) 11.7 (5.9) 7 (3.2) 4.9 (3.6) 4.9 (0.2) 5 (0) 4.9 (0.2) 2.7 (1.3) 3.7 (0.6) 3.6 (0.6) 1.5 (1.1) 4.8 (0.4) 3.5 (0.9) 2.4 (1.1) 1.6 (0.6) 1.4 (0.5) a Tri refers to trilingual participants, mono refers to monolinguals. OV represents the original version (no subtitles), FR refers to the version with French subtitles, and GB to the version with English subtitles. Data presented for the language self-rating are the mean of the answer given by the participants on four criteria (reading comprehension, reading frequency, oral production and oral comprehension). 112 X. Aparicio and D. Bairstow This being a preliminary study, full statistical analysis was not carried out from lack of enough participants for it to be relevant. A descriptive analysis is provided in order to point out the main tendency followed by our results, encouraging further data collection. To begin, the results of the personal data questionnaires were analysed in order to sort the participants into groups depending on their proficiency in each of the languages of this study. The experimental results of seven participants, that did not meet the language requirements, were excluded from the study report. Next, the mean scores for each item of the comprehension questionnaire were represented graphically in order to examine the way in which the movie data were processed by each group. The items were coded depending on the type of information they question (visual- or dialogue-related), but also their level of importance within the story (level 1 being the most central data and level 2, the more secondary information). Indeed, concerning visual data, centrality was defined as elements presented in the middle of the screen, that are integral part of the story (the cake presented to the main character for example), whereas secondary information relates to elements peripheral to the story and screen, presented in the background and not particularly relevant to the storyline (the clothes worn by secondary characters for instance). In the same way, central dialogue information is part of the exchanges between main characters and that has an incidence on the story, while secondary information is just passing comments that are not connected to the main plot. Thus, four types of items were obtained: 8 central visual items (V1-1 through 8); 8 secondary visual items (V2-1 through 8); 8 central dialogue items (D1-1 through 8); and 8 secondary dialogue items (D2-1 through 8), dispatched chronologically throughout the film (see Appendix for an example of each type of item; the evolution of comprehension according to visual (V) and dialogue (D) items are presented in Figure 2). This first graph shows the overall pattern of film comprehension as the film extract unfolds. Indeed, the bold black line represents the mean comprehension score of all participants, independently of the version of the movie they were shown. The upper and lower dotted curves represent the standard deviations at each point in time and provide a good picture of the variations in comprehension scores due to the different versions of the movie presented to the participants. Looking at this representation, we can see that film comprehension is far from being a linear phenomenon and that processing the different elements that make up a movie can be either positively or negatively influenced. Many factors can certainly have such an influence, although the experimental setting was designed to minimise the effect of such factors as movie length, number of characters, 14 Mean comprehension scores Downloaded by [University of California, San Diego] at 08:18 26 February 2016 3. Results 12 Mean comprehension 10 8 Upper error 6 4 Lower error 2 0 -2 V1-1 D1-1 V2-1 V2-2 V2-3 V1-5 D1-2D2-4D1-3 V2-5 V2-6 D2-5D1-6D1-8 V1-8 V2-8 Items Figure 2. Evolution of comprehension according to visual (V) and dialogue (D) mean scores. Downloaded by [University of California, San Diego] at 08:18 26 February 2016 International Journal of Multilingualism 113 plot complexity, dialogue complexity and speech intelligibility (speed and accents); thus we can see that the mean comprehension pattern follows a series of drops that do not take place at random times. Indeed, we can see that the lowest comprehension scores are achieved for items referring to information provided around the time of language switches (illustrated by the vertical dashed lines). Furthermore, Table 2 also shows that the English monolinguals (MonoEN) seem to achieve the lowest comprehension level throughout the movie in comparison to all three other groups, which could indicate that processing linguistic information alternately via an audio or a visual media has a negative impact on general comprehension. When comparing the results of the three other groups (TRIOV, TRIFR and MONOFR), the mean comprehension levels appear near-equivalent. This could imply that the presence of subtitles in the viewers’ first language can help to overcome language barriers (French subtitles in the MONOFR condition), provided this language is not also partially present in the dialogues (English used either in the dialogues or in the subtitles in the MONOEN condition). Thus, the fact that the French monolinguals (MonoFR) achieve an overall higher level of comprehension than the English monolinguals (MonoEN) seems to imply that it is better for comprehension that subtitles in the viewers’ mother tongue be presented throughout the sequence (French condition), rather than having subtitles during only those parts in which a different language is spoken (English condition). This will of course have to be tested with further research. In this table, the mean number of correct answers for each type of item shows that visual information is globally processed in a better way when no subtitles are on-screen (TRIOV) than when they are present. This result is in line with most findings in the field (Bairstow & Lavaur, 2012; Lavaur & Nava, 2008) evidencing that the presence of subtitles can distract the viewer from fully processing the visual information provided. However, concerning dialogue information, the mean scores are generally higher when subtitles in the viewer’s language are continuously shown (TRIFR and MONOFR) than when they are absent (TRIOV) or only shown when Spanish is spoken (MONOEN). When looking at the detailed scores, we can see that the negative effect of the presence of subtitles on visual information processing seems to be stronger on secondary elements (V2) rather than on central information (V1). This could be explained by the central position of the subtitles on the screen which is also where most of the relevant visual action will take place, making it easier to switch from the subtitles to that zone of the picture, rather than to a more peripheral part of the screen. Concerning dialogue information, the results in Table 3 show that both central (D1) and secondary (D2) information is better processed by the French monolinguals seeing the French-subtitled version of the film (MONOFR). This indicates that seeing a movie Table 2. Mean comprehension scores (and standard deviations) for each language level (trilinguals and monolinguals) and each version of the films (original version – OV, version with French subtitles – FR, version with English subtitles – EN, according to the spoken language). English dialogues TRIOV TRIFR MonoFR MonoEN Spanish dialogues Mean SD Mean SD 6.7 7.5 9.2 5.0 4.2 3.4 3.3 2.7 9.5 9.8 10.2 6.6 2.7 2.5 2.2 1.8 114 X. Aparicio and D. Bairstow Table 3. Mean scores (and standard deviations) to each type of item (visual or dialogues, central or secondary) by each language level (trilingual or monolingual) and for each version of the film (original, subtitled in French or subtitled in English). Downloaded by [University of California, San Diego] at 08:18 26 February 2016 Visual Group/version Central (V1) TRIOV TRIFR MONOFR MONOEN 10.25 9.25 10.50 6.00 (1.98) (2.05) (1.41) (2.07) Dialogue Secondary (V2) 9.00 6.50 7.25 5.13 (3.07) (3.82) (3.81) (3.39) Central (D1) 7.25 8.75 11.00 6.00 (4.23) (2.92) (0.53) (2.00) Secondary (D2) 7.13 11.25 10.50 6.88 (4.05) (1.04) (2.00) (1.56) with dialogues in foreign languages and subtitles in the mother tongue is effective for a good comprehension of linguistic information. When the viewer has some knowledge of the languages used in the dialogues however, the presence of subtitles seems to have a positive effect primarily on the processing of secondary elements (D2). The subtitles therefore appear to enable the viewer to process details of linguistic information that were not readily available through listening to the dialogues presented in the trilinguals’ L2 (English) and L3 (Spanish). Finally, both central and secondary dialogue information seem to be more difficult to process when the linguistic information is given in turn in either the dialogues or in the subtitles (MONOGB). This alteration of dialogue comprehension could be linked to the discontinuous transmission mode of linguistic information, switching from an oral mode (dialogues) to a written on (subtitles) throughout the film and shall have to be the subject of more extensive research in the future. 4. Discussion Four groups differing in terms of language knowledge and use (trilinguals and monolinguals) were asked to watch an excerpt of a movie in its original version or with subtitles in their native language. The experimental conditions involved respectively two (original version with English and Spanish dialogues, either with or without English subtitles) or three languages (additional French subtitles), and the performance of the participants was evaluated according to their good processing of visual aspects, dialogues and languages involved in the film. Our explorative study was designed to examine the influence of the knowledge of the participants concerning the languages used on-screen on the dimensions associated with film comprehension (dialogues and visual components). The choice of these different populations of participants was made in order to evaluate different combinations of languages with regards to the version of the movie visualised. The study was carried out keeping in mind that the trilingual participants are used to dealing with language switching aspects, as opposed to the monolingual participants, who only have a very low knowledge of any other languages apart from their mother tongue. 4.1. General comprehension According to our findings, the comprehension of a movie is far from being frozen during the viewing, as the viewers tend to lean more on one dimension or the other (visual or dialogue) as the story unfolds. Of course, the results obtained here are modulated by the Downloaded by [University of California, San Diego] at 08:18 26 February 2016 International Journal of Multilingualism 115 version of the movie and the participants’ skills in terms of language proficiency, but the evolution of comprehension is verified across our four groups. The mean scores show that the general performance of the different groups concerning both dialogue and visual aspects are very close, except for the group of monolingual English speakers, which is on average below the three other groups. This difference could originate in the fact that, when monolingual English speakers process the original version of the film with English subtitles, they must process linguistic information presented alternately in both spoken dialogues and on a visual modality (subtitles). These participants therefore have to change viewing strategies at each language switch as it is very different to watch a film that is not subtitled and one in which the viewer must read the subtitles to achieve satisfactory comprehension. This constant switching of viewing strategies may be the cause of the lower overall comprehension scores for this group and may be linked to an inability to inhibit (or not properly) one modality or the other (or to handle both modalities; for cognitive control, see Bialystok, Craik, & Luk, 2008; Aparicio, Heidlmayr, & Isel, 2015). This hypothesis shall have to be tested in future research with an appropriated paradigm and measuring tool. Concerning the performances of the other groups, clear differences seem to arise depending on the version watched and their language abilities (Bairstow, 2011; Lavaur & Bairstow, 2011; Lavaur & Nava, 2008). The comparison between French and English monolinguals clearly highlights better performance for French participants. An attempt to explain the monolingual English group’s lower results has been provided in the previous paragraph. For the second group, and given that French monolinguals are not familiar with the two languages spoken, it is clear that they can freely concentrate on the visual modality (images and subtitles), although the auditory channel is never totally ignored (as proved by the high scores to the language recognition questions). Moreover, when comparing the French monolinguals, seeing the film with subtitles in their mother tongue, with the trilinguals seeing the original version, we can see that similar comprehension levels are achieved. This result indicates that subtitles do enable satisfactory comprehension of a film and are thus an effective method to overcome language barriers, confirming the results of previous studies (Bairstow, 2011; Bairstow & Lavaur, 2012; Grignon, Lavaur, & Blanc, 2007; Lavaur & Bairstow, 2011). However, although visual elements are mainly correctly processed by the viewers seeing the subtitled film, we can see that the processing of secondary visual elements is affected, since lower scores are obtained. These results confirm data collected in previous experiments run by the authors that seem to point to the fact that while the presence of subtitles do not necessarily impair visual processing as such, it may however impede in the perception or processing of more secondary information. This trend needs to be explored, however, notably to determine whether this effect is linked to the more peripheral location of this type of visual information on the screen, or whether it is a strategy to ignore information that seems less important in order to concentrate limited cognitive resources on more important data processing. In the present study, the two groups of trilinguals represent the central issue, because they have knowledge of each of the three languages presented within the movie. The comparison of these two groups confirms our hypothesis concerning the different effects of the presence of subtitles on both dimension of film comprehension (visual and dialogue). According to our results, the presence of subtitles leads to lower performances on the visual dimension, while better results for dialogue comprehension are obtained. This seems to indicate that more attention is focused on the subtitles presented in their mother tongue than to the spoken dialogues (in their second and third languages). The 116 X. Aparicio and D. Bairstow Downloaded by [University of California, San Diego] at 08:18 26 February 2016 higher dialogue-related information processing can therefore be linked to the fact that the first language has a higher level of activation inside the mental lexicon and is basically automatically processed (because of the well-demonstrated automaticity of reading). 4.2. Language switching and language effects As was mentioned in the theoretical framework presented at the beginning of this chapter, language switching is often associated with a cost, which originates in the difficulty of inhibiting one language in order to activate the other one, even if all languages possess a residual level of activation (Aparicio & Lavaur, 2013; Dijkstra, 2005). Here, the two groups of monolingual participants are considered as control groups, because they have a very limited knowledge of the other languages presented on the screen. In the absence of representations of words from an unknown language in the mental lexicon, the interferences are of course non-existent. Nevertheless, very interesting findings arise when looking at the performance of both groups of trilinguals. For both groups, there is a disruption consecutive to a language switch leading to a drop in the mean number of correct answers. Interestingly, this cost associated with language switching seems to affect more strongly the visual components of comprehension. This could be due to the fact that the activity of switching from one language to another strongly requires linguistic components in order to successfully process this unexpected language switch. As this activity requires attentional processes and the intervention of executive functions (Aparicio & Lavaur, 2013; Aparicio et al., 2015; Bialystok et al., 2008), this could explain the lower score obtained for visual information. In other words, it is possible that during and straight after a language switch, the participants’ cognitive resources are temporarily focused in the act of re-establishing linguistic coherence, inducing an instantaneous drop in movie information processing. Once again this hypothesis will need to be tested in an appropriate paradigm enabling online measures of cognitive effort. Moreover, trilingual participants are clearly more accurate at determining which language was spoken at a key moment of the film (79.6% and 79.2% of accuracy) compared to monolingual participants (61% for French monolinguals, 69.4% for English monolinguals). These findings show that the knowledge of the languages, associated with a greater ability to deal with language switching, permits a better ability to identify languages. For monolinguals, although they are able to identify that the spoken language has changed, the identification of the language spoken at a key moment is less efficient. Nevertheless, the better results obtained by English monolinguals in comparison with the French ones is probably due to a strategy bias, given that if it was not their mother tongue which was spoken, it was necessarily Spanish. To finish, it is also important to notice that in the trilinguals’ performances after viewing the original version, there is a lower level of general comprehension when Spanish is spoken on the screen. This effect could be due to the fact that Spanish is the latest acquired language of our participants and, as demonstrated by Aparicio et al. (2012), age of acquisition as well as proficiency could account for semantic processing in multilinguals. 4.3. Conclusions This preliminary study has provided some interesting data concerning multilingualism and multilingual situations, in addition to effects of language switching. Many differences in processing have been found depending on various factors, such as language International Journal of Multilingualism 117 Downloaded by [University of California, San Diego] at 08:18 26 February 2016 knowledge, number of languages on the screen, presentation mode (spoken dialogues and/or written subtitles) and components of comprehension (visual or linguistic information). Some hypotheses concerning the reasons behind these differences have been provided, notably in relation to cognitive load and working memory limitation, which constitute good leads for future research. In this view, this line of research appears as very interesting for investigating many different aspects of multilingual situations in a wide range of experimental studies. Considering the large number of dimensions that can be examined using multilingual films as a basis, a rich, long and interesting future can be predicted for these studies. Disclosure statement No potential conflict of interest was reported by the authors. Note 1. As it is more and more difficult nowadays to find monolingual people, in the present study the term monolingual is associated with a very scarce use and knowledge of a second language (below 10%). References Altarriba, J., & Basnight-Brown, D. M. (2007). Methodological considerations in performing semantic- and translation-priming experiments across languages. Behavior Research Methods, 39(1), 1–18. doi:10.3758/BF03192839 Aparicio, X., Heidlmayr, K., & Isel, F. (2015). Effect of inhibition training on language switching and Stroop tasks: Evidence from late bilinguals and simultaneous interpreters. Language and Cognitive Processes. Manuscript submitted for publication. Aparicio, X., & Lavaur, J.-M. (2013). Recognising words in three languages: Effects of language dominance and language switching. International Journal of Multilingualism, 11, 164–181. doi:10.1080/14790718.2013.783583 Aparicio, X., Midgley, K. J., Holcomb, P. J., Pu, H., Lavaur, J.-M., & Grainger, J. (2012). Language effects in trilinguals: An ERP Study. Frontiers Psychology, 3(402), 1–10. Bairstow, D. (2011). Audiovisual processing while watching subtitled films: A cognitive approach. In A. Şerban, A. Matamala, & J. M. Lavaur (Eds.), Audiovisual translation in close-up: Practical and theoretical approaches (pp. 205–219). Bern: P. Lang. Bairstow, D., & Lavaur, J.-M. (2012). Audiovisual information processing by monolinguals and bilinguals: Effects of intralingual and interlingual subtitles. In A. Remael, P. Orero, & M. Carroll (Eds.), Audiovisual translation and media accessibility at the crossroads (pp. 273–294). Amsterdam/New York, NY: Rodopi. Baldo, M. (2009). Subtitling multilingual films. The case of Lives of the Saints, an ItalianCanadian TV screenplay. In F. Federici (Ed.), Translating voices for audiovisuals (pp. 117– 135). Roma: Aracne, chapter 6. Bentahila, A., & Davies, E. E. (1995). Patterns of code-switching and patterns of language contact. Lingua, 96, 75–93. doi:10.1016/0024-3841(94)00035-K Bialystok, E., Craik, F., & Luk, G. (2008). Cognitive control and lexical access in younger and older bilinguals. Journal f Experimental Psychology: Learning, Memory, and Cognition, 34, 859–873. Brunick, K. L., Cutting, J. E., & Delong, J. E. (2013). Low-level features of film: What they are and why we would be lost without them. In A. Shimamura (Ed.), Psychocinematics: The aesthetic science of movies (pp. 133–148). New York, NY: Oxford. Bultena, S., Dijkstra, T., & Van Hell, J. G. (in press). Processing language switches in sentence context: Evidence from self-paced reading. Bilingualism: Language and Cognition, 1–17. doi:10.1017/S1366728914000145 Cantone, K. F. (2007). Code-switching in bilingual children. Studies in Theoretical Psycholinguistics, 37. Springer Netherlands. Downloaded by [University of California, San Diego] at 08:18 26 February 2016 118 X. Aparicio and D. Bairstow Carroll, N. (1996). Theorizing the moving image. Cambridge: Cambridge University Press. Chauncey, K., Grainger, J., & Holcomb, P. J. (2008). Code-switching effects in bilingual word recognition: A masked priming studies with ERPs. Brain and Language, 105, 161–174. doi:10.1016/j.bandl.2007.11.006 Christoffels, I. K., Firk, C., & Schiller, N. O. (2007). Bilingual language control: An event-related brain potential study. Brain Research, 1147, 192–208. doi:10.1016/j.brainres.2007.01.137 Costa, A., & Santesteban, M. (2004). Lexical access in bilingual speech production: Evidence from language switching in highly proficient bilinguals and L2 learners. Journal of Memory and Language, 50, 491–511. doi:10.1016/j.jml.2004.02.002 Dijkstra, T. (2005). Bilingual visual word recognition and lexical access. In J. F. Kroll & A. de Groot (Eds.), Handbook of bilingualism: Psycholinguistic approaches (pp. 179–200). Cambridge: Cambridge University Press. Dijkstra, T., & Van Hell, J.G. (2003). Testing the language mode hypothesis using trilinguals. International Journal of Bilingual Education and Bilingualism, 6(1), 2–16. doi:10.1080/ 13670050308667769 D’Ydewalle, G., Praet, C., Verfaillie, K., & Van Rensbergen, J. (1991). Watching subtitled television: Automatic reading behaviour. Communication Research, 18, 650–666. Grainger, J., & Beauvillain, C. (1987). Language blocking and lexical access in bilinguals. The Quarterly Journal of Experimental Psychology, 39, 295–319. doi:10.1080/14640748708401788 Grainger, J., & Dijkstra, T. (1992). On the representation and use of language information in bilinguals. In R. J. Harris (Ed.), Cognitive processing in bilinguals. Amsterdam: North Holland. Grainger, J., Midgley, K. J., & Holcomb, P. J. (2010). Re-thinking the bilingual interactiveactivation model from a developmental perspective (BIA-d). In M. Kail & M. Hickmann (Eds.), Language acquisition across linguistic and cognitive systems (pp. 267–284). New York, NY: John Benjamins. Grignon, P., Lavaur, J.-M., & Blanc, N. (2007, July 8–11). The effect of subtitles on film understanding. Presented at the Seventeenth annual meeting of the society for Text and Discourse. Glasgow, Scotland. Heiss, C. (2004). Dubbing multilingual films: A new challenge? Meta, 49, 208–220. Lavaur, J.-M., & Bairstow, D. (2011). Languages on the screen: Is film comprehension related to the viewer's fluency level and to the languages in the subtitles? International Journal of Psychology, 46, 455–462. doi:10.1080/00207594.2011.565343 Lavaur, J.-M., & Nava, S. (2008). Interferences related to intra-lingual subtitling on pictures processing of a movie. In J.-M. Hoc & Y. Corson (Eds.), Published acts of the 2007 conference of the French Society of Psychology (pp. 59–64). Nantes, France. Lemhöfer, K., Dijkstra, A., & Michel, M. C. (2004). Three languages, one ECHO: Cognate effects in Trilingual word recognition. Language and Cognitive Processes, 19, 585–611. Marian, V. (2009). Audio-visual integration during bilingual language processing. In A. Pavlenko (Ed.), The bilingual mental lexicon: Interdisciplinary approaches (pp. 52–78). Bristol: Multilingual Matters. Merleau-Ponty, M. (1947). Cinema and the new psychology. Modern Times, 3, 930–947. Meuter, R. F. I. (2005). Language selection in bilinguals: mechanisms and processes. In J. F. Kroll & A. de Groot (Eds.), Handbook of bilingualism: Psycholinguistic approaches (pp. 349–368). Cambridge: Cambridge University Press. Mitry, J. (2001). Esthetics and psychology of cinema. Paris: Cerf. Moreno, E. M., Federmeier, K. D., & Kutas, M. (2002). Switching languages, switching palabras (words): An electrophysiological study of code switching. Brain and Language, 80, 188–207. doi:10.1006/brln.2001.2588 Muysken, P. (2000). Bilingual speech: A typology of code mixing. Cambridge: Cambridge University Press. Thomas, M. S. C., & Allport, A. (2000). Language switching costs in bilingual visual word recognition. Journal of Memory and Language, 43(1), 44–66. doi:10.1006/jmla.1999.2700 Van Heuven, W. J. B., Dijkstra, T., & Grainger, J. (1998). Orthographic neighborhood effect in bilingual word recognition. Journal of Memory and Language, 39, 458–483. doi:10.1006/ jmla.1998.2584 Zacks, J. M., & Magliano, J. P. (2011). Film, narrative, and cognitive neuroscience. In F. Bacci & D. Melcher (Eds.), Art and the senses (pp. 435–454). New York, NY: Oxford University Press. International Journal of Multilingualism 119 Downloaded by [University of California, San Diego] at 08:18 26 February 2016 Appendix. Examples of each type of item from the comprehension questionnaire (visual central and secondary; dialogue central and secondary), with the multiplechoice answers (correct answer in bold characters). Type of item Item Visual central (V1) Visual secondary (V2) Dialogue central (D1) The young lady enters a house and speaks with: When Anita arrives in the garden, her aunt: Dialogue secondary (D2) A young man; A woman; A man; A young lady; I don’t know Gives her a drink; Gives her a hug; Gives her a slice of cake; Gives her a seat; I don’t know Her mother reproaches her for quitting her job. Anita explains that: She couldn’t stand the long hours; She wasn’t getting along with the manager; She wasn’t being paid enough; She wasn’t getting along with her colleagues; I don’t know The mother finishes by saying that Sixteen; Fifteen; Fourteen; Thirteen; I she’s been working since the don’t know age of: Note: The items shown in this table are examples and are not necessarily consecutive within the experimental questionnaire.