Hostname: page-component-669899f699-rg895 Total loading time: 0 Render date: 2025-05-04T06:43:19.016Z Has data issue: false hasContentIssue false
  • English
  • Français

Effects of modality preference and working memory capacity on captioned videos in enhancing L2 listening outcomes

Published online by Cambridge University Press:  14 February 2020

Emily Fen Kam Open the ORCID record for Emily Fen Kam [Opens in a new window] ,
Yeu-Ting Liu Open the ORCID record for Yeu-Ting Liu [Opens in a new window]  and
Wen-Ta Tseng Open the ORCID record for Wen-Ta Tseng [Opens in a new window]
Emily Fen Kam
Affiliation:
National Taiwan Normal University, Taiwan (kamemilyfen@gmail.com)
Yeu-Ting Liu*
Affiliation:
National Taiwan Normal University, Taiwan (yeutingliu@ntnu.edu.tw)
Wen-Ta Tseng
Affiliation:
National Taiwan University of Science and Technology, Taiwan (wenta.tseng@ntust.edu.tw)
*
*All correspondence regarding this publication should be addressed to Yeu-Ting Liu (Email: yeutingliu@ntnu.edu.tw).
Get access Rights & Permissions [Opens in a new window]

Abstract

Captioned video is widely used to enhance second language (L2) learners’ exposure to oral input beyond the classroom setting, and captioning has been found to provide an instantaneous, useful visual aid for parsing and understanding L2 oral discourse. Nevertheless, a meta-analysis has shown that captioning exerts a selective effect on L2 learners with different profiles. This study investigated whether L2 learners’ modality preferences (visual vs. auditory) and working memory capacity (high vs. low) would modulate the effect of full captions on L2 listening outcome. Results from 60 participants revealed that both cognitive variables affected their L2 listening to different extents. Notably, working memory capacity modulates the impact of L2 learners’ preferred modality on their listening outcome. Modality preference did not exert any significant impact on the listening outcome of L2 learners with lower working memory capacity. For L2 learners with high working memory capacity, their modality preference played a pivotal role in modulating their listening outcome; in this case, auditory learners had the best listening performance viewing the video without captions, whereas visual learners did best when watching the captioned video. These findings speak to the need for taking individual differences into consideration when employing captioned videos.

Keywords

listening comprehensionmultimedia learningcaptionsindividual differences
Type
Regular papers
Information
ReCALL , Volume 32 , Issue 2 , May 2020 , pp. 213 - 230
Check for updates
Copyright
© European Association for Computer Assisted Language Learning 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Alptekin, C. & Erçetin, G. (2009) Assessing the relationship of working memory to L2 reading: Does the nature of comprehension process and reading span task make a difference? System, 37(4): 627–639. https://doi.org/10.1016/j.system.2009.09.007CrossRefGoogle Scholar
Chang, C.-C., Tseng, K.-H. & Tseng, J.-S. (2011) Is single or dual channel with different English proficiencies better for English listening comprehension, cognitive load and attitude in ubiquitous learning environment? Computers & Education, 57(4): 2313–2321. https://doi.org/10.1016/j.compedu.2011.06.006CrossRefGoogle Scholar
Chen, Y.-R., Liu, Y.-T. & Todd, A. G. (2018) Transient but effective? Captioning and adolescent EFL learners’ spoken vocabulary acquisition. English Teaching & Learning, 42(1): 25–56. https://doi.org/10.1007/s42321-018-0002-8CrossRefGoogle Scholar
Friedman, N. P. & Miyake, A. (2004) The reading span test and its predictive power for reading comprehension ability. Journal of Memory and Language, 51(1): 136–158. https://doi.org/10.1016/j.jml.2004.03.008CrossRefGoogle Scholar
Garza, T. J. (1991) Evaluating the use of captioned video materials in advanced foreign language learning. Foreign Language Annals, 24(3): 239–258. https://doi.org/10.1111/j.1944-9720.1991.tb00469.xCrossRefGoogle Scholar
Gass, S., Winke, P., Isbell, D. R. & Ahn, J. (2019) How captions help people learn languages: A working-memory, eye-tracking study. Language Learning & Technology, 23(2): 84–104. https://doi.org/10125/44684Google Scholar
Hasler, B. S., Kersten, B. & Sweller, J. (2007) Learner control, cognitive load and instructional animation. Applied Cognitive Psychology, 21(6): 713–729. https://doi.org/10.1002/acp.1345CrossRefGoogle Scholar
Huang, H.-C. & Eskey, D. E. (1999) The effects of closed-captioned television on the listening comprehension of intermediate English as a second language (ESL) students. Journal of Educational Technology Systems, 28(1): 75–96. https://doi.org/10.2190/RG06-LYWB-216Y-R27GCrossRefGoogle Scholar
Kozan, K., Erçetin, G. & Richardson, J. C. (2015) Input modality and working memory: Effects on second language text comprehension in a multimedia learning environment. System, 55: 63–73. https://doi.org/10.1016/j.system.2015.09.001CrossRefGoogle Scholar
Lee, M. & Révész, A. (2018) Promoting grammatical development through textually enhanced captions: An eye-tracking study. The Modern Language Journal, 102(3): 557–577. https://doi.org/10.1111/modl.12503CrossRefGoogle Scholar
Lee, P.-J., Liu, Y.-T. & Tseng, W.-T. (2019) One size fits all? In search of the desirable caption display for second language learners with different caption reliance in listening comprehension. Language Teaching Research. Advance online publication. https://doi.org/10.1177/1362168819856451CrossRefGoogle Scholar
Leveridge, A. N. & Yang, J. C. (2013) Testing learner reliance on caption supports in second language listening comprehension multimedia environments. ReCALL, 25(2): 199–214. https://doi.org/10.1017/S0958344013000074CrossRefGoogle Scholar
Leveridge, A. N. & Yang, J. C. (2014) Learner perceptions of reliance on captions in EFL multimedia listening comprehension. Computer Assisted Language Learning, 27(6): 545–559. https://doi.org/10.1080/09588221.2013.776968CrossRefGoogle Scholar
Liu, Y.-T. & Todd, A. G. (2014) Dual-modality input in repeated reading for foreign language learners with different learning styles. Foreign Language Annals, 47(4): 684–706. https://doi.org/10.1111/flan.12113CrossRefGoogle Scholar
Linck, J. A., Osthus, P., Koeth, J. T. & Bunting, M. F. (2014) Working memory and second language comprehension and production: A meta-analysis. Psychonomic Bulletin & Review, 21(4): 861–883. https://doi.org/10.3758/s13423-013-0565-2CrossRefGoogle ScholarPubMed
Markham, P. & Peter, L. (2003) The influence of English language and Spanish language captions on foreign language listening/reading comprehension. Journal of Educational Technology Systems, 31(3): 331–341. https://doi.org/10.2190/BHUH-420B-FE23-ALA0CrossRefGoogle Scholar
Markham, P. L., Peter, L. A. & McCarthy, T. J. (2001) The effects of native language vs. target language captions on foreign language students’ DVD video comprehension. Foreign Language Annals, 34(5): 439–445. https://doi.org/10.1111/j.1944-9720.2001.tb02083.xCrossRefGoogle Scholar
Mayer, R. E. (2001) Multimedia learning. New York: Cambridge University Press. https://doi.org/10.1017/CBO9781139164603Google Scholar
Mayer, R. E. (2005) Principles for managing essential processing in multimedia learning: Segmenting, pretraining, and modality principles. In Mayer, R. E. (ed.), Cambridge handbook of multimedia learning. New York: Cambridge University Press, 169–182. https://doi.org/10.1017/CBO9780511816819.012CrossRefGoogle Scholar
Mayer, R. E. & Moreno, R. (2003) Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1): 43–52. https://doi.org/10.1207/S15326985EP3801_6CrossRefGoogle Scholar
Mirzaei, M. S., Meshgi, K., Akita, Y. & Kawahara, T. (2017) Partial and synchronized captioning: A new tool to assist learners in developing second language listening skill. ReCALL, 29(2): 178–199. https://doi.org/10.1017/S0958344017000039CrossRefGoogle Scholar
Montero Perez, M., Noortgate, W. V. D. & Desmet, P. (2013) Captioned video for L2 listening and vocabulary learning: A meta-analysis. System, 41(3): 720–739. https://doi.org/10.1016/j.system.2013.07.013CrossRefGoogle Scholar
Moreno, R. & Mayer, R. E. (2007) Interactive multimodal learning environments. Educational Psychology Review, 19(3): 309–326. https://doi.org/10.1007/s10648-007-9047-2CrossRefGoogle Scholar
Peters, E. (2019) The effect of imagery and on-screen text on foreign language vocabulary learning from audio-visual input. TESOL Quarterly, 53: 1008–1032. https://doi.org/10.1002/tesq.531CrossRefGoogle Scholar
Pujadas, G. & Muñoz, C. (2019) Extensive viewing of captioned and subtitled TV series: A study of L2 vocabulary learning by adolescents. The Language Learning Journal, 47(4): 479–496. https://doi.org/10.1080/09571736.2019.1616806CrossRefGoogle Scholar
Reid, J. M. (ed.) (1995) Learning styles in the ESL/EFL classroom. Boston: Heinle & Heinle.Google Scholar
Révész, A. (2012) Working memory and the observed effectiveness of recasts on different L2 outcome measures. Language Learning, 62(1): 93–132. https://doi.org/10.1111/j.1467-9922.2011.00690.xCrossRefGoogle Scholar
Sagarra, N. (2017) Longitudinal effects of working memory on L2 grammar and reading abilities. Second Language Research, 33(3): 341–363. https://doi.org/10.1177/0267658317690577CrossRefGoogle Scholar
Seufert, T., Schütze, M. & Brünken, R. (2009) Memory characteristics and modality in multimedia learning: An aptitude–treatment–interaction study. Learning and Instruction, 19(1): 28–42. https://doi.org/10.1016/j.learninstruc.2008.01.002CrossRefGoogle Scholar
Suárez, M. D. M. & Gesa, F. (2019) Learning vocabulary with the support of sustained exposure to captioned video: Do proficiency and aptitude make a difference? The Language Learning Journal, 47(4): 497–517. https://doi.org/10.1080/09571736.2019.1617768CrossRefGoogle Scholar
Sueyoshi, A. & Hardison, D. M. (2005) The role of gestures and facial cues in second language listening comprehension. Language Learning, 55(4): 661–699. https://doi.org/10.1111/j.0023-8333.2005.00320.xCrossRefGoogle Scholar
Taylor, G. (2005) Perceived processing strategies of students watching captioned video. Foreign Language Annals, 38(3): 422–427. https://doi.org/10.1111/j.1944-9720.2005.tb02228.xCrossRefGoogle Scholar
Turner, M. L. & Engle, R. W. (1989) Is working memory capacity task dependent? Journal of Memory and Language, 28(2): 127–154. https://doi.org/10.1016/0749-596X(89)90040-5CrossRefGoogle Scholar
Winke, P., Gass, S. & Sydorenko, T. (2010) The effects of captioning videos used for foreign language listening activities. Language Learning & Technology, 14(1): 65–86.Google Scholar
Winke, P., Gass, S. & Sydorenko, T. (2013) Factors influencing the use of captions by foreign language learners: An eye‐tracking study. The Modern Language Journal, 97(1): 254–275. https://doi.org/10.1111/j.1540-4781.2013.01432.xCrossRefGoogle Scholar