Partial and synchronized captioning: A new tool for second language listening development

Written by
Language: English

  • Book title: "CALL Design: Principles and Practice - Proceedings of the 2014 EUROCALL Conference, Groningen, The Netherlands"
  • Editors: Sake Jager, Linda Bradley, Estelle J. Meima, Sylvie Thouësny.
  • Pages: 230-236
  • Article DOI: 10.14705/rpnet.2014.000223
  • Book DOI: 10.14705/rpnet.2014.9781908416209
  • Publication date:
  • Licence: CC BY-NC-ND 4.0
  • Publisher:

© 2014 Maryam Sadat Mirzaei, Yuya Akita, Tatsuya Kawahara.

Crossref Similarity Check logo


This study investigates a novel method of captioning, partial and synchronized, as a listening tool for second language (L2) learners. In this method, the term partial and synchronized caption (PSC) pertains to the presence of a selected set of words in a caption where words are synced to their corresponding speech signal, using a state-of-the-art automatic speech recognition (ASR) technology. The system automatically selects words/phrases which are likely to hinder the learner’s listening comprehension and discards the rest. To evaluate the system, the performance of 58 Kyoto University students was assessed by a listening comprehension test on TED talks, under three conditions: no caption, full caption and PSC. Analysis of results revealed that while reducing the textual density of captions to less than 30%, PSC realizes comprehension performance as well as the full caption condition. Moreover, it gains higher scores compared to other conditions for a new segment of the same video without any captions. The findings suggest that PSC can be incorporated into CALL systems as an alternative method to enhance L2 listening comprehension.

Keywords: listening comprehension, partial and synchronized caption, word frequency, speech rate, ASR, CALL.


Coxhead, A. (2000). A new academic word list. TESOL quarterly, 34(2), 213-238. doi:10.2307/3587951

Davies, M. (2008-). The Corpus of Contemporary American English: 450 million words, 1990-present. Retrieved from

Garza, T. J. (1991). Evaluating the use of captioned video materials in advanced foreign language learning. Foreign Language Annals, 24(3), 239-258. doi:10.1111/j.1944-9720.1991.tb00469.x

Griffiths, R. (1992). Speech rate and listening comprehension: Further evidence of the relationship. TESOL Quarterly, 26(2), 385-390. doi:10.2307/3587015

Nation, I. S. P., & Beglar, D. (2007). A vocabulary size test. The Language Teacher, 31(7), 9-13.

Nation, I. S. P., & Webb, S. A. (2011). Researching and analyzing vocabulary. Boston: Heinle Cengage Learning.

Pujolà, J. T. (2002). CALLing for help: Researching language learning strategies using help facilities in a web-based multimedia program. ReCALL, 14(2), 235-262. doi:10.1017/S0958344002000423

Rogers, C. V., & Medley, F. W. (1988). Language with a purpose: Using authentic materials in the foreign language classroom. Foreign Language Annals, 21(5), 467-478. doi:10.1111/j.1944-9720.1988.tb01098.x

Vandergrift, L. (2004). 1. Listening to learn or learning to listen? Annual Review of Applied Linguistics, 24, 3-25. doi:10.1017/S0267190504000017

Wingfield, A. (2000). Speech perception and the comprehension of spoken language in adult aging. In D. Park & N. Schwarz (Eds), Cognitive Aging: A Primer (pp.175–195). Philadelphia, PA: Psychology Press.

How to cite

Citation is provided in standard text format below. For full citation export options, click Export citation.

Mirzaei, Maryam Sadat; Akita, Yuya; Kawahara, Tatsuya. (2014). Partial and synchronized captioning: A new tool for second language listening development. In Sake Jager, Linda Bradley, Estelle J. Meima, Sylvie Thouësny (Eds), CALL Design: Principles and Practice - Proceedings of the 2014 EUROCALL Conference, Groningen, The Netherlands (pp. 230-236).

Request permissions

This article is published under the Attribution-NonCommercial-NoDerivatives International 4.0 (CC BY-NC-ND 4.0) licence. Under this licence, the contents are freely available online (as PDF files) for anybody to read, download, copy, and redistribute provided that the AUTHOR(s), EDITORIAL TEAM and PUBLISHER are properly cited. Commercial use and derivative works are, however, not permitted.

Permission is not required for the republication of tables, figures or illustrations, as long as they are reproduced accurately and the source material is fully cited. It may be the case that the licence does not give you all of the permissions necessary for your intended use. If this is your current situation, please do feel free to ask the copyright holders, i.e. the authors; email addresses are listed on individual papers.

From the same authors

Errors in automatic speech recognition versus difficulties in second language listening
Mirzaei, Maryam Sadat; Meshgi, Kourosh; Akita, Yuya; Kawahara, Tatsuya.
Leveraging automatic speech recognition errors to detect challenging speech segments in TED talks
Mirzaei, Maryam Sadat; Meshgi, Kourosh; Kawahara, Tatsuya.
Listening difficulty detection to foster second language listening with a partial and synchronized caption system
Mirzaei, Maryam Sadat; Meshgi, Kourosh; Kawahara, Tatsuya.