Sentence recognition from articulatory movements for silent speech interfaces

Jun Wang, Ashok K Samal, Jordan R. Green, Frank Rudzicz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

25 Citations (Scopus)

Abstract

Recent research has demonstrated the potential of using an articulation-based silent speech interface for command-and-control systems. Such an interface converts articulation to words that can then drive a text-to-speech synthesizer. In this paper, we have proposed a novel near-time algorithm to recognize whole-sentences from continuous tongue and lip movements. Our goal is to assist persons who are aphonic or have a severe motor speech impairment to produce functional speech using their tongue and lips. Our algorithm was tested using a functional sentence data set collected from ten speakers (3012 utterances). The average accuracy was 94.89% with an average latency of 3.11 seconds for each sentence prediction. The results indicate the effectiveness of our approach and its potential for building a real-time articulation-based silent speech interface for clinical applications.

Original languageEnglish (US)
Title of host publication2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012 - Proceedings
Pages4985-4988
Number of pages4
DOIs
StatePublished - Oct 23 2012
Event2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012 - Kyoto, Japan
Duration: Mar 25 2012Mar 30 2012

Publication series

NameICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
ISSN (Print)1520-6149

Conference

Conference2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012
CountryJapan
CityKyoto
Period3/25/123/30/12

Fingerprint

Command and control systems

Keywords

  • Sentence recognition
  • laryngectomy
  • silent speech interface
  • support vector machine

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Wang, J., Samal, A. K., Green, J. R., & Rudzicz, F. (2012). Sentence recognition from articulatory movements for silent speech interfaces. In 2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012 - Proceedings (pp. 4985-4988). [6289039] (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). https://doi.org/10.1109/ICASSP.2012.6289039

Sentence recognition from articulatory movements for silent speech interfaces. / Wang, Jun; Samal, Ashok K; Green, Jordan R.; Rudzicz, Frank.

2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012 - Proceedings. 2012. p. 4985-4988 6289039 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, J, Samal, AK, Green, JR & Rudzicz, F 2012, Sentence recognition from articulatory movements for silent speech interfaces. in 2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012 - Proceedings., 6289039, ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 4985-4988, 2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012, Kyoto, Japan, 3/25/12. https://doi.org/10.1109/ICASSP.2012.6289039
Wang J, Samal AK, Green JR, Rudzicz F. Sentence recognition from articulatory movements for silent speech interfaces. In 2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012 - Proceedings. 2012. p. 4985-4988. 6289039. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). https://doi.org/10.1109/ICASSP.2012.6289039
Wang, Jun ; Samal, Ashok K ; Green, Jordan R. ; Rudzicz, Frank. / Sentence recognition from articulatory movements for silent speech interfaces. 2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012 - Proceedings. 2012. pp. 4985-4988 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).
@inproceedings{480668c5ecfd4a9b863cbb0b72164c48,
title = "Sentence recognition from articulatory movements for silent speech interfaces",
abstract = "Recent research has demonstrated the potential of using an articulation-based silent speech interface for command-and-control systems. Such an interface converts articulation to words that can then drive a text-to-speech synthesizer. In this paper, we have proposed a novel near-time algorithm to recognize whole-sentences from continuous tongue and lip movements. Our goal is to assist persons who are aphonic or have a severe motor speech impairment to produce functional speech using their tongue and lips. Our algorithm was tested using a functional sentence data set collected from ten speakers (3012 utterances). The average accuracy was 94.89{\%} with an average latency of 3.11 seconds for each sentence prediction. The results indicate the effectiveness of our approach and its potential for building a real-time articulation-based silent speech interface for clinical applications.",
keywords = "Sentence recognition, laryngectomy, silent speech interface, support vector machine",
author = "Jun Wang and Samal, {Ashok K} and Green, {Jordan R.} and Frank Rudzicz",
year = "2012",
month = "10",
day = "23",
doi = "10.1109/ICASSP.2012.6289039",
language = "English (US)",
isbn = "9781467300469",
series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",
pages = "4985--4988",
booktitle = "2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012 - Proceedings",

}

TY - GEN

T1 - Sentence recognition from articulatory movements for silent speech interfaces

AU - Wang, Jun

AU - Samal, Ashok K

AU - Green, Jordan R.

AU - Rudzicz, Frank

PY - 2012/10/23

Y1 - 2012/10/23

N2 - Recent research has demonstrated the potential of using an articulation-based silent speech interface for command-and-control systems. Such an interface converts articulation to words that can then drive a text-to-speech synthesizer. In this paper, we have proposed a novel near-time algorithm to recognize whole-sentences from continuous tongue and lip movements. Our goal is to assist persons who are aphonic or have a severe motor speech impairment to produce functional speech using their tongue and lips. Our algorithm was tested using a functional sentence data set collected from ten speakers (3012 utterances). The average accuracy was 94.89% with an average latency of 3.11 seconds for each sentence prediction. The results indicate the effectiveness of our approach and its potential for building a real-time articulation-based silent speech interface for clinical applications.

AB - Recent research has demonstrated the potential of using an articulation-based silent speech interface for command-and-control systems. Such an interface converts articulation to words that can then drive a text-to-speech synthesizer. In this paper, we have proposed a novel near-time algorithm to recognize whole-sentences from continuous tongue and lip movements. Our goal is to assist persons who are aphonic or have a severe motor speech impairment to produce functional speech using their tongue and lips. Our algorithm was tested using a functional sentence data set collected from ten speakers (3012 utterances). The average accuracy was 94.89% with an average latency of 3.11 seconds for each sentence prediction. The results indicate the effectiveness of our approach and its potential for building a real-time articulation-based silent speech interface for clinical applications.

KW - Sentence recognition

KW - laryngectomy

KW - silent speech interface

KW - support vector machine

UR - http://www.scopus.com/inward/record.url?scp=84867591892&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867591892&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2012.6289039

DO - 10.1109/ICASSP.2012.6289039

M3 - Conference contribution

AN - SCOPUS:84867591892

SN - 9781467300469

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

SP - 4985

EP - 4988

BT - 2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012 - Proceedings

ER -