Studies on Training Text Selection for Conversational Finnish Language Modeling

Loading...
Thumbnail Image
Journal Title
Journal ISSN
Volume Title
Conference article in proceedings
Date
2013
Major/Subject
Mcode
Degree programme
Language
en
Pages
8
256-263
Series
10th International Workshop on Spoken Language Translation, (IWSLT 2013), Heidelberg, 5 Dec 2013 - 6 Dec 2013
Abstract
Current ASR and MT systems do not operate on conversational Finnish, because training data for colloquial Finnish has not been available. Although speech recognition performance on literary Finnish is already quite good, those systems have very poor baseline performance in conversational speech. Text data for relevant vocabulary and language models can be collected from the Internet, but web data is very noisy and most of it is not helpful for learning good models. Finnish language is highly agglutinative, and written phonetically. Even phonetic reductions and sandhi are often written down in informal discussions. This increases vocabulary size dramatically and causes word-based selection methods to fail. Our selection method explicitly optimizes the perplexity of a subword language model on the development data, and requires only very limited amount of speech transcripts as development data. The language models have been evaluated for speech recognition using a new data set consisting of generic colloquial Finnish.
Description
VK: coin
Keywords
Other note
Citation
Enarvi , S & Kurimo , M 2013 , Studies on Training Text Selection for Conversational Finnish Language Modeling . in 10th International Workshop on Spoken Language Translation, (IWSLT 2013), Heidelberg, 5 Dec 2013 - 6 Dec 2013 . pp. 256-263 . < http://workshop2013.iwslt.org/downloads/Studies_on_Training_Text_Selection_for_Conversational_Finnish_Language_Modeling.pdf >