Unsupervised Spoken Term Discovery on Untranscribed Speech

Abstract: (Part of the abstract) In this thesis, we investigate the use of unsupervised spoken term discovery in tackling this problem. Unsupervised spoken term discovery aims to discover topic-related terminologies in a speech without knowing the phonetic properties of the language and content. It can be further divided into two parts: Acoustic segment modelling (ASM) and unsupervised pattern discovery. ASM learns the phonetic structures of zero-resource language audio with no phonetic knowledge available, generating self-derived "phonemes". The audio are labelled with these "phonemes" to obtain "phoneme" sequences. Unsupervised pattern discovery searches for repetitive patterns in the "phoneme" sequences. The discovered patterns can be grouped to determine the keywords of the audio. Multilingual neural network with bottleneck layer is used for feature extraction. Experiments show that bottleneck features facilitate the training of ASM compared to conventional features such as MFCC. The unsupervised spoken term discovery system is experimented with online lectures covering different topics by different speakers. It is shown that the system learns the phonetic information of the language and can discover frequent spoken terms that align with text transcription. By using information retrieval technology such as word embedding and TFIDF, it is shown that the discovered keywords can be further used for topic comparison.