Phoneme-Grapheme Dictionary-Based Prompting for Robust Proper Noun Recognition in Japanese ASR

Recent End-to-End (E2E) speech recognition models still struggle with low-frequency words such as proper nouns. Whereas Whisper models utilize prompt-based contextual biasing for improvement, Japanese presents a challenge because of its non-unique grapheme-phoneme correspondence, which limits the biasing effectiveness. Furthermore, rare kanji and special symbols often lead to out-of-vocabulary (OOV) issues, preventing model processing. To address these problems, we propose a novel speech recognition method that leverages a dictionary for proper nouns, which stores their graphemes and phonemes, along with assigned special tokens. Our method biases the model by providing phoneme sequences from the dictionary as prompts. When input speech matches these sequences, the model outputs special tokens, which are then replaced with the corresponding correct graphemes during post-processing by utilizing dictionary entries. Experiments on Japanese CSJ datasets demonstrate that our method drastically reduces the proper noun error rate from 24.5% to 0.6% compared with the conventional grapheme-based prompting method.