UPDF AI

Unsupervised Analysis for Decipherment Problems

Kevin Knight,Anish Nair,Nishit Rathod,Kenji Yamada

2006 · DOI: 10.3115/1273073.1273138
Annual Meeting of the Association for Computational Linguistics · 82 Citations

TLDR

Techniques for understanding errors and significantly increasing performance in natural language decipherment problems using unsupervised learning are described.

Abstract

We study a number of natural language decipherment problems using unsupervised learning. These include letter substitution ciphers, character code conversion, phonetic decipherment, and word-based ciphers with relevance to machine translation. Straightforward unsupervised learning techniques most often fail on the first try, so we describe techniques for understanding errors and significantly increasing performance.