ASR Linguistics Intern

A Moving Experience. Motivation Statistical and neural network ASR systems normalize text before training models with it. It works in two directions: our ‘tokenizer’ not only splits on whitespace, but also normalizes text as it is spoken. Numbers in digits, dates, time expressions etc. will be written out in words, spelling will be normalized, and some other minor rewrites will be done in order to bring all training material into our token space. On the other hand, the end user does not want to see ‘first of July nineteen ninety two’, so will need the inverse normalization to ‘July 1, 1992’ as a step we call ‘formatting’. Currently both the tokenizer and the formatter are largely driven by grammar rules.…