Intelligibility tests for synthetic speech subjective evaluation: The semantically unpredictable sentences approach for European Portuguese Free

In this paper a test proposal for European Portuguese (EP) synthetic speech quality evaluation as well as some of its application are presented. For this purpose we have built a semantically unpredictable sentences (SUS) corpus of 25 sentences with different lengths that were chosen according to the standard EP phonological, syllabic and prosodic specific features. The goal is to create a standard test linguistically focused that constitutes a base for a more accurate comparison and ranking of different synthesis techniques. The tests were conducted in two distinct conditions: one performed in a controlled environment with listeners ranging from 16 to 60 years old; the other performed through the web by a large group of listeners of different ages, each one in their own environment. The listeners had only one chance of listening to each sentence. Of course in the second case this could not be controlled. The evaluation parameters considered were the number of (1) totally well identified sentences; (2) nonidentified sentences; (3) misunderstood words in each sentence; (4) misunderstood words in the whole corpus; (5) misunderstood words according to the length of the sentence; (6) errors in the perception of a word according to its position in the sentence.