Phonetic variation in speech informs listeners not only about the linguistic message but also about talkers (e.g., gender, age, and emotion). In most episodic theories of speech perception, this indexical variation is accommodated via an acoustically-detailed exemplar lexicon. This view assumes lexical and indexical information are coupled, but speakers use acoustic patterns productively to convey information independent of the words they utter. We investigated the effects of emotional prosody on word recognition to test whether indexical information affects word recognition independent of lexical information. First, we compared the recognition of emotion word targets (UPSET) preceded by semantically unrelated words spoken with emotionally related or unrelated prosody (pineapple[AngryVoice] or pineapple[NeutralVoice]). Second, we investigated the effects of emotional prosody on semantically-related target words (pineapple[AngryVoice] or pineapple[NeutralVoice]—FRUIT). Recognition of both emotionally related and semantically related targets was facilitated by prime words spoken with angry prosody. These data suggest that indexical variation in speech influences word recognition beyond acoustically-detailed lexical representations. We suggest listeners simultaneously process acoustic variation for indexical and lexical meaning and argue that emotional prosody activates emotion features and categories, independent of lexical access.