An unresolved issue in speech perception concerns whether top-down lexical information influences early perceptual representations. This was addressed using the event-related potential (ERP) technique to measure semantic priming effects on the auditory N1, an index of initial acoustic cue encoding. Participants saw visual primes that formed a clear association with the target (Association: “MARCHING band”), led to no specific association (Neutral: “BUTTER bomb”), or consisted of a non-word Mask. Auditory targets were stop consonants varying in voice onset time (VOT) between voiced (/b,d,g/) and voiceless (/p,t,k/) pairs. Participants were faster to identify the initial sound of the target in the Association condition than the Neutral and Mask conditions, and ERP responses showed the expected bottom-up effect of stimulus VOT (larger N1s for shorter VOTs). In Experiment 1, Association primes produced smaller N1s when targets were perfectly predictive, suggesting a top-down attentional effect. In Experiment 2, ambiguous and unexpected VOTs were added, and the results demonstrated that ambiguous VOTs in the Association condition were encoded similarly to the voicing endpoint that matched the semantic context (i.e., larger N1s for voiced expectations). These results provide the first ERP evidence that top-down lexical information directly influences early perceptual responses.