Sensitivity to interaural phase difference (IPD) cues is critical for localizing sound and hearing in background noise. Recent work from multiple laboratories has explored electrophysiologic methods for assaying IPD sensitivity with the aim of developing an objective measure of binaural hearing. Such a measure would be useful in a variety of clinical and research applications such as understanding the effects of aging on binaural hearing; fine-tuning hearing aids/cochlear implants to maximize binaural benefit; and assessing neural damage after a traumatic brain injury. Here, we present data from an ongoing study evaluating a novel “multi-level” (brainstem, midbrain, and cortex) simultaneous measure of IPD acuity. Amplitude modulated (20, 40, and 80 AM) carrier tones (125, 250, and 500 Hz) with IPDs embedded in them (7 deg–180 deg) were used to generate auditory evoked potentials in young, normal hearing listeners. Neural responses corresponding to the carrier tones (frequency following responses), amplitude modulation envelopes (envelope following responses), and cortical change detection (acoustic change complex) were measured simultaneously to provide a multi-level snapshot of neural IPD processing. Parametric data presented here demonstrate feasibility and limitations of this approach for measuring IPD encoding at multiple levels of the auditory system.