Small differences in the arrival time of sound between the two ears [interaural time differences (ITDs)] provide important cues for directional hearing and speech understanding in noise. Deaf subjects with bilateral cochlear implants (CIs) show relatively poor sensitivity to ITDs. It is unclear whether these limitations are due to differences in binaural brain circuits activated by electric compared to acoustic stimulation, mismatches in electric activation site across ears, or deafness-induced degradations in neural ITD processing. To identify potential limitations of electric ITD coding, we compared electric and acoustic ITD coding (i.e., ITD tuning and discrimination thresholds) in the same population of auditory brainstem and midbrain neurons in normal hearing gerbils. When compared in the same neurons, ITD coding to acoustic stimulation did not predict coding to electric stimulation. However, on a population level, neurons demonstrated surprising similarities in acoustic and electric ITD processing. Importantly, even short periods of deafness (2 weeks) severely degraded electric ITD processing. The findings suggest that discrepancies in ITD discrimination between bilateral CI users and normal hearing listeners are primarily due to deafness-induced changes in neural ITD processing rather than differences in the binaural brain circuits activated by either electric or acoustic stimulation.