The classic view of sensorineural hearing loss (SNHL) is that the “primary” targets are hair cells, and that cochlear-nerve loss is “secondary” to hair cell degeneration. Our work in mouse and guinea pig has challenged that view. In noise-induced hearing loss, exposures causing only reversible threshold shifts (and no hair cell loss) nevertheless cause permanent loss of >50% of cochlear-nerve/hair-cell synapses. Similarly, in age-related hearing loss, degeneration of cochlear synapses precedes both hair cell loss and threshold elevation. This primary neural degeneration has remained hidden for two reasons: (1) the spiral ganglion cells, the cochlear neural elements commonly assessed in studies of SNHL, survive for years despite loss of synaptic connection with hair cells, and (2) the degeneration is selective for cochlear-nerve fibers with high thresholds. Although not required for threshold detection in quiet (e.g., threshold audiometry, auditory brainstem response threshold), these high-threshold fibers are critical for hearing in noisy environments. Our research suggests that (1) primary neural degeneration is an important contributor to the perceptual handicap in SNHL, and (2) noise exposure guidelines should be re-evaluated, as they are based on the faulty premise that threshold audiograms are the most sensitive measures of noise damage.