The presence of noise during the performance of cognitive task, commonly causes a subjective experience of annoyance, which can lead to a decline in performance. This tendency is stronger for meaningful noise such as a conversation than for meaningless noise such as heating, ventilating, and air-conditioning noise. This paper first focus on the degree of meaningfulness of noise, then discuss the psychological impression of annoyance during auditory or visual cognitive tasks under the meaningful noise or meaningless noise. In addition, we considered how the brain responds on transient event-related potentials (ERPs), elicited by external auditory or visual stimuli, can be measured using electroencephalography (EEG). The present experiment was designed to determine the effects of meaningfulness of the noise on selective attention to stimuli under the odd-ball paradigm. To this end, we examined differences in the N100 and P300 ERPs of these components. The principal component analysis (PCA) was adopted to define a set of components. Our results suggested that degree of meaningfulness of noise has a strong influence on not only the psychological impression of annoyance but also the selective attention to stimuli in cognitive tasks.