Social isolation and loneliness are significant risk factors for overall health, especially for older adults with vision or hearing impairment. This study was designed to understand the impact of the COVID-19 pandemic restrictions on the social /emotional wellbeing in adults with sensory impairment. Three groups of older adults -- vision loss (VL, N = 13), hearing loss (HL, N = 12 with cochlear implants and N = 12 with hearing aids), and controls (CTL, N = 18) -- were recruited from the Twin Cities community. They were interviewed at 6-week intervals from April 2020 to July 2021 concerning their wellbeing and social isolation. Overall, results show the widespread impact of the pandemic on social interactions and emotional wellbeing of older adults with sensory loss. All three groups had worse scores on the patient health questionnaire (PHQ-9) after the start of the pandemic. Older people with visual impairments were especially vulnerable to changes that led to a loss of independence. Although facial coverings impaired speech intelligibility for older people with hearing loss, the quieter acoustic environment during stay-at-home restrictions of gatherings may have mediated the negative effects. Results highlight unique factors that have affected older persons with sensory loss.