This study investigates the impact of visual and auditory secondary tasks on the driving performance of participants (14 younger and 14 older) with simulated visual impairment. Participants drove around a closed road circuit under single- and dual-task conditions. Driving performance measures included road sign recognition, detection and avoidance of low-contrast hazards, gap judgment, and time to complete the course. Driving with two levels of visual impairment was compared against a baseline condition: goggles designed to replicate the effects of cataracts or blur (uncorrected refractive error visual impairment were used to simulate), and goggles were used to reduce binocular visual acuity to a mean level of 20/40. Secondary tasks required participants to add orally pairs of numbers presented through a computer speaker (auditorily) or via a dashboard-mounted monitor (visually). Results indicate that visual impairment significantly reduces driving performance (P < 0.05) and the differences are greatest under the cataract condition. Multitasking (e.g., talking on a cell phone or using in-vehicle navigational devices) further exacerbated these effects, and the visual dual task had a greater detrimental effect on driving performance than did the auditory dual task (P < 0.05), particularly for the older drivers. Overall, results indicate that multitasking impairs driving performance and the effects are exacerbated for older drivers and younger drivers with visual impairment. This finding has important implications as driving and in-vehicle environments become increasingly complex and older people comprise the fastest-growing segment of the driving population.