This paper focused on the challenges in educating the visually impaired and modalities for ensuring quality assurance in tertiary institutions of learning in Nigeria. It examined the global challenges in the higher educational system and made it clear that the visually impaired are those with visual problems be it partial, low vision, or total blindness, as such need higher education to be liberated from the shackles of pity and sympathy. The paper found out that the educational needs of the visually impaired include computer application, optical aids, and Braille writing materials, issues of mobility, funding, library resources, personnel availability and physical infrastructural facilities. Recommendations were proffered to tackle the issues and challenges identified.