VisionAssist: Enhancing Accessibility for Individuals with Visual Impairment Through AI

This project explores how AI can support individuals with visual impairments by developing a system that converts images containing text or math into audio and Braille in near real-time. Using a fine-tuned OCR model, the system achieves high accuracy and low latency, demonstrating that AI can be a powerful tool for improving accessibility to educational content.

Millions of people around the world are partially or totally blind. Such people face significant challenges in their educational pursuits and therefore they often struggle in life. Can Artificial Intelligence and latest developments in technology help visually impaired people understand literary content easier? To address this question, I have developed an Artificial Intelligence based proof of concept system that accurately converts any image with textual/mathematical data into Braille and Audio mediums as close to real-time as possible. The system takes an image as input and extracts text from it through optical character recognition (OCR) using a finetuned and optimized version of opensource PaddlePaddle OCR model. The system then has two different output options, an audio-based format, and a braille-based tactile format. Word Error Rate (WER) and Character Error Rate (CER) were used as evaluation metrics to evaluate performance of the system on different types of data. My system achieves a weighted average WER of approximately 7% with an average processing time of around 1 sec per 500 characters. The evaluation data shows that the system achieved the goal of having an accurate detection of the data in near real-time speed. My prototype outperforms much bigger and commercial models like ChatGPTv4 in terms of latency as well. This project shows that technology and AI can be an additional tool to help visually impaired people better understand educational content which can eventually uplift them in life.