Bringing Vision To The Blind With Unity Game Tech

My mum suffers from a whole bunch of complications related to diabetes including multiple amputations, kidney failure, micro-strokes, nerve damage, poor circulation and blindness. Despite all this she's pretty technically savvy and has adapted to use voice controlled technology like Siri pretty damned well. Although Siri often messes up because of the stroke related slurred speech.

During a hospital stay in 2015 there was a big, sunny window next to her bed. We noticed that she could easily see the silhouettes of people walking past, and get a general sense of size and shape of the person. Although she could often tell if someone had a raised arm, she wasn't able to distinguish if the person was holding up any fingers.

The fact that she could see very large shadows sparked of thoughts in my mind about how this could be used for her advantage. I had my laptop and my Google Cardboard with me, so I got to work to build something with a high contrast that looked huge though the VR glasses.

It didn't take long until I built what could only be described as a giant wall clock. A simple affair, that was white on black. It had two buttons to switch the display between "Time" and "Date". When gazing at the buttons they grew in size to become more visible. Originally I had two sets: white on black; and black on white, but after some testing it became clear that white on black was easier for her to see. The pointer events for the buttons were quickly hooked up to Google's text-to-speech to give audio feedback as to which button was being selected.

After the build I gave it to my mum without telling her anything about it and just asked her to have a look and see if she could see anything. After a short amount of time she asked, "Does that say... 3:34pm?". I replied that it does and she became quite excited! after a little while she exclaimed, "It just changed! It now says 3:35pm!".

The excitement in her voice was amazing and I still get a bit teary now thinking about it.

Even though she has adapted to voice controlled/feedback devices, including a watch that will tell her the time, the novelty that she could actually see the time for herself for the first time in 5 years brought her so much joy and a small sense of independance.

We know that games often help pioneer new technologies like video and audio by pushing hardware requirements, but they can also be used to push the boundaries of assistive technologies.

I'm grateful that my interest in game development, and the free technologies from Google and Unity has been able to bring a sense of independance to my mum.

At the moment I am saving up to get a couple of pairs of the Moverio BT-200 Smart Glasses, as I'm sure I can use the AR in this to provide augmented visual enhancement through audio feedback and give her even more independence both in her home and outside.