Over 30,000 people in the 18-49 year age range were registered as blind or partially sighted in 2000. In a survey we found 58% of white goods (e.g. microwaves, etc.) having some form of display - to provide information during use or programming. Naturally, these are of little use to people who can't see them!
We are attempting to develop devices that will provide an audio representation of the information contained in the display. This is either all of the information displayed, the significant information or just the parts of the display that have changed since it was last analysed.
Our method is to define a profile for each type of display. The profile is tailored to the specific display.
We capture images using a webcam. The webcam was a deliberate choice as this is hardware that is likely to be in the possession of the target users. It also gives low quality images that might well be similar to those derived from cameras in smart devices.
The image is processed to find the extent of the display, using markers that have been placed to indicate its corners. It is also resampled to give the equivalent front-on view of the display.
The image is dynamically thresholded to separate the background and display components.
The profile is used to define the regions of the display that correspond to characters and icons. A simple test is used to indicate which portions of the display are "on" or "off", we therefore recognise characters. The profile is used to assemble a text string corresponding to the correct output. The Microsoft speech engine is used for text-to-speech translation.
Tests indicated that the method is successful if the lighting conditions are appropriate. More interesting, the blind users find no difficulty in capturing images that are useable: in focus, correct orientation, etc.
The next step of this development is to port the software to a smart device (mobile phone or PDA) and to dispense with the markers.
The system is descrbed in a paper published in the IEEE Transactions on Neural Systems and
Rehabilitation Engineering
The RNIB offers a service to transform maps into a tactile format, suitable for
Braille readers. Each map takes about a week to produce. They are sold at much less than the cost price. Our aim is to provide software that will
partially automate the conversion process, thereby speeding it up. We have or are producing software to detect coloured areas, linear features,
icons and text in maps of many types.
Last modified
March 2008