Looking into Artificial Vision for the Blind

The Braille Monitor – November, 2000 Edition

by Peter M. Scialli, Ph.D.

From the Editor: Peter Scialli is no stranger to Braille Monitor readers, but we usually associate his name with Shrinkwrapped Computing, the company he runs. This time, however, he has been thinking about his original interest, the working of the human brain, for Peter's Ph.D. is in psychology.

Like many others of us, he has been disturbed by recent misinformation and exaggerated claims in the media about artificial vision. I always dread announcements that some scientist has made a breakthrough in this field. I know that my telephone will soon be ringing off the hook with people hoping that their own or a loved one's vision can be restored. They can be pretty frantic in their desire to regain something that has usually been quite recently lost, and the last thing they want to hear is cautious statements about sober reality and minuscule advances.

In the following article Dr. Scialli explains the most recent research in artificial vision and places it in the proper context. This should help us all deal more effectively with those cockeyed optimists who are sure that science is about to turn our lives right-side-up, or is it upside-down? This is what he says:

On January 18, 2000, an article appeared in the New York Times which described the invention of "Artificial Vision for the Blind." The Times piece, along with coverage of the same news in other print and broadcast media, gave rise to a brief but intense flood of information on the technology responsible for the story. My purpose for writing this article is not to cast doubt on the technology, for it appears to be scientifically sound, but rather to call attention to the near-term harm that a popular news story, particularly one proclaiming artificial vision, may cause for blind people everywhere.

The news story is the result of an original paper published by Dr. William Dobelle in the Journal of the American Society of Artificial Internal Organs. In this paper Dr. Dobelle describes a volunteer patient named Jerry who was able to perform some overtly visual tasks as a result of having a camera and computer connected to his brain's visual cortex through surgically implanted electrodes. As we learn from the paper, the images that the patient "sees" are actually composed of a small number of white dots which he views against his otherwise dark visual field. The arrangement of these dots corresponds roughly to what is viewed by a small electronic pinhole camera affixed to Jerry's eyeglasses. In Dobelle's research Jerry can see as many as twenty phosphines or points of light in his visual field. These points of light are roughly analogous to picture elements on a computer or television screen. Note that by today's standards a computer monitor of even average quality will display 2,400 or more picture elements.

Still, with practice Jerry was able to perform some simple visual tasks using the feedback produced by his artificial vision.

Some general description of the human visual system is in order to help illustrate the similarity (or difference) between true eyesight and artificial vision as it now exists or is likely to exist in the foreseeable future. Unlike a video camera the human eye does not simply convert individual points of light into corresponding electrical signals. Instead, that is only the first step in a vast network of biochemical processes which result in human vision. The eye itself does nothing more than focus incoming light onto the retina.

The retina does not simply convert light to electricity, however. The retina is composed of at least two layers of cells which do much to process incoming information. The cells of the retina are actually more closely related to the cells of the brain than to any other material in the body. As such, they have a high capacity to act differentially in accordance with a given set of circumstances. They do not simply react like a digital circuit where a value is either "on" or "off." The incoming light image is actually focused on the innermost layer of cells, where it is differentiated primarily according to its color or lack thereof. This outer nuclear layer, as it is called, is where the rod cells and cone cells reside. The cones react to color, whereas the rods generally do not.

The electrochemical signals sent from the outer layer back toward the front of the retina to what is called the inner nuclear layer carry a great deal of information about color, contrast, brightness, position, and movement. Further processing is done in the inner layer. Some cells are excited by the presence of color whereas others are inhibited. Still other cells have different regions which react differently depending on a combination of connections made to them. In brief, a great deal of computing power has been applied to incoming light signals before they ever get to the visual cortex, which is located at the top and rear of the human brain.

Following processing by the retina, visual information is sent along the optic nerve toward the visual cortex. However, this is not a straight path. The optic nerve has branches which themselves may have branches to other parts of the brain. Some of these are involved in memory and recognition. Others may be involved in shaping the physiological response to visual input, forming associations and so on.

When signals reach the visual cortex via the optic nerve, they influence not only the surface of the brain but many layers of cells beneath it. The nerve again branches, the branches branch, those branches branch, and so on. In addition, every cell influences the behavior of the cells around it, and in turn each cell is influenced by a combination of inputs from many places. To add to the complexity, the entire system changes many times each second. As confusing as this description is, it only superficially describes the known mechanisms of human vision. Much more is still unknown.

In striking contrast is the artificial vision system described in Dr. Dobelle's article. The world, as seen by a very limited camera, is reduced by the computer to a few dozen discrete points of light. These points are either on or off. As described by Dr. Dobelle, Jerry learned to interpret the meaning of the visual impressions more quickly than was expected. After a good deal of practice, Jerry could place a hat on the head of a mannequin using only the feedback provided by the artificial vision. He could also identify the figure of a capital letter E lying on its back.

As reported in the original paper, the apparatus provides a very limited field of view such that Jerry had to scan the area around him using repeated head movements in order to interpret it correctly. Imagine trying to read a sentence in Braille in which each Braille cell is the size of a restaurant menu and each dot the size of a golf ball. It would take considerable time and effort to accomplish the task.

What is apparent in Dr. Dobelle's paper, but not so apparent in the popular reports of artificial vision, is that what the patient experiences bears little resemblance to human eyesight. The circuitry which provides the artificial vision is limited to direct stimulation of points along the surface of Jerry's visual cortex. Given enough of these points, each resulting in the appearance of a spot of light, the patient can respond appropriately. It is unlikely that Jerry can describe or act upon new visual data without being told what it is and practicing with the arrangement of light spots it produces. Dr. Dobelle holds out hope that, if enough discrete points of light can be displayed in the patient's visual field, Jerry may one day be able to view the world in a way similar to viewing an instant replay on the lighted scoreboard of a sports stadium.

If this degree of resolution is attained, then the patient may be able to engage in some description of or reaction to his visual surroundings. Dr. Dobelle even suggests that one day gray scaling of the images may be possible. He does not, however, hold out any hope that color vision will ever be possible with his system. He notes that the capacity for one's visual cortex to respond to color quickly diminishes if that capacity is not used for a prolonged period of time.

Dr. Dobelle and others have been working toward producing functional artificial vision for many years. He has nearly arrived at a commercially viable product. The question that we must ask is, has he arrived at eyesight where there was none before? Through this superficial description of the human visual process, I hope I have conveyed that human vision, and other physiological functions, comprise an unbelievably vast and poorly understood collection of very complex, very rapid interactions.

I want to be clear that, if we have been misled, it has not been by Dr. Dobelle. He writes forthrightly about the capabilities as well as the deficiencies of his work. He imagines, and correctly so, that at some point in the future researchers will create good, though not perfect, vision as a result of his pioneering work. For the present, however, he imagines that artificial vision may be able to take its place alongside a white cane or guide dog as a useful aid for orientation and mobility. It is clear, however, that we are still several years from creating artificial vision that matches, let alone surpasses the more traditional mobility devices.

The public, and to a lesser extent the blind, have been misled by the media. The same mass media that hold blind people out as helpless creatures to be pitied have, as a result of Dr. Dobelle's recent success, now trumpeted the discovery of a cure for the terrible blight of blindness. The artificial vision enjoyed by Jerry is in reality a stepping stone toward some truly astounding developments that will come only over the course of decades. They represent marvelous science whose practical use for the average blind person will appear in the distant future.

Given the limitations of artificial vision as described, why then pursue it at all? The reasoning, as I will describe, is no different from other attempts to address the issue of blindness. In Dr. Dobelle's abstract--the summary of the paper which appears directly under the title and author's name--these are the first words: "Blindness is more feared by the public than any ailment except cancer and AIDS." Unfortunately, Dr. Dobelle is probably right in this observation. His recognition of this fear provides the motivation to create a commercially viable system of artificial vision.

It is plausible to assume that there will be no shortage of funding for Dr. Dobelle's research and development in artificial vision given that for most of the public eliminating blindness is as urgent a calling as eliminating cancer or AIDS. It is natural to fear these diseases and perhaps to some extent to fear those afflicted with them. At the root of many of the problems faced by the blind are the public attitudes of fear, concern, and awe. If an injection was announced one day which would instantly eliminate AIDS or cancer, we would all applaud it as a miracle of science. We would not understand patients' reluctance to submit to the injection, nor would we be supportive of their desire to continue through the rest of their life using more traditional, less miraculous methods of coping.

The blind are confronted with this circumstance exactly. Since the New York Times article, among others, how many of us have been told by well meaning acquaintances that a cure for blindness now exists? How many of us have been encouraged to write a letter or otherwise make contact with someone who can give us the gift of artificial vision? Keep in mind that for the present the so- called gift of artificial vision is described by its creator as being not yet as useful as a long white cane or guide dog. Also note that the artificial vision enjoyed by Jerry requires surgical implantation of electrodes into one's brain.

I believe that we should thank Dr. Dobelle as well as Jerry and other volunteer patients for their hard work and personal sacrifice. They are doing ground-breaking work in a field that in perhaps the twenty-second century will eliminate blindness for most people. It is profoundly unfortunate that their work may in the short term produce increased misunderstanding of blindness and the capabilities of blind people.

(back) (next) (contents)