IP: AS IF YOU WERE THERE: Matching Machine Vision to Human Vision

[Dave Farber <dave () farber net>]

http://hybridvigor.net/human/pubs/index.html

AS IF YOU WERE THERE: Matching Machine Vision to Human Vision

Richard Jay Solomon
University of Pennsylvania
Program on Vision Science & Advanced Networking

ABSTRACT & ACKNOWLEDGMENTS

A large variety of advanced electronic imaging equipment is available for
collecting and disseminating visual information. However, despite the
ever-expanding capabilities of new devices, the limiting factor for
understanding and reacting to information displayed and collected is the
human perceptual system.

Where science has had any discernable input (other than guesswork and
trial-and-error) in the design of photographic and television systems,
parameters for imaging systems have been set primarily via psychophysical
measurements. Psychophysics, the study of human reactions to physical
stimuli or input, is limited to determining reactions from external stimuli;
it does not study how the brain works. It is difficult to explain just how
the human perceptual system reacts to these stimuli. What has emerged as a
key design problem for a system that more accurately replicates "presence"
is that much of the psychophysical data used in the past to engineer
high-performance networked imaging systems ‹ i.e., simple reactions to
stimuli ‹ is not consistent with the known workings of the human
neurological system, which tries to explain how or why we react.[1]

The latest neurological data, using direct brain scanning devices such as
functional magnetic resonance imaging (fMRI) and positron emission
tomography (PET) scans [2], indicate that the human sensory system is much
more sensitive, yet at the same time is much more selective to information
stimuli from visual displays and auditory sources, than had been previously
understood by engineering designers. Seemingly contradictory, these dual
insights imply that the designs of information transmitting, storage and
processing devices have to be more closely coupled to the human perceptual
system in order to gain a better "impedance" match between opto-electronics
and our neurons.[3]

Compared to older theories based on psychophysical measurements, many of the
results recently published in the neurological literature about how the
human vision system works are surprising and counter-intuitive. This new
research forces us to question long-held assumptions about how electronic
transmission components, cameras, displays, processors, and even audio
speakers should work. We can use this new information to design much more
accurate and believable electronic systems that would replicate a scene as
if the observer were present.

That is to say, for critical scientific, medical, archival and engineering
objectives ‹ contrary to the design of consumer entertainment appliances ‹
it is simply not acceptable to discard potential perceptual inputs to the
human neurological system based on erroneous ideas of what we can perceive
and not perceive, even if the picture looks "pretty good" to the untrained
observer, or at least good enough based on what we've become accustomed to,.
The human vision system is much too complicated and capable for us to settle
for simplistic design objectives such as those found in most off-the-shelf
compression, transmission and display systems. Making pretty pictures for
television and movies is easy compared to providing critical information for
technical analysis and archival storage.

[0.1] Acknowledgments

I wish to thank my colleagues Eric Rosenthal of Creative Technology LLC,
David Farber at the University of Pennsylvania, and Tice de Young of the
National Aeronautics and Space Administration who kindly contributed some of
the concepts in this paper. I take responsibility for all
misinterpretations, however.

For archives see:
http://www.interesting-people.org/archives/interesting-people/