Stefano Soatto

Title: "Actionable Information as The Link Between Vision and Control"

Abstract: I will discuss a notion of visual information as complexity _not_ of the raw data, but of the 
images after the effects of nuisance factors such as viewpoint and illumination are discounted. It is 
rooted in ideas of J. J. Gibson, and stands in contrast to traditional information as entropy or coding 
length of the data regardless of its use, and regardless of the nuisance factors affecting it. Its 
computation is made possible by a recent characterization of the set of images modulo viewpoint and 
contrast changes, that induce group (invertible) transformations on the domain and range of the image. 
The non-invertibility of nuisances such as occlusion and quantization induces an ``information gap'' that 
can only be bridged by controlling the data acquisition process.Measuring visual information entails 
early vision operations, tailored to the structure of the nuisances so as to be ``lossless'' with respect 
to visual decision and control tasks (as opposed to data transmission and storage tasks implicit in 
traditional information theory). I illustrate these ideas on visual exploration, whereby a ``Shannonian 
Explorer'' navigates unaware of the structure of the physical space surrounding it, while a ``Gibsonian 
Explorer'' is guided by the topology of the environment, despite measuring only images of it, without 
performing 3D reconstruction. This operational definition of visual information suggests desirable 
properties that a visual representation should possess to best accomplish vision-based decision and 
control tasks.