The focus of this research is to acquire models of interaction automatically by observing interactions between humans. The acquired model can then be used to enable interaction between a human and a synthetic interactive partner.

In this first experiment, two individuals shaking hands are observed in side view. A large video corpus of typical handshake interactions has been used and training data has been extracted automatically by tracking individuals and representing them by their profiles. We are considering interaction as a joint behaviour between humans, therefore interaction is represented as the joint behaviour of object silhouettes. A large set of prototypes is used to form a representation of the joint behaviour space where each of these prototypes represents interaction over an extended time period. An individual prototype is not generative. It is a compressed representation of interaction (encoding temporal history) and in order to be used generatively for prediction we connect the prototypes into a Markov chain.

Having acquired the behaviour model, a stochastic algorithm (based on Isard and Blake Condensation algorithm) has been used to propagate multiple hypothesis through the behaviour model. The maximum likelihood interaction hypothesis provides a natural response to the humans actions.

For more details see:

Johnson, N., A. Galata, D. Hogg (1998). The Acquisition and Use of Interaction Behaviour Models. IEEE Computer Vision and Pattern Recognition (CVPR'98), Santa Barbara, IEEE Computer Society Press (.pdf).