Prediction of the Inter-Observer Visual Congruency (IOVC) and application to image ranking

Abstract

This paper proposes an automatic method for predicting the inter-observer visual congruency (IOVC). The IOVC reflects the congruence or the variability among different subjects looking at the same image. Predicting this congruence is of interest for image processing applications where the visual perception of a picture matters such as website design, advertisement, etc. This paper makes several new contributions. First, a computational model of the IOVC is proposed. This new model is a mixture of low-level visual features extracted from the input picture where model's parameters are learned by using a large eye-tracking database. Once the parameters have been learned, it can be used for any new picture. Second, regarding low-level visual feature extraction, we propose a new scheme to compute the depth of field of a picture. Finally, once the training and the feature extraction have been carried out, a score ranging from 0 (minimal congruency) to 1 (maximal congruency) is computed. A value of 1 indicates that observers would focus on the same locations and suggests that the picture presents strong locations of interest. A second database of eye movements is used to assess the performance of the proposed model. Results show that our IOVC criterion outperforms the Feature Congestion measure [Rosenholtz et al., 2007]. To illustrate the interest of the proposed model, we have used it to automatically rank personalized photograph.

Paper, presentation and poster

• ACM 2011 (full paper) [pdf]
• Presentation [pdf]
• Poster [pdf]

Some figures of the paper

• Examples of pictures associated with their corresponding inter-observer congruency. IOVC is in the range of 0 (strongest) to 1 (lowest).
  
  
• Image ranking application (based on the prediction of inter-observer dispersion): the 49 pictures below, containing various contents have been sorted out in function of their interestingness (from top-left (highest congruency) to bottom-right (lowest congruency)).
  
  

Citation

Mancas, M. and Le Meur, O. (2013). Memorability of natural scene: the role of attention, ICIP 2013 .

@INPROCEEDINGS{LeMeur_2011,
  author    = {O. {Le Meur} and T. Baccino and A. Roumy},
  title     = {Prediction of the Inter-Observer Visual Congruency (IOVC) and application to image ranking},
  booktitle = {ACM Multimedia},  
  year      = {2011}
}}