SHIVPRO: An EU FP7 Marie-Curie Project

Home Context Objectives People Publication Activities

Context

Saliency-aware High-resolution Video Processing (SHIVPRO)
PF7-PEOPLE-2011-IIF

Objectives

• A new spatiotemporal saliency model for efficiently processing high-resolution videos:
  This project will propose a new framework to efficiently compute the spatiotemporal saliency of high resolution video. It will bring distinct advantages compared to existing works and will ease the design of saliency based applications. Dr. Zhi Liu and Dr. Olivier Le Meur collaborate on developing this novel spatiotemporal saliency model, and demonstrate its performance in the emerging high-resolution video processing applications as follows:

• Spatiotemporal salient object detection;
• On the basis of HEVC, we will further exploit the visual redundancy for more efficient compression of high-resolution videos with the aid of saliency model;
• Efficient video retargeting to adapt to various displays with different resolutions.

People

Zhi Liu received the B.E. and M.E. degrees from Tianjin University, Tianjin, China, and the Ph.D. degree from Institute of Image Processing and Pattern Recognition, Shanghai Jiaotong University, Shanghai, China, in 1999, 2002, and 2005, respectively. He is currently a Professor with the School of Communication and Information Engineering, Shanghai University, Shanghai, China. From Aug. 2012 to Aug. 2014, he was a Visiting Researcher with the SIROCCO Team, IRISA/INRIA-Rennes, France, with the support by EU FP7 Marie Curie Actions. He has published more than 120 refereed technical papers in international journals and conferences. His research interests include saliency models, image/video segmentation, image/video retargeting, video coding, and multimedia communication. He was a TPC member in ICME 2014, WIAMIS 2013, IWVP 2011, PCM 2010, ISPACS 2010, etc. He co-organized special sessions on visual attention, saliency models, and applications at WIAMIS 2013 and ICME 2014. He is an area editor of Signal Processing: Image Communication and served as a guest editor for the special issue on Recent Advances in Saliency Models, Applications and Evaluations in this journal. He is a senior member of IEEE.

Olivier Le Meur received the Ph.D. degree from the University of Nantes in 2005. From 1999 to 2009, he worked in the media and broadcasting industry. In 2003, he joined the research center of Thomson-Technicolor at Rennes where he supervised a research project concerning the modeling of human visual attention. He has been an Associate Professor for image processing at the University of Rennes 1 since 2009. In the SIROCCO team of IRISA/INRIA-Rennes, his research interests are dealing with the understanding of the human visual attention. It includes computational modeling of the visual attention and saliency-based applications (video compression, objective assessment of video quality, retargeting).

Publication

Journals

6. J. Li, Z. Liu, X. Zhang, O. Le Meur, and L. Shen, Spatiotemporal saliency detection based on superpixel-level trajectory, Signal Processing: Image Communication, vol. 38, pp. 100-114, Oct. 2015.
5. O. Le Meur, and Z. Liu, Saccadic model of eye movements for free-viewing condition, Vision Research, vol. 116, part B, pp. 152-164, Nov. 2015. [Presentation]. [project page]
4. Z. Liu, X. Zhang, S. Luo and O. Le Meur, Superpixel-Based Spatiotemporal Saliency Detection, IEEE Transactions on Circuits and Systems for Video Technology, vol. 24, no. 9, pp. 1522-1540, Sep. 2014. [Code] [Salient object detection in video]

3. Z. Liu, W. Zou and O. Le Meur, Saliency Tree: A Novel Saliency Detection Framework, IEEE Transactions on Image Processing, vol. 23, no. 5, pp. 1937-1952, May 2014. [pdf]

Materials: [Code] [Ground truths] [Proposed saliency maps]

Note that the test images and the original ground truths of the five datasets can be downloaded from the websites for different papers including [34], [35], [23], [56], [57], [58], [51], [59] (see Section VI-A in our paper). For evaluating saliency detection performance, we provide here the ground truths used in our paper.

• For MSRA and ASD datasets, we made corrections on the binary ground truths for the following 10 images, which are not accurately labelled in the original ground truths.
3_95_95588.png; 3_95_95642.png; 3_95_95684.png; 3_95_95922.png; 3_96_96016.png; 3_97_97769.png; 3_97_97770.png; 3_97_97915.png; 3_98_98012.png; 3_109_109233.png.
• For SED and SOD datasets, the binary ground truth for salient object(s) in each image is generated based on the results labelled by multiple users in the original datasets.
• PASCAL-1500 is a challenging dataset introduced by us for saliency detection.

2. Z. Liu, W. Zou, L. Li, L. Shen and O. Le Meur, Co-saliency detection based on hierarchical segmentation, IEEE Signal Processing Letters, vol. 21(1), 2014 [pdf] [Results] [IF=1.674].

1. Z. Liu, O. Le Meur, S. Luo and L. Shen, Saliency detection using regional histograms, Optics Letters, Vol. 38, N°5, March 2013 [pdf] [Results] [IF=3.39].

Conferences

2014

1. L. Li, Z. Liu, W. Zou, X. Zhang, and O. Le Meur, Co-saliency detection based on region-level fusion and pixel-level refinement, International Conference on Multimedia Expo (ICME), Chengdu, July 2014.
2. X. Zhang, Z. Liu, H. Li, X. Zhao, and P. Zhang, Statistical moving object detection based on imbalanced learning, International Conference on Multimedia Expo (ICME), Chengdu, July 2014.
3. O. Le Meur, and Z. Liu, Saliency aggregation: Does unity make strength?, Asian Conference on Computer Vision (ACCV), Singapore, Nov. 2014.

2013

1. Z. Liu, O. Le Meur and S. Luo, Superpixel-based saliency detection, WIAMIS 2013, [pdf] [Presentation].
2. S. Luo, Z. Liu, L. Li, X. Zou and O. Le Meur, Efficient saliency detection using regional color and spatial information, EUVIP, 2013 [pdf].

Activities

• Special Issue: Zhi Liu, Olivier Le Meur, Ali Borji and Hongliang Li (Guest Editors), Recent Advances in Saliency Models, Applications and Evaluations, Signal Processing: Image Communication, vol. 38, Oct. 2015. [link to this special issue]
• Special session Visual saliency: emerging models and applications in multimedia processing at ICME, Chengdu, July 2014. (Co-organized by Zhi Liu, Olivier Le Meur and Xiang Zhang)
• Special session Visual attention, a multidisciplinary topic: from behavioral studies to computer vision applications at WIAMIS, Paris, July 2013. (Co-organized by Olivier Le Meur and Zhi Liu):
• Konstantinos Apostolakis* and Petros Daras, An application framework for implicit sentiment human-centered tagging using attributed affect, [Presentation]
• Zhi Liu*, Olivier Le Meur and Shuhua Luo, Superpixel-based saliency detection, [Presentation]
• Jie Sun and Congyan Lang*, Sample Specific Late Fusion for Saliency Detection
• Furkan Isikdogan* and Albert Salah, Affine invariant salient patch descriptors for image retrieval
• Antoine Coutrot* and Nathalie Guyader, Toward the introduction of auditory information in dynamic visual attention models

• A talk Salient object detection and segmentation was presented at IRISA/INRIA-Rennes (Nov. 30, 2012), INSA-Rennes (May 21, 2013), and LUTIN Lab, Paris (Oct. 1, 2013).
• A talk High-performance region-based saliency detection for images and videos was presented at Technicolor, Rennes (Apr. 9, 2014) and IRCCyN, University of Nantes (Apr. 24, 2014).