Saliency-based navigation in omnidirectional image

Abstract

Omnidirectional images describe the color information at a given position from all directions. Affordable 360° cameras have recently been developed leading to an explosion of the 360° data shared on the social networks. However, an omnidirectional image does not contain interesting content everywhere. Some part of the images are indeed more likely to be looked at by some users than others. Knowing these regions of interest might be useful for 360° image compression, streaming or even editing. In this paper, a new approach based on 2D image saliency is proposed both to model the user navigation within a 360° image, and to detect which parts of an omnidirectional content might draw users' attention.

Cube projection

Instead of mapping the sphere to a single image, some works have proposed to decompose the sphere into several subregions and to map these regions to independent images. The mostly adopted approach is called the cube representation.

Optimal navigation path (green line)

Our objective is to find the optimal horizontal path given a saliency-based energy function. Below, the green line corresponds to the best navigation with respect to predicted saliency maps.








The blue line corresponds to the equator.

Video

Examples of video thumbnails when considering the equator path (left) and the saliency-based path (right).

ClassRoom

Best viewed in fullscreen.

Theatre

Best viewed in fullscreen.

Basket

Best viewed in fullscreen.

Restaurant

Best viewed in fullscreen.

Software

• Binaries will be available soon.

BibTex

Maugey, T., Le Meur, O. and Liu Z. (2017). Saliency-based navigation in omnidirectional image, MMSP , 2017.

@inproceedings{Maugey2017,
  Title = {Saliency-based navigation in omnidirectional image},
  Author = {Maugey, Thomas, Le Meur, Olivier and Liu, Zhi},
  booktitle = {IEEE 19th International Workshop on Multimedia Signal Processing},
  volume={xx},
  number={xx},
  pages={xx},
  year={2017},
  organization={IEEE}
}