How to measure the relevance of a retargeting approach?

Abstract

Most cell phones today can receive and display video content. Nonetheless, we are still significantly behind the point where premium made for mobile content is mainstream, largely available, and affordable. Significant issues must be overcome. The small screen size is one of them. Indeed, the direct transfer of conventional contents (\textit{i.e.} not specifically shot for mobile devices) will provide a video in which the main characters or objects of interest may become indistinguishable from the rest of the scene. Therefore, it is required to retarget the content. Different solutions exist, either based on distortion of the image, on removal of redundant areas, or cropping. The most efficient ones are based on dynamic adaptation of the cropping window. They significantly improve the viewing experience by zooming in the regions of interest. Currently, there is no common agreement on how to compare different solutions. A retargeting metric is proposed in order to gauge its quality. Eye-tracking experiments, zooming effect through coverage ratio and temporal consistency are introduced and discussed.

Paper's figures

Click on the pictures to look at the original full resolution pictures.

• Figure 1:

Example of a retarget picture with the seam-carving approach. Original picture on the left; retargeted picture on the right.




• Figure 2:

Pictures extracted from the Sports clip. Red points correspond to visual fixations from eye-tracking experiments. Red boxes are the cropping windows.




• Figure 3:

General description of the proposed automatic retargeting process. Main operations are the visual attention model, the cropping window extraction and the temporal consistency.




• Figure 4:

Visual comparison of still pictures for the seam carving and dynamic reframing schemes. Top row is the computed saliency heat maps (the reddish pixels are salient, the blue ones are not). Second row is the original picture with the cropping window in white. Third row is the resulting cropped picture.




• Figure 5:

Percentage of human fixation points in cropping window. Blue squares indicate the average values over time (the standard deviation is also given). The pink triangle and the purple diamond respectively correspond to the frame with the lowest percentage and the average of the ten percent lowest values.




• Figure 6:

Temporal evolution of the center of the cropping window (just the spatial coordinate X is presented) on the left-hand side and the coverage value on the right for the Sports clip. Concerning the position of the cropping window, the temporal evolution is given after the cropping window extraction, the Kalman and the median filter.

Supplementary materials

A clip is available on the home page of P. Guillotel here (600 Mo!!). It shows the result of the proposed retargeting. An example is given below. On the left-hand side, this is the original video clip, on which the cropping window is drawn (white box). On the left, this is the retargeted video (only the content of the white box is displayed).

Citation

Chamaret, C., Le Meur, O., Guillotel, P. and Chevet, J.C. (2013). How to measure the relevance of a retargeting approach?, workshop ECCV'2010. .

@CONFERENCE{Chamaret2010,
 author = {Chamaret, C., Le Meur, O., Guillotel, P. and Chevet, J.C.},
 title  = {How to measure the relevance of a retargeting approach?},
 booktitle= {Workshop Media Retargeting, ECCV 2010},
 year   = {2010},
}