Image Representation, Editing and Perception

Objectives

The objective of this lecture is twofold. On the one hand, we will study the different tools classically used for representing, processing and editing images. At the crossroad of mathematics and informatics, these tools will be studied under the perspective of classical problems in image processing as for example image inpainting or denoising. On the second hand, we will introdue the basics of deep learning for image processing. In this context, we will present computational models of visual perception (e.g. saliency models) aiming to detect in an automatic manner the most salient areas of an image.

To summarize, during these courses, the students will study how images are handled along the whole image/video processing chain: from the way they are represented until the way they are perceived.

Teaser

Lectures

Lectures will take place in (To be defined). (TM=Thomas MAUGEY; OLM=Olivier LE MEUR)

• Lecture 1 (TM) 14/09/2020: Introduction [slides]
• Lecture 2-3 (TM) 14-16/09/2020: Projection models for perspective and spherical cameras [slides]
• Lecture 4 (TM) 16/09/2020: Pixel Organisation and Representation [slides]
• Lecture 5 (TM) 21/09/2020: Color representation [slides]
• Lecture 6-7-8 (TM) 21-23/09/2020: Transforms and dictionaries [slides]
• Lecture 9-10-11-12 (OLM) 28-30/09/2020: Image processing. Tools and applications [slides]
• Lecture 13-14 (OLM) 05/10/2020: Basic of deep learning for image processing [slides]
• Lecture 15-16-16-17 (TM) 07-12/10/2020: Advanced methods for image and video processing [slides] [slides]
• Lecture 19-20 (OLM) 14/10/2020: Example of Computational models of visual attention based on deep learning [slides]

Evaluation

The final score will be composed of two marks:

• 04/11/2020: written exam (2h)
• 09/11/2020: a research paper to read, summarize and present

Research papers (2019-2020)

• GRAPH: Chen, S., Varma, R., Sandryhaila, A., & Kovacevic, J. (2015). Discrete Signal Processing on Graphs: Sampling Theory, IEEE transactions on signal processing, 63(24), 6510-6523.
• GRAPH: Thanou, D., Chou, P. A., & Frossard, P. (2016). Graph-based compression of dynamic 3D point cloud sequences. IEEE Transactions on Image Processing, 25(4), 1765-1778.
• GRAPH: Kovnatsky, A., Bronstein, M. M., Bronstein, A. M., Glashoff, K., & Kimmel, R. (2013, May). Coupled quasi-harmonic bases. In Computer Graphics Forum (Vol. 32, No. 2pt4, pp. 439-448). Oxford, UK: Blackwell Publishing Ltd.
• LIGHT FIELD: Hog, M., Sabater, N., & Guillemot, C. (2017). Superrays for Efficient Light Field Processing. IEEE Journal of Selected Topics in Signal Processing, 11(7), 1187-1199.
• SALIENCY+DEEP: Tavakoli, H. R., Borji, A., Rahtu, E., & Kannala, J. (2019). DAVE: A Deep Audio-Visual Embedding for Dynamic Saliency Prediction. arXiv preprint arXiv:1905.10693.
• HDR+DEEP: Endo, Y., Kanamori, Y., & Mitani, J. (2017). Deep reverse tone mapping. ACM Trans. Graph., 36(6), 177-1.
• 360+DEEP: Monroy, R., Lutz, S., Chalasani, T., & Smolic, A. (2018). Salnet360: Saliency maps for omni-directional images with cnn. Signal Processing: Image Communication, 69, 26-34.

Research papers (2018-2019)

• SPARSITY: Benoît, L., Mairal, J., Bach, F., & Ponce, J. (2011, June). Sparse image representation with epitomes. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on (pp. 2913-2920). IEEE. [pdf]
• GRAPH: Rotondo, I., Cheung, G., Ortega, A., & Egilmez, H. (2015). Designing sparse graphs via structure tensor for block transform coding of images. APSIPA ACS, Hong Kong, China. [pdf]
• GRAPH: Egilmez, H. E., Chao, Y. H., Ortega, A., Lee, B., & Yea, S. (2016, September). GBST: Separable transforms based on line graphs for predictive video coding. In Image Processing (ICIP), 2016 IEEE International Conference on (pp. 2375-2379). IEEE. [pdf]
• LIGHT FIELD: Frigo, O., & Guillemot, C. (2017, September). Epipolar Plane Diffusion: An Efficient Approach for Light Field Editing. In British Machine Vision Conference (BMVC). [pdf]
• FOURIER: Ng, R. (2005, July). Fourier slice photography. In ACM transactions on graphics (TOG) (Vol. 24, No. 3, pp. 735-744). ACM. [pdf]
• GRAPH: Zhang, C., Florêncio, D., & Chou, P. A. (2015). Graph signal processing-a probabilistic framework. Microsoft Res., Redmond, WA, USA, Tech. Rep. MSR-TR-2015-31. [pdf]
• LIGHT FIELD: Tao, M. W., Hadap, S., Malik, J., & Ramamoorthi, R. (2013). Depth from combining defocus and correspondence using light-field cameras. In Proceedings of the IEEE International Conference on Computer Vision (pp. 673-680). [pdf]
• DICTIONARY: Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2010). Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11(Jan), 19-60. [pdf]
• OBJECT DETECTION: Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on (Vol. 1, pp. I-I). IEEE. [pdf]
• TEXTURE: Karacan, L., Erdem, E., & Erdem, A. (2013). Structure-preserving image smoothing via region covariances. ACM Transactions on Graphics (TOG), 32(6), 176. [pdf]
• SALIENCY+DEEP: Cornia, M., Baraldi, L., Serra, G., & Cucchiara, R. (2016, December). A deep multi-level network for saliency prediction. In Pattern Recognition (ICPR), 2016 23rd International Conference on (pp. 3488-3493). IEEE. [pdf]
• SALIENCY+DEEP: Kümmerer, M., Wallis, T. S., & Bethge, M. (2016). DeepGaze II: Reading fixations from deep features trained on object recognition. arXiv preprint arXiv:1610.01563. [pdf]