About me

My Name is Nicolas Courty, and I am currently an associate professor at the University of Bretagne Sud since Septembre 2004. My research activities take part in the OBELIX group (IRISA laboratory from University of Bretagne-Sud). My main research objectives are data analysis/synthesis schemes, machine learning and visualization problems, with applications in computer vision, remote sensing and computer graphics.

Keywords: Statistical learning, kernel methods, manifold and geometric approaches, optimal control, optimal transport, remote sensing, hyperspectral image analysis, character and crowd animation

News

Here are some latest actualities:
  • PCV 2014: Our paper 'A group-lasso active set strategy for multiclass hyperspectral image classification' with Devis Tuia and with Rémi Flamary was selected for the best paper award at the PCV conference !
  • ECML 2014: Two of my papers will be presented at ECML, one in a special issue of MACH and the other one in the regular session. See you there !
  • Beginning 2014: I spent two months in EPFL/LASIG to work with Devis Tuia. How nice !
  • August 2013: Out for the GSI (Geometric Science of Information) conference
  • June 2013: New version of the AGORASET dataset ! Check the link
  • April 2013: I obtained my 'Habilitation à diriger des recherches' (HDR). You can find the document here:




Experience

Short Vitae

  • 2013. Habilitation à diriger des recherches (HDR) from University of Bretagne Sud
  • 2012. Invited professor with a Senior Chinese Academy of Science (CAS) fellowship at the Institute of Automation (CASIA), Beijing, China
  • 2004. Assistant professor at the University of Bretagne Sud
  • 2003. INRIA Post doctoral fellowship in Porto Alegre, Brazil, under the direction of Soraia Musse
  • 2002. Ph.D. degree at IRISA, funded by FranceTelecom R&D, under the direction of Bruno Arnaldi.
  • 1999. DEA Imagerie Numérique et Intelligence Artificielle at National University of Rennes, equivalent to a Master Degree in Computer Science, specialized in computer graphics and artificial intelligence,
  • 1999. Engineer diploma at INSAR (Institut National Des Sciences Appliquées de Rennes), Computer Science department, a French engineering school, equivalent to a Master Degree in Engineering.

Students

I am (or was) implied in the advising process of the following phd thesis:
  • Alexis Héloir, who obtained his phd thesis in January 2008, with Sylvie Gibet and Franck Multon,
  • Charly Awad, who obtained his phd thesis in February 2011, with Sylvie Gibet,
  • Pierre Allain, who finished his phd thesis in January 2012, with Thomas Corpetti,
  • Thibaut LeNaour, (started in 2011), with Sylvie Gibet.

Work

This is an overview of more-or-less recent projects (from 2007).

Machine learning

2014 - SAGA: Sparse and Geometry Aware Matrix Factorization.We propose a new non-negative matrix factorization technique which (1) allows the decomposition of the original data on multiple latent factors accounting for the geometrical structure of the manifold embedding the data; (2) provides an optimal representation with a controllable level of sparsity; (3) has an overall linear complexity allowing handling in tractable time large and high dimensional datasets. It operates by coding the data with respect to local neighbors with non-linear weights. This locality is obtained as a consequence of the simultaneous sparsity and convexity constraints.



2014 - Domain adaptation with Optimal transport.We present a new and original method to solve the domain adaptation problem using optimal transport. By searching for the best transportation plan between the probability distribution functions of a source and a target domain, a non-linear and invertible transformation of the learning samples can be estimated. Any standard machine learning method can then be applied on the transformed set, which makes our method very generic. We propose a new optimal transport algorithm that incorporates label information in the optimization: this is achieved by combining an efficient matrix scaling technique together with a majoration of a non-convex regularization term. By using the proposed optimal transport with label regularization, we obtain significant increase in performance compared to the original transport solution. The proposed algorithm is computationally efficient and effective, as illustrated by its evaluation on a toy example and a challenging real life vision dataset, against which it achieves competitive results with respect to state-of-the-art methods.

2013 - Subsampling manifolds.In the Hilbert space reproducing the Gaussian kernel, projected data points are located on an hypersphere. Following some recent works on geodesic analysis on that particular manifold, we propose a method which purpose is to select a subset of input data by sampling the corresponding hypersphere. The selected data should represent correctly the input data, while also maximizing the diversity. We show how these two opposite objectives can be characterized in terms of Karcher variance optimization.

Joint work with Thomas Burger (CNRS, CEA Grenoble)

2012 - Geodesic Analysis over the Gaussian RKHS hypersphere. Using kernels to embed non linear data into high dimensional spaces where linear analysis is possible has become utterly classical. In the case of the Gaussian kernel however, data are distributed on a hypersphere in the corresponding Reproducing Kernel Hilbert Space (RKHS). Inspired by previous works in non-linear statistics, this work investigates the use of dedicated tools to take into account this particular geometry.



2011 - Perturbo, a new classification algorithm based on the spectrum perturbations of the Laplace-Beltrami operator . In our framework, the manifold of each class is characterized by its Laplace-Beltrami operator. A classification criterion is established thanks to a measure of the magnitude of the spectrum perturbation of this operator. The first experiments show good performances against classical algorithms of the state-of-the-art. Moreover, from this measure is derived an efficient policy to design sampling queries in a context of active learning.



Crowd Simulation, Analysis and Control

2007 - Crowd Motion Capture from video. We are trying in this project to design a data-driven animation technique for crowd animation. Our method extracts a continuous flow from a video of a real crowd. This information is then used as input in our animation technique. This work is done in collaboration with Thomas Corpetti

[project page]

2009 - Crowd Motion Analysis from video. Analyzing the crowd dynamics from video sequences is an open challenge in computer vision. Under a high crowd density assumption, we characterize the dynamics of the crowd flow by two related information: velocity and a disturbance potential which accounts for several elements likely to disturb the flow (the density of pedestrians, their interactions with the flow and the environment). The aim of this work to simultaneously estimate from a sequence of crowded images those two quantities. We demonstrate the efficiency of our approach on both synthetic and real crowd videos. P. Allain, T. Corpetti, N. Courty. Crowd Flow Characterization with Optimal Control Theory. In Proc. of the The Ninth Asian Conference on Computer Vision (ACCV 2009), LNCS, Xi'an, China, Septembre 2009.

2013 - Particle and Crowd Control. Controlling several and possibly independent moving agents in order to reach global goals is a tedious task that has applications in many engineering fields such as robotics or computer animation. Together, the different agents form a whole called swarm, which may display interesting collective behaviors. When the agents are driven by their own dynamics, controlling this swarm is known as the particle swarm control problem. In that context, several strategies, based on the control of individuals using simple rules, exist. This work defends a new and original method based on a centralized approach. More precisely, we propose a framework to control several particles with constraints either expressed on a per-particle basis, or expressed as a function of their environment.

P. Allain, N. Courty, T. Corpetti. Particle Swarm Control. In Optimal Control and Applications, To appear.

2013 - AGORASET We showcase a simulation-based crowd video dataset to be used for evaluation of low-level video crowd analysis methods, such as tracking or segmentation. Most of the time, an exact ground truth associated to real videos is difficult and time-consuming to produce, prone to errors, and these difficulties rise exponentially with the apparent density of the crowd in the image. We propose a synthetic crowd dataset to help researchers evaluate their methods against an objective and temporally dense synthetic ground truth. This dataset, named AGORASET, can be found HERE


2013 - Ground Truth For Pedestrian Analysis This work investigates the use of synthetic 3D scenes to generate ground truth of pedestrian segmentation in 2D crowd video data. Manual segmentation of objects in videos is indeed one of the most time-consuming type of assisted labeling. A big gap in computer vision research can not be filled due to this lack of temporally dense and precise segmentation ground truth on large video samples. Such data is indeed essential to introduce machine learning techniques for automatic pedestrian segmentation, as well as many other application involving occluded people. We present a new dataset of 1.8 millions pedestrian silhouettes presenting human-to-human occlusion patterns likely to be seen in real crowd video data. To our knowledge, it is the first publicly available large dataset of pedestrian in crowd silhouettes.

Published at CVPR Workshop on ground truth. Joint work with Clément Creusot, Toshiba. More information on his webpage.

Remote sensing Imagery

2012 - Classification of hyperspectral images with Mathematical Morphology. We present a new method for the spectral-spatial classification of hyperspectral images, by means of morphological features and manifold learning. In particular, mathematical morphology has proved to be an invaluable tool for the description of remote sensing images. However, its application to hyperspectral data is problematic, due to the absence of a complete lattice structure at higher dimensions. We address this issue by following up previous experimental indications on the interest of classwise orderings.

Joint work with Sébastien Lefèvre and Erhan Aptoula

2013 - Monitoring urban transformation in the old foreign concessions of Shanghai from 1987 to 2012. This paper is concerned with morphological change analysis in the old foreign concessions of Shanghai from 1969 to 2010. To that end, we use a series of 17 Landsat TM and Landsat ETM + images on which we estimate some feature parameters. The analysis of the resulting time series enables to isolate changes from traditional constructions to new buildings or gardens. Our results show that 70 % of the old urban pattern was converted in modern highrise buildings and green spaces.

Joint work with Antoine Lefebvre and Thomas Corpetti



Human Character Animation

2011 - Virtual Signer Together with Sylvie Gibet, we have worked on a virtual signer capable of signing. A full evaluation and description of our virtual signer has been published in ACM Transactions on Interactive Intelligent Systems. Please find the download link here.






2007 - Sequential Monte Carlo methods in computer animation. In this project we try to design bayesian filter to control human figure in a realistic and plausible way. Through this method, we try to provide accurate and flexible means of generating human gestures under kinematics and physical constraints. This work is a result of a collaboration with Elise Arnaud from LJK in Grenoble, France.

This work has been awarded Best Paper at the AMDO 2008 conference

  • N. Courty, E. Arnaud. Inverse Kinematics using Sequential Monte Carlo Methods. In International Conference on Articulated Motion and Deformable Objects (AMDO 2008), (Best Paper Award), LNCS, Volume 5098, p. 1-10, Mallorca, Spain, Juillet 2008.

Download the technical report.

2010 - Conditional Stochastic Simulation. In a context of interactive applications, adapting motion capture data to new situations or producing variants of them are known as non trivial tasks. We propose an original method that produces motions that preserve the statistical properties of a reference motion while ensuring some constraints. This method uses principles of conditional stochastic simulation to achieve this goal. Notably, a new real time algorithm, performing sequentially and producing the desired motion is introduced.

This work has been done in collaboration with Anne Cuzol (LMBA, UBS).

2013 - Spatio-temporal coupling with the 3D+t motion Laplacian. Motion editing requires the preservation of certain geometric details and also temporal information as contained in the accelerations and decelerations. During editing, this informationshould be preserved at best. We propose a new representation of the motion based on the Laplacian expression of a super-graph: the set of graph given by the skeleton over time, these graphs being otherwise linked from one to another. Moreover, the lengths of the skeleton segments being invariants during the editing, we add this constraint into our algorithm. Through this Laplacian representation of the motion, we propose an application which allows an easy and interactive editing, correction or retargeting of the new movement.

This work is part of Thibaut LeNaour phd thesis

Signal Processing on Rotational Data

2007 - Human Motion Data Analysis for data on Riemmannian manifolds. In this project we have tried to find out an exact algorithm to compute the Principal Geodesic Analysis (PGA) of data on SO(3). The Principal Geodesics Analysis is an extension of PCA which was first designed by T. Fletcher at Chapel Hill. This work was performed in collaboration with Salem Said and Nicolas Le Bihan from GIPSA Lab. [project page]


2008 - Bilateral Human Motion Filtering: new method to process motion data that tends to preserve some characteristic features of human motions. It is based on an adaptation of the well-known bilateral filter to orientation data.
  • N. Courty. Bilateral Human Motion Filtering. In Proc. of the 16th European Signal Processing Conference (EUSIPCO 2008), Lausanne, Switzerland, Août 2008.

2009 - Human Motion Compression with PGA. In collaboration with Maxime Tournier, XiaoMao, Lionel Reveret and Elise Arnaud, we developped a human motion compression scheme based on principal geodesic analysis and a special inverse kinematics algorithm.
This work has been published at Eurographics 2009 and has won the third best paper prize.

Publications

List of publications over the last previous 4 years. The complete list can be found on HAL or my google scholar webpage.
2015
[33] Multiclass feature learning for hyperspectral image classification: sparse and hierarchical solutions (Devis Tuia, Rémi Flamary, Nicolas Courty), In ISPRS Journal of Photogrammetry and Remote Sensing, Elsevier, 2015. [bibtex] [pdf]
2014
[32]Optimal transport for Domain adaptation (Nicolas Courty, Remi Flamary, Alain Rakotomamonjy, Devis Tuia), In NIPS, Workshop on Optimal Transport and Machine Learning, 2014. [bibtex]
[31]Optimal transport with Laplacian regularization (Remi Flamary, Nicolas Courty, Alain Rakotomamonjy, Devis Tuia), In NIPS, Workshop on Optimal Transport and Machine Learning, 2014. [bibtex]
[30] Domain adaptation with regularized optimal transport (N. Courty, R. Flamary, D. Tuia), In Proceedings of ECML/PKDD 2014, 2014. [bibtex] [pdf]
[29] SAGA: Sparse And Geometry-Aware non-negative matrix factorization through non-linear local embedding (N. Courty, X. Gong, J. Vandel, T. Burger), In Machine Learning, 2014. [bibtex] [pdf]
[28] Network-based correlated correspondence for unsupervised domain adaptation of hyperspectral satellite images (J. Rebetez, D. Tuia, N. Courty), In Proceedings of ICPR2014, 2014. [bibtex] [pdf]
[27]A group-lasso active set strategy for multiclass hyperspectral image classification (D. Tuia, N. Courty, R. Flamary), In Photogrammetric Computer Vision (PCV), 2014. [bibtex]
[26] An end-member based ordering relation for the morphological description of hyperspectral images (E. Aptoula, N. Courty, S. Lefèvre), In Proceedings of the IEEE International Conference on Image Processing (ICIP), 2014. [bibtex] [pdf]
[25]Using the AGORASET dataset : assessing for the quality of crowd video analysis methods (N. Courty, P. Allain, C. Creusot, T. Corpetti), In Pattern Recognition Letters, to appear, 2014. [bibtex]
[24]PerTurbo manifold learning algorithm for weakly labelled hyperspectral image classification (Thomas Burger Nicolas Courty Sébastien Lefèvre Laetitia Chapel), In IEEE JSTARS, special issue on machine learning, 2014. [bibtex]
2013
[23] Spatio-temporal coupling with the 3D+t motion Laplacian (Thibaut Le Naour, Nicolas Courty, Sylvie Gibet), In Computer Animation and Virtual Worlds, 2013. [bibtex] [pdf]
[22] Ground Truth for Pedestrian Analysis and Application to Camera Calibration (Clément Creusot, Nicolas Courty), In Proceedings of the Ground Truth Workshop at the Computer Vision and Pattern Recognition Conference (CVPR) 2013, 2013. [bibtex] [pdf]
[21] A kernel view on manifold sub-sampling based on Karcher variance optimization (Nicolas Courty, Thomas Burger), In Geometric sciences of information - LNCS 8085, 2013. [bibtex] [pdf]
[20] Monitoring urban transformation in the old foreign concessions of Shanghai from 1987 to 2012 (Lefebvre Antoine, Nicolas Courty, Thomas Corpetti), In International Symposium on Remote Sensing for Environment - Earth observation and Global Environmental Change, 2013. [bibtex] [pdf]
[19]Unsupervised Dense Crowd Detection by Multiscale Texture Analysis (Antoine Fagette, Nicolas Courty, Daniel Racoceanu, Jean-Yves Dufour), In Pattern Recognition Letters, special issue on Pattern Recognition and Crowd Analysis, to appear, 2013. [bibtex]
[18]Particle Swarm Control (Pierre Allain, Nicolas Courty, Thomas Corpetti), In Optimal Control, Applications and Methods, to appear, 2013. [bibtex]
[17]Optimal Crowd Editing (P. Allain, N. Courty, T. Corpetti), In Graphical Models, GMOD, to appear, 2013. [bibtex]
2012
[16] Fast Motion retrieval with the distance input space (Thibaut Le Naour, Nicolas Courty, Sylvie Gibet), In Motion in Games - 5th International Conference, MIG 2012, (Marcelo Kallmann, Kostas E. Bekris, eds.), volume 7660, 2012. [bibtex] [pdf]
[15] Cinématique guidée par les distances (Thibaut Le Naour, Nicolas Courty, Sylvie Gibet), In Revue Électronique Francophone d'Informatique Graphique, volume Vol. 6, 2012. [bibtex] [pdf]
[14] A classwise supervised ordering approach for morphology based hyperspectral image classification (Nicolas Courty, Erchan Aptoula, Sébastien Lefèvre), In Proceedings of the 21st ICPR, 2012. [bibtex] [pdf]
[13] Geodesic Analysis on the Gaussian RKHS hypersphere (Nicolas Courty, Thomas Burger, Pierre-François Marteau), In proceedings of ECML-PKDD 2012, 2012. [bibtex] [pdf]
[12] CLASSWISE HYPERSPECTRAL IMAGE CLASSIFICATION WITH PERTURBO METHOD (Laetitia Chapel, Thomas Burger, Nicolas Courty, Sébastien Lefèvre), In IGARSS 2012, 2012. [bibtex] [pdf]
2011
[11] Continuous Control of Lagrangian Data (Pierre Allain, Nicolas Courty, Thomas Corpetti), In Informatics in Control, Automation and Robotics, Springer, volume 132, 2011. [bibtex] [pdf]
[10] The SignCom System for Data-Driven Animation of Interactive Virtual Signers : Methodology and Evaluation (Sylvie Gibet, Nicolas Courty, Kyle Duarte, Thibaut Le Naour), In ACM Transaction on Interactive Intelligent Systems, volume 1, 2011. [bibtex] [pdf]
[9] Perturbations of the Laplace-Beltrami Operator (Nicolas Courty, Thomas Burger, Johann Laurent), In ECML-PKDD, 2011. [bibtex] [pdf]
[8] PerTurbo: a new classification algorithm based on the spectrum perturbations of the Laplace-Beltrami operator (Nicolas Courty, Thomas Burger, Laurent D. Johann), In ECML-PKDD 2011, 2011. [bibtex] [pdf]
2010
[7] Why is the Creation of a Virtual Signer Challenging Computer Animation ? (Nicolas Courty, Sylvie Gibet), In Motion in Games 2010, 2010. [bibtex] [pdf]
[6] Conditional Stochastic Simulation for Character Animation (Nicolas Courty, Anne Cuzol), In Computer Animation and Virtual Worlds (best selected papers from CASA 2010), 2010. [bibtex] [pdf]
[5] Conditional stochastic simulation for character animation (Nicolas Courty, Anne Cuzol), In COMPUTER ANIMATION AND VIRTUAL WORLDS, 2010. [bibtex] [pdf]
2009
[4] Motion Compression using Principal Geodesics Analysis (Maxime Tournier, Xiaomao Wu, Nicolas Courty, Elise Arnaud, Lionel Reveret), In Computer Graphics Forum (Proceedings of Eurographics 2009), EG, 2009. [bibtex] [pdf]
[3] A Combined Semantic and Motion Capture Database for Real-Time Sign Language Synthesis (Charly Awad, Nicolas Courty, Kyle Duarte, Thibaut Le Naour, Sylvie Gibet), In Proc. of the 9th Int. Conference on Intelligent Virtual Agent (IVA 2009), 2009. [bibtex] [pdf]
[2] A Database Architecture For Real-Time Motion Retrieval (Charly Awad, Nicolas Courty, Sylvie Gibet), In Proc. of the 7th International Workshop on Content-Based Multimedia Indexing (CBMI 2009) (IEEE CS, ed.), 2009. [bibtex] [pdf]
[1] Crowd Flow Characterization with Optimal Control Theory (Pierre Allain, Nicolas Courty, Thomas Corpetti), In Ninth Asian Conference on Computer Vision (ACCV 2009), 2009. [bibtex] [pdf]

Contact

Address

Laboratoire IRISA
Campus de Tohannic
56000 Vannes, France
Phone : (+33)2 97 01 72 13
Fax : (+33)2 97 01 72 79

In order to reach me

Here is my mail : ncourty _at_ irisa.fr