This is an overview of more-or-less recent projects (from 2007).

## Machine learning

**2014 - SAGA: Sparse and Geometry Aware Matrix Factorization**.We propose a new non-negative matrix factorization technique which (1) allows the decomposition of the original data on multiple latent factors accounting for the geometrical structure of the manifold embedding the data; (2) provides an optimal representation with a controllable level of sparsity; (3) has an overall linear complexity allowing handling in tractable time large and high dimensional datasets. It operates by coding the data with respect to local neighbors with non-linear weights. This locality is obtained as a consequence of the simultaneous sparsity and convexity constraints.

**2014 - Domain adaptation with Optimal transport**.We present a new and original method to solve the domain adaptation problem using optimal transport. By searching for the best transportation plan between the probability distribution functions of a source and a target domain, a non-linear and invertible transformation of the learning samples can be estimated. Any standard machine learning method can then be applied on the transformed set, which makes our method very generic. We propose a new optimal transport algorithm that incorporates label information in the optimization: this is achieved by combining an efficient matrix scaling technique together with a majoration of a non-convex regularization term. By using the proposed optimal transport with label regularization, we obtain significant increase in performance compared to the original transport solution. The proposed algorithm is computationally efficient and effective, as illustrated by its evaluation on a toy example and a challenging real life vision dataset, against which it achieves competitive results with respect to state-of-the-art methods.

**2013 - Subsampling manifolds**.In the Hilbert space reproducing the Gaussian kernel, projected data points are located on an hypersphere. Following some recent works
on geodesic analysis on that particular manifold, we propose a method which purpose is to select a subset of input data
by sampling the corresponding hypersphere. The selected data should represent correctly the input data, while also maximizing the
diversity. We show how these two opposite objectives can be characterized in terms of Karcher variance optimization.

Joint work with Thomas Burger (CNRS, CEA Grenoble)

**2012 - Geodesic Analysis over the Gaussian RKHS hypersphere**. Using kernels to embed
non linear data into high dimensional spaces where linear analysis is possible has become utterly classical.
In the case of the Gaussian kernel however, data are distributed on a hypersphere in the corresponding Reproducing Kernel Hilbert Space (RKHS). Inspired by previous works in non-linear statistics, this work investigates the use of dedicated tools to take into account this particular geometry.

**2011 - Perturbo, a new classification algorithm based on the spectrum perturbations of the Laplace-Beltrami operator **. In our framework, the manifold of each class is characterized by its Laplace-Beltrami operator. A classification criterion is established thanks to a measure of the magnitude of the spectrum perturbation of this operator. The first experiments show good performances against classical algorithms of the state-of-the-art. Moreover, from this measure is derived an efficient policy to design sampling queries in a context of active learning.

## Crowd Simulation, Analysis and Control

**2007 - **Crowd Motion Capture from video. We are trying in this project to design a data-driven animation technique for crowd animation. Our method extracts a continuous
flow from a video of a real crowd. This information is then used as input in our animation technique. This work is done in collaboration
with Thomas Corpetti

[project page]
**2009 - Crowd Motion Analysis from video**. Analyzing the crowd dynamics from video sequences is an open challenge
in computer vision. Under a high crowd density assumption, we characterize
the dynamics of the crowd flow by two related information: velocity and a disturbance
potential which accounts for several elements likely to disturb the flow
(the density of pedestrians, their interactions with the flow and the environment).
The aim of this work to simultaneously estimate from a sequence of crowded
images those two quantities. We demonstrate the efficiency of our approach on both synthetic and
real crowd videos.
P. Allain, T. Corpetti, N. Courty. Crowd Flow Characterization with Optimal Control Theory. In Proc. of the The Ninth Asian Conference on Computer Vision (ACCV 2009), LNCS, Xi'an, China, Septembre 2009.

**2013 - Particle and Crowd Control**. Controlling several and possibly independent moving agents in order to reach global goals is a tedious
task that has applications in many engineering fields such as robotics or computer animation. Together, the
different agents form a whole called swarm, which may display interesting collective behaviors. When the
agents are driven by their own dynamics, controlling this swarm is known as the particle swarm control
problem. In that context, several strategies, based on the control of individuals using simple rules, exist. This
work defends a new and original method based on a centralized approach. More precisely, we propose a
framework to control several particles with constraints either expressed on a per-particle basis, or expressed
as a function of their environment.

P. Allain, N. Courty, T. Corpetti. Particle Swarm Control. In Optimal Control and Applications, To appear.

**2013 - AGORASET** We showcase a simulation-based crowd video dataset to be used for evaluation of
low-level video crowd analysis methods, such as tracking or segmentation. Most of the time, an
exact ground truth associated to real videos is difficult and time-consuming to produce, prone
to errors, and these difficulties rise exponentially with the apparent density of the crowd in the
image. We propose a synthetic crowd dataset to help researchers evaluate their methods against
an objective and temporally dense synthetic ground truth. This dataset, named AGORASET, can be found HERE

**2013 - Ground Truth For Pedestrian Analysis** This work investigates the use of synthetic 3D scenes to generate ground truth of pedestrian segmentation in 2D crowd video data. Manual segmentation of objects in videos is indeed one of the most time-consuming type of assisted labeling. A big gap in computer vision research can not be filled due to this lack of temporally dense and precise segmentation ground truth on large video samples. Such data is indeed essential to introduce machine learning techniques for automatic pedestrian segmentation, as well as many other application involving occluded people. We present a new dataset of 1.8 millions pedestrian silhouettes presenting human-to-human occlusion patterns likely to be seen in real crowd video data. To our knowledge, it is the first publicly available large dataset of pedestrian in crowd silhouettes.

Published at CVPR Workshop on ground truth. Joint work with Clément Creusot, Toshiba. More information on his webpage.

## Remote sensing Imagery

**2012 - Classification of hyperspectral images with Mathematical Morphology**. We present a new method for the spectral-spatial classification of hyperspectral images, by means of
morphological features and manifold learning. In particular, mathematical morphology has proved to be an invaluable tool for the description of remote sensing images.
However, its application to hyperspectral data is problematic, due to the absence of a complete lattice structure at higher dimensions. We address this issue
by following up previous experimental indications on the interest of classwise orderings.

Joint work with Sébastien Lefèvre and Erhan Aptoula

**2013 - Monitoring urban transformation in the old foreign concessions of Shanghai from 1987 to 2012**. This paper is concerned with morphological change analysis in the old foreign
concessions of Shanghai from 1969 to 2010. To that end, we use a series of 17 Landsat TM
and Landsat ETM + images on which we estimate some feature parameters. The analysis of the
resulting time series enables to isolate changes from traditional constructions to new buildings
or gardens. Our results show that 70 % of the old urban pattern was converted in modern highrise
buildings and green spaces.

Joint work with Antoine Lefebvre and Thomas Corpetti

## Human Character Animation

**2011 - Virtual Signer** Together with Sylvie Gibet, we have worked on a virtual signer capable of signing. A full evaluation and description of our virtual signer has been published in ACM Transactions on Interactive Intelligent Systems. Please find the download link here.

**2007 - **Sequential Monte Carlo methods in computer animation. In this project we try to design bayesian filter to control human figure
in a realistic and plausible way. Through this method, we try to provide accurate and flexible means of generating
human gestures under kinematics and physical constraints. This work is a result of a collaboration with Elise Arnaud from LJK in Grenoble, France.

This work has been awarded Best Paper at the AMDO 2008 conference

- N. Courty, E. Arnaud. Inverse Kinematics using Sequential Monte Carlo Methods. In International Conference on Articulated Motion and Deformable Objects (AMDO 2008), (Best Paper Award), LNCS, Volume 5098, p. 1-10, Mallorca, Spain, Juillet 2008.

Download the technical report.

**2010 - Conditional Stochastic Simulation**. In a context of interactive applications, adapting motion capture data to new situations or producing variants of them are known as non trivial tasks. We propose an original method
that produces motions that preserve the statistical properties of a reference motion while
ensuring some constraints. This method uses principles of conditional stochastic simulation
to achieve this goal. Notably, a new real time algorithm, performing sequentially and
producing the desired motion is introduced.

This work has been done in collaboration with Anne Cuzol (LMBA, UBS).

**2013 - Spatio-temporal coupling with the 3D+t motion Laplacian**. Motion editing requires the preservation of certain geometric details and also temporal
information as contained in the accelerations and decelerations. During editing, this informationshould be preserved at best. We propose a new representation of the motion based on
the Laplacian expression of a super-graph: the set of graph given by the skeleton over time, these graphs being otherwise linked from one to another. Moreover, the lengths
of the skeleton segments being invariants during the editing, we add this constraint into our algorithm. Through this Laplacian representation of the motion, we propose an
application which allows an easy and interactive editing, correction or retargeting of the new movement.

This work is part of Thibaut LeNaour phd thesis

## Signal Processing on Rotational Data

**2007 - **Human Motion Data Analysis for data on Riemmannian manifolds. In this project we have tried to find out an exact
algorithm to compute the Principal Geodesic Analysis (PGA) of data on SO(3). The Principal Geodesics Analysis is an extension
of PCA which was first designed by T. Fletcher at Chapel Hill. This work was performed in collaboration with Salem Said and Nicolas Le Bihan from GIPSA Lab. [project page]

**2008 - **Bilateral Human Motion Filtering: new method to process motion data that tends to preserve some characteristic features of human motions. It
is based on an adaptation of the well-known bilateral filter to orientation data.

- N. Courty. Bilateral Human Motion Filtering. In Proc. of the 16th European Signal Processing Conference (EUSIPCO 2008), Lausanne, Switzerland, Août 2008.

**2009 - **Human Motion Compression with PGA. In collaboration with Maxime Tournier, XiaoMao, Lionel Reveret and Elise Arnaud, we developped a human motion compression scheme based on principal geodesic analysis and a special inverse kinematics algorithm.

This work has been published at Eurographics 2009 and has won the third **best paper prize**.