ENCOV

Endoscopy and Computer Vision

EnCoV is a research group located at the Faculty of Medicine in Clermont-Ferrand. It is part of the Image-Guided Therapies research axis of Institut Pascal, a joint venture between CNRS, Université Clermont Auvergne and SIGMA.

Back to index

Project FLEXABLE (ERC Starting Consolidator Grant, 2013-2018)

Project FLEXABLE is a Seventh Framework Programme funded by the European Research Council (ERC). The project is conducted by the EnCoV group.

Contact: flexable@udamail.fr

Description

Project FLEXABLE lies in the scientific field of Computer Vision which studies the computational interpretation of images. In particular, 3D Computer Vision seeks to recover depth or the 3D shape of the observed environment from images. One of the most successful and mature techniques in 3D Computer Vision is Shape-from-Motion which is based on the well-established theory of Multiple-View Geometry. This uses multiple images and assumes that the observed environment is rigid.

The world is however made of objects which move and undergo deformations rather than remaining rigid. Researchers have tried to extend Shape-from-Motion to a deformable environment for about a decade, yet with only very limited success to date. We believe that there are two main reasons for this. Firstly there is still a lack of a solid underlying theory for Deformable Shape-from-Motion. Fundamental questions, such as what kinds of deformation can facilitate unambiguous 3D reconstruction, are not yet answered. Secondly practical solutions have not yet come about because the Motion cue has been mostly used on its own. For accurate and dense 3D shape results, it must be combined with other visual cues, since it is certainly weaker in the deformable than in the rigid case. Furthermore, strong priors are needed, requiring one to bridge the gap with object recognition.

This project focuses on developing these two key areas and in doing so will include three main lines of research: theory, its computational implementation, and its real-world application. Deformable Multiple-View Geometry will generalize the existing rigid theory to the deformable case and will provide researchers with a rigorous mathematical framework that underpins the use of Motion as a proper visual cue for Deformable 3D Reconstruction. Our theory will require us to introduce new mathematical tools from differentiable projective manifolds to Computer Vision. Our implementation will study and develop new computational means for solving the difficult inference problems formulated in our theory. Because we strongly believe that Motion is not sufficient for solving the general Deformable 3D Reconstruction problem alone, it will be combined with other visual cues including Shading. Finally, we will develop cutting-edge applications of our framework specific to Minimally Invasive Surgery, with a focus in interventional laparoscopy, for which there is a very high need for 3D Computer Vision.

Members

Collaborators

Alumni

Postdocs

PhD students

Interns

Collaborators

Publications by year

Local Deformable 3D Reconstruction with Cartan’s Connections
S. Parashar, D. Pizarro and A. Bartoli
IEEE Transactions on Pattern Analysis and Machine Intelligence, accepted May 2019
[paper]
A Differential-Algebraic Projective Framework for the Deformable Single-View Geometry of the 1D Perspective Camera
A. Bartoli
Journal of Mathematical Imaging and Vision, accepted April 2019
[paper]
The Highlight Ovals
A. Bartoli
Journal of Mathematical Imaging and Vision, accepted February 2019
[paper]
Equiareal Shape-from-Template
D. Casillas-Perez, D. Pizarro, D. Fuentes-Jimenez, M. Mazo and A. Bartoli
Journal of Mathematical Imaging and Vision, accepted October 2018
[paper]
3DVFX: 3D Video Editing using Non-Rigid Structure-from-Motion
S. Parashar and A. Bartoli
EG - Eurographics, 2019
[paper]
[video]
Isometric Non-Rigid Shape-from-Motion with Riemannian Geometry Solved in Linear Time
S. Parashar, D. Pizarro and A. Bartoli
IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(10):2442-2454, October 2018
[paper]
Inextensible Non-Rigid Structure-from-Motion by Second-Order Cone Programming
A. Chhatkuli, D. Pizarro, T. Collins and A. Bartoli
IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(10):2428-2441, October 2018
[paper]
Plane-Based Resection for Metric Affine Cameras
A. Bartoli and T. Collins
Journal of Mathematical Imaging and Vision, 60(7):1037-1064, September 2018
[paper]
Contributions to Monocular Deformable 3D Reconstruction: Curvilinear Objects and Multiple Visual Cues
Mathias Gallardo
PhD thesis (thèse de doctorat), Université Clermont Auvergne, September 2018 [supervision: T. Collins and A. Bartoli (director)] [document in English]
[thesis]
Image-Based Models for Specularity Propagation in Diminished Reality
S. Hadj Said, M. Tamaazousti and A. Bartoli
IEEE Transactions on Visualization and Computer Graphics, 24(7):2140-2152, July 2018
[paper]
Model-Based Active Learning to Detect an Isometric Deformable Object in the Wild with a Deep Architecture
S. Sankar and A. Bartoli
Computer Vision and Image Understanding, 171:69-82, June 2018
[paper]
A Geometric Model for Specularity Prediction on Planar Surfaces with Multiple Light Sources
A. Morgand, M. Tamaazousti and A. Bartoli
IEEE Transactions on Visualization and Computer Graphics, 24(5):1691-1704, May 2018
[paper]
Augmented Reality in a Tumor Resection Model
P. Chauvet, T. Collins, C. Debize, L. Novais-Gameiro, B. Pereira, A. Bartoli, M. Canis and N. Bourdel
Surgical Endoscopy, 32(3):1192-1201, March 2018
- Junior researcher prize from Clermont-Ferrand's university hospital (prix jeune chercheur du CHU de Clermont-Ferrand), 2017
- Computer-aided surgery innovation prize from the National Academy of Surgery (prix de l’innovation médico-chirurgicale assistée par ordinateur du Labex CAMI décerné par l'Académie nationale de Chirurgie), 2017
[paper]
A 3D Deformable Model-Based Framework for the Retrieval of Near-Isometric Flattenable Objects using Bag-of-Visual-Words
R. Rantoson and A. Bartoli
Computer Vision and Image Understanding, 167:89-108, February 2018
[paper]
Fast Shape-from-Template Using Local Features
M. Famouri, A. Bartoli and Z. Azimifar
Machine Vision and Applications, 29(1):73-93, January 2018
[paper]
Self-Calibrating Isometric Non-Rigid Structure-from-Motion
S. Parashar, A. Bartoli and D. Pizarro
ECCV - European Conference on Computer Vision, 2018
[paper]
Rolling Shutter Pose and Ego-motion Estimation using Shape-from-Template
Y. Lao, O. Ait-Aider and A. Bartoli
ECCV - European Conference on Computer Vision, 2018
[paper]
A Multiple-View Geometric Model of Specularities on Non-Planar Shapes with Application to Dynamic Retexturing
A. Morgand, M. Tamaazousti and A. Bartoli
IEEE Transactions on Visualization and Computer Graphics, special issue: selected papers from ISMAR 2017, 23(11):2485-2493, November 2017
[paper]
Image-based Deformable 3D Reconstruction using Differential Geometry and Cartan's Connections
Shaifali Parashar
PhD thesis (thèse de doctorat), Université Clermont Auvergne, November 2017 [supervision: D. Pizarro and A. Bartoli (director)] [document in English]
Université Clermont Auvergne PhD prize (prix de thèse de la fondation de l'Université Clermont Auvergne), 2018
[thesis]
Particle-SfT: a Provably-Convergent, Fast Shape-from-Template Algorithm
E. Özgür and A. Bartoli
International Journal of Computer Vision, 123(2):184-205, June 2017
[paper]
Planar Structure-from-Motion with Affine Camera Models: Closed-Form Solutions, Ambiguities and Degeneracy Analysis
T. Collins and A. Bartoli
IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6):1237-1255, June 2017
[paper]
A Stable Analytical Framework for Isometric Shape-from-Template by Surface Integration
A. Chhatkuli, D. Pizarro, A. Bartoli and T. Collins
IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(5):833-850, May 2017
[paper]
Generalizing the Prediction Sum of Squares Statistic and Formula, Application to Linear Fractional Image Warp and Surface Fitting
A. Bartoli
International Journal of Computer Vision, 122(1):61-83, March 2017
[paper]
EasyFlow: Increasing the Convergence Basin of Variational Image Matching with a Feature-Based Cost
J. Braux-Zin, R. Dupont, A. Bartoli and M. Tamaazousti
IET Computer Vision, 11(2):122-134, March 2017
[paper]
A Multiple-View Geometric Model of Specularities on Non-Uniformly Curved Surfaces
A. Morgand, M. Tamaazousti and A. Bartoli
VRST - ACM Symposium on Virtual Reality Software and Technology, 2017
[paper]
Dense Non-Rigid Structure-from-Motion and Shading with Unknown Albedos
M. Gallardo, T. Collins and A. Bartoli
ICCV - IEEE International Conference on Computer Vision, 2017
[paper]
Visualizing In-Organ Tumors in Augmented Monocular Laparoscopy
E. Özgür, A. Lafont and A. Bartoli
ISMAR - IEEE International Symposium on Mixed and Augmented Reality, 2017
[paper]
[video]
A Multiple-View Geometric Model of Specularities on Non-Planar Shapes with Application to Dynamic Retexturing
A. Morgand, M. Tamaazousti and A. Bartoli
ISMAR - IEEE International Symposium on Mixed and Augmented Reality, 2017
[paper (TVCG17)]
Local Analytic and Global Convex Methods for the 3D Reconstruction of Isometric Deformable Surfaces
Ajad Chhatkuli
PhD thesis (thèse de doctorat), Université d'Auvergne, December 2016 [supervision: D. Pizarro and A. Bartoli (director)] [document in English]
[thesis]
Schwarps: Locally Projective Image Warps Based on 2D Schwarzian Derivatives
D. Pizarro, R. Khan and A. Bartoli
International Journal of Computer Vision, 119(2):93-109, September 2016
[paper]
3D Conformation of a Flexible Fiber in a Turbulent Flow
G. Verhille and A. Bartoli
Experiments in Fluids, 57(7):117-127, July 2016
[paper]
Using Shading and a 3D Template to Reconstruct Complex Surface Deformations
M. Gallardo, T. Collins and A. Bartoli
BMVC - British Machine Vision Conference, 2016
[paper]
Can we Jointly Register and Reconstruct Creased Surfaces by Shape-from-Template Accurately?
M. Gallardo, T. Collins and A. Bartoli
ECCV - European Conference on Computer Vision, 2016
[paper]
A Perspective on Non-Isometric Shape-from-Template
A. Bartoli and E. Özgür
ISMAR - IEEE International Symposium on Mixed and Augmented Reality, 2016
[paper]
An Empirical Model for Specularity Prediction with Application to Dynamic Retexturing
A. Morgand, M. Tamaazousti and A. Bartoli
ISMAR - IEEE International Symposium on Mixed and Augmented Reality, 2016
[paper]
Isometric Non-Rigid Shape-from-Motion in Linear Time
S. Parashar, D. Pizarro and A. Bartoli
CVPR - IEEE International Conference on Computer Vision and Pattern Recognition, 2016
[paper]
Inextensible Non-Rigid Shape-from-Motion by Second Order Cone Programming
A. Chhatkuli, D. Pizarro, T. Collins and A. Bartoli
CVPR - IEEE International Conference on Computer Vision and Pattern Recognition, 2016
[paper]
Shape-from-Template
A. Bartoli, Y. Gérard, F. Chadebecq, T. Collins and D. Pizarro
IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(10):2099-2118, October 2015
Université d'Auvergne research prize (prix recherche de l'Université d'Auvergne), 2016
[paper]
Metric Corrections of the Affine Camera
A. Bartoli, T. Collins and D. Pizarro
Computer Vision and Image Understanding, 135:141-156, June 2015
[paper]
As-Rigid-As-Possible Volumetric Shape-from-Template
S. Parashar, D. Pizarro, A. Bartoli and T. Collins
ICCV - IEEE International Conference on Computer Vision, 2015
[paper]
Shape-from-Template in Flatland
M. Gallardo, D. Pizarro, A. Bartoli and T. Collins
CVPR - IEEE International Conference on Computer Vision and Pattern Recognition, 2015
[paper]
A Linear Least-Squares Solution to Elastic Shape-from-Template
A. Malti, A. Bartoli and R. Hartley
CVPR - IEEE International Conference on Computer Vision and Pattern Recognition, 2015
[paper]
Infinitesimal Plane-Based Pose Estimation
T. Collins and A. Bartoli
International Journal of Computer Vision, 109(3):252-286, September 2014
[paper]
Monocular Template-Based 3D Surface Reconstruction: Convex Inextensible and Nonconvex Isometric Methods
F. Brunet, A. Bartoli and R. Hartley
Computer Vision and Image Understanding, 125:138-154, August 2014
[paper]
Combining Conformal Deformation and Cook-Torrance Shading for 3D Reconstruction in Laparoscopy
A. Malti and A. Bartoli
IEEE Transactions on Biomedical Engineering, 61(6):1684-1692, June 2014
[paper]
Non-Rigid Shape-from-Motion for Isometric Surfaces using Infinitesimal Planarity
A. Chhatkuli, D. Pizarro and A. Bartoli
BMVC - British Machine Vision Conference, 2014
[paper]
Schwarps: Locally Projective Image Warps Based on 2D Schwarzian Derivatives
R. Khan, D. Pizarro and A. Bartoli
ECCV - European Conference on Computer Vision, 2014
[paper (IJCV16)]
Using Isometry to Classify Correct/Incorrect 3D-2D Correspondences
T. Collins and A. Bartoli
ECCV - European Conference on Computer Vision, 2014
[paper]
An Analysis of Errors in Graph-based Keypoint Matching and Proposed Solutions
T. Collins, P. Mesejo and A. Bartoli
ECCV - European Conference on Computer Vision, 2014
[paper]
Stable Template-Based Isometric 3D Reconstruction in All Imaging Conditions by Linear Least-Squares
A. Chhatkuli, D. Pizarro and A. Bartoli
CVPR - IEEE International Conference on Computer Vision and Pattern Recognition, 2014
[paper (PAMI16)]
Live Image Parsing in Uterine Laparoscopy
A. Chhatkuli, A. Bartoli, A. Malti and T. Collins
ISBI - IEEE International Symposium on Biomedical Imaging, 2014
[paper]
A Robust Analytical Solution to Isometric Shape-from-Template with Focal Length Calibration
A. Bartoli, D. Pizarro and T. Collins
ICCV - IEEE International Conference on Computer Vision, 2013
[paper]
A General Dense Image Matching Framework Combining Direct and Feature-based Costs
J. Braux-Zin, R. Dupont and A. Bartoli
ICCV - IEEE International Conference on Computer Vision, 2013
[paper (IETCV16)]
Combining Features and Intensity for Wide-Baseline Non-Rigid Surface Registration
J. Braux-Zin, R. Dupont and A. Bartoli
BMVC - British Machine Vision Conference, 2013
[paper (IETCV16)]
Isowarp and Conwarp: Warps that Exactly Comply with Weak-Perspective Projection of Deforming Objects for 3D Reconstruction
D. Pizarro, A. Bartoli and T. Collins
BMVC - British Machine Vision Conference, 2013
[paper]
Monocular Template-Based 3D Reconstruction of Extensible Surfaces with Local Linear Elasticity
A. Malti, R. Hartley, A. Bartoli and J.-H. Kim
CVPR - IEEE International Conference on Computer Vision and Pattern Recognition, 2013
[paper]
Template-Based Isometric Deformable 3D Reconstruction with Sampling-Based Focal Length Self-Calibration
A. Bartoli and T. Collins
CVPR - IEEE International Conference on Computer Vision and Pattern Recognition, 2013
[paper]

Website created and maintained by Adrien Bartoli.