Teaching

Courses

2021

1. Geometric Computer Vision: from Images to 3D Models Arrigoni Federica, and Magri Luca [Abstract] [Webpage]

   This course presents the mathematical models underpinning several 3D vision algorithms, with a particular emphasis on synchronization and multi-model fitting, two streams of research that have been recently combined to derive better understanding of a 3D scene. The first part of the course describes the basic geometric tools of photogrammetric computer vision, and how they can be combined together to implement a modern 3D reconstruction pipeline. The second part of the course introduces the concept of “synchronization”, that is a general framework to solve problems involving multiple entities (e.g., images or 3D point clouds) organized as a “graph”, where the task is to seek for global consistency. Instances of synchronization include pose-graph optimization, multi-view matching and 3D point cloud registration. The third part of the course is devoted to “multi-model fitting”; the problem of robustly fitting a single parametric model will be first introduced, to move then to the general case of multiple models, with a focus on methods addressing this problem from a clustering perspective. Examples include fitting geometric primitives (e.g., lines or circles) to points in the plane and fitting geometric models (e.g., fundamental matrices or homographies) to correspondences in two images. Finally, the connections between synchronization and multi-model fitting are explained, with particular emphasis to the motion segmentation problem, where the task is to detect moving objects in a dynamic 3D scene.

Tutorials

2022

1. Inside Plato’s door: a tour in Multi-view Geometry Magri Luca, and Arrigoni Federica [Abstract] [Webpage]

   Nowadays we are experiencing a resurgence of interest in the traditional methods of Computer Vision. While the dazzling success of deep learning has inspired the research throughout the last decade, the geometry of vision has never ceased to be a fertile ground for investigation. This tutorial offers a self-contained overview of multi-view geometry, an essential tool in Computer Vision that is at the core of several classical and modern algorithms involving a variety of applications, including 3D reconstruction, motion segmentation, visual SLAM and augmented reality. Aimed primarily at an audience unfamiliar with these topics, this tutorial will guide the attendees on a tour that starts from the geometric foundations of 3D vision to reach up some of the research challenges that are still open in the field. The course will introduce classical topics such as camera models, camera calibration, the epipolar geometry, and then move to multi-view relations. We will also address the problem of robust estimation and structure from motion. Special emphasis will be put on rigorous mathematical formulations and on the practical aspects that separate theory from efficient and working implementations.

2020

1. Synchronization and Cycle Consistency in Computer Vision Birdal Tolga, Arrigoni Federica, Huang Qixing, and Guibas Leonidas [Abstract] [Webpage]

   Many of the computer vision problems involve processing multiple entities be it objects, shapes, views or scenes. In such cases, graphs a.k.a. networks, are the key data structures for storing and organizing information. Yet, relationships between individual entities that had to be encoded in the edges often remain pairwise, or rather local. One of the most well accepted methods of seeking a global agreement is enforcing cycle-consistency, where the local errors are distributed over the entire graph such that the composition of maps/transforms along the cycles is close to the identity map. This art of consistently recovering absolute quantities from a collection of ratios is known as synchronization. From training generative adversial networks to geometric structure from motion algorithms, from temporal video understanding to image-to-image translation, this capability of imposing consistency benefits a wide variety of vision tasks.

2. Synchronization: a general framework for mosaicking, 3D reconstruction, matching and segmentation problems Arrigoni Federica, Maset Eleonora, Bernard Florian, and Fusiello Andrea [Abstract] [Webpage]

   The synchronization problem has attracted a lot of attention in the community thanks to its application in a variety of Computer Vision tasks. The goal of “synchronization” is to infer the unknown states of a network of nodes, where only the ratio (or difference) between pairs of states can be measured. Typically, states are represented by elements of a group, such as the Symmetric Group or the Special Euclidean Group. The former can for example represent local labels of a set of features, as it occurs in multi-view matching applications. The latter can for example represent camera reference frames (e.g., in the context of structure from motion or pose graph optimization), or local coordinates of 3D points when dealing with 3D registration. Other applications include image mosaicking (where states are represented as homographies) and motion segmentation (where states are represented as binary matrices).