Siddharth Choudhary

Lead Computer Vision Researcher, Magic Leap

Email, Google Scholar, Github, Linkedin

I am a Lead Computer Vision Researcher at Magic Leap. I finished my Ph.D. from the School of Interactive Computing at Georgia Institute of Technology. I was advised by Professor Henrik I. Christensen and Professor Frank Dellaert. I completed my Bachelors and Masters in Computer Science from IIIT Hyderabad where I was advised by Prof. P J Narayanan.


My primary research interests are in the areas of Robotics and Computer Vision and its intersection with Machine Learning and High Performance Computing. Over the last few years, I have been excited by the potential of AR/VR technology and interested in Computer Vision research needed to enable ubiquitous AR/VR head-mounted displays.

News


  • 2019.06 - Gave a tutorial in CVPR 2019 on Perception at Magic Leap.
  • 2017.10 - Joined the computer vision team at Magic Leap.
  • 2017.08 - Gave a successful demo of decentralized exploration and mapping system during the Micro Autonomous Systems and Technology (MAST) Collaborative Technology Alliance (CTA) Capstone event. The event was covered in The Economist, Phys, APG News, ARL News.

Research Projects


Distributed Object based SLAM with known Object Models


We propose a multi robot SLAM approach that uses 3D objects as landmarks for localization and mapping. The approach is fully distributed in that the robots only communicate during rendezvous and there is no centralized server gathering the data. Moreover, it leverages local computation at each robot (e.g., object detection and object pose estimation) to reduce the communication burden. We show that object-based representations reduce the memory requirements and information exchange among robots, compared to point-cloud-based representations; this enables operation in severely bandwidth-constrained scenarios. We test the approach in simulations and field tests, demonstrating its advantages over related techniques: our approach is as accurate as a centralized method, scales well to large teams, and is resistant to noise.


Publication:

[1] S. Choudhary, L. Carlone, C. Nieto, J. Rogers, Z. Liu, H. I. Christensen, F. Dellaert. Multi Robot Object-based SLAM, in ISER 2016. [pdf] [bibtex]

[2] S. Choudhary, L. Carlone, C. Nieto, J. Rogers, H. I. Christensen, F. Dellaert. Distributed Mapping with Privacy and Communication Constraints: Lightweight Algorithms and Object-based Models, in IJRR 2017. [arxiv] [code] [bibtex]

Distributed Object based SLAM with Joint Object Modeling and Mapping


We extend the previous work on Distributed Object based SLAM to the case where object models are previously unknown and are modeled jointly with Distributed Object-based SLAM. We show that this approach further reduces the memory required to store the object models while maintaining the accuracy at the same level as the state of art RGB-D mapping approaches.


Publication:

[1] S. Choudhary, A. Trevor, H. I. Christensen, F. Dellaert. SLAM with Object Discovery, Modeling and Mapping, in IROS 2014. [pdf] [www] [bibtex]

[2] S. Choudhary. Distributed Object based SLAM. Ph.D Thesis (2017). [pdf] [ppt]


Distributed Pose Graph Optimization


We propose a distributed algorithm to estimate the 3D trajectories of multiple cooperative robots from relative pose measurements. Our approach leverages recent results which show that the maximum likelihood trajectory is well approximated by a sequence of two quadratic subproblems. The main contribution of the present work is to show that these subproblems can be solved in a distributed manner, using the distributed Gauss-Seidel (DGS) algorithm. Our approach has several advantages. It requires minimal information exchange, which is beneficial in presence of communication and privacy constraints. It has an anytime flavor: after few iterations the trajectory estimates are already accurate, and they asymptotically convergence to the centralized estimate. The DGS approach scales well to large teams, and it has a straightforward implementation. We test the approach in simulations and field tests, demonstrating its advantages over related techniques.


Publication:

[1] S. Choudhary, L. Carlone, C. Nieto, J. Rogers, H. I. Christensen, F. Dellaert. Distributed Trajectory Estimation with Privacy and Communication Constraints: a Two-Stage Distributed Gauss-Seidel Approach, in ICRA 2016. [pdf] [ppt] [www] [bibtex]


Improving the Efficiency of Structure from Motion


Large scale reconstructions of camera matrices and point clouds have been created using structure from motion from community photo collections. Such a dataset is rich in information; we can interpret it as a sampling of the geometry and appearance of the underlying space. In this dissertation, we encode the visibility information between and among points and cameras as visibility probabilities. The conditional visibility probability of a set of points on a point (or a set of cameras on a camera) can be used to select points (or cameras) based on their dependence or independence. We use it to efficiently solve the problems of image localization and feature triangulation. We show how the conditional probability can be combined with other measures to prioritize a set of points (or cameras) for matching and use it for fast guided search of points for the image localization problem. We define the problem of feature triangulation as the estimation of 3D coordinate of a given 2D feature using the SfM data. Our approach can guide the search to quickly identify a subset of cameras in which the feature is visible.
Other than image localization and feature triangulation, bundle adjustment is a key component of the reconstruction pipeline and often its slowest and the most computational resource intensive. It hasn't been parallelized effectively so far. We also a present a hybrid implementation of sparse bundle adjustment on the GPU using CUDA, with the CPU working in parallel. The algorithm is decomposed into smaller steps, each of which is scheduled on the GPU or the CPU. We develop efficient kernels for the steps and make use of existing libraries for several steps. Our implementation outperforms the CPU implementation significantly, achieving a speedup of 30-40 times over the standard CPU implementation for datasets with upto 500 images on an Nvidia Tesla C2050 GPU.


Publication:

[1] S. Choudhary and P. J. Narayanan. Visibility Probability Structure from SfM Datasets and Applications, in ECCV 2012. [pdf] [www] [bibtex]

[2] S. Choudhary, S. Gupta, P. J. Narayanan. Practical Time Bundle Adjustment for 3D Reconstruction on GPU, in ECCV 2010. [pdf] [ppt]

S. Choudhary. Improving the Efficiency of SfM and its Applications, MS Thesis 2012. [pdf][www]

Softwares


Data-Efficient Decentralized Visual SLAM

This is the code for the 2018 ICRA paper Data-Efficient Decentralized Visual SLAM by Titus Cieslewski, Siddharth Choudhary and Davide Scaramuzza.

Distributed Pose Graph Optimization

This library is an implementation of the algorithm described in Distributed Trajectory Estimation with Privacy and Communication Constraints: a Two-Stage Distributed Gauss-Seidel Approach (ICRA 2016, IJRR 2017). The core library is developed in C++ language.

Planning to Calibrate

This library is an implementation of the algorithm described in Active Planning Based Extrinsic Calibration of Exteroceptive Sensors in Unkown Environments (IROS 2016). The core library is developed in C++ and MATLAB.

OmniMapper

A multi-modular mapping framework.


Others


Tutorial on Bundle Adjustment

This is a tutorial on Bundle Adjustment which I made for a reading group.