Siddharth Choudhary

I am a Principal Applied Scientist in Amazon AGI. I closely collaborate with AWS AI Labs. Before this, I was an applied scientist in Amazon Halo where I worked on problems at the intersection of Computer Vision and Health.

Prior to Amazon, I was a Principal Computer Vision Researcher at Magic Leap. I finished my Ph.D. from the School of Interactive Computing at Georgia Institute of Technology. I was advised by Professor Henrik I. Christensen and Professor Frank Dellaert. I completed my Bachelors and Masters in Computer Science from IIIT Hyderabad where I was advised by Prof. P J Narayanan.

Email  /  CV  /  Google Scholar  /  Twitter  /  LinkedIn  /  Github

I'm interested in multimodal LLMs and its applications in computer vision and robotics.

Multimodal LLMs

Amazon Nova is a new line of foundation models delivering advanced AI capabilities. The family includes: Nova Pro (comprehensive multimodal model), Nova Lite (fast, cost-effective multimodal model), Nova Micro (efficient text model), Nova Canvas (image generation), and Nova Reel (video generation).

We address hallucination in Generative Vision-Language Models by introducing Multi-Modal Mutual-Information Decoding (M3ID), which amplifies the influence of reference images, reducing hallucinated responses by up to 28% without compromising linguistic fluency.

RAVEN is a multi-task retrieval-augmented vision-language model framework that improves performance on various tasks through efficient fine-tuning without requiring additional parameters.

3D Human Reconstruction and Health CVML

Propose a CNN based model called MeasureNet for accurately and reliably predicting body measurements and waist-hip ratio. Model is trained using realistic synthetic dataset.

A fully articulated Gaussian splatting model for human avatars. Our model includes both rigid and non-rigid skinning components, and a Neural Color Field for implicit color regularization.

Optimizing a SMPL+D mesh and an efficient, multi-resolution texture representation for novel view synthesis and pose synthesis.

A two-stage transformer-based system analyzes real-time fitness videos using pose sequences to detect exercise repetitions, classify movements, and identify form errors with severity levels.

Augmented Reality

An approach for multiuser and scalable 3D object detection, based on distributed data association and fusion.

Distributed Object SLAM

Decentralized visual SLAM combining distributed PGO using Gauss-Seidel and efficient, distributed place recognition using NetVLAD features.

Decentralized object based SLAM combining distributed PGO with joint modeling and mapping of object landmarks.

Propose a distributed implementation of the two-stage pose graph optimization, using Successive Over-Relaxation (SOR) and the Jacobi Over-Relaxation (JOR) as workhorses to split the computation among the robots. Extends it to work with object-based map models.

Multi robot SLAM approach that uses 3D objects as landmarks for localization and mapping. Leverages local computation at each robot (e.g., object detection and object pose estimation) to reduce the communication burden.

Leverages recent results which show that the maximum likelihood trajectory is well approximated by a sequence of two quadratic subproblems and solves it in a distributed manner, using the distributed Gauss-Seidel (DGS) algorithm.

Plans a trajectory which actively reduces the uncertainty of the robot's calibration given a rough initial calibration estimate.

Scalable SLAM using multiblock Alternating Direction Method of Multipliers (ADMM).

Information theoretic algorithm to efficiently reduce the number of landmarks and poses in a SLAM estimate without compromising the accuracy of the estimated trajectory.

Propose an approach for online object discovery and object modeling, and extend a SLAM system to utilize these discovered and modeled objects as landmarks to help localize the robot in an online manner.

Structure from Motion / GPU Computing

Encode the visibility information between and among points and cameras as visibility probabilities for improved localization and triangulation.

Hybrid implementation of sparse bundle adjustment on the GPU using CUDA, with the CPU working in parallel.

Credits to Jon Barron for website template.