TopoGaussian: Inferring Internal Topology Structures from Visual Clues

We present TopoGaussian, a mesh-free holistic pipeline for inferring the interior structure of an opaque object with easily accessible photos and videos as input.

Traditional approaches require intrusive sensors, operate on expensive equipment, or reconstruct surface information only. Our pipeline combines Gaussian splatting with a novel particle-based differentiable simulator for hollowed objects. Based on the gradients from this simulator, we provides two optimizers, including a new shape optimization scheme based on a particle representation, and a method based on neural implicit surface. The resultant pipeline is holistic and meshless, supporting modeling, simulation, and optimization of the interior geometry in a hollowed object on a unified point-cloud representation.

We showcase the results of our pipeline on several synthetic examples and four real-world tasks, involving rigid and soft objects and report the successful deduction of their interior designs that lead to visually plausible motions observed in the video input. Based on the output, we further performed 3D-print and validates the effect of our methods. These results highlight the potential of our pipeline in 3D-vision, soft-robotics, and manufacturing applications.