Umeå University's logo

This thesis presents a pipeline for physically accurate reconstruction of log pile scenes from RGB-D data, connecting real-world perception and physics-based simulation. The proposed method integrates SAM-6D, a zero-shot 6D pose estimation framework, with AGX Dynamics, a high-fidelity physics engine. Starting from RGB-D images and a CAD model reference of a log, SAM-6D identifies and performs 6D pose estimation for each individual log. The resulting poses and segmentation masks are used to infer the terrain beneath occluded regions through interpolation, generating an initial terrain guess. Direct simulation of the predicted scene, however, does not often result in stable configurations. To address this, a heightfield optimization process is introduced.The terrain under each log is perturbed locally, and candidate configurations are evaluated in simulation using a loss function that penalizes deviation from the predicted poses, accumulated linear and angular velocity after spawn, and terrain distortion. The system is evaluated on synthetic log pile scenes under varying conditions in three different tests: AGX generated log pile scenes, repeated optimization on a poorly performing configuration, and added environmental complexities using Blender. Results show that the optimized simulations achieve median position errors of 18 mm, which is 7% error relative to the chosen log diameter, and angular deviations below 1° after letting the logs settle for 78 AGX-generated scenes, with a resulting error decrease of 59%. The heightfield optimization also demonstrates consistency across 57 repeated runs on a log pile configuration withpoor initial stability, resulting in a 94% improvement. The pipeline successfully segmented all three logs in 5 out of 10 Blender generated scenes. For these, the optimized simulations achieved a median position error of 23% relative to the log diameter and angular deviations of 1.6°.