Version 1 of a round wooden garden table with a vase of dried flowers, cut out of its garden automatically with the same pipeline as my LEGO Bonsai Tree v2 and LEGO Bulldozer v1.
How it was made:
- Photos and camera positions: the "garden" scene of the Mip-NeRF 360 dataset (Barron et al., CVPR 2022), 185 photos.
- Segmentation: Meta's SAM 3 tracked the table through all 185 photos from the text prompts "round wooden table", "wooden vase", "dried flower arrangement", "green soccer ball", "table legs" and "black round tray" - no manual masking. Frames where part of the table dropped out of the mask were refilled automatically by re-tracking from neighbouring frames, and a second pass checked every frame for the vase, ball, legs and tray.
- Splat-guided mask repair: a first splat was trained, turned into a 3D voxel grid, and holes enclosed by the table (the dark tray under the vase, which SAM had missed) were filled in 3D. That shape was projected back into all 185 photos to add what the masks missed, and the splat was trained again on the repaired masks.
- Training: Nerfstudio's splatfacto-big (gsplat), 30,000 steps, trained only on the masked table.
- Clean cut-out: each Gaussian kept or removed by how much it actually paints inside vs outside the masks across all views (the FlashSplat method). Then faint fringe along the edges and smears where the feet meet the ground were removed (the underside and feet were never photographed), stray clusters removed, and the model centred and scaled for viewing.
Credits:
- Mip-NeRF 360: Barron, Mildenhall, Verbin, Srinivasan & Hedman, "Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields", CVPR 2022.
- SAM 3 by Meta AI.
- Shen, Yang & Wang, "FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally", ECCV 2024.
- 3D Gaussian Splatting: Kerbl et al., SIGGRAPH 2023. Training: Nerfstudio + gsplat. Cameras supplied with the dataset (computed with COLMAP).
Built step by step with help from Claude (Anthropic).