🏥 Preoperative-to-Intraoperative Registration for Minimally Invasive Surgery
Align preoperative imaging (CT/MRI) with intraoperative data for surgical navigation and AR overlay.
| Pipeline | Input | Method | Use Case |
|---|---|---|---|
| 🎥 CT-to-Video AR Overlay | Surgical video + CT mesh | Depth estimation + ICP + projection | Live AR surgical navigation |
| 📦 Volume Registration | CT/MRI volumes (.nii.gz) | SimpleITK intensity-based (rigid/affine/deformable) | Preop CT → intraop CBCT/MRI |
| ☁️ Point Cloud Registration | Surface models (.ply/.obj/.stl) | FPFH global + ICP refinement | CT surface → depth camera/3D scan |
| 📍 Landmark Registration | Paired fiducial coordinates | SVD-based rigid transform | Optical tracking fiducials |
| 🔬 Surface Extraction | CT/MRI volume | Marching cubes + smoothing | Generate 3D models from volumes |
Register CT 3D Model onto Live Surgical Video
Pipeline: Upload a surgical video frame → Depth Anything V2 estimates depth → depth map converted to 3D point cloud → CT mesh registered via ICP → mesh projected back onto video as translucent AR overlay.
Inputs: (1) Surgical video frame (image/video) (2) CT-derived mesh (.stl/.obj/.ply)
How it works:
- Depth estimation — Depth Anything V2 predicts per-pixel depth from a single RGB frame
- 3D reconstruction — Depth map + estimated camera intrinsics → intraoperative point cloud
- Registration — CT mesh auto-scaled and aligned to depth point cloud via ICP
- AR rendering — Registered mesh vertices projected to 2D and drawn as wireframe/filled/points overlay
30 120
0.1 1
1 20
10 200