fragments: local geometric surfaces

Make fragments

1. Register RGBD image pairs
   1. For adjacent RGBD images, an identity matrix is used as the initialization.
   2. For non-adjacent RGBD images, wide baseline matching is used as the initialization.
2. Multiway registration
   1. The process of aligning multiple pieces of geometry in a global space.
   2. The input is a set of geometries (e.g., point clouds or RGBD images) {𝐏𝑖}.
   3. The output is a set of rigid transformations {𝐓𝑖}
   4. So that the transformed point clouds {𝐓𝑖𝐏𝑖} are aligned in the global space.
3. Make a fragment
   1. Once the poses are estimated, RGBD integration is used to reconstruct a colored fragment from each RGBD sequence.
   2. Open3D implements a scalable RGBD image integration algorithm.
   3. Make ScalableTSDFVolume and integrate it with series of rgbd

Register fragments

Once the fragments of the scene are created, then align them in a global space. Input arguments: ["path_dataset"] should have subfolders fragments which stores fragments in .ply files and a pose graph in a .json file.

1. Preprocess point cloud
   1. Downsamples a point cloud to make it sparser and regularly distributed.
   2. Normals and FPFH feature are precomputed.
      1. The FPFH feature is a 33-dimensional vector that describes the local geometric property of a point. A nearest neighbor query in the 33-dimensinal space can return points with similar local geometric structures. See [Rasu2009] for details.
2. Compute initial registration
   1. Computes a rough alignment between two fragments.
   2. If the fragments are neighboring fragments, the rough alignment is determined by an aggregating RGBD odometry obtained from Make fragments.
   3. Otherwise, register_point_cloud_fpfh is called to perform global registration. Note that global registration is less reliable according to [Choi2015].
3. Pairwise global registration
   1. Uses RANSAC or Fast global registration for pairwise global registration.
   2. With proper configuration, the accuracy of fast global registration is even comparable to ICP. Please refer to [Zhou2016] for more experimental results.
4. Multiway registration
   1. Update_posegraph_for_scene
      1. Builds a pose graph for multiway registration of all fragments.
      2. Each graph node represents a fragment and its pose which transforms the geometry to the global space.
   2. Optimize_posegraph_for_scene

Refine registration

Perform pairwise registration on the pairs detected by Register fragments. and then perform multiway registration.

Input arguments: ["path_dataset"] should have subfolders fragments  which stores fragments in .ply files and a pose graph in a .json file.

1. Fine-grained registration
   1. Down-sampling and estimating normals
   2. ICP registration
      1. point_to_point
      2. point_to_plane
      3. color [recommended]
         1. Optimize a joint photometric and geometric objective that locks the alignment along both the normal direction and the tangent plane.
         2. Colored ICP is sensitive to threshold.
         3. Color information prevents drift
      4. generalized
2. Multiway registration
   1. Do the same process of step 7 with the local refined registration

Integrate scene