# Fuse 3D-Points in Bundle Adjustment?

I'm actually implementing my own Pose-Estimation/- and -Refinement pipeline. For this purpose I use one moving mono-camera. Then I take the consecutive images to estimate the pose and triangulate the points (nothing special). In the last step I refine the poses and 3D-points with a bundle adjustment approach.

Generating 3D points with triangulation from consecutive image pairs will give me multiple estimations for one real-world 3D point. In fact, all the estimations refer to the same point. For my understanding, these estimations of the same 3D-points have to be fused in some way. Otherwise the poses were not linked anymore through a common point (see also image below). Further, looking at the equation for the re-projection error in different publications:

turns out, that 3D point (vector a) is only related to j and not to the cameraindex i.

Do I understand that right or do I have to use a different set of 3D points for each camera view? Suppose I've to merge the 3D points, is there any preferable strategy?

• About reprojection error: the way I understand it, the vectors `a[j]` represent actual points in 3D space (the estimated 3D coords of the black dots) - in "world space"; this is why they are independent of the camera index `i` - and why there's only one set of points. `Q(a[j], b[i])` then (re)projects those points back onto each view [i]. Then, for each view, you compare the original point `x` (from the image) with the result of Q by finding the distance between them in "camera space" - you are adjusting the estimate and the camera positions, by minimizing the error in camera space. – Filip Milovanović Oct 6 at 16:13
• As for your other question - and I'm guessing here - you can probably just do an average of several estimates (for the same 3D point) to get a single "merged" point (just do vector sum, and scale by inverse of the number of estimates), and use that as the initial guess for `a[j]`, then minimize the error from there. – Filip Milovanović Oct 6 at 16:24