0

I'm actually implementing my own Pose-Estimation/- and -Refinement pipeline. For this purpose I use one moving mono-camera. Then I take the consecutive images to estimate the pose and triangulate the points (nothing special). In the last step I refine the poses and 3D-points with a bundle adjustment approach.

Generating 3D points with triangulation from consecutive image pairs will give me multiple estimations for one real-world 3D point. In fact, all the estimations refer to the same point. For my understanding, these estimations of the same 3D-points have to be fused in some way. Otherwise the poses were not linked anymore through a common point (see also image below). Further, looking at the equation for the re-projection error in different publications:

Core equation Bundle Adjustment

turns out, that 3D point (vector a) is only related to j and not to the cameraindex i.

Stolen from: SBA: A Software Package for Generic Sparse Bundle Adjustment, MANOLIS I. A. LOURAKIS and ANTONIS A. ARGYROS

Do I understand that right or do I have to use a different set of 3D points for each camera view? Suppose I've to merge the 3D points, is there any preferable strategy?

Thanks in advance!

Edit: I know, there are already countless implementations for BA. I want to use it for further development...

Improve this question
4
  • 2
    Hi. This is more of a question for one of these SE sites, you'll probably have better luck there: Computer Science, Computer Graphics or even Signal Processing.
    – Filip Milovanović
    Commented Oct 6, 2019 at 16:13
  • About reprojection error: the way I understand it, the vectors a[j] represent actual points in 3D space (the estimated 3D coords of the black dots) - in "world space"; this is why they are independent of the camera index i - and why there's only one set of points. Q(a[j], b[i]) then (re)projects those points back onto each view [i]. Then, for each view, you compare the original point x (from the image) with the result of Q by finding the distance between them in "camera space" - you are adjusting the estimate and the camera positions, by minimizing the error in camera space.
    – Filip Milovanović
    Commented Oct 6, 2019 at 16:13
  • As for your other question - and I'm guessing here - you can probably just do an average of several estimates (for the same 3D point) to get a single "merged" point (just do vector sum, and scale by inverse of the number of estimates), and use that as the initial guess for a[j], then minimize the error from there.
    – Filip Milovanović
    Commented Oct 6, 2019 at 16:24
  • At first, thanks for your reply and the recommendations to further sites. About the reprojection error: I'm totally conform with your explanation. That's exactly how I understood it so far. For now I average the 3D-points, that derived from one common physical point. It seems to be a fair method to do this, but I still wonder why nobody mentioned this step in the publications. Maybe it's just to obvious... Thanks again for your answers. I will leave this thread open until I finished my implementation and testing.
    – MattDom
    Commented Oct 9, 2019 at 15:02

1 Answer 1

Reset to default
1

So, everything works now as expected and the results seem to be correct. I took the average of all 3D-Point belonging to one physical point. Therefore this question should be considered as answered.

Just a point for further investigation: Taking the average may not be a really robust way. It would be useful to implement something like an outlier control. But this was not part of the question at core.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.