Calculating homography of multiple markers on a single plane

问题

I've got an algorithm that calculates the homography of a number of markers in a single image, using OpenCV's findHomography. This works fine, but due to the size of the markers/resolution of the images, the rotation calculated is often quite inaccurate (mainly around the x and z axes of the marker, assuming y is up). I'd love to find a way to improve this.

I have a hypothesis that the fact that I know all the markers are on the same plane (they're on a table) should help improve the homography - I then just need to work out the rotation of the marker around a single axis - but I don't know enough to put this in to practice. Can anyone give me any guidance on this?

This is an example of the situation I'm trying to deal with (the images on the markers are different in reality).

回答1:

I like the idea of getting higher precision for the estimated homography by using multiple marker images spread across the scene.

This approach can be realized using Nonlinear Optimization. But it’s a little tricky.

Assumptions, Model, Data

Let’s take your example of 3 printed marker images placed across a table and assume all your assumptions hold.

And now let’s assume that for each of the 3 printed marker images you’ve found N point correspondences.

Point correspondences here means that for a printed marker image you can associate N image pixel positions with corresponding N real world positions (x,y) in centimeters (or some other unit) with respect to a known real world "printed marker coordinate system" (for example, with coordinate (0,0) located at the center of a marker image).

The problem is that the "printed marker coordinate system" is only known locally for each printed marker image. What we need is to bring those coordinate systems together and find a way to express "printed marker coordinate system" positions in a single, global coordinate system. So we need to figure out a relation between them.

Relation between each marker’s homographies

We want to compute a homography which uses all the correspondence data from different printed marker images. Therefore, it is important to understand how they relate to each. Specifically, we want to find a common coordinate system for all marker images in order to formulate an optimization problem.

Now, based on the point correspondences, and using conventional methods, compute 3 homographies:

• H1 maps "printed marker coordinate system" positions of marker image #1 to image pixel coordinates
• H2 maps "printed marker coordinate system" positions of marker image #2 to image pixel coordinates
• H3 maps "printed marker coordinate system" positions of marker image #3 to image pixel coordinates

Let’s assume shortly that no measurement and computation errors are present (just hypothetically). Then, the homographies differ only in a rotation

and translation

of the respective "printed marker coordinate systems".

Then, the following equations hold:

with the rotation and translation defined as follows:

Putting everything together

That means that H1 equals H2 "up to" a rotation

and translation

and H1 equals H3 "up to" a rotation

and translation

.

This gives us a way for transforming positions of marker image #2 into the coordinate system of marker image #1 (case for positions of marker image #3 can be handled analogous):

Now, in a real world scenario, these equations will hold only approximately. But nevertheless they give a motivation for estimating the rotation and translation between any pair of printed marker images:

Now, using atan2, we can find estimates of six auxiliary variables

which describe how the printed marker images #2 and #3 are located relative to #1.

That’s all we need for preparation to have start values and a model to formulate a nonlinear optimization problem.

Nonlinear optimization problem

The final goal is to find 8 coefficients of a homography matrix H which uses all point correspondences from all printed marker images at the same time:

Using the above model, we can define a system of nonlinear equations. It will have 8 + 6 unknowns:

• 8 homography coefficients
• 6 auxiliary variables
  
  which describe how the correspondences from marker image #2 and #3 will be mapped to the marker image #1 coordinate system

Using the N point correspondences of 3 marker images, we get N*3*2 = N*6 equations. That means, we need N>=3 point correspondences in order to have more equations than unknowns. Interestingly, the minimum N=3 correspondences per marker image in case of the multiple marker image scenario is less than the minimum N=4 in the case of a single marker image scenario.

I’ll not add the formulation of the nonlinear optimization problem in more detail at this point as the general idea of how to approach the problem was already explained.

For solving the nonlinear optimization problem, Newton’s method can be used, for example.

Remark

All matrices in the above equations need to be normalized like follows:

回答2:

The homography model assumes that your markers are on a plane. This is not additional information that you can add. If your markers were not and a plane the function would not work at all.

If your markers are all in the image plane, i.e. all images are top-view images, then you can use an affine model (or lower) instead of the homography. It is not clear from your question if this is the case.

If all your markers are always on the same plane, you can calculate the plane homography from all of them simultaneously. The result should be more numerically stable the more points you have and the more they span the image. It should be more accurate than calculating each marker's homography independently.

The question is what your corresponding points are - you would need to know the relative position of the markers in the image as well.
If you use the same reference marker image the calculation will most likely fail.

来源：https://stackoverflow.com/questions/31424844/calculating-homography-of-multiple-markers-on-a-single-plane