How to find correct rotation from one vector to another?

Question

I have two objects, and each object has two vectors:

• normal vector
• up vector

Like on this image:

Answer 1

First of all, I make the claim that there is only one such transformation that will align the orientation of the two objects. So we needn't worry about finding the shortest one.

Let the object that will be rotated be called a, and call the object that stay stationary b. Let x and y be the normal and up vectors respectively for a, and similarly let u and v be these vectors for b. I will assume x, y, u, and v are unit length, and that is x is orthogonal to y, and u is orthogonal to v. If any of this is not the case code can be written to correct this (via planar projection and normalization).

Now let’s construct matrices defining the “world space” the orientation of a and b. (let ^ denote the cross product) construct z as x ^ y, and construct c as a ^ b. Writing x, y, z and a, b, c to columns of each matrix gives us the two matrices, call them A and B respectively. (the cross product here gives us a unit length and mutually orthogonal vector since the same is true of the operands)

The change of coordinate system transformation to obtain B in terms of A is A^-1 (the inverse of matrix A, where ^ denotes a generalization of an exponent), in this case A^-1 can be computed as A^T, the transpose, since A is an orthogonal matrix by construction. Then the physical transformation to B is just matrix B itself. So, transforming an object by A^-1, and then by B will give the desired result. However these transformations can be concatenated into one transformation by multiplying B on the right into A^-1 on the left.

You end up with this matrix (assuming no arithmetic errors):

 _                                                                                                                                                                                             _
| x0*u0+x1*u1+x2*u2                                    x0*v0+x1*v1+x2*v2                                    x0*(u1*v2-u2*v1)+x1*(u2*v0-u0*v2)+x2*(u0*v1-u1*v0)                                  |
|                                                                                                                                                                                               |
| y0*u0+y1*u1+y2*u2                                    y0*v0+y1*v1+y2*v2                                    y0*(u1*v2-u2*v1)+y1*(u2*v0-u0*v2)+y2*(u0*v1-u1*v0)                                  |
|                                                                                                                                                                                               |
| (x0*y2-x2*y1)*u0+(x2*y0-x0*y2)*u1+(x0*y1-x1*y0)*u2   (x0*y2-x2*y1)*v0+(x2*y0-x0*y2)*v1+(x0*y1-x1*y0)*v2   (x0*y2-x2*y1)*(u1*v2-u2*v1)+(x2*y0-x0*y2)*(u2*v0-u0*v2)+(x0*y1-x1*y0)*(u0*v1-u1*v0) |
|_                                                                                                                                                                                             _|