Sort a set of 3-D points in clockwise/counter-clockwise order

Question

In 3-D space I have an unordered set of, say, 6 points; something like this:

           (A)*
                          (C)*
(E)*
                         (F)         


        

Answer 1

The notion of "clockwise" or "counterclockwise" is not well-defined without an axis and orientation! (proof: What if you looked at those points from the other side of your monitor screen, or flipped them, for example!)

You must define an axis and orientation, and specify it as an additional input. Ways to specify it include:

• a line (1x=2y=3z), using the right-hand rule
• a (unit) vector (A_x, A_y, A_z), using the right-hand rule; this is the preferred way to do so

In order to determine the orientation, you have to look deeper at your problem: You must define a "up" and "down" size of the mesh. Then for each set of points, you must take the centroid (or another "inside" point) and construct a unit vector pointing "up" which is normal to the surface. (One way to do this would be to find the least-squares-fit plane, then find the two perpendicular vectors through that point, picking the one in the "up" direction.)

---

You will need to use any of the above suggestions to determine your axis. This will allow you to reformulate your problem as follows:

Inputs:

• the set of points {P_i}
• an axis, which we shall call "the z-axis" and treat as a unit vector centered on the centroid (or somewhere "inside") of the points
• an orientation (e.g. counterclockwise) chosen by one of the above methods

Setup:

• For all points, pick two mutually-orthogonal unit vectors to the axis, which we shall call "the y-axis" and "the x-axis". (Just rotate the z-axis unit-vector 90 degrees in two directions, http://en.wikipedia.org/wiki/Rotation_matrix#Basic_rotations )

Algorithm:

• For each point P, project P onto the x-axis and y-axis (using the dot product), then use http://en.wikipedia.org/wiki/Atan2

Once you have the angles, you can just sort them.

Answer 2

I can't attest for the efficiency of this code, but it works, and you can optimize parts of it as needed, I'm just not good at it.
Code is in C#, using system collection classes, and linq.
Vector3 is a class with floats x, y, z, and static vector math functions.
Node is a class with Vector3 variable called pos

//Sort nodes with positions in 3d space.
//Assuming the points form a convex shape.
//Assuming points are on a single plain (or close to it).

public List<Node> sortVerticies( Vector3 normal, List<Node> nodes ) {

    Vector3 first = nodes[0].pos;

    //Sort by distance from random point to get 2 adjacent points.
    List<Node> temp = nodes.OrderBy(n => Vector3.Distance(n.pos, first ) ).ToList();

    //Create a vector from the 2 adjacent points,
    //this will be used to sort all points, except the first, by the angle to this vector.
    //Since the shape is convex, angle will not exceed 180 degrees, resulting in a proper sort.
    Vector3 refrenceVec = (temp[1].pos - first);

    //Sort by angle to reference, but we are still missing the first one.
    List<Node> results = temp.Skip(1).OrderBy(n => Vector3.Angle(refrenceVec,n.pos - first)).ToList();

    //insert the first one, at index 0.
    results.Insert(0,nodes[0]);

    //Now that it is sorted, we check if we got the direction right, if we didn't we reverse the list.
    //We compare the given normal and the cross product of the first 3 point.
    //If the magnitude of the sum of the normal and cross product is less than Sqrt(2) then then there is more than 90 between them.
    if ( (Vector3.Cross( results[1].pos-results[0].pos, results[2].pos - results[0].pos ).normalized + normal.normalized).magnitude < 1.414f ) {
        results.Reverse();
    }

    return results;
}