Given two rectangles with x, y, width, height in pixels and a rotation value in degrees -- how do I calculate the closest distance of their outlines toward each other?
This question depends on what kind of distance. Do you want, distance of centers, distance of edges or distance of closest corners?
I assume you mean the last one. If the X and Y values indicate the center of the rectangle then you can find each the corners by applying this trick
//Pseudo code
Vector2 BottomLeftCorner = new Vector2(width / 2, heigth / 2);
BottomLeftCorner = BottomLeftCorner * Matrix.CreateRotation(MathHelper.ToRadians(degrees));
//If LUA has no built in Vector/Matrix calculus search for "rotate Vector" on the web.
//this helps: http://www.kirupa.com/forum/archive/index.php/t-12181.html
BottomLeftCorner += new Vector2(X, Y); //add the origin so that we have to world position.
Do this for all corners of all rectangles, then just loop over all corners and calculate the distance (just abs(v1 - v2)).
I hope this helps you