If for instance, we set a -vendor-transform: rotate(40deg)
css attribute on a rectangle
You can get the current transformation matrix that is applied to an element by using getComputedStyle(). You can use this to transform the current mouse position to its position in transformed space and see whether the click/drag events are within the element boundary and/or corners. Good resources for this:
http://www.useragentman.com/blog/2011/01/07/css3-matrix-transform-for-the-mathematically-challenged/
http://www.eleqtriq.com/2010/05/css-3d-matrix-transformations/
BTW, as you're experiencing, this is non-trivial to code. We had to do it for Sencha Animator, and it was a beast.