A point inside the collider ignores it when ray casting Unity

Question

Recently I got the idea to make my navigation mesh in unity. To do this, I split all baked objects into voxels. And then I project each vertex down to the distance of the si