So I thought I had finally got everything setup on Windows ... then ran into this issue.
Current setup
URL: ssh://user@host:port/myapp.git
I get message: "remote end gung-up, unexpected EOFs, index-pack failed" while cloning git repo but was able to workaround this issue with copssh http://sourceforge.net/projects/sereds/files/Copssh
The root cause is Cygwin openssh package.
I removed original openssh from Cygwin, installed copssh and set GIT_SSH variable to point copssh binaries. After this I simply added copssh binaries in the PATH and now I'm using copssh instead of openssh - there is no difference.
Everything works fine and issue does not appear any more.
The other solution is to use mSysGit package.