Our site is currently having this problem. Basically it only happen when we click some particular links where it will pop-up a new window.
This is the error message we
I've found that some developers will for some reason define the server's private IP outside of IIS in an unexpected location, like a nonstandard config file (i.e. not web.config) or a text file. This can cause internal operation to fail even when the service is started, ports aren't being blocked, reg keys are correct, etc.
Kaseya, in particular, places a file called serveripinternal.txt in the root IIS directory of the VSA server. I've seen the text of your error when somebody running their own Kaseya instance changed the server's internal IP. The server will be reachable, IIS will respond, and the login page will come up - but login will fail with the cited message.