I have an HttpHandler that is run on a client page (cross domain, not on our IIS server, etc) and when they click on our embedded link it fires off the Handler on our server
I think you have to implement the empty interface IReadOnlySessionState, so the context will be loaded.
edit to add:
According to Michael Morton's answer, you can also implement IRequiresSessionState, which will give you write access also to the Session object