I set up 404 handler page in web.config, but it works ONLY when extension of URL is .aspx (or other which is handled by ASP.NET). I know I can setup static HTML page in webs
You can setup wild card mapping in IIS (Application configuration/Mappings/Wildcard mappings/ - just set aspnet_isapi.dll as executable and uncheck the Verify that file exists box) that will route all incoming requests to your app - so you can control the behavior directly from it.
You don't have to setup static page in your IIS application settings. Imho, you should be able to setup valid url (e.g. /error_handler.aspx) from your app that will be used as landing page in case of specific server error.