Why is D3D10SDKLayers.dll loaded during my DX11 game?

Question

SEE EDIT UPDATES BELOW. Original question has been modified!

I have a working window that uses a DX11 device. My problem comes along when I try to go fullscreen using Alt+Enter. If the window is not focused, i get a debug output that says:

'MyGame.exe': Loaded 'C:\Windows\SysWOW64\D3D10SDKLayers.DLL', Cannot find or open the PDB file

and then the warning

DXGI Warning: IDXGISwapChain::Present: Fullscreen presentation inefficiencies incurred due to application not using IDXGISwapChain::ResizeBuffers appropriately, specifying a DXGI_MODE_DESC not available in IDXGIOutput::GetDisplayModeList, or not using DXGI_SWAP_CHAIN_FLAG_ALLOW_MODE_SWITCH

I'm fairly certain a DX11 game should NOT be loading D3D10SDKLayers.dll, especially during the middle of execution. From the MSDN docs, I gathered that this dll is loaded upon device creation: if a device is created with the appropriate layer flag, this DLL is loaded automatically. So i checked to see if my device creation methods were being called somehow during the middle of execution, and they are not. I only have 2 spots in my game where device creation exists, and neither spot is getting hit. EDIT: after checking MSDN, it seems this dll is only a debug dll and might just be loaded to print out the warning itself, with no other purpose.

Explicit reproduction cases:

1) Alt+Enter 6 times (3 fullscreen transition cycles, both ways, starting windowed), on 7th the dll is loaded and the warning pops up. This happens regardless of window focusing.

Method Call Hierarchy Summary (going fullscreen):

1) ToggleFullscreen() - my method, only method called by Alt+Enter
2) ResizeTargetAndBuffers() - my method, sub-methods below
3) DXGISwapChain->ResizeTarget(frontBufferDesc) resizes front buffer to specified res
4) DXGISwapChain->GetFullscreenState() to determine fullscreen state
5) DXGISwapChain->SetFullscreenState(TRUE, NULL) to go fullscreen
6) ResizeDXGIBuffers(width, height, TRUE) my method, resizes back buffer, sub-method below
7) DXGISwapChain->ResizeBuffers(count, width, height, format, flags) to resize back buffer
8) DXGISwapChain->ResizeTarget(frontBufferDesc) prevents refresh rate issues. RefreshRate member zeroed out as per MSDN best practices.
9) DXGISwapChain->GetFullscreenState() to determine fullscreen state

Method Call Hierarchy Summary (going windowed):

1) ToggleFullscreen() - my method, only method called by Alt+Enter
2) ResizeTargetAndBuffers() - my method, sub-methods below
3) DXGISwapChain->ResizeTarget(backBufferDesc) resizes front buffer to specified res
4) DXGISwapChain->GetFullscreenState() to determine fullscreen state
5) DXGISwapChain->SetFullscreenState(FALSE, NULL) to go fullscreen
6) DXGISwapChain->ResizeTarget(backBufferDesc) to resize front buffer to window's res (helped resolve some res issues)
7) ResizeDXGIBuffers(width, height, FALSE) my method, resizes back buffer, sub-methods below
8) DXGISwapChain->ResizeBuffers(count, width, height, format, flags) to resize back buffer
9) DXGISwapChain->GetFullscreenState() to determine fullscreen state

The repercussions of this are quite severe. My low-level keyboard hook that catches Alt+Enter is no longer called, so windows is able to do its automatic Alt+Enter handling, which bypasses my ToggleFullscreen method entirely and sets the window to desktop resolution. This causes the buffers to be the wrong size (since I didn't set them, windows did), causing the inefficiency warning, and messing with the variables in my program which no longer have correct knowledge of buffer sizes and whether the window is fullscreen or not.

Any ideas as to what might be causing this?

P.S. If you need code samples be specific about what you want to see and I will try to put it up if possible. I can't put up the entire code listing.

EDIT: Device Creation code is below.

         hr = D3D11CreateDevice(    pAdapter,
                                    driverType,
                                    NULL,
                                    rDeviceSettings.m_CreateFlags,
                                    &rDeviceSettings.m_eD3DDeviceFeatureLevel,
                                    1,
                                    D3D11_SDK_VERSION,
                                    &pGraphicsDevice,
                                    &eFeatureLevel,
                                    &pDeviceContextI
            );


            if ( FAILED( hr ) ) {
                pAdapter = NULL;
                // Remote desktop does not allow you to enumerate the adapter.  In this case, we let D3D11 do the enumeration.
                if ( driverType == D3D_DRIVER_TYPE_UNKNOWN ) { 
                    hr = D3D11CreateDevice( pAdapter,
                                            driverType,
                                            NULL,
                                            rDeviceSettings.m_CreateFlags,
                                            &rDeviceSettings.m_eD3DDeviceFeatureLevel,
                                            1,
                                            D3D11_SDK_VERSION,
                                            &pGraphicsDevice,
                                            &eFeatureLevel,
                                            &pDeviceContextI
                    );
                }

The first call succeeds 99% of the time, namely when you are not using remote desktop, so I will focus only on it. I give it the adapter, driverType as D3D_DRIVER_TYPE_HARDWARE, m_CreateFlags as D3D11_CREATE_DEVICE_DEBUG, and m_eFeatureLevel as D3D_FEATURE_LEVEL_11_0. Pretty standard call, and it always succeeds.

EDIT UPDATE 1: After some extensive debugging, I've found that when the dll is loaded and the inefficiency warning pops up, some very interesting circumstances arise. They are listed below:

1) VS2010 debugger no longer triggers breakpoints in the key hook.
2) Printed output no longer works in the key hook.
3) window may become non-re-sizable if it was re-sizable before
4) window may become un-movable.
5) a trio of threads exit.

EDIT UPDATE 2: The first edit update may have incorrect assumptions; if i find it does, i will delete it. It turns out that my low-level key hook is no longer being called (i think, since no breakpoints or print statements work inside of it), so if something in my program is unregistering it accidentally then that would cause all of the above problems. Testing this tomorrow...

EDIT UPDATE 3: I'm not sure what is going on anymore. I tested the same clean project on my home computer and work computer, and got different results. At home, I can Alt+Enter indefinitely without any problems occuring, but at work Alt+Enter the 7th time causes the key hook to no longer be called and the buffer problems to happen.

EDIT UPDATE 4: More testing (at work). The key hook is definitely being removed after the 3rd transition into windowed mode. It no longer prints inside the key hook method at all, and breakpoints are not triggered, regardless of what key is pressed. I think I'm going to open a separate question regarding this, as all the other problems I describe above are just consequences of this key hook not calling ToggleFullscreen(). For reference, I have provided the key hook code below.

LRESULT _stdcall MyClass::WindowsKeyHook( s32 nCode, WPARAM wParam, LPARAM lParam ) {
    printf("Key hook called, nCode: %d. ", nCode);
    if( nCode < 0 || nCode != HC_ACTION )  { // do not process message 
        return CallNextHookEx( MyClassVar.GetWindowsKeyHook(), nCode, wParam, lParam );
    }
    printf(" Key hook status ok.\n");

    BOOL bEatKeystroke = FALSE;
    KBDLLHOOKSTRUCT* p = ( KBDLLHOOKSTRUCT* )lParam;
    switch( wParam ) {
        //NOTE: Alt seems to be a system key when it is PRESSED, but a regular key when it is released...
        case WM_SYSKEYDOWN:
            if(p->vkCode == VK_MENU || p->vkCode == VK_LMENU || p->vkCode == VK_RMENU) {
                MyClassVar.SetAltPressed(TRUE);
            }
            if(MyClassVar.IsAltPressed() && p->vkCode == VK_RETURN) {
                bEatKeystroke = TRUE;
                MyClassVar.SetAltEnterUsed(TRUE);
                printf("Alt+Enter used.\n");
            }
            break;
        case WM_SYSKEYUP:
            //NOTE: releasing alt+enter causes a SYSKEYUP message with code 0x13: PAUSE key...
            break;
        case WM_KEYDOWN:
            break;
        case WM_KEYUP: {
            if(p->vkCode == VK_MENU || p->vkCode == VK_LMENU || p->vkCode == VK_RMENU) {
                MyClassVar.SetAltPressed(FALSE);
            }
            bEatKeystroke = ( !MyClassVar.IsShortcutKeysAllowed() &&
                                ( p->vkCode == VK_LWIN || p->vkCode == VK_RWIN ) );
            break;
        }
    }

    if( bEatKeystroke ) {
        return 1;
    }
    else {
        return CallNextHookEx( MyClassVar.GetWindowsKeyHook(), nCode, wParam, lParam );
    }
}

The printf statements show that the key hook is being called until right after the 6th Alt+Enter. This is the 3rd transition into windowed mode. It is not necessary for me to go fullscreen a 4th time, as I thought before, to cause the problem. All methods called by MyClassVar are inline to make the key hook go as fast as possible, as I know there is a timeout on windows key hooks. The actual processing of Alt+Enter is handled by a thread in MyClass.

Also, can someone make this not a community wiki? I think this question is far too specific to be of use as a wiki. The only reason it became one is I've been updating it regularly with edits.

Answer 1

I resolved this problem by taking out the key hook entirely. The proper way to handle Alt+Enter is to create your swap chain, then call IDXGIFactory1::MakeWindowAssosciation( m_hWnd, DXGI_MWA_NO_ALT_ENTER). Then you can handle Alt+Enter with your own code from your windows message procedure using WM_SYSKEYDOWN and WM_KEYUP. I hope this helps others! I had a hell of a time getting this to work, so if you are having a hard time getting it to work in your own app, send me a message and I'll try to help you!