Solved:
* Workable Solution: @sbi
* Explanation for what really happens: @Hans
* Explanation for why OpenFile doesn't pass through "DELETE PENDING": @Benjamin
The Problem:
Our software is in large part an interpreter engine for a proprietary scripting language. That scripting language has the ability to create a file, process it, and then delete the file. These are all separate operations, and no file handles are kept open in between these operations. (i.e. during the file create a handle is created, used for writing, then closed. During the file processing portion, a separate file handle opens the file, reads from it, and is closed at EOF. and Finally, delete uses ::DeleteFile which only has use of a filename, not a file handle at all).
Recently we've come to realize that a particular macro (script) fails sometimes to be able to create the file at some random subsequent time (i.e. it succeeds during the first hundred iterations of "create, process, delete", but when it comes back to creating it a hundred and first time, Windows replies "Access Denied").
Looking deeper into the issue, I have written a very simple program that loops over something like this:
while (true) {
HANDLE hFile = CreateFileA(pszFilename, FILE_ALL_ACCESS, FILE_SHARE_READ, NULL, CREATE_NEW, FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile == INVALID_HANDLE_VALUE)
return OpenFailed;
const DWORD dwWrite = strlen(pszFilename);
DWORD dwWritten;
if (!WriteFile(hFile, pszFilename, dwWrite, &dwWritten, NULL) || dwWritten != dwWrite)
return WriteFailed;
if (!CloseHandle(hFile))
return CloseFailed;
if (!DeleteFileA(pszFilename))
return DeleteFailed;
}
As you can see, this is direct to the Win32 API, and pretty darn simple. I create a file, write to it, close the handle, delete it, rinse, repeat...
But somewhere along the line, I'll get an Access Denied (5) error during the CreateFile() call. Looking at sysinternal's ProcessMonitor, I can see that the underlying issue is that there is a pending delete on the file while I'm trying to create it again.
Questions:
* Is there a way to wait for the delete to complete?
* Is there a way to detect that a file is pending deletion?
We have tried the first option, by simply WaitForSingleObject() on the HFILE. But the HFILE is always closed before the WaitForSingleObject executes, and so WaitForSingleObject always returns WAIT_FAILED. Clearly, trying to wait for the closed handle doesn't work.
I could wait on a change notification for the folder that the file exists in. However, that seems like an extremely overhead-intensive kludge to what is a problem only occasionally (to wit: in my tests on my Win7 x64 E6600 PC it typically fails on iteration 12000+ -- on other machines, it can happen as soon as iteration 7 or 15 or 56 or never).
I have been unable to discern any CreateFile() arguments that would explicitly allow for this ether. No matter what arguments CreateFile has, it really is not okay with opening a file for any access when the file is pending deletion. And since I can see this behavior on both an XP box and on an x64 Win7 box, I am quite certain that this is core NTFS behavior "as intended" by Microsoft. So I need a solution that allows the OS to complete the delete before I attempt to proceed, preferably w/o tying up CPU cycles needlessly, and without the extreme overhead of watching the folder that this file is in (if possible).
Thanks for taking the time to read this and post a response. Clarifying Questions welcome!
[1] Yes, this loop returns on a failure to write or a failure to close which leaks, but since this is a simple console test app, the app itself exits, and Windows guarantees that all handles are closed by the OS when a process completes. So no leaks exist here.
bool DeleteFileNowA(const char * pszFilename)
{
// determine the path in which to store the temp filename
char szPath[MAX_PATH];
strcpy(szPath, pszFilename);
PathRemoveFileSpecA(szPath);
// generate a guaranteed to be unique temporary filename to house the pending delete
char szTempName[MAX_PATH];
if (!GetTempFileNameA(szPath, ".xX", 0, szTempName))
return false;
// move the real file to the dummy filename
if (!MoveFileExA(pszFilename, szTempName, MOVEFILE_REPLACE_EXISTING))
return false;
// queue the deletion (the OS will delete it when all handles (ours or other processes) close)
if (!DeleteFileA(szTempName))
return false;
return true;
}
Why don't you first rename the file to be deleted, and then delete it?
Use GetTempFileName() to obtain a unique name, then use MoveFile() to rename the file. Then delete the renamed file. If the actual deletion is indeed asynchronous and might conflict with the creation of the same file (as your tests seems to indicate), this should solve the problem.
Edit: Of course, if your analysis is right and file operations happen somewhat asynchronous, this might introduce the problem that you attempt to delete the file before the renaming is done. But then you could always keep trying to delete in a background thread.
Edit #2: If Hans is right (and I'm inclined to believe his analysis), then moving might not really help, because you might not be able to actually rename a file that's open by another process. (But then you might, I don't know this.) If that's indeed the case, the only other way I can come up with is "keep trying". You would have to wait for a few milliseconds and retry. Keep a timeout to give up when this doesn't help.
There are other processes in Windows that want a piece of that file. The search indexer is an obvious candidate. Or a virus scanner. They'll open the file for full sharing, including FILE_SHARE_DELETE, so that other processes aren't heavily affected by them opening the file.
That usually works out well, unless you create/write/delete at a high rate. The delete will succeed but the file cannot disappear from the file system until the last handle to it got closed. The handle held by, say, the search indexer. Any program that tries to open that pending-delete file will be slapped by error 5.
This is otherwise a generic problem on a multitasking operating system, you cannot know what other process might want to mess with your files. Your usage pattern seems unusual, review that first. A workaround would be to catch the error, sleep and try again. Or moving the file into the recycle bin with SHFileOperation().
Silly suggestion - since it fails so infrequently, how about simply waiting some milliseconds on failure and trying again? Or, if latency is important, switch to another file name, leaving the old file to be deleted later.
- Is there a way to detect that a file is pending deletion?
Use GetFileInformationByHandleEx function with FILE_STANDARD_INFO structure.
But the function can't solve your problem. @sbi's solution either.
This maybe not your particular issue, but it's possible so I suggest you get out Process Monitor (Sysinternals) and see.
I had exactly the same problem and discovered that Comodo Internet Security (cmdagent.exe) was contributing to the problem. Previously I had a dual-core machine, but when I upgraded to an Intel i7 suddenly my working software (jam.exe by Perfore software) no longer worked because it had the same pattern (a delete then create, but no check). After debugging the problem I found GetLastError() was returning access denied, but Process Monitor reveals a 'delete pending'. Here is the trace:
10:39:10.1738151 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
10:39:10.1738581 AM jam.exe 5032 QueryAttributeTagFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS Attributes: ANCI, ReparseTag: 0x0
10:39:10.1738830 AM jam.exe 5032 SetDispositionInformationFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS Delete: True
10:39:10.1739216 AM jam.exe 5032 CloseFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1739438 AM jam.exe 5032 IRP_MJ_CLOSE C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1744837 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1788811 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1838276 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1888407 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1936323 AM System 4 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS SyncType: SyncTypeOther
10:39:10.1936531 AM System 4 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1936647 AM System 4 IRP_MJ_CLOSE C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1939064 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat DELETE PENDING Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0
10:39:10.1945733 AM cmdagent.exe 1188 CloseFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1946532 AM cmdagent.exe 1188 IRP_MJ_CLOSE C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1947020 AM cmdagent.exe 1188 IRP_MJ_CLOSE C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS
10:39:10.1948945 AM cfp.exe 1832 QueryOpen C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat FAST IO DISALLOWED
10:39:10.1949781 AM cfp.exe 1832 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat NAME NOT FOUND Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
10:39:10.1989720 AM jam.exe 5032 CreateFile C:\Users\Dad\AppData\Local\Temp\jam5032t1.bat SUCCESS Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: 0, OpenResult: Created
As you can see, there is a request to delete followed by several attempts to open the file again by jam.exe (it's an fopen in a loop). You can see cmdagent.exe presumably had the file open as it closes its handle and then suddenly jam.exe is able to now open the file.
Of course, the suggested solution to wait and try again, and it works just fine.
Since you're creating a new file, processing it, then deleting it, it sounds like you don't really care about what the file name is. If that's truly the case, you should consider always creating a temporary file. That way, each time through the process, you don't have to care that the file didn't yet get deleted.
I actually had the same issue while using the LoadLibrary(path) I couldn't delete the file in path.
The solution was to "close the handle" or use the FreeLibrary(path) method.
NOTE: Please read the "Remarks" on MSDN regarding the FreeLibrary().
If CreateFile returns INVALID_HANDLE_VALUE then you should determine what GetLastError returns in your particular situation (pending delete) and loop back to CreateFile based on that error code only.
Edit
Would the FILE_FLAG_DELETE_ON_CLOSE flag buy you anything?
I may be late to the party but on Vista/Win7 there is DeleteFileTransacted which deletes file using transactions which ensures they are deleted (flushes file buffers etc.). For XP compatibility this is not an option though.
Another idea how this might be done is to OpenFile with flag OF_CREATE which sets length to zero if file exists or creates it if it doesn't and then to call FlushFileBuffers on file handle to wait for this operation (making file zero length) to complete. On completion the file is of size 0 and then simply call DeleteFile.
You can later test if the file exists or if it has zero-length to treat it the same way.
According to [1] you could use NtDeleteFile to avoid the asynchronous nature of DeleteFile. Also [1] gives some details on how DeleteFile works.
Unfortunately the official documentation on NtDeleteFile [2] doesn't mention any particular details on this issue.
[1] http://undocumented.ntinternals.net/index.html?page=UserMode%2FUndocumented%20Functions%2FNT%20Objects%2FFile%2FNtDeleteFile.html [2] https://msdn.microsoft.com/en-us/library/windows/hardware/ff566435(v=vs.85).aspx
The best answer was given by sbi, but in the interest of completeness, some people might also want to know about a new way now available from Windows 10 RS1/1603. It involves calling the SetFileInformationByHandle API with class FileDispositionInfoEx, and setting flags FILE_DISPOSITION_DELETE | FILE_DISPOSITION_POSIX_SEMANTICS. See the full answer by RbMm.
I think this is just by poor design in the file system. I have seen same problem when I worked with communication ports, opening/closing them.
Unfortunately I think the most simple solution would be to just retry to create the file a number of times if you get an INVALID_HANDLE_VALUE. GetLastError() might also give you a better way of detecting this particular INVALID_HANDLE_VALUE.
I would have preferred overlapped I/O, but there CloseHandle() and DeleteFile() don't handle overlapped operations :(
来源：https://stackoverflow.com/questions/3764072/c-win32-how-to-wait-for-a-pending-delete-to-complete