I\'ve build a simple test which creates and deletes a file (name does not change) in an infinite loop. The test does run for a couple of seconds (sometimes over 77,000 iteration
I've found an explanation while writing the question. I still posted the question because I wanted to share what I learned.
My application is not the only process on the system accessing files. The Windows Search Index Service for example could open this file because it wants to add it to it's index. Or the windows Explorer if it is updating the view.