I just stumbled upon an undocumented behavior of the GetFiles methods in System.IO.Directory.
Whenever the searchPattern parameter
As an addendum to this, had a related issue when using a json config file in an MSTest dll.
When I called it Test_Settings_Develop.json, Directory.GetFiles found it, but said it did not exist, so my tests failed to run, on attempting to load it.
Renamed it to TestSettings_Develop.json and it worked.