UWP Recursive File Search in C# is awfully slow

问题

I am using the following code to search recursively the images in a selected folder. I have over 120000 images in that folder (Yes, I'm a photographer!) and it is incredibly slow, for e.g. I have to stop it after 10 minutes and it's not done yet. In comparison, my Python code (interpreted!) does the same in less than 2 minutes.

Is there any way to make this code more efficient? It works fine, that's ok, but only very slowly...

    public List<StorageFile> _allFiles;
    public List<StorageFile> ShuffledFiles;
    public int i = 0;
    public int ri = 0;
    public Boolean random = false;
    public int numfiles = 0;

    //Get the starting folder for recursive search
    private static async Task<StorageFolder> SelectFolderAsync()
    {
        var folderPicker = new Windows.Storage.Pickers.FolderPicker
        {
            SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.Desktop
        };
        //Selects the folder with a FolderPicker and returns the selected StorageFolder
        folderPicker.FileTypeFilter.Add("*");
        StorageFolder folder = await folderPicker.PickSingleFolderAsync();
        return folder;
    }

    //Get the list of files recursively
    private async Task GetFilesInFolder(StorageFolder folder)
    {
        var items = await folder.GetItemsAsync();
        foreach (var item in items)
        {
            //If it's a folder, read each file in it and add them to the list of files "_allFiles"
            if (item is StorageFile )
            {
                StorageFile typetest = item as StorageFile;
                String ext = typetest.FileType.ToLower();
                if ((ext == ".jpg") || (ext == ".jpeg") || (ext == ".tiff") || (ext == ".cr2") || (ext == ".nef") || (ext == ".bmp") || (ext == ".png"))
                { _allFiles.Add(item as StorageFile);
                    numfiles = numfiles + 1;
                    //Display the file count so I can track where it's at...
                    cmdbar.Content = "Number of slides:"+numfiles.ToString();
                }
            }
            else
            //otherwise, recursively search the folder
                await GetFilesInFolder(item as StorageFolder);
        }
    }


    //Select the directory, load the files and display the first file
    private async void LoadMediaFile(object sender, TappedRoutedEventArgs e)
    {
        StorageFolder root = await SelectFolderAsync();
        //Initialises the file list _allFiles, the filecount numfiles, and the pointers to the list i and ri
        _allFiles = new List<StorageFile>();
        numfiles = 0;
        //Reads the files recursively into the list
        await GetFilesInFolder(root);
    }

回答1:

I don't have so many photo that I quickly can test but two things you can try.

Use the System.IO namespace; I noticed some improvements improvements in my app when I switched to that API.
Don't manually iterate over it by try to use the seaerch api: https://docs.microsoft.com/en-us/windows/uwp/files/quickstart-listing-files-and-folders#query-files-in-a-location-and-enumerate-matching-files (I think would be the best approach)

回答2:

Is there any way to make this code more efficient?

Derive from Universal Windows Platform - File System Monitoring in Universal Windows Platform Apps

The system is now able to provide a list of all the changes that are happening in a library, from a picture being taken all the way up to entire folders being deleted. This is a huge help if you’re looking to build a cloud backup provider, track files being moved off the device or even just display the most recent photos.

It means that the system will create database to record the index of the file. And Windows Storage API has provide CreateFileQueryWithOptions that use file index to query file efficiently.

StorageFolder photos = KnownFolders.CameraRoll;
// Create a query containing all the files your app will be tracking
QueryOptions option = new QueryOptions(CommonFileQuery.DefaultQuery,
  supportedExtentions);
option.FolderDepth = FolderDepth.Shallow;
// This is important because you are going to use indexer for notifications
option.IndexerOption = IndexerOption.UseIndexerWhenAvailable;
StorageFileQueryResult resultSet =
  photos.CreateFileQueryWithOptions(option);
// Indicate to the system the app is ready to change track
await resultSet.GetFilesAsync(0, 1);
// Attach an event handler for when something changes on the system
resultSet.ContentsChanged += resultSet_ContentsChanged;

And this related blog you could refer. Change Tracking: More Betterness.

回答3:

Ok, tried using the Windows.Storage.Search API. Using the code below, I scan a subtree of 70,000 files in 1 min 45 sec. The recursive code above (in my original question) takes 1 min 32 sec (faster...!). Interestingly enough, I would have thought the recursive code would have taken more time and resources because it has more overhead?!?!?!

And my Python interpreted code takes only 3 seconds!!! For the same thing!!!

There has got to be a better way, no? Microsoft engineers around, any hints?

    //This code is actually taken almost literally from the Microsoft example 
    // given here: https://docs.microsoft.com/en-us/uwp/api/windows.storage.search.queryoptions
   private async Task GetFilesInFolder(StorageFolder folder)
    {
            List<string> fileTypeFilter = new List<string>();
            fileTypeFilter.Add(".png");
            fileTypeFilter.Add(".jpg");
            fileTypeFilter.Add(".jpeg");
            fileTypeFilter.Add(".tiff");
            fileTypeFilter.Add(".nef");
            fileTypeFilter.Add(".cr2");
            fileTypeFilter.Add(".bmp");
            QueryOptions queryOptions = new QueryOptions(Windows.Storage.Search.CommonFileQuery.OrderByName, fileTypeFilter);
            queryOptions.FolderDepth = FolderDepth.Deep;
            queryOptions.IndexerOption = IndexerOption.UseIndexerWhenAvailable;
            StorageFileQueryResult queryResult = folder.CreateFileQueryWithOptions(queryOptions);
            var files = await queryResult.GetFilesAsync();

            if (files.Count == 0)
            {
                await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () => { cmdbar.Content = "Nothing found!" ; });
            }
            else
            {
                // Add each file to the list of files (this takes 2 seconds)
                foreach (StorageFile file in files)
                {
                    _allFiles.Add(file as StorageFile);
                    numfiles = numfiles + 1;
                    //Display the file count so I can track where it's at...
                    await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () => { cmdbar.Content = "Number of slides:" + numfiles.ToString(); });
                }
            }
    }

来源：https://stackoverflow.com/questions/52214932/uwp-recursive-file-search-in-c-sharp-is-awfully-slow

标签

list

file

uwp