Swift saving multiple images to the filesystem at once, High CPU

问题

I have the following func, I need to clear out all picture columns in a DB and move to the file system. When I did this all in one go, there was too much memory and it would crash. I switched to a recursive function and do the writes and batches of 20.

There are about 6 tables which I need to do this for. There is 2 and half gigs of data in my Realm DB. This gets switched to 40mb after I call my 6 recursive functions, taking images out and compressing Realm.

I can see very high memory usage as my functions are called and phones with less RAM would not be able to handle it.

How can I free up memory and CPU in between each function?

public static func clearEqCatPics(){

        let docsDir = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
        let eqcatPicDir = docsDir.appendingPathComponent(util_Constants.DIR_EQCAT_PICS)

        do {
            var realm : Realm? = try Realm()
            let predicate = NSPredicate(format: "icon != %@", "")
            let categories = realm!.objects(STD_EQ_category.self).filter(predicate).sorted( by: [SortDescriptor(keyPath: "displayorder", ascending: true), SortDescriptor(keyPath: "_id", ascending: true)] )

            if (categories.count > 0)
            {
                realm?.beginWrite()
                let upper = categories.count > 20 ? 20 : categories.count

                var actualCounter = upper
                for i in 0..<upper{
                    autoreleasepool{
                        if let proPicData = Data(base64Encoded: categories[actualCounter - 1].icon, options: .ignoreUnknownCharacters) {


                            let filename = eqcatPicDir.appendingPathComponent(categories[actualCounter - 1]._id.description+".jpg")

                            (proPicData as NSData).writeToURL2(named: filename, completion: { (result, url) -> Void in

                            })

                            categories[actualCounter - 1].icon = ""

                        }
                        else{
                            categories[actualCounter - 1].icon = ""
                        }
                    }

                    actualCounter = actualCounter - 1

                }

                try realm?.commitWrite()
                let eqcatNew = realm!.objects(STD_EQ_category.self).filter(predicate)
                print("$$$$$$$$$$$$$$$$$$$$ 2. eqcatNew COUNT : \(eqcatNew.count) $$$$$$$$$$$$$$$$$$$$")
                realm = nil
                if eqcatNew.count > 0 {
                    clearEqCatPics()
                }
            }
            realm = nil
        }
        catch let error as NSError {
            print("error realm \(error.localizedDescription)")

        }

    }

where writeToURL2 IS:

I needed to get rid of the weak self in my extension because I was getting past the guard let for multiple items and loads were being skipped

extension NSData {

    func writeToURL2(named:URL, completion: @escaping (_ result: Bool, _ url:NSURL?) -> Void)  {


        let tmpURL = named as NSURL

        //[weak self]
        DispatchQueue.global(qos: .background).async {  () -> Void in

            //guard let strongSelf = self else { print("self was weak"); completion (false, tmpURL); return }

            self.write(to: tmpURL as URL, atomically: true)
            var error:NSError?
            if tmpURL.checkResourceIsReachableAndReturnError(&error) {
                print("We have it")
                completion(true, tmpURL)
            } else {
                print("We Don't have it\(error?.localizedDescription)")
                completion (false, tmpURL)
            }

        }

    }
}

EDIT:

I changed my writeToURL in my for loop to the following:

do {
     try proPicData.write(to: filename, options: [.atomic])                               
}
catch let err as NSError{
     print("err : \(err.localizedDescription)")
}

It helped with memory, But sometimes I get Thread1: EXC_BAD_ACCESS pointing to the line try proPicData.write...

Still have very high CPU usage. Is there anyway to clear out CPU usage in between each function call?

回答1:

You are fetching all of the objects in your Realm at the same time, that is what is using up so much memory

let categories = realm!.objects(STD_EQ_category.self).filter(predicate).sorted( by: [SortDescriptor(keyPath: "displayorder", ascending: true), SortDescriptor(keyPath: "_id", ascending: true)] )

The writing to files is not what is using up memory, although that might have something to do with the CPU usage.

I would recommend putting a limit on the categories fetch so that you don't load all categories and their images into memory. Otherwise, try to come up with a fetch predicate that would otherwise limit the data in a sensible way.

来源：https://stackoverflow.com/questions/55932996/swift-saving-multiple-images-to-the-filesystem-at-once-high-cpu