How is filters UIScrollView/UICollectionView in Apple's Photos app implemented that it opens so fast?

Question

I\'m not asking about the exact code but the overall idea.

Here is my problem: I\'m trying to create something similar to filter choosing UI in Photos app. I\'ve tr

Answer 1

The question seems to be how to display the CIImage resulting from a Core Image CIFilter as fast as possible — so fast that it appears instantly when the view controller appears; so fast, in fact, that the user can adjust the CIFilter parameters using sliders and so forth, and the image will redisplay live and keep up with the adjustment.

The answer is to use Metal Kit, and in particular a MTKView. The rendering work is moved off onto the device's GPU and is extremely fast, fast enough to come in under the refresh rate of the device's screen so that there is no discernible lag as the user twiddles the sliders.

I have a simple demonstration where the user applies a custom chain of filters called VignetteFilter:

As the user slides the slider, the amount of vignetting (the inner circle) changes smoothly. At every instant of sliding, a new filter is being applied to the original image and the filter is rendered, over and over as the user slides the slider, keeping up in synch with the user's movements.

The view at the bottom, as I said, is an MTKView. MTKView is not hard to work with in this way; it does require some preparation but it's all boilerplate. The only tricky part is actually getting the image to come out where you want it.

Here's the code for my view controller (I'm omitting everything but the slider and the display of the filtered image):

class EditingViewController: UIViewController, MTKViewDelegate {
    @IBOutlet weak var slider: UISlider!
    @IBOutlet weak var mtkview: MTKView!

    var context : CIContext!
    let displayImage : CIImage! // must be set before viewDidLoad
    let vig = VignetteFilter()
    var queue: MTLCommandQueue!

    // slider value changed
    @IBAction func doSlider(_ sender: Any?) {
        self.mtkview.setNeedsDisplay()
    }

    override func viewDidLoad() {
        super.viewDidLoad()

        // preparation, all pure boilerplate

        self.mtkview.isOpaque = false // otherwise background is black
        // must have a "device"
        guard let device = MTLCreateSystemDefaultDevice() else {
            return
        }
        self.mtkview.device = device

        // mode: draw on demand
        self.mtkview.isPaused = true
        self.mtkview.enableSetNeedsDisplay = true

        self.context = CIContext(mtlDevice: device)
        self.queue = device.makeCommandQueue()

        self.mtkview.delegate = self
        self.mtkview.setNeedsDisplay()
    }

    func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
    }

    func draw(in view: MTKView) {
        // run the displayImage thru the CIFilter
        self.vig.setValue(self.displayImage, forKey: "inputImage")
        let val = Double(self.slider.value)
        self.vig.setValue(val, forKey:"inputPercentage")
        var output = self.vig.outputImage!

        // okay, `output` is the CIImage we want to display
        // scale it down to aspect-fit inside the MTKView
        var r = view.bounds
        r.size = view.drawableSize
        r = AVMakeRect(aspectRatio: output.extent.size, insideRect: r)
        output = output.transformed(by: CGAffineTransform(
            scaleX: r.size.width/output.extent.size.width, 
            y: r.size.height/output.extent.size.height))
        let x = -r.origin.x
        let y = -r.origin.y

        // minimal dance required in order to draw: render, present, commit
        let buffer = self.queue.makeCommandBuffer()!
        self.context!.render(output,
            to: view.currentDrawable!.texture,
            commandBuffer: buffer,
            bounds: CGRect(origin:CGPoint(x:x, y:y), size:view.drawableSize),
            colorSpace: CGColorSpaceCreateDeviceRGB())
        buffer.present(view.currentDrawable!)
        buffer.commit()
    }
}