How to capture depth data from camera in iOS 11 and Swift 4?

Question

I\'m trying to get depth data from the camera in iOS 11 with AVDepthData, tho when I setup a photoOutput with the AVCapturePhotoCaptureDelegate the photo.depthData is nil.

Answer 1

There are two ways to do this, and you are trying to do both at once:

1. Capture depth data along with the image. This is done by using the photo.depthData object from photoOutput(_:didFinishProcessingPhoto:error:). I explain why this did not work for you below.
2. Use a AVCaptureDepthDataOutput and implement depthDataOutput(_:didOutput:timestamp:connection:). I am not sure why this did not work for you, but implementing depthDataOutput(_:didOutput:timestamp:connection:) might help you figure out why.

I think that #1 is a better option, because it pairs the depth data with the image. Here's how you would do that:

@IBAction func capture(_ sender: Any) {

    let settings = AVCapturePhotoSettings(format: [AVVideoCodecKey: AVVideoCodecType.jpeg])
    settings.isDepthDataDeliveryEnabled = true
    self.sessionOutput?.capturePhoto(with: settings, delegate: self)

}

// ...

override func viewDidLoad() {
    // ...
    self.sessionOutput = AVCapturePhotoOutput()
    self.sessionOutput.isDepthDataDeliveryEnabled = true
    // ...
}

Then, depth_map shouldn't be nil. Make sure to read both this and this (separate but similar pages) for more information about obtaining depth data.

For #2, I'm not quite sure why depthDataOutput(_:didOutput:timestamp:connection:) isn't being called, but you should implement depthDataOutput(_:didDrop:timestamp:connection:reason:) to see if depth data is being dropped for some reason.