CVMetalTextureCacheCreateTextureFromImage returns -6660 on macOS 10.13

问题

I'm recording the screen from my iPhone device to my Mac. As a preview layer I am collecting sample buffers directly from a AVCaptureVideoDataOutput, from which I'm creating textures and rendering them with Metal. The problem I'm having is that code that worked in macOS prior to 10.13 stopped working after updating to 10.13. Namely,

CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(_currentSampleBuffer);

if (!imageBuffer) return;

CVPixelBufferLockBaseAddress(imageBuffer,0);
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);

CVMetalTextureRef metalTexture = NULL;
CVReturn result = CVMetalTextureCacheCreateTextureFromImage(nil,
                                                            self.textureCache,
                                                            imageBuffer,
                                                            nil,
                                                            self.pixelFormat,
                                                            width,
                                                            height,
                                                            0,
                                                            &metalTexture);

if (result == kCVReturnSuccess) {
    self.texture = CVMetalTextureGetTexture(metalTexture);
}

Returns result = -6660, which translates to a generic kCVReturnError, as can be seen on the official Apple docs, and the metalTexture = NULL.

The pixel format I'm using is MTLPixelFormatBGRG422, since the samples coming from camera are 2vuy.

As a workaround to creating metalTexture from sampleBuffer, I am now creating an intermediate NSImage like so:

    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(_currentSampleBuffer);
    NSCIImageRep *imageRep = [NSCIImageRep imageRepWithCIImage:[CIImage imageWithCVImageBuffer:imageBuffer]];

    NSImage *image = [[NSImage alloc] initWithSize:[imageRep size]];
    [image addRepresentation:imageRep];

and creating a MTLTexture from that. That is obviously a subpar solution to using CVMetalTextureCacheCreateTextureFromImage directly.

Once again, the code in question works perfectly fine in macOS < 10.13, I'd like to know if anyone has similar issues, and if so, do you have any ideas how to overcome this?

回答1:

I've come across the same issue, the problem was not asking for Metal compatibility when configuring the AVCaptureVideoDataOutput. I guess the system started to check this in macOS 10.13, possibly to apply some optimization when not requested.

The solution was to add the kCVPixelBufferMetalCompatibilityKey to the videoSettings property of AVCaptureVideoDataOutput.

In Objective-C:

outputCapture.videoSettings = @{
  /* ... */
  (NSString *)kCVPixelBufferMetalCompatibilityKey: @YES
};

In Swift:

outputCapture.videoSettings = [
  /* ... */
  kCVPixelBufferMetalCompatibilityKey as String: true
]

I think this warrants a radar, to ask Apple to at least print a warning message when this occurs. I'll update this if I get to it.

回答2:

I found a workaround for this, which keeps the 2vuy format in the pixel buffer, but the bad thing is that you make a copy of the pixel buffer data which affects performance. I'm posting this for future reference, or if anyone else finds it useful. Basically we intercept the pixel buffer and then add attributes while copying the data.

NSDictionary *attributes = @{
                             @"IOSurfaceCoreAnimationCompatibility": @YES
                             };
CVPixelBufferRef copy = NULL;

CVPixelBufferCreate(kCFAllocatorDefault,
                    CVPixelBufferGetWidth(pixelBuffer),
                    CVPixelBufferGetHeight(pixelBuffer),
                    CVPixelBufferGetPixelFormatType(pixelBuffer),
                    (__bridge CFDictionaryRef)attributes,
                    &copy);

CVPixelBufferLockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);
CVPixelBufferLockBaseAddress(copy, 0);

void *baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer);
void *copyBaseAddress = CVPixelBufferGetBaseAddress(copy);

memcpy(copyBaseAddress, baseAddress, CVPixelBufferGetDataSize(pixelBuffer));

CVPixelBufferUnlockBaseAddress(copy, 0);
CVPixelBufferUnlockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);

来源：https://stackoverflow.com/questions/46549906/cvmetaltexturecachecreatetexturefromimage-returns-6660-on-macos-10-13

标签

objective-c

macos

metal