Passing Metal texture2d_array to SceneKit shader modifier

Question

I want to create shader modifier for SCNMaterial (with SCNShaderModifierEntryPointSurface) and pass Metal texture2d_array as custom input. To achieve that I try to use key-value coding setting parameter of type SCNMaterialProperty. From SceneKit manual on SCNMaterialProperty property contents:

You can set a value for this property using any of the following types:

• ...
• A texture (SKTexture, MDLTexture, MTLTexture, or GLKTextureInfo)
• ...

So, I construct MTLTexture:

• Create MTLTextureDescriptor with type MTLTextureType2DArray
• Create MTLTexture with MTLDevice's newTextureWithDescriptor:
• Fill texture with data using [texture replaceRegion:mipmapLevel:withBytes:bytesPerRow:];

Then I create SCNMaterialProperty, assign my created MTLTexture to its contents and pass it to material:

[material setValue:aTexture forKey:@"tex"];

Then I attach this shader modifier:

#pragma arguments
texture2d_array tex;

#pragma body
// do sampling from 1st slice
constexpr sampler smp(coord::normalized, address::repeat, filter::nearest);
_surface.diffuse = tex.sample(smp, _surface.diffuseTexcoord, 0);

And I get:

Backtrace:

I also tried to pass texture directly with [material setValue:aTexture forKey:@"tex"]; But then I'm getting error:

/Library/Caches/com.apple.xbs/Sources/Metal/Metal-56.6/ToolsLayers/Debug/MTLDebugRenderCommandEncoder.mm:1600: 
failed assertion `textureType mismatch at texture binding at index 8 for tex[0].'

Also, I tried passing MTLTexture2D in SCNMaterialProperty contents and It fails with exception in C3DImageGetImage type. Passing MTLTexture2D directly via [material setValue:forKey] results in everything white - wrong texture being sampled.

When analysing 'Capture GPU Frame' output, I see that wrong texture bound during draw call.

Maybe someone managed to pass texture2d_array to shader modifier? Is it possible at all? I could use metal command buffer to bind manually, but how to get access to it on per-material basis?

Thank you all.

Answer 1

        #include <metal_stdlib>
        #include <metal_texture>
        using namespace metal;
        #pragma arguments
        texture2d<float> mask;

        #pragma body
        constexpr sampler quadSampler(address::clamp_to_edge, filter::linear);

        _surface.diffuse = mask.sample(quadSampler, _surface.diffuseTexcoord);

then on swift side

let mask = SCNMaterialProperty(contents: MDLTexture(named: "gameFrame.png")!)
doorFrame.geometry?.firstMaterial?.setValue(mask, forKey: "mask")