How to read data from a UTexture2D in C++

Question

I am trying to read the pixel data from a populated UTexture2D in an Unreal Engine C++ project. Before I post the question here, I tried to use the method described in this link: https://answers.unrealengine.com/questions/25594/accessing-pixel-values-of-texture2d.html. However, it doesn't work for me. All pixel values I got from the texture are some garbage data.

I just want to get the depth values from the SceneCapture2D and a post-processing material that contains SceneTexture: Depth node. I need the depth values available in C++ so that I can do further processing with OpenCV. In Directx11, staging texture can be used for CPU read, but in the unreal engine, I don't know how to create a 'staging texture' like Dx11 has. I can't get the correct pixel values from the current method which makes me think I may try to access a no-CPU-readable texture.

Here is my experimental code for reading data back from an RGB UTexture2D.

Initialize the RGB Texture:

VideoTextureColor= UTexture2D::CreateTransient(640, 480, PF_B8G8R8A8);
VideoTextureColor->UpdateResource();
VideoUpdateTextureRegionColor = new FUpdateTextureRegion2D(0, 0, 0, 0, 640, 480);
ColorRegionData = new FUpdateTextureRegionsData;

PixelDepthData.Init(FColor(0, 0, 0, 255), 640 * 480);

// Populate the texture with blue color
for (int i = 0; i < 640; i++) {
    for (int j = 0; j < 480; j++) {
        int idx = j * 640 + i;
        PixelDepthData[idx].B = 255;
        PixelDepthData[idx].G = 0;
        PixelDepthData[idx].R = 0;
        PixelDepthData[idx].A = 255;
    }
}

UpdateTextureRegions(
    VideoTextureColor,
    (int32)0,
    (uint32)1,
    VideoUpdateTextureRegionColor,
    (uint32)(4 * 640),
    (uint32)4,
    (uint8*)PixelDepthData.GetData(),
    false,
    ColorRegionData
);

Then, update read its value back to the PixelDepthData (TArray type) array and update this texture with values storing in the PixelDepthData, which is its old value.

UpdateTextureRegions(
    VideoTextureColor,
    (int32)0,
    (uint32)1,
    VideoUpdateTextureRegionColor,
    (uint32)(4 * 640),
    (uint32)4,
    (uint8*)PixelDepthData.GetData(),
    false,
    ColorRegionData
);

ENQUEUE_UNIQUE_RENDER_COMMAND_ONEPARAMETER(
    FRealSenseDelegator,
    ARealSenseDelegator*, RealSenseDelegator, this,
    {
        FColor* tmpImageDataPtr = static_cast<FColor*>((RealSenseDelegator->VideoTextureColor)->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_ONLY));
        for (uint32 j = 0; j < 480; j++) {
            for (uint32 i = 0; i < 640; i++) {
                uint32 idx = j * 640 + i;
                RealSenseDelegator->PixelDepthData[idx] = tmpImageDataPtr[idx];
                RealSenseDelegator->PixelDepthData[idx].A = 255;
            }
        }
        (RealSenseDelegator->VideoTextureColor)->PlatformData->Mips[0].BulkData.Unlock();
    }
);

All I got is a white color texture instead of a blue color texture in the visualization scene.

Does anyone know how to read the data of the UTexture2D Object?

Answer 1

I figured that out. You have to get the UTexture2D's RHI texture reference first, and then use RHILockTexture2D to read it's data, and you have to do it in the RenderThread. The following code just an example:

FTexture2DResource* uTex2DRes = (FTexture2DResource*)(RealSenseDelegator->VideoTexturePixelDepth)->Resource;
float* cpuDataPtr = (float*)RHILockTexture2D(
        uTex2DRes->GetTexture2DRHI(),
        0,
        RLM_ReadOnly,
        destStride,
        false);

for (uint32 j = 0; j < 480; j++) {
    for (uint32 i = 0; i < 640; i++) {
        uint32 idx = j * 640 + i;

        // TODO Read the pixel data right here

    }
}
RHIUnlockTexture2D(uTex2DRes->GetTexture2DRHI(), 0, false);

To do this in the Render Thread, you have to use the Macro such as ENQUEUE_UNIQUE_RENDER_COMMAND_ONEPARAMETER // If you just one to pass one parameter to the render thread, use this one.+