Simulating refraction in SceneKit

Question

I am trying to create an ios 9 app for a project which will visualise 3d scenes with these special kind of theoretical lenses called glenses.

A ray tracing program called TIM has already been written from the ground up for simulating these glenses and more, but it's infeasable to simply port this to ios.

My understanding from searching the site (i.e. this answer and many others on shaders) is that it should be possible but I'm having a hard time getting the desired effect.

I decided that I would start by implementing a simpler form of refraction first before moving onto the more complex glens type refraction:

I was able to get a barrel distortion (fish eye lens) effect working on the camera using SCNTechnique, however it seems you can only use techniques on the camera or the whole scene, not individual pieces of geometry.

After that I tried to get the barrel distortion effect applied to the geometry by injecting opengl code using the SCNMaterial's shaderModifiers property:

    var shaders = [SCNShaderModifierEntryPoint: String]()

    try! shaders[SCNShaderModifierEntryPoint.fragment] = String(contentsOfFile: Bundle.main.path(forResource: "FishEye", ofType: "fsh")!, encoding: String.Encoding.utf8)

    try! shaders[SCNShaderModifierEntryPoint.geometry] = String(contentsOfFile: Bundle.main.path(forResource: "FishEye", ofType: "vsh")!, encoding: String.Encoding.utf8)

    let material = SCNMaterial()
    material.shaderModifiers = shaders
    object.geometry?.materials = [material]

I used slightly modified shaders found here:

fisheye.vsh

varying vec2 uv;

#pragma body

gl_Position = a_position;
uv = a_position.xy;

fisheye.fsh

uniform sampler2D colorSampler;
const float PI = 3.1415926535;
const float barrelPower = 0.5;
uniform vec2 rg;
uniform vec2 uv2;
varying vec2 uv;
uniform float d;
uniform vec2 xy;
uniform vec2 Vertex_UV;

vec2 Distort(vec2 p)
{   
    float theta  = atan(p.y, p.x);
    float radius = length(p);
    radius = pow(radius, barrelPower);
    p.x = radius * cos(theta);
    p.y = radius * sin(theta);
    return 0.5 * (p + 1.0);
}

#pragma body
#pragma transparent

vec2 xy = 2.0 * Vertex_UV.xy - 1.0;
vec2 rg = 2.0 * uv.xy - 1.0;
vec2 uv2;
float d = length(xy);
if (d < 1.0){
    uv2 = Distort(xy);
}else{
    uv2 = uv.xy;
}

gl_FragColor = texture2D(colorSampler, uv2);

These shaders compile and are loaded onto my object in the scene but do nothing; the object is opaque and white when it is nearly transparent without the injected shaders, and the #pragma transparent directive should make it transparent anyway.

To make it clear, what I'm trying to achieve here is having a 3d lens in a scene which you can look through and see the refracted image of whatever is on the other side.

Any help would be very much appreciated!

Answer 1

If you want to use your own vertex and fragment shader instead of SceneKit default shader program, you have too use SCNProgram instead of SCNShaderModifierEntryPoint.

SCNShaderModifierEntryPoints are only allow to modify default shader program of Swift.

let material = SCNMaterial()
            let program:SCNProgram = SCNProgram()
            do {
                program.vertexShader = try String(contentsOfFile: Bundle.main.path(forResource: "fisheye", ofType: "vsh")!, encoding: String.Encoding.utf8)
            } catch let error {
                print("shaderReadingError:\(error)")
            }
            do {
                program.fragmentShader = try String(contentsOfFile: Bundle.main.path(forResource: "fisheye", ofType: "fsh")!, encoding: String.Encoding.utf8)
            } catch let error {
                print("shaderReadingError:\(error)")
            }

    // and also your vertex shader has lack. 
    // You have to add some geometry source and transformation matrix to the vertex shader first with setSemantic method. 

            program.setSemantic(SCNGeometrySource.Semantic.vertex.rawValue, forSymbol: "vPosition", options: nil)
            program.setSemantic(SCNGeometrySource.Semantic.texcoord.rawValue, forSymbol: "uv", options: nil)
            program.setSemantic(SCNModelViewProjectionTransform, forSymbol: "uMVPMatrix", options: nil)





    // and also your fragment shader expect some realtime data like
    // colorSampler, rg, uv2, d, xy, Vertex_UV
    // you have to use handleBinding block to update this values before rendering the object.
            material.handleBinding(ofSymbol: "resolution", handler: { (programId:UInt32, location:UInt32, node:SCNNode?, renderer:SCNRenderer) in


            })


            material.program = program
            yourNode.geometry.firstMaterial = material