Vertex shader attribute mapping in GLSL

Question

I'm coding a small rendering engine with GLSL shaders:

Each Mesh (well, submesh) has a number of vertex streams (eg. position,normal,texture,tangent,etc) into one big VBO and a MaterialID.

Each Material has a set of textures and properties (eg. specular-color, diffuse-color, color-texture, normal-map,etc)

Then I have a GLSL shader, with it's uniforms and attributes. Let's say:

uniform vec3 DiffuseColor;
uniform sampler2D NormalMapTexture;
attribute vec3 Position;
attribute vec2 TexCoord;

I'm a little bit stuck in trying to design a way for the GLSL shader to define the stream mappings (semantics) for the attributes and uniforms, and then bind the vertex streams to the appropriate attributes.

Something in the lines of saying to the mesh :"put your position stream in attribute "Position" and your tex coordinates in "TexCoord". Also put your material's diffuse color in "DiffuseColor" and your material's second texture in "NormalMapTexture"

At the moment I am using hard-coded names for the attributes (ie. vertex pos is always "Position" ,etc) and checking each uniform and attribute name to understand what the shader is using it for.

I guess I'm looking for some way of creating a "vertex declaration", but including uniforms and textures too.

So I'm just wondering how people do this in large-scale rendering engines.

Edit:

Recap of suggested methods:

1. Attribute/Uniform semantic is given by the name of the variable (what I'm doing now) Using pre-defined names for each possible attribute.The GLSL binder will query the name for each attribute and link the vertex array based on the name of the variable:

//global static variable

semantics (name,normalize,offset) = {"Position",false,0} {"Normal",true,1},{"TextureUV,false,2}

 ...when linking
for (int index=0;index<allAttribs;index++)
{
   glGetActiveAttrib(program,index,bufSize,length,size[index],type[index],name);      
   semantics[index]= GetSemanticsFromGlobalHardCodedList(name);
} 
... when binding vertex arrays for render
 for (int index=0;index<allAttribs;index++)
{
    glVertexAttribPointer(index,size[index],type[index],semantics[index]->normalized,bufferStride,semantics[index]->offset);

}  

2. Predefined locations for each semantic

GLSL binder will always bind the vertex arrays to the same locations.It is up to the shader to use the the appropriate names to match. (This seems awfully similar to method 1, but unless I misunderstood, this implies binding ALL available vertex data, even if the shader does not consume it)

.. when linking the program...
glBindAttribLocation(prog, 0, "mg_Position");
glBindAttribLocation(prog, 1, "mg_Color");
glBindAttribLocation(prog, 2, "mg_Normal");

3. Dictionary of available attributes from Material, Engine globals, Renderer and Mesh

Maintain list of availlable attributes published by the active Material, the Engine globals, the current Renderer and the current Scene Node.

eg:

 Material has (uniformName,value) =  {"ambientColor", (1.0,1.0,1.0)}, {"diffuseColor",(0.2,0.2,0.2)}
 Mesh has (attributeName,offset) = {"Position",0,},{"Normals",1},{"BumpBlendUV",2}

then in shader:

 uniform vec3 ambientColor,diffuseColo;
 attribute vec3 Position;

When binding the vertex data to the shader, the GLSL binder will loop over the attribs and bind to the one found (or not? ) in the dictionary:

 for (int index=0;index<allAttribs;index++)
    {
       glGetActiveAttrib(program,index,bufSize,length,size[index],type[index],name);      
      semantics[index] = Mesh->GetAttributeSemantics(name);
}

and the same with uniforms, only query active Material and globals aswell.

Answer 1

Attributes:

Your mesh has a number of data streams. For each stream you can keep the following info: (name, type, data).

Upon linking, you can query the GLSL program for active attributes and form an attribute dictionary for this program. Each element here is just (name, type).

When you draw a mesh with a specified GLSL program, you go through programs attribute dictionary and bind the corresponding mesh streams (or reporting an error in case of inconsistency).

Uniforms:

Let the shader parameter dictionary be the set of (name, type, data link). Typically, you can have the following dictionaries:

• Material (diffuse,specular,shininess,etc) - taken from the material
• Engine (camera, model, lights, timers, etc) - taken from engine singleton (global)
• Render (custom parameters related to the shader creator: SSAO radius, blur amount, etc) - provided exclusively by the shader creator class (render)

After linking, the GLSL program is given a set of parameter dictionaries in order to populate it's own dictionary with the following element format: (location, type, data link). This population is done by querying the list of active uniforms and matching (name, type) pair with the one in dictionaries.

Conclusion: This method allows for any custom vertex attributes and shader uniforms to be passed, without hard-coded names/semantics in the engine. Basically only the loader and render know about particular semantics:

• Loader fills out the mesh data streams declarations and materials dictionaries.
• Render uses a shader that is aware of the names, provides additional parameters and selects proper meshes to be drawn with.

Answer 2

From my experience, OpenGL does not define the concept of attributes or uniforms semantics.

All you can do is define your own way of mapping your semantics to OpenGL variables, using the only parameter you can control about these variables: their location.

If you're not constrained by platform issues, you could try to use the 'new' GL_ARB_explicit_attrib_location (core in OpenGL 3.3 if I'm not mistaken) that allows shaders to explicitly express which location is intended for which attribute. This way, you can hardcode (or configure) which data you want to bind on which attribute location, and query the shaders' locations after it's compiled. It seems that this feature is not yet mature, and perhaps subject to bugs in various drivers.

The other way around is to bind the locations of your attributes using glBindAttribLocation. For this, you have to know the names of the attributes that you want to bind, and the locations you want to assign them.

To find out the names used in a shader, you can:

• query the shader for active attributes
• parse the shader source code to find them yourself

I would not recommend using the GLSL parsing way (although it may suit your needs if you're in simple enough contexts): the parser can easily be defeated by the preprocessor. Supposing that your shader code becomes somewhat complex, you may want to start using #includes, #defines, #ifdef, etc. Robust parsing supposes that you have a robust preprocessor, which can become quite a heavy lift to set up.

Anyway, with your active attributes names, you have to assign them locations (and/or semantics), for this, you're alone with your use case.

In our engine, we happily hardcode locations of predefined names to specific values, such as:

glBindAttribLocation(prog, 0, "mg_Position");
glBindAttribLocation(prog, 1, "mg_Color");
glBindAttribLocation(prog, 2, "mg_Normal");
...

After that, it's up to the shader writer to conform to the predefined semantics of the attributes.

AFAIK it's the most common way of doing things, OGRE uses it for example. It's not rocket science but works well in practice.

If you want to add some control, you could provide an API to define the semantics on a shader basis, perhaps even having this description in an additional file, easily parsable, living near the shader source code.

I don't get into uniforms where the situation is almost the same, except that the 'newer' extensions allow you to force GLSL uniform blocks to a memory layout that is compatible with your application.

I'm not satisfied by all this myself, so I'll be happy to have some contradictory information :)

Answer 3

You may want to consider actually parsing the GLSL itself.

The uniform/attribute declaration syntax is pretty simple. You can come up with a small manual parser that looks for lines that start with uniform or attribute, get the type and name and then expose some C++ API using strings. This will save you the trouble of hard coded names. If you don't want to get your hands dirty with manual parsing a couple of likes of Spirit would do the trick.
You probably won't want to fully parse GLSL so you'll need to make sure you don't do anything funny in the decelerations that might alter the actual meaning. One complication that comes to mind is conditional compilation using macros in the GLSL.