What's the use of metaprogramming?

Question

I\'ve read:

• Wikipedia
• Code Generation vs. Metaprogramming
• The art of Metaprogramming
• Metaprogramming at c2.com

and I

Answer 1

I can give my own specific example: I am developing ABSE, which is a meta-programming approach. With ABSE you create a model (actually, a tree) where each item is an "Atom". This Atom represents a "concept" and contains the necessary meta-data for its definition.

In ABSE, the implementation of a concept is actually a "mini-program".

Then, the host modeler (AtomWeaver, developed alongside ABSE) takes the model and "weaves" a generator program out of all its Atoms. That program is then run, generating the desired artifacts (source code, data, etc).

So, the ABSE workflow is:

1. Create a discrete concept (a fraction of the meta-metaprogram)
2. Reuse that concept in a model (effectively building the metaprogram)
3. Host modeler weaves and runs the metaprogram
4. The metaprogram generates your final program

At first sight this looks like a lot of redundant, complex work, but it is actually quite straightforward if you grasp the concept.

Advantages of meta-programming (not exclusive to ABSE)?:

• Changing the model and regenerating a complete system (Imagine refactoring features instead of source lines).
• Changing a few definitions in the model can result in distinct programs (a Software Product Family).
• By reusing templates, you can change the template's code, regenerate and get your code changed in dozens, hundreds of places.
• Many others, really

Metaprogramming, code generation, program transformation are new exciting worlds in software development, IMHO. However, metaprogramming requires a new skill: meta-thinking.

We can define meta-thinking as "thinking about how you think about your own development". A kind of class reflection, applied on yourself. In practice, you must find out your own development patterns, isolate them, make them generic, and then turn them into metaprograms using your favorite technique, being it ABSE, DSL's, DSM, etc.

Answer 2

Metaprogramming based libraries/code help write directly explicit and simple code that will generate implementation details code for you, depending on parameters used.

Boost is full of (C++) libraries that demonstrate what can be achieved with metaprogramming. Some good (and maybe hard to understand) examples are Proto that allow implementation of DSL, Spirit that allow to write a compiler using EBNF grammar directly inside the code, and many other blow-minding libraries.

Answer 3

We use meta-programming a lot to create properties in VBA. We have various Excel spreadsheets with many headers on them and we want to define getter/setter properties for each header, allowing us to manipulate cells under that header. Manually doing this would be a nightmare.

The meta programming framework of choice for us was Notepad++ and its find/replace regular expressions capabilities. Here is how we meta-programmed our properties:

• Copy a list of headers from Excel to Notepad++
• Record a Notepad++ macro to clean up the data (remove whitespaces and special characters). At the end of this we have a list of newline separated strings.
• Manually copy the list to another .CSV file and use Excel to generate a list of line numbers. Then copy back to Notepad++.
• Write a regex to convert a property name into a property definition, adding all the whitespace, keywords etc. Use the line number as the column number in our property definition.

At the end of this we have a process that's a mixture of manual steps, recorded macros and a regex that we can re-apply every time we want properties for a sheet. And we did! To great effect.

That's the power of meta-programming. When to use it is a matter of experience/intuition. But I recommend answering this question:

Will if be quicker for me to just code this directly, or can I automate some/all of the process, and speed up my process?

That gives you a line to draw beyond which meta-programming is no longer useful. If you can just code it quicker, even if it's 10 repetitions, just do it! Only if it's hundreds of repetitions, or it's something you expect to reuse many times in future then meta program it.

Another point is that there are degrees here. I once wrote a Java program to create a bunch of files for adding a new IntelliJ inspection to an inspections coding project. That was a fair bit of overhead: creating the Java project and compiling it etc. On the other hand, Notepad++ find/replace is just a tiny step above manually typing stuff yourself. The advice here is to start doing things manually and then automate as you see a need, only up to the point where it makes sense. No need for a Java program when Notepad++ will do. No need for Notepad++ when manually typing it will do.

Answer 4

Imagine a guy who builds cars. Say it's the same thing as using a computer.
At some point he realizes he's always doing the same thing, more or less.
So he builds factories to build cars, and it's much better. He's now programming !
Nevertheless, once again, at some point, he realizes he's always doing the same thing, to some extent.
Now he decides to build factories that build factories that build cars. That's metaprogramming.

Metaprogramming is immensely powerful, but one glitch in the system makes all advantages turn into monster difficulties. So master it and use it... Or stay away !

Answer 5

I'll try to explain my concrete example of using meta programming techniques.

I've created a program tool which will generate ASP.NET web page source code from any MS Access data entry form. The technique that I used was to create my own ASP.NET text templates for each type of form control. I simply plugged in the values such as TOP, LEFT, HEIGHT, WIDTH, CONTROLSOURCE from the MS Access form objects meta data. For example, my template for an ASP.NET text box looks like this:

 <asp:TextBox ID="**ID**" runat="server" style="z-index: 1; left: **LL**px; top: **TOP**px; position: absolute"  Text='<%# Bind("[**CTLSOURCE**]") %>' />

after getting the textbox control meta data values, my program generates the code for the text box

<asp:TextBox ID="txtCustomerID" runat="server" style="z-index: 1; left: 50px; top: 240px; position: absolute"  Text='<%# Bind("[CustomerID]") %>' />

My program generates the entire web page source code for one MS Access form in 2-3 seconds.The alternative is to code by hand the ASP.NET web page from scratch; a task that could potentially take hours or even days.

Imagine an MS Access database with 24-35 forms. To hand code each and every form as an ASP.NET web page source code could take weeks if not months. Using a conversion tool with meta programming techniques , in this case, reduces development time for the web pages from weeks and months to hours .