Building multiple executables with similar rules

I am writing something like an interactive tutorial for C++. The tutorial will consist of two parts: one is compiled into a library (I'm using Scons to build that), and the other (the lessons) is shipped with the tutorial to be compiled by the end user. I'm currently looking for a good, easy way for people to build these lessons.

Basically, the second part is a directory with all the lessons in it, each in its own directory. Each lesson will have at least a lesson.cpp and a main.cpp file, there may be also other files, the existence of which I will not know until after it is shipped -- the end user will create these. It will look something like this:

all_lessons/
    helloworld/
        lesson.cpp
        main.cpp
    even_or_odd/
        lesson.cpp
        main.cpp
    calculator/
        lesson.cpp
        main.cpp
        user_created_add.cpp

Each of these will need to be compiled according to almost the same rules, and the command for compiling should be possible to run from one of the lesson directories (helloworld/, etc.).

Seeing as the rest of the project is built with Scons, it would make sense to use it for this part, too. However, Scons searches for the SConstruct file in the directory it is run from: would it be acceptable to put a SConstruct file in each lesson directory, plus a SConscript in the all_lessons/ directory that gives the general rules? This seems to go against the typical way Scons expects projects to be organised: what are the potential pitfalls of this approach? Could I put a SConstruct file instead of the SConscript one, and thereby make it possible to build from either directory (using exports to avoid endless recursion, I'm guessing)?

Also, I may at some point want to replace the lesson.cpp with a lesson.py that generates the necessary files; will Scons allow me to do this easily with builders, or is there a framework that would be more convenient?

In the end, I want to end up with the following (or equivalent with different build systems):

all_lessons/
    SConstruct
    helloworld/
        SConstruct
        lesson.cpp
        main.cpp
    even_or_odd/
        SConstruct
        lesson.py
        main.cpp
    calculator/
        SConstruct
        lesson.cpp
        main.cpp
        user_created_add.cpp

Running scons all in the all_lessons directory would need to:

Run even_or_odd/lesson.py to generate even_or_odd/lesson.cpp.
Realise that user_created_add.cpp also needs to be compiled.
Produce an executable for each lesson.

Running scons in even_or_odd/, or scons even_or_odd in all_lessons/ should produce an executable identical to the one above (same compile flags).

Summary:

Is Scons suitable for/capable of this?
Does Scons work well when SConscript files are above SConstruct files?
Does Scons work well with multiple SConstrcut files for one project, SConscripting each other?
Is the Scons builder system suitable for using Python scripts to generate C++ files?
Is there any advantage of using a different build system/writing my own build framework that I'm missing?

Any further comments are, of course, welcome.

Thanks.

You can actually do this with a few lines of GNU Make.

Below are two makefiles that allow building and cleaning from all_lessons directory and individual project directories. It assumes that all C++ sources in that directory comprise an executable file which gets named after its directory. When building and cleaning from the top level source directory (all_lessons) it builds and cleans all the projects. When building and cleaning from a project's directory it only builds and cleans the project's binaries.

These makefiles also automatically generate dependencies and are fully parallelizable (make -j friendly).

For the following example I used the same source file structure as you have:

$ find all_lessons
all_lessons
all_lessons/even_or_odd
all_lessons/even_or_odd/main.cpp
all_lessons/Makefile
all_lessons/helloworld
all_lessons/helloworld/lesson.cpp
all_lessons/helloworld/main.cpp
all_lessons/project.mk
all_lessons/calculator
all_lessons/calculator/lesson.cpp
all_lessons/calculator/user_created_add.cpp
all_lessons/calculator/main.cpp

To be able to build from individial project directories project.mk must be symlinked as project/Makefile first

[all_lessons]$ cd all_lessons/calculator/
[calculator]$ ln -s ../project.mk Makefile
[helloworld]$ cd ../helloworld/
[helloworld]$ ln -s ../project.mk Makefile
[even_or_odd]$ cd ../even_or_odd/
[even_or_odd]$ ln -s ../project.mk Makefile

Let's build one project:

[even_or_odd]$ make
make -C .. project_dirs=even_or_odd all
make[1]: Entering directory `/home/max/src/all_lessons'
g++ -c -o even_or_odd/main.o -Wall -Wextra   -MD -MP -MF even_or_odd/main.d even_or_odd/main.cpp
g++ -o even_or_odd/even_or_odd even_or_odd/main.o  
make[1]: Leaving directory `/home/max/src/all_lessons'
[even_or_odd]$ ./even_or_odd
hello, even_or_odd

Now build all projects:

[even_or_odd]$ cd ..
[all_lessons]$ make
g++ -c -o calculator/lesson.o -Wall -Wextra   -MD -MP -MF calculator/lesson.d calculator/lesson.cpp
g++ -c -o calculator/user_created_add.o -Wall -Wextra   -MD -MP -MF calculator/user_created_add.d calculator/user_created_add.cpp
g++ -c -o calculator/main.o -Wall -Wextra   -MD -MP -MF calculator/main.d calculator/main.cpp
g++ -o calculator/calculator calculator/lesson.o calculator/user_created_add.o calculator/main.o  
g++ -c -o helloworld/lesson.o -Wall -Wextra   -MD -MP -MF helloworld/lesson.d helloworld/lesson.cpp
g++ -c -o helloworld/main.o -Wall -Wextra   -MD -MP -MF helloworld/main.d helloworld/main.cpp
g++ -o helloworld/helloworld helloworld/lesson.o helloworld/main.o  
[all_lessons]$ calculator/calculator 
hello, calculator
[all_lessons]$ helloworld/helloworld
hello, world

Clean one project:

[all_lessons]$ cd helloworld/
[helloworld]$ make clean
make -C .. project_dirs=helloworld clean
make[1]: Entering directory `/home/max/src/all_lessons'
rm -f helloworld/lesson.o helloworld/main.o helloworld/main.d helloworld/lesson.d helloworld/helloworld
make[1]: Leaving directory `/home/max/src/all_lessons'

Clean all projects:

[helloworld]$ cd ..
[all_lessons]$ make clean
rm -f calculator/lesson.o calculator/user_created_add.o calculator/main.o even_or_odd/main.o helloworld/lesson.o helloworld/main.o calculator/user_created_add.d calculator/main.d calculator/lesson.d even_or_odd/main.d  calculator/calculator even_or_odd/even_or_odd helloworld/helloworld

The makefiles:

[all_lessons]$ cat project.mk 
all :
% : forward_ # build any target by forwarding to the main makefile
    $(MAKE) -C .. project_dirs=$(notdir ${CURDIR}) $@
.PHONY : forward_

[all_lessons]$ cat Makefile 
# one directory per project, one executable per directory
project_dirs := $(shell find * -maxdepth 0 -type d )

# executables are named after its directory and go into the same directory
exes := $(foreach dir,${project_dirs},${dir}/${dir})

all : ${exes}

#  the rules

.SECONDEXPANSION:

objects = $(patsubst %.cpp,%.o,$(wildcard $(dir ${1})*.cpp))

# link
${exes} : % : $$(call objects,$$*) Makefile
    g++ -o $@ $(filter-out Makefile,$^) ${LDFLAGS} ${LDLIBS}

# compile .o and generate dependencies
%.o : %.cpp Makefile
    g++ -c -o $@ -Wall -Wextra ${CPPFLAGS} ${CXXFLAGS} -MD -MP -MF ${@:.o=.d} $<

.PHONY: clean

clean :
    rm -f $(foreach exe,${exes},$(call objects,${exe})) $(foreach dir,${project_dirs},$(wildcard ${dir}/*.d)) ${exes}

# include auto-generated dependency files
-include $(foreach dir,${project_dirs},$(wildcard ${dir}/*.d))

As an exercise in learning scons, I've tried to answer your question. Unfortunately, I'm no expert, so I can't tell you what's the best/ideal way, but here's a way that works.

Scons is suitable for/capable of this. (This is exactly what build tools are for.)
Not applicable. (I don't know.)
Scons seems to work well with multiple SConstrcut files for one project, SConscripting each other.
The Scons builder system can use Python scripts to generate C++ files.
A different build system? To each his own.

Using the hierarchy you defined, there's a SConstruct file in each folder. You can run scons in a subfolder to build that project or at the top level to build all projects (not sure how you'd alias "all" to the default build). You can run scons -c to clean the project and scons automatically figures out which files it created and cleans them (including the generated lesson.cpp).

However, if you want compiler flags to propagate from the top-level file down, I think it's better to use SConscript files -- except I'm not sure about making these compile on their own.

./SConstruct

env = Environment()
env.SConscript(dirs=['calculator', 'even_or_odd', 'helloworld'], name='SConstruct')

./calculator/SConstruct and ./calculator/helloworld

env = Environment()
env.Program('program', Glob('*.cpp'))

./even_or_odd/SConstruct

env = Environment()

def add_compiler_builder(env):
    # filename transformation
    suffix = '.cpp'
    src_suffix = '.py'

    # define the build method
    rule = 'python $SOURCE $TARGET'

    bld = Builder(action = rule,
                  suffix = suffix,
                  src_suffix = src_suffix)
    env.Append(BUILDERS = {'Lesson' : bld})

    return env

add_compiler_builder(env)

env.Lesson('lesson.py')
env.Program('program', Glob('*.cpp'))

Using SConscripts

I convert the subfolder's SConstructs to SConscripts and can lift the code build specifics out of the subfolders, but then you need to run scons -u to build in a subfolder (to search upwards for the root SConstruct).

./SConstruct

def default_build(env):
    env.Program('program', Glob('*.cpp'))

env = Environment()
env.default_build = default_build

Export('env')
env.SConscript(dirs=['calculator', 'even_or_odd', 'helloworld'])

./helloworld/SConscript, etc...

Import('env')
env.default_build(env)

Is it essential that the command for compiling be run from the lesson directory? If not then I would personally create all_lessons/makefile with the following contents:

lessons = helloworld even_or_odd calculator

all: $(lessons)

# for each $lesson, the target is $lesson/main built from $lesson/main.cpp and $lesson/lesson.cpp
# NB: the leading space on the second line *must* be a tab character
$(lessons:%=%/main): %/main: %/main.cpp %/lesson.cpp
   g++ -W -Wall $+ -o $@

All lessons could then be built with "make" or "make all" in the all_lessons directory, or a specific lesson with e.g. "make helloworld/main".

As far as I have found, this is the best solution available:

The directory is structured in the same way, but instead of having multiple SConstruct files, the lessons have a SConscript file each, where defaults are overridden as necessary. The SConstruct files are generated by an external script as necessary, and SCons is invoked.

An overview:

all_lessons/
    helloworld/
        SConscript
        lesson.cpp
        main.cpp
    even_or_odd/
        SConscript
        lesson.py
        main.cpp
    calculator/
        SConscript
        lesson.cpp
        main.cpp
        user_created_add.cpp

Using Glob, the SConscript file can make all files with the cpp extension be compiled. It can also use a builder (either one invoking a simple command, or a fully-fledged one) that will generate the lesson, meaning it's possible to even just store the lesson as metadata and have it generated on the spot.

So, to answer the questions:

Yes.
I don't know, but it is not required for the purpose of this.
As far as I have seen, no (issues with paths relative to SConstruct, amongst other things).
Yes, with several options being available.
I don't know.

Downsides to the suggested approach: this does require making a meta-build system separately. The number of files where options can be specified is higher, and the SConscript files give a lot of room for error.

Here is my way.

# SConstruct or SConscript

def getSubdirs(dir) :   
    lst = [ name for name in os.listdir(dir) if os.path.isdir(os.path.join(dir, name)) and name[0] != '.' ]
    return lst

env = Environment()
path_to_lessons = '' # path to lessons
# configure your environment, set common rules and parameters for all lessons
for lesson in getSubdirs(path_to_lessons) :
    lessonEnv = env.Clone()
    # configure specific lesson, for example i'h ve checked SConscript file in lesson dir
    # and if it exist, execute it with lessonEnv and append env specific settings
    if File(os.path.join(path_to_lessons, lesson, 'lesson.scons')).exists() :
        SConscript(os.path.join(lesson, 'lesson.scons', export = ['lessonEnv'])

    # add lesson directory to include path
    lessonEnv.Append(CPPPATH = os.path.join(path_to_lessons, lesson));

    lessonEnv.Program(lesson, Glob(os.path.join(path_to_lessons, lesson, '*.cpp'))

Now you have :

env - core Environment that contain common rules and parameters for all lessons
lessonEnv - clone of core env, but if you have lesson.scons in specific lesson dir, you can additional configure that environment or rewrite some parameters.

来源：https://stackoverflow.com/questions/7123431/building-multiple-executables-with-similar-rules

标签

c++

build

build-automation

scons