Is it possible to split a SWIG module for compilation, but rejoin it when linking?

问题

I hit this issue about two years ago when I first implemented our SWIG bindings. As soon as we exposed a large amount of code we got to the point where SWIG would output C++ files so large the compiler could not handle them. The only way I could get around the issue was to split up the interfaces into multiple modules and to compile them separately.

This has several downsides:

• Each module must know about dependencies in other modules. I have a script to generate the interface files which handles this side of things, but it adds extra complexity.

• Each additional module increases the time that the dynamic linker requires to load in the code. I have added an init.py file that imports all the submodules, so that the fact that the code is split up is transparent to the user, but what is always visible is the long load times.

I'm currently reviewing our build scripts / build process and I wanted to see if I could find a solution to this issue that was better than what I have now. Ideally, I'd have one shared library containing all the wrapper code.

Does anyone know how I can acheive this with SWIG? I've seen some custom code written in Ruby for a specific project, where the output is post-processed to make this possible, but when I looked at the feasibility for Python wrappers it does not look so easy.

回答1:

I just did equivalent hack for TCL library: I use several SWIG modules, generating several .cpp files that are compiled in several .o files but compile them all in a single .so file that is loaded by a single TCL "load" command.

The idea is to creates a top swig module (Top) that calls initialization functions of all sub-modules (Sub1 and Sub2):

%module Top
%header %{
  extern "C" {
    SWIGEXPORT int Sub1_Init(Tcl_Interp *);
    SWIGEXPORT int Sub2_Init(Tcl_Interp *);
  }
%}
%init %{
    if (Sub1_Init(interp) != TCL_OK) {return TCL_ERROR;}
    if (Sub2_Init(interp) != TCL_OK) {return TCL_ERROR;}
%}

There's nothing special in the submodules files. I end up with file Top.so that I load from TCL with command "load ./Top.so"

I don't know python but's likely to be similar. You may need to understand how the python extensions are loaded, though.

回答2:

If split properly, the modules don't necessarily need to have the same dependencies as the others - just what's necessary to do compilation. If you break things up appropriately, you can have libraries without cyclic dependencies. The issue with using multiple libraries is that by default, SWIG declares its runtime code statically, and as a result, as problems passing objects from one module to another. You need to enable a shared version of the SWIG runtime code.

From the documentation (SWIG web page documentation link is broken):

The runtime functions are private to each SWIG-generated module. That is, the runtime functions are declared with "static" linkage and are visible only to the wrapper functions defined in that module. The only problem with this approach is that when more than one SWIG module is used in the same application, those modules often need to share type information. This is especially true for C++ programs where SWIG must collect and share information about inheritance relationships that cross module boundaries.

Check out that section in your downloaded documentation (section 16.2 The SWIG runtime code), and it'll give you details on how to enable this so that objects can be properly handled when passed from one module to the other.

FWIW, I've not worked with Python SWIG, but have done Tcl SWIG.

来源：https://stackoverflow.com/questions/697749/is-it-possible-to-split-a-swig-module-for-compilation-but-rejoin-it-when-linkin

标签

c++

python

swig