sympy autowrap (cython): limit of # of arguments, arguments in array form?

Question

I have the following issue:

I want to use autowrap to generate a compiled version of a sympy matrix, with cells containing sympy expressions. Depending on the specification of my problem, the number of arguments can get very large.

I ran into the following 2 issues:

• The number of arguments that autowrap accepts seems to be limited to 509.

i.e., this works:

import sympy
from sympy.utilities.autowrap import autowrap

x = sympy.symbols("x:509")
exp = sum(x)
cyt = autowrap(exp, backend="cython", args=x) 

and this fails to compile:

x = sympy.symbols("x:510")
exp = sum(x)
cyt = autowrap(exp, backend="cython", args=x) 

The message I get seems not very telling:

[...] (Full output upon request)
Generating code
c:\users\[classified]\appdata\local\temp\tmp2zer8vfe_sympy_compile\wrapper_module_17.c(6293) : fatal error C1001: An internal error has occurred in the compiler.
(compiler file 'f:\dd\vctools\compiler\utc\src\p2\hash.c', line 884)
 To work around this problem, try simplifying or changing the program near the locations listed above.
Please choose the Technical Support command on the Visual C++ 
 Help menu, or open the Technical Support help file for more information
LINK : fatal error LNK1257: code generation failed
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed with exit status 1257

Is there any way around this? I would like to use versions of my program that need ~1000 input variables. (I have no understanding of C/cython. Is this an autowrap limitation, a C limitation ...?)

Partly connected to the above:

• Can one compile functions that accept the arguments as array.

Is there any way to generate code that accepts a numpy array as input? I specifically mean one array for all the arguments, instead of providing the arguments as list. (Similar to lambdify using a DeferredVector). ufuncify supports array input, but as I understand only for broadcasting/vectorizing the function.

I would hope that an array as argument could circumvent the first problem above, which is most pressing for me. Apart from that, I would prefer array input anyways, both because it seems faster (no need to unpack the numpy array I have as input into a list), and also more straightforward and natural.

Does anyone have any suggestions what I can do? Also, could anyone tell me whether f2py has similar limitations? This would also be an option for me if feasible, but I don't have it set up to work currently, and would prefer to know whether it helps at all before investing the time. Thanks!

---

Edit:

I played around a bit with the different candidates for telling autowrap that the input argument will be something in array form, rather than a list of numbers. I'll document my steps here for posterity, and also to increase chances to get some input:

• sympy.DeferredVector

Is what I use with lambdify for the same purpose, so I thought to give it a try. However, warning:

A = sympy.DeferredVector("A")
expression = A[0]+A[1]
cyt = autowrap(expression, backend="cython", args=A)

just completely crashed my OS - last statement started executing, (no feedback), everything got really slow, then no more reactions. (Can only speculate, perhaps it has to do with the fact that A has no shape information, which does not seem to bother lambdify, but might be a problem here. Anyways, seems not the right way to go.)

• All sorts of array-type objects filled with the symbols in the expression to be wrapped.

e.g.

x0 ,x1 = sympy.symbols("x:2")
expression = x0 + x1
cyt = autowrap(expression, backend="cython", args=np.array([x0,x1]))

Still wants unpacked arguments. Replacing the last row by

cyt = autowrap(expression, backend="cython", args=[np.array([x0,x1])])

Gives the message

CodeGenArgumentListError: ("Argument list didn't specify: x0, x1 ", [InputArgument(x0), InputArgument(x1)])

Which is a recurrent theme to this approach: also happens when using a sympy matrix, a tuple, and so on inside the arguments list.

• sympy.IndexedBase

This is actually used in the autowrap examples; however, in a (to me) inintuitive way, using an equation as the expression to be wrapped. Also, the way it is used there seems not really feasible to me: The expression I want to cythonize is a matrix, but its cells are themselves longish expressions, which I cannot obtain via index operations.

The upside is that I got a minimal example to work:

X = sympy.IndexedBase("X",shape=(1,1))
expression = 2*X[0,0]
cyt = autowrap(expression, backend="cython", args=[X])

actually compiles, and the resulting function correctly evaluates - when passed a 2d-np.array.

So this seems the most promising avenue, even though further extensions to this approach I keep trying fail.

For example this

X = sympy.IndexedBase("X",shape=(1,))
expression = 2*X[0]
cyt = autowrap(expression, backend="cython", args=[X])

gets me

[...]\site-packages\sympy\printing\codeprinter.py", line 258, in _get_expression_indices " rhs indices in %s" % expr)
ValueError: lhs indices must match non-dummy rhs indices in 2*X[0]

even though I don't see how it should be different from the working one above.

Same error message when sticking to two dimensions, but increasing the size of X:

X = sympy.IndexedBase("X",shape=(2,2))
expression = 2*X[0,0]+X[0,1]+X[1,0]+X[1,1]
cyt = autowrap(expression, backend="cython", args=[X])

ValueError: lhs indices must match non-dummy rhs indices in 2*X[0, 0] + X[0, 1] + X[1, 0] + X[1, 1]

I tried snooping around the code for autowrap, but I feel a bit lost there...

So I'm still searching for a solution and happy for any input.

Answer 1

Passing the argument as an array seems to work OK

x = sympy.MatrixSymbol('x', 520, 1)

exp = 0
for i in range(x.shape[0]):
    exp += x[i]

cyt = autowrap(exp, backend='cython')

arr = np.random.randn(520, 1)
cyt(arr)
Out[48]: -42.59735861021934

arr.sum()
Out[49]: -42.597358610219345