问题
I need some feedback on C# - cPython integration for scientific application.
The scenario is the following: C# for data acquisition and visualization - CPython for data manipulation using several and changing third part library to implement domain specific tasks.
General use case:
- C# code get realtime data from device
- C# code pass data to cPython code and ask for elaboration
- Cpython code does the magic
- Cpython code pass result back to c# code
- C# visualizes data in a WPF application
Both c# and cPython code (item 1),3) and 5)) have already been optimized for speed. Data to be passed in both direction (item 2) and 4)) are essentially arrays of 10E6 double (no complex data structure involved).
I'd like to find an interfacing solution with good performance (speed in going from 1 to 5) and decoupling capabilities who also minimizes the development on client (c# code) side. Actually both side run on the same machine, but I'd like the solution to scale out.
Solutions I've tried - issues:
- a) C# + porting python algorithm on .NET - too many code to write/rewrite on domain side, lack of specialized library cPython provide for almost everything on scientific side
- b) C# embedding ironpython + wrapping/porting c++ extension on managed code (with and without ironclad) - borderline performance caused by the marshaling of structures used during algorithms execution - dll hell when upgrading version of subcomponents.
- c) C# embedding ironpython + rpc between ironpython and python - issues in maintainability, not satisfied by having to "pass" data between the three C#/ironpython/python .
- d) XML-RPC, JSON-RPC - not satisfied by performance (speed).
Solutions not already tried which I plan to evaluate:
- e) Apache Thrift
- f) ZeroRPC
Thanks in advance for any criticism, comments and suggestions.
回答1:
I have a similar requirement (C#<->Python RPC) I've just been experimenting with Apache Thrift and I've very impressed. It's very fast: 50% slower than Pyro4/pickle, 5x faster than RPyC. This is for sending numpy arrays around as binary data. The remaining overhead with Thrift is likely the copying of the array data into/out of strings. If Thrift could be refactored to take buffer objects in place of strings, I would expect it to be as fast as Pyro.
The code-generation seems very clean and the API is nice. The main thing it lacks is documentation. Still, I've been able to figure everything out from the basic tutorial, tests and browsing the source.
As far as I can see ZeroRPC doesn't (yet) have a C# implementation.
来源：https://stackoverflow.com/questions/13141966/fast-and-scalable-rpc-between-c-sharp-and-cpython