Cannot marshal 'return value': Invalid managed/unmanaged type combination

问题

I have this function in an unmanaged library and I want to call it in C#:

unsigned char __stdcall InitDev(unsigned char comport, long BaudRate)

and this is in C#

[System.Security.SuppressUnmanagedCodeSecurity]
[DllImport("*.dll", CallingConvention = CallingConvention.Cdecl,CharSet =CharSet.Auto)]
public static extern byte[] InitDev(byte[] comport, [Out]long BaudRate);

but when I call this function in C# I get the following error

"Cannot marshal 'return value': Invalid managed/unmanaged type combination."

string COM = "COM3";
byte[] ptr = new byte[1];
try
{
    ptr = InitDev(Encoding.ASCII.GetBytes(COM), 9600);
}
catch (Exception ex)
{
    MessageBox.Show(ex.Message);
}

I am new to this if a little guidance is provided I will be able to resolve this issue

回答1:

Assuming your native call is something like unsigned char* InitDef(unsigned char*, long long &baudRate) then you probably want to use IntPtr in the place of your unsigned char*. But first since it's managed/unmanaged mixing you should allocate that byte[] buffer for input and output :

byte[] pInBuffer = Encoding.ASCII.GetBytes("COM3");
// allocate unmanaged memory
IntPtr inputBuffer = Marshal.AllocHGlobal(pInBuffer.Length * sizeof(byte));
Marshal.Copy(pInBuffer, 0, inputBuffer, pInBuffer.Length);

// now your inputBuffer contains a native pointer to the `unsigned char*`
// and since your function returns a new pointer to some `unsigned char*`
// just retrieve it to IntPtr
IntPtr result = InitDev(inputBuffer, 9600);

// free your allocated memory
Marshal.FreeHGlobal(inputBuffer);

At this stage your result contains the value returned from InitDev.

new definition of InitDev function

[System.Security.SuppressUnmanagedCodeSecurity]
[DllImport("*.dll", CallingConvention = CallingConvention.Cdecl,CharSet =CharSet.Auto)]
public static extern IntPtr InitDev(IntPtr comport, [Out] long BaudRate);

All of the necessary calls, you can find on this msdn page

EDIT :
Since your native call looks like this :extern unsigned char __stdcall InitDev(unsigned char comport,long BaudRate);
I'm guessing that the first parameter is just the last digit ( index ) of your "COMn" because unsigned char's lowest possible value is 0 and COM's lowest index can be 0.

Try using this snippet :

[System.Security.SuppressUnmanagedCodeSecurity]
[DllImport("*.dll", CallingConvention = CallingConvention.Cdecl,CharSet =CharSet.Auto)]
public static extern byte InitDev(byte comport, long BaudRate);

And then call it like so :

byte comIdx = 3;
byte result = InitDev(comIdx, 9600);
// byte should yield correct result now

来源：https://stackoverflow.com/questions/45283745/cannot-marshal-return-value-invalid-managed-unmanaged-type-combination

标签

c++

marshalling