How do I connect to the VMware's pipe and receive debug information from the running OS?

Question

I'm trying to catch debug information from VMware. This might be an easy task because you can redirect all debug information from VMware's OS into the named pipe like is nicely described here. It works fine with WinDbg but I want to create my own application which will do exactly the same. So I've decided to connect to the named pipe provided by VMware and read from it.

I'm able to connect to that named pipe but I'm getting meaningless result when reading from the pipe. I've simplified the real code into the unsafe infinite loop. It's enough to have a memo and button on a form and use the following sample code.

I suppose I'm wrong with the ReadFile part of a code because I'm getting meaningless string results even if I have read maybe appropriate BytesRead count. Another strange thing is that after VMware's OS boots, nothing else I can read from the pipe even if the most of the debug info should come at that moment.

I've mixed this code from this article and this answer.

procedure TForm1.Button1Click(Sender: TObject);
const
  BufferSize = 1024;

var
  Buffer: PChar;
  BytesRead: DWORD;
  BytesInPipe: DWORD;
  BytesToRead: DWORD;
  PipeName: String;
  PipeHandle: THandle;
  SecAttrs: SECURITY_ATTRIBUTES;
  SecDescr: SECURITY_DESCRIPTOR;

begin
  InitializeSecurityDescriptor(@SecDescr, SECURITY_DESCRIPTOR_REVISION);
  SetSecurityDescriptorDacl(@SecDescr, True, nil, False);
  SecAttrs.lpSecurityDescriptor := @SecDescr;
  SecAttrs.nLength := SizeOf(SECURITY_ATTRIBUTES);
  SecAttrs.bInheritHandle := True;

  PipeName := '\\.\pipe\com_1';

  PipeHandle := CreateFile(PChar(PipeName), GENERIC_READ or GENERIC_WRITE, 0, @SecAttrs, OPEN_EXISTING, 0, 0);
  Memo1.Lines.Add('CreateFile; PipeHandle = '+IntToStr(PipeHandle));

  Buffer := AllocMem(BufferSize + 1);

  repeat
    repeat
      PeekNamedPipe(PipeHandle, nil, 0, nil, @BytesInPipe, nil);
      BytesToRead := Min(BufferSize, BytesInPipe);

      if BytesToRead > 0 then
        begin
          BytesRead := 0;
          Memo1.Lines.Add('PeekNamedPipe; BytesInPipe = '+IntToStr(BytesInPipe)+'; BytesToRead = '+IntToStr(BytesToRead));

          if ReadFile(PipeHandle, Buffer[0], BytesToRead, BytesRead, nil) then
            begin
              Buffer[BufferSize] := #0;
              // OemToAnsi(Buffer, Buffer); // without this line I'm getting the following log

              Memo1.Lines.Add('ReadFile; BytesToRead = '+IntToStr(BytesToRead)+'; BytesRead = '+IntToStr(BytesRead));
              Memo1.Lines.Add('ReadFile; Buffer = '+String(Buffer));
            end;
        end;
    until
      (BytesToRead = 0);

    Application.ProcessMessages;
  until
    (Tag = 1);
end;

Here's what I caugth; after those lines (which came at WMware's OS boot) nothing else comes through the pipe. I'm afraid it's such kind of a protocol.

CreateFile; PipeHandle = 240
PeekNamedPipe; BytesInPipe = 1; BytesToRead = 1
ReadFile; BytesToRead = 1; BytesRead = 1
ReadFile; Buffer = 0
PeekNamedPipe; BytesInPipe = 16; BytesToRead = 16
ReadFile; BytesToRead = 16; BytesRead = 16
ReadFile; Buffer = 000
PeekNamedPipe; BytesInPipe = 169; BytesToRead = 169
ReadFile; BytesToRead = 169; BytesRead = 169
ReadFile; Buffer = 0
PeekNamedPipe; BytesInPipe = 74; BytesToRead = 74
ReadFile; BytesToRead = 74; BytesRead = 74
ReadFile; Buffer = 
PeekNamedPipe; BytesInPipe = 28; BytesToRead = 28
ReadFile; BytesToRead = 28; BytesRead = 28
ReadFile; Buffer = DOWS\system32\ntkrnlpa.exe
PeekNamedPipe; BytesInPipe = 1; BytesToRead = 1
ReadFile; BytesToRead = 1; BytesRead = 1
ReadFile; Buffer = 0OWS\system32\ntkrnlpa.exe
PeekNamedPipe; BytesInPipe = 261; BytesToRead = 261
ReadFile; BytesToRead = 261; BytesRead = 261
ReadFile; Buffer = 000
PeekNamedPipe; BytesInPipe = 26; BytesToRead = 26
ReadFile; BytesToRead = 26; BytesRead = 26
ReadFile; Buffer = WS\system32\ntkrnlpa.exe
PeekNamedPipe; BytesInPipe = 288; BytesToRead = 288
ReadFile; BytesToRead = 288; BytesRead = 288
ReadFile; Buffer = 0000
PeekNamedPipe; BytesInPipe = 1; BytesToRead = 1
ReadFile; BytesToRead = 1; BytesRead = 1
ReadFile; Buffer = 0000
PeekNamedPipe; BytesInPipe = 218; BytesToRead = 218
ReadFile; BytesToRead = 218; BytesRead = 218
ReadFile; Buffer = 000
PeekNamedPipe; BytesInPipe = 69; BytesToRead = 69
ReadFile; BytesToRead = 69; BytesRead = 69
ReadFile; Buffer = 
PeekNamedPipe; BytesInPipe = 1; BytesToRead = 1
ReadFile; BytesToRead = 1; BytesRead = 1
ReadFile; Buffer = 0Ì]Â
PeekNamedPipe; BytesInPipe = 287; BytesToRead = 287
ReadFile; BytesToRead = 287; BytesRead = 287
ReadFile; Buffer = 000

I'm trying to do it with non Unicode Delphi 2007 on Windows 7 with Administrator rights with UAC turned off. Maybe I'm totally wrong; maybe I need to communicate with the pipe, not only read from it. Can anyone suggest me what I'm doing wrong ? Does anyone has an experience with it ?

Thanks a lot
Regards

Answer 1

This is much more complex than I thought. In case that VMware only connects the simulated serial port to the client's pipe (99.9% ?) I have the honor to the kernel debugger of the virtual OS. What means to implement the protocol described in the following sources. Thanks to this answer as a starting point I realized I will let this stuff for a long winter times.

Edit:
Since I need only to catch OutputDebugString messages on the virtual machine and send them to the real machine I've decided to use pipes and the network connection to the virtual machine.

http://www.cnitblog.com/torch/articles/9429.html
http://www.qqread.com/soft-engineering/e300828.html
http://www.developerfusion.com/article/84367/kernel-and-remote-debuggers/

Answer 2

As Windows is a Unicode system, I guess the bytes you receive are Unicode strings. Try to declare Buffer as PWideChar and see what you get. Alternatively switch to a Unicode capable Delphi version.