Serialize execution of symstore via Powershell or BATCH

Question

We are working to integrate a step into our continuous integration (CI) server (CruiseControl.NET). We want to register the debug symbols *.pdb generated from our build process into a Microsoft Symbol Server. As implemented by Microsoft, a symbol server is a directory structure Visual Studio uses to find the *.pdb debug symbols for C++/C# executables. Microsoft provides a command symstore that takes debug symbols in one directory and populates the central symbol store directory as appropriate.

The trouble is symstore explicitly states it is not safe to run concurrently.

What approaches or strategies can we try to prohibit the concurrent execution of the symstore command via BATCH or Powershell scripts?

We are flexible on approach, but because we run on a Windows platform, BATCH and Powershell are preferred solutions.

Clarification:

For our use-case, symstore needs to be runnable from two different CI servers which will save the symbols on a common network drive.

Resources:

symstore:: http://msdn.microsoft.com/en-us/library/windows/desktop/ms681417(v=vs.85).aspx

Answer 1

You can use a locked file as a simple semaphore to serialize events. When you redirect stdout to a file in a batch file, it establishes an exclusive write lock on that file. No other process can open up the same file for write access. The lock will automatically be released when the process finishes, no matter how it ends (clean exit, CTRL-C, Exception failure, etc)

A batch file can attempt to redirect 9 to a lock file, and if it fails, loop back until success. The symstore command is only run while the lock is in place. A non-standard file handle (stream?) is used so that the lock does not interfere with stdin, stdout, or stderr processing.

So you just need to make sure you never call symstore directly. Instead you always call it via a batch script. Something like the following (serializeSymstore.bat):

@echo off
setlocal

:loop

:: Save stderr definition and redirect stderr to nul
:: to hide possible redirection error when establishing lock.
8>&2 2>nul (

  %= Attempt to establish the lock and restore stderr =%
  9>"\\centralServer\somePath\symstore.lock" 2>&8 (

    %= If got here then lock is established throughout all commands =%
    %= in this set of parentheses.                                  =%

    %= Execute your command =%
    symstore %*

    %= Save the return code =%
    call set "rtnCd=%%errorlevel%%"

    %= The next command is a very fast way to clear the ERRORLEVEL. =%
    %= We don't want symstore failure to trigger a loop.            =%
    (call )
  )

) || (
  %= If entered here then failed to establish lock.                 =%
  %= Wait 1 second and then loop back to retry.                     =%
  %= Replace with PING delay if TIMEOUT not universally available.  =%
  timeout 1 /nobreak >nul
  goto loop
)

:: Exit with appropriate return code
exit /b %rtnCd%

Without comments, it becomes a tiny bit of code

@echo off
setlocal

:loop
8>&2 2>nul (
  9>"\\centralServer\somePath\symstore.lock" 2>&8 (
    symstore %*
    call set "rtnCd=%%errorlevel%%"
    (call )
  )
) || (
  timeout 1 /nobreak >nul
  goto loop
)
exit /b %rtnCd%

I have found this primitive and simple strategy to be extremely effective in many projects. I must confess that I have not tested the lock and release characteristics on remote machines. But I believe it should be reliable as long as all machines are Windows.

The only drawback I am aware of is that there is no FIFO queue. If multiple overlapping requests are received, then it's a random luck of the draw as to which process gets to go next. But the processes will be serialized.

EDIT:
I've read splattered bits' original answer prior to editing. He questions whether file locking is reliable on remote machines. I did some quick Google searches and there does appear to be some issues with file locking on UNC paths. If you run into problems, you may have better luck redirecting to a file on a mapped drive letter instead of directly through a UNC path. This is all theory - I've done no testing. Please be sure to do adequate testing before committing to this solution. Note that PUSHD is a convenient way to temporarily assign a drive letter to a UNC path without knowing what drive letters are available. POPD will unmap the drive assignment.

Answer 2

In order to lock access to the network drive, you'll need a third-party that both your CI services talk to. This third-party would then handle access to the network drive. This third party could be:

• MSMQ
• Row in a database table
• WCF service on symbol server which runs symstore and your CCNet builds talk to it to trigger symstore
• CCNet on the symbol server with a project/job that can be triggered remotely
• A scheduled job on the symbol server that can be triggered remotely from your CI servers

Answer 3

Use a file in the shared directory as semaphore to avoid concurrent executions.

:checkfile
if exist %cidir%\sem.txt goto :wait10secs
echo gotit! >%cidir%\sem.txt
doit
del %cidir%\sem.txt
goto :eof
:wait10secs
ing 192.0.2.2 -n 1 -w 10000 > nul
goto :checkfile

Be prepared for debugging all strange ways your batch can fail and all nasty racing conditions.