Something making NServiceBus lose messages

Question

I have an NServiceBus configuration that is working great on developers machines and in my Development Environment.

However, when I move it to my Test Environment my messages just start getting tossed.

Here is the system:

1. An app gets a TCP message from a Mainframe system and sends it to a MSMQ (call it FromMainframe).
2. An application hosted in IIS has a "Handle" method for that MSMQ and processes the messages from the mainframe.

In my Test Environment, step two only half way happens. The message is popped off the MSMQ, but not processed by my application.

Effectively my data is LOST! NServiceBus removes them from the Queue but I never get to process them. They are not even in the error queue!

These are the things I have tried in an attempt to figure out what is happening:

1. Check the Config files
2. Attach a remote debugger to the process to see what the Handle method is doing
   • The Handle method is never called (but when I attach to the Development Environment my breakpoint in my Handle method is hit and it all works flawlessly).
3. Redeploy my Dev version to the Test Envioronment and try step 2 again (just in case the versions were not exactly the same.)
4. Check the Config files again
5. Check that the Error queue is not filling up
   • The error queue stays empty (I wish it would fill up, then my data would not be LOST).
6. Check for any other process that may be pulling stuff from my MSMQs
   • I Turned off my IIS website and the messages in the FromMainframe queue start to backup.
   • When I turn it back on, the messages disappear fairly fast (but still not all at once). The speed that they disappear is too fast for them to be processed by my Handle method.
7. Check Config files yet again.
8. Run the NServiceBusTools\MsmqUtils\Runner.exe \i
   • I ran it, rebooted, ran it again and again for good measure!
9. Check the Configs again (I must have missed SOMETHING right?)
10. Check the Development Environment Configs are not pointing to the Test Environment
    • I don't think it is possible to use another computer's MSMQ as your input queue, but it does not hurt to check.
11. Look for any catch blocks that could be silently killing my message.
12. One last check of the Config files.
13. Recreate my Test Environment on another machine (it worked flawlessly)
14. Run my stuff outside of IIS.
    
    • When I host outside of IIS (using NServiceBus.Host.exe) it all works fine. So it has to be an IIS thing right?
15. Go crazy and hope that stack overflow can offer any kind of insight.

Answer 1

So I know enough about what happened to throw out an "Answer".

When I setup my NServiceBus self hosting I had a call that loaded the message handlers.

NServiceBus.Configure.With().LoadMessageHandlers()

(There are more configurations, but I omitted them for brevity)

When you call this, NServiceBus scans the assmeblies for a class that implements IHandleMessages<T>.

So, somehow, on my Test Environment Machine, the ServiceBus scan of the directory for a class that calls IHandleMessages was failing to find my class (even though the assembly was absolutely there).

Turns out that if NServiceBus does not find something that handles a message it will THROW IT AWAY!!!

This is a total design bug in my opinion. The whole idea of NServiceBus is to not lose your data, but in this case it does just that!

Now, once you know about this pitfall, there are several ways around it.

1. Expressly state what your handler(s) should be:

   NServiceBus.Configure.With().LoadMessageHandlers<First<MyMessageType>>()

2. Even further protection is to add another handler that will handle "Everything else". IMessage is the base for all message payloads, so if you put a handler on it, it will pickup everything.
   If you set IMessage to handle after your messages get handled, then it will handle everything that NServiceBus can't find a handler for. If you throw and exception in that Handle method that will cause NServiceBus to to move the message to the error queue. (What I think should be the default behavior.)