Detecting whether or not (and why) Application Insights can send telemetry data to Azure

Question

I'm developing a Windows Desktop application and have successfully linked the Application Insights Core assemblies. I'm using TrackTrace, TrackEvent, etc. to send custom telemetry.

On some workstations telemetry is successfully sent to the Azure portal, and on some others it is not, although calls to TrackTrace, Flush, etc. succeed (or at least return without throwing an exception. TelemetryClient.IsEnabled() returns true. Both workstations use an InMemoryChannel, with the same endpoint https://dc.services.visualstudio.com/v2/track and the sending interval is 30 seconds.

Is there an API function I can call in my application to get the connection status of the TelemetryClient? Something that would tell me that the client is successfully connected, or that it encountered error x while trying, and still has y telemetry packets waiting to be sent.

I'm not looking for a checklist like reinstall the NuGet package (I did...), Make sure your firewall allows traffic to port xxx (it does...) or try to install kb...871 (I did that too...). What I'd like is a status report that I can log somewhere on the client workstation while my application runs, at least acknowledging in the status bar (Yes I know status bars are so old fashioned these days) that there is a problem.

First update - Getting the queue size

First victory, I was able to get the queue size. I wanted to do that without creating my own channel implementation (yet). This however is of little help to detect outages, because the queue will drain even if the transmitter is unable to transmit the telemetry items (it will just discard them) - more on this later. At least you know the transmitter thread is running...

private ITelemetryChannel _TelemetryChannel;
private InMemoryChannel _InMemoryChannel;
private object _TelemetryBuffer;
private object _BufferLock;
private object _InMemoryTransmitter;

_TelemetryChannel = TelemetryConfiguration.Active?.TelemetryChannel;
if (_TelemetryChannel != null && _TelemetryChannel is InMemoryChannel)
{
  _InMemoryChannel = (InMemoryChannel)_TelemetryChannel;
  _TelemetryBuffer = GetInstanceField (_InMemoryChannel, "buffer");
  _BufferLock = GetInstanceField (_TelemetryBuffer, "lockObj");
  _InMemoryTransmitter = GetInstanceField (_InMemoryChannel, "transmitter");
}

public int GetTelemetryQueueSize ()
{
    if (_BufferLock != null)
    {
        lock (_BufferLock)
        {
            object l = GetInstanceField (_TelemetryBuffer, "items");
            if (l is List<ITelemetry>)
            {
                return ((List<ITelemetry>)l).Count;
            }
        }
    }
    return -1;
}

You also need an utility function to use reflection to access the private fields of the objects (the buffer and transmitter are internal sealed...) I made these as error resistant as possible, they could be more concise.

private static object GetInstanceField (Type type, object instance, string fieldName)
{
    if (instance == null)
    {
        return null;
    }
    try
    {
        BindingFlags bindFlags = BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static;
        FieldInfo field = type.GetField (fieldName, bindFlags);
        return field.GetValue (instance);
    }
    catch
    {
        return null;
    }
}

private static object GetInstanceField (object instance, string fieldName)
{
    if (instance == null)
    {
        return null;
    }
    return GetInstanceField (instance.GetType (), instance, fieldName);
}

Answer 1

OK, I managed to make it work... There is no API to do it, so I created a new custom Telemetry Channel that actually reports the errors, and that I can rely on.

Edit: it appears from comments that ReliableTelemetryChannel is not an appropriate name for this class. It should be named ProbingTelemetryChannel. Thanks.

using System;
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.ApplicationInsights.Extensibility.Implementation;

namespace Streambolics.Telemetry
{
  public class ReliableTelemetryChannel : ITelemetryChannel
  {
    private Uri _EndpointAddress;
    private int _Attempts, _Successes, _Failures;
    private Exception _LastFailure;
    private DateTime _LastFailureTime;

    public ReliableTelemetryChannel ()
    {
      EndpointAddress = TelemetryConfiguration.Active?.TelemetryChannel?.EndpointAddress;
    }

    public bool? DeveloperMode {
        get { return true; }
        set { }
    }

    public string EndpointAddress {
        get { return _EndpointAddress?.ToString (); }
        set {
            if (String.IsNullOrEmpty (value))
                _EndpointAddress = null;
            else
                _EndpointAddress = new Uri (value);
        }
    }

    public void Flush () { }

    public void Send (ITelemetry item)
    {
      _Attempts++;

      try
      {
        item.Timestamp = DateTime.Now;
        byte[] data = JsonSerializer.Serialize (new ITelemetry[] { item });
        var transmission = new Transmission (_EndpointAddress, data, "application/x-json-stream", JsonSerializer.CompressionType);
        transmission.SendAsync ().GetAwaiter ().GetResult ();
        _Successes++;
      }
      catch (Exception ex)
      {
          _Failures++;
          _LastFailure = ex;
          _LastFailureTime = DateTime.Now;
      }
    }

    protected virtual void Dispose (bool disposing) { }
    public void Dispose ()
    { Dispose (true); }
  }
}

Now it's just a matter of creating the channel and a client on that channel:

var _ReliableChannel = new ReliableTelemetryChannel ();
var _ReliableConfiguration = new TelemetryConfiguration ();
_ReliableConfiguration.TelemetryChannel = _ReliableChannel;
var _ReliableClient = new TelemetryClient (_ReliableConfiguration);
_ReliableClient.InstrumentationKey = "...";

Now I just regularly send a probe, and query the channel for error statistics:

_ReliableClient.TrackEvent ("TelemetryProbe");
GlobalLog.Log ("Probe attempts {0} Last Error {1}", _ReliableChannel.Attempts, _ReliableChannel.LastFailure);

It doesn't solve the global problem of detecting whether the active configuration works (the one that I use to send regular telemetry, with buffering etc.), but at least I can safely assume that if my Reliable channel is working, the regular one is working too.

Answer 2

Note: the persistence channel was discontinued shortly after I wrote this.

You got it to work, but all you did was creating a Persistence Channel that already exist.

You should've done the following:

  using Microsoft.ApplicationInsights.Channel;
  using Microsoft.ApplicationInsights.Extensibility;
  ...

  // Set up 
  TelemetryConfiguration.Active.InstrumentationKey = "YOUR INSTRUMENTATION KEY";

  TelemetryConfiguration.Active.TelemetryChannel = new PersistenceChannel();

On Application Exit, make sure you call the

telemetryClient.Flush()

to flush the telemetry data to the channel.

What this does is write your telemetry data to disk and periodically flush the data to the Azure cloud. This is great for intermittent internet connectivity and also for cases where the app closes before all data could be sent.

On startup, you can add a tiny sleep delay to ensure that the data from a previous session is sent to the cloud.

The code samples has been taken from the Application Insights Documentation where more detail can be found.