Is there a way to get the metrics from a kafka server remotely in a c# program?

问题

My current project in c# requires getting the number of under replicated partitions from a kafka server. I can view this in jconsole under the mbeans section, but I need to get the value in a c# program. I tried using NetMX with this code to make the initial connection.

AppDomain.CurrentDomain.SetPrincipalPolicy(PrincipalPolicy.WindowsPrincipal);
INetMXConnector connector = NetMXConnectorFactory.Connect(new Uri("http://<myserver>:<jmxport>"), null);    
IMBeanServerConnection remoteServer = connector.MBeanServerConnection;

A "Section not found" error is thrown at the second line and I was wondering if anyone could help with this please?

回答1:

I don't think NetMX will connect to a Java JMX endpoint. It's an "independent" implementation specifically for DotNet.

Your best bet is to load the Jolokia java-agent into your target Kafka servers. Jolokia exposes JMX via an HTTP/REST interface which you can access via C#.

You could try IKVM (a java byte code to CLR which will provide you a Java JMX compatible DLL you can invoke against. IKVM is a compiler that converts Java byte code to .NET byte code. The standard JMX remoting works fine from a C# client.

回答2:

In reference to your Jolokia configuration, perhaps you need to fully qualify the path of the jar. Mine looks like this and works:

export JOLOKIA_HOME=/libs/java/jolokia/1.3.7
export JOLOKIA_JAR=$JOLOKIA_HOME/jolokia-jvm-1.3.7-agent.jar
export KAFKA_OPTS="-javaagent:$JOLOKIA_JAR=port=7778,host=* $KAFKA_OPTS"

When I start Kafka in non-daemon mode, it prints this:

I> No access restrictor found, access to any MBean is allowed
Jolokia: Agent started with URL http://10.8.36.121:7778/jolokia/

Then I point my browser to http://localhost:7778/jolokia/search/: and I get:

{
  "request": {
    "mbean": "*:*",
    "type": "search"
  },
  "value": [
    "kafka.network:name=ResponseQueueTimeMs,request=ListGroups,type=RequestMetrics",
    "kafka.server:delayedOperation=topic,name=PurgatorySize,type=DelayedOperationPurgatory",
    "kafka.server:delayedOperation=Fetch,name=NumDelayedOperations,type=DelayedOperationPurgatory",
    "kafka.network:name=RemoteTimeMs,request=Heartbeat,type=RequestMetrics",
<-- SNIP -->
    "kafka.network:name=LocalTimeMs,request=Offsets,type=RequestMetrics"
  ],
  "timestamp": 1504188793,
  "status": 200
}

回答3:

1) Use jolokia jar to convert JMX to Http by adding:

KAFKA_OPTS: javaagent:/usr/share/java/kafka/jolokia-jvm-1.6.0-agent.jar -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost -Dcom.sun.management.jmxremote.rmi.port=9999 -Djava.security.auth.login.config=/var/private/sasl_acl/kafka.server.jaas.config.

2) You will get a http endpoint now you can try a sample get request to check whether it works http://localhost:8778/jolokia/read//java.lang:type=Memory/HeapMemoryUsage

3) Use standard rest api libraries to consume above endpoint.

来源：https://stackoverflow.com/questions/45964498/is-there-a-way-to-get-the-metrics-from-a-kafka-server-remotely-in-a-c-sharp-prog

标签

apache-kafka

jmx