apache-storm

Storm logviewer page not found

巧了我就是萌 提交于 2019-12-04 00:17:56
I'm able to submit a topology job in the multi-tenant cluster. The job is running. However, the logviewer page is not available. Is there any way to solve this issue? you need to start the logviewer before you click on topology port to see logviewer. To start logviewer run: $ storm logviewer same as you run $ storm list I faced the same issue for logviewer's home page, but directly navigating to a particular log file that exists in the logs folder works. Try this: MachineIP:8000/log?file=worker-6700.log 来源: https://stackoverflow.com/questions/25538327/storm-logviewer-page-not-found

Apache Storm : Metrics log file is empty

我只是一个虾纸丫 提交于 2019-12-03 21:51:06
I am trying to follow the example here https://www.endgame.com/blog/storm-metrics-how here is my storm.yaml storm.zookeeper.servers: - localhost supervisor.slots.ports: - 6700 - 6701 - 6702 - 6703 - 6704 nimbus.host: localhost ui.port: 8080 ui.host: localhost storm.log.dir: /path/to/storm/logdir topology.max.spout.pending: 5000 I tried running the topology in local and cluster mode. the metrics.log file is created at the location /path/to/storm/logdir but the file is empty! am i missing some configuration? The problem is with the current log4j2 setup of Metrics in Storm and the fix is a little

Connection refused error in Storm

扶醉桌前 提交于 2019-12-03 17:09:51
I am new in Storm.. I am stucked with below error java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) 1178482 [Thread-11-SendThread(localhost:2000)] WARN org.apache.zookeeper.ClientCnxn - Session 0x1417cd58578000b for server null, unexpected error, closing socket connection and attempting reconnect Sometimes my topology works fine but when i tried again i got above error.Searched a lot in google

How to write logs to a file using Log4j and Storm Framework?

筅森魡賤 提交于 2019-12-03 13:49:51
I am having bit of an issue in logging to a file using log4j in storm . Before submitting my topology , i.e in my main method I wrote some log statements and configured the logger using : PropertyConfigurator.configure(myLog4jProperties) Now when I run my topology using my executable jar in eclipse - its working fine and log files are being created as supposed. OR When i run my executable jar using "java -jar MyJarFile someOtherOptions", i can see log4j being configured and the files are formed correctly and logging is done on both files and console (as defined in my log4j.properties) BUT when

Does storm replay tuple which processing has timed out?

半腔热情 提交于 2019-12-03 13:36:31
问题 It's mentioned in the storm documentation, that storm replays tuple which processing has timed out. My question is if the storm do this automatically (without calling fail() on the origin spout) or is this rather responsibility of the origin spout to replay the tuple (the fail() is called and replay should be implemented inside or even somewhere externally)? 回答1: In order to have a proper replay on a timeout, you must anchor the tuple with an id when you emit it from the spout. When the

How to achieve multi-tenancy in the context of Kafka and storm?

守給你的承諾、 提交于 2019-12-03 11:12:19
What are the best practices, for building a multi-tenant app in the context of Kafka and storm? For example: creating topic for each tenant and consume multi-topics spout (using wildcard). I think that topic for each tenant is the right choice . The naming convention might be something like this: topic_base_name_tenant_id . The reasons are: It allows flexible configuration for each tenant (like @Sebastian mentioned earlier). Clearer logical separation. Now let's say that we will use different approach. For example, partition for each tenant. It might be problematic, since: You are limiting the

How to disable/turn off the logging feature from Storm

笑着哭i 提交于 2019-12-03 09:03:08
问题 I want to turn off the logging feature offered by default when we run from local cluster. Currently its logging so many information on the console. Below is the example of log: 261 [main] INFO backtype.storm.daemon.task - Shut down task Getting-Started-Toplogie-1-1376388324:2 2261 [main] INFO backtype.storm.daemon.task - Shutting down task Getting-Started-Toplogie-1-1376388324:1 2262 [Thread-24] INFO backtype.storm.util - Async loop interrupted! 2276 [main] INFO backtype.storm.daemon.task -

Stopping Storm: the right way

為{幸葍}努か 提交于 2019-12-03 08:51:11
问题 Is there a clean way to stop storm without killing it with "kill XXX" where XXX is the PID? I run "storm kill topology-name" to kill the topology, but after that, is there a clean way to shutdown workers, nimbus, supervisor and ui? I didn't find any command which corresponds to this from the documentation: https://github.com/nathanmarz/storm/wiki/Command-line-client 回答1: Command to kill a topology (as you rightly mentioned): storm kill topology-name To shutdown the cluster you need to run the

Storm max spout pending

半世苍凉 提交于 2019-12-03 08:48:02
问题 This is a question regarding how storm's max spout pending works. I currently have a spout that reads a file and emits a tuple for each line in the file (I know storm is not the best solution for dealing with files but I do not have a choice for this problem). I set the max spout pending to 50k to throttle how many tuples get into the topology to be processed. However, I see this number not having any effect in the topology. I see all records in a file being emitted every time. My guess is

Proper way to ACK in Storm in a chain of bolts

一个人想着一个人 提交于 2019-12-03 08:25:49
问题 Just want to make sure I got how Ack-ing works in Storm. I have 1 spout and 2 bolts chained together. Spout emits tuple to Bolt1 which in turn will emit a tuple to Bolt 2. I want Bolt 2 to ack the initial tuple sent from Spout and I'm not sure how. In order to guarantee fault tolerance (ie: tuples are resent) I want to ack in bolt 2 the tuple emitted by Spout just in case it fails somewhere in the process so it can be resent. Consider this example: Spout: _collector.emit(new Values(queue