Troubleshooting onyx-kafka not writing to a topic. Howto run kafka in docker swarm. Error setting runtime volume size (/dev/shm)?

问题

I'm trying to i) troubleshoot a simple onyx-kafka job not writing to a topic. More details are given here. And you can try it out in this sample project. I think the reason is because there's only one kafka node.

So I tried ii) launching kafka confluentinc/cp-kafka:3.3.1 (with zookeeper confluentinc/cp-zookeeper:3.3.1) running docker (17.09.0-ce, build afdb6d4) in swarm mode. But then I get this error.

Warning: space is running low in /dev/shm (shm) threshold=167,772,160 usable=58,716,160

A) With docker compose, I could just configure the mounted /dev/shm disk to have more capacity with the configuration option shm_size: 1G (see here).

Is there an equivalent for docker swarm? I just need to control the size of the disk volume. But I don’t see such an option in docker compose’s deploy configuration (see here). And the docker service --mount option doesn't seem to solve the problem. Any options here?

B) I don't quite grok the Memory volume mount described here. So don't know if that's a workable option.

C) Thanks to Andrew Mulholland (@itwasntandy) from the devops-engineers Slack channel, I can fake it by passing in --mount type=tmpfs,dst=/dev/shm,tmpfs-size=768000000 as options to a call to docker service create ... (see here).

But the offical documentation says that the tmpfs option is not supported for docker stack deploy. This optiononly applies to docker compose's service configuration. It doesn't apply to docker's swarm mode.

Summary. So is there any way to get that option into my existing docker-compose.yml file, in the swarm configuration?

I’d love to be able to run docker stack deploy --compose-file docker-compose.yml my-app. Details are here. Sample project is here. Ultimately, I want to:

• i) run a docker stack, with a service which has onyx-kafka write to a topic (see here). And for that, I think I need to...
• ii) launch docker (17.09.0-ce, build afdb6d4) in swarm mode, with the images kafka confluentinc/cp-kafka:3.3.1 (with zookeeper confluentinc/cp-zookeeper:3.3.1). But I'm running into this memory error issue.

来源：https://stackoverflow.com/questions/47401805/troubleshooting-onyx-kafka-not-writing-to-a-topic-howto-run-kafka-in-docker-swa