RabbitMQ

【消息队列】如何处理消息丢失的问题

丶灬走出姿态 提交于 2020-01-16 11:05:36
一、RabbitMQ 1)生产者弄丢了数据   生产者将数据发送到rabbitmq的时候,可能因为网络问题导致数据就在半路给搞丢了。 1.可以选择用rabbitmq提供的事务功能,在生产者发送数据之前开启rabbitmq事务(channel.txSelect),然后发送消息,如果消息没有成功被rabbitmq接收到,那么生产者会收到异常报错,此时就可以回滚事务(channel.txRollback),然后重试发送消息;如果收到了消息,那么可以提交事务(channel.txCommit)。但是问题是,开始rabbitmq事务机制,基本上吞吐量会下来,因为太耗性能。 2.(推荐)可以开启confirm模式,在生产者那里设置开启confirm模式之后,你每次写的消息都会分配一个唯一的id,然后如果写入了rabbitmq中,rabbitmq会给你回传一个ack消息,告诉你说这个消息ok了。如果rabbitmq没能处理这个消息,会回调你一个nack接口,告诉你这个消息接收失败,你可以重试。而且你可以结合这个机制自己在内存里维护每个消息id的状态,如果超过一定时间还没接收到这个消息的回调,那么你可以重发。      事务机制和cnofirm机制最大的不同在于,事务机制是同步的,你提交一个事务之后会阻塞在那儿,但是confirm机制是异步的,你发送个消息之后就可以发送下一个消息

Nodejs: Consume rabbitmq messages in topic exchange

久未见 提交于 2020-01-16 09:08:13
问题 I am trying to use rabbitmq topic exchange . In rabbitmq web management i created a queue called queue1 with this details : After creating queue i used default exchange amq.topic and i binded queue1 to this exchange with anonymous.info routing key: After push some message to this queue1 : Now i want to consume these messages so in order to i wrote this script: var amqp = require('amqplib/callback_api'); amqp.connect(uri, (error0, connection) => { if (error0) { throw error0; } connection

Nodejs: Consume rabbitmq messages in topic exchange

馋奶兔 提交于 2020-01-16 09:08:01
问题 I am trying to use rabbitmq topic exchange . In rabbitmq web management i created a queue called queue1 with this details : After creating queue i used default exchange amq.topic and i binded queue1 to this exchange with anonymous.info routing key: After push some message to this queue1 : Now i want to consume these messages so in order to i wrote this script: var amqp = require('amqplib/callback_api'); amqp.connect(uri, (error0, connection) => { if (error0) { throw error0; } connection

celery --分布式任务队列

一曲冷凌霜 提交于 2020-01-16 08:51:17
celery --分布式任务队列 一、介绍 celery是一个基于python开发的分布式异步消息任务队列,用于处理大量消息,同时为操作提供维护此类系统所需的工具。 它是一个任务队列,专注于实时处理,同时还支持任务调度。如果你的业务场景中需要用到异步任务,就可以考虑使用celery 二、实例场景 1、你想对100台机器执行一条批量命令,可能会花很长时间 ,但你不想让你的程序等着结果返回,而是给你返回 一个任务ID,你过一段时间只需要拿着这个任务id就可以拿到任务执行结果, 在任务执行ing进行时,你可以继续做其它的事情。 2、你想做一个定时任务,比如每天检测一下你们所有客户的资料,如果发现今天 是客户的生日,就给他发个短信祝福 三、优点 1、简单:一但熟悉了celery的工作流程后,配置和使用还是比较简单的 2、高可用:当任务执行失败或执行过程中发生连接中断,celery 会自动尝试重新执行任务 3、快速:一个单进程的celery每分钟可处理上百万个任务 4、灵活:几乎celery的各个组件都可以被扩展及自定制 四、入门 celery 需要一个解决方案来发送和接受消息,通常,这是以称为消息代理的单独服务的形式出现的 有以下几种解决方案,包括: 一:RabbitMQ(消息队列,一种程序之间的通信方式) rabbitmq 功能齐全,稳定,耐用且易于安装。它是生产环境的绝佳选择。

CentOS7下源码包方式安装rabbitmq

。_饼干妹妹 提交于 2020-01-16 08:18:08
1.先安装erlang http://www.cnblogs.com/justphp/p/6093880.html 2.下载rabbitmq rpm包: wget http://www.rabbitmq.com/releases/rabbitmq-server/v3.5.0/rabbitmq-server-3.5.0-1.noarch.rpm 3.安装rabbitmq rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc rpm -ivh rabbitmq-server-3.5.0-1.noarch.rpm 4.启动rabbitmq服务 service rabbitmq-server restart 查看状态 rabbitmqctl status 可能报错: Error: unable to connect to node rabbit@localhost: nodedown rabbit@localhost: connected to epmd (port 4369) on localhost epmd reports: node 'rabbit' not running at all no other nodes on localhost suggestion: start the node

傻瓜式在mac上安装rabbitmq

三世轮回 提交于 2020-01-15 22:32:46
这种安装方式不需要自己手动下载任何东西,包括rabbitmq 一、安装Homebrew Homebrew官网:https://brew.sh/index_zh-cn.html 打开终端,输入上面的命令 ypdeMacBook-Air:~ yp$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" ==> This script will install: /usr/local/bin/brew /usr/local/share/doc/homebrew /usr/local/share/man/man1/brew.1 /usr/local/share/zsh/site-functions/_brew /usr/local/etc/bash_completion.d/brew /usr/local/Homebrew ==> The following existing directories will be made group writable: /usr/local/bin /usr/local/lib ==> The following existing directories will have their owner set to

rabbitmq queues filling up with celery tasks

社会主义新天地 提交于 2020-01-15 18:48:30
问题 I am using Celery to call multiple hardware units by their ip address. Each unit will return a list of values. Application code below # create a list of tasks modbus_calls = [] for site in sites: call = call_plc.apply_async((site.name, site.address), expires=120) # expires after 2 minutes? modbus_calls.append(call) # below checks all tasks are complete (values returned), then move forward out of the while loop ready_list = [False] while not all(ready_list): ready_list = [] for task in modbus

Is it necessary to rebuild RabbitMQ connection each time a message is to be sent

帅比萌擦擦* 提交于 2020-01-15 16:48:32
问题 I have a Spring 3 application that receives messages via a non-RabbitMQ receiver, processes them and forwards via RabbitMQ. Each time a message is to be sent a new RabbitMQ connection is built. This seems a bit wasteful. I am just wondering if this is really necessary or if there is a reason why the connection cannot be held in a Singleton and only built once (for multiple sends). This is the sending method: private void send(String routingKey, String message) throws Exception { String

Why does basicReject does not work with Apache Qpid?

a 夏天 提交于 2020-01-15 09:27:13
问题 I'm using qpid-broker for integration testing my spring-boot-start-amqp application which uses basicGet (autoAck=false), basicAck and basicReject for handling the messages. basicReject (with requeue=false) works fine with my external rabbitmq instance but doesn't work with the qpid-broker . I have tested my code with an external RabbitMQ instance where everything works fine but with the embedded Apache Qpid server the test fails because basicReject is not working properly. Getting the message

Why does basicReject does not work with Apache Qpid?

会有一股神秘感。 提交于 2020-01-15 09:26:27
问题 I'm using qpid-broker for integration testing my spring-boot-start-amqp application which uses basicGet (autoAck=false), basicAck and basicReject for handling the messages. basicReject (with requeue=false) works fine with my external rabbitmq instance but doesn't work with the qpid-broker . I have tested my code with an external RabbitMQ instance where everything works fine but with the embedded Apache Qpid server the test fails because basicReject is not working properly. Getting the message