[异步][jms][activeMq]如何做到重试机制不会导致一条消息被多次执行.

淘宝海量存储之单机事务面临的问题及解决办法

http://blog.csdn.net/jiao_fuyou/article/details/15499261

这篇文章讲的比较好.核心关键词: 幂等.

Message Queue ActiveMQ:  redeliver,如何做到重试机制不会导致一条消息被多次执行.

         http://stackoverflow.com/questions/4934386/avoiding-duplicated-messages-on-jms-activemq

这里面提到了两个. 一个是可集成到Mq的camel的组件  Idempotent Consumer (  Idempotent Receiver  http://camel.apache.org/idempotent-consumer.html  内含一个概念  Idempotent Receiver   大致看了下,有 jpa. 老实说没看懂如何实现 , 邮件组中也有人问,见下面附件 ),

一个是业务端实现方案(官方文档建议  http://activemq.apache.org/should-i-use-xa.html   ). 

另外一个是XA/jpa分布式事务.( http://activemq.apache.org/should-i-use-xa.html )

     答:     关键词 ①消息不丢失 ,②消息不重复执行  是一个问题的两个方面. 不能顾此失彼.

               1.方案.业务端增加一张表存放,消息是否执行. 每次业务事务commit后,告知server端已经处理该消息.  即使server 因为接受 timeout 重发,导致多个业务机器收到消息.也不会导致重复处理.

              想到一点,不需要去专门判断是 unique sqlException 还是其他的Exception.这个可以直接当作偶然Exception. 让Mq重试.

下次重试的时候遇到

   业务逻辑检查checkMessageExist(), 再丢弃这条消息,不再重试.

  这个方案做到了①消息不丢失 ,②消息不重复执行 依赖消息重复发送.

消息不丢失,消息不重复执行.jpg

              2,一种思路.让server记录. 当返回acknowledge时,进行判断是否已经执行, 如果否,那么执行次数+1, 告知用户commit事务. 如果是就告知用户端,进行事务回滚.

            此方案缺陷, 例如acknowledge接受成功,但是业务端 事务commit失败 (停电,硬盘出错),导致该消息丢失.

这个方案只做到了②消息不重复执行, 没做到①消息不丢失 .某些情况下有可能消息丢失.

 

      

第二个方案里涉及了偶发性故障的问题.

       数据库硬盘坏了,停电了没有持久化成功.和持久化成功了硬盘坏了有区别,前者数据丢失,完全无法恢复,后者也有概率会丢失,但是OP会通过机制尽可能不丢失,如果真的丢失了.那不不是人能控制的.比如未做异地容灾的备份,比如地球毁灭.

       对于前者我们业务端通过"编程自动重试"或者"同步返回错误让使用者重新上传"可以避免前者的消息/数据丢失.  

     编程自动重试对异步框架而言更加重要,上面的activeMq就是通过重试避免消息的丢失.
      后者的数据无法恢复并不是通过业务端的编程能够控制的.需要Op通过某些机制尽可能的保证.

Pasted from: <http://mail-archives.apache.org/mod_mbox/activemq-users/200711.mbox/%3cec6e67fd0711050843i3bdc758ch8b801f046e96a526@mail.gmail.com%3e>

 

See the background here... http://activemq.apache.org/should-i-use-xa.html   Typically you either need to implement your own duplicate message detector, use an idempotent consumer... 幂等消费者.  核心关键 http://activemq.apache.org/camel/idempotent-consumer.html or use XA.   Another approach 方法 I've used with customers is that for JPA stuff that might fail (因为jpa可能失败), use a JPA endpoint in Camel as part of your JMS send (so it really just does a JPA insert under the covers using your JpaTemplate object and using the same transaction), then in the ActiveMQ broker bridge from your entity to a queue.   e.g. in your code send to destination   new CamelDestination("jpa:com.acme.MyEntity");   then in your activemq.xml have a rule...   <route>   <from "jpa:com.acme.MyEntity"/>   <to "activemq:MyQueue"/> </route>   This then means rather than having the XA issue; you can stick to simple JDBC operations and transactions; then get the broker to poll the database and bridge it to a message queue in a separate transaction (which typically will have less risk of creating duplicates).   If you're not using XA you really should have some kinda duplicate detection in there just in case though.     On 05/11/2007, dmd <dmd17@cornell.edu> wrote: > > Hi all, > > I am using JPA (currently with OpenJPA) to persist the execution state of a > process.  I sometimes need to use the same transaction to consume a message, > update the state of the process and persist it to disk, and generate some > message. > > Essentially I have two services which communicate via JMS and both need to > recover from failures and not lose their state or messages in transit. > > Now, I can't yet figure out how to configure the system in order to share > the transactional context...  Furthermore, I don't know if that requires > that both the ActiveMQ broker and the JPA must use the same DB.  (Obviously > there is a requirement that the DB itself be highly available.  We are using > MS SQL Server with a shared storage configuration.) > > Any help would be appreciated, including someone telling me that I'm on > totally the wrong track and I should use some other technologies. > > Cheers, > Dan > > -- > View this message in context: http://www.nabble.com/ActiveMQ-and-JPA-with-global-transactions-tf4752732s2354.html#a13590338 > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > >