Idempotent Consumer

一、Idempotent Consumer

The Idempotent Consumer from the EIP patterns is used to filter out duplicate messages.

This pattern is implemented using the IdempotentConsumer class. This uses an Expression to calculate a unique message ID string for a given message exchange; this ID can then be looked up in the IdempotentRepository to see if it has been seen before; if it has the message is consumed; if its not then the message is processed and the ID is added to the repository.

The Idempotent Consumer essentially acts like a Message Filter to filter out duplicates.

Camel will add the message id eagerly to the repository to detect duplication also for Exchanges currently in progress.
On completion Camel will remove the message id from the repository if the Exchange failed, otherwise it stays there.

Options

The Idempotent Consumer has the following options:

Option	Default	Description
eager	true	Camel 2.0: Eager controls whether Camel adds the message to the repository before or after the exchange has been processed. If enabled before then Camel will be able to detect duplicate messages even when messages are currently in progress. By disabling Camel will only detect duplicates when a message has successfully been processed.
messageIdRepositoryRef	null	A reference to a IdempotentRepository to lookup in the registry. This option is mandatory when using XML DSL.

Using the Fluent Builders

The following example will use the header myMessageId to filter out duplicates

RouteBuilder builder = new






















 RouteBuilder() {











    public










 void configure() {











        errorHandler(deadLetterChannel("mock:error"










));























        from("seda:a"










)   .idempotentConsumer(header("myMessageId"










),











                    MemoryIdempotentRepository.memoryIdempotentRepository(200))











            .to("seda:b"










);











    }











};

The above example will use an in-memory based MessageIdRepository which can easily run out of memory and doesn't work in a clustered environment. So you might prefer to use the JPA based implementation which uses a database to store the message IDs which have been processed

from("direct:start"






















).idempotentConsumer(























        header("messageId"






















),























        jpaMessageIdRepository(lookup(JpaTemplate.class), PROCESSOR_NAME)























).to("mock:result"






















);

In the above example we are using the header messageId to filter out duplicates and using the collection myProcessorName to indicate the Message ID Repository to use. This name is important as you could process the same message by many different processors; so each may require its own logical Message ID Repository.

For further examples of this pattern in use you could look at the junit test case

Spring XML example

The following example will use the header myMessageId to filter out duplicates

<camelContext xmlns="http://camel.apache.org/schema/spring"










 >






















    <route>






















        <from uri="direct:start"










 />






















        <idempotentConsumer messageIdRepositoryRef="myRepo"










 >






















            <!-- use the messageId header as key for identifying duplicate messages -->

































            <header>






















messageId</header>






















            <!-- if not a duplicate send it to this mock endpoint -->

































            <to uri="mock:result"










 />






















        </idempotentConsumer>






















    </route>






















</camelContext>

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started , you may also find the Architecture useful particularly the description of Endpoint and URIs . Then you could try out some of the Examples first before trying this pattern out.

 

-------------------------------------------------------------------------------------------------------------------------------------------

 

-------------------------------------------------------------------------------------------------------------------------------------------

 

-------------------------------------------------------------------------------------------------------------------------------------------

 

二、

Contents of /camel/trunk/camel-core/src/test/java/org/apache/camel/processor/IdempotentConsumerTest.java

Parent Directory | Revision Log

---

Revision 927722 - ( show annotations ) ( download )
Fri Mar 26 08:03:00 2010 UTC (10 months ago) by davsclaus
File size: 6938 byte(s)

CAMEL-2576: Renamed redeliverDelay to redeliveryDelay.

1	/**
2	* Licensed to the Apache Software Foundation (ASF) under one or more
3	* contributor license agreements. See the NOTICE file distributed with
4	* this work for additional information regarding copyright ownership.
5	* The ASF licenses this file to You under the Apache License, Version 2.0
6	* (the "License"); you may not use this file except in compliance with
7	* the License. You may obtain a copy of the License at
8	*
9	* http://www.apache.org/licenses/LICENSE-2.0
10	*
11	* Unless required by applicable law or agreed to in writing, software
12	* distributed under the License is distributed on an "AS IS" BASIS,
13	* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14	* See the License for the specific language governing permissions and
15	* limitations under the License.
16	*/
17	package org.apache.camel.processor;
18
19	import org.apache.camel.ContextTestSupport;
20	import org.apache.camel.Endpoint;
21	import org.apache.camel.Exchange;
22	import org.apache.camel.Message;
23	import org.apache.camel.Processor;
24	import org.apache.camel.builder.RouteBuilder;
25	import org.apache.camel.component.mock.MockEndpoint;
26	import org.apache.camel.processor.idempotent.MemoryIdempotentRepository;
27
28	/**
29	* @version $Revision$
30	*/
31	public class IdempotentConsumerTest extends ContextTestSupport {
32	protected Endpoint startEndpoint;
33	protected MockEndpoint resultEndpoint;
34
35	@Override
36	public boolean isUseRouteBuilder() {
37	return false;
38	}
39
40	public void testDuplicateMessagesAreFilteredOut() throws Exception {
41	context.addRoutes(new RouteBuilder() {
42	@Override
43	public void configure() throws Exception {
44	from("direct:start").idempotentConsumer(
45	header("messageId"), MemoryIdempotentRepository.memoryIdempotentRepository(200)
46	).to("mock:result");
47	}
48	});
49	context.start();
50
51	resultEndpoint.expectedBodiesReceived("one", "two", "three");
52
53	sendMessage("1", "one");
54	sendMessage("2", "two");
55	sendMessage("1", "one");
56	sendMessage("2", "two");
57	sendMessage("1", "one");
58	sendMessage("3", "three");
59
60	assertMockEndpointsSatisfied();
61	}
62
63	public void testFailedExchangesNotAddedDeadLetterChannel() throws Exception {
64	context.addRoutes(new RouteBuilder() {
65	@Override
66	public void configure() throws Exception {
67	errorHandler(deadLetterChannel("mock:error").maximumRedeliveries(2).redeliveryDelay(0).logStackTrace(false));
68
69	from("direct:start").idempotentConsumer(
70	header("messageId"), MemoryIdempotentRepository.memoryIdempotentRepository(200)
71	).process(new Processor() {
72	public void process(Exchange exchange) throws Exception {
73	String id = exchange.getIn().getHeader("messageId", String.class);
74	if (id.equals("2")) {
75	throw new IllegalArgumentException("Damm I cannot handle id 2");
76	}
77	}
78	}).to("mock:result");
79	}
80	});
81	context.start();
82
83	// we send in 2 messages with id 2 that fails
84	getMockEndpoint("mock:error").expectedMessageCount(2);
85	resultEndpoint.expectedBodiesReceived("one", "three");
86
87	sendMessage("1", "one");
88	sendMessage("2", "two");
89	sendMessage("1", "one");
90	sendMessage("2", "two");
91	sendMessage("1", "one");
92	sendMessage("3", "three");
93
94	assertMockEndpointsSatisfied();
95	}
96
97	public void testFailedExchangesNotAddedDeadLetterChannelNotHandled() throws Exception {
98	context.addRoutes(new RouteBuilder() {
99	@Override
100	public void configure() throws Exception {
101	errorHandler(deadLetterChannel("mock:error").handled(false).maximumRedeliveries(2).redeliveryDelay(0).logStackTrace(false));
102
103	from("direct:start").idempotentConsumer(
104	header("messageId"), MemoryIdempotentRepository.memoryIdempotentRepository(200)
105	).process(new Processor() {
106	public void process(Exchange exchange) throws Exception {
107	String id = exchange.getIn().getHeader("messageId", String.class);
108	if (id.equals("2")) {
109	throw new IllegalArgumentException("Damm I cannot handle id 2");
110	}
111	}
112	}).to("mock:result");
113	}
114	});
115	context.start();
116
117	// we send in 2 messages with id 2 that fails
118	getMockEndpoint("mock:error").expectedMessageCount(2);
119	resultEndpoint.expectedBodiesReceived("one", "three");
120
121	sendMessage("1", "one");
122	sendMessage("2", "two");
123	sendMessage("1", "one");
124	sendMessage("2", "two");
125	sendMessage("1", "one");
126	sendMessage("3", "three");
127
128	assertMockEndpointsSatisfied();
129	}
130
131	public void testFailedExchangesNotAdded() throws Exception {
132	context.addRoutes(new RouteBuilder() {
133	@Override
134	public void configure() throws Exception {
135	// use default error handler
136	errorHandler(defaultErrorHandler());
137
138	from("direct:start").idempotentConsumer(
139	header("messageId"), MemoryIdempotentRepository.memoryIdempotentRepository(200)
140	).process(new Processor() {
141	public void process(Exchange exchange) throws Exception {
142	String id = exchange.getIn().getHeader("messageId", String.class);
143	if (id.equals("2")) {
144	throw new IllegalArgumentException("Damm I cannot handle id 2");
145	}
146	}
147	}).to("mock:result");
148	}
149	});
150	context.start();
151
152	resultEndpoint.expectedBodiesReceived("one", "three");
153
154	sendMessage("1", "one");
155	sendMessage("2", "two");
156	sendMessage("1", "one");
157	sendMessage("2", "two");
158	sendMessage("1", "one");
159	sendMessage("3", "three");
160
161	assertMockEndpointsSatisfied();
162	}
163
164	protected void sendMessage(final Object messageId, final Object body) {
165	template.send(startEndpoint, new Processor() {
166	public void process(Exchange exchange) {
167	// now lets fire in a message
168	Message in = exchange.getIn();
169	in.setBody(body);
170	in.setHeader("messageId", messageId);
171	}
172	});
173	}
174
175	@Override
176	protected void setUp() throws Exception {
177	super.setUp();
178
179	startEndpoint = resolveMandatoryEndpoint("direct:start");
180	resultEndpoint = getMockEndpoint("mock:result");
181	}
182
183	}

二、

org.apache.camel.processor.idempotent
Class IdempotentConsumer

java.lang.Object










  




org.apache.camel.impl.ServiceSupport










      




org.apache.camel.processor.idempotent.IdempotentConsumer

All Implemented Interfaces:

Navigate < Processor >, Processor , Service

---

public class IdempotentConsumer










   
   
    
    extends 
    
    ServiceSupport

    
    


    
    


   
   





   
   
    
    implements 
    
    Processor

    
    


    
    

, 
    
    Navigate

    
    


    
    

<
    
    Processor

    
    


    
    

>
   
   

An implementation of the Idempotent Consumer pattern.

 

 

Version:

$Revision: 835732 $

---

Constructor Summary
IdempotentConsumer (Expression  messageIdExpression, IdempotentRepository <String > idempotentRepository, boolean eager, Processor  processor)

 

Method Summary
protected  void	doStart ()
protected  void	doStop ()
IdempotentRepository <String >	getIdempotentRepository ()
Expression	getMessageIdExpression ()
Processor	getProcessor ()
boolean	hasNext ()           Are there more outputs?
List <Processor >	next ()           Next group of outputs
protected  void	onDuplicateMessage (Exchange  exchange, String  messageId)           A strategy method to allow derived classes to overload the behaviour of processing a duplicate message
void	process (Exchange  exchange)           Processes the message exchange
String	toString ()

 

Methods inherited from class org.apache.camel.impl.ServiceSupport
addChildService , getStatus , getVersion , isRunAllowed , isStarted , isStarting , isStopped , isStopping , removeChildService , start , stop

 

Methods inherited from class java.lang.Object
clone , equals , finalize , getClass , hashCode , notify , notifyAll , wait , wait , wait

 

Constructor Detail

IdempotentConsumer

public IdempotentConsumer









(Expression




 messageIdExpression,





                          IdempotentRepository




<String




> idempotentRepository,





                          boolean eager,





                          Processor




 processor)

Method Detail

toString

public String




 toString









()

Overrides:

toString in class Object

---

process

public void process









(Exchange




 exchange)





             throws Exception

Description copied from interface: Processor

Processes the message exchange

 

Specified by:

process in interface Processor

Parameters:

exchange - the message exchange

Throws:

Exception - if an internal processing error has occurred.

---

next

public List




<Processor




> next









()

Description copied from interface: Navigate

Next group of outputs

 

Specified by:

next in interface Navigate <Processor >

Returns:

next group or null if no more outputs

---

hasNext

public boolean hasNext









()

Description copied from interface: Navigate

Are there more outputs?

 

Specified by:

hasNext in interface Navigate <Processor >

Returns:

true if more outputs

---

getMessageIdExpression

public Expression




 getMessageIdExpression









()

---

getIdempotentRepository

public IdempotentRepository




<String




> getIdempotentRepository









()

---

getProcessor

public Processor




 getProcessor









()

---

doStart

protected void doStart









()





                throws Exception

Specified by:

doStart in class ServiceSupport

Throws:

Exception

---

doStop

protected void doStop









()





               throws Exception

Specified by:

doStop in class ServiceSupport

Throws:

Exception

---

onDuplicateMessage

protected void onDuplicateMessage









(Exchange




 exchange,





                                  String




 messageId)

A strategy method to allow derived classes to overload the behaviour of processing a duplicate message

 

Parameters:

exchange - the exchange

messageId - the message ID of this exchange

-------------------------------------------------------------------------------------------------------------------------------------------

 

-------------------------------------------------------------------------------------------------------------------------------------------

 

-------------------------------------------------------------------------------------------------------------------------------------------

 

 

三、rg.apache.camel.spi

Interface IdempotentRepository<E>

All Known Implementing Classes:

FileIdempotentRepository , MemoryIdempotentRepository

---

public interface IdempotentRepository<E>

Access to a repository of Message IDs to implement the Idempotent Consumer pattern.

The add and contains methods is operating according to the Set contract.

 

 

Version:

$Revision: 782534 $

---

Method Summary
boolean	add (E  key)           Adds the key to the repository.
boolean	confirm (E  key)           Confirms the key, after the exchange has been processed sucesfully.
boolean	contains (E  key)           Returns true if this repository contains the specified element.
boolean	remove (E  key)           Removes the key from the repository.

 

Method Detail

add

boolean add





















(E










 key)

Adds the key to the repository.

 

Parameters:

key - the key of the message for duplicate test

Returns:

true if this repository did not already contain the specified element

---

contains

boolean contains





















(E










 key)

Returns true if this repository contains the specified element.

 

Parameters:

key - the key of the message

Returns:

true if this repository contains the specified element

---

remove

boolean remove





















(E










 key)

Removes the key from the repository.

Is usually invoked if the exchange failed.

 

Parameters:

key - the key of the message for duplicate test

Returns:

true if the key was removed

---

 

confirm

boolean confirm





(E


 key)  Confirms the key, after the exchange has been processed sucesfully.



Parameters:


key


- the key of the message for duplicate test



Returns:


true


 if the key was confirmed




 
 



------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------------------------------

四、Fluent Builders

Camel provides fluent builders for creating routing and mediation rules using a type-safe IDE friendly way which provides smart completion and is refactoring safe.