RocketMQ支持SQL92的语法规范。但是采用MessageFilter方式实现时,运行发现无法正常过滤消息。
//DefaultMQPushConsumer的签名方法,采用MessageFilter实现类过滤消息
public void subscribe(String topic, String fullClassName, String filterClassSource) throws MQClientException {
PushConsumer如下:
/**
* java -cp .:lib/* demo.rocketmq.consumer.PushConsumerDemo1 g1<br>
*/
public class PushConsumerDemo1 {
public static void main(String[] args) throws MQClientException, IOException {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer(args[0]);
consumer.setNamesrvAddr("localhost:9876");
//上次消费的位置
consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);
//该种方式已经不再支持了,4.3.0以后只支持MessageSelector.bySql形式过滤,
consumer.subscribe("TopicTest", "demo.rocketmq.consumer.TimestampMessageFilter"
, MixAll.file2String("/usr/local/rocketmq/rocketmq-all-4.4.0/distribution/target/apache-rocketmq/demo-tom/lib/TimestampMessageFilter.java"));
consumer.registerMessageListener(new MessageListenerConcurrently() {
@Override
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), msgs);
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
consumer.start();
Runtime.getRuntime().addShutdownHook(new Thread(() -> consumer.shutdown()));
}
}
MessageFilter实现类:
public class TimestampMessageFilter implements MessageFilter {
@Override
public boolean match(MessageExt msg, FilterContext context) {
Long t = Long.parseLong(msg.getUserProperty("timestamp"));
System.out.println("MessageFilter msg timestamp : " + t);
if (t <= System.currentTimeMillis()) {
return true;
}
return false;
}
}
RocketMQ日志 如下:
/root/logs/rocketmqlogs/rocketmq_client.log.
2019-07-10 09:45:07,007 WARN RocketmqClient - register message class filter failed, because no filter server, ConsumerGroup: g1 Topic: TopicTest ClassName: demo.rocketmq.consumer.TimestampMessageFilter
2019-07-10 09:45:09,009 ERROR RocketmqClient - pullKernelImpl exception
org.apache.rocketmq.client.exception.MQClientException: Find Filter Server Failed, Broker Addr: 172.19.0.1:10911 topic: TopicTest
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
at org.apache.rocketmq.client.impl.consumer.PullAPIWrapper.computPullFromWhichFilterServer(PullAPIWrapper.java:265)
at org.apache.rocketmq.client.impl.consumer.PullAPIWrapper.pullKernelImpl(PullAPIWrapper.java:196)
at org.apache.rocketmq.client.impl.consumer.DefaultMQPushConsumerImpl.pullMessage(DefaultMQPushConsumerImpl.java:418)
at org.apache.rocketmq.client.impl.consumer.PullMessageService.pullMessage(PullMessageService.java:83)
at org.apache.rocketmq.client.impl.consumer.PullMessageService.run(PullMessageService.java:96)
at java.lang.Thread.run(Thread.java:748)
/root/logs/rocketmqlogs/broker.log
2019-07-11 17:27:22 INFO FilterServerManagerScheduledThread1 - CallShell: <sh /usr/local/rocketmq/rocketmq-all-4.4.0/distribution/target/apache-rocketmq/bin/startfsrv.sh -c conf/2m-noslave/broker-a.properties -n localhost:9876> OK
上面的CallShell虽然没有在代码里面报错,但是其实执行并不成功,日志上的信息有点误导,从RocketMQ4.3开始,FiltersrvStartup相关的类已经被移除了。所以无法采用之前的方式启动filter服务。
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Error: Could not find or load main class org.apache.rocketmq.filtersrv.FiltersrvStartup
官网Filter文档给出的样例:sql92
例子中给出的方法是采用MessageSelector形式过滤,当前(>=4.3.0)应该是不再支持MessageFilter形式的过滤了,后续会不会废除或者重新以其他方式实现还有待进一步观察。
附:采用下面的形式过滤是可以的:
/**
* java -cp .:lib/* demo.rocketmq.consumer.PullConsumerDemo1 g1
* 拉模式,客户端主动拉取消息,
*/
public class PullConsumerDemo1 {
public static void main(String[] args) throws InterruptedException, RemotingException, MQClientException, MQBrokerException {
DefaultMQPullConsumer consumer = new DefaultMQPullConsumer(args[0]);
consumer.setNamesrvAddr("localhost:9876");
consumer.start();
Runtime.getRuntime().addShutdownHook(new Thread(() -> consumer.shutdown()));
MessageQueue mq = new MessageQueue("TopicTest", "broker-a", 1);
long offset = 0;
//注意:Pull模式的sql过滤在4.4.0才支持,Push模式的以前就支持
PullResult result = consumer.pull(mq, MessageSelector.bySql("timestamp <= " + System.currentTimeMillis()),
offset, 3);
while (true) {
offset = result.getNextBeginOffset();
System.out.printf("Pull Result: %s, next offset: %s %n", result.getPullStatus(), offset);
if (result.getPullStatus() == PullStatus.FOUND) {
List<MessageExt> list = result.getMsgFoundList();
System.out.printf("Pull Messages: %d ,content: %s %n", list.size(), list);
} else {
System.out.println("Sleep 10s..");
Thread.sleep(10000L);
}
result = consumer.pull(mq, MessageSelector.bySql("timestamp <= " + System.currentTimeMillis()),
offset, 3);
}
}
}
MessageFilter相关:issues452