用途
本文演示如何通过kettle向rocketmq发送消息。
技术
- kettle5.4
- rocketmq-4.6.0
整体流程
生成记录
用于模拟业务数据:
增加序列
用于模拟主键:
Java代码
- 用途:
拼接业务数据,生成消息体,本例中仅拼接:id+name;
k1为输出消息发送的状态。 - 截图:
- 源码:
import java.io.UnsupportedEncodingException;
import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.common.RemotingHelper;
import org.apache.rocketmq.remoting.exception.RemotingException;
// Instantiate with a producer group name.
private DefaultMQProducer producer=null;
public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException
{
try {
if (first){
first = false;
producer = new DefaultMQProducer("g1");
// Specify name server addresses.
producer.setNamesrvAddr("localhost:9876");
// Launch the instance.
producer.start();
logBasic(">>>>>成功连接到RocketMQ!");
}
Object[] r = getRow();
if (r == null) {
// Shut down once the producer instance is not longer in use.
producer.shutdown();
logBasic(">>>>>成功关闭RocketMQ连接!");
setOutputDone();
return false;
}
//----------------------------------------
String name = get(Fields.In, "name").getString(r);
Long id = get(Fields.In, "id").getInteger(r);
// Create a message instance, specifying topic, tag and message body.
Message msg = new Message("TopicTest" /* Topic */, "TagA" /* Tag */,
(id+":"+name).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
);
logBasic(">>>>>开始发送数据!");
// Call send message to deliver message to one of brokers.
SendResult sendResult = producer.send(msg);
String v=sendResult.getSendStatus().toString();
logBasic("send result:"+ v);
//----------------------------------------
r = createOutputRow(r, data.outputRowMeta.size());
// Set a value in a new output field
get(Fields.Out, "k1").setValue(r, v);
// Send the row on to the next step.
putRow(data.outputRowMeta, r);
} catch (UnsupportedEncodingException e) {
logError("UnsupportedEncodingException", e);
} catch (MQClientException e) {
logError("MQClientException", e);
} catch (RemotingException e) {
logError("RemotingException", e);
} catch (MQBrokerException e) {
logError("MQBrokerException", e);
} catch (InterruptedException e) {
logError("InterruptedException", e);
}
return true;
}
写日志
用于调试与验证数据,默认留空即可输出全部变量消息:
消息发送端执行日志
消息消费端接收示例
消费端源码
import java.io.UnsupportedEncodingException;
import java.util.List;
import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.message.MessageExt;
import org.apache.rocketmq.remoting.common.RemotingHelper;
public class Consumer {
public static void main(String[] args) throws InterruptedException, MQClientException {
// Instantiate with specified consumer group name.
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("please_rename_unique_group_name");
// Specify name server addresses.
consumer.setNamesrvAddr("localhost:9876");
// Subscribe one more more topics to consume.
consumer.subscribe("TopicTest", "*");
// Register callback to execute on arrival of messages fetched from brokers.
consumer.registerMessageListener(new MessageListenerConcurrently() {
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), msgs);
for(MessageExt me:msgs) {
try {
String msg=new String(me.getBody(),RemotingHelper.DEFAULT_CHARSET);
System.out.println("recive msg:"+msg);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
// Launch the consumer instance.
consumer.start();
System.out.printf("Consumer Started.%n");
}
}
kettle依赖jar包
commons-validator-1.6.jar
fastjson-1.2.61.jar
netty-all-4.0.42.Final.jar
netty-tcnative-boringssl-static-1.1.33.Fork26.jar
rocketmq-client-4.6.0.jar
rocketmq-common-4.6.0.jar
rocketmq-logging-4.6.0.jar
rocketmq-remoting-4.6.0.jar
未完待续
本文已实现基于rocketmq的消息发送功能,后期计划将消息的接收功能补上。