storm ack机制//

1、原理\\
Storm实现一组特殊的'acker’ task来track每一个spout tuple, 同时acker task的个数你可以根据tuple的数量来配置,所有被产生的tuple都会有一个随机的64bit的id用于被track。tuple之间通过emit时的anchor形成tuple tree, 并且每个tuple都知道产生它的spout tuple的id (通过不断的copy传递)\\
Acker需要为每个spout tuple存储spout tuple id, task id, ack val,这个ack val, 64 bit number, 用于表示整个tuple tree的状况, 产生方法是tuple tree中所有created和acked的tuple的id进行异或,当ack val值为0的时候, 即表示tuple tree被完成。\\

2、实现\\

public class AckMain {

	public static void main(String[] args) {

		TopologyBuilder tb=new TopologyBuilder();
		tb.setSpout("1", new MessageSpout(),1);
		tb.setBolt("2", new SplitBolt()).shuffleGrouping("1");
		tb.setBolt("3", new FileWriteBolt()).shuffleGrouping("2");
		Config con=new Config();
		con.setDebug(false);
		LocalCluster local=new LocalCluster();
		local.submitTopology("xx", con, tb.createTopology());
	}
}

public class MessageSpout implements IRichSpout{
	private int index=0;
	private String[]lines=new String []{
			"0,zero",
			"1,one",
			"2,two",
			"3,three",
			"4,four",
			"5,five",
			"6,six",
			"7,seven",
			"8,eight",
			"9,nine"
	};
	private SpoutOutputCollector spoutOutputCollector;

	@Override
	public void ack(Object msId) {

		System.out.println("ack...MessageId........"+msId);
	}

	
	@Override
	public void fail(Object arg0) {

		System.out.println("fail....messageId...."+arg0);
		spoutOutputCollector.emit(new Values(lines[(Integer)arg0]), arg0);
	}

	@Override
	public void nextTuple() {
		if(index < lines.length){
			String l=lines[index];
			spoutOutputCollector.emit(new Values(l), index);
			index++;
		}
	}

	@Override
	public void open(Map arg0, TopologyContext arg1, SpoutOutputCollector spoutOutputCollector) {
		this.spoutOutputCollector=spoutOutputCollector;
		
	}

	@Override
	public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {

		outputFieldsDeclarer.declare(new Fields("line"));
	}


}


 
public class SplitBolt implements IRichBolt{

	OutputCollector outputCollector;
	@Override
	public void execute(Tuple input) {

		String line=input.getString(0);
		String []lines=line.split(",");
		for(String word:lines){
			outputCollector.emit(input, new Values(word)); //把接收的sentence进行切分,emit需要实现anchor,即第一个参数必须是Tuple。
		}
		outputCollector.ack(input);
	}

	@Override
	public void prepare(Map arg0, TopologyContext arg1, OutputCollector outputCollector) {

		this.outputCollector=outputCollector;
	}

	@Override
	public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {
		outputFieldsDeclarer.declare(new Fields("word"));
	}
}
public class FileWriteBolt implements IRichBolt{

	OutputCollector outputCollector;
	private FileWriter fw;
	int count=0;
	@Override
	public void execute(Tuple input) {

		String word=input.getString(0);
		if(count==5)
			outputCollector.fail(input);//当发送第五个单词时触发fail方法。
		else{
			try {
				fw.write(word);
				fw.write("....");
				fw.flush();
			} catch (IOException e) {
				e.printStackTrace();
				
			}
			outputCollector.ack(input);
		}
		count++;
	}

	@Override
	public void prepare(Map map, TopologyContext topologyContext, OutputCollector outputCollector) {
		this.outputCollector=outputCollector;
		try {
			fw=new FileWriter("d://000.txt");
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		
	}

	@Override
	public void declareOutputFields(OutputFieldsDeclarer arg0) {
	}
}





  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值