storm-Custom Grouping

Custom Grouping 

实现backtype.storm.grouping.CustomStreamGrouping接口即可完成用户自定义Grouping

例如：单次计数按照第一单词的第一个字母mod task数的余数来分配

package CostumerGroup;

import backtype.storm.grouping.CustomStreamGrouping;
import backtype.storm.task.TopologyContext;
import backtype.storm.tuple.Fields;

import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;

/**
 * Created by hjw on 17/5/26.
 */
public class ModuleGrouping implements CustomStreamGrouping {//,Serializable
    int numTasks = 0;
    private List<Integer>  targetTasks;

    @Override
    public void prepare(TopologyContext topologyContext, Fields fields, List<Integer> targetTasks) {
        numTasks =  targetTasks.size();
        this.targetTasks = targetTasks;
    }
    @Override
    public List<Integer> chooseTasks(List<Object> values) {
        List<Integer> boltIDs = new ArrayList<Integer>();
        if(values.size() >0 ){
            String str = values.get(0).toString();
            if (str.isEmpty())
                boltIDs.add(targetTasks.get(0));
            else
                boltIDs.add(targetTasks.get((int)(str.charAt(0))%numTasks));//根据余数分配
        }
        return boltIDs;
    }
}

应用：

	builder.setBolt("word-normalizer", new WordNormalizer()).
				customGrouping("word-reader", new ModuleGrouping());

其他场景：点击打开链接

业务中遇到一个问题想让用户的uid按照分段的规则grouping到对应的task上面，于是采用uid%k的方法将相同模值的记录在一个task进行业务处理，自己实现了ModStreamingGrouping