declareOutputFields方法中声明了该bolt/spout输出的字段个数,供下游使用,在该bolt中的execute方法中,emit发射的字段个数必须和声明的相同,否则报错:Tuple created with wrong number of fields. Expected 2 fields but got 1 fields。
fieldsGrouping方法设定从上一个bolt中获取哪些fields。fieldsGrouping("2", new Fields("ahah","bb")),Fields中的参数名字必须和上一个bolt中声明的字段名字相同,否则报错:“Topology submission exception. (topology name='HelloStorm') #<InvalidTopologyException InvalidTopologyException(msg:Component: [3] subscribes from stream: [default] of component [2] with non-existent fields: #{"ahah1"})>”
测试Spout代码如下:
public class Produce extends BaseRichSpout {
private SpoutOutputCollector collector;
@Override
public void nextTuple() {
String[] str1=new String[]{"aaa","bbb","ccc","ddd"};
String[] str2=new String[]{"ee","ff","gg","kk"};
Random random=new Random();
String word1=str1[random.nextInt(str1.length)];
String word2=str2[random.nextInt(str2.length)];
collector.emit(new Values(word1,word2));
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
@Override
public void open(Map arg0, TopologyContext arg1, SpoutOutputCollector arg2) {
this.collector=arg2;
}
@Override
public void declareOutputFields(OutputFieldsDeclarer arg0) {
arg0.declare(new Fields("ppee","ttt"));
}
}
测试Output代码如下:public class Output extends BaseBasicBolt{
@Override
public void execute(Tuple tuple, BasicOutputCollector arg1) {
String str=tuple.getString(0);
String str1=tuple.getString(1);
arg1.emit(new Values(str,str1));
/*此处发射的字段个数必须要和declareOutputFields中声明的字段个数相同,否则报错:
* Tuple created with wrong number of fields. Expected 2 fields but got 1 fields
*/
}
@Override
public void declareOutputFields(OutputFieldsDeclarer arg0) {
arg0.declare(new Fields("ahah","bb"));
}
}
测试writer代码如下:
public class Writer extends BaseBasicBolt{
@Override
public void execute(Tuple arg0, BasicOutputCollector arg1) {
// TODO Auto-generated method stub
String str=arg0.getString(0);
String str1=arg0.getString(1);
System.out.println(str+" "+str1+" "+this);
}
@Override
public void prepare(Map stormConf, TopologyContext context) {
}
@Override
public void declareOutputFields(OutputFieldsDeclarer arg0) {
arg0.declare(new Fields("hehe"));
}
}
测试Main类如下:
public class main {
public static void main(String[] args) throws Exception {
TopologyBuilder builder=new TopologyBuilder();
builder.setSpout("1", new Produce(),1);
builder.setBolt("2", new Output(), 4).shuffleGrouping("1");
builder.setBolt("3", new Writer(), 8).fieldsGrouping("2", new Fields("ahah","bb"));
/*设定从上一个bolt中获取"ahah"字段,这个“ahah”必须要和上一个bolt中的declare声明的字段相同,
* 否则报错,“Topology submission exception. (topology name='HelloStorm')
* #<InvalidTopologyException InvalidTopologyException(msg:Component: [3] subscribes from
* stream: [default] of component [2] with non-existent fields: #{"ahah1"})>”
*
*/
Config con=new Config();
con.setDebug(false);
con.setNumWorkers(5);
// StormSubmitter.submitTopology("first", con, builder.createTopology());
LocalCluster cluster = new LocalCluster();
cluster.submitTopology("HelloStorm", con, builder.createTopology());
}
}
在运行测试的过程中发现一个有趣的现象,在builder.setBolt("3", new Writer(), 8).fieldsGrouping("2", new Fields("ahah","bb"))中的Fields中如果以第二个字段"bb"分组,则只打印出一个对象的哈希值,如果以第一个字段''ahah''或者两个字段则正常打印出多个对象的哈希值。《后续:经leader指导,是因为发射数据量的问题,发射数据改成"word+System.currentTimeMillis()",则按正常设置生成相应的Task对象》
fieldsGrouping分组的含义是同id的tuple同时只能在一个Task上。