declareOutputFields方法中声明了该bolt/spout输出的字段个数,供下游使用,在该bolt中的execute方法中,emit发射的字段个数必须和声明的相同,否则报错:Tuple created with wrong number of fields. Expected 2 fields but got 1 fields。
测试Output代码如下:
测试Main类如下:
在运行测试的过程中发现一个有趣的现象,在builder.setBolt("3", new Writer(), 8).fieldsGrouping("2", new Fields("ahah","bb"))中的Fields中如果以第二个字段"bb"分组,则只打印出一个对象的哈希值,如果以第一个字段''ahah''或者两个字段则正常打印出多个对象的哈希值。《后续:经leader指导,是因为发射数据量的问题,发射数据改成"word+System.currentTimeMillis()",则按正常设置生成相应的Task对象》
fieldsGrouping方法设定从上一个bolt中获取哪些fields。fieldsGrouping("2", new Fields("ahah","bb")),Fields中的参数名字必须和上一个bolt中声明的字段名字相同,否则报错:“Topology submission exception. (topology name='HelloStorm') #<InvalidTopologyException InvalidTopologyException(msg:Component: [3] subscribes from stream: [default] of component [2] with non-existent fields: #{"ahah1"})>”
测试Spout代码如下:
- public class Produce extends BaseRichSpout {
- private SpoutOutputCollector collector;
- @Override
- public void nextTuple() {
- String[] str1=new String[]{"aaa","bbb","ccc","ddd"};
- String[] str2=new String[]{"ee","ff","gg","kk"};
- Random random=new Random();
- String word1=str1[random.nextInt(str1.length)];
- String word2=str2[random.nextInt(str2.length)];
- collector.emit(new Values(word1,word2));
- try {
- Thread.sleep(100);
- } catch (InterruptedException e) {
- e.printStackTrace();
- }
- }
- @Override
- public void open(Map arg0, TopologyContext arg1, SpoutOutputCollector arg2) {
- this.collector=arg2;
- }
- @Override
- public void declareOutputFields(OutputFieldsDeclarer arg0) {
- arg0.declare(new Fields("ppee","ttt"));
- }
- }
- public class Output extends BaseBasicBolt{
- @Override
- public void execute(Tuple tuple, BasicOutputCollector arg1) {
- String str=tuple.getString(0);
- String str1=tuple.getString(1);
- arg1.emit(new Values(str,str1));
- /*此处发射的字段个数必须要和declareOutputFields中声明的字段个数相同,否则报错:
- * Tuple created with wrong number of fields. Expected 2 fields but got 1 fields
- */
- }
- @Override
- public void declareOutputFields(OutputFieldsDeclarer arg0) {
- arg0.declare(new Fields("ahah","bb"));
- }
- }
测试writer代码如下:
- public class Writer extends BaseBasicBolt{
- @Override
- public void execute(Tuple arg0, BasicOutputCollector arg1) {
- // TODO Auto-generated method stub
- String str=arg0.getString(0);
- String str1=arg0.getString(1);
- System.out.println(str+" "+str1+" "+this);
- }
- @Override
- public void prepare(Map stormConf, TopologyContext context) {
- }
- @Override
- public void declareOutputFields(OutputFieldsDeclarer arg0) {
- arg0.declare(new Fields("hehe"));
- }
- }
- public class main {
- public static void main(String[] args) throws Exception {
- TopologyBuilder builder=new TopologyBuilder();
- builder.setSpout("1", new Produce(),1);
- builder.setBolt("2", new Output(), 4).shuffleGrouping("1");
- builder.setBolt("3", new Writer(), 8).fieldsGrouping("2", new Fields("ahah","bb"));
- /*设定从上一个bolt中获取"ahah"字段,这个“ahah”必须要和上一个bolt中的declare声明的字段相同,
- * 否则报错,“Topology submission exception. (topology name='HelloStorm')
- * #<InvalidTopologyException InvalidTopologyException(msg:Component: [3] subscribes from
- * stream: [default] of component [2] with non-existent fields: #{"ahah1"})>”
- *
- */
- Config con=new Config();
- con.setDebug(false);
- con.setNumWorkers(5);
- // StormSubmitter.submitTopology("first", con, builder.createTopology());
- LocalCluster cluster = new LocalCluster();
- cluster.submitTopology("HelloStorm", con, builder.createTopology());
- }
- }
在运行测试的过程中发现一个有趣的现象,在builder.setBolt("3", new Writer(), 8).fieldsGrouping("2", new Fields("ahah","bb"))中的Fields中如果以第二个字段"bb"分组,则只打印出一个对象的哈希值,如果以第一个字段''ahah''或者两个字段则正常打印出多个对象的哈希值。《后续:经leader指导,是因为发射数据量的问题,发射数据改成"word+System.currentTimeMillis()",则按正常设置生成相应的Task对象》
fieldsGrouping分组的含义是同id的tuple同时只能在一个Task上。