Azure Stream Analytics的BadArgument错误

       最近在使用Azure Stream Analytics链接到event hub处理其中的数据时,遇到了下面的BadArgument的错误。该错误发生在Sample Data阶段,在此之前Test检测时正常的。难道是数据格式有错误? 配置的event hub读取格式为JSON UTF-8,经过检查可以排除是由于数据格式造成的这个问题。

Error code:BadArgument\nError message: There was an error while reading sample input.Please check if the input source is configured correctly and data is in correctformat

          反复尝试后发现,这个问题实际上是由于event hub的Consumer Group的问题。默认情况下, event hub创建了名为$default的 Consumer Group,如果在从event hub读取数据是不指明consumer group,那么$default将被用来读取数据。但是,event hub另有一条特殊的规定:

The publish/subscribe mechanism of Event Hubs is enabled throughconsumer groups. A consumer group is a view (state, position, or offset) of an entire Event Hub. Consumer groups enable multiple consuming applications to each have a separate view of the event stream, and to read the stream independently at their own pace and with their own offsets.

In a stream processing architecture, each downstream application equates to a consumer group. If you want to write event data to long-term storage, then that storage writer application is a consumer group. Complex event processing can then be performed by another, separate consumer group. You can only access partitions through a consumer group. Each partition can only have one active reader from a given consumer group at a time. There is always a default consumer group in an Event Hub, and you can create up to 20 consumer groups for a Standard tier Event Hub.

      也就是说,每个partition只能有一个来自于同一个consumer group的应用读取。默认的$default,可能有其它应用在读取,所以再使用它就会产生问题。解决的办法就是创建一个新的consumer group,专门用它来连接Azure Stream Analytics job,这样就不会有问题了。


  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值