使用apache drill的一些问题

1、在hive中生成的parquet,在drill中用不了,报错

2、在drill中查询太大的(例如13M以上)的json文件,就报错:

WARN util.ResourceLeakDetector: LEAK: ByteBuf was GC'd before being released correctly.  The following stack trace shows where the leaked object was created, rather than where you failed to release it.
io.netty.util.ResourceLeakException: io.netty.buffer.PooledUnsafeDirectByteBufL@1f830e0
        at io.netty.util.ResourceLeakDetector$DefaultResourceLeak.<init>(ResourceLeakDetector.java:174)
        at io.netty.util.ResourceLeakDetector.open(ResourceLeakDetector.java:116)
        at io.netty.buffer.PooledByteBufL.<init>(PooledByteBufL.java:45)
        at io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:53)
        at io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:32)
        at io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:39)
        at io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:36)
        at io.netty.util.Recycler.get(Recycler.java:40)
        at io.netty.buffer.PooledUnsafeDirectByteBufL.newInstance(PooledUnsafeDirectByteBufL.java:44)
        at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:397)
        at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:93)
        at io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:235)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
        at org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
        at org.apache.drill.exec.vector.BitVector.allocateNew(BitVector.java:53)
        at org.apache.drill.exec.vector.NullableIntVector.allocateNew(NullableIntVector.java:119)
        at org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:12)
        at org.apache.drill.exec.store.VectorHolder.allocateNew(VectorHolder.java:87)
        at org.apache.drill.exec.store.VectorHolder.reset(VectorHolder.java:74)
        at org.apache.drill.exec.store.json.JSONRecordReader.resetBatch(JSONRecordReader.java:161)
        at org.apache.drill.exec.store.json.JSONRecordReader.next(JSONRecordReader.java:117)
        at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:95)
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
        at org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)


暂时放弃使用drill了,试试impala去。。。

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值