本文整理匯總了Java中org.apache.arrow.memory.RootAllocator類的典型用法代碼示例。如果您正苦於以下問題：Java RootAllocator類的具體用法？Java RootAllocator怎麽用？Java RootAllocator使用的例子？那麽恭喜您, 這裏精選的類代碼示例或許可以為您提供幫助。
RootAllocator類屬於org.apache.arrow.memory包，在下文中一共展示了RootAllocator類的33個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於我們的係統推薦出更棒的Java代碼示例。
示例1: main
點讚 3
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
public static void main(String []args) {
try (final BufferAllocator bufferAllocator = new RootAllocator(SabotConfig.getMaxDirectMemory())) {
final Path tableDir = new Path(args[0]);
final FileSystem fs = tableDir.getFileSystem(new Configuration());
if (fs.exists(tableDir) && fs.isDirectory(tableDir)) {
Map dictionaryEncodedColumns = createGlobalDictionaries(fs, tableDir, bufferAllocator).getColumnsToDictionaryFiles();
long version = getDictionaryVersion(fs, tableDir);
Path dictionaryRootDir = getDictionaryVersionedRootPath(fs, tableDir, version);
for (ColumnDescriptor columnDescriptor: dictionaryEncodedColumns.keySet()) {
final VectorContainer data = readDictionary(fs, dictionaryRootDir, columnDescriptor, bufferAllocator);
System.out.println("Dictionary for column [" + columnDescriptor.toString() + " size " + data.getRecordCount());
BatchPrinter.printBatch(data);
data.clear();
}
}
} catch (IOException ioe) {
logger.error("Failed ", ioe);
}
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:20，
示例2: testLocalDictionaries
點讚 3
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void testLocalDictionaries() throws IOException {
try (final BufferAllocator bufferAllocator = new RootAllocator(SabotConfig.getMaxDirectMemory())) {
final CodecFactory codecFactory = CodecFactory.createDirectCodecFactory(fs.getConf(), new ParquetDirectByteBufferAllocator(bufferAllocator), 0);
Pair, Set> dictionaries1 =
LocalDictionariesReader.readDictionaries(fs, new Path(tableDirPath, "phonebook1.parquet"), codecFactory);
Pair, Set> dictionaries2 =
LocalDictionariesReader.readDictionaries(fs, new Path(tableDirPath, "phonebook2.parquet"), codecFactory);
Pair, Set> dictionaries3 =
LocalDictionariesReader.readDictionaries(fs, new Path(tableDirPath, "phonebook3.parquet"), codecFactory);
Pair, Set> dictionaries4 =
LocalDictionariesReader.readDictionaries(fs, new Path(partitionDirPath, "phonebook4.parquet"), codecFactory);
assertEquals(2, dictionaries1.getKey().size()); // name and kind have dictionaries
assertEquals(1, dictionaries2.getKey().size());
assertEquals(1, dictionaries3.getKey().size());
assertEquals(1, dictionaries4.getKey().size());
assertEquals(0, dictionaries1.getValue().size());
assertEquals(1, dictionaries2.getValue().size()); // skip name
assertEquals(1, dictionaries3.getValue().size()); // skip name
assertEquals(1, dictionaries4.getValue().size()); // skip name
}
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:25，
示例3: testConvertBitsToBytes
點讚 3
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void testConvertBitsToBytes() {
try (
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
BitVector bits = new BitVector("$bits$", allocator);
UInt1Vector bytes = new UInt1Vector("$bits$", allocator);
) {
int count = 100;
for (int i = 0; i < count ; i++) {
bits.getMutator().setSafe(i, i % 2);
}
bits.getMutator().setValueCount(count);
ArrowBuf oldBuf = bits.getBuffer();
oldBuf.retain();
SerializedField.Builder fieldBuilder = TypeHelper.getMetadataBuilder(bits);
ArrowBuf newBuf = convertBitsToBytes(allocator, fieldBuilder, oldBuf);
TypeHelper.load(bytes, fieldBuilder.build(), newBuf);
for (int i = 0; i < count ; i++) {
assertEquals(i % 2, bytes.getAccessor().get(i));
}
newBuf.release();
}
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:27，
示例4: sortData
點讚 3
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
private int sortData(int[] values){
try(final SplayTreeImpl tree = new SplayTreeImpl(values);
BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
ArrowBuf data = allocator.buffer((values.length + 1) * SplayTree.NODE_SIZE )){
data.setZero(0, data.capacity());
tree.setData(data);
for(int i = 0; i < values.length; i++){
tree.put(i);
}
SplayIterator iter = tree.iterator();
Integer previous = null;
while(iter.hasNext()){
int index = iter.next();
if(previous != null){
assertTrue(previous <= values[index]);
}
}
return tree.getComparisonCount();
}
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:26，
示例5: createKKStoreProvider
點讚 3
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Override
KVStoreProvider createKKStoreProvider() throws Exception {
allocator = new RootAllocator(20 * 1024 * 1024);
pool = new CloseableThreadPool("test-remoteocckvstore");
localFabricService = new FabricServiceImpl("localhost", 45678, true, 300, 2, pool, allocator, 0, Long.MAX_VALUE);
localFabricService.start();
remoteFabricService = new FabricServiceImpl("localhost", 45679, true, 300, 2, pool, allocator, 0, Long.MAX_VALUE);
remoteFabricService.start();
localKVStoreProvider = new LocalKVStoreProvider(DremioTest.CLASSPATH_SCAN_RESULT, DirectProvider.wrap(localFabricService), allocator, "localhost", null, true, true, true, false);
localKVStoreProvider.start();
remoteKVStoreProvider = new RemoteKVStoreProvider(DremioTest.CLASSPATH_SCAN_RESULT, DirectProvider.wrap(remoteFabricService), allocator, "localhost", "localhost", localFabricService.getPort());
remoteKVStoreProvider.start();
return remoteKVStoreProvider;
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:18，
示例6: T_setLong_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setLong_1() throws IOException{
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.LONG , "target" , allocator , parent );
memoryAllocator.setLong( 0 , (long)100 );
memoryAllocator.setLong( 1 , (long)200 );
memoryAllocator.setLong( 5 , (long)255 );
memoryAllocator.setLong( 1000 , (long)10 );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readLong().longValue() , (long)100 );
reader.setPosition( 1 );
assertEquals( reader.readLong().longValue() , (long)200 );
reader.setPosition( 5 );
assertEquals( reader.readLong().longValue() , (long)255 );
for( int i = 6 ; i < 1000 ; i++ ){
reader.setPosition( i );
assertEquals( reader.readLong() , null );
}
reader.setPosition( 1000 );
assertEquals( reader.readLong().longValue() , (long)10 );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:29，
示例7: T_setLong_2
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setLong_2() throws IOException{
IColumn column = new PrimitiveColumn( ColumnType.LONG , "boolean" );
column.add( ColumnType.LONG , new LongObj( (long)100 ) , 0 );
column.add( ColumnType.LONG , new LongObj( (long)200 ) , 1 );
column.add( ColumnType.LONG , new LongObj( (long)255 ) , 5 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new OptimizeLongColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.LONG , "target" , allocator , parent );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readLong().longValue() , (long)100 );
reader.setPosition( 1 );
assertEquals( reader.readLong().longValue() , (long)200 );
reader.setPosition( 5 );
assertEquals( reader.readLong().longValue() , (long)255 );
reader.setPosition( 2 );
assertEquals( reader.readLong() , null );
reader.setPosition( 3 );
assertEquals( reader.readLong() , null );
reader.setPosition( 4 );
assertEquals( reader.readLong() , null );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:37，
示例8: T_setDouble_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setDouble_1() throws IOException{
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.DOUBLE , "target" , allocator , parent );
memoryAllocator.setDouble( 0 , (double)0.1 );
memoryAllocator.setDouble( 1 , (double)0.2 );
memoryAllocator.setDouble( 5 , (double)0.255 );
memoryAllocator.setDouble( 1000 , (double)0.1 );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readDouble().doubleValue() , (double)0,1 );
reader.setPosition( 1 );
assertEquals( reader.readDouble().doubleValue() , (double)0.2 );
reader.setPosition( 5 );
assertEquals( reader.readDouble().doubleValue() , (double)0.255 );
for( int i = 6 ; i < 1000 ; i++ ){
reader.setPosition( i );
assertEquals( reader.readDouble() , null );
}
reader.setPosition( 1000 );
assertEquals( reader.readDouble().doubleValue() , (double)0.1 );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:29，
示例9: T_setDouble_2
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setDouble_2() throws IOException{
IColumn column = new PrimitiveColumn( ColumnType.DOUBLE , "boolean" );
column.add( ColumnType.DOUBLE , new DoubleObj( (double)100 ) , 0 );
column.add( ColumnType.DOUBLE , new DoubleObj( (double)200 ) , 1 );
column.add( ColumnType.DOUBLE , new DoubleObj( (double)255 ) , 5 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new OptimizeDoubleColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.DOUBLE , "target" , allocator , parent );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readDouble().doubleValue() , (double)100 );
reader.setPosition( 1 );
assertEquals( reader.readDouble().doubleValue() , (double)200 );
reader.setPosition( 5 );
assertEquals( reader.readDouble().doubleValue() , (double)255 );
reader.setPosition( 2 );
assertEquals( reader.readDouble() , null );
reader.setPosition( 3 );
assertEquals( reader.readDouble() , null );
reader.setPosition( 4 );
assertEquals( reader.readDouble() , null );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:37，
示例10: T_setBoolean_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setBoolean_1() throws IOException{
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.BOOLEAN , "target" , allocator , parent );
memoryAllocator.setBoolean( 0 , true );
memoryAllocator.setBoolean( 1 , false );
memoryAllocator.setBoolean( 5 , true );
memoryAllocator.setBoolean( 1000 , true );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readBoolean().booleanValue() , true );
reader.setPosition( 1 );
assertEquals( reader.readBoolean().booleanValue() , false );
reader.setPosition( 5 );
assertEquals( reader.readBoolean().booleanValue() , true );
for( int i = 6 ; i < 1000 ; i++ ){
reader.setPosition( i );
assertEquals( reader.readBoolean() , null );
}
reader.setPosition( 1000 );
assertEquals( reader.readBoolean().booleanValue() , true );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:29，
示例11: T_setBoolean_2
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setBoolean_2() throws IOException{
IColumn column = new PrimitiveColumn( ColumnType.BOOLEAN , "boolean" );
column.add( ColumnType.BOOLEAN , new BooleanObj( true ) , 0 );
column.add( ColumnType.BOOLEAN , new BooleanObj( false ) , 1 );
column.add( ColumnType.BOOLEAN , new BooleanObj( true ) , 5 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new DumpBooleanColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.BOOLEAN , "target" , allocator , parent );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readBoolean().booleanValue() , true );
reader.setPosition( 1 );
assertEquals( reader.readBoolean().booleanValue() , false );
reader.setPosition( 5 );
assertEquals( reader.readBoolean().booleanValue() , true );
reader.setPosition( 2 );
assertEquals( reader.readBoolean() , null );
reader.setPosition( 3 );
assertEquals( reader.readBoolean() , null );
reader.setPosition( 4 );
assertEquals( reader.readBoolean() , null );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:36，
示例12: T_setString_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setString_1() throws IOException{
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.STRING , "target" , allocator , parent );
memoryAllocator.setString( 0 , "a" );
memoryAllocator.setString( 1 , "b" );
memoryAllocator.setString( 5 , "c" );
memoryAllocator.setString( 1000 , "a b c" );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readText().toString() , "a" );
reader.setPosition( 1 );
assertEquals( reader.readText().toString() , "b" );
reader.setPosition( 5 );
assertEquals( reader.readText().toString() , "c" );
for( int i = 6 ; i < 1000 ; i++ ){
reader.setPosition( i );
assertEquals( reader.readText() , null );
}
reader.setPosition( 1000 );
assertEquals( reader.readText().toString() , "a b c" );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:29，
示例13: T_setString_2
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setString_2() throws IOException{
IColumn column = new PrimitiveColumn( ColumnType.STRING , "boolean" );
column.add( ColumnType.STRING , new StringObj( "a" ) , 0 );
column.add( ColumnType.STRING , new StringObj( "b" ) , 1 );
column.add( ColumnType.STRING , new StringObj( "c" ) , 5 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new OptimizeDumpStringColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.STRING , "target" , allocator , parent );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readText().toString() , "a" );
reader.setPosition( 1 );
assertEquals( reader.readText().toString() , "b" );
reader.setPosition( 5 );
assertEquals( reader.readText().toString() , "c" );
reader.setPosition( 2 );
assertEquals( reader.readText() , null );
reader.setPosition( 3 );
assertEquals( reader.readText() , null );
reader.setPosition( 4 );
assertEquals( reader.readText() , null );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:37，
示例14: T_setBytes_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setBytes_1() throws IOException{
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.BYTES , "target" , allocator , parent );
memoryAllocator.setBytes( 0 , "a".getBytes() );
memoryAllocator.setBytes( 1 , "b".getBytes() );
memoryAllocator.setBytes( 5 , "c".getBytes() );
memoryAllocator.setBytes( 1000 , "a b c".getBytes() );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( new String( reader.readByteArray() ) , "a" );
reader.setPosition( 1 );
assertEquals( new String( reader.readByteArray() ) , "b" );
reader.setPosition( 5 );
assertEquals( new String( reader.readByteArray() ) , "c" );
for( int i = 6 ; i < 1000 ; i++ ){
reader.setPosition( i );
assertEquals( reader.readByteArray() , null );
}
reader.setPosition( 1000 );
assertEquals( new String( reader.readByteArray() ) , "a b c" );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:29，
示例15: T_setBytes_2
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setBytes_2() throws IOException{
IColumn column = new PrimitiveColumn( ColumnType.STRING , "boolean" );
column.add( ColumnType.STRING , new BytesObj( "a".getBytes() ) , 0 );
column.add( ColumnType.STRING , new BytesObj( "b".getBytes() ) , 1 );
column.add( ColumnType.STRING , new BytesObj( "c".getBytes() ) , 5 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new DumpBytesColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.BYTES , "target" , allocator , parent );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( new String( reader.readByteArray() ) , "a" );
reader.setPosition( 1 );
assertEquals( new String( reader.readByteArray() ) , "b" );
reader.setPosition( 5 );
assertEquals( new String( reader.readByteArray() ) , "c" );
reader.setPosition( 2 );
assertEquals( reader.readByteArray() , null );
reader.setPosition( 3 );
assertEquals( reader.readByteArray() , null );
reader.setPosition( 4 );
assertEquals( reader.readByteArray() , null );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:36，
示例16: T_setFloat_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setFloat_1() throws IOException{
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.FLOAT , "target" , allocator , parent );
memoryAllocator.setFloat( 0 , (float)0.1 );
memoryAllocator.setFloat( 1 , (float)0.2 );
memoryAllocator.setFloat( 5 , (float)0.255 );
memoryAllocator.setFloat( 1000 , (float)0.1 );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readFloat().floatValue() , (float)0,1 );
reader.setPosition( 1 );
assertEquals( reader.readFloat().floatValue() , (float)0.2 );
reader.setPosition( 5 );
assertEquals( reader.readFloat().floatValue() , (float)0.255 );
for( int i = 6 ; i < 1000 ; i++ ){
reader.setPosition( i );
assertEquals( reader.readFloat() , null );
}
reader.setPosition( 1000 );
assertEquals( reader.readFloat().floatValue() , (float)0.1 );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:29，
示例17: T_setFloat_2
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setFloat_2() throws IOException{
IColumn column = new PrimitiveColumn( ColumnType.FLOAT , "boolean" );
column.add( ColumnType.FLOAT , new FloatObj( (float)100 ) , 0 );
column.add( ColumnType.FLOAT , new FloatObj( (float)200 ) , 1 );
column.add( ColumnType.FLOAT , new FloatObj( (float)255 ) , 5 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new OptimizeFloatColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.FLOAT , "target" , allocator , parent );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readFloat().floatValue() , (float)100 );
reader.setPosition( 1 );
assertEquals( reader.readFloat().floatValue() , (float)200 );
reader.setPosition( 5 );
assertEquals( reader.readFloat().floatValue() , (float)255 );
reader.setPosition( 2 );
assertEquals( reader.readFloat() , null );
reader.setPosition( 3 );
assertEquals( reader.readFloat() , null );
reader.setPosition( 4 );
assertEquals( reader.readFloat() , null );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:37，
示例18: T_setInteger_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setInteger_1() throws IOException{
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.INTEGER , "target" , allocator , parent );
memoryAllocator.setInteger( 0 , 100 );
memoryAllocator.setInteger( 1 , 200 );
memoryAllocator.setInteger( 5 , 255 );
memoryAllocator.setInteger( 1000 , 10 );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readInteger().intValue() , 100 );
reader.setPosition( 1 );
assertEquals( reader.readInteger().intValue() , 200 );
reader.setPosition( 5 );
assertEquals( reader.readInteger().intValue() , 255 );
for( int i = 6 ; i < 1000 ; i++ ){
reader.setPosition( i );
assertEquals( reader.readInteger() , null );
}
reader.setPosition( 1000 );
assertEquals( reader.readInteger().intValue() , 10 );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:29，
示例19: T_setInteger_2
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setInteger_2() throws IOException{
IColumn column = new PrimitiveColumn( ColumnType.INTEGER , "boolean" );
column.add( ColumnType.INTEGER , new IntegerObj( 100 ) , 0 );
column.add( ColumnType.INTEGER , new IntegerObj( 200 ) , 1 );
column.add( ColumnType.INTEGER , new IntegerObj( 255 ) , 5 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new OptimizeLongColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.INTEGER , "target" , allocator , parent );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readInteger().intValue() , 100 );
reader.setPosition( 1 );
assertEquals( reader.readInteger().intValue() , 200 );
reader.setPosition( 5 );
assertEquals( reader.readInteger().intValue() , 255 );
reader.setPosition( 2 );
assertEquals( reader.readInteger() , null );
reader.setPosition( 3 );
assertEquals( reader.readInteger() , null );
reader.setPosition( 4 );
assertEquals( reader.readInteger() , null );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:37，
示例20: T_setShort_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setShort_1() throws IOException{
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.SHORT , "target" , allocator , parent );
memoryAllocator.setShort( 0 , (short)100 );
memoryAllocator.setShort( 1 , (short)200 );
memoryAllocator.setShort( 5 , (short)255 );
memoryAllocator.setShort( 1000 , (short)10 );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( (short)( reader.readCharacter().charValue() ) , (short)100 );
reader.setPosition( 1 );
assertEquals( (short)( reader.readCharacter().charValue() ) , (short)200 );
reader.setPosition( 5 );
assertEquals( (short)( reader.readCharacter().charValue() ) , (short)255 );
for( int i = 6 ; i < 1000 ; i++ ){
reader.setPosition( i );
assertEquals( reader.readCharacter() , null );
}
reader.setPosition( 1000 );
assertEquals( (short)( reader.readCharacter().charValue() ) , (short)10 );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:29，
示例21: T_setShort_2
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setShort_2() throws IOException{
IColumn column = new PrimitiveColumn( ColumnType.SHORT , "boolean" );
column.add( ColumnType.SHORT , new ShortObj( (short)100 ) , 0 );
column.add( ColumnType.SHORT , new ShortObj( (short)200 ) , 1 );
column.add( ColumnType.SHORT , new ShortObj( (short)255 ) , 5 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new OptimizeLongColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.SHORT , "target" , allocator , parent );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( (short)( reader.readCharacter().charValue() ) , (short)100 );
reader.setPosition( 1 );
assertEquals( (short)( reader.readCharacter().charValue() ) , (short)200 );
reader.setPosition( 5 );
assertEquals( (short)( reader.readCharacter().charValue() ) , (short)255 );
reader.setPosition( 2 );
assertEquals( reader.readCharacter() , null );
reader.setPosition( 3 );
assertEquals( reader.readCharacter() , null );
reader.setPosition( 4 );
assertEquals( reader.readCharacter() , null );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:36，
示例22: T_setByte_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setByte_1() throws IOException{
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.BYTE , "target" , allocator , parent );
memoryAllocator.setByte( 0 , (byte)100 );
memoryAllocator.setByte( 1 , (byte)200 );
memoryAllocator.setByte( 5 , (byte)255 );
memoryAllocator.setByte( 1000 , (byte)10 );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readByte().byteValue() , (byte)100 );
reader.setPosition( 1 );
assertEquals( reader.readByte().byteValue() , (byte)200 );
reader.setPosition( 5 );
assertEquals( reader.readByte().byteValue() , (byte)255 );
for( int i = 6 ; i < 1000 ; i++ ){
reader.setPosition( i );
assertEquals( reader.readByte() , null );
}
reader.setPosition( 1000 );
assertEquals( reader.readByte().byteValue() , (byte)10 );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:29，
示例23: T_setByte_2
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setByte_2() throws IOException{
IColumn column = new PrimitiveColumn( ColumnType.BYTE , "boolean" );
column.add( ColumnType.BYTE , new ByteObj( (byte)100 ) , 0 );
column.add( ColumnType.BYTE , new ByteObj( (byte)200 ) , 1 );
column.add( ColumnType.BYTE , new ByteObj( (byte)255 ) , 5 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new OptimizeLongColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
IMemoryAllocator memoryAllocator = ArrowMemoryAllocatorFactory.getFromMapVector( ColumnType.BYTE , "target" , allocator , parent );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
MapReader rootReader = parent.getReader();
FieldReader reader = rootReader.reader( "target" );
reader.setPosition( 0 );
assertEquals( reader.readByte().byteValue() , (byte)100 );
reader.setPosition( 1 );
assertEquals( reader.readByte().byteValue() , (byte)200 );
reader.setPosition( 5 );
assertEquals( reader.readByte().byteValue() , (byte)255 );
reader.setPosition( 2 );
assertEquals( reader.readByte() , null );
reader.setPosition( 3 );
assertEquals( reader.readByte() , null );
reader.setPosition( 4 );
assertEquals( reader.readByte() , null );
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:37，
示例24: main
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
public static void main(String[] args) {
try (final BufferAllocator bufferAllocator = new RootAllocator(SabotConfig.getMaxDirectMemory())) {
final FileSystem fs = FileSystem.getLocal(new Configuration());
final Path filePath = new Path(args[0]);
final CodecFactory codecFactory = CodecFactory.createDirectCodecFactory(fs.getConf(), new ParquetDirectByteBufferAllocator(bufferAllocator), 0);
final Pair, Set> dictionaries = readDictionaries(fs, filePath, codecFactory);
for (Map.Entry entry : dictionaries.getLeft().entrySet()) {
printDictionary(entry.getKey(), entry.getValue());
}
System.out.println("Binary columns which are not dictionary encoded: " + dictionaries.getRight());
} catch (IOException ioe) {
logger.error("Failed ", ioe);
}
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:15，
示例25: testPadValues
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void testPadValues() {
int originalTypeByteWidth = 8;
int targetTypeByteWidth = 12;
try (
BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
UInt1Vector bytes = new UInt1Vector("$bits$", allocator);
) {
int count = 100;
for (int i = 0; i < count * 8 ; i++) {
bytes.getMutator().setSafe(i, i % 8);
}
bytes.getMutator().setValueCount(count * 8);
ArrowBuf oldBuf = bytes.getBuffer();
oldBuf.retain();
SerializedField.Builder fieldBuilder = TypeHelper.getMetadataBuilder(bytes);
ArrowBuf newBuf = padValues(allocator, fieldBuilder, oldBuf, originalTypeByteWidth, targetTypeByteWidth);
fieldBuilder.setValueCount(count * 12); // since we're loading it in a bytes vector
TypeHelper.load(bytes, fieldBuilder.build(), newBuf);
for (int i = 0; i < count ; i++) {
for (int byteIndex = 0; byteIndex < 8 ; byteIndex++) {
assertEquals((i * 8 + byteIndex) % 8, bytes.getAccessor().get(i * 12 + byteIndex));
}
}
newBuf.release();
}
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:30，
示例26: emptyvalues
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void emptyvalues() throws Exception {
try (BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
final VectorContainer c = new VectorContainer();) {
NullableVarCharVector col1 = new NullableVarCharVector("col1", allocator);
c.add(col1);
NullableVarCharVector col2 = new NullableVarCharVector("col2", allocator);
c.add(col2);
NullableIntVector col3 = new NullableIntVector("col3", allocator);
c.add(col3);
c.allocateNew();
final int records = c.setAllCount(2000);
final PivotDef pivot = PivotBuilder.getBlockDefinition(
new FieldVectorPair(col1, col1),
new FieldVectorPair(col2, col2),
new FieldVectorPair(col3, col3)
);
try (
final FixedBlockVector fbv = new FixedBlockVector(allocator, pivot.getBlockWidth());
final VariableBlockVector var = new VariableBlockVector(allocator, pivot.getVariableCount());) {
Pivots.pivot(pivot, records, fbv, var);
try (LBlockHashTable bht = new LBlockHashTable(HashConfig.getDefault(), pivot, allocator, 16000, 10, ResizeListener.NO_OP);) {
final long keyFixedAddr = fbv.getMemoryAddress();
final long keyVarAddr = var.getMemoryAddress();
for (int keyIndex = 0; keyIndex < records; keyIndex++) {
assertEquals(0, bht.add(keyFixedAddr, keyVarAddr, keyIndex));
}
}
}
}
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:38，
示例27: setupServer
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Before
public void setupServer() throws Exception{
allocator = new RootAllocator(20 * 1024 * 1024);
pool = new CloseableThreadPool("test-fabric");
fabric = new FabricServiceImpl(address, 45678, true, 300, 2, pool, allocator, 0, Long.MAX_VALUE);
fabric.start();
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:8，
示例28: createKKStoreProvider
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Override
KVStoreProvider createKKStoreProvider() throws Exception {
allocator = new RootAllocator(20 * 1024 * 1024);
pool = new CloseableThreadPool("test-remoteindexedkvstore");
localFabricService = new FabricServiceImpl("localhost", 45678, true, 300, 2, pool, allocator, 0, Long.MAX_VALUE);
localFabricService.start();
final Provider fab = new Provider(){
@Override
public FabricService get() {
return localFabricService;
}};
remoteFabricService = new FabricServiceImpl("localhost", 45679, true, 300, 2, pool, allocator, 0, Long.MAX_VALUE);
remoteFabricService.start();
final Provider rfab = new Provider(){
@Override
public FabricService get() {
return remoteFabricService;
}};
localKVStoreProvider = new LocalKVStoreProvider(DremioTest.CLASSPATH_SCAN_RESULT, fab, allocator, "localhost", null, true, true, true, false);
localKVStoreProvider.start();
remoteKVStoreProvider = new RemoteKVStoreProvider(DremioTest.CLASSPATH_SCAN_RESULT, rfab, allocator, "localhost", "localhost", localFabricService.getPort());
remoteKVStoreProvider.start();
return remoteKVStoreProvider;
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:28，
示例29: setUpPDFSService
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@BeforeClass
public static void setUpPDFSService() throws Exception {
root = new RootAllocator(Long.MAX_VALUE);
EndpointProvider provider = new EndpointProvider();
client = new ServiceHolder(root, provider, PDFSMode.CLIENT, "client");
data = new ServiceHolder(root, provider, PDFSMode.DATA, "data");
provider.add(data);
clientFS = client.fileSystem;
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:10，
示例30: T_setArray_1
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void T_setArray_1() throws IOException{
IColumn column = new ArrayColumn( "array" );
List value = new ArrayList();
value.add( new StringObj( "a" ) );
value.add( new StringObj( "b" ) );
value.add( new StringObj( "c" ) );
column.add( ColumnType.ARRAY , value , 0 );
column.add( ColumnType.ARRAY , value , 1 );
column.add( ColumnType.ARRAY , value , 2 );
column.add( ColumnType.ARRAY , value , 3 );
column.add( ColumnType.ARRAY , value , 6 );
ColumnBinaryMakerConfig defaultConfig = new ColumnBinaryMakerConfig();
ColumnBinaryMakerCustomConfigNode configNode = new ColumnBinaryMakerCustomConfigNode( "root" , defaultConfig );
IColumnBinaryMaker maker = new DumpArrayColumnBinaryMaker();
ColumnBinary columnBinary = maker.toBinary( defaultConfig , null , column );
BufferAllocator allocator = new RootAllocator( 1024 * 1024 * 10 );
SchemaChangeCallBack callBack = new SchemaChangeCallBack();
MapVector parent = new MapVector("root", allocator, new FieldType(false, Struct.INSTANCE, null, null), callBack);
parent.allocateNew();
ListVector listVector = parent.addOrGetList( "target" );
IMemoryAllocator memoryAllocator = new ArrowArrayMemoryAllocator( allocator , listVector );
maker.loadInMemoryStorage( columnBinary , memoryAllocator );
ListVector.Accessor accessor = listVector.getAccessor();
for( int i = 0 ; i < 7 ; i++ ){
System.out.println( "count:" + accessor.getInnerValueCountAt(i) );
System.out.println( "obj:" + accessor.getObject(i) );
}
/*
reader.setPosition( 0 );
assertEquals( reader.read().booleanValue() , true );
reader.setPosition( 1 );
assertEquals( reader.readBoolean().booleanValue() , false );
reader.setPosition( 5 );
assertEquals( reader.readBoolean().booleanValue() , true );
reader.setPosition( 2 );
assertEquals( reader.readBoolean() , null );
reader.setPosition( 3 );
assertEquals( reader.readBoolean() , null );
reader.setPosition( 4 );
assertEquals( reader.readBoolean() , null );
*/
}
開發者ID:yahoojapan，項目名稱:multiple-dimension-spread，代碼行數:49，
示例31: before
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Before
public void before() {
this.allocator = new RootAllocator(Integer.MAX_VALUE);
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:5，
示例32: simple
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Test
public void simple() throws Exception {
String[] col1arr = {"hello", "my", "hello", "hello", null, null};
String[] col2arr = {"every", "every", "every", "none", null, null};
Integer[] col3arr = {1, 1, 1, 1, 1, 1};
try (BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
final VectorContainer c = new VectorContainer();) {
NullableVarCharVector col1 = new NullableVarCharVector("col1", allocator);
TestVarBinaryPivot.populate(col1, col1arr);
c.add(col1);
NullableVarCharVector col2 = new NullableVarCharVector("col2", allocator);
TestVarBinaryPivot.populate(col2, col2arr);
c.add(col2);
NullableIntVector col3 = new NullableIntVector("col3", allocator);
TestIntPivot.populate(col3, col3arr);
c.add(col3);
final int records = c.setAllCount(col1arr.length);
final PivotDef pivot = PivotBuilder.getBlockDefinition(
new FieldVectorPair(col1, col1),
new FieldVectorPair(col2, col2),
new FieldVectorPair(col3, col3)
);
try (
final FixedBlockVector fbv = new FixedBlockVector(allocator, pivot.getBlockWidth());
final VariableBlockVector var = new VariableBlockVector(allocator, pivot.getVariableCount());) {
Pivots.pivot(pivot, records, fbv, var);
try (LBlockHashTable bht = new LBlockHashTable(HashConfig.getDefault(), pivot, allocator, 16000, 10, ResizeListener.NO_OP);) {
final long keyFixedAddr = fbv.getMemoryAddress();
final long keyVarAddr = var.getMemoryAddress();
int[] expectedOrdinals = {0, 1, 0, 2, 3, 3};
int[] actualOrdinals = new int[expectedOrdinals.length];
for (int keyIndex = 0; keyIndex < records; keyIndex++) {
actualOrdinals[keyIndex] = bht.add(keyFixedAddr, keyVarAddr, keyIndex);
}
assertArrayEquals(expectedOrdinals, actualOrdinals);
}
}
}
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:47，
示例33: setup
點讚 2
import org.apache.arrow.memory.RootAllocator; //導入依賴的package包/類
@Before
public void setup(){
this.allocator = new RootAllocator(Long.MAX_VALUE);
}
開發者ID:dremio，項目名稱:dremio-oss，代碼行數:5，
注：本文中的org.apache.arrow.memory.RootAllocator類示例整理自Github/MSDocs等源碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。
117
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
