BitmapIndex的Block的概念

allen@allen-laptop:~/Desktop/hive-0.8.1$ cd bin
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ ./hive
12/03/28 15:12:47 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Logging initialized using configuration in jar:file:/home/allen/Desktop/hive-0.8.1/lib/hive-common-0.8.1.jar!/hive-log4j.properties
Hive history file=/tmp/allen/hive_job_log_allen_201203281512_352230653.txt
hive> show tables;
OK
default__table02_compact_index__
default__table03_bitmap_index__
default__test_bitmap_index__
default__triple_dual_index__
table02
table03
test
triple
Time taken: 4.069 seconds
hive> create table test_rc(class int,name string) 
    > row format delimited
    > fields terminated by '\t'
    > stored as rcfile;
OK
Time taken: 0.713 seconds
hive> insert overwrite table test_rc
    > select * from test;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201203281512_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201203281512_0001
Kill Command = /home/allen/Hadoop/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201203281512_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2012-03-28 15:15:26,774 Stage-1 map = 0%,  reduce = 0%
2012-03-28 15:15:28,903 Stage-1 map = 100%,  reduce = 0%
2012-03-28 15:15:32,028 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201203281512_0001
Ended Job = 783802278, job is filtered out (removed at runtime).
Moving data to: hdfs://localhost:9000/tmp/hive-allen/hive_2012-03-28_15-15-15_976_476836116449539941/-ext-10000
Loading data to table default.test_rc
Deleted hdfs://localhost:9000/user/hive/warehouse/test_rc
Table default.test_rc stats: [num_partitions: 0, num_files: 1, num_rows: 0, total_size: 226, raw_data_size: 0]
7 Rows loaded to test_rc
MapReduce Jobs Launched: 
Job 0: Map: 1   HDFS Read: 61 HDFS Write: 226 SUCESS
Total MapReduce CPU Time Spent: 0 msec
OK
Time taken: 17.53 seconds
hive> ll
    > ;
FAILED: Parse Error: line 1:0 cannot recognize input near 'll' '<EOF>' '<EOF>'

hive> exit;
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -ls /user/hive/warehosue/;
12/03/28 15:16:00 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
ls: Cannot access /user/hive/warehosue/: No such file or directory.
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -ls /
12/03/28 15:16:15 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Found 3 items
drwxr-xr-x   - allen supergroup          0 2012-03-26 11:47 /home
drwxr-xr-x   - allen supergroup          0 2012-03-26 11:47 /tmp
drwxr-xr-x   - allen supergroup          0 2012-03-25 23:40 /user
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -ls /user
12/03/28 15:16:21 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
^[[AFound 2 items
drwxr-xr-x   - allen supergroup          0 2012-03-06 18:07 /user/allen
drwxr-xr-x   - allen supergroup          0 2012-03-25 23:40 /user/hive
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -ls /user/hive
12/03/28 15:16:28 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Found 1 items
drwxr-xr-x   - allen supergroup          0 2012-03-28 15:15 /user/hive/warehouse
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -ls /user/hive/warehouse;
12/03/28 15:16:37 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Found 7 items
drwxr-xr-x   - allen supergroup          0 2012-03-22 14:21 /user/hive/warehouse/default__table03_bitmap_index__
drwxr-xr-x   - allen supergroup          0 2012-03-22 15:48 /user/hive/warehouse/default__test_bitmap_index__
drwxr-xr-x   - allen supergroup          0 2012-03-25 23:40 /user/hive/warehouse/default__triple_dual_index__
drwxr-xr-x   - allen supergroup          0 2012-03-12 21:49 /user/hive/warehouse/table03
drwxr-xr-x   - allen supergroup          0 2012-03-22 15:47 /user/hive/warehouse/test
drwxr-xr-x   - allen supergroup          0 2012-03-28 15:15 /user/hive/warehouse/test_rc
drwxr-xr-x   - allen supergroup          0 2012-03-25 23:25 /user/hive/warehouse/triple
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -ls /user/hive/warehouse/test_rc
12/03/28 15:16:51 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Found 1 items
-rw-r--r--   1 allen supergroup        226 2012-03-28 15:15 /user/hive/warehouse/test_rc/000000_0
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ ./hive
hive            hive-config.sh  
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ ./hive
12/03/28 15:17:03 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Logging initialized using configuration in jar:file:/home/allen/Desktop/hive-0.8.1/lib/hive-common-0.8.1.jar!/hive-log4j.properties
Hive history file=/tmp/allen/hive_job_log_allen_201203281517_609509912.txt
hive> create index bitmap_index on table test_rc(class)
    > as 'org.apache.hadoop.hive.ql.index.bitmap.BitmapIndexHandler'
    > with deferred rebuild;
OK
Time taken: 9.709 seconds
hive> alter index bitmap_index on test_rc rebuild;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201203281512_0002, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201203281512_0002
Kill Command = /home/allen/Hadoop/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201203281512_0002
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2012-03-28 15:19:52,372 Stage-1 map = 0%,  reduce = 0%
2012-03-28 15:20:00,720 Stage-1 map = 100%,  reduce = 0%
2012-03-28 15:20:13,238 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201203281512_0002
Loading data to table default.default__test_rc_bitmap_index__
Deleted hdfs://localhost:9000/user/hive/warehouse/default__test_rc_bitmap_index__
Table default.default__test_rc_bitmap_index__ stats: [num_partitions: 0, num_files: 1, num_rows: 0, total_size: 260, raw_data_size: 0]
MapReduce Jobs Launched: 
Job 0: Map: 1  Reduce: 1   HDFS Read: 186 HDFS Write: 260 SUCESS
Total MapReduce CPU Time Spent: 0 msec
OK
Time taken: 47.457 seconds
hive> exit;                                       
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -ls /user/hive/warehouse/;
12/03/28 15:20:33 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Found 8 items
drwxr-xr-x   - allen supergroup          0 2012-03-22 14:21 /user/hive/warehouse/default__table03_bitmap_index__
drwxr-xr-x   - allen supergroup          0 2012-03-22 15:48 /user/hive/warehouse/default__test_bitmap_index__
drwxr-xr-x   - allen supergroup          0 2012-03-28 15:20 /user/hive/warehouse/default__test_rc_bitmap_index__
drwxr-xr-x   - allen supergroup          0 2012-03-25 23:40 /user/hive/warehouse/default__triple_dual_index__
drwxr-xr-x   - allen supergroup          0 2012-03-12 21:49 /user/hive/warehouse/table03
drwxr-xr-x   - allen supergroup          0 2012-03-22 15:47 /user/hive/warehouse/test
drwxr-xr-x   - allen supergroup          0 2012-03-28 15:15 /user/hive/warehouse/test_rc
drwxr-xr-x   - allen supergroup          0 2012-03-25 23:25 /user/hive/warehouse/triple
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -ls /user/hive/warehouse/default__test_rc_bitmap_index__
12/03/28 15:20:50 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Found 1 items
-rw-r--r--   1 allen supergroup        260 2012-03-28 15:20 /user/hive/warehouse/default__test_rc_bitmap_index__/000000_0
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -cat /user/hive/warehouse/default__test_rc_bitmap_index__/000000_0
12/03/28 15:21:08 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
1	hdfs://localhost:9000/user/hive/warehouse/test_rc/000000_0	151	7248589934592810
2	hdfs://localhost:9000/user/hive/warehouse/test_rc/000000_0	151	6248589934592420
3	hdfs://localhost:9000/user/hive/warehouse/test_rc/000000_0	151	324858993459240
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ 

allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ ./hive
12/03/28 15:27:14 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Logging initialized using configuration in jar:file:/home/allen/Desktop/hive-0.8.1/lib/hive-common-0.8.1.jar!/hive-log4j.properties
Hive history file=/tmp/allen/hive_job_log_allen_201203281527_106840917.txt
hive> create table test_seq(class int,name string)                  
    > row format delimited                                          
    > fields terminated by '\t'                                     
    > stored as sequencefile;                                       
OK
Time taken: 8.551 seconds
hive> insert overwrite table test_seq                               
    > select * from test;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201203281512_0003, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201203281512_0003
Kill Command = /home/allen/Hadoop/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201203281512_0003
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2012-03-28 15:28:39,889 Stage-1 map = 0%,  reduce = 0%
2012-03-28 15:28:45,165 Stage-1 map = 100%,  reduce = 0%
2012-03-28 15:28:51,441 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201203281512_0003
Ended Job = -244635386, job is filtered out (removed at runtime).
Moving data to: hdfs://localhost:9000/tmp/hive-allen/hive_2012-03-28_15-28-24_955_4897112826733271333/-ext-10000
Loading data to table default.test_seq
Deleted hdfs://localhost:9000/user/hive/warehouse/test_seq
Table default.test_seq stats: [num_partitions: 0, num_files: 1, num_rows: 0, total_size: 232, raw_data_size: 0]
7 Rows loaded to test_seq
MapReduce Jobs Launched: 
Job 0: Map: 1   HDFS Read: 61 HDFS Write: 232 SUCESS
Total MapReduce CPU Time Spent: 0 msec
OK
Time taken: 29.214 seconds
hive> create index bitmap_index on table test_seq(class)            
    > as 'org.apache.hadoop.hive.ql.index.bitmap.BitmapIndexHandler'
    > with deferred rebuild;                                        
OK
Time taken: 0.319 seconds
hive> alter index bitmap_index on test_seq rebuild;                 
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201203281512_0004, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201203281512_0004
Kill Command = /home/allen/Hadoop/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201203281512_0004
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2012-03-28 15:31:40,847 Stage-1 map = 0%,  reduce = 0%
2012-03-28 15:31:43,931 Stage-1 map = 100%,  reduce = 0%
2012-03-28 15:31:52,160 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201203281512_0004
Loading data to table default.default__test_seq_bitmap_index__
Deleted hdfs://localhost:9000/user/hive/warehouse/default__test_seq_bitmap_index__
Table default.default__test_seq_bitmap_index__ stats: [num_partitions: 0, num_files: 1, num_rows: 0, total_size: 608, raw_data_size: 0]
MapReduce Jobs Launched: 
Job 0: Map: 1  Reduce: 1   HDFS Read: 402 HDFS Write: 608 SUCESS
Total MapReduce CPU Time Spent: 0 msec
OK
Time taken: 21.936 seconds
hive> exit;
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$ hadoop fs -cat /user/hive/warehouse/default__test_seq_bitmap_index__/000000_0
12/03/28 15:32:19 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
1	hdfs://localhost:9000/user/hive/warehouse/test_seq/000000_0	87	124858993459210
2	hdfs://localhost:9000/user/hive/warehouse/test_seq/000000_0	107	124858993459210
3	hdfs://localhost:9000/user/hive/warehouse/test_seq/000000_0	129	124858993459210
2	hdfs://localhost:9000/user/hive/warehouse/test_seq/000000_0	153	124858993459210
1	hdfs://localhost:9000/user/hive/warehouse/test_seq/000000_0	172	124858993459210
2	hdfs://localhost:9000/user/hive/warehouse/test_seq/000000_0	191	124858993459210
1	hdfs://localhost:9000/user/hive/warehouse/test_seq/000000_0	212	124858993459210
allen@allen-laptop:~/Desktop/hive-0.8.1/bin$