hive strict模式

set hive.mapred.mode=nonstrict;
set hive.mapred.mode=strict;

hive> set hive.mapred.mode;
hive.mapred.mode=nonstrict
hive> set hive.mapred.mode=strict;
hive> select key, value from src order by key,value;
FAILED: Error in semantic analysis: line 1:36 In strict mode, limit must be specified if ORDER BY is present value
hive>


HiveConf:
HIVEMAPREDMODE("hive.mapred.mode", "nonstrict"),

tianzhao@ubuntu:~/hive/trunk/hive-0.6.0/conf$ grep -r "hive.mapred.mode" ../conf/
../conf/hive-default.xml: <name>hive.mapred.mode</name>

hive-default.xml:
<property>
<name>hive.mapred.mode</name>
<value>nonstrict</value>
<description>The mode in which the hive operations are being performed. In strict mode, some risky queries are not allowed to run</description>
</property>


strict:
出现的地方:
(1)
private Operator genJoinReduceSinkChild(QB qb, QBJoinTree joinTree,
Operator child, String srcName, int pos) throws SemanticException {

// Use only 1 reducer in case of cartesian product
if (reduceKeys.size() == 0) {
numReds = 1;

// Cartesian product is not supported in strict mode
if (conf.getVar(HiveConf.ConfVars.HIVEMAPREDMODE).equalsIgnoreCase(
"strict")) {
throw new SemanticException(ErrorMsg.NO_CARTESIAN_PRODUCT.getMsg());
}
}

}

hive> set hive.mapred.mode=strict;
hive> EXPLAIN SELECT subq.key, tab.value FROM src subq JOIN src tab where subq.key < 200;
FAILED: Error in semantic analysis: In strict mode, cartesian product is not allowed. If you really want to perform the operation, set hive.mapred.mode=nonstrict


(2)
private Operator genReduceSinkPlan(String dest, QB qb, Operator input,
int numReducers) throws SemanticException {

if (sortExprs == null) {
sortExprs = qb.getParseInfo().getOrderByForClause(dest);
if (sortExprs != null) {
assert numReducers == 1;
// in strict mode, in the presence of order by, limit must be specified
Integer limit = qb.getParseInfo().getDestLimit(dest);
if (conf.getVar(HiveConf.ConfVars.HIVEMAPREDMODE).equalsIgnoreCase(
"strict")
&& limit == null) {
throw new SemanticException(ErrorMsg.NO_LIMIT_WITH_ORDERBY
.getMsg(sortExprs));
}
}
}

}

(3)
public static PrunedPartitionList prune(Table tab, ExprNodeDesc prunerExpr,
HiveConf conf, String alias,
Map<String, PrunedPartitionList> prunedPartitionsMap) throws HiveException {

// If the "strict" mode is on, we have to provide partition pruner for
// each table.
if ("strict".equalsIgnoreCase(HiveConf.getVar(conf,
HiveConf.ConfVars.HIVEMAPREDMODE))) {
if (!hasColumnExpr(prunerExpr)) {
throw new SemanticException(ErrorMsg.NO_PARTITION_PREDICATE
.getMsg("for Alias \"" + alias + "\" Table \""
+ tab.getTableName() + "\""));
}
}

}

strict模式在下面三种情况下有限制:
(1) partition表需要加上分区裁剪
(2) order by 只有一个reduce,需要加上limit
(3) join时,如果只有一个reduce,笛卡尔积不支持。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值