Neither the HADOOP_CONF_DIR nor the YARN_CONF_DIR environment variable is set. The Flink YARN Client needs one of these to be set to properly load the Hadoop configuration for accessing YARN.
Please note that the Client requires the YARN_CONF_DIR
or HADOOP_CONF_DIR
environment variable to be set to read the YARN and HDFS configuration.
广播变量:
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn/
./bin/yarn-session.sh -n 2 -tm 2000
2017-05-24 18:39:47,775 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf.cloudera.yarn
2017-05-24 18:39:47,845 INFO org.apache.flink.yarn.YarnClusterDescriptor - Using values:
2017-05-24 18:39:47,845 INFO org.apache.flink.yarn.YarnClusterDescriptor - TaskManager count = 2
2017-05-24 18:39:47,846 INFO org.apache.flink.yarn.YarnClusterDescriptor - JobManager memory = 1024
2017-05-24 18:39:47,846 INFO org.apache.flink.yarn.YarnClusterDescriptor - TaskManager memory = 2000
2017-05-24 18:39:49,054 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-05-24 18:39:49,150 WARN org.apache.flink.yarn.YarnClusterDescriptor - The configuration directory ('/var/lib/hadoop-hdfs/flink-1.2.1/conf') contains both LOG4J and Logback configuration files. Please delete or rename one of them.
2017-05-24 18:39:49,170 INFO org.apache.flink.yarn.Utils - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/conf/logback.xml to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/logback.xml
2017-05-24 18:39:49,626 INFO org.apache.flink.yarn.Utils - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/conf/log4j.properties to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/log4j.properties
2017-05-24 18:39:49,671 INFO org.apache.flink.yarn.Utils - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/lib to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/lib
2017-05-24 18:39:51,389 INFO org.apache.flink.yarn.Utils - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/lib/flink-dist_2.10-1.2.1.jar to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/flink-dist_2.10-1.2.1.jar
2017-05-24 18:39:53,143 INFO org.apache.flink.yarn.Utils - Copying from /var/lib/hadoop-hdfs/flink-1.2.1/conf/flink-conf.yaml to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/flink-conf.yaml
2017-05-24 18:39:53,203 INFO org.apache.flink.yarn.YarnClusterDescriptor - Submitting application master application_1495186603292_0062
2017-05-24 18:39:53,280 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1495186603292_0062
2017-05-24 18:39:53,280 INFO org.apache.flink.yarn.YarnClusterDescriptor - Waiting for the cluster to be allocated
2017-05-24 18:39:53,284 INFO org.apache.flink.yarn.YarnClusterDescriptor - Deploying cluster, current state ACCEPTED
17-05-24 18:40:05,290 INFO org.apache.flink.yarn.YarnClusterDescriptor - YARN application has been deployed successfully.
2017-05-24 18:39:47,845 INFO org.apache.flink.yarn.YarnClusterDescriptor - Using values:
2017-05-24 18:39:47,845 INFO org.apache.flink.yarn.YarnClusterDescriptor - TaskManager count = 2
2017-05-24 18:39:47,846 INFO org.apache.flink.yarn.YarnClusterDescriptor - JobManager memory = 1024
2017-05-24 18:39:47,846 INFO org.apache.flink.yarn.YarnClusterDescriptor - TaskManager memory = 2000
2017-05-24 18:39:49,054 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-05-24 18:39:49,150 WARN org.apache.flink.yarn.YarnClusterDescriptor - The configuration directory ('/var/lib/hadoop-hdfs/flink-1.2.1/conf') contains both LOG4J and Logback configuration files. Please delete or rename one of them.
2017-05-24 18:39:49,170 INFO org.apache.flink.yarn.Utils - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/conf/logback.xml to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/logback.xml
2017-05-24 18:39:49,626 INFO org.apache.flink.yarn.Utils - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/conf/log4j.properties to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/log4j.properties
2017-05-24 18:39:49,671 INFO org.apache.flink.yarn.Utils - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/lib to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/lib
2017-05-24 18:39:51,389 INFO org.apache.flink.yarn.Utils - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/lib/flink-dist_2.10-1.2.1.jar to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/flink-dist_2.10-1.2.1.jar
2017-05-24 18:39:53,143 INFO org.apache.flink.yarn.Utils - Copying from /var/lib/hadoop-hdfs/flink-1.2.1/conf/flink-conf.yaml to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/flink-conf.yaml
2017-05-24 18:39:53,203 INFO org.apache.flink.yarn.YarnClusterDescriptor - Submitting application master application_1495186603292_0062
2017-05-24 18:39:53,280 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1495186603292_0062
2017-05-24 18:39:53,280 INFO org.apache.flink.yarn.YarnClusterDescriptor - Waiting for the cluster to be allocated
2017-05-24 18:39:53,284 INFO org.apache.flink.yarn.YarnClusterDescriptor - Deploying cluster, current state ACCEPTED
17-05-24 18:40:05,290 INFO org.apache.flink.yarn.YarnClusterDescriptor - YARN application has been deployed successfully.
启动程序成功。