flink

最新推荐文章于 2022-08-02 10:55:18 发布
最新推荐文章于 2022-08-02 10:55:18 发布
阅读量211 收藏
点赞数
文章标签： flink
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/spencer_tseng/article/details/103360985
版权 
[root@master flink-1.4.0]# ./bin/yarn-session.sh
Using the result of 'hadoop classpath' to augment the Hadoop classpath: /home/zwf/hadoop-2.7.7/etc/hadoop:/home/zwf/hadoop-2.7.7/share/hadoop/common/lib/*:/home/zwf/hadoop-2.7.7/share/hadoop/common/*:/home/zwf/hadoop-2.7.7/share/hadoop/hdfs:/home/zwf/hadoop-2.7.7/share/hadoop/hdfs/lib/*:/home/zwf/hadoop-2.7.7/share/hadoop/hdfs/*:/home/zwf/hadoop-2.7.7/share/hadoop/yarn/lib/*:/home/zwf/hadoop-2.7.7/share/hadoop/yarn/*:/home/zwf/hadoop-2.7.7/share/hadoop/mapreduce/lib/*:/home/zwf/hadoop-2.7.7/share/hadoop/mapreduce/*:/home/zwf/hadoop-2.7.7/contrib/capacity-scheduler/*.jar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/zwf/flink-1.4.0/lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/zwf/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2019-12-03 00:58:36,063 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.rpc.address, localhost
2019-12-03 00:58:36,067 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.rpc.port, 6123
2019-12-03 00:58:36,068 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.heap.mb, 1024
2019-12-03 00:58:36,068 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.heap.mb, 1024
2019-12-03 00:58:36,068 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.numberOfTaskSlots, 1
2019-12-03 00:58:36,074 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.preallocate, false
2019-12-03 00:58:36,074 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: parallelism.default, 1
2019-12-03 00:58:36,074 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: web.port, 8081
2019-12-03 00:58:38,197 WARN  org.apache.hadoop.util.NativeCodeLoader                       - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-12-03 00:58:38,648 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to root (auth:SIMPLE)
2019-12-03 00:58:38,792 ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - Missing required argument n
Usage:
   Required
     -n,--container <arg>   Number of YARN container to allocate (=Number of Task Managers)
   Optional
     -D <arg>                        Dynamic properties
     -d,--detached                   Start detached
     -f6,--flip6                     Specify this option to start a Flip-6 Yarn session cluster.
     -id,--applicationId <arg>       Attach to running YARN session
     -j,--jar <arg>                  Path to Flink jar file
     -jm,--jobManagerMemory <arg>    Memory for JobManager Container [in MB]
     -n,--container <arg>            Number of YARN container to allocate (=Number of Task Managers)
     -nm,--name <arg>                Set a custom name for the application on YARN
     -q,--query                      Display available YARN resources (memory, cores)
     -qu,--queue <arg>               Specify YARN queue.
     -s,--slots <arg>                Number of slots per TaskManager
     -st,--streaming                 Start Flink in streaming mode
     -t,--ship <arg>                 Ship files in the specified directory (t for transfer)
     -tm,--taskManagerMemory <arg>   Memory per TaskManager Container [in MB]
     -z,--zookeeperNamespace <arg>   Namespace to create the Zookeeper sub-paths for high availability mode
Exception in thread "main" java.lang.IllegalArgumentException: Missing required argument n
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createClusterSpecification(FlinkYarnSessionCli.java:356)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:676)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:514)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:511)
	at org.apache.flink.runtime.security.HadoopSecurityContext$$Lambda$8/2085002312.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:511)
[root@master flink-1.4.0]# ./bin/yarn-session.sh -n 1 -tm 1024 -s 2
Using the result of 'hadoop classpath' to augment the Hadoop classpath: /home/zwf/hadoop-2.7.7/etc/hadoop:/home/zwf/hadoop-2.7.7/share/hadoop/common/lib/*:/home/zwf/hadoop-2.7.7/share/hadoop/common/*:/home/zwf/hadoop-2.7.7/share/hadoop/hdfs:/home/zwf/hadoop-2.7.7/share/hadoop/hdfs/lib/*:/home/zwf/hadoop-2.7.7/share/hadoop/hdfs/*:/home/zwf/hadoop-2.7.7/share/hadoop/yarn/lib/*:/home/zwf/hadoop-2.7.7/share/hadoop/yarn/*:/home/zwf/hadoop-2.7.7/share/hadoop/mapreduce/lib/*:/home/zwf/hadoop-2.7.7/share/hadoop/mapreduce/*:/home/zwf/hadoop-2.7.7/contrib/capacity-scheduler/*.jar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/zwf/flink-1.4.0/lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/zwf/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2019-12-03 01:02:58,718 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.rpc.address, localhost
2019-12-03 01:02:58,722 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.rpc.port, 6123
2019-12-03 01:02:58,722 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.heap.mb, 1024
2019-12-03 01:02:58,723 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.heap.mb, 1024
2019-12-03 01:02:58,723 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.numberOfTaskSlots, 1
2019-12-03 01:02:58,728 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.memory.preallocate, false
2019-12-03 01:02:58,728 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: parallelism.default, 1
2019-12-03 01:02:58,729 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: web.port, 8081
2019-12-03 01:03:00,221 WARN  org.apache.hadoop.util.NativeCodeLoader                       - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-12-03 01:03:00,555 INFO  org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop user set to root (auth:SIMPLE)
2019-12-03 01:03:00,779 WARN  org.apache.flink.yarn.YarnClusterDescriptor                   - Neither the HADOOP_CONF_DIR nor the YARN_CONF_DIR environment variable is set. The Flink YARN Client needs one of these to be set to properly load the Hadoop configuration for accessing YARN.
2019-12-03 01:03:00,909 INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at master.hadoop/127.0.0.1:8050
2019-12-03 01:03:03,268 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:04,270 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:05,273 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:06,275 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:07,278 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:08,280 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:09,282 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:10,284 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:11,287 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:12,289 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:43,371 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:44,385 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:45,399 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:46,402 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:47,404 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:48,407 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:49,409 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:50,413 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:51,415 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:03:52,418 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:23,429 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:24,431 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:25,434 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:26,436 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:27,440 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:28,443 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:29,446 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:30,448 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:31,451 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:04:32,454 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:03,465 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:04,467 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:05,470 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:06,472 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:07,476 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:08,480 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:09,483 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:10,485 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:11,488 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:12,491 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:43,502 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:44,507 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:45,512 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:46,515 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:47,518 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:48,520 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:49,523 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:50,528 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:51,530 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:05:52,533 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:23,542 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:24,545 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:25,548 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:26,553 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:27,556 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:28,561 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:29,563 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:30,565 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:31,568 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-12-03 01:06:32,570 INFO  org.apache.hadoop.ipc.Client                                  - Retrying connect to server: master.hadoop/127.0.0.1:8050. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
^[^Z
[1]+  Stopped                 ./bin/yarn-session.sh -n 1 -tm 1024 -s 2
[root@master flink-1.4.0]#

 

spencer_tseng
关注
retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
u012516914的专栏
07-29 4091
[root@qa bin]# hadoop fs -ls / Warning: $HADOOP_HOME is deprecated. 14/07/29 13:25:35 INFO ipc.Client: Retrying connect to server: centhost.centdomain/10.110.30.30:9000. Already tried 0 time(s); retr
CDH集群下hadoop异常PriviledgedActionException as:root (auth:SIMPLE)解决
03-27 6715
在hadoop的应用平台看到的错误如下，根据这个错误参考了网络上的一些解决方法，没有解决这个问题，后来找了下yarn的日志 Application application_1553509480321_0011 failed 2 times due to AM Container for appattempt_1553509480321_0011_000002 exit...
flink1.10三节点集群之yarn-session模式
ASN_forever的博客
05-19 1万+
提前搭建Hadoop集群环境，可以参考这篇文章。 本文章的集群环境： 机器 ip 服务 flink1 172.21.89.128 flink2 172.21.89.129 flink3 172.21.89.130 安装配置flink 安装参考这篇文章 启动一个yarn session（分配1g的jobmanager和4g的taskmanager） [root@flink1 flink-1.10.0]# ./bin/yar...
Flink继续填坑之hadoop又出问题了
i_monster的博客
04-18 3799
最近主机坏掉，尝试修改了一下配置，本来的服务器发生故障无法运行，想修改换到另一台电脑上，但是在修改的过程中执行hadoop语句产生了一个错误本来我以为这个是因为免密登陆产生的原因，于是就重新做了一遍免密登陆的操作，但是之后还是这个问题，至于问题的原因并不清楚，再慢慢查现在赶紧把方法记录下来，真的是很简单，就是把图上那个阴影部分代码复制运行一边就可以了，额，人生之路太过于艰辛，有一个大佬能传授传授学...
Flink学习9：配置idea开发flink-Scala程序环境
hzp666的博客
08-02 3052
我的CSDN资源中有ideeeeeeaaaaa。
flink1.18.1安装包
最新发布
05-29
Flink 1.18.1 是一个高度可扩展的开源流处理框架，适用于实时数据流分析和批处理。在Linux环境下安装Flink 1.18.1，需要遵循以下步骤，确保系统具备必要的环境并正确配置。 首先，确保你的Linux系统上已经安装了...
flink1.15.4
08-09
《Flink 1.15.4在Linux环境中的应用详解》 Apache Flink是一个流行的开源流处理框架，被广泛用于实时数据处理和分析。在Linux操作系统中部署和使用Flink，可以充分利用其高效、可扩展和高可用的特性。Flink 1.15.4...
flink-1.16.2下载安装包，flink-1.16.2-bin-scala-2.12.zip
02-29
Apache Flink是一个流行的开源流处理和批处理框架，它提供了高度容错的数据处理能力，能够实时处理大规模数据流。Flink 1.16.2是该软件的一个稳定版本，针对Scala 2.12构建，这意味着它可以充分利用Scala语言的强大...
flink读取kafka数据.zip
01-04
在大数据处理领域，Apache Flink 是一款强大的流处理框架，它可以高效地处理实时和批处理数据。本项目涉及的关键技术包括Flink与Kafka的集成、数据计算后的存储以及流式数据导入MySQL。以下是这些技术的详细说明： ...
flink-1.19.0-bin-scala-2.12.tgz flink-1.16.3-bin-scala-2.12.tgz
05-21
标题和描述中提到的"flink-1.19.0-bin-scala-2.12.tgz"和"flink-1.16.3-bin-scala-2.12.tgz"是Apache Flink的两个不同版本的二进制发行版。Flink是一个开源的流处理和批处理框架，它提供了低延迟、高吞吐量的数据...
Flink 的yarn 两种提交方式
wudonglianga的专栏
03-30 3681
Flink 的yarn 两种提交方式 标题第一种 1.yarn-session.sh + flink run(提交任务) 命令： 优缺点： 特点：需要事先申请资源，启动JobManager和TaskManger 优点：不需要每次递交作业申请资源，而是使用已经申请好的资源，从而提高执行效率 缺点：作业执行完成以后，资源不会被释放，因此一直会占用系统资源 应用场景：适合作业递交比较频繁的场景，小作业比较多的场景 /opt/module/flink-1.12.0/bin/yarn-session.sh -n
PriviledgedActionException as:root (auth:SIMPLE) cause:0: No such file or directory
okakio的博客
11-12 1720
刚学MapReduce. 写WordCount的代码(代码没问题的情况下) , 运行后报错: 错误代码 19/11/12 07:57:19 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 19/11/12 07:57:19 INFO jvm.JvmMetric...
hadoop3的简单安装方法（单节点）
热门推荐
lucien7l的博客
06-24 1万+
前言    目前，网络上关于Hadoop安装的资料很多，过程比较详细，各种问题的解决方案基本都能找到，但资料质量参差不齐，随着hadoop版本的更新，有些方法已经不适用。lz初学hadoop，在这里面碰到不少坑，在此对hadoop安装方法进行梳理，既作为自己看的笔记，也希望能与大家交流。安装过程依照Apache官网的安装文档，对安装中遇到的问题，见招拆招。安装环境    电脑系统：windows ...
Hadoop1.2.1配置用户认证——simple类型
weixin_34306676的博客
08-03 577
2019独角兽企业重金招聘Python工程师标准>>> ...
Hadoop初入门的坑
jiayoudeBOKE的博客
01-06 6530
这几天初入门hadoop，测试一些demo遇到了许多问题，有些问题想了好久才找到办法解决，因此写下这篇入坑小博客，学习编程，解决问题是我们的核心，所以我们遇到难题应该兴奋并解决之。以后我要写多点博客，记录编程的点点滴滴，愿与各位共勉之。下面，正题。 ！！！hadoop中遇到问题一定要找相应的日志！！！ 一、环境 1、一个linux系统(Centos7)的服务器(不是虚拟机)，下面贴出/e
ambari部署大数据私有化集群运行Flink报错解决处理方案
梦想成真那天
06-29 1139
报错如下: [ansible@ALIBJ-TEST-OnPre-BigData1 bin]$ ./start_hour.sh Setting HADOOP_CONF_DIR=/etc/hadoop/conf because no HADOOP_CONF_DIR was set. Using the result of 'hadoop classpath' to augment the Hadoop classpath: /usr/hdp/3.1.4.0-315/hadoop/conf:/usr/hdp/3.
Hadoop yarn nodemanager启动提示 Retrying connect to server: 0.0.0.0/0.0.0.0:8031.
cl723401的博客
10-06 3544
Hadoop yarn nodemanager启动提示 Retrying connect to server: 0.0.0.0/0.0.0.0:8031.  INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8031 2018-10-06 08:52:33,562 INFO...
Flink数据转换详解
"Flink Data Transformation 是 Apache Flink 中的核心概念，它涉及到对数据流的一系列操作，用于将原始数据转化为有价值的信息。Flink作为一个强大的大数据处理框架，其数据转换功能对于构建实时数据处理管道至关...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值