spark java 日志打印,Spark Java应用程序UDF中的日志未出现在控制台或 executor日志文件中...

log4j.rootLogger=INFO, Console_Appender, File_Appender

log4j.appender.Console_Appender=org.apache.log4j.ConsoleAppender

log4j.appender.Console_Appender.Threshold=INFO

log4j.appender.Console_Appender.Target=System.out

log4j.appender.Console_Appender.layout=org.apache.log4j.PatternLayout

log4j.appender.Console_Appender.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

log4j.appender.File_Appender=org.apache.log4j.rolling.RollingFileAppender

log4j.appender.File_Appender.Threshold=INFO

log4j.appender.File_Appender.File=/opt/spark_log/app_log.txt

log4j.appender.File_Appender.RollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy

log4j.appender.File_Appender.TriggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy

log4j.appender.File_Appender.RollingPolicy.FileNamePattern=/opt/spark_log/app_log.%d{MM-dd-yyyy}.%i.txt.gz

log4j.appender.File_Appender.RollingPolicy.ActiveFileName=/opt/spark_log/app_log.txt

log4j.appender.File_Appender.TriggeringPolicy.MaxFileSize=1000

log4j.appender.File_Appender.layout=org.apache.log4j.PatternLayout

log4j.appender.File_Appender.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c - %m%n

log4j.logger.myLogger=INFO,File_Appender

Set the default spark-shell log level to WARN. When running the spark-shell, the

log level for this class is used to overwrite the root logger's log level, so that

the user can have different defaults for the shell and regular Spark apps.

log4j.logger.org.apache.spark.repl.Main=WARN

Settings to quiet third party logs that are too verbose

log4j.logger.org.spark-project.jetty=WARN

log4j.logger.org.spark-project.jetty.util.component.AbstractLifeCycle=ERROR

log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFO

log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO