camus执行任务，偶发性异常

异常信息：

*********************************

MapReduce异常信息

*********************************

2015-07-20 14:16:01,045 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.require.client.cert;  Ignoring.

2015-07-20 14:16:01,049 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.

2015-07-20 14:16:01,050 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.client.conf;  Ignoring.

2015-07-20 14:16:01,053 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.keystores.factory.class;  Ignoring.

2015-07-20 14:16:01,057 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.server.conf;  Ignoring.

2015-07-20 14:16:01,081 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.

2015-07-20 14:16:01,358 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties

2015-07-20 14:16:01,463 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).

2015-07-20 14:16:01,463 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started

2015-07-20 14:16:01,479 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:

2015-07-20 14:16:01,479 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1435737529069_0025, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@63a6d923)

2015-07-20 14:16:01,631 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.

2015-07-20 14:16:02,065 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /mnt/sdb1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sdc1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sdd1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sde1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sdf1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sdg1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025

2015-07-20 14:16:02,271 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.require.client.cert;  Ignoring.

2015-07-20 14:16:02,274 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.

2015-07-20 14:16:02,275 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.client.conf;  Ignoring.

2015-07-20 14:16:02,276 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.keystores.factory.class;  Ignoring.

2015-07-20 14:16:02,279 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.server.conf;  Ignoring.

2015-07-20 14:16:02,295 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.

2015-07-20 14:16:02,668 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id

2015-07-20 14:16:03,198 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]

2015-07-20 14:16:03,454 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: com.linkedin.camus.etl.kafka.mapred.EtlSplit@d9f20e5

2015-07-20 14:16:03,582 INFO [main] org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library

2015-07-20 14:16:03,584 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]

2015-07-20 14:16:03,595 INFO [main] com.linkedin.camus.etl.kafka.mapred.EtlRecordReader: 

topic:hive-xm-syslog5 partition:0 beginOffset:80 estimatedLastOffset:120

2015-07-20 14:16:03,603 INFO [main] com.linkedin.camus.etl.kafka.common.KafkaReader: bufferSize=10485760

2015-07-20 14:16:03,603 INFO [main] com.linkedin.camus.etl.kafka.common.KafkaReader: timeout=30000

2015-07-20 14:16:04,029 INFO [main] com.linkedin.camus.etl.kafka.common.KafkaReader: Connected to leader tcp://m105:9092 beginning reading at offset 80 latest offset=120

2015-07-20 14:16:05,657 INFO [main] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore

2015-07-20 14:16:05,702 INFO [main] org.apache.hadoop.hive.metastore.ObjectStore: ObjectStore, initialize called

2015-07-20 14:16:06,018 INFO [main] DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored

2015-07-20 14:16:06,019 INFO [main] DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored

2015-07-20 14:16:06,686 ERROR [BoneCP-pool-watch-thread] com.jolbox.bonecp.PoolWatchThread: Error in trying to obtain a connection. Retrying in 7000ms

java.sql.SQLException: A read-only user or a user in a read-only datab

ase is not permitted to disable read-only mode on a connection.

at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)

at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)

at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)

at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)

at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)

at org.apache.derby.impl.jdbc.EmbedConnection.setReadOnly(Unknown Source)

at com.jolbox.bonecp.ConnectionHandle.setReadOnly(ConnectionHandle.java:1324)

at com.jolbox.bonecp.ConnectionHandle.<init>(ConnectionHandle.java:262)

at com.jolbox.bonecp.PoolWatchThread.fillConnections(PoolWatchThread.java:115)

at com.jolbox.bonecp.PoolWatchThread.run(PoolWatchThread.java:82)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.sql.SQLException: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.

at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)

at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)

... 13 more

Caused by: ERROR 25505: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.

at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)

at org.apache.derby.impl.sql.conn.GenericAuthorizer.setReadOnlyConnection(Unknown Source)

at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.setReadOnly(Unknown Source)

... 8 more

原因及方案解决：

m102节点找不到对应的Hive表元数据地址，造成程序偶尔运行异常，在hive-site.xml中配置Hive元数据信息

  <property>

    <name>hive.metastore.uris</name>

    <value>thrift://m103:9083,thrift://m105:9083</value>

  </property>

转载于:https://my.oschina.net/u/1474514/blog/484253