#./happlier
a error is
[root@dmining05 mysql2hdfs]# ./happlier
The default data warehouse directory in HDFS will be set to /usr/hive/warehouse
Change the default data warehouse directory? (Y or N) n
Enter either Y or N:N
loadFileSystems error:
(unable to get stack trace for java.lang.NoClassDefFoundError exception: ExceptionUtils::getStackTrace error.)
hdfsBuilderConnect(forceNewInstance=0, nn=default, port=0, kerbTicketCachePath=(NULL), userName=(NULL)) error:
(unable to get stack trace for java.lang.NoClassDefFoundError exception: ExceptionUtils::getStackTrace error.)
Couldnot connect to HDFS file system
the error may be due to the following cause
http://stackoverflow.com/questions/21064140/hadoop-c-hdfs-test-running-exception
#echo $CLASSPATH
/root/hadoop-2.6.0/etc/hadoop:
/root/hadoop-2.6.0/share/hadoop/common/lib/*:
/root/hadoop-2.6.0/share/hadoop/common/*:
/root/hadoop-2.6.0/share/hadoop/hdfs:
/root/hadoop-2.6.0/share/hadoop/hdfs/lib/*:
/root/hadoop-2.6.0/share/hadoop/hdfs/*:
/root/hadoop-2.6.0/share/hadoop/yarn/lib/*:
/root/hadoop-2.6.0/share/hadoop/yarn/*:
/root/hadoop-2.6.0/share/hadoop/mapreduce/lib/*:
/root/hadoop-2.6.0/share/hadoop/mapreduce/*:
/root/hadoop-2.6.0/contrib/capacity-scheduler/*.jar
For hadoop versions 2.0.0 and above, the classpath doesn't support wild characters. If you add the jars explicitly to the CLASSPATH, your app will work.
Can JNI be made to honour wildcard expansion in the classpath?
One way is to build a file name setclass.sh
#!/bin/bash
hcp=$(hadoop classpath)
#echo $hcp
arr=(${hcp//:/ })
len=${#arr[@]}
let len-=1
echo $len
j=0
export CLASSPATH=/etc/hadoop/conf
for i in ${arr[@]}
do
# echo $i
if [ $j -eq 0 ]; then
export CLASSPATH=$i
elif [ $j -eq $len ]; then
echo $i
else
export CLASSPATH=$CLASSPATH:$i
fi
let j+=1
done
then source it, not to execute it
#source setclass.sh
#echo $CLASSPATH
This work well in current session, if want to make it work gloably,
#$echo $CLASSPATH >> ~/.bashrc
#source ~/.bashrc
open another consele to check it .
mysql configuration in /etc/my.cnf likes
[mysqld]
basedir = /usr/local/mysql
datadir = /var/lib/mysql
port = 3306
#socket = /var/lib/mysql/mysql.sock
user=mysql
bind_address = ::
#bin log conf
log_bin = masterbin_log
binlog_checksum = NONE
binlog_format = ROW
server-id = 2
#./happlier mysql://root@127.0.0.1:3306 hdfs://localhost:9000
the msg is
the right way is
#./happlier mysql://root:123456@127.0.0.1:3306 hdfs://localhost:9000
(you should provide password to access mysql)
but this warning still exists
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Refrences
http://innovating-technology.blogspot.com/2013/04/mysql-hadoop-applier-part-1.html
http://innovating-technology.blogspot.com/2013/04/mysql-hadoop-applier-part-2.html
http://paddy-w.iteye.com/blog/2023656
http://www.tuicool.com/articles/NfArA3i