http://blog.csdn.net/eric_sunah/article/details/7262486
http://www.educity.cn/wenda/430259.html
http://blog.csdn.net/wficeboy/article/details/5692475
http://note.youdao.com/iyoudao/?p=439&keyfrom=cidian
http://blog.csdn.net/ttlyfast/article/details/1800890
http://blog.chinaunix.net/uid-27164517-id-3299073.html
http://blog.csdn.net/songjinshi/article/details/6816776
window--preferences--java--Installed JREs 选中jre.1.6点击Edit,
在Default VM arguments:中添加:-Xmx1024M -Xms512M -XX:MaxPermSize=256m
--------------------------------------------------------------------------------------------------------------------------------------------------------------
15/01/19 11:15:07 ERROR security.UserGroupInformation: PriviledgedActionException as:Administrator cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator2075593207\.staging to 0700
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator2075593207\.staging to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:918)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at com.litsoft.mapreduced.WordCountTest.main(WordCountTest.java:33)
解决方法是,修改/hadoop-1.0.4/src/core/org/apache/hadoop/fs/FileUtil.java里面的checkReturnValue,注释掉即可(有些粗暴,在Window下,可以不用检查)
- ......
-
private static void checkReturnValue(boolean rv, File p, -
FsPermission permission -
) throws IOException { -
-
} - ......
--------------------------------------------------------------------------------------------------------------------------------------------------------------
运行时如果报
java.lang.RuntimeException: java.lang.NoSuchMethodException: com.litsoft.mapreduced.WordCountTest$MyMapper.<init>()
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:214)
Caused by: java.lang.NoSuchMethodException: com.litsoft.mapreduced.WordCountTest$MyMapper.<init>()
at java.lang.Class.getConstructor0(Unknown Source)
at java.lang.Class.getDeclaredConstructor(Unknown Source)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
... 3 more
是由于class MyMapper extends Mapper<LongWritable,Text,Text,LongWritable>{}类前没加static,所以刚开始就没初始化,但是加上static的话,就出初始化,也可以不加static,把MyMapper变成外面的类,不是内部类:直接class MyMapper extends Mapper<LongWritable,Text,Text,LongWritable>{},然后job.setReducerClass(new MyReducer().getClass());job.setMapperClass(new MyMapper().getClass());就可以了,参考:
初始化被延迟到了对静态方法(构造器隐式地是静态的)或者非常数静态域进行首次引用时才执行
--------------------------------------------------------------------------------------------------------------------------------------------------------------
hadoop jar src,这里的src是指linux下的路径,不是hdfs下的路径
--------------------------------------------------------------------------------------------------------------------------------------------------------------
http://www.haodaima.net/art/2614232 -hadoop源码解析
-------------------------------------------------------------------------------------------------------------------------------------------------------------
http://blog.sina.com.cn/s/blog_b30d7fc80101df9n.html oracle安装
jms:http://leonmau.iteye.com/blog/910275
hive ha :http://www.aboutyun.com/thread-10938-1-1.html
hive 操作 :http://blog.csdn.net/xiaoping8411/article/details/7605039 ----------2015-03-25
http://www.cnblogs.com/oftenlin/p/3592005.html----hadoop缓存机制