自定义博客皮肤VIP专享

*博客头图:

格式为PNG、JPG,宽度*高度大于1920*100像素,不超过2MB,主视觉建议放在右侧,请参照线上博客头图

请上传大于1920*100像素的图片!

博客底图:

图片格式为PNG、JPG,不超过1MB,可上下左右平铺至整个背景

栏目图:

图片格式为PNG、JPG,图片宽度*高度为300*38像素,不超过0.5MB

主标题颜色:

RGB颜色,例如:#AFAFAF

Hover:

RGB颜色,例如:#AFAFAF

副标题颜色:

RGB颜色,例如:#AFAFAF

自定义博客皮肤

-+

蚂蚁绊大象

有备无患

  • 博客(10)
  • 资源 (2)
  • 收藏
  • 关注

原创 hadoop2.2.0 边角料(四)

看了一上午,终于找到了HDFS写一个新的block做了个啥。DFSClient传数据最终是在DFSOutputStream中完成的。它内部的DataStreamer来完成真正的传输工作。1.setup streamnextBlockOutputStreamlocateFollowingBlock调用namenode.addBlock向NN申请新的block,那么弄得会返回一个新的b

2014-01-07 14:42:03 757

原创 hadoop2.2.0 边角料(三)

hadoop 2.2.0 使用protobuf实现消息的串行化,这篇blog主要是以BlockChecksum为例,详细看一下如何发送请求,并接收响应。1. DFSClient在getFileChecksum()中连接datanode IOStreamPair pair = connectToDN(socketFactory, connectToDnViaHostname,

2014-01-03 11:01:08 858

原创 HDFS修改代码(一)

加入RsyncCopy的过程如下:编译过后把/home/ChenKang/RsyncHadoopCompile/hadoop-tools/hadoop-rsynccopy/target目录下的hadoop-rsynccopy-2.2.0-sources.jar拷贝到/home/ChenKang/hadoop-2.2.0/share/hadoop/tools/sources目录下ha

2014-01-02 16:55:28 733

转载 ssh-keygen的使用方法

一、概述1、就是为了让两个linux机器之间使用ssh不需要用户名和密码。采用了数字签名RSA或者DSA来完成这个操作2、模型分析假设 A (192.168.20.59)为客户机器,B(192.168.20.60)为目标机;要达到的目的:A机器ssh登录B机器无需输入密码;加密方式选 rsa|dsa均可以,默认dsa 二、具体操作流程

2014-01-02 13:20:36 656

原创 函数闭包原理

正在看...

2013-12-26 14:34:00 476

原创 hadoop2.2.0 边角料(二)

上一篇文章忘了提了,这一组博客是我为了要做一个hdfs的项目,对hadoop学习的一组文章。并且只关注了我觉得会用到的API和相关流程,主要是了解DFSClient的工作过程,涉及到DFSClient与NN和DN的通信。以前没有用过java,只了解一些基本语法,对hadoop的了解也比较粗浅,如果有什么理解错误的地方,希望看到这个博客的人能给我指出来,先谢过了。这次看的问题是如何从Datano

2013-12-23 16:59:29 789

原创 hadoop2.2.0 边角料(一)

hadoop dfs -copyFromLocal 是如何执行的?bin/hadoop脚本调用hdfs脚本hdfs调用org.apache.hadoop.fs.FsShell类FssShell的main函数调用res = ToolRunner.run(shell, argv);//shell是FsShell的一个实例ToolRunner.run调用的是FsShell中的run函数,其中

2013-12-23 14:05:11 1183

原创 git学习

本文是《版本控制之道-----使用git》的学习笔记基本概念:1.版本库:分为本地版本库和上游版本库2.工作目录3.标签:用标签记录项目的关键节点,是项目内部版本号的一个别名4.分支:git主分支默认为master,分支主要用于支持项目不同的发布版本,支持一个特定功能的开发5.合并:需要了解冲突处理6.索引(index):索引即暂存区(staging area),用

2013-12-21 17:10:31 587

原创 ubuntu 12.04 + hadoop 2.2 单机部署

http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/SingleCluster.html1.Mapreduce Tarball$ mvn clean install -DskipTests$ cd hadoop-mapreduce-project$ mvn clean install asse

2013-12-16 16:13:06 774

原创 Ubuntu 12.04下编译hadoop 2.2.0

本文大部分内容源自参考文献1。1.使用Maven3.1.1的时候出现了很多错误,果断放弃。我选择环境如下:Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 21:51:28+0800)Maven home: /home/hadoop/apache-maven-3.0.5

2013-12-16 14:16:48 1284

Google Hacks

Everyone knows that Google lets you search billions of web pages. But few people realize that Google also gives you hundreds of cool ways to organize and play with information. Since we released the last edition of this bestselling book, Google has added many new features and services to its expanding universe: Google Earth, Google Talk, Google Maps, Google Blog Search, Video Search, Music Search, Google Base, Google Reader, and Google Desktop among them. We've found ways to get these new services to do even more. The expanded third edition of Google Hacks is a brand-new and infinitely more useful book for this powerful search engine. You'll not only find dozens of hacks for the new Google services, but plenty of updated tips, tricks and scripts for hacking the old ones. Now you can make a Google Earth movie, visualize your web site traffic with Google Analytics, post pictures to your blog with Picasa, or access Gmail in your favorite email client. Industrial strength and real-world tested, this new collection enables you to mine a ton of information within Google's reach. And have a lot of fun while doing it: Search Google over IM with a Google Talk bot Build a customized Google Map and add it to your own web site Cover your searching tracks and take back your browsing privacy Turn any Google query into an RSS feed that you can monitor in Google Reader or the newsreader of your choice Keep tabs on blogs in new, useful ways Turn Gmail into an external hard drive for Windows, Mac, or Linux Beef up your web pages with search, ads, news feeds, and more Program Google with the Google API and language of your choice For those of you concerned about Google as an emerging Big Brother, this new edition also offers advice and concrete tips for protecting your privacy. Get into the world of Google and bend it to your will!

2010-12-23

南开大学的信号与系统课件

南开大学数学院的课件,还是不错的。 南开大学数学院的课件,还是不错的。

2010-03-11

空空如也

TA创建的收藏夹 TA关注的收藏夹

TA关注的人

提示
确定要删除当前文章?
取消 删除