- 博客(360)
- 资源 (40)
- 问答 (2)
- 收藏
- 关注
RSS订阅转载 Hadoop源码分析之二(RPC机制之Call处理)
上一篇介绍整个RPC Server端的处理机制。http://www.linuxidc.com/Linux/2013-01/77251.htm下面介绍在整个处理机制中怎么把具体的Request Call转换并调用到整体的实现逻辑。主要以NameNode Client PRC Server作为例子来说明，整个转换通过Google Protocol Buffer RPC来实现。
2016-06-21 18:25:45
1412
转载 Hadoop源码分析之一(RPC机制之Server)
网上已经有很多关于Hadoop源码分析的好文，在这记录的目的是把自己在看Hadoop源码时自己的一些体会，方便日后查询巩固。相关阅读：Hadoop源码分析之二(RPC机制之Call处理) http://www.linuxidc.com/Linux/2013-01/77252.htm想了解Hadoop是如何工作的，首先需要了解Hadoop RPC原理。Hadoop提供了一个统一的RP
2016-06-21 18:24:21
989
转载 基于柯西矩阵的Erasure Code技术详解
原文 http://alanwu.blog.51cto.com/3652632/1410132一、概述Erasure Code 可以应用于分布式存储系统中，替代多份数据拷贝的数据冗余方式，从而可以提高存储空间利用率。此外， Erasure code 还可以应用于传统 RAID 系统中，增加数据冗余度，支持多块盘同时发生故障，从而可以提高数据可靠性。采用范德蒙矩阵可以构
2016-06-05 18:52:48
2838
转载 呼之欲出！比Spark快10倍的Hadoop3.0有哪些实用新特性？
Apache hadoop 项目组最新消息，hadoop3.x以后将会调整方案架构，将Mapreduce 基于内存+io+磁盘，共同处理数据。其实最大改变的是hdfs,hdfs 通过最近black块计算，根据最近计算原则，本地black块，加入到内存，先计算，通过IO，共享内存计算区域，最后快速形成计算结果。1. Hadoop 3.0简介Hadoop
2016-06-05 16:29:30
3388
转载 HDFS EC：将纠删码技术融入HDFS
原文转自：http://geek.csdn.net/news/detail/77338在HDFS中，可靠性通过多副本的方式来实现，从而较低的存储利用率成为时下基于HDFS应用的主要问题之一。本文将详细介绍HDFS一个新的特性——Erasure Coding（EC）， 它在保证同等（或者更高）可靠性的情况下将存储利用率提高了近一倍。背景近些年，随着大数据技术的发展，HDFS作
2016-06-05 16:11:41
8401
转载 Reed Solomon纠删码
纠删码是存储领域常用的数据冗余技术， 相比多副本复制而言， 纠删码能够以更小的数据冗余度获得更高数据可靠性。 Reed Solomon Coding是存储领域常用的一种纠删码，它的基本原理如下： 给定n个数据块d1, d2,..., dn，n和一个正整数m， RS根据n个数据块生成m个校验块， c1, c2,..., cm。 对于任意的n和m， 从n个原始数据块和m 个校验块中任取n块就
2016-06-02 16:26:05
6068
转载 Erasure Coding（纠删码）深入分析
原文地址：Coding（纠删码）深入分析" style="text-decoration:none; color:rgb(42,112,155)">Erasure Coding（纠删码）深入分析作者：Eric雪菲1.前言Swift升级到2.0大版本后宣称开始支持纠删码，这其实是一个很有意义的特性，主要是能够在一定程度上解决3副本空间浪费太多的问题。因为3副本这一点是swift推
2016-06-02 16:23:43
4341
转载 Hadoop 3.0纠删码(Erasure Coding)：节省一半存储空间
随着大数据技术的发展，HDFS作为Hadoop的核心模块之一得到了广泛的应用。为了系统的可靠性，HDFS通过复制来实现这种机制。但在HDFS中每一份数据都有两个副本，这也使得存储利用率仅为1/3，每TB数据都需要占用3TB的存储空间。随着数据量的增长，复制的代价也变得越来越明显：传统的3份复制相当于增加了200%的存储开销，给存储空间和网络带宽带来了很大的压力。因此，在保证可靠性的前提下如何提高存
2016-06-02 15:20:01
11137
转载 Centos Yum安装Chrome浏览器
之前Google就说了，由于CentOS/RHEL 6已经是过期的系统，所以不再会有Chrome了。 虽然后来由于引起了社区的抗议，从而改口，不再提CentOS/RHEL 6是过期系统了；但是，目前在CentOS/RHEL 6上已经没有Chrome可以下载使用了。其实，根本的原因不是CentOS/RHEL 6有多老，连Windows XP和停止更新的Ubuntu 10.04都能继续使用Ch
2016-06-01 13:38:38
36666
1
原创 Centos 下设置各种代理上网
假设我们要设置代理为 IP：PORT1、网页上网网页上网设置代理很简单，在firefox浏览器下 Edit-->>Preferences-->>Advanced-->>Network在Connection下点击Settings，里面的manual proxy configuration里设置IP和PORT即可2、yum代理设置编辑文件为：/etc/yum.conf
2016-06-01 13:20:42
10279
1
原创 Java 调用本地Native接口指引
Java本地接口允许java用户用代码与其他语言接口交互的一种方法，native方法允许java语言调用其他语言，增加了代码的灵活性1、Java本地接口的好处通过使用Java接口，可以增加代码的灵活性与重用性，假如我们想在java里用一段C语言的代码实现的功能，我们不用全部重写所有C语言代码，而是可以加一个C语言接口类，与java交互即可。通过java接口，还可以提升java程序的性能
2016-05-31 20:00:56
2872
转载 protocol buffer安装
1.下载 Hadoop使用protocol buffer进行通信，需要下载和安装 protobuf-2.5.0.tar.gz。百度一下就有很多资源下载2.安装 使用tar -zxf protobuf-2.5.0.tar.gz命令解压后得到是 protobuf-2.5.0的源码， cd protobuf-2.5.0 进入目录 假如 你
2016-05-18 10:02:13
406
原创 Hadoop介绍
1、Hadoop介绍Hadoop是Apache软件基金会旗下的一个开源分布式计算平台，为用户提供了一个系统底层细节透明的分布式架构，通过Hadoop，可以将大量的廉价机器的计算资源组织起来，解决单机不能解决的海量数据的处理问题。Hadoop主要有以下几个优点：高可靠性：提供按位处理的存储和计算能力值得用户信赖。高扩展性：可以轻松地从小量集群扩展到数以千计的节点中。
2016-05-17 12:37:59
4215
原创 VNC Viewer 设置屏幕分辨率-解决屏幕分辨率问题
1.第一种方法：使用geometry参数进行调整vncserver -geometry 1280x1024即可，之后通过window下vnc连接后的ubuntu分辨率即为1280x1024了,注意这里的X是小写的x而不是*2.第二种方法：修改配置文件vncservers[root@secdb ~]# vi /etc/sysconfig/vncservers# The V
2016-05-16 15:14:08
107589
5
转载 VNC Viewer配置
转载自：http://www.cnblogs.com/kerrycode/p/3315793.htmlVNC概述 VNC (Virtual Network Computing)是虚拟网络计算机的缩写。VNC 是一款优秀的远程控制工具软件，由著名的 AT&T 的欧洲研究实验室开发的。VNC 是在基于 UNIX 和 Linux操作系统的免费的开源软件，远程控制能力强大，高效
2016-05-16 15:09:47
33928
原创 LeetCode 38 Count and Say(C,C++,Java,Python)
Problem:The count-and-say sequence is the sequence of integers beginning as follows:1, 11, 21, 1211, 111221, ...1 is read off as "one 1" or 11.11 is read off as "two 1s" or 21.21 i
2015-05-19 21:36:24
3507
原创 LeetCode 37 Sudoku Solver (C,C++,Java,Python)
Problem:Write a program to solve a Sudoku puzzle by filling the empty cells.Empty cells are indicated by the character '.'.You may assume that there will be only one unique solution.
2015-05-19 13:11:05
2909
原创 LeetCode 36 Valid Sudoku (C,C++,Java,Python)
Problem:Determine if a Sudoku is valid, according to: Sudoku Puzzles - The Rules.The Sudoku board could be partially filled, where empty cells are filled with the character '.'.A p
2015-05-18 22:56:46
3858
原创 LeetCode 35 Search Insert Position (C,C++,Java,Python)
Problem:Given a sorted array and a target value, return the index if the target is found. If not, return the index where it would be if it were inserted in order.You may assume no duplicates
2015-05-18 20:33:01
1708
原创 LeetCode 34 Search for a Range (C,C++,Java,Python)
Problem:Given a sorted array of integers, find the starting and ending position of a given target value.Your algorithm's runtime complexity must be in the order of O(log n).If the target
2015-05-14 21:28:11
1881
原创 LeetCode 33 Search in Rotated Sorted Array (C,C++,Java,Python)
Problem:Suppose a sorted array is rotated at some pivot unknown to you beforehand.(i.e., 0 1 2 4 5 6 7 might become 4 5 6 7 0 1 2).You are given a target value to search. If found in t
2015-05-14 19:59:49
703
原创 LeetCode 32 Longest Valid Parentheses (C,C++,Java,Python)
Problem:Given a string containing just the characters '(' and ')', find the length of the longest valid (well-formed) parentheses substring.For "(()", the longest valid parentheses substri
2015-05-14 17:16:32
1233
原创 LeetCode 31 Next Permutation (C,C++,Java,Python)
Problem:Implement next permutation, which rearranges numbers into the lexicographically next greater permutation of numbers.If such arrangement is not possible, it must rearrange it as the l
2015-05-14 14:36:42
2095
原创 LeetCode 30 Substring with Concatenation of All Words (C,C++,Java,Python)
Problem:You are given a string, s, and a list of words, words, that are all of the same length. Find all starting indices of substring(s) in s that is a concatenation of each word in wordsexac
2015-05-14 13:13:52
2106
原创 LeetCode 29 Divide Two Integers (C,C++,Java,Python)
Problem:Divide two integers without using multiplication, division and mod operator.If it is overflow, return MAX_INT.Solution:不能乘除就加减就行了，但是一个问题是加减有可能速度太慢，因此需要转换，由于任何一个数都能表示成二进制，所以有d
2015-05-13 09:27:20
2943
1
原创 LeetCode 28 Implement strStr() (C,C++,Java,Python)
Problem:Implement strStr().Returns the index of the first occurrence of needle in haystack, or -1 if needle is not part of haystack.Update (2014-11-02):The signature of the function ha
2015-05-12 11:59:17
2412
原创 LeetCode 27 Remove Element (C,C++,Java,Python)
Problem:Given an array and a value, remove all instances of that value in place and return the new length.The order of elements can be changed. It doesn't matter what you leave beyond the ne
2015-05-11 20:40:46
1957
原创 LeetCode 26 Remove Duplicates from Sorted Array (C,C++,Java,Python)
Problem:Given a sorted array, remove the duplicates in place such that each element appear only once and return the new length.Do not allocate extra space for another array, you must do this
2015-05-11 20:25:19
1114
原创 LeetCode 25 Reverse Nodes in k-Group (C,C++,Java,Python)
Problem:Given a linked list, reverse the nodes of a linked list k at a time and return its modified list.If the number of nodes is not a multiple of k then left-out nodes in the end should r
2015-05-11 19:55:08
1505
原创 LeetCode 24 Swap Nodes in Pairs (C,C++,Java,Python)
Problem:Given a linked list, swap every two adjacent nodes and return its head.For example,Given 1->2->3->4, you should return the list as 2->1->4->3.Your algorithm should use only con
2015-05-11 16:37:22
2214
1
原创 LeetCode 23 Merge k Sorted Lists (C,C++,Java,Python)
Problem:Merge k sorted linked lists and return it as one sorted list. Analyze and describe its complexity.Solution:采用胜者树的方法，胜者树介绍：胜者树，假如链表数组长度为n，链表元素总个数为k，那么时间复杂度为O(k*log(n))题目大意：
2015-05-11 15:39:00
1452
原创 LeetCode 22 Generate Parentheses (C,C++,Java,Python)
Problem:Given n pairs of parentheses, write a function to generate all combinations of well-formed parentheses.For example, given n = 3, a solution set is:"((()))", "(()())", "(())()", "
2015-05-10 21:11:15
2231
原创 LeetCode 21 Merge Two Sorted Lists (C,C++,Java,Python)
Problem:Merge two sorted linked lists and return it as a new list. The new list should be made by splicing together the nodes of the first two lists.Solution:两个有序链表，每次取头部最小的那个元素，然后将这个元素从原来链表
2015-05-10 17:15:00
1768
1
原创 LeetCode 20 Valid Parentheses (C,C++,Java,Python)
Problem:Given a string containing just the characters '(', ')', '{', '}', '[' and ']', determine if the input string is valid.The brackets must close in the correct order, "()" and "()[]{}
2015-05-10 16:41:00
4179
原创 LeetCode 19 Remove Nth Node From End of List (C,C++,Java,Python)
Problem:Given a linked list, remove the nth node from the end of list and return its head.For example, Given linked list: 1->2->3->4->5, and n = 2. After removing the second node fro
2015-05-10 16:07:27
1253
原创 LeetCode 18 4Sum (C,C++,Java,Python)
Problem:Given an array S of n integers, are there elements a, b, c, and d in S such that a + b + c + d = target? Find all unique quadruplets in the array which gives the sum of target.Note
2015-05-10 15:04:02
2488
原创 LeetCode 17 Letter Combinations of a Phone Number(C,C++,Java,Python)
Problem:Given a digit string, return all possible letter combinations that the number could represent.A mapping of digit to letters (just like on the telephone buttons) is given below.
2015-05-10 13:39:32
3817
原创 LeetCode 16 3Sum Closest(C,C++,Java,Python)
Problem:Given an array S of n integers, find three integers in S such that the sum is closest to a given number, target. Return the sum of the three integers. You may assume that each input woul
2015-05-09 22:26:02
2585
原创 LeetCode 15 3Sum (C,C++,Java,Python)
Problem:Given an array S of n integers, are there elements a, b, c in S such that a + b + c = 0? Find all unique triplets in the array which gives the sum of zero.Note:Elements in a
2015-05-09 21:06:57
5674
安装Wine时需要的Flex最新版
2013-05-10
Linux与Windows相互远程工具
2013-05-10
Nodejs实战经典书籍
2015-05-08
Pattern Recongition Quiz
2014-11-22
精通Windows Sockets网络开发：基于Visual C++实现-带源码
2013-05-21
leetcode C语言 int**是怎么用的？
2015-09-11
Ext中json解析如何获取属性名
2015-08-05
TA创建的收藏夹 TA关注的收藏夹
TA关注的人