2013年06月_天外有菌

11月 10月 09月 08月 07月 06月 05月

原创【Hadoop】Why Writable Interface

在前面学习RPC系统的时候，可以看到client和server相互通讯都是用Writable类型来序列化。实际上，Writable Interface是一种Hadoop内置的序列化机制，MapReduce中的key, value都需要是Writable。最初一直不太理解，为何Hadoop需要自定义一种序列化机制，而不重用Java内置的序列化机制。《Hadoop The Definiti

2013-06-08 16:51:03 1357 2

原创【Notes】Effective Java 2nd -- Concurrency

Item 66: Synchronize access to shared mutable data1. Not only does synchronization prevent a thread from observing an object in an inconsistent state, but it ensures thateach thread entering a synch

2013-06-03 11:59:20 1112

原创 A Hadoop data pipeline to analyze applicaction performance

1. IntroductionIn recent years, Hadoop has been under the spotlight for its flexible and scalable architecture to store and process big data on commodity machines. One of its common use cases is to

2013-06-01 21:50:51 3066

原创【Hadoop】RPC Client Workflow

Client端比Server端简单，只有两种线程：Client线程：发起RPC的主线程Connection线程：每一个ConnectionId (由组成)唯一对应一个Connection线程，用以接受server端的response如上图所示，基本的工作流程如下:Client线程将输入参数值封装成Call，Call在原来的基础上增加了RPC完成标志和RPC返

2013-06-01 15:40:18 1917