2013年01月_macyang

转载 session介绍

目录一、Session二、Cookies三、Cookies机制四、Session机制五、Cookies机制与Session机制的区别和联系六、常见问题七、Session的用法 Session是WEB上有效的信息交互手段，因其使用方便、稳定、安全、可靠而被众多WEB开发者所认知。尤其在互联网身份验证、网上电子购物等方面的应用更

2013-01-31 23:11:25 1860

转载 Apache Kafka --- A high-throughput distributed messaging system.

Why we built thisKafka is a messaging system that was originally developed at LinkedIn to serve as the foundation for LinkedIn's activity stream and operational data processing pipeline. It is now

2013-01-27 20:26:30 6525

转载 A utility for importing/exporting between hbase and csv file

除了可以使用hbase自带的org.apache.hadoop.hbase.mapreduce.TsvImporterMapper，下面这个实现的很基础也很不错，不过是单线程的。HBase is a NOSQL distributed database system that runs a cluster of machines on top of Hadoop. HBase and

2013-01-20 19:32:45 6425 1

转载 HBase性能优化方法总结

本文主要是从HBase应用程序设计与开发的角度，总结几种常用的性能优化方法。有关HBase系统配置级别的优化，这里涉及的不多，这部分可以参考：淘宝Ken Wu同学的博客。[转发者注明：关于使用多线程去读取hbase全表数据，推荐先将rowkey根据线程的个数划分为多段，然后将每段 start-key ~ end-key丢给线程去执行！]1. 表的设计1.1 Pre-Cre

2013-01-19 23:17:22 3858 1

转载 HADOOP-HBase MapReduce Examples

1. HBase MapReduce Read ExampleThe following is an example of using HBase as a MapReduce source in read-only manner. Specifically, there is a Mapper instance but no Reducer, and nothing is being emi

2013-01-19 20:34:00 2736

转载 An Unorthodox Approach To Database Design : The Coming Of The Shard

Update 4: Why you don’t want to shard. by Morgon on the MySQL Performance Blog.Optimize everything else first, and then if performance still isn’t good enough, it’s time to take a very bitter medici

2013-01-18 22:35:41 738

转载 Understanding the Java ClassLoader

Tutorial tipsold I take this tutorial?The Java ClassLoader is a crucial, but often overlooked, component of the Java run-time system. It is the class responsible for finding and loading class file

2013-01-15 23:56:52 549

转载 Hadoop是怎么分块的

hadoop的分块有两部分，其中第一部分更为人熟知一点。第一部分就是数据的划分（即把File划分成Block），这个是物理上真真实实的进行了划分，数据文件上传到HDFS里的时候，需要划分成一块一块，每块的大小由hadoop-default.xml里配置选项进行划分。 dfs.block.size 67108864 The default blo

2013-01-14 14:10:46 481

转载 Understanding the parallelism of a Storm topology

In the past few days I have been test-driving Twitter’s Storm project, which is a distributed real-time data processing platform. One of my findings so far has been that the quality of Storm’s docum

2013-01-14 13:47:31 654

转载 What are some time-saving tips that every Linux user should know?

Here is a selection of command-line tips that I've found useful when working on Linux. The emphasis is on somewhat less-known techniques that are generally important or useful to technical users. It

2013-01-13 22:09:26 732

转载 What have been Facebook's greatest technical accomplishments?

I ran the infrastructure software team at facebook for five years, and was involved in most of the projects listed in the other answers. I consider the greatest accomplishment while I was there to b

2013-01-13 22:08:35 575

转载 Signing JAR Files

You use the JAR Signing and Verification Tool to sign JAR files. You invoke the JAR Signing and Verification Tool by using the jarsigner command, so we'll refer to it as "Jarsigner" for short.To sig

2013-01-13 21:59:39 671

转载 Java keytool and keystore tutorials

Summary: A collection of Java keytool and keystore tutorials, including genkey, export, list and import examples.I've been working with the Java keytool command a lot lately, as I needed to learn

2013-01-13 21:42:20 2504

转载 jarsigner - JAR 签名和校验工具

jarsigner - JAR 签名和校验工具为 Java 归档 (JAR) 文件产生签名，并校验已签名的 JAR 文件的签名。结构jarsigner [ options ] jar-file aliasjarsigner -verify [ options ] jar-file说明jarsigner为 Java 归档 (JAR) 文

2013-01-11 22:37:06 5029

转载自定义java.policy配置(如何让你的类禁止读写文件？禁止创建Socket对象？)

最近做的项目有一个特殊的需求，由于系统是CS的，客户端需要自己编写JAVA类上传至服务器运行，整个流程如下：1.客户端编写JAVA类，并实现特定接口2.由客户端的JAVA编译器将其编译成class3.客户端通过socket将class文件上传至服务器4.服务器将class文件放至自己的classpath中，加载并执行问题出在最后一步上，服务器虽然由我们完全控

2013-01-08 11:42:36 2368 1

转载 Scalable Web Architecture and Distributed Systems

Open source software has become a fundamental building block for some of the biggest websites. And as those websites have grown, best practices and guiding principles around their architectures have e

2013-01-06 21:46:42 630

转载 Scaling Up And Out

Most attention today is focused on adding nodes or cloud instances to scale out systems. Guest editor Nikita Shamgunov emphasizes the importance of scaling systems vertically as well.Recentl

2013-01-06 21:10:21 637

转载 Nginx源码分析-内存池

Nginx的内存池实现得很精巧，代码也很简洁。总的来说，所有的内存池基本都一个宗旨：申请大块内存，避免“细水长流”。一、创建一个内存池nginx内存池主要有下面两个结构来维护，他们分别维护了内存池的头部和数据部。此处数据部就是供用户分配小块内存的地方。//该结构用来维护内存池的数据块，供用户分配之用。 typedef struct { u_char *last; //当

2013-01-03 22:41:50 600

转载网站统计中的数据收集原理及实现

网站数据统计分析工具是网站站长和运营人员经常使用的一种工具，比较常用的有谷歌分析、百度统计和腾讯分析等等。所有这些统计分析工具的第一步都是网站访问数据的收集。目前主流的数据收集方式基本都是基于javascript的。本文将简要分析这种数据收集的原理，并一步一步实际搭建一个实际的数据收集系统。数据收集原理分析简单来说，网站统计分析工具需要收集到用户浏览目标网站的行为（如打开某

2013-01-03 22:24:14 1551

转载 Nginx @tbdata

Nginx启动初始化过程（一）Nginx启动初始化过程(二)Nginx进程管理之master进程Nginx进程管理之worker进程Nginx的master和worker进程间的通信Nginx事件驱动的初始化Nginx的connections数组Nginx源码分析-事件循环Nginx源码分析-Epoll模块Nginx源码分析-内存池http://www.alidata.org/arch

2013-01-03 11:20:17 540

转载 Damn Cool Algorithms: Cardinality Estimation

Suppose you have a very large dataset - far too large to hold in memory - with duplicate entries. You want to know how many duplicate entries, but your data isn't sorted, and it's big enough that sort

2013-01-01 20:01:52 704

原创 2013年度技术规划的主题

1） linux内核方面的知识. e.g: 中断、调度、系统调用、内存管理等等2) python coding e.g: 成为资深用户3）分布式系统相关知识积累. e.g: hadoop, storm4）数据挖掘和算法方面的学习虽然这些对我来说早已经入门了，很多相关知识也积累了很久，但是还是想将它们继续作为自己成长的一部分，继续学习，不能因为其他事情把它们丢弃：）

2013-01-01 13:35:53 490

Mac Track