自定义博客皮肤VIP专享

*博客头图：

格式为PNG、JPG，宽度*高度大于1920*100像素，不超过2MB，主视觉建议放在右侧，请参照线上博客头图

请上传大于1920*100像素的图片！

博客底图：

图片格式为PNG、JPG，不超过1MB，可上下左右平铺至整个背景

栏目图：

图片格式为PNG、JPG，图片宽度*高度为300*38像素，不超过0.5MB

主标题颜色：

RGB颜色，例如：#AFAFAF

Hover：

RGB颜色，例如：#AFAFAF

副标题颜色：

RGB颜色，例如：#AFAFAF

预览 取消 提交

自定义博客皮肤

-+
上一步保存

ae5555的专栏

  • 博客(8)
  • 资源 (7)
  • 收藏
  • 关注
全部文章> 2015年05月
排序：
按最后发布时间
按访问量
RSS订阅
12月 11月 10月 09月 08月 07月 06月 05月 04月 03月 02月 01月

原创 Hadoop优化

1.hadoop的inputformat提供了在block的基础上更细粒度控制mapper输入块的大小,设置split的大小范围。FileInputFormat.setMaxInputSplitSize(job,size)FileInputFormat.setMinInputSplitSize(job,size)这种方法可以控制mapper的数量。也可以在配置文件中配置，mapre

2015-05-19 17:36:52 515

原创 Spark/Java8 lambda表达式

lambda表达式的本质是一个“语法糖”，由编译器转换为常规的代码，由此可以编写更少的代码来实现同样的功能。lambda表达式是SAM类型，SAM类型是一个具有单一抽象方法的接口。lambda表达式在运行期表示为一个函数接口，函数接口是一种只定义一个抽象方法的接口。同时，增加了新注解@FunctionalInterface，防止往这个接口添加方法。lambda表达式的语法由参数列表

2015-05-19 16:47:47 1570

原创 JSoup 代码总结

常用的html解析器有jsoup、htmlparser等。File  input=new File("xxx.html");Document   doc=Jsoup.parse(input,"UTF-8","http://xxx");Document   doc=Jsoup.connect("http://xxx").get();查找元素Element   conten

2015-05-18 16:40:15 406

原创 Hdfs/代码总结

1.计算hdfs上文件大小对每个文件，filestatus.getLen()。2.获取当前正在运行job的数量Configuration conf=new Configuration();conf.set("mapred.job.tracker","URL");JobClient client=new JobClient(new JobConf(conf));cli

2015-05-18 16:23:47 465

转载 linux/shell

shell中执行一个字符串命令cmd="xxxx"$cmd或者eval $cmd

2015-05-18 15:50:49 342

原创 Spark/RDD

每一个Spark应用(Job),都是由一个驱动程序(Driver)构成，它运行用户的main函数。RDD，是一个元素集合，划分到集群的各个节点上，可以被并行操作。两种类型的RDD，并行集合(Scala集合)和hadoop数据集。这两种类型的RDD都可以通过相同的方式进行操作。并行集合并行集合是通过调用SparkContext的parallelize方法，在一个已经存在的Sc

2015-05-06 21:14:41 457

原创 Spark:Transformation和Action

Transformation操作：map(func)    对调用map的RDD数据集中的每个element都使用func，然后返回一个新的RDD。groupByKey(numTasks)  返回(K,Seq[V]),相当于hadoop中的reduce函数接收K-V对，输出K-Vlist。reduceByKey(func,[numTasks])   在groupByKey产生的(K,S

2015-05-06 20:39:55 408

原创 Spark

Spark特点1.Spark是为集群计算中的特定类型的工作负载而设计，即在并行操作之间重用工作数据集的工作负载。为了优化这些类型的工作负载，Spark引进了内存集群计算的概念，可在内存集群计算中将数据集缓存在内存中，以缩短访问延迟。2.Spark提供的数据集操作类型有很多种。3.Spark充分利用内存进行缓存，比较合适做迭代式的运算，有相当一部分机器学习算法是通过多次迭代收敛的算法，适

2015-05-05 14:44:21 631

0day安全:软件漏洞分析技术
0day安全:软件漏洞分析技术

2013-07-04

linux网络安全技术与实现.pdf
第 1 章 防火墙的基本概念 ····················································································· 1 1.1 TCP/IP的基本概念 ························································································································ 2 1.1.1 应用层 ···································································································································· 2 1.1.2 传输层 ···································································································································· 3 1.1.3 网络层 ···································································································································· 4 1.1.4 链路层 ···································································································································· 4 1.2 数据包传输 ····································································································································· 4 1.3 TCP、UDP及Socket的关系 ········································································································ 9 1.4 何谓防火墙 ··································································································································· 12 1.5 防火墙的判断依据 ······················································································································ 14 1.5.1 各层数据包包头内的信息 ·································································································· 14 1.5.2 数据包所承载的数据内容 ·································································································· 16 1.5.3 连接状态 ······························································································································ 16 1.6 防火墙的分类 ······························································································································· 17 1.6.1 数据包过滤防火墙 ·············································································································· 17 1.6.2 应用层防火墙 ······················································································································ 18 1.7 常见的防火墙结构 ······················································································································ 19 1.7.1 单机防火墙 ·························································································································· 19 1.7.2 网关式防火墙 ······················································································································ 20 1.7.3 透明防火墙 ·········

2013-03-15

HTTP协议详解.pdf
引言.................................................................................................................................................................... 3 一、HTTP 协议详解之URL 篇........................................................................................................................... 3 二、HTTP 协议详解之请求篇........................................................................................................................... 3 三、HTTP 协议详解之响应篇........................................................................................................................... 4 四、HTTP 协议详解之消息报头篇................................................................................................................... 5 1、普通报头.............................................................................................................................................. 5 2、请求报头.............................................................................................................................................. 6 3、响应报头.............................................................................................................................................. 7 4、实体报头.............................................................................................................................................. 7 五、利用telnet 观察http 协议的通讯过程.................................................................................................8 1、打开telnet......................................................................................................................................... 8 2、连接服务器并发送请求...................................................................................................................... 9 3、实验结果：.......................................................................................................................................... 9 4、注意事项............................................................................................................................................ 10 六、HTTP 协议相关技术补充.........................................................................................................................

2013-03-13

第9章_linux防火墙配置.pdf
linux防火墙配置 iptables防火墙命令行和图形界面配置

2013-03-13

WPF编程宝典——C# 2010版
WPF编程宝典——C# 2010版

2013-02-23

ARM7技术参考手册
ARM7技术参考手册

2012-11-13

空空如也

TA创建的收藏夹 TA关注的收藏夹

TA关注的人

提示
确定要删除当前文章？
取消 删除