关闭

Mac 环境变量设置

文章转载处:http://www.cnblogs.com/shineqiujuan/p/4693404.html  感谢原作者: mac 一般使用bash作为默认shell Mac系统的环境变量,加载顺序为: /etc/profile /etc/paths ~/.bash_profile ~/.bash_login ~/.profile ~/.bashrc 当然/...
阅读(37) 评论(0)

数据序列化avro\thrift\protocol buffer | kyro

Thrift 不仅仅是个高效的序列化工具,它是一个完整的 RPC (跨进程间服务调用)框架体系   (facebook http://dongxicheng.org/search-engine/thrift-framework-intro/ http://www.cnblogs.com/cyfonly/p/6059374.html https://thrift.apache.org/...
阅读(109) 评论(0)

how to make run command include the provided dependencies when using sbt

The easiest way to add provided dependencies to debug a task with IntelliJ is to: Right-click src/main/scala Select Mark Directory as... > Test Sources Root This tells IntelliJ to treat src/mai...
阅读(39) 评论(0)

Java多线程内存模型学习

转载出处: http://www.cnblogs.com/hewenwu/p/3662529.html  good! 特此感谢 reference: http://rainyear.iteye.com/blog/1734311  good!                  http://www.cnblogs.com/huanmin/p/6156192.html...
阅读(45) 评论(0)

Iterable和iterator

(1) Iterable:大部分collection直接实现了改接口(trait)  A base trait for iterable collections. This is a base trait for all Scala collections that define an iterator method to step through one-by-one ...
阅读(37) 评论(0)

linux查看java进程|线程池信息

1.根据端口号57638查看对应进程pid/name=1463/java         netstat -apn | grep  57638   tcp        0      0 ::ffff:172.20.5.10:57638    :::*                        LISTEN      1463/java            tcp        0...
阅读(52) 评论(0)

spark 读取alluxio: Caused by: java.io.IOException: Block 38416150429696 is not available in Alluxio

使用Alluxio, 读取文件报错: 30-09-2017 10:41:14 CST gpsWarning INFO - Driver stacktrace: 30-09-2017 10:41:14 CST gpsWarning INFO - at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGSch...
阅读(39) 评论(0)

How_to_design_partition_key_in_cassandra

-.  how to design partition key in cassandra? 1. Cassandra Data Modeling – Primary , Clustering , Partition , Compound Keys https://blog.knoldus.com/2016/10/18/cassandra-data-modeling-primar...
阅读(31) 评论(0)

cassandra materialized view And index

CREATE TABLE mytest1.wordstest (     word text PRIMARY KEY,     c1 int,     count int ) -. index 1. 不指定partition, 普通查询 select * from mytest1.wordstest  where count = 40 ALLOW FILTER...
阅读(42) 评论(0)

cassandra Internal data structure

Understanding How CQL3 Maps to Cassandra’s Internal Data Structure ? http://opensourceconnections.com/blog/2013/07/24/understanding-how-cql3-maps-to-cassandras-internal-data-structure/ h...
阅读(39) 评论(0)

hdfs 小文件合并

(1)使用hive insert overwrite  insert overwrite table test select * from test  (2)使用hdfs 命令: 合并src目录下的多个小文件 为 本地磁盘的一个文件 hdfs dfs  -getmerge     src: 是hdfs 目录 localdst: 是本地磁盘文件 example: hdfs...
阅读(29) 评论(0)

hive 文件存储parquet格式, sparkSql解析部分字段为null

hive 版本2.1.0, hdfs 上同一份数据 (1) 用hive 建表, 部分字段hive 读取有值, sparksql读取值为null CREATE EXTERNAL TABLE business_log.rule_business_log2 (                            id bigint,                             bu...
阅读(33) 评论(0)

hive几种存储格式: Storage Format

(1) hive table 的几种 Storage Format: Hive supports built-in and custom-developed file formats. See CompressedStorage for details on compressed table storage. The following are some of the formats bu...
阅读(27) 评论(0)

linux vim查找替换撤销

查找·替换详解 转载文章出处: http://www.cnblogs.com/longjshz/p/5775584.html 感谢原作者 (1) Linux下的vi编辑命令中查找·替换详解 一、查找  查找命令  /pattern :向下查找pattern匹配字符串  ?pattern:向上查找pattern匹配字符串  使用了查找...
阅读(35) 评论(0)

sbt assembly遇到的两个问题

1.  deduplicate file appears in two different jar: If multiple files share the same relative path (e.g. a resource named application.conf in multiple dependency JARs), the default strategy is to ...
阅读(65) 评论(0)

scala 关键字seal

简单记一下 1.  http://www.cnblogs.com/rollenholt/p/4192758.html 2. case class override toString case class Person(id: Int, id2: Int) { override def toString = this.productIterator.mkStr...
阅读(34) 评论(0)

scala/java 可变长度参数

1.  转载文章出处:http://www.cnblogs.com/rollenholt/p/4112833.html Scala 允许你指明函数的最后一个参数可以是重复的。这可以允许客户向函数传入可变长度参数列表。想要标注一个重复参数,在参数的类型之后放一个星号。例如: scala> def echo(args: String*) = for (arg <- args) println(...
阅读(47) 评论(0)

What does [B >: A] do in Scala?

(1) down voteaccepted [B >: A] is a lower type bound. It means that B is constrained to be a supertype of A. B 是A的超类 (2) Similarly [B  is an upper type bound, mea...
阅读(40) 评论(0)

cassandra节点异常:java.lang.OutOfMemoryError: Direct buffer memory

宕机两次了: 频繁发送多次请求到cassandra集群, 整个集群down掉, 查看日志发现: debug.log文件: ERROR [ReadStage-146] 2017-09-13 10:40:08,415 JVMStabilityInspector.java:141 - JVM state determined to be unstable.  Exiting for...
阅读(68) 评论(0)

sbt项目中多模块(多个子项目)配置

http://www.scala-sbt.org/0.13/docs/Multi-Project.html (官方文档永远是最好的, 完整的 看不懂多看几遍) http://www.scala-sbt.org/0.13/docs/zh-cn/Multi-Project.html  中文翻译 http://wiki.jikexueyuan.com/project/sbt-ge...
阅读(74) 评论(0)
57条 共3页1 2 3 下一页 尾页
    个人资料
    • 访问:7792次
    • 积分:558
    • 等级:
    • 排名:千里之外
    • 原创:47篇
    • 转载:10篇
    • 译文:0篇
    • 评论:2条
    文章分类
    最新评论