小丑八怪的博客

数据如下：shop1,2019-01-18,500shop1,2019-02-10,500shop1,2019-02-10,200shop1,2019-02-11,600shop1,2019-02-12,400shop1,2019-02-13,200shop1,2019-02-15,100shop2,2019-02-10,100shop2,2019-02-11,100shop2,2019-02-13,100shop2,2019-03-15,100shop2,2019-04-15,10

数据如下+---+-------------------+-------------------+----+| id| start_time| end_time|flow|+---+-------------------+-------------------+----+| 1|2020-02-18 14:20:30|2020-02-18 14:46:30| 20|| 1|2020-02-18 14:47:20|2020-02-18 15:20:30|

部分数据展示uid,dtguid01,2018-03-01guid01,2018-02-28guid01,2018-03-01guid01,2018-03-04guid01,2018-03-02guid01,2018-03-05guid01,2018-03-06guid01,2018-03-07guid02,2018-03-01guid02,2018-03-02guid02,2018-03-03guid02,2018-03-06...import org.apache.spa

部分数据展示uid,dtguid01,2018-03-01guid01,2018-02-28guid01,2018-03-01guid01,2018-03-04guid01,2018-03-02guid01,2018-03-05guid01,2018-03-06guid01,2018-03-07guid02,2018-03-01guid02,2018-03-02guid02,2018-03-03guid02,2018-03-06...import org.apache.spa

IP规则数据1.0.1.0|1.0.3.255|16777472|16778239|亚洲|中国|福建|福州||电信|350100|China|CN|119.306239|26.0753021.0.8.0|1.0.15.255|16779264|16781311|亚洲|中国|广东|广州||电信|440100|China|CN|113.280637|23.1251781.0.32.0|1.0.63.255|16785408|16793599|亚洲|中国|广东|广州||电信|440100|China|CN

/** * 数据分析： * uid,startTime, endTime, downFlow, lag() over , flag , sum_over * 1,2020-02-18 14:20:30,2020-02-18 14:46:30,20, 2020-02-18 14:20:30 0 0 * 1,2020-02-18 14:47:20,2020-02-18 15:20:30,30, 2020-02-18 14:46:30 0 0 * 1,2020-02-18 15:37:

/** * 数据： * shop1,2019-01-18,500 * shop1,2019-02-10,500 * shop1,2019-02-10,200 * shop1,2019-02-11,600 * shop1,2019-02-12,400 * shop1,2019-02-13,200 * shop1,2019-02-15,100 * shop2,2019-02-10,100 * shop2,2019-02-11,100 * shop2,2019-02-13,100 * sh

第一种实现方式/** * guid01,2018-02-28 1 * guid01,2018-03-01 2 * guid01,2018-03-01 * guid01,2018-03-02 3 * guid01,2018-03-04 4 * guid01,2018-03-05 5 * guid01,2018-03-06 6 * guid01,2018-03-07 7 * * 思路：根据uid分组，组内按日期排序，开个窗口row_num * 日期减去row_num 得

spark的persist算子使用//设置Kryo序列化方式，占用内存小，默认使用java序列化sc.getConf.set("spark.serializer","org.apache.spark.serializer.KryoSerializer")val rdd = sc.textFile("hdfs://linux01:8020/data/teacher.log")import org.apache.spark.storage.StorageLevelrdd.persist(Storage

1 创建一个Maven项目2 在pom.xml中添加依赖和插件<!-- 定义了一些常量 --> <properties> <maven.compiler.source>1.8</maven.compiler.source> <maven.compiler.target>1.8</maven.compiler.target> <scala.version>2.1

hive端测试--hive端建表create table tb_log(log string)partitifirstd by (dt string);load data local inpath '/root/event.log' into table tb_log partition (dt='20201007');--解析json数据,并对数据进行预处理create table tb_ods_log asselectif(account='',deviceId,accoun

数据a,2017-02-05,200a,2017-02-06,300a,2017-02-07,200a,2017-02-08,400a,2017-02-10,600b,2017-02-05,200b,2017-02-06,300b,2017-02-08,200b,2017-02-09,400b,2017-02-10,600c,2017-01-31,200c,2017-02-01,300c,2017-02-02,200c,2017-02-03,400c,2017-02-10,60

1 SummingMergeTree将相同主键的所有的数据累加--建表，插入数据create table sum_mergetree(id Int8 ,name String ,cDate Date ,cost Float64 )engine=SummingMergeTree(cost)order by id partition by name ;insert into sum_mergetree values(1,'zss','2020-12-06',70);insert int

MergeTreecreate table tb_tree1(id String ,name String ,city String)engine = MergeTree primary key id order by id;插入数据insert into tb_tree1 values('id004','wb','HB'),('id005','DHT','HN'),('id006','ZXX','HLJ');insert into tb_tree1 values('id0

启动service clickhouse-server startclickhouse-client -m– 数据库create database db_name ;use db_name ; 切换数据库 ;select currentDatabase() ;┌─currentDatabase()─┐│ db_doit19 │└───────────────────┘drop database db_doit19 ;– 建表– 数据类型Int8

一、集合数据类型在hive中的数据类型 ：int bigint double string timestampstruct 结构体 类似于java对象array 数组map 键值对{“name”: “songsong”,“friends”: [“bingbing” , “lili”] ,“children”: {“xiao song”: 18 ,“xiaoxiao song”: 14},“address”: {“street”: “hui long guan”

连续登录的天数数据如下：name login_datea,2020-11-01a,2020-11-05a,2020-11-04b,2020-11-04b,2020-11-05a,2020-11-03a,2020-11-02b,2020-11-08a,2020-11-07b,2020-11-01b,2020-11-02a,2020-11-06b,2020-11-061、创建表，导入数据create table login(name string ,login_date

行转列行转列 将多行的数据合并到一列中collect_set -- 去重collect_list 这两个都是聚合函数 将收集的多行数据聚集成一个数组集合concat 拼接 参数是可变参数 拼接字符串concat_ws 参数一 拼接符 参数二 可变个数的字符串/数组表内容zz 处女座 Bnn 射手座 Abb 处女座 Bgg 白羊座 Atg 射手座 Acreate table tb_teacher(name string ,xz string ,xx

case when 和聚合函数表格内容如下：zs A 男ls A 男ss B 男fj A 女rb B 女hh B 女需求 结果如下；+--------+----+----+| dname | m | f |+--------+----+----+| A | 2 | 1 || B | 1 | 2 |+--------+----+----+--创建表create table tb_emp(name string ,dname string

1 导出数据方式1.1export tb_name to HDFS_PATH1.2如果数据是普通的文本数据 在shell客户端使用get下载数据hdfs dfs -get /tb_name/filehive端： hive> dfs -get /文件 1.3不开启hive终端 执行SQL语句 hive -e "use dafault; select * from tb_user "; hive -e "select * from tb_user " >&g