自定义博客皮肤VIP专享

*博客头图:

格式为PNG、JPG,宽度*高度大于1920*100像素,不超过2MB,主视觉建议放在右侧,请参照线上博客头图

请上传大于1920*100像素的图片!

博客底图:

图片格式为PNG、JPG,不超过1MB,可上下左右平铺至整个背景

栏目图:

图片格式为PNG、JPG,图片宽度*高度为300*38像素,不超过0.5MB

主标题颜色:

RGB颜色,例如:#AFAFAF

Hover:

RGB颜色,例如:#AFAFAF

副标题颜色:

RGB颜色,例如:#AFAFAF

自定义博客皮肤

-+

Widsom的博客

人的原动力来自对未来的美好憧憬

  • 博客(4)
  • 资源 (10)
  • 收藏
  • 关注

原创 Hive表命名规范

Hive表命名规范一、命名规则业务产品_层名_计算周期&&统计周期_表类型_存储类型_表的业务含义举个例子:1.浏览器业务产品的 每天计算的 hive 搜索事件的 事实表 明细数据 browser_dwd_dd_face_hive_search2.浏览器业务产品 每天全量 城市 hive维表 browser_dim_da_dic_hive_city3.浏览器业务产品 每天 搜索网站 次数统计 排名 browser_al_dd_rpt_hive_search_web_ran

2020-09-30 17:47:38 5148

原创 Hive之MapReduce性能优化

Hive优化一、Hive任务创建文件数优化1.1 Map端文件合并减少Map任务数量一般来说,HDFS的默认文件块大小是128M,如果在Hive执行任务时,发现Map端的任务过多,且执行时间多数不超过一分钟,建议通过参数,划分(split)文件的大小,合并小文件。如:set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;set mapreduce.input.fileinputformat.split.m

2020-09-20 22:05:02 3696

原创 Hive添加列、修改列(调整位置)、删除列操作等

1.添加1列或多列# 添加1列alter table table_name add columns( user_id bigint comment '用户ID');# 添加多列alter table table_name add columns( name string comment '用户名称', city string comment '城市', sex string comment '用户性别', age s

2020-09-11 18:35:51 23884 2

原创 Hive Archive合并文件归档,减少小文件数量(推荐)

我们在使用Hive存储时,有时会遇到Hive表的文件大小不大,但是文件数量众多;这是可能会遇到HDFS的储存空间没到阈值,但文件数量超过阈值。如果小文件太多,容易影响整个集群的性能。那么对于小文件多的问题,有以下处理方法:1.输出时减少reduce的数量;但可能会导致job运行变慢2.使用Spark重新读取小文件,修改分区并写入,这里不建议使用repartition(),推荐使用coalesce()repartition()会增加文件的大小,因为要历经shuffle阶段,但coalesce()不会经

2020-09-10 18:46:24 1424 1

内存检测分析工具

内存分析工具

2017-06-19

Android逆向助手

android逆向助手,反编译工具

2017-06-19

GifCam录屏工具

录屏工具,保存的格式是GIF

2017-06-19

fiddler4抓包工具

Fiddler 4抓包工具

2017-06-19

commons-beanutils-1.8.3.jar

commons-beanutils-1.8.3.jar

2017-02-25

阿里巴巴Java开发手册 高清.pdf版

阿里巴巴Java开发手册,包含编程规约,异常日志,MySQL规约,工程规约,安全规约

2017-02-10

Bugly实现热更新Demo

使用bugly实现热更新

2017-02-07

Gradle Rescipes for Android

Gradle 官网资料

2016-11-15

Gradle for Android

About This Book, Create custom Gradle tasks and plugins for your Android projects, Configure different build variants, each with their own dependencies and properties, Manage multi-module projects, and integrate modules interdependently, Who This Book Is For, If you are an experienced Android developer wanting to enhance your skills with the Gradle Android build system, then this book is for you. As a prerequisite, you will need some knowledge of the concepts of Android application development., What You Will Learn, Build new Android apps and libraries using Android Studio and Gradle, Migrate projects from Eclipse to Android Studio and Gradle, Manage the local and remote dependencies of your projects, Create multiple build variants, Include multiple modules in a single project, Integrate tests into the build process, Create custom tasks and plugins for Android projects, In Detail, Gradle is an open source build automation system that introduces a Groovy-based domain-specific language (DSL) to configure projects. Using Gradle makes it easy for Android developers to manage dependencies and set up the entire build process., This book begins by taking you through the basics of Gradle and how it works with Android Studio. Furthermore, you will learn how to add local and remote dependencies to your project. You will work with build variants, such as debug and release, paid and free, and even combinations of these things. The book will also help you set up unit and integration testing with different libraries and will show how Gradle and Android Studio can make running tests easier. Finally, you will be shown a number of tips and tricks on the advanced customization of your application's build process. By the end of this book, you will be able to customize the entire build process, and create your own tasks and plugins for your Gradle builds.

2016-11-15

空空如也

TA创建的收藏夹 TA关注的收藏夹

TA关注的人

提示
确定要删除当前文章?
取消 删除