weixin_43003792的博客

一。Spark基础架构二：Spark内置模块三 Spark Shell1.yarn application -listdef flatMap[U](f: String => TraversableOnce[U])(implicit evidence$4: scala.reflect.ClassTag[U]): org.apache.spark.rdd.RDD[U]sc.textFile(“in”).flatMap(x=>x.split(" ")).map((x)=>{(x,

一.flume自定义sink.public class MySink extends AbstractSink implements Configurable {//定义两个属性 前后缀private String prefix;private String subfix;//获取Log对象Logger logger = LoggerFactory.getLogger(MySink.class);@Overridepublic void configure(Context context)

一.自定义source1.需求 hadoop203 监控本机端口44444，将内容分类发送给hadoop204,hadoop205，包含hello的给hadoop204,不包含的给hadoop205.都打印到控制台。(1)自定义过滤器。public class TypeInterceptor implements Interceptor {private List addHeaderEverts; //定义添加过头的eventlist；//初始化方法@Overridepublic void

1.启动./hiveserver22.client启动beeline3.!connect jdbc:hive2://hadoop102:100004. Enter username for jdbc:hive2://hadoop102:10000: atguiguEnter password for jdbc:hive2://hadoop102:10000:Connected to: Apache Hive (version 1.2.1)5. show databases;6.DDLcrea

分区 /boot 200M/swap 2G/ 全部1.关闭防火墙 service iptables stop //服务关闭2.chkconfig iptables off //关闭开机自启

**1.搜索查找类命令 find -name "in."2.ll | grep shenzhen 过滤查找。3.