CatherineHuangTT的博客

1.java heap space异常如果仅设置set mapred.map.child.java.opts=-Xmx2048m;（默认值:mapred.map.child.java.opts=-Djava.net.preferIPv4Stack=true -Xmx1610612736）set mapred.reduce.child.java.opts=-Xmx2048m;（默认值:mapre...

def fib(n): #write Fibonacci series up to n #在Java和C语言中不能赋予两个不同的变量不同的值，但是在Python中是可以的 a,b = 0, 1 while a < n: print(a, end=', ') a, b = b, a+b print()#Now call the fun

for n in range(2, 10): for x in range(2, n): if n % x == 0: #连接符用逗号和加号都可 print(n, 'equals', x, '*', x) break else: # loop fell throug

quit; exit;当我们在CLI或者Beeline下，我们可以用quit或者exit命令来退出交互式的shell。set;在交互式shell中输入 上面的命令可以查看用户和Hive重写的配置文件的变量；set =在交互式shell中设置参数，set mapred.reduce.tasks=32;这里具体列举下参数配置的三种方式：（1）在Hive的配置文件hive-site

//上界 <:class Pair[T <: Comparable[T]] { def shuishuai(first:T,second:T):T={ if(first.compareTo(second)<0) first else second }}object Test{ def main(args: Array[String]): Unit = {

/*下界 >: 去公安部们身份证，儿子的身份证父亲当然可以拿走 */class Farther(val name:String)class Child(name:String) extends Farther(name:String)object GetIDCard { def getIdCard[R >: Child](person:R): Unit ={ pri

/*视图边界 A <% B左边的类型是右边类型或者是其子类。如果不是，左边类型会隐士转换为右边的类型 */class Person(var name:String){ def sayHello = println("hello my name is "+name); def makeFirends(p:Person): Unit ={ sayHello

class Masterclass Professional extends Masterclass Card[-T](val name:String) //支持逆变，父类来了当然可以class Workerobject Test { def enterMeet(card: Card[Professional]): Unit = { println("welcome to

package day3.lesson07class Masterclass Professional extends Masterclass Card[+T](val name:String) //支持协变 子类来了也可以class Workerobject Test { def enterMeet(card: Card[Master]): Unit = { pr

seasons = ['Spring', 'Summer', 'Fall', 'Winter']li1 = list(enumerate(seasons))print(li1)li2 = list(enumerate(seasons, start=1))print(li2)"""result:[(0, 'Spring'), (1, 'Summer'), (2, 'Fall'), (3

""""如果你需要遍历序列数字，Python的内置函数可以帮你遍历序列数字"""for i in range(20) : print(i)"""result:01234从遍历的结果可以得出，range是 [ ) 的，遍历从0开始"""#range(遍历的起始数字，遍历的结束数字（不包括）)for i in range(5,10) : print(i

#Python的for可以遍历 a list or a string#Measure some stringswords = ['happy', 'smile', 'sunshine']for word in words : print(word, len(word))"""result: happy 5 smile 5 sunshin

x = int(input("Please ente an integer:"))#y = float(input("Please enter a float"))if x < 0 : x = 0 print("Negative changed to zero")elif x == 0 : print("zero") print('zero')elif

import org.apache.log4j.{Level, Logger}import org.apache.spark.SparkConfimport org.apache.spark.streaming.dstream.{DStream, ReceiverInputDStream}import org.apache.spark.streaming.{Seconds, Streamin

import org.apache.log4j.{Level, Logger}import org.apache.spark.SparkConfimport org.apache.spark.streaming.dstream.ReceiverInputDStreamimport org.apache.spark.streaming.{Seconds, StreamingContext}

import org.apache.log4j.{Logger,Level}import org.apache.spark.SparkConfimport org.apache.spark.streaming.dstream.DStreamimport org.apache.spark.streaming.{Seconds, StreamingContext}object SparkStr

import org.apache.spark.SparkConfimport org.apache.spark.broadcast.Broadcastimport org.apache.spark.rdd.RDDimport org.apache.spark.streaming.dstream.{DStream, ReceiverInputDStream}import org.apach

import org.apache.spark.SparkConfimport org.apache.spark.streaming.dstream.{DStream, ReceiverInputDStream}import org.apache.spark.streaming.{Seconds, StreamingContext}import org.apache.log4j.{Logge

1.MySQL的连接池package streaming.utilsimport java.sql.{Connection, DriverManager}object MysqlPool { private val max=8 ;//连接池的连接总数 private val connectionNum=10;//每次产生的连接数 private var conNum=0;/

2,从Kafka里面拿数据的两种方式(1) push(推过来的)   kafka，flume  ->  Exeuctor内存-》磁盘  处理    1）整个任务出问题了    2）整个集群宕机了    3）机房停电了       数据有可能重复消费，也有可能漏了 Spark 1.3 以后 (2) pull(拉)    sparkStreaming自己去维