RDD,Spark SQL,DF排序

一、单一字段排序

1、用RDD
RDD使用takeOrdered(num,key=None)方法排序资料

升序排列
a = userrdd.takeOrdered(5, key=lambda x: int(x[1]))
print(a)

降序
a = userrdd.takeOrdered(5, key=lambda x: -int(x[1]))
print(a)

2、Spark SQL
使用关键字order by

降序
sqlContxt.sql('''
select userid,age,gender,occupation,salary from user_table order by age desc
''').show(5)


升序排列
sqlContxt.sql('''
select userid,age,gender,occupation,salary from user_table order by age 
''').show(5)

3、df排列

df.select('userid','occupation','gender','age').orderBy('age').show(5)
df.select('userid','occupation','gender','age').orderBy('age'，ascending=0).show(5)

二、多字段排序
1、rdd 排列

# RDD 排序 lambda x:(-int(x[1]),x[2]))年龄降幂排列，性别生序排列
a = userrdd.takeOrdered(5, key=lambda x: (-int(x[1]), x[2]))
print(a)

2 、Spark SQL

sqlContxt.sql('''
select userid,age,gender,occupation,salary from user_table order by age desc,gender
''').show()

3、DF排列

df.orderBy(['age', 'gender'], ascending=[0, 1]).show(5)

df.orderBy(df.age.desc(), df.gender).show(5)