今日小结【大数据+python】---部分技能演练

1.如何删除分区

alter table db.tb drop if exists partition (partition1='${partition1}', partition2='${partition2}');

2.如何导入文件至textfile

hive -v -e "

set mapred.job.queue.name='${queue_name}';

use db;

hadoop fs -rm targetfile

hadoop fs -cp sourcefile targetfile

load data inpath 'targetfile' into db.tb;

 

";

3.python多线程

Demo:

from threading import Thread

threads = []

for obj,arg in objs:

    thread = Thread(target=obj, args=(arg,), daemon=False)

    threads.append(thread)

for thread in threads:

    thread.start()

    #thread.join()

4.python多进程

Demo:

from multiprocessing import Process

def main():

    ps = []

    for obj, arg in objs:

        process = Process(target=obj, args=(arg, ))

        ps.append(process)

    for process in ps:

        process.start()

 

 

if __name__ == '__main__':

    main()