shell熟悉

202212月中旬，第二任导师要离职了，他手头的事情暂时交接给我，其中有些运维的部分，因此花些时间熟悉常用的Shell，目前主要通过牛客网来刷刷题。
xargs 是一条 Unix 和类 Unix操作系统的常用命令；它的作用是将参数列表转换成小块分段传递给其他命令，以避免参数列表过长的问题。

[hadoop@emr-header-2 test]$ cat for_practice_shell.txt 
anla
nihao 
this is the third line
anla
nihao 
this



[hadoop@emr-header-2 test]$ cat for_practice_shell.txt | xargs -n1
anla
nihao
this
is
the
third
line
anla
nihao
this
[hadoop@emr-header-2 test]$ cat for_practice_shell.txt | xargs -n2
anla nihao
this is
the third
line anla
nihao this
[hadoop@emr-header-2 test]$ cat for_practice_shell.txt | xargs -n3
anla nihao this
is the third
line anla nihao
this
[hadoop@emr-header-2 test]$ cat for_practice_shell.txt | xargs -n4
anla nihao this is
the third line anla
nihao this
[hadoop@emr-header-2 test]$ cat for_practice_shell.txt | xargs -n5
anla nihao this is the
third line anla nihao this
[hadoop@emr-header-2 test]$ cat for_practice_shell.txt | xargs
anla nihao this is the third line anla nihao this

awk 的条件和动作之间 不要随意换行，除非有判断逻辑。动作之间须换行。
如：去掉所有包含this的句子

[hadoop@emr-header-2 test]$ cat for_practice_shell.txt 
that is your bag
is this your bag?
to the degree or extent indicated.
there was a court case resulting from this incident
welcome to nowcoder

#非目标结果
[hadoop@emr-header-2 test]$ awk '                      
    $0~/this/
    {print $0}
' for_practice_shell.txt 
that is your bag
is this your bag?
is this your bag?
to the degree or extent indicated.
there was a court case resulting from this incident
there was a court case resulting from this incident
welcome to nowcoder

#目标结果
[hadoop@emr-header-2 test]$ awk '$0!~/this/{print}' for_practice_shell.txt 
that is your bag
to the degree or extent indicated.
welcome to nowcoder

awk格式化输出 链接
uniq-c 统计相同的字符串,但是会有空格
awk如何进行字符串拼接？格式：变量 空格 “”
注意：+、+=号只能做数值运算，不能做字符串拼接。

awk '{
    str_s = $1 ":" $2
    print(str_s)
}'

自测输入：

aaa sss
bb aaa

输出：

aaa:sss
bb:aaa

awk -F"[1,2,3,4,5]" '{print(NF "," $1 "," $2 "," $3 "," $4 "," $5)}'
#元素1 元素2 元素3 即有两个分隔符 ，即 NF-1
#ps.以 1 2 3 4 5 分隔后，若两个分隔符连续，会有一个空字符串为一个元素

自测输入：

a12b8
10ccc
2521abc
9asf

输出：

3,a,,b8,,
2,,0ccc,,,
5,,,,,abc
1,9asf,,,,

SQLAlchemy是Python编程语言下的一款开源软件。提供了SQL工具包及对象关系映射（ORM）工具，使用MIT许可证发行。
SQLAlchemy“采用简单的Python语言，为高效和高性能的数据库访问设计，实现了完整的企业级持久模型”。

选 parquet 是因为兼容历史原因，impala支持parquet 不支持orc。
选 snappy 是因为 parquet 和 snappy 整合的好。

查询监听端口的程序

(base) [root@emr-header-2 airflow优化]# netstat -anop | grep 40001
tcp        0      0 0.0.0.0:40001           0.0.0.0:*               LISTEN      26443/java           off (0.00/0/0)
(base) [root@emr-header-2 airflow优化]# ps -ef | grep 26443
root       740 20711  0 14:01 pts/0    00:00:00 grep --color=auto 26443
hadoop   26443     1  0 2月09 ?       06:21:03 java -jar ../jar/data-dict.jar