一. 前提
1.1 数据准备
545-1 2024/07/20 18:20:21 [ERROR] MPX001 eventController=aupay transactionId=A545 {"event":"ERROR","auid":"kddi_XXXasd1","category":{"id":"110","name":"APPLE 1","amout":10,"price":12}}
545-2 2024/07/20 18:20:22 [INFO] MPX001 eventController=alipay transactionId=Bljk {"event":"INFO","auid":"kddi_XXXasd2","category":{"id":"111","name":"APPLE 2","amout":11,"price":13}}
545-3 2024/07/20 18:20:23 [ERROR] MPX001 eventController=paypay transactionId=Ijkhjk {"event":"ERROR","auid":"kddi_XXXasd3","category":{"id":"112","name":"APPLE 3","amout":12,"price":14}}
545-4 2024/07/20 18:20:24 [INFO] MPX001 eventController=alipay transactionId=C9joj {"event":"INFO","auid":"kddi_XXXasd4","category":{"id":"113","name":"APPLE 4","amout":13,"price":15}}
1.2 需求
- 从log中提取出
[ERROR]
相关的日志 - 然后再从日志中进一步提取
eventController
,transactionId
,JSON数据中的auid
和name
- 服务器中没有安装
jq
库,无法通过此种方式来处理json数据。
1.3 分析
- ①先提取
[ERROR]
相关的日志,过滤掉INFO
的日志 - ②再提取出
eventController
和transactionId
字段 - ③然后可以先尝试提取出所有的json数据
- 使用
{.*}
正则表达式
- 使用
- ④然后可以进一步缩小范围,提取
category
相关的json数据- 使用
"category":{.*}
正则表达式
- 使用
- ⑤然后可以进一步提取
auid
和name
所对应的值auid":"[^"]*"
name":"[^"]*"
- ⑥最后再通过sed命令将各字段转置到一行上
sed ':loop; N; $!b loop; ;s/\n\([tan]\)/ \1/g'
二. 数据提取
2.1 提取所有的json数据
{.*}
grep -E "\[ERROR\]\sMPX001" ./result.log | \
grep -o -e "eventController=\S*" -e "transactionId=\S*" -e "{.*}" | \
sed ':loop; N; $!b loop; ;s/\n\([t{]\)/ \1/g'
⏹效果如下
fengyehong@ubuntu:~/jmw_work_space/20270720$ grep -E "\[ERROR\]\sMPX001" ./result.log | \
> grep -o -e "eventController=\S*" -e "transactionId=\S*" -e "{.*}" | \
> sed ':loop; N; $!b loop; ;s/\n\([t{]\)/ \1/g'
eventController=aupay transactionId=A545 {"event":"ERROR","auid":"kddi_XXXasd1","category":{"id":"110","name":"APPLE 1","amout":10,"price":12}}
eventController=paypay transactionId=Ijkhjk {"event":"ERROR","auid":"kddi_XXXasd3","category":{"id":"112","name":"APPLE 3","amout":12,"price":14}}
2.2 提取子项目的全部json数据
"category":{.*}
grep -E "\[ERROR\]\sMPX001" ./result.log | \
grep -o -e "eventController=\S*" -e "transactionId=\S*" -e '"category":{.*}' | \
sed ':loop; N; $!b loop; ;s/\n\([t"]\)/ \1/g'
⏹效果如下
fengyehong@ubuntu:~/jmw_work_space/20270720$ grep -E "\[ERROR\]\sMPX001" ./result.log | \
> grep -o -e "eventController=\S*" -e "transactionId=\S*" -e '"category":{.*}' | \
> sed ':loop; N; $!b loop; ;s/\n\([t"]\)/ \1/g'
eventController=aupay transactionId=A545 "category":{"id":"110","name":"APPLE 1","amout":10,"price":12}}
eventController=paypay transactionId=Ijkhjk "category":{"id":"112","name":"APPLE 3","amout":12,"price":14}}
2.3 提取指定项目的json数据
auid":"[^"]*"
name":"[^"]*"
[
和]
:定义一个字符类。^
:在字符类中表示否定,意味着字符类匹配所有不在括号内的字符。"
:字符类中唯一被否定的字符。*
:表示前面的模式(即 [^"])可以出现零次或多次。
因此,[^"]*
表示匹配由任意数量的非双引号字符组成的字符串,包括零个字符的情况。
grep -E "\[ERROR\]\sMPX001" ./result.log | \
grep -o -e "eventController=\S*" -e "transactionId=\S*" -e 'auid":"[^"]*"' -e 'name":"[^"]*"' | \
sed ':loop; N; $!b loop; ;s/\n\([tan]\)/ \1/g'
⏹效果如下
fengyehong@ubuntu:~/jmw_work_space/20270720$ grep -E "\[ERROR\]\sMPX001" ./result.log | \
> grep -o -e "eventController=\S*" -e "transactionId=\S*" -e 'auid":"[^"]*"' -e 'name":"[^"]*"' | \
> sed ':loop; N; $!b loop; ;s/\n\([tan]\)/ \1/g'
eventController=aupay transactionId=A545 auid":"kddi_XXXasd1" name":"APPLE 1"
eventController=paypay transactionId=Ijkhjk auid":"kddi_XXXasd3" name":"APPLE 3"