Chapter 7 Intermediate Shell Tool I

最新推荐文章于 2024-07-19 11:44:44 发布

DB架构

最新推荐文章于 2024-07-19 11:44:44 发布

阅读量220

点赞数

分类专栏： Linux 文章标签： bash linux 开发语言

本文链接：https://blog.csdn.net/u011868279/article/details/126006643

版权

Linux 专栏收录该内容

43 篇文章 0 订阅

订阅专栏

1.1 Sifting Through Files for a String

The grep command searches through files looking for the expression you supply:

[maxwell@MaxwellDBA test]$ grep printf *.c
grep: *.c: No such file or directory
[maxwell@MaxwellDBA test]$ grep bash *.sh
check_arg_count.sh:#!/usr/bin/env bash
checkfile.sh:#!/usr/bin/env bash
checkstr.sh:#!/usr/bin/env bash
check_unset_parms.sh:#!/usr/bin/env bash
chmod_all2.sh:#!/usr/bin/env bash
chmod_all.sh:#!/usr/bin/env bash
dbinit.sh:#!/usr/bin/env bash
donors.sh:#!/bin/bash
ext.sh:#!/bin/bash
fun_calc.sh:#!/usr/bin/env bash
func_choice1.sh:#!/bin/bash
func_choose.sh:#!/bin/bash
myscript.sh:#!/bin/bash
readability.sh:#!/bin/bash
rpncalc.sh:#!/usr/bin/env bash
select_dir.sh:#!/bin/bash
simpls.sh:#!/bin/bash
strvsnum.sh:#!/usr/bin/env bash
suffixer.sh:#!/usr/bin/env bash
suffixer.sh:# rename files that end in .bad to be .bash
suffixer.sh:    mv "${FN}" "${FN%bad}bash"
trackmatch.sh:#!/usr/bin/env bash
use_up_option.sh:#!/usr/bin/env bash
[maxwell@MaxwellDBA test]$

[maxwell@MaxwellDBA test]$ grep bash /home/maxwell/shelllearning/test/*.sh /home/maxwell/shelllearning/anotherdir/*.sh
/home/maxwell/shelllearning/test/check_arg_count.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/checkfile.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/checkstr.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/check_unset_parms.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/chmod_all2.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/chmod_all.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/dbinit.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/donors.sh:#!/bin/bash
/home/maxwell/shelllearning/test/ext.sh:#!/bin/bash
/home/maxwell/shelllearning/test/fun_calc.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/func_choice1.sh:#!/bin/bash
/home/maxwell/shelllearning/test/func_choose.sh:#!/bin/bash
/home/maxwell/shelllearning/test/myscript.sh:#!/bin/bash
/home/maxwell/shelllearning/test/readability.sh:#!/bin/bash
/home/maxwell/shelllearning/test/rpncalc.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/select_dir.sh:#!/bin/bash
/home/maxwell/shelllearning/test/simpls.sh:#!/bin/bash
/home/maxwell/shelllearning/test/strvsnum.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/suffixer.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/suffixer.sh:# rename files that end in .bad to be .bash
/home/maxwell/shelllearning/test/suffixer.sh:    mv "${FN}" "${FN%bad}bash"
/home/maxwell/shelllearning/test/trackmatch.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/test/use_up_option.sh:#!/usr/bin/env bash
/home/maxwell/shelllearning/anotherdir/func_choice1.sh:#!/bin/bash
/home/maxwell/shelllearning/anotherdir/func_choose.sh:#!/bin/bash
[maxwell@MaxwellDBA test]$

If you don't want to see the specific filenames,you may turn this off using the -h switch to grep:

[maxwell@MaxwellDBA test]$ grep -h bash *.sh 
#!/usr/bin/env bash
#!/usr/bin/env bash
#!/usr/bin/env bash
#!/usr/bin/env bash
#!/usr/bin/env bash
#!/usr/bin/env bash
#!/usr/bin/env bash
#!/bin/bash
#!/bin/bash
#!/usr/bin/env bash
#!/bin/bash
#!/bin/bash
#!/bin/bash
#!/bin/bash
#!/usr/bin/env bash
#!/bin/bash
#!/bin/bash
#!/usr/bin/env bash
#!/usr/bin/env bash
# rename files that end in .bad to be .bash
    mv "${FN}" "${FN%bad}bash"
#!/usr/bin/env bash
#!/usr/bin/env bash

If you don't want to see the actual lines from the file, but only a count of the number of times the expression is found, then use the -c option.

[maxwell@MaxwellDBA test]$ grep -c bash *.sh 
check_arg_count.sh:1
checkfile.sh:1
checkstr.sh:1
check_unset_parms.sh:1
chmod_all2.sh:1
chmod_all.sh:1
dbinit.sh:1
donors.sh:1
ext.sh:1
fun_calc.sh:1
func_choice1.sh:1
func_choose_man1.sh:0
func_choose.sh:1
myscript.sh:1
quoted.sh:0
readability.sh:1
rpncalc.sh:1
select_dir.sh:1
simplest.sh:0
simpls.sh:1
strvsnum.sh:1
suffixer.sh:3
trackmatch.sh:1
tricky.sh:0
trythis.sh:0
use_up_option.sh:1
[maxwell@MaxwellDBA test]$

1.2 Getting Just the Filename from a Search

Use the -l option of grep to get just the filenames:

[maxwell@MaxwellDBA test]$ grep -l bash *.sh
check_arg_count.sh
checkfile.sh
checkstr.sh
check_unset_parms.sh
chmod_all2.sh
chmod_all.sh
dbinit.sh
donors.sh
ext.sh
fun_calc.sh
func_choice1.sh
func_choose.sh
myscript.sh
readability.sh
rpncalc.sh
select_dir.sh
simpls.sh
strvsnum.sh
suffixer.sh
trackmatch.sh
use_up_option.sh
[maxwell@MaxwellDBA test]$

1.3 Getting a Simple True/False from a Search

Use -q, the "quiet" option for grep. Or,for maximum portability,just throw the output away by redirecting it into /dev/null.Either way, your answer is in the bash return status variable $?

[maxwell@MaxwellDBA test]$ grep -q bash fun_calc.sh
[maxwell@MaxwellDBA test]$ if [ $? -eq 0 ] ; then echo yes ; else echo nope ; fi
yes
[maxwell@MaxwellDBA test]$

If we list multiple filenames after grep -q, then grep stops searching after the very first occurence of the search string being found.if you really need to read through all the files, then rather than use -q you can do this :

[maxwell@MaxwellDBA test]$ grep bash fun_calc.sh > /dev/null
[maxwell@MaxwellDBA test]$ if [ $? -eq 0 ] ; then echo yes ; else echo nope ; fi
yes
[maxwell@MaxwellDBA test]$

The redirecting to /dev/null sends the output to a special kind of device, a bit bucket, that just throws everything you give it away.

1.4 Searching for Text While Ignoring Case

Use the -i option on grep to ignore case:

[maxwell@MaxwellDBA test]$ grep -i bash *sh
check_arg_count.sh:#!/usr/bin/env bash
checkfile.sh:#!/usr/bin/env bash
checkstr.sh:#!/usr/bin/env bash
check_unset_parms.sh:#!/usr/bin/env bash
chmod_all2.sh:#!/usr/bin/env bash
chmod_all.sh:#!/usr/bin/env bash
dbinit.sh:#!/usr/bin/env bash
donors.sh:#!/bin/bash
ext.sh:#!/bin/bash
fun_calc.sh:#!/usr/bin/env bash
func_choice1.sh:#!/bin/bash
func_choose.sh:#!/bin/bash
myscript.sh:#!/bin/bash
readability.sh:#!/bin/bash
rpncalc.sh:#!/usr/bin/env bash
select_dir.sh:#!/bin/bash
simpls.sh:#!/bin/bash
strvsnum.sh:#!/usr/bin/env bash
suffixer.sh:#!/usr/bin/env bash
suffixer.sh:# rename files that end in .bad to be .bash
suffixer.sh:    mv "${FN}" "${FN%bad}bash"
trackmatch.sh:#!/usr/bin/env bash
trackmatch.sh:  echo Track ${BASH_REMATCH[2]} is ${BASH_REMATCH[3]}
trackmatch.sh:  mv "$CDTRACK" "Track${BASH_REMATCH[2]}"
use_up_option.sh:#!/usr/bin/env bash
[maxwell@MaxwellDBA test]$

A case-insensitive search finds messages written "BASH","bash","Bash",as well as ones like "BaSH" and "bAsH".This option is particularly useful for finding words anywhere that you might have mixed-case text, including words that might be capitalized at the beginning of a sentence or email addresses.

1.5 Doing a Search in a Pipeline

Just pipe your result into grep

If you also want to have grep search error messages that come from the previous command, be sure to redirect its error output into standard output before the pipe:

gcc bigbadcode.c 2>&1 | grep -i error

This command attempts to compile some hypothetical, hairy piece of code. We redirect standard error into standard output ( 2>&1) before we proceed to pipe (|) the output into grep, where it will search case-insensitively (-i) looking for the string error.

1.6 Paring Down What the Search Finds

Pipe the results into grep -v with an expression that describes what you don’t want to see.

# you’re not sure if it’s always written as Dec, so to be sure to catch them all you type:

[maxwell@MaxwellDBA test]$ grep -i dec logfile
error on Jan 01: not a decimal number
error on Feb 13: base converted to Decimal
warning on Mar 22: using only decimal numbers
error on Dec 16 : the actual message you wanted
error on Jan 01: not a decimal number

# A quick and dirty solution in this case is to pipe the first result into a second grep and tell the second grep to ignore any instances of “decimal”:

[maxwell@MaxwellDBA test]$ grep -i dec logfile | grep -vi decimal
error on Dec 16 : the actual message you wanted
[maxwell@MaxwellDBA test]$ grep -i dec logfile | grep -vi decimal | grep -vi decimate
error on Dec 16 : the actual message you wanted

# a better solution would be to use a more powerful regular expression to match the December date. one that looked for "Dec" followed by a space and two digits:
[maxwell@MaxwellDBA test]$ grep 'Dec [0-9][0-9]' logfile
error on Dec 16 : the actual message you wanted
[maxwell@MaxwellDBA test]$ grep Dec\ [0-9\ ][0-9] logfile
error on Dec 16 : the actual message you wanted
[maxwell@MaxwellDBA test]$

1.8 Search for an SSN

[maxwell@MaxwellDBA test]$ grep '[0-9]\{3\}-\{0,1\}[0-9]\{2\}-\{0,1\}[0-9]\{4\}' datafile
grep: datafile: No such file or directory
[maxwell@MaxwellDBA test]$

[0-9]\{3\} -\{0,1\} [0-9]\{2\} -\{0,1\} [0-9]\{4\}

The first grouping says “any digit” then “exactly 3 times.” The next grouping says “a dash” then “0 or 1 time.” The third grouping says “any digit” then “exactly 2 times.” The next grouping says “a dash” then “0 or 1 time.” The last grouping says “any digit” then “exactly 4 times.”

1.9 Grepping Compressed Files.

zgrep is simply a grep that understands various compressed and uncompressed files (which types are understood varies from system to system).

zgrep 'search term' /var/log/messages*

zcat is simply a cat that understands various compressed and uncompressed files (which types are understood varies from system to system).

1.10 Keeping Some Output,Discarding the Rest

The following code prints the first word of every line of input:

The awk utility reads data from the filename supplied on the command line, or from standard input if no filename is given.

[maxwell@MaxwellDBA test]$ awk '{print $1}' myinput.file
zcat
(which
Words
supplied
[maxwell@MaxwellDBA test]$ awk '{print $1}' < myinput.file
zcat
(which
Words
supplied
[maxwell@MaxwellDBA test]$ 
[maxwell@MaxwellDBA test]$ cat myinput.file | awk '{print $1}'
zcat
(which
Words
supplied
[maxwell@MaxwellDBA test]$ cat myinput.file
zcat is simply a cat that understands various compressed and uncompressed files
(which types are understood varies from system to system).
Words are delineated by whitespace. The awk utility reads data from the filename
supplied on the command line, or from standard input if no filename is given. Therefore, you can redirect the input from a file, like this
[maxwell@MaxwellDBA test]$

1.11 Keeping Only a Portion of a Line of Output

Pipe ls into awk,and just pull out the fields that you need.
awk has a built-in variable called NF that holds the number of fields found on the current line, $NF always refers to the last field.

Just remember that you don’t use a $ to read the value of an awk variable (unlike bash variables). NF is a valid variable reference by itself. Adding a $ before it changes its meaning from “the number of fields on the current line” to “the last field on the current line.”

[maxwell@MaxwellDBA test]$ ls -l | awk '{print $1, $NF}'
total 156
-rw-rw-r-- amazing.data
-rw-rw-r-- another.file
-rw-rw-r-- a.out
-rwxr-x--- asdfwednnrw2rs2esff.txt
-rw-rw-r-- check_arg_count.sh
-rw-rw-r-- checkfile.sh
-rw-rw-r-- checkstr.sh
-rw-rw-r-- check_unset_parms.sh
-rw-rw-r-- chmod_all2.sh
-rw-rw-r-- chmod_all.sh
-rwxr-x--- cong.txt
-rw-rw-r-- dashes
[maxwell@MaxwellDBA test]$

1.12 Reversing the Words on Each Line

[maxwell@MaxwellDBA test]$ awk '{for (i=NF; i>0; i--) {printf "%s ", $i;} printf "\n" }'
sdjsf
sdjsf 
fsf
fsf 
ssfs
ssfs 
sfs
sfs 
sfs
sfs 
^C
[maxwell@MaxwellDBA test]$ awk '{                                                       
                               for (i=NF; i>0; i--) {
                                   printf "%s", $i;
                               }
                               printf "\n"
                           }'
sdafada
sdafada
ddsda
ddsda
fssdn
fssdn
^C
[maxwell@MaxwellDBA test]$

1.13 Summing a List of Numbers

Use awk both to isolate the field to be summed and to do the summing.

[maxwell@MaxwellDBA test]$ ls -l | awk '{sum += $5} END {print sum}'
9527
[maxwell@MaxwellDBA test]$ ls -l
total 156
-rw-rw-r-- 1 maxwell maxwell   0 Jul 26 10:20  amazing.data
-rw-rw-r-- 1 maxwell maxwell  65 Jul 21 21:10  another.file
-rw-rw-r-- 1 maxwell maxwell  10 Jul 21 15:39  a.out
-rwxr-x--- 1 maxwell maxwell  49 Jul 21 20:43  asdfwednnrw2rs2esff.txt
-rw-rw-r-- 1 maxwell maxwell 487 Jul 25 10:17  check_arg_count.sh
-rw-rw-r-- 1 maxwell maxwell 487 Jul 26 10:19  checkfile.sh
-rw-rw-r-- 1 maxwell maxwell 292 Jul 26 14:35  checkstr.sh
-rw-rw-r-- 1 maxwell maxwell 263 Jul 25 12:23  check_unset_parms.sh
-rw-rw-r-- 1 maxwell maxwell 199 Jul 25 09:36  chmod_all2.sh
-rw-rw-r-- 1 maxwell maxwell 160 Jul 25 07:55  chmod_all.sh
-rwxr-x--- 1 maxwell maxwell  19 Jul 21 15:40  cong.txt
-rw-rw-r-- 1 maxwell maxwell 399 Jul 26 20:23  dashes
-rw-rw-r-- 1 maxwell maxwell 185 Jul 26 20:36  dbinit.sh
-rw-rw-r-- 1 maxwell maxwell  15 Jul 21 15:40  def.conf
-rwxrwxr-x 1 maxwell maxwell 123 Jul 22 12:01  donors.sh
-rw-rw-r-- 1 maxwell maxwell 867 Jul 25 05:43  embedded_documenta

1.14 Counting String Values

Use awk's associative arrays for your counting.

[maxwell@MaxwellDBA test]$ ls -lR /usr/local | awk -f asar.awk
ls: cannot open directory '/usr/local/aegis/globalcfg/aegisdb': Permission denied
ls: cannot open directory '/usr/local/aegis/globalcfg/cache': Permission denied
ls: cannot open directory '/usr/local/aegis/globalcfg/quara_hex': Permission denied
ls: cannot open directory '/usr/local/aegis/globalcfg/quarantine': Permission denied
ls: cannot open directory '/usr/local/aegis/globalcfg/vulfilebackup': Permission denied
root owns 5306 files
[maxwell@MaxwellDBA test]$ cat asar.awk
#
# cookbook filename: asar.awk
#
NF > 7 {
        user[$3]++
     }
 END {
         for (i in user)
             {
                printf "%s owns %d files\n", i, user[i]
             }
     }

[maxwell@MaxwellDBA test]$

1.15 Showing Data As a Quick and Easy Histogram

Use the associate arrays of awk, as discussed in the previous recipe:

[maxwell@MaxwellDBA test]$ cat hist.awk
#
# cookbook filename: hist.awk
#
function max(arr, big)
{
   big = 0
   for (i in user)
   {
       if (user[i] > big) { big=user[i];}
   }
   return big
}

NF > 7 {
        user[$3]++
     }
 END {
        # for scaling
        maxm = max(user);
        for (i in user)
        {
          #printf "%s owns %d files\n",i,user[i]
          scaled = 60 * user[i] / maxm ;
          printf "%-10.10s [%8d]:", i, user[i]
          for (i=0; i<scaled; i++) {
              printf "#";
          }
          printf "\n"
        }
     }
[maxwell@MaxwellDBA test]$ ls -lR /usr/local | awk -f hist.awk
ls: cannot open directory '/usr/local/aegis/globalcfg/aegisdb': Permission denied
ls: cannot open directory '/usr/local/aegis/globalcfg/cache': Permission denied
ls: cannot open directory '/usr/local/aegis/globalcfg/quara_hex': Permission denied
ls: cannot open directory '/usr/local/aegis/globalcfg/quarantine': Permission denied
ls: cannot open directory '/usr/local/aegis/globalcfg/vulfilebackup': Permission denied
root       [    5306]:############################################################
[maxwell@MaxwellDBA test]$

1.16 Showing a Paragraph of Text After a Found Phrase

[maxwell@MaxwellDBA test]$ awk -f para.awk < myinput.file
[maxwell@MaxwellDBA test]$ cat para.awk
/keyphrase/ { flag=1 }
{ if (flag == 1) { print $0 }}
/^$/ { flag=0 }
[maxwell@MaxwellDBA test]$