The Introduction of Shell Script

Typographical Conventions Used in This Tutorial

Code segments and script output will be displayed as monospaced text.
Command-line entries will be preceded by the Dollar sign ($). If your prompt is different, enter the command:

PS1="$ " ; export PS1

[maxwell@oracle-db-19c shell_projects]$ PS1="$ "; export PS1
$ 
$ 
$ echo '#!/bin/sh' > my-script.sh
$ echo 'echo Hello World' >> my-script.sh
$ chmod 775 my-script.sh
$ ./my-script.sh
Hello World
$ ls -ltr
total 12
-rw-rw-r--. 1 maxwell maxwell 244 Mar  7 19:57 sorted_data.sh
-rw-rw-r--. 1 maxwell maxwell  84 Mar  7 20:00 source_data.csv
-rwxrwxr-x. 1 maxwell maxwell  27 Mar  9 13:09 my-script.sh
$ 

2. Philosophy

• The speed at which an interpreted program will run as compared to a C program, or even an interpreted Perl program.
• Since it is easy to write a simple batch-job type shell script, there are a lot of poor quality shell scripts around.

One weakness in many shell scripts is lines such as:

cat /tmp/myfile | grep "mystring"

which would run much faster as:

grep "mystring" /tmp/myfile

3. Variables 

$ 
$ vi var.sh
$ chmod 775 var.sh
$ ./var.sh
Hello World
$ cat var.sh
#!/bin/sh
MY_MESSAGE="Hello World"
echo $MY_MESSAGE
$ 

$ vim var2.sh
$ chmod 775 var2.sh
$ ./var2.sh
What is your name？
Maxwell 
Hello Maxwell - hope you're well.
$ cat var2.sh
#!/bin/sh
echo What is your name？
read MY_NAME
echo "Hello $MY_NAME - hope you're well."
$ 

4.Scope of Variables

Variables in the Bourne shell do not have to be declared, as they do in languages like C. But if you try to read an undeclared variable, the result is the empty string. You get no warnings or errors. This can cause some subtle bugs - if you assign
MY_OBFUSCATED_VARIABLE=Hello
and then
echo $MY_OSFUCATED_VARIABLE
Then you will get nothing (as the second OBFUSCATED is mis-spelled).

$ 
$ vim myvar2.sh
$ chmod a+rx myvar2.sh
$ ./myvar2.sh
MYVAR is: 
MYVAR is: hi there
$ cat myvar2.sh
#!/bin/sh
echo "MYVAR is: $MYVAR"
MYVAR="hi there"
echo "MYVAR is: $MYVAR"
$ 

$ 
$ MYVAR=hello
$ ./myvar2.sh  
MYVAR is: 
MYVAR is: hi there
$ 
$ 

We need to export the variable for it to be inherited by another program - including a shell script.

$ 
$ MYVAR=hello
$ ./myvar2.sh  
MYVAR is: 
MYVAR is: hi there
$ 
$ 
$ export MYVAR
$ ./myvar2.sh 
MYVAR is: hello
MYVAR is: hi there
$ 

Once the shell script exits, its environment is destroyed. But MYVAR keeps its value of hello within your interactive shell.
In order to receive environment changes back from the script, we must source the script - this effectively runs the script within our own interactive shell, instead of spawning another shell to run it.
We can source a script via the "." (dot) command:

$ 
$ MYVAR=hello
$ echo $MYVAR
hello
$
$ . ./myvar2.sh
MYVAR is: hello
MYVAR is: hi there
$ echo $MYVAR
hi there
$ 
$ 

$ vim user.sh      
$ chmod 775 user.sh
$ ./user.sh        
What is your name?
Maxwell
Hello Maxwell
I will create you a file called Maxwell_file
$ cat user.sh
#!/bin/sh
echo "What is your name?"
read USER_NAME
echo "Hello $USER_NAME"
echo "I will create you a file called ${USER_NAME}_file"
touch "${USER_NAME}_file"
$ 

5. Wildcards(通配符)

$ cp /tmp/a/* /tmp/b/
$ cp /tmp/a/*.txt /tmp/b/
$ cp /tmp/a/*.html /tmp/b/


$ mv *.txt *.bak

6. Escape Characters

Certain characters are significant to the shell; we have seen, for example, that the use of double quotes (") characters affect how spaces and TAB characters are treated, for example:

$ 
$ echo Hello     World
Hello World
$ echo "Hello    World"
Hello    World
$ 

$ 
$ echo Hello     World
Hello World
$ echo "Hello    World"
Hello    World
$ 
$ 
$ 
$ echo "Hello \"World\""
Hello "World"
$ 
$ echo "Hello " Word ""
Hello  Word 
$ 

In the first example, * is expanded to mean all files in the current directory.
In the second example, *txt means all files ending in txt.
In the third, we put the * in double quotes, and it is interpreted literally.
In the fourth example, the same applies, but we have appended txt to the string.

$ ls -ltr
total 32
-rw-rw-r--. 1 maxwell maxwell 244 Mar  7 19:57 sorted_data.sh
-rw-rw-r--. 1 maxwell maxwell  84 Mar  7 20:00 source_data.csv
-rwxrwxr-x. 1 maxwell maxwell  92 Mar  9 13:15 my-script.sh
-rwxrwxr-x. 1 maxwell maxwell 255 Mar  9 14:32 first2.sh
-rwxrwxr-x. 1 maxwell maxwell  52 Mar  9 14:36 var.sh
-rwxrwxr-x. 1 maxwell maxwell  91 Mar  9 14:42 var2.sh
-rwxrwxr-x. 1 maxwell maxwell  75 Mar  9 14:56 myvar2.sh
-rwxrwxr-x. 1 maxwell maxwell 158 Mar  9 15:14 user.sh
-rw-rw-r--. 1 maxwell maxwell   0 Mar  9 15:18 Maxwell_file
$ echo *
first2.sh Maxwell_file my-script.sh myvar2.sh sorted_data.sh source_data.csv user.sh var2.sh var.sh
$ echo *.sh
first2.sh my-script.sh myvar2.sh sorted_data.sh user.sh var2.sh var.sh
$ echo "*"
*
$ echo "*txt"
*txt
$ 

A quote is ", backslash is \, backtick is `.
A few spaces are    and dollar is $. $X is 5.

$ 
$ export X=5
$ echo "A quote is \", backslash is \\, backtick is \`."
A quote is ", backslash is \, backtick is `.
$ echo "A few spaces are and dollar is \$. \$X is ${X}."
A few spaces are and dollar is $. $X is 5.
$ 

We have seen why the " is special for preserving spacing. Dollar ($) is special because it marks a variable, so $X is replaced by the shell with the contents of the variable X. Backslash (\) is special because it is itself used to mark other characters off; we need the following options for a complete shell:

$ 
$ echo "This is \\ a backslash"
This is \ a backslash
$ echo "This is \" a quote and this is \\ a backslash"
This is " a quote and this is \ a backslash
$ 

7.Loops

Most languages have the concept of loops: If we want to repeat a task twenty times, we don't want to have to type in the code twenty times, with maybe a slight change each time.
As a result, we have for and while loops in the Bourne shell. This is somewhat fewer features than other languages, but nobody claimed that shell programming has the power of C.

$ 
$ vim for.sh
$ sh -x for.sh
+ for i in 1 2 3 4 5
+ echo 'Looping ... number 1'
Looping ... number 1
+ for i in 1 2 3 4 5
+ echo 'Looping ... number 2'
Looping ... number 2
+ for i in 1 2 3 4 5
+ echo 'Looping ... number 3'
Looping ... number 3
+ for i in 1 2 3 4 5
+ echo 'Looping ... number 4'
Looping ... number 4
+ for i in 1 2 3 4 5
+ echo 'Looping ... number 5'
Looping ... number 5
$
$ cat for.sh
#!/bin/sh
for i in 1 2 3 4 5
do
  echo "Looping ... number $i"
done
$ 

$ 
$ vim for2.sh
$ chmod a+rx for2.sh
$ ./for2.sh
Looping ... i is set to hello
Looping ... i is set to 1
Looping ... i is set to first2.sh
Looping ... i is set to for2.sh
Looping ... i is set to for.sh
Looping ... i is set to Maxwell_file
Looping ... i is set to my-script.sh
Looping ... i is set to myvar2.sh
Looping ... i is set to sorted_data.sh
Looping ... i is set to source_data.csv
Looping ... i is set to user.sh
Looping ... i is set to var2.sh
Looping ... i is set to var.sh
Looping ... i is set to 2
Looping ... i is set to goodbye
$ ls -ltr
total 40
-rw-rw-r--. 1 maxwell maxwell 244 Mar  7 19:57 sorted_data.sh
-rw-rw-r--. 1 maxwell maxwell  84 Mar  7 20:00 source_data.csv
-rwxrwxr-x. 1 maxwell maxwell  92 Mar  9 13:15 my-script.sh
-rwxrwxr-x. 1 maxwell maxwell 255 Mar  9 14:32 first2.sh
-rwxrwxr-x. 1 maxwell maxwell  52 Mar  9 14:36 var.sh
-rwxrwxr-x. 1 maxwell maxwell  91 Mar  9 14:42 var2.sh
-rwxrwxr-x. 1 maxwell maxwell  75 Mar  9 14:56 myvar2.sh
-rwxrwxr-x. 1 maxwell maxwell 158 Mar  9 15:14 user.sh
-rw-rw-r--. 1 maxwell maxwell   0 Mar  9 15:18 Maxwell_file
-rw-rw-r--. 1 maxwell maxwell  68 Mar  9 16:16 for.sh
-rwxrwxr-x. 1 maxwell maxwell  83 Mar  9 16:19 for2.sh
$ cat for2.sh
#!/bin/sh
for i in hello 1 * 2 goodbye
do
  echo "Looping ... i is set to $i"
done
$ 

While Loops

while loops can be much more fun!

$ ./while.sh   
Please type something in (bye to quit)
hello
You typed: hello
Please type something in (bye to quit)
bye
You typed: bye
$ cat while.sh
#!/bin/sh
INPUT_STRING=hello
while [ "$INPUT_STRING" != "bye" ]
do
  echo "Please type something in (bye to quit)"
  read INPUT_STRING
  echo "You typed: $INPUT_STRING"
done
$ 

The colon (:) always evaluates to true; whilst using this can be necessary sometimes, it is often preferable to use a real exit condition. Compare quitting the above loop with the one below; see which is the more elegant. Also think of some situations in which each one would be more useful than the other:

$ vim while2.sh       
$ chmod a+rx while2.sh
$ ./while2.sh         
Please type something in (^C to quit)
hello
You typed: hello
Please type something in (^C to quit)
good job!
You typed: good job!
Please type something in (^C to quit)
quit
You typed: quit
Please type something in (^C to quit)
^C
$ cat while2.sh
#!/bin/sh
while :

do
  echo "Please type something in (^C to quit)"
  read INPUT_STRING
  echo "You typed: $INPUT_STRING"

done
$ 

This reads the file "myfile.txt", one line at a time, into the variable "$input_text". The case statement then checks the value of $input_text. If the word that was read from myfile.txt was "hello" then it echoes the word "English". If it was "gday" then it will echo Australian. If the word (or words) read from a line of myfile.txt don't match any of the provided patterns, then the catch-all "*" default will display the message "Unknown Language: $input_text" - where of course "$input_text" is the value of the line that it read in from myfile.txt.

$ 
$ vim while3.sh
$ vim myfile.txt
$ chmod a+rx while3.sh
$ ./while3.sh
Unknown Language: this file is called myfile.txt and we are using it as an example input.
English
Australian
French
Unknown Language: hola
$     
$ cat myfile.txt
this file is called myfile.txt and we are using it as an example input.
hello
gday
bonjour
hola
$ cat while3.sh
#!/bin/sh
while read input_text
do
  case $input_text in
        hello)       echo English    ;;
        howdy)       echo American   ;;
        gday)        echo Australian ;;
        bonjour)     echo French     ;;
        "guten tag") echo German     ;;
        *)           echo Unknown Language: $input_text
                ;;
  esac
done < myfile.txt
$ 

A handy Bash (but not Bourne Shell) tip I learned recently is:

mkdir rc{0,1,2,3,4,5,6,S}.d

instead of the more cumbersome:

for runlevel in 0 1 2 3 4 5 6 S
do
  mkdir rc${runlevel}.d
done

And this can be done recursively, too:

$ 
$ cd /
$ ls -ld {,usr,usr/local}/{bin,sbin,lib}
lrwxrwxrwx.  1 root root     7 Jun 22  2021 /bin -> usr/bin
lrwxrwxrwx.  1 root root     7 Jun 22  2021 /lib -> usr/lib
lrwxrwxrwx.  1 root root     8 Jun 22  2021 /sbin -> usr/sbin
dr-xr-xr-x.  2 root root 49152 Jan 15 11:54 usr/bin
dr-xr-xr-x. 40 root root  4096 Jan 10 09:01 usr/lib
drwxr-xr-x.  2 root root    63 Nov 26 11:20 usr/local/bin
drwxr-xr-x.  2 root root     6 Jun 22  2021 usr/local/lib
drwxr-xr-x.  2 root root     6 Jun 22  2021 usr/local/sbin
dr-xr-xr-x.  2 root root 20480 Jan 15 11:54 usr/sbin
$ 

8. Test

Test is used by virtually every shell script written. It may not seem that way, because test is not often called directly. test is more frequently called as [. [ is a symbolic link to test, just to make shell programs more readable. It is also normally a shell builtin (which means that the shell itself will interpret [ as meaning test, even if your Unix environment is set up differently):

$ 
$ type [
[ is a shell builtin
$ which [
/usr/bin/[
$ ls -l /usr/bin/[
-rwxr-xr-x. 1 root root 54872 Jun 24  2022 '/usr/bin/['
$ ls -l /usr/bin/test
-rwxr-xr-x. 1 root root 54816 Jun 24  2022 /usr/bin/test
$ 

This means that '[' is actually a program, just like ls and other programs, so it must be surrounded by spaces:

if [$foo = "bar" ]

will not work; it is interpreted as if test$foo = "bar" ], which is a ']' without a beginning '['. Put spaces around all your operators. 

Test is most often invoked indirectly via the if and while statements. It is also the reason you will come into difficulties if you create a program called test and try to run it, as this shell builtin will be called instead of your program!
The syntax for if...then...else... is:

if [ ... ]
then
  # if-code
else
  # else-code
fi

Note that fi is if backwards! This is used again later with case and esac.

if [ ... ]; then
  # do something
fi

You can also use the elif, like this:

if  [ something ]; then
 echo "Something"
 elif [ something_else ]; then
   echo "Something else"
 else
   echo "None of the above"
fi

This will echo "Something" if the [ something ] test succeeds, otherwise it will test [ something_else ], and echo "Something else" if that succeeds. If all else fails, it will echo "None of the above".

$ X=5        
$ export X   
$ ./test.sh  
X is more than zero
X is more than or equal to zero
X is not the string "hello"
X is of nonzero length
No such file: 5
$ cat test.sh           
#!/bin/sh
if [ "$X" -lt "0" ]
then
   echo "X is less than zero"
fi
if [ "$X" -gt "0" ]; then
   echo "X is more than zero"
fi
[ "$X" -le "0" ] && \
      echo "X is less than or equal to zero"
[ "$X" -ge "0" ] && \
      echo "X is more than or equal to zero"
[ "$X" = "0" ] && \
       echo "X is the string or number \"0\""
[ "$X" = "hello" ] && \
       echo "X is matches the string \"hello\""
[ "$X" != "hello" ] && \
       echo "X is not the string \"hello\""
[ -n "$X" ] && \
       echo "X is of nonzero length"
[ -f "$X" ] && \
       echo "X is the path of a real file" || \
       echo "No such file: $X"
[ -x "$X" ] && \
       echo "X is the path of an executable file"
[ "$X" -nt "/etc/passwd" ] && \
      echo "X is a file Which is newer than /etc/passwd"
$