Shell Scripting part1

 1.Introduction

PS1="$ " ; export PS1

[maxwell@oracle-db-19c shell_20230320]$ PS1="$ " ; export PS1
$ echo '#!/bin/sh' > my-script.sh
$ echo 'echo Hello World' >> my-script.sh
$ chmod 755 my-script.sh
$ ./my-script.sh
Hello World
$ cat my-script.sh
#!/bin/sh
echo Hello World
$ chmod a+rx my-script.sh
$ ./my-script.sh
Hello World
$ 

2.Philosphy

One weakness in many shell scripts is lines such as:

cat /tmp/myfile | grep "mystring"

which would run much faster as:

grep "mystring" /tmp/myfile

 3. A First Script

[maxwell@oracle-db-19c shell_20230320]$ vim first.sh
[maxwell@oracle-db-19c shell_20230320]$ chmod 755 first.sh
[maxwell@oracle-db-19c shell_20230320]$ ./first.sh
Hello World
[maxwell@oracle-db-19c shell_20230320]$ cat first.sh
#!/bin/sh
# This is a comment!
echo Hello World        # This is a comment, too!
[maxwell@oracle-db-19c shell_20230320]$ 

$ cat first2.sh
#!/bin/sh
# This is a comment!
echo "Hello      World"  # This is a comment, too!
echo "Hello World"
echo "Hello * World"
echo Hello * World
echo Hello      World
echo "Hello" World
echo Hello "    " World
echo "Hello"*"World"
echo `hello` world
echo 'hello' world
$ 

$ vim first2.sh
$ chmod 775 first2.sh
$ ./first2.sh
Hello      World
Hello World
Hello * World
Hello first2.sh first.sh my-script.sh World
Hello World
Hello World
Hello       World
Hello * World
./first2.sh: line 11: hello: command not found
world
hello world
$ cat first2.sh
#!/bin/sh
# This is a comment!
echo "Hello      World"       # This is a comment, too!
echo "Hello World"
echo "Hello * World"
echo Hello * World
echo Hello      World
echo "Hello" World
echo Hello "     " World
echo "Hello "*" World"
echo `hello` world
echo 'hello' world
$ 

4.Variables -Part I

$ 
$ vim var.sh
$ chmod 775 var.sh
$ ./var.sh
Hello World
$ 
$ cat var.sh
#!/bin/sh
MY_MESSAGE="Hello World"
echo $MY_MESSAGE
$ 

If you assign a string to a variable then try to add 1 to it, you will not get away with it:

$ 
$ x="hello"
$ expr $x + 1
expr: non-integer argument
$ 

This is because the external program expr only expects numbers.But there is no syntactic difference between:

$ 
$ 
$ MY_MESSAGE="Hello World"
$ MY_SHORT_MESSAGE=hi
$ MY_NUMBER=1
$ MY_PI=3.142
$ MY_OTHER_PI="3.142"
$ MY_MIXED=123abc
$ 

We can interactively set variable names using the read command; the following script asks you for your name then greets you personally:

$ 
$ vim var2.sh
$ chmod 775 var2.sh
$ ./var2.sh
What is your name?
Maxwell
Hello Maxwell - hope you're well.
'$ 
$ cat var2.sh
#!/bin/sh
echo What is your name?
read MY_NAME
echo "Hello $MY_NAME - hope you're well."
$ 

Scope of Variables

Variables in the Bourne shell do not have to be declared, as they do in languages like C. But if you try to read an undeclared variable, the result is the empty string. You get no warnings or errors. This can cause some subtle bugs - if you assign
MY_OBFUSCATED_VARIABLE=Hello
and then
echo $MY_OSFUCATED_VARIABLE
Then you will get nothing (as the second OBFUSCATED is mis-spelled).

There is a command called export which has a fundamental effect on the scope of variables. In order to really know what's going on with your variables, you will need to understand something about how this is used.

$ 
$ vim myvar2.sh
$ chmod 775 myvar2.sh
$ 
$ ./myvar2.sh
MYVAR is: 
MYVAR is : hi there
$ 
$ cat myvar2.sh
#!/bin/sh
echo "MYVAR is: $MYVAR"
MYVAR="hi there"
echo "MYVAR is : $MYVAR"
$ 

We need to export the variable for it to be inherited by another program - including a shell script. 

$ 
$ 
$ MYVAR=hello
$ 
$ ./myvar2.sh
MYVAR is: 
MYVAR is : hi there
$ 
$ export MYVAR
$ 
$ ./myvar2.sh 
MYVAR is: hello
MYVAR is : hi there
$ 
$ 
$ echo $MYVAR
hello
$ 
$ 

Once the shell script exits, its environment is destroyed. But MYVAR keeps its value of hello within your interactive shell.
In order to receive environment changes back from the script, we must source the script - this effectively runs the script within our own interactive shell, instead of spawning another shell to run it.
We can source a script via the "." (dot) command:

$ 
$ 
$ MYVAR=hello
$ echo $MYVAR
hello
$ 
$ . ./myvar2.sh
MYVAR is: hello
MYVAR is : hi there
$ 
$ echo $MYVAR
hi there
$ 

$ 
$ vim user.sh
$ chmod 775 user.sh
$ 
$ ./user.sh
What is your name?
Maxwell
Hello Maxwell
I will create you a file called Maxwell_file
$ ls -ltr
total 28
-rwxr-xr-x. 1 maxwell maxwell  81 Mar 20 16:57 first.sh
-rwxr-xr-x. 1 maxwell maxwell  27 Mar 20 17:10 my-script.sh
-rwxrwxr-x. 1 maxwell maxwell 273 Mar 20 17:21 first2.sh
-rwxrwxr-x. 1 maxwell maxwell  52 Mar 20 17:24 var.sh
-rwxrwxr-x. 1 maxwell maxwell  89 Mar 20 17:35 var2.sh
-rwxrwxr-x. 1 maxwell maxwell  76 Mar 20 18:38 myvar2.sh
-rwxrwxr-x. 1 maxwell maxwell 158 Mar 20 18:56 user.sh
-rw-rw-r--. 1 maxwell maxwell   0 Mar 20 18:57 Maxwell_file
$ 
$ cat user.sh
#!/bin/sh
echo "What is your name?"
read USER_NAME
echo "Hello $USER_NAME"
echo "I will create you a file called ${USER_NAME}_file"
touch "${USER_NAME}_file"
$ 

5.Wildcards

Wildcards are really nothing new if you have used Unix at all before.

$ cp /tmp/a/* /tmp/b/
$ cp /tmp/a/*.txt /tmp/b/
$ cp /tmp/a/*.html /tmp/b/

Now how would you list the files in /tmp/a/ without using ls /tmp/a/?
How about echo /tmp/a/*? What are the two key differences between this and the ls output? How can this be useful? Or a hinderance?
How could you rename all .txt files to .bak? Note that

$ mv *.txt *.bak

6.Escape Characters

Certain characters are significant to the shell; we have seen, for example, that the use of double quotes (") characters affect how spaces and TAB characters are treated, for example:

$ 
$ 
$ echo Hello      World
Hello World
$ echo "Hello       World"
Hello       World
$ 
$ echo "Hello   \"World\""
Hello   "World"
$ 
$ echo "Hello   " World ""
Hello    World 
$ 
$ echo "Hello   "World""
Hello   World
$ 

Most characters (*, ', etc) are not interpreted (ie, they are taken literally) by means of placing them in double quotes (""). They are taken as is and passed on to the command being called. An example using the asterisk (*) goes:

$ 
$ echo *
first2.sh first.sh Maxwell_file my-script.sh myvar2.sh user.sh var2.sh var.sh
$ 
$ echo "*"
*
$ 
$ echo "*.sh"
*.sh
$ 

In the first example, * is expanded to mean all files in the current directory.
In the second example, *txt means all files ending in txt.
In the third, we put the * in double quotes, and it is interpreted literally.
In the fourth example, the same applies, but we have appended txt to the string.

However, ", $, `, and \ are still interpreted by the shell, even when they're in double quotes.
The backslash (\) character is used to mark these special characters so that they are not interpreted by the shell, but passed on to the command being run (for example, echo).
So to output the string: (Assuming that the value of $X is 5):

A quote is ", backslash is \, backtick is `.
A few spaces are    and dollar is $. $X is 5.

$ X=5
$ echo "A quote is \", backslash is \\, backtick is \`."   
A quote is ", backslash is \, backtick is `.
$ echo "A few spaces are    and dollar is \$. \$X is ${X}."
A few spaces are    and dollar is $. $X is 5.
$ 

We have seen why the " is special for preserving spacing. Dollar ($) is special because it marks a variable, so $X is replaced by the shell with the contents of the variable X. Backslash (\) is special because it is itself used to mark other characters off; we need the following options for a complete shell:

$ 
$ echo "This is \\ a backslash"
This is \ a backslash
$ 
$ echo "This is \" a quote and this is \\ a backslash"
This is " a quote and this is \ a backslash
$ 

 7.Loops

For Loops

for loops iterate through a set of values until the list is exhausted:

$ 
$ vim for.sh
$ 
$ chmod 775 for.sh
$ 
$ ./for.sh 
Looping ... number 1
Looping ... number 2
Looping ... number 3
Looping ... number 4
Looping ... number 5
$ 
$ cat for.sh
#!/bin/sh
for i in 1 2 3 4 5
do
  echo "Looping ... number $i"
done
$ 

Try this code and see what it does. Note that the values can be anything at all:

$ vim for2.sh
$ 
$ chmod 775 for2.sh
$ 
$ ./ for2.sh 
-bash: ./: Is a directory
$ ./for2.sh
Looping ... i is set to hello
Looping ... i is set to 1
Looping ... i is set to first2.sh
Looping ... i is set to first.sh
Looping ... i is set to for2.sh
Looping ... i is set to for.sh
Looping ... i is set to Maxwell_file
Looping ... i is set to my-script.sh
Looping ... i is set to myvar2.sh
Looping ... i is set to user.sh
Looping ... i is set to var2.sh
Looping ... i is set to var.sh
Looping ... i is set to 2
Looping ... i is set to goodbye
$ pwd
/home/maxwell/projects/shell_20230320
$
$ ls
first2.sh  first.sh  for2.sh  for.sh  Maxwell_file  my-script.sh  myvar2.sh  user.sh  var2.sh  var.sh
$ 
$ 
$ cat for2.sh
#!/bin/sh
for i in hello 1 * 2 goodbye
do
  echo "Looping ... i is set to $i"
done

$ 
$ 

This is well worth trying. Make sure that you understand what is happening here. Try it without the * and grasp the idea, then re-read the Wildcards section and try it again with the * in place. Try it also in different directories, and with the * surrounded by double quotes, and try it preceded by a backslash (\*)  

$ 
$ vim for2.sh
$ 
$ ./for2.sh
Looping ... i is set to hello
Looping ... i is set to 1
Looping ... i is set to *
Looping ... i is set to 2
Looping ... i is set to goodbye
$ 
$ cat for2.sh
#!/bin/sh
for i in hello 1 \* 2 goodbye
do
  echo "Looping ... i is set to $i"
done

$ 

While Loops

while loops can be much more fun! (depending on your idea of fun, and how often you get out of the house... )

$ 
$ vim while.sh
$ 
$ chmod 775 while.sh
$ 
$ ./while.sh
Please type something in (bye to quit)
what happen here
You typed: what happen here
Please type something in (bye to quit)
bye
You typed: bye
$ 
$ cat while.sh
#!/bin/sh
INPUT_STRING=hello
while [ "$INPUT_STRING" != "bye" ]
do
  echo "Please type something in (bye to quit)"
  read INPUT_STRING
  echo "You typed: $INPUT_STRING"

done
$ 

The colon (:) always evaluates to true; whilst using this can be necessary sometimes, it is often preferable to use a real exit condition. Compare quitting the above loop with the one below; see which is the more elegant. Also think of some situations in which each one would be more useful than the other:

$ 
$ vim while2.sh
$ 
$ chmod 775 while2.sh
$ 
$ ./while2.sh
Please type something in (^C to quit)
hello Maxwell
You typed: hello Maxwell
Please type something in (^C to quit)
^C
$ 
$ cat while2.sh
#!/bin/sh
while :
do
  echo "Please type something in (^C to quit)"
  read INPUT_STRING
  echo "You typed: $INPUT_STRING"
done
$ 

$ 
$ vim while3.sh      
$ chmod 775 while3.sh
$ ./while3.sh        
Unknown Language: this file is called myfile.txt and we are using it as an example input.
Engslish
Australlian
French
Unknown Language: hola
$ 
$ cat while3.sh
#!/bin/sh
while read input_text
do
  case $input_text in
       hello)              echo Engslish    ;;
       howdy)              echo American    ;;
       gday)               echo Australlian ;;
       bonjour)            echo French      ;;
       "guten tag")        echo German      ;;
       *)                  echo Unknown Language: $input_text
  esac
done < myfile.txt
$ 
$ 
$ cat myfile.txt
this file is called myfile.txt and we are using it as an example input.
hello
gday
bonjour
hola
$ 

A handy Bash (but not Bourne Shell) tip I learned recently 

mkdir rc{0,1,2,3,4,5,6,S}.d

instead of the more cumbersome:

for runlevel in 0 1 2 3 4 5 6 S
do
  mkdir rc${runlevel}.d
done

And this can be done recursively, too:

$ 
$ cd /
$ ls -ld {,usr,usr/local}/{bin,sbin,lib}
lrwxrwxrwx.  1 root root     7 Jun 22  2021 /bin -> usr/bin
lrwxrwxrwx.  1 root root     7 Jun 22  2021 /lib -> usr/lib
lrwxrwxrwx.  1 root root     8 Jun 22  2021 /sbin -> usr/sbin
dr-xr-xr-x.  2 root root 49152 Jan 15 11:54 usr/bin
dr-xr-xr-x. 40 root root  4096 Jan 10 09:01 usr/lib
drwxr-xr-x.  2 root root    63 Nov 26 11:20 usr/local/bin
drwxr-xr-x.  2 root root     6 Jun 22  2021 usr/local/lib
drwxr-xr-x.  2 root root     6 Jun 22  2021 usr/local/sbin
dr-xr-xr-x.  2 root root 20480 Jan 15 11:54 usr/sbin
$ 

8. Test

Test is used by virtually every shell script written. It may not seem that way, because test is not often called directly. test is more frequently called as [. [ is a symbolic link to test, just to make shell programs more readable. It is also normally a shell builtin (which means that the shell itself will interpret [ as meaning test, even if your Unix environment is set up differently):

$ 
$ 
$ type [
[ is a shell builtin
$ 
$ which [
/usr/bin/[
$ 
$ ls -l /usr/bin/[
-rwxr-xr-x. 1 root root 54872 Jun 24  2022 '/usr/bin/['
$ 
$ ls -l /usr/bin/test
-rwxr-xr-x. 1 root root 54816 Jun 24  2022 /usr/bin/test
$ 

 This means that '[' is actually a program, just like ls and other programs, so it must be surrounded by spaces:

if [$foo = "bar" ]

will not work; it is interpreted as if test$foo = "bar" ], which is a ']' without a beginning '['. Put spaces around all your operators. I've highlighted the mandatory spaces with the word 'SPACE' - replace 'SPACE' with an actual space; if there isn't a space there, it won't work:

Note: Some shells also accept "==" for string comparison; this is not portable, a single "=" should be used for strings, or "-eq" for integers.

Test is a simple but powerful comparison utility. For full details, run man test on your system, but here are some usages and typical examples.

Test is most often invoked indirectly via the if and while statements. It is also the reason you will come into difficulties if you create a program called test and try to run it, as this shell builtin will be called instead of your program!
The syntax for if...then...else... is:

if [ ... ]
then
  # if-code
else
  # else-code
fi

Also, be aware of the syntax - the "if [ ... ]" and the "then" commands must be on different lines. Alternatively, the semicolon ";" can separate them:

if [ ... ]; then
  # do something
fi

You can also use the elif, like this:

if  [ something ]; then
 echo "Something"
 elif [ something_else ]; then
   echo "Something else"
 else
   echo "None of the above"
fi

This will echo "Something" if the [ something ] test succeeds, otherwise it will test [ something_else ], and echo "Something else" if that succeeds. If all else fails, it will echo "None of the above".

$ vim test.sh
$ 
$ X=5
$ export X
$ ./test.sh
X is more than zero
X is more than or equal to zero
X is not the string "hello"
X is of nonzero length
No such file: 5
$ 
$ X=hello
$ ./test.sh
./test.sh: line 2: [: hello: integer expression expected
./test.sh: line 6: [: hello: integer expression expected
./test.sh: line 9: [: hello: integer expression expected
./test.sh: line 11: [: hello: integer expression expected
X matches the string "hello"
X is of nonzero length
No such file: hello
$ 
$ cat test.sh
#!/bin/sh
if [ "$X" -lt "0" ]
then
  echo "X is less than zero"
fi
if [ "$X" -gt "0" ]; then
  echo "X is more than zero"
fi
[ "$X" -le "0" ] && \
       echo "X is less than or equal to  zero"
[ "$X" -ge "0" ] && \
       echo "X is more than or equal to zero"
[ "$X" = "0" ] && \
       echo "X is the string or number \"0\""
[ "$X" = "hello" ] && \
       echo "X matches the string \"hello\""
[ "$X" != "hello" ] && \
       echo "X is not the string \"hello\""
[ -n "$X" ] && \
       echo "X is of nonzero length"
[ -f "$X" ] && \
       echo "X is the path of a real file" || \
       echo "No such file: $X"
[ -x "$X" ] && \
       echo "X is the path of an executable file"
[ "$X" -nt "/etc/passwd" ] && \
       echo "X is a file which is newer than /etc/passwd"
$