UNIX.Shells.by.Example.4th.Edition StudyNote

Some basic command:
touch:Create a file if it's already exists it will update the modified date
mkdir:create a directory:mkdir jobs/log/test/test2
cd:direct to a directory
pwd:displays the current working directory
cp:copy files:cp /jobs/log/test/test2/test3.txt test4.txt:copy test3 to test3 at current directory
with an -i option, it will check before overridden a existed file.
rm:remove a file
with an -r option,it removes a directory
find:Format:find pathname -options
paramaters:
-name:search by name
-mtime-n+n: searche by modified time
-type search by file type
ExAMPLE:find . -name \*.txt -print>lslist 
echo find . -name \*.txt


Basic Commands for VI editor
I:Insert words
ESC: Change mode from Insert to command
dd:delete a line
dw:delete a word
o:INSERT a new line




Chapter 13,The Interactive Bash Shell


13.1 Introduction
PS:shows what processes are running
$ps: 
UID:User's Identification
PID:process Identification
PPID:Parent PID
STIME:Start Time
COMMAND:What processes are running


13.3 The Command Line
If a command line is too long, the backslash character(\),followed by a newline,will allow
you to continue typing on the next line.


type:To find out the type of command you are using--a built-in command,an alias,a function
or an executable.


Echo $?:The exit status is a number between 0 and 255.
if the status returned is 0,the command was successful in its execution,
when the exit status is nonzero,the command is failed in some way.


13.3.5 Multiple Commands at the Command Line
A command line can consist of multiple commands. Each command is separated by
a ;
Example $ls;pwd;date


Commands may also be grouped so that all of the output is either piped to another
command or redirected to a file
Example $(ls;pwd;date)>testFile


13.3.7 Conditiobnal Execution of Commands
$cc progm1.c -o prgm1 && prgm1
if cc program can compile prgm1.c then prgm1 will be executed.
||if not successfully executed then the program after || would be executed.


13.4 Job Control
Ctrl-C or Ctrl-\ to terminate a job,
Ctrl-Z sends the job to background and suspends it.


$jobs :To display the jobs currently in the background
$jobs %%: To displays the most recent command put in the job table.
%jobs -1:Displays the process running in the background and the PID numbers of those jobs.
-r option lists all running jobs and the -s optiono lists all stopped jobs.


13.9. Filename Substitution
*          Matches zero or more characters
?          Matches exactly one character
[abc]      matches on character in the set of a,b, or c :ls [abc].txt
{a,ile,ax} matches for a character or set of characters :ls {sub,aa}*   mkdir /logs/test/{test1,test2,test3}
\          Escapes or disables the metacharacter 
echo ?.txt 
a.txt b.txt
echo \?.txt
\?.txt
noglob variable is set or if the set command is give a -f option
means metacharacters represent themselves; They are not used as wildcards
unset noglob or set +f means metacharacters will be expanded.




13.10. Variables
Format: delcare variable=value
Example: round=world
$round //a dollar sign is used to extract the value stored there
world


name="Alice Zhang"//when assigning a value,there can be no whitespace surrounding the equal sign
//so the quotes are needed to hide the whitespace
echo $name
Alice Zhang


The scope of local variables is confined to the shell in which the variable is created.




Environment Variables
The export command is used either after assigning a value or the variable is set.
The declare command given the -x option will do the same.
Format:exprot variable=value
variable=value;export variable
declare -x variable=value
Example:
$NAME="John Smith"
$export NAME
$echo $NAME
John Smith
$bash
$echo $NAME
John Smith
$declare name="Jennerif"
$echo $name
Jennerif
$exit
$echo $name
John Smith


The printf Command
Format: printf [argument...]
Example:
$printf "%s is his name." "john"
$printf "%.3f" 100.1234 
100.123


13.11 Quoting
Quoting is used to protect special metacharacters from interpretation
and prevent parameter expansion. There are three methods of quoting:
the backslash,single quotes and double quotes.
 
Special metacharacters requiring qutoes
; & ()  {} | < > newline space/tab $ * [] ?
the backslash is not interpreted when enclosed in single quotes
$echo '\$5.00'
\$5.00


13.13.Arithmetic Expansion
FORMAT $[expression]
$((expression))
EXAMPLE:
echo $[2+2*4]


13.15.Arrays
FORMAT:
declare -a variable_name
variable=(item1 item2 item3)
EXAMPLE:
name=(tom dick harry)
$echo ${name[0]}
tom
$echo ${name[*]}
tom dick harry


Use the unset command followed by the array name to delete the whole array
unset ${name[0]}:this will remove tom


13.16 FUNCTIONS
FORMAT: function Function_name {commands;commands;}
EXAMPLE:
$function welcome {echo "Hi $1 and $2";} #used two positional parameters
$welcome tom joe #the arguments tom and joe are assigned to $1 and $2.
Hi tom and joe
$set jane anna lizzy #these variables will not affect the ones in function
$echo $*
jane anna lizzy
$welcome johan joe
hi johan and joe
$echo $1 $2
johan joe
$unset -f welcom #unsets the function


13.17. Standard I/O and Redirection
The Here Document
if the terminator is preceded by the << operate,
leading tabs, and only tabs may precede the final terminator
Example:
$cat<<Finish
>Hello $LOGNAME
>Date
Finish




Chapter 14. Programming the Bash Shell
cat>avi
if($#=1)
then
cp $1 $jobs/New
fi
vi $1
exit 0
[ctrl+d]


chmod +x avi


./avi


14.2 Reading User Input
EXAMPLE:
$Read
$Echo hi $REPLY #a line is read from standard input and stored in the built-in REPLY variable.
-----------------------------------------
$read first last
a b c d
$echo "hello $first and $last"
hello a and b c d # the first word of input to the variable first,all the
#rest of the words up to the vaiable last
-------------------------------------------
read -p "Enter your job title:" #waits for input and stores input in REPLY variable.
-------------------------------------------
read -a friends
echo "say hi to ${friends[2]}" #with the -a option,the read command takes input as array of words.Array indices 
#start at 0.




14.3.Arithmetic
$declare -i num # -i option creates an integer
$num=hello
$echo $num
0  #assign the string to int integer variable num causes the
string to be stored as 0
the(())(let command) syntax is used for arithmetic operations;
 
Using Different Bases:
FORMAT:variable=base#number
EMAMPLE:2#101   #base is 2,number is in base 2


The INTEGER
the let command is a bash built-in command that is used to
perform integer anrithmetic.
$EXAMPLE:i=5
$let i=i+1
echo $i
6


THE FLOATING-POINT ARITHMETIC
bash supports only integer arithmetic,but the bs,awk can performance complex calculations
x=1.1
y=2.11
echo "$x+$y"|bc


14.11 Positional Parameters
$0 is for the script name.
$1 for the first argument,$2 for the second argument and so on.
$# variable shows the number of positional paramenters.
$* is to display all of them.
use set -- to clear all the parameters


EXAMPLE:
cat>greetings2
echo "this script is called $0"
echo "$0 $1 and $2"
echo "the number of parameters is $#"
[ctrl+d]


chmod +x greetings2


1.  ./greetings2 
this script is called ./greetings2
./greetings2 and
the number of parameters is 0
2.  ./greetings2 tommy david mike
this script is called ./greetings2
./greetings tommy david
the number of parameters is 3.


14.5 Conditional Constructs and Flow Control
14.4.2 the Built-in test and let Commands
test or [] commadn is used to evaluate strings,numbers and perform file testing,
it returns an exit status. If exit status is o the expression is true; if the exit
status is 1, the expression evaluates to false.
EXAMPLE:
$name=tom
$test $name!=tom
$echo $?
1


$[ $name=tom ] #it must be spaces after the bracket.
$echo $?
0


$x=5
$y=2
$ [ $x -gt $y ] #gt stands for greater
$ echo $?
0




The test Command Operators Test Operator
 Tests True If
 
String Test
 


[ string1 = string2 ]  String1 is equal to String2 (space surrounding = required).


[ string1==string2 ]   (Can be used instead of the single = sign on bash versions 2.x.)
 
[ string1 != string2 ]
 String1 is not equal to String2 (space surrounding != required).
 
[ string ]
 String is not null.
 
[ –z string ]
 Length of string is zero.
 
[ –n string ]
 Length of string is nonzero.
 
[ –l string ]
 Length of string (number of characters).
 
  EXAMPLE


test –n $word      or     [ –n $word ]
test tom = sue      or     [ tom = sue ]


 
Logical Test
 
[ string1 –a string1 ]
 Both string1 and string2 are true.
 
[ string1 –o string2 ]
 Either string1 or string2 is true.
 
[ ! string1 ]
 Not a string1 match.
 
Logical Test (Compound Test)[a]
 
[[ pattern1 && pattern2 ]]
 Both pattern1 and pattern2 are true.
 
[[ pattern1 || pattern2 ]]
 Either pattern1 or Fis true.
 
[[ ! pattern ]]
 Not a pattern match.
 
Integer Test
 
[ int1 –eq int2 ]
 Int1 is equal to int2.
 
[ int1 –ne int2 ]
 Int1 is not equal to int2.
 
[ int1 –gt int2 ]
 Int1 is greater than int2.
 
[ int1 –ge int2 ]
 Int1 is greater than or equal to int2.
 
[ int1 –lt int2 ]
 Int1 is less than int2.
 
[ int1 –le int2 ]
 Int1 is less than or equal to int2.
 
Binary Operators for File Testing
 
[ file1 –nt file2 ]
 True if file1 is newer than file2 (according to modification date).
 
[ file1 –ot file2 ]
 True if file1 is older than file2.
 
[ file1 –ef file2 ]
 True if file1and file2 have the same device or inode numbers.
 
14.5.3. The if Command
FORMAT:
if command
then
   command
   command
fi


(Using test for numbers and strings -- old format)
    if test expression


    then


        command


    fi


            or




    if [ string/numeric expression ]  then


             command


    fi


-------------------------------------
(Using test for strings -- new format)


    if [[ string expression ]]     then


             command


    fi


(Using let for numbers -- new format)


   if (( numeric expression ))




EXAMPLE:
   echo  "Are you o.k. (y/n) ?"
    read answer
   if [ "$answer" = Y -o "$answer" = y ]
    then
        echo  "Glad to hear it."
   fi
---------------------------------------------------
  if (( $# != 2 ))[a]          # [ $# -ne 2 ]


    then


        echo  "Usage:   $0 mdays size " 1>&2


        exit 1


  fi
---------------------------------------------------
$ cat > bigfiles
if (( $1<0 || $1>30 ))
then
echo "out of range"
exit 2
fi


$ chmod +x bigfiles
$ ./bigfiles 400
out of range


Checking for Null Value
Use double quotes to stand for null values
EXAMPLE:
if [ "$name" = "" ]    
then
    echo The name variable is null
fi


Nested if Commands
if  command
then
    command(s)
else
    command(s)
fi


expr:calcution
EXAMPLE: 
$expr 10 + 10 #space between options
20




14.5.8 The Case Command
FORMAT
case variable in
value1)
    command(s)
    ;;
value2)
    command(s)
    ;;
*)
command(s)
    ;;
esac


EXAMPLE:
case "$1" in  #the script with parameters must use positional parmeters
blue)
echo "choose blue"
;;
green)
echo "choose green"
;;
red)
echo "choose red"
;;
*)
echo "not match"
;;
esac
echo "out of case"


Creating Menus with the here document and case document.
The here document is used to create a menu of choices that will
be displayed to the screen. 
EXAMPLE:
echo "select a terminal type"
cat<< ENDIT
1)unix
2)xterm
3)sun
ENDIT
read choice
case "$choice" in
1) 
echo "unix"
;;
2) 
echo "xterm"
;;
3)echo "sun"
;;
*)
echo "Error"
;;
esac
echo "the tere is $choice"




Looping Command
The for Command
FORMAT:
for variable in word_list
do
    command(s)
done


EXAMPLE1:
$ for name in tom jerry jack
> do 
> echo "name is $name"
> done


EXAMPLE2:
$ cat > mynames   
tome
jerry
kate
alice


$ for person in $(cat mynames) #The contents of a file called mylist are displayed.
> do 
> echo $person
> done


EXAMPLE3:
$ cat > memo1 <<EOF #Create three files
> EOF
$ cat > memo2 << EOF
> EOF
$ cat > memo3 << EOF
> EOF


$ for file in memo[1-3] #the wordlist will consist of all files in the current working directory with name
#starting with memo and ending with numbers
> do 
> echo "filename is $file"
> done


EXAMPLE4:
for name in $*
do
echo "hi $name"
done


$ ./testname peter mary
hi peter 
hi mary


EXAMPLE5:
for file
do
.
.
.
#the sell expands the * to all filenames in the current working directory.


14.6.3
The While Command
FORMAT
while command
do
command(s)
done


EXAMPLE1:
num=0
while [ $num -lt 10 ]
do
echo "$num"
let num+=1
done
echo "finisheed"


EXAMPLE2:
read answer
while [ $answer != "a" ]
do
echo "wrong answer"
read answer
done


EXAMPLE3:
go=start
while [[ -n "$go" ]]
do
echo "hello Shell"
read go
if [[ "$go" == "q" && "$go" != "w" ]]
then
echo "goodbye shell"
go=
fi
done




14.6.4 The until Command
FORMAT
until command
do
    command(s)
done


EXAMPLE1:




14.6.5 The select Command
FORMAT:
select var in wordlist
do
    command(s)
done




EXAMPLE1:
PS3="select a program"
select pro in ls pwd date  #The PS3 prompt is assigned the string that will appear below the menu that the select loop displays
do
echo "your choose $pro"
$pro
done




EXAMPLE2:
PS3="Make your choice"
select choice in a b c
do
case "$choice" in
a)
echo "you choice a"
;;
b|c)
echo "you choice b or c"
;;
*)
echo "try again"
;;
esac
done




14.6.6 Looping Control Commands


The shift command shifts the parameter list to the left a specified number of times.
The shift command without an argument shifts the parameter list once to the left.
Once the list is shifted, the parameter is removed permanently.
FORMAT
shift [n]


EXAMPLE1:
bash-3.00$ set `date`       
bash-3.00$ echo $*
Thu Jun 19 04:44:25 EDT 2014
bash-3.00$ shift 5
bash-3.00$ echo $*
2014


EXAMPLE2:
$while [ $# -gt 0 ]
$do
$echo $*
$shift
$done
while [ $# -gt 0 ]
do
echo $*
shift
done




The Break Command
The break command causes an exit from the innermost loop,
so if you have nested loops, the break command takes a number as an argument, 
allowing you to break out of a specific outer loop. If you are nested in three loops,
the outermost loop is loop number 3, the next nested loop is loop number 2, 
and the innermost nested loop is loop number 1. 




The Continue Command:
The continue command returns control to the top of the loop if some condition becomes true.
Format continue [n]


EXAMPLE1:
cat>list
Peter
john
may


for name in $(cat list)
do
if [ "$name" = "john" ]
then
continue
else
echo "send maile to $name"
fi
done


EXAMPLE2:


for month in Jan Feb Mar Apr
do
   for week in 1 2 3 4
      do
        echo "process $month?"
        read ans
        if [ "$ans" = "N" ]
        then
           continue 2            #back to the outer loop
         else
         echo "process month $month week $week?"
         read ans
         if [ "$ans" = "N" ]
         then
         continue                #back to the inner loop
         else
         echo "now processing week $week of $month"
         fi
        fi
done
done


14.6.7. Input can be piped or redirected to a loop from a file. 
Output can also be piped or redirected to a file from a loop. 


EXAMPLE1:
$cat > memo
abc
def
ghi


#script:put line numbers on all lines of memo
$count=1
$cat $1 | while read line  #read files one by one
$do
$echo "$count.$line"
$$let count+=1
$done > tmp$$
$mv tmp$$ $1   #The tmp file is renamed to the filename assigned to $1.




Connect to Database
nzsql -t -d qsdb_dev -u qsdb_dev_admin -pw password -o /tmp/tmpsqltest2.txt -c "  select * from IVDB_OPTION_INFO" -F '|' 


awk
cat /tmp/tmpsqltest2.txt | awk '{print $1}' $1 means first column $n means n column


YesterdayCount=$(awk 'NR>3 && NR<5 {print $2}' /tmp/sqltest.txt) #get line 3 to line 5 2 column
TodayCount=`sed -n '5,5p' /tmp/sqltest.txt` 




expr support matheratic operation * needs add \
EXAMPLE:`expr 3 \* 2`
expr doesn't support float operation. Use bc