Notes for The Advanced Bash Scripting Guide

BASH is not as rigorous as Python.

BASICS

Special Characters

# ; ;; . “ ‘ , \ / ` : ! * ** ? $ ${} $’’ $\ast ,$@ $? $$ () {xxx,yyy,zzz,…} {a…z} {} [] [[]] (()) > &> >& >> < <> << <<< <,> <,\> | || & && - – = + + % ~ ^ Ctl-H Ctl-Z Ctl-W

Introduction to Variables and Parameters

Variable Substitution

$Variable ${Variable}

Variable Assignment

=, ``, $()

Bash variables are untyped

Special variable types

Local, Environmental, Positional

Quoting

“” ‘’ \

Exit

The equivalent of a bare exit is exit $? or even just omitting the exit.

Tests

Test Constructs

Test, [, [[, ((

if [ condition1 ]
then
 ...
elif [ condition2 ]
then
 ...
else
 ...
fi

File test operators

-e -f -s -d -b -c -p -h -S -t -r -w -x -g -u -k -O -G -N -nt -ot -ef !

integer comparison

-eq -ne -gt -ge -lt -le < <= > >=

string comparison

= == != < > -z -n

compound comparison

-a -o && ||

Operations and Related Topics

Operators

assignment

=

arithmetic operators

+ - * / ** % += -= *= /= %=

bitwise operators

<< <<= >> >>= & &= | |= ~ ^ ^=

logical (boolean) operators

! && ||

miscellaneous operators

,

Numerical Constants

31, 024, 0xff, 64#@_

Operator Precedence

Beyond the Basics

Another Look at Variables

Internal Variables

$HOME, $IFS, $LC_COLLATE, $PATH, $PPID, $UID, $0,$1,etc, $#, $*, $@, $?, $$

Generate random integer

RANDOM=$$ #Reseed the random number generator using script process ID.
number=$RANDOM

Manipulating Variables

Manipulating Strings

${#strvar}
${strvar:pos:len}
${strvar#shortestfrontsubstring}
${strvar##longestlfrontsubstring}
${strvar%shortestbacksubstring} e.g. mv $filename ${filename%$1}$2
${strvar%%longestlbacksubstring}
${strvar/substring/replacement}
${strvar//substring/replacement}

Parameter Substitution

${parameter} same as $parameter,e.g.PATH=${PATH}:/opt/bin
${parameter=default}

Loops and Branches

Loops

for arg in [list]
do
 command(s)...
done
while [ condition ]
do
 command(s)...
done

Nested Loops

Loop Control

Break, continue

Testing and Branching

case "$Keypress" in
 [[:lower:]] ) echo "Lowercase letter";;
 [[:upper:]] ) echo "Uppercase letter";;
 [0-9] ) echo "Digit";;
 * ) echos "Punctuation, whitespace, or other";;
esac

Command Substitution

`` $()

Arithmetic Expansion

$(())e.g.z=$(($z+3)), (( n += 1 ))

Commands

Builtins

I/O: Echo, printf, read
Variables: let, set, eval, unset, export, getopts,
File system: cd, pwd
Script: source/., exit, exec,
Commands: true, false, type, kill, help

Job Control Commands

&, jobs, fg, bg, wait, kill -9 %N, killall,

External Filters, Programs and Commands

Basic Commands

ls, cat, cp, mv, rm, rmdir, chmod, chown, chattr, ln, man

Complex Commands

find ~/ -name 'core\*' -exec rm {} \;  
find /home/bozo/projects -mtime -1;  
find "$DIR" -type f -atime +5 -exec rm {} \;  
find /etc -exec grep '[0-9][0-9]\*[.][0-9][0-9]\*[.][0-9][0-9]\*[.][0-9][0-9]\*' {} \;  
ls | xargs -n 8 echo  
find / -type f  -print0 | xargs -0 grep -liwZ GUI | xargs -0 rm -f  
ls *gif | xargs -t -n1 -P2 gif2png    
find . -print0 | xargs -0 COMMAND      
while IFS="" read -r -d "" file <&4 ; do COMMAND "$file"; done 4< <(find . -print0)   

Time/Date Commands

Date, time, touch, sleep, clock

Text Processing Commands

Sort, uniq, cut, head, tail, grep (-i -w -l -r -n -v -c -P), sed, awk, wc, tr, nl, mkfifo

File and Archiving Commands

Tar(-c -z -j), rpm, file, which, diff, patch, basename, dirname, md5sum, sha1sum, openssl, make, more, less

Communication Commands

Nslookup, dig, traceroute, ping, ftp, telnet, wget, ssh, rsync, mailto, wall

Terminal Control Commands

Reset, clear

Math Commands

Factor, bc

Miscellaneous Commands

Seq, run-parts, tee, yes, dd(if,of,bs,count), od,

System and Administrative Commands

Users, groups, chown, useradd, userdel, id, w, su, sudo, last, passwd, wall, uname, lsof, nc, free, du, df, uptime, logrotate, ps, pstree, top, nohup, init, halt, reboot, shutdown, service, nmap, ifconfig, ip, netstat, route, iptables, tcpdump, mount, umount, sync, fdisk, lspci, chroot, makedev, dump, restore, ulimit, quota, umask, lsmod, insmod, modprobe, rmmod, modinfo, env, ldd, watch, fuser

Advanced Topics

Regular Expressions

A Brief Introduction to Regular Expressions

=~
A character set, a anchor, and/or modifiers
Escaped “angle brackets” – <…> – mark word boundaries.
egrep ‘re(a|e)d’ misc.txt

Globbing

Globbing interprets the standard wildcard characters – * and ?, character lists in square brackets, and certain other special characters (such as ^ for negating the sense of a match).

Here Documents

COMMAND <<InputComesFromHERE
…
InputComesFromHERE

: <<COMMENTBLOCK
echo "This line will not echo."
This is a comment line missing the "#" prefix.
This is another comment line missing the "#" prefix.
&*@!!++=
The above line will cause no error message,
because the Bash interpreter will ignore it.
A here document supports parameter and command substitution.
COMMENTBLOCK

Here Strings

String=“This is a string of words.”
read -r -a Words <<< “$String”

I/O Redirections

COMMAND_OUTPUT >
: > filename
> filename 
COMMAND_OUTPUT >>
1>filename
1>>filename
2>filename
2>>filename
2>&1
>>filename 2>&1
2>&1 | [command(s)
i>&j
0< FILENAME
< FILENAME
j<>filename

Using exec

exec 3<> myfile.txt
while read line <&3
do {
 echo "$line"
 (( Lines++ ));
}
done
exec 3>&-

exec 5<>/dev/tcp/www.net.cn/80
echo -e "GET / HTTP/1.0\n" >&5
cat <&5

Redirecting Code Blocks

for name in `seq $line_count`
do
 read name
 echo "$name"
 if [ "$name" = "$FinalName" ]
 then
 break
 fi 
done < "$Filename" > "$Savefile"

Applications

Subshells

A subshell is a child process launched by a shell (or shell script).
An external command in a script forks off a subprocess, whereas a Bash builtin does not.
A command list embedded between parentheses runs as a subshell.
Processes may execute in parallel within different subshells.

Restricted Shells

Process Substitution

>(command_list)  
<(command_list)  
There is no space between the "<" or ">" and the parentheses.  
Process substitution can compare the contents of two directories.  
diff <(ls $first_directory) <(ls $second_directory)  
read -a list < <( od -Ad -w24 -t u2 /dev/urandom ) #Read a list of random numbers from /dev/urandom.  

Functions

function_name () {
command...
}

Complex Functions and Function Complexities

file_excerpt ()
{ 
while read line # "while" does not necessarily need [ condition ]
do
 echo "$line" | grep $1 | awk -F":" '{ print $5 }'
done
} <$file
file_excerpt $pattern
# Try this:
Function ()
{
 {
 ...
 } < file
}

Local Variables

fact ()
{
 local number=$1
 # Variable "number" must be declared as local,
 #+ otherwise this doesn't work.
 if [ "$number" -eq 0 ]
 then
 factorial=1 # Factorial of 0 = 1.
 else
 let "decrnum = number - 1"
 fact $decrnum # Recursive function call (the function calls itself).
 let "factorial = $number * $?"
 fi
 return $factorial
}

Recursion Without Local Variables

#!/bin/bash
# recursion-def.sh
# A script that defines "recursion" in a rather graphic way.
RECURSIONS=10
r_count=0
sp=" "
define_recursion ()
{
 ((r_count++))
 sp="$sp"" "
 echo -n "$sp"
 echo "\"The act of recurring ... \"" # Per 1913 Webster's dictionary.
 while [ $r_count -le $RECURSIONS ]
 do
 define_recursion
 done
}
echo
echo "Recursion: "
define_recursion
echo
exit $?

Aliases

List Contructs

command-1 && command-2 && command-3 && … command-n
command-1 || command-2 || command-3 || … command-n

Arrays

base64_charset=( {A…Z} {a…z} {0…9} + / = )

Indirect References

The notation is $$var, usually preceded by an eval (and sometimes an echo).

a=b
b=c
eval x=\$$a
echo $x

/dev and /proc

/dev

/dev/loop0, /dev/null, /dev/zero, /dev/urandom, /dev/sda1, /dev/udp, /dev/tcp

/proc

/proc/loadavg, /proc/meminfo, /proc/cpuinfo, etc.

Network Programming

lsof -ni | grep TCP

Of Zeros and Nulls

cat /dev/null > /var/log/messages
: > /var/log/messages
dd if=/dev/zero of=$FILE bs=$BLOCKSIZE count=$blocks

Debugging

Options

Miscellany

Interactive and non-interactive shells and scripts

Shell Wrappers

Tests and Comparisons: Alternatives

Recursion

RANGE=10
MAXVAL=9
i=$RANDOM
let "i %= $RANGE"
if [ "$i" -lt "$MAXVAL" ]
then
 echo "i = $i"
 ./$0
fi
exit 0

Optimization

Assorted Tips

Security Issues

Portability Issues

Appendix C.

sed

sed "/^$pattern/d" "$filename"  
sed -n '/xzy/p' $filename  
8d  
/^$/d  
1,/^$/d  
/Jones/p  
s/Windows/Linux/  
s/BSOD/stability/g  
s/ *$//  
s/00*/0/g  
echo "Working on it." | sed -e '1i How far are you along?'  
5i 'Linux is great.' file.txt  
/GUI/d  
s/GUI//g  
sed -n 's/.*url="\([^"]*\)".*/\1/p')  
sed -e 's/\.//g' -e 's/\,//g' -e 's/ //g' "$1"  
sed -e 's:^0[bBxX]::'  

awk

cut -d ' ' -f2,3 filename is equivalent to awk -F'[ ]' '{ print $2, $3 }' filename  
awk 'BEGIN{FS=":"}{print $1}' < "$PASSWORD_FILE"  

Appendix H. Important Files

startup files

/etc/profile, /etc/bashrc, $HOME/.bash_profile, $HOME/.bashrc

logout file

$HOME/.bash_logout

data files

/etc/passwd, /etc/shadow, /etc/group

Reference https://tldp.org/LDP/abs/html/