Linux: Bash String and Array Manipulation

why: linux - difference between $@ and $* in bash script - Stack Overflow

==> patterned/batch input arguments processing

Array variables

quick tutorial: Unix / Linux - Using Shell Arrays

more detailed: Array variables

10.2.1. Creating arrays

An array is a variable containing multiple values. Any variable may be used as an array. There is no maximum limit to the size of an array, nor any requirement that member variables be indexed or assigned contiguously. Arrays are zero-based: the first element is indexed with the number 0.

Indirect declaration is done using the following syntax to declare a variable:

ARRAY[INDEXNR]=value

The INDEXNR is treated as an arithmetic expression that must evaluate to a positive number.

Explicit declaration of an array is done using the declare built-in:

declare -a ARRAYNAME

A declaration with an index number will also be accepted, but the index number will be ignored. Attributes to the array may be specified using the declare and readonly built-ins. Attributes apply to all variables in the array; you can't have mixed arrays.

Array variables may also be created using compound assignments in this format:

ARRAY=(value1 value2 ... valueN)

Each value is then in the form of [indexnumber=]string. The index number is optional. If it is supplied, that index is assigned to it; otherwise the index of the element assigned is the number of the last index that was assigned, plus one. This format is accepted by declare as well. If no index numbers are supplied, indexing starts at zero.

Adding missing or extra members in an array is done using the syntax:

ARRAYNAME[indexnumber]=value

Remember that the read built-in provides the -a option, which allows for reading and assigning values for member variables of an array.

10.2.2. Dereferencing the variables in an array

In order to refer to the content of an item in an array, use curly braces. This is necessary, as you can see from the following example, to bypass the shell interpretation of expansion operators. If the index number is @ or *, all members of an array are referenced.

[bob in ~] ARRAY=(one two three) [bob in ~] echo ${ARRAY[*]} one two three [bob in ~] echo $ARRAY[*] one[*] [bob in ~] echo ${ARRAY[2]} three [bob in ~] ARRAY[3]=four [bob in ~] echo ${ARRAY[*]} one two three four

Referring to the content of a member variable of an array without providing an index number is the same as referring to the content of the first element, the one referenced with index number zero.

10.2.3. Deleting array variables

The unset built-in is used to destroy arrays or member variables of an array:

[bob in ~] unset ARRAY[1] [bob in ~] echo ${ARRAY[*]} one three four [bob in ~] unset ARRAY [bob in ~] echo ${ARRAY[*]} <--no output-->

10.2.4. Examples of arrays

Practical examples of the usage of arrays are hard to find. You will find plenty of scripts that don't really do anything on your system but that do use arrays to calculate mathematical series, for instance. And that would be one of the more interesting examples...most scripts just show what you can do with an array in an oversimplified and theoretical way.

The reason for this dullness is that arrays are rather complex structures. You will find that most practical examples for which arrays could be used are already implemented on your system using arrays, however on a lower level, in the C programming language in which most UNIX commands are written. A good example is the Bash history built-in command. Those readers who are interested might check the built-ins directory in the Bash source tree and take a look at fc.def, which is processed when compiling the built-ins.

Another reason good examples are hard to find is that not all shells support arrays, so they break compatibility.

After long days of searching, I finally found this example operating at an Internet provider. It distributes Apache web server configuration files onto hosts in a web farm:

#!/bin/bash if [ $(whoami) != 'root' ]; then echo "Must be root to run $0" exit 1; fi if [ -z $1 ]; then echo "Usage: $0 </path/to/httpd.conf>" exit 1 fi httpd_conf_new=$1 httpd_conf_path="/usr/local/apache/conf" login=htuser farm_hosts=(web03 web04 web05 web06 web07) for i in ${farm_hosts[@]}; do su $login -c "scp $httpd_conf_new ${i}:${httpd_conf_path}" su $login -c "ssh $i sudo /usr/local/apache/bin/apachectl graceful" done exit 0

First two tests are performed to check whether the correct user is running the script with the correct arguments. The names of the hosts that need to be configured are listed in the array farm_hosts. Then all these hosts are provided with the Apache configuration file, after which the daemon is restarted. Note the use of commands from the Secure Shell suite, encrypting the connections to remote hosts.

Thanks, Eugene and colleague, for this contribution.

Dan Richter contributed the following example. This is the problem he was confronted with:

"...In my company, we have demos on our web site, and every week someone has to test all of them. So I have a cron job that fills an array with the possible candidates, uses date +%W to find the week of the year, and does a modulo operation to find the correct index. The lucky person gets notified by e-mail."

And this was his way of solving it:

#!/bin/bash # This is get-tester-address.sh # # First, we test whether bash supports arrays. # (Support for arrays was only added recently.) # whotest[0]='test' || (echo 'Failure: arrays not supported in this version of bash.' && exit 2) # # Our list of candidates. (Feel free to add or # remove candidates.) # wholist=( 'Bob Smith <bob@example.com>' 'Jane L. Williams <jane@example.com>' 'Eric S. Raymond <esr@example.com>' 'Larry Wall <wall@example.com>' 'Linus Torvalds <linus@example.com>' ) # # Count the number of possible testers. # (Loop until we find an empty string.) # count=0 while [ "x${wholist[count]}" != "x" ] do count=$(( $count + 1 )) done # # Now we calculate whose turn it is. # week=`date '+%W'` # The week of the year (0..53). week=${week#0} # Remove possible leading zero. let "index = $week % $count" # week modulo count = the lucky person email=${wholist[index]} # Get the lucky person's e-mail address. echo $email # Output the person's e-mail address.

This script is then used in other scripts, such as this one, which uses a here document:

email=`get-tester-address.sh` # Find who to e-mail. hostname=`hostname` # This machine's name. # # Send e-mail to the right person. # mail $email -s '[Demo Testing]' <<EOF The lucky tester this week is: $email Reminder: the list of demos is here: http://web.example.com:8080/DemoSites (This e-mail was generated by $0 on ${hostname}.) EOF

note that

`command` not 'command' is command substitution, the same as $(command)

==> use the latter since it is much more reader friendly

see why here: BashFAQ/082 - Greg's Wiki

String

detailed guide: Manipulating Strings

quick tutorial: String Manipulation in Shell Scripting - GeeksforGeeks

String Manipulation in Shell Scripting

String Manipulation is defined as performing several operations on a string resulting change in its contents. In Shell Scripting, this can be done in two ways: pure bash string manipulation, and string manipulation via external commands. 

Basics of pure bash string manipulation:

1. Assigning content to a variable and printing its content: In bash, ‘$‘ followed by the variable name is used to print the content of the variable. Shell internally expands the variable with its value. This feature of the shell is also known as parameter expansion. Shell does not care about the type of variables and can store strings, integers, or real numbers.
Syntax:

VariableName='value'
echo $VariableName
or
VariableName="value"
echo ${VariableName}
or
VariableName=value
echo "$VariableName"

Note:

==>see shell - Difference between single and double quotes in Bash - Stack Overflow

Single quotes won't interpolate anything, but double quotes will. For example: variables, backticks, certain \ escapes, etc.

==> but $var and ${var} is effectively the same here

There should not be any space around the “=” sign in the variable assignment. When you use VariableName=value, the shell treats the “=” as an assignment operator and assigns the value to the variable. When you use VariableName = value, the shell assumes that VariableName is the name of a command and tries to execute it.

Example:

2. To print length of string inside Bash Shell: ‘#‘ symbol is used to print the length of a string.

Syntax:

variableName=value
echo ${#variablename}

Example:

3. Concatenate strings inside Bash Shell using variables: In bash, listing the strings together concatenates the string. The resulting string so formed is a new string containing all the listed strings.

Syntax:

var=${var1}${var2}${var3}
or
var=$var1$var2$var3
or
var="$var1""$var2""$var3"

To concatenate any character between the strings:

The following will insert "**" between the strings
var=${var1}**${var2}**${var3}
or
var=$var1**$var2**$var3
or
var="$var1"**"$var2"**"$var3"

The following concatenate the strings using space:
var=${var1} ${var2} ${var3}
or
var="$var1" "$var2" "$var3"
or
echo ${var1} ${var2} ${var3}

Note: While concatenating strings via space, avoid using var=$var1 $var2 $var3. Here, the shell assumes $var2 and $var3 as commands and tries to execute them, resulting in an error.

==> ' ' or space is special for Bash in short.

Example:

4. Concatenate strings inside Bash Shell using an array: In bash, arrays can also be used to concatenate strings. 

Syntax:

To create an array:
arr=("value1" value2 $value3)

To print an array:
echo ${arr[@]}

To print length of an array:
echo ${#arr[@]}

Using indices (index starts from 0):
echo ${arr[index]}

Note: echo ${arr} is the same as echo ${arr[0]}

Example:

5. Extract a substring from a string: In Bash, a substring of characters can be extracted from a string.

Syntax:

${string:position}  --> returns a substring starting from $position till end
${string:position:length} --> returns a substring of $length characters starting from $position.

Note: $length and $position must be always greater than or equal to zero. 

If the $position is less than 0, it will print the complete string.

If the $length is less than 0, it will raise an error and will not execute.

Example:

6. Substring matching: In Bash, the shortest and longest possible match of a substring can be found and deleted from either front or back.

Syntax:

To delete the shortest substring match from front of $string:
${string#substring}

To delete the shortest substring match from back of $string:
${string%substring}

To delete the longest substring match from front of $string:
${string##substring}

To delete the shortest substring match from back of $string of $string:
${string%%substring}

Example:

In the above example: 

• The first echo statement substring ‘*.‘ matches the characters ending with a dot, and # deletes the shortest match of the substring from the front of the string, so it strips the substring ‘Welcome.‘.
• The second echo statement substring ‘.*‘ matches the substring starting with a dot and ending with characters, and % deletes the shortest match of the substring from the back of the string, so it strips the substring ‘.GeeksForGeeks‘
• The third echo statement substring ‘*.‘ matches the characters ending with a dot, and ## deletes the longest match of the substring from the front of the string, so it strips the substring ‘Welcome.to.‘
• The fourth echo statement substring ‘.*‘ matches the substring starting with a dot and ending with characters, and %% deletes the longest match of the substring from the back of the string, so it strips the substring ‘.to.GeeksForGeeks‘.