|
|
Home | Switchboard | Unix Administration | Red Hat | TCP/IP Networks | Neoliberalism | Toxic Managers |
| (slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and bastardization of classic Unix | |||||||
This page is based on a lecture by Prof Nikolai Bezroukov, which in turn was partially based on Chapter 6 of Linux Shell Scripting with Bash by Ken O. Burtch
An expression in bash can be understood as formula that calculates some value. There are several types of expressions in bash. We will mention only three:
bash simultaneously is a scripting language and a macro processor. So before expression is calculated its content is subjected to several passes of analysis on each of which specific language element are processed (which is called macro expressions). In other languages there is a separate "preprocessor" phase, performed by a separate program. But in bash they are integrated into interpreter. During this process of macroexpansion macrovariables are replaced by their values. Each such action is usually called macrosubstitution or simply substitution. In case of double quoted literals the process of substituting of values of Bash variables into their values is also called interpolation (the term more widely used in Perl then in bash).
Bash performs certain types of macrosubstitution only outside literals(single or double quoted strings and backticked string are called literals) . For example Bash expands tilde to the user home directory only outside any literals
Outside literals after the values of variables and backticked strings were substituted Bash performs two additional operations:
The result is passed to syntax phase of Bash interpreter, where the syntax analyzer try to split script text into statements. If errors are found the process is aborted. After that script is executed one statement at a time.
The order is important and can cause subtle problems in scripts. Suppose you assign a path with a tilde in it in a variable (tile means home directory in Bash):
ls -d ~/bin /home/joeuser/bin
You can see the tilde here work as expected. Now let's enclose the expression ~/bin in double quotes:
$ TEMP="~/bin" $ ls $TEMP ~/bin not found
You can see that this does not work -- tilde substitution double quoted literals not performed as explained above. In double quotes literals tilde symbol is not treated as marcosymbol, represnting home directory, but as a regular symbol. that means that within any literal including double quote literals the tilde will remain "as is". In such cases you should use the $HOME variable instead.
Conditional expressions were not part of the original Borne shell (which was a blunder on the part of its designer even in the situation when memory was tight -- ancient computers that run Unix often have one megabyte (megabyte, not gigabyte) of memory or even less.
In any case in original Borne shell conditional expressions were implemented via external command called test which accepts sequence of arguments which are interpreted by this program as expression. As later version of shell (Korn shell) introduced double square brackets and double round brackets expressions which are much better deal, this archaic way should better be forgotten (double square brackets or double round brackets expressions should be used exclusively in you new scrips), but it is preserved for compatibility so we need to know about its existence.
the test command evaluates expression and return the corresponding "return code" -- 0 if expression is true, and non-zero it is false.
Now let's see hot it issued within bash if command, which allow to structure control stream of your scripts into two branches: then branch which is executed if the condition is true and (optional) else branch which is executed, if the conditions false.
Bash allows for a wide variety of tests for files. It can test the type of file, the accessibility of the file, compare the ages of files, or other file attributes. The following is a complete list of Bash file tests:
File testing is commonly used in the sanity checks at the beginning of a script. They can be used to check that all files are present and readable (and writable, if necessary). All commands must be executable
#!/bin/bash # # Nikolai Bezroukov shopt -s -o nounset # get machine from SGE $PE_HOSTFILE variable
cat /dev/null > machines # make ./machine a zero length file and add lines to it using while loop
cat $PE_HOSTFILE | while read line; do
host=`echo $line | cut -d" " -f1`
cores=`echo $line | cut -d" " -f2`
while (($cores > 0 )) ; do
echo $host >> machines
let cores--
done
done
## done with $PE_HOSTFILE
# Sanity check
if [[ -f ./machines ]] ; then echo "The ./machines file exists. We can proceed" else echo "The machines file does not exits. Exiting ..." exit fi
This script fragment ensures that the who command is executable and that the temp file named in TMP either doesn't exist or exists and can be overwritten.
The built-in let command performs math calculations on variables without declaring then to be integer. let expects a string containing a variable, an equals sign, and an expression to calculate. the result is assigned to the variable.
let "large_run=$cores*2" printf "%d" $large_run
One good thing about ((...)) is that they are impressive: if you forget to put dollar sign in front of the variable name to extract the value of this this variable, it still will interpret the statement correctly. Which is a big plus for users of other languages, other then Perl:
let "large_run=cores*2" printf "%d" $large_run
Double quote are required with let because you need to prevent bash from interpreting special characters such as "*" and "/" that are used in bash for other purposes too. This way you can avoid subtle ambiguity due to double meaning of * and / in bash. You do not need to do this is you use ((..)), so this is preferable form for complex arithmetic expressions and the only one that is suitable for use in if , while and other conditional statement. The same can better be written as
(( large_run=$cores*3 )) printf "%d" $large_run
Another example
$ (( double_cores = 2*$cores )) $ echo $double_coresA couple more examples
(( RC = 5 / 2 )) printf "5 divided by 2 is %d\n" "$RC" 5 divided by 2 is 2 (( RC = 5 % 2 )) printf "remainder of 5 divided by 2 is %d\n" "$RC" remainder of 5 divided by 2 is 1
Here are the arithmetic operators in order of precedence. Many of these operators will be familiar to C programmers:
The parentheses must be escaped to prevent the shell from treating them as a reference to a subshell.
In if statement true is represented by the value of 1, and false by 0. Any value other than 1 or 0 is treated as true, but the logical operators themselves only return 1 or 0.Unlike string comparisons, let provides a full complement of numeric comparisons. These are of limited value because most of comparisons are tested with the test command in the if command, resulting in two sets of tests. In logical expressions, 0 is a failure.
declare -i RC=(( 1> 0 )
If (( $RC )) ; then printf "1 greater than 0 is TRUE: %d\n" else printf "something is really wrong %d\n" $RC exit fi
As we already know, self-referential operators are shorthand notations that combine assignment with one of the other basic operations.
This is the only case where let is superior of ((..)) expressions.
declare NODE="HP_blade" declare -i cores=20 if [[ $NODE == 'HP_blade" ]] ; let cores+=8 } printf "We will use %d cores" cores
The other self-referential operators are multiplication (*=), division (/=), remainder (%=), subtraction (-=), right shift (<<=), left shift (>>=), bitwise AND (&=), bitwise exclusive OR (^=) , and bitwise OR (|=).
Notice that certain kinds of self-referential operations are impossible to express with the shorthand operators. count=count-10 can be written as count-=10 but count=10-count must be written out in full.
The increment (++) operator adds one to a variable. The decrement operator (--) subtracts one from a variable.
$ RUNNO=`cat ./runno" $ let RUNNO++" printf "================= RUN %d ========================\n" "$RUNNO"
The self-referential operators cannot be used with integer variables without the let statement.
$ COST+=5 bash: COST+=5: command not found
Parentheses are allowed in the expressions.
if (( $cores=(20+8)*3 )) then printf "WARNING ! You are probably too greedy and this amount of cores will porbably slow down your computation %d" cores fi
Assignment in the let command is an operator that returns the value being assigned. As a result, multiple variables can be assigned at once.
(( threads=cores=16 )) printf "wr will be using are %d threads and %d cores\n" $threads $cores
let can evaluate more than one expression at a time. Several, small assignments can be combined on one line.
let "SUM=5+5" "SUM2=10+5"
NOTE: Excessive numbers of assignments in a single let command will lead to readability problems in a script. When each let is on a single line, it's easier to look through the script for a specific assignment.
The conditional expression operator (?) is shorthand for an if statement when one of two different expression are evaluated based on the left condition. Because this is a feature of the let command, it works only with numeric expressions, not strings. The following example constrains a truth value to a 1 or a 0.
cores=16 let "NODE= $cores > 20 ? 'HP_blade' : 'Dell_blade' printf "%d\n" $cores
Often you are better off using regular if.
input_file_count=`ls -l *.in | wc -l`
if (( $input_file_count == 0 )) ; then echo No input file provides. Exiting..." exit fi
Again, in most case it is better to use double parentheses then let. Two exceptions are simple assignment of a type a=b+c and self referential oprators. In both cases you do not need to use double quotes with let unless you use * or /.
Double parenthesis are also natural in conditional statement such as if and various loops (which we will study later). For example while in while loop:
declare -i count=0; while (( count < 10 )) ; do printf "%d\n" "$count" let count++ done
To review let arithmetic, the script temperature.bash, shown in Listing 6.2, converts a temperature from Fahrenheit to Celsius.
#!/bin/bash # # temperature.bash: Convert Fahrenheit to Celsius # # Ken O. Burtch # CVS $Header$ shopt -s -o nounset declare -i FTEMP # Fahrenheit temperature declare -i CTEMP # Celsius temperature # Title printf "%s\n" "Fahrenheit-Celsius Conversion" printf "\n" # Get the value to convert read -p "Enter a Fahrenheit temperature: " FTEMP # Do the conversion (( CTEMP=(5*($FTEMP-32) )/9 )) printf "The Celsius temperature is %d\n" "$CTEMP" exit 0
Bash pattern recognition is called globbing. Globbing is used to match filenames given to a command, and it is also used by the Korn shell test command to match strings.
$ ls *.txt notes.txt project_notes.txt
The pattern-recognition feature works by supplying wildcard symbols that Bash will attempt to match to a string or a filename.
The asterisk (*) character represents zero or more characters. The Korn shell test can be used to match the value of a variable to a string with an asterisk.
# Future of the companies predictor
COMPANY="Abracadabra" if [[ $COMPANY = A* ]] ; then printf "The company name begins with a letter A. This company will probably prosper... \n" fi if [[ $COMPANY = Z* ]] ; then printf "The company name begins with a letter Z. this company probably will go down...\n" fi
This behavior doesn't work when quotation marks are used. Quotation marks tell the Korn shell test command not to interpret special characters. A test for "A*" would indicate a file named "A*".
COMPANY="Behemot" if [[ $COMPANY = B* ]] ; then printf "The company name is starting with letter B\n" fi
The question mark (?) character is a wildcard representing any single character.
COMPANY="Behemot" if [[ $COMPANY = ???? ]] ; then printf "The company name is 4 characters long\n" fi
You can specify a set of characters using square brackets. You can list individual characters or ranges.
if [[ $COMPANY = [ABC]* ]] ; then printf "The company name begins with a A, B or C\n" fi if [[ $COMPANY = [A-Z]* ]] ; then printf "The company name begins with a letter an uppercase letter\n" fi if [[ $COMPANY = [A-Z0-9]* ]] ; then printf "The company name begins with a letter an uppercase "\ "letter or number\n" fi
Partial ranges are allowed. If the start of the range is omitted, the range is all ASCII values up to that character. Likewise, the end of the range can be omitted to specify any ASCII characters after the range start character.
If the first character in a square bracket set is a exclamation point (!) or caret (^), characters not in the set count as a match.
Bash defines short forms for common ranges called character classes:
For example, [:lower:] is equivalent to the range [a-z].
COMPANY="2nd Rate Solutions" if [[ $COMPANY = [[:digit:]]*]] ; then printf "Company name starts with a digit\n" fi
There are a number of shell options that affect pattern matching. Filename pattern matching can be turned off completely by setting the noglob shell option (shopt -s -o noglob). If the nullglob option is set (with shopt -s nullglob), the pattern is discarded if not matched. In the previous example, ls would receive no argument and would list all the files in the current directory. You can disable case sensitivity by setting the nocaseglob option (shopt -s nocaseglob). Likewise, you can turn off the special treatment of leading periods in filenames by setting the dotglob option (shopt -s dotglob).
These shell options have a wide-ranging effect; use them with care. Change the setting for as few lines of a script as possible and clearly comment the effect of the change.
The GLOBIGNORE variable also affects how filenames are matched. This variable resembles the PATH variable; it's a colon-separated list of filenames that Bash will not attempt to match. If you are aware of files with pattern symbols in their names, adding them to the GLOBIGNORE list will ensure that Bash will treat them as files and not as globbing expressions.
If you need more sophisticated pattern recognition, or if you need to apply a pattern to an entire file, you can use the grep family of commands.
One filename can be expanded into multiple filenames using curly braces. Inside the curly braces, use commas to list each substring in order to build a new argument. Brace expansion is typically used to specify a root filename with different endings.
printf "%s %s %s\n" "Files should be named:" orders{.txt,.out}
Files should be named: orders.txt orders.out
Because Bash expands the first line to this:
printf "%s %s %s\n" "Files should be named:" orders.txt orders.out
three %s codes are necessary, one for each parameter to printf.
The braces can contain file-matching characters. The actual file matching occurs after the braces are processed by Bash.
If a dollar sign is followed by a string in single quotes, the string is searched for ANSI C escape sequences and the sequences are replaced by the corresponding characters. The acceptable escape sequences are similar to the special format escape codes used by the printf command:
If the contents of curly braces start with an exclamation point (!) and end with an asterisk (*), a list of all variables starting with the specified letters is returned.
$ COMPANY="Nightlight Inc."
printf "%s\n" "${!COMP*}"
COMPANY
A dollar sign with curly braces and a leading number sign (#) returns the length of the variable's value.
printf "%s\n" "${#COMPANY}"
15
An asterisk(*) or at sign (@) returns the number of parameters to the shell script.
printf "%s\n" "${#*}"
0
This is identical to the value of $* built in variable
If a trailing colon question mark appears after the variable name, the message following the question mark is returned and the Bash script exits. This provides a crude form of a sanity check. The message is optional.
printf "Company is %s\n" ${COMPANY:? Error: Not defined.}
If a trailing colon followed by a number appears after the variable name, a substring is returned. The number indicates the first position of the substring, minus one. The first character in the string is character 0. Optionally, a colon and a second number can follow, which indicates the length of the substring.
printf "%s\n" "${COMPANY:5}"
light Inc.
printf "%s\n" "${COMPANY:5:5}"
light
If a trailing number sign appears after the variable name, the substring returned has the matching pattern removed. One number sign matches the smallest possible substring and two number signs matches the largest possible substring. The expression returns the characters to the right of the pattern.
printf "%s\n" "${COMPANY#Ni*}"
ghtlight Inc.
printf "%s\n" "${COMPANY##Ni*}"
printf "%s\n" "${COMPANY##*t}"
Inc.
printf "%s\n" "${COMPANY#*t}"
light Inc.
Using percent signs (%), the expression returns the characters to the left of the pattern.
printf "%s\n" "${COMPANY%t*}"
Nightligh
printf "%s\n" "${COMPANY%%t*}"
Nigh
Here is one useful idiom: obtaining script name from the built-in variable $0
declare -r SCRIPT_NAME=${0##*/} # the name of this script
If the variable is followed by a slash (/), the first occurrence of the pattern following the slash is replaced. Following the pattern, there is a second slash and the replacement string. If the variable is followed by two slashes, all occurrences of the pattern are replaced.
printf "%s\n" "${COMPANY/Inc./Incorporated}"
Nightlight Incorporated
printf "You are the I in %s" "${COMPANY//i/I}"
You are the I in NIghtlIght Inc.
If the pattern begins with a number sign (#), the pattern matches at the beginning of the variable's value. If the pattern ends with a percent sign (%), the pattern matches at the end of the variable's value. Other occurrences are ignored.
$ COMPANY="NightLight Night Lighting Inc."
printf "%s\n" "$COMPANY"
NightLight Night Lighting Inc.
printf "%s" "${COMPANY//Night/NIGHT}"
NIGHTLight NIGHT Lighting Inc.
printf "%s" "${COMPANY//#Night/NIGHT}"
NIGHTLight Night Lighting Inc.
If no new value is indicated, the matching substring is deleted.
$ COMPANY="Nightlight Inc."
printf "%s\n" "${COMPANY/light}"
Night Inc.
Ranges can also be used. For example, to delete all the punctuation in a string, use the range [:punct:]:
printf "%s" "${COMPANY//[[:punct:]]}"
Nightlight Inc
Using an asterisk or at sign instead of a variable applies the substitutions to all the parameters to the shell script. Likewise, an array with an asterisk or at sign applies the substitutions to all elements in the array.
When parentheses are used adder the dollar sign , the command inside them is executed. This has the same effect as enclosing the commands in backquotes.
printf "There are %d files in this directory\n" "$(ls -1 | wc -l)" There are 28 files in this directory printf "There are %d files in this directory\n" `ls -1 | wc -l` There are 28 files in this directory
|
|
Switchboard | ||||
| Latest | |||||
| Past week | |||||
| Past month | |||||
Google matched content |
...
Society
Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers : Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy
Quotes
War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda : SE quotes : Language Design and Programming Quotes : Random IT-related quotes : Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce : Bernard Shaw : Mark Twain Quotes
Bulletin:
Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 : Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law
History:
Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds : Larry Wall : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS : Programming Languages History : PL/1 : Simula 67 : C : History of GCC development : Scripting Languages : Perl history : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history
Classic books:
The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month : How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite
Most popular humor pages:
Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor
The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D
Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.
FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...
|
|
You can use PayPal to to buy a cup of coffee for authors of this site |
Disclaimer:
The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.
Last modified: March, 12, 2019
