Brace Expansion

Everyone has done one of the following to make a quick backup of a file test:

cp test test.old 
cp test.old test 

But retyping or even copying name if it is long is a inconvenient and time consuming. As Unix came from world of teletypes with their slow speed people invented a better way -- factoring common part of the filename for such cases:

 cp test{, ".old"}
 cp test{".old",}  

These two commands are doing exactly the same thing as the first two, but with less typing.   The curly brace ({) in this context means  "brace expansion". The preamble (in our case test,) is prepended to each of the strings in the comma-separated list found within the curly braces, creating a new word for each string. Brace expansion can take place anywhere in your command string, can occur multiple times in a line and can be nested. Brace expansion expressions are evaluated left to right. Some examples:

touch a{1,2,3}_sample
ls /usr/{,local/}
mkdir -p /db/vendors/{dell,hp,cisco}

Brace expansion can be nested.

 touch a{1,2,3{,orig}}

The shell will expand it to:

 a1 a2 a3 a3orig

echo {a..f}{1..9}.txt

"A sequence expression takes the form {x..y}, where x and y are either integers or single characters. When integers are supplied, the expression expands to each number between x and y, inclusive. When characters are supplied, the expression expands to each character lexicographically between x and y, inclusive. Note that both x and y must be of the same type".

NEWS CONTENTS

• Bash Brace Expansion By Mitch Frazier ( May 30, 2008 | Linux Journal )
• Bash Brace Expansion Tutorial 6 Examples of Expanding Expressions within Braces
• Shell scripting and brace expansion

Old News ;-)

Bash Brace Expansion By Mitch Frazier

May 30, 2008 | Linux Journal

Bash brace expansion is used to generate stings at the command line or in a shell script. The syntax for brace expansion consists of either a sequence specification or a comma separated list of items inside curly braces "{}". A sequence consists of a starting and ending item separated by two periods "..".

Some examples and what they expand to:

  {aa,bb,cc,dd}  => aa bb cc dd
  {0..12}        => 0 1 2 3 4 5 6 7 8 9 10 11 12
  {3..-2}        => 3 2 1 0 -1 -2
  {a..g}         => a b c d e f g
  {g..a}         => g f e d c b a

If the brace expansion has a prefix or suffix string then those strings are included in the expansion:

  a{0..3}b       => a0b a1b a2b a3b

Brace expansions can be nested:

  {a,b{1..3},c}  => a b1 b2 b3 c

Counted loops in bash can be implemented a number of ways without brace expansion:

# Three expression for loop:
for (( i = 0; i < 20; i++ ))
do
    echo $i
done
# While loop:
i=0
while [[ $i -lt 20 ]]
do
    echo $i
    let i++
done
# For loop using seq:
for i in $(seq 0 19)
do
    echo $i
done

A counted for loop using bash sequences requires the least amount of typing:

for i in {0..19}
do
    echo $i
done

But beyond counted for loops, brace expansion is the only way to create a loop with non-numeric "indexes":

for i in {a..z}
do
    echo $i
done

Brace expansion can also be useful when passing multiple long pathnames to a command. Instead of typing:

  # rm /a/long/path/foo /a/long/path/bar

You can simply type:

  # rm /a/long/path/{foo,bar}

Brace expansion is enabled via the "set -B" command and the "-B" command line option to the shell and disabled via "set +B" and "+B" on the command line.

Bash Brace Expansion Tutorial 6 Examples of Expanding Expressions within Braces

If you see the output of the following two for statement, you could identify the above pitfall.

$ cat var_seq.sh
# Print 1 to 4 using sequences.
for i in {1..4}
do
        echo $i
done
start=1
end=4

# Print 1 to 4 using through variables
echo "Sequences expressed using variables"
for i in {$start..$end}
do
        echo $i
done

$ ./var_seq.sh
1
2
3
4
Sequences expressed using variables
{1..4}

Brace expansion only happens once, right after the command line is tokenized. So this works:

$ echo foo{1,2,3}
foo1 foo2 foo3

That’s great, but this does not work:

$ myline=foo{1,2,3}
$ echo $myline
foo{1,2,3}

This does not work since the brace expansion (foo{1,2,3} -> foo1 foo2 foo3) happens prior to the shell parameter expansion ($myline -> foo{1,2,3}). To examplify the order, try the reverse experiment. It should work out the same way:

$ one=1
$ two=2
$ echo values_{$one,$two}
values_1 values_2

So it makes sense, but if we still want to force our $myline variable value through brace expansion, we’ll need to have bash evaluate it twice. for this, we’ll need the bash builtin ‘eval’ command. eval evaluates a command line for you, and using it with echo and backtics you can get it to double evaluate our variable, like this:

$ myline=foo{1,2,3}
$ evaluatedline=`eval echo $myline`
$ echo $evaluatedline
foo1 foo2 foo3

And there you go. Now, using a similar mechanism, we can iterate through our file based list of brace expressions, expand each one and copy the files I want to a new location and sequenced name. The full script to do this is here:

#!/bin/bash
OUTPUT_DIR=./quicktime/qt_input_files
OUTPUT_FILE_PREFIX=p
OUTPUT_FILE_SUFFIX=.jpg
FILE_OF_PATTERNS=file-list.txt

# make the output directory if it doesn't exist
if [ ! -e $OUTPUT_DIR ]
then
 mkdir -p $OUTPUT_DIR
fi

# clear out previous run's output
# use this construct since passing such a huge amount (12,000) of
#   files to rm (via rm *.jpg)
#    will fail otherwise (due to 'too many args' error)
ls -tr $OUTPUT_DIR | xargs -t -I{} rm -f ${OUTPUT_DIR}/{}

COUNT=1
# $FILE_OF_PATTERNS is the path to a file that contains path specifiers,
#   one per line, like this:
# 2004-10-26/cam_2004-10-26_{08,09_00}*.jpg
# each line specifies some files we want to include in the animation
# do this for each pattern spec I want to grab
for i in `egrep -o '^[^#]*'  $FILE_OF_PATTERNS`
do
 echo =========== PROCESSING: $i ================
 # force the brace expression through another evaluation, like so:
 foo=`eval echo $i`
 # for every file that matches this sub pattern, copy that image
 #   to a file named with a more simple sequence (p1.jpg, p2.jpg, ...)
 for file in `ls $foo 2> /dev/null`
 do
   cp -p ${file} ${OUTPUT_DIR}/${OUTPUT_FILE_PREFIX}${COUNT}${OUTPUT_FILE_SUFFIX}
   let COUNT=$COUNT+1
 done
done

Shell scripting and brace expansion

Now try it in a shell script:

#!/bin/bash
HOSTS="$1"
for i in $HOSTS
do
ping $i
# rest of logic
done

And then executed script by typing command:

$ ./myscript host{1..5}.my.com 

It will not expand to host1.my.com, host2.my.com..... :/? It took me more than two hours, finally while chatting with my friend he told me to replace HOSTS="$1" with HOSTS="$@". Bingo it worked!

According to bash man page,"A sequence expression takes the form {x..y}, where x and y are either integers or single characters. When integers are supplied, the expression expands to each number between x and y, inclusive. When characters are supplied, the expression expands to each character lexicographically between x and y, inclusive. Note that both x and y must be of the same type". $@ is a special shell variable which. expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. I must admit I need to master shell shell scripting skills ;)

Recommended Links

Softpanorama Recommended

• Shell expansion
• Bash Brace Expansion Tutorial- 6 Examples of Expanding …
• Bash Reference Manual - The GNU Operating System

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: December 06, 2015