Bash: my cookbook

TOC

• split a string into charaters
• read a given line
• sort by tab and numeric
• table merge
• array index
• loop
• cheat keys

# split a string into chraters

cat [file] | grep -o  .

cat [file] | fold -w1    # seems not working with unicode


# Read a given line

sed -n '90,101p' [file]



# Sort by tab and numeric

sort -t$'\t' --key 8,8 --key 3,3n --key 12,12 --key 9.9 --key 1,1 

# Table merge

paste -d , date1.csv date2.csv 

# Array Index

${arr[*]}         # All of the items in the array
${!arr[*]}        # All of the indexes in the array
${#arr[*]}        # Number of items in the array
${#arr[0]}        # Length of item zero

http://www.linuxjournal.com/content/bash-arrays


# Loop

http://www.cyberciti.biz/faq/bash-for-loop/

Examples

This type of for loop is characterized by counting. The range is specified by a beginning (#1) and ending number (#5). The for loop executes a sequence of commands for each member in a list of items. A representative example in BASH is as follows to display welcome message 5 times with for loop:

#!/bin/bash
for i in 1 2 3 4 5
do
   echo "Welcome $i times"
done

Sometimes you may need to set a step value (allowing one to count by two's or to count backwards for instance). Latest bash version 3.0+ has inbuilt support for setting up ranges:

#!/bin/bash
for i in {1..5}
do
   echo "Welcome $i times"
done

Bash v4.0+ has inbuilt support for setting up a step value using {START..END..INCREMENT} syntax:

#!/bin/bash
echo "Bash version ${BASH_VERSION}..."
for i in {0..10..2}
  do
     echo "Welcome $i times"
 done

Sample outputs:

Bash version 4.0.33(0)-release...
Welcome 0 times
Welcome 2 times
Welcome 4 times
Welcome 6 times
Welcome 8 times
Welcome 10 times

The seq command (outdated)

WARNING! The seq command print a sequence of numbers and it is here due to historical reasons. The following examples is only recommend for older bash version. All users (bash v3.x+) are recommended to use the above syntax.

The seq command can be used as follows. A representative example in seq is as follows:

#!/bin/bash
for i in $(seq 1 2 20)
do
   echo "Welcome $i times"
done

There is no good reason to use an external command such as seq to count and increment numbers in the for loop, hence it is recommend that you avoid using seq. The builtin command are fast.

Three-expression bash for loops syntax

This type of for loop share a common heritage with the C programming language. It is characterized by a three-parameter loop control expression; consisting of an initializer (EXP1), a loop-test or condition (EXP2), and a counting expression (EXP3).

for (( EXP1; EXP2; EXP3 ))
do
 command1
 command2
 command3
done

A representative three-expression example in bash as follows:

#!/bin/bash
for (( c=1; c<=5; c++ ))
do
   echo "Welcome $c times"
done
 

Sample output:

Welcome 1 times
Welcome 2 times
Welcome 3 times
Welcome 4 times
Welcome 5 times

How do I use for as infinite loops?

Infinite for loop can be created with empty expressions, such as:

#!/bin/bash
for (( ; ; ))
do
   echo "infinite loops [ hit CTRL+C to stop]"
done

Conditional exit with break

You can do early exit with break statement inside the for loop. You can exit from within a FOR, WHILE or UNTIL loop using break. General break statement inside the for loop:

for I in 1 2 3 4 5
do
  statements1      #Executed for all values of ''I'', up to a disaster-condition if any.
  statements2
  if (disaster-condition)
  then
 break           #Abandon the loop.
  fi
  statements3          #While good and, no disaster-condition.
done

Following shell script will go though all files stored in /etc directory. The for loop will be abandon when /etc/resolv.conf file found.

#!/bin/bash
for file in /etc/*
do
 if [ "${file}" == "/etc/resolv.conf" ]
 then
  countNameservers=$(grep -c nameserver /etc/resolv.conf)
  echo "Total  ${countNameservers} nameservers defined in ${file}"
  break
 fi
done

Early continuation with continue statement

To resume the next iteration of the enclosing FOR, WHILE or UNTIL loop use continue statement.

for I in 1 2 3 4 5
do
  statements1      #Executed for all values of ''I'', up to a disaster-condition if any.
  statements2
  if (condition)
  then
 continue   #Go to next iteration of I in the loop and skip statements3
  fi
  statements3
done

This script make backup of all file names specified on command line. If .bak file exists, it will skip the cp command.

#!/bin/bash
FILES="$@"
for f in $FILES
do
        # if .bak backup file exists, read next file
 if [ -f ${f}.bak ]
 then
  echo "Skiping $f file..."
  continue  # read next file and skip cp command
 fi
        # we are hear means no backup file exists, just use cp command to copy file
 /bin/cp $f $f.bak
done

# Cheat Keys

Source: http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/fto.html

!! - Last command
  !foo - Run most recent command starting with 'foo...' (ex. !ps, !mysqladmin)
  !foo:p - Print command that !foo would run, and add it as the latest to
  command history
  !$ - Last 'word' of last command ('/path/to/file' in the command 'ls -lAFh
  /path/to/file', '-uroot' in 'mysql -uroot')
  !$:p - Print word that !$ would substitute
  !* - All but first word of last command ('-lAFh /path/to/file' in the command
  'ls -lAFh /path/to/file', '-uroot' in 'mysql -uroot')
  !*:p - Print words that !* would substitute

  ^foo^bar - Replace 'foo' in last command with 'bar', print the result, then
  run. ('mysqladmni -uroot', run '^ni^in', results in 'mysqladmin -uroot')

  {a,b,c} passes words to the command, substituting a, b, and c sequentially
  (`cp file{,.bk}` runs `cp file file.bk`)

  Ctrl + a - Jump to the start of the line
  Ctrl + b - Move back a char
  Ctrl + c - Terminate the command
  Ctrl + d - Delete from under the cursor
  Ctrl + e - Jump to the end of the line
  Ctrl + f - Move forward a char
  Ctrl + k - Delete to EOL
  Ctrl + l - Clear the screen
  Ctrl + r - Search the history backwards
  Ctrl + R - Search the history backwards with multi occurrence
  Ctrl + t - Transpose the current char with the previous
  Ctrl + u - Delete backward from cursor
  Ctrl + w - Delete backward a word
  Ctrl + xx - Move between EOL and current cursor position
  Ctrl + x @ - Show possible hostname completions
  Ctrl + z - Suspend/ Stop the command
  Ctrl + x; Ctrl + e - Edit line into your favorite editor

  Alt + < - Move to the first line in the history
  Alt + > - Move to the last line in the history
  Alt + ? - Show current completion list
  Alt + * - Insert all possible completions
  Alt + / - Attempt to complete filename
  Alt + . - Yank last argument to previous command
  Alt + b - Move backward
  Alt + c - Capitalize the word
  Alt + d - Delete word
  Alt + f - Move forward
  Alt + l - Make word lowercase
  Alt + n - Search the history forwards non-incremental
  Alt + p - Search the history backwards non-incremental
  Alt + r - Recall command
  Alt + t - Transpose the current word with the previous
  Alt + u - Make word uppercase
  Alt + back-space - Delete backward from cursor

  (Here "2T" means Press TAB twice)
  $ 2T - All available commands(common)
  $ (string)2T - All available commands starting with (string)
  $ /2T - Entire directory structure including Hidden one
  $ (dir)2T - Only Sub Dirs inside (dir) including Hidden one
  $ *2T - Only Sub Dirs inside without Hidden one 
  $ ~2T - All Present Users on system from "/etc/passwd"
  $ $2T - All Sys variables
  $ @2T - Entries from "/etc/hosts"
  $ =2T - Output like ls or dir
  .bash_profile = sourced by login shell, 
  .bashrc = sourced by all shells, 
  .bash_aliases = should be sourced by .bashrc

Run something:
  for i in a b c; do $i 'hello'; done

Do something on a bunch of files:
  for i in *.rb; do echo "$i"; done

If syntax:
  if [[ -e .ssh ]]; then echo "hi"; fi

Numerical comparison:
  i=0; if (( i <= 1 )); then echo "smaller or equal"; else echo "bigger"; fi

file check flags:
  -e:  file exists
  -f:  regular file (non directory)
  -d:  directory
  -s:  non-zero file
  -x:  execute permission

Avoid duplicates in your history:
  export HISTIGNORE="&:ls:ls *:[bf]g:exit"