More info is in this tutorial Bash scripting cheatsheet

all value expansions (ie. all syntax with a $ prefix) can only expand inside quoted arguments if the argument was double-quoted . Single quotes will turn the dollar-syntax into literal characters, causing bash to output the dollar rather than expand its value in-place!

“Value expansions ($...) must always be double-quoted.”

Never leave a value expansion unquoted.

Redirection

File Redirection

0 is standard input, 1 is standard output, 2 is standard error

[x] > file, [x] < file

Make File descriptor(FD) x write to / read from file.

echo hello > ~/World

read line < ~/Word

rm file 2>/dev/null

File Descriptor copying

[x] >& y, [x] <& y

make FD x write to / read from FD y’s stream

the connection to the stream used by FD y is copied to FD x

curl cip.cc > result 2>&1

ping localhost > result 2>&1

# exec can be used to change the file descriptors of bash itself,
# and if you use an x that doesn't yet exist,
# bash will create a new file descriptor ("plug") for you with that number.
# - in command is to close new FD 3 we'd created before.
# >&- is to close FD 1, <&- is to close FD0
exec 3>&1 >mylog; echo moo; exec 1>&3 3>&-

Appending file redirection

[x] >> file

make FD x append to end of the file

A stream to file is opened for writing in append mode and is connected to file descriptor x. The regular file redirection operator > empties the file’s contents when it opens the file so that only your bytes will be in the file.

echo hello >> ~/world

echo world >> ~/world

Redirecting standard output and standard error

&>file

Make both FD 1 (standard output) and FD 2 (standard error) write to file

This is a convenience operator which does the same thing as >file 2>&1 but is more concise. Again, you can append rather than truncate by doubling the arrow: &>>file

ping localhost &>result

Here documents

<<delimiter
	Here document
delimiter

Make FD 0 read from the string between delimiters

Here-Documents are great for reading blocks of text to command line.

cat << EOF
this is within here document
I can write as many lines as I like
and terminate with line of demiliter only
EOF	//end of heredoc

Here strings

<<< string

Make FD 0 read from the string

Here strings are very similar to here documents but more concise. They are generally preferred over here documents.

cat <<< "This,
is the here strings. tab will also be read."

Moving file decipher

[x]>&y-, [x]<&y-

Replace FD x with FD y and close FD y

Easy way of [x]>&y, y>&-

# 3>&1-: copy FD 1 to FD 3 and close FD 1.
#  >&3-: copy FD 3 to FD 1 and close FD 3.
exec 3>&1- >mylog; echo moo; exec >&3-

Reading and writing with file descriptor

[x] <> file

Open FD x for both reading and writing to file

The file descriptor at x is opened with a stream to the file that can be used for writing as well as reading bytes. Usually you’ll use two file descriptors for this. One of the rare cases where this is useful is when setting up a stream with a read/write device such as a network socket.

exec 5<>aFile
cat >&5 "Hello world"	# make FD 1 write to where FD 5 currently writing, copy file descriptor FD 5 to FD 1
cat <&5			# make FD 0 read from where FD 5 currently reading, copy file descriptor FD 5 to FD 0, then cat will send content to FD 1

Exercise

Q: fix exec 3>&2 2>log; echo 'Hello!'; exec 2>&3 so that the message is properly saved into the log file and such that FD 3 is properly closed afterwards:

A:

  1. exec 3>&1- 3>log; echo 'Hello!'; exec 1>&3-
  2. exec 3>&1 1>&- 3>log; echo 'Hello!'; exec 1>&3 3>&-

Expansion

pathname expansion

  1. pattern expansion is performed by bash before command even execute

    file * will show info about all file in current directory. * will expand to content before file execute.

  2. A glob is the name of the type of pattern supported by the bash shell.

    basic glob name supported by bashexplanation

    example

    Those glob will only affect current directory, explicit expression is required to working on other directory. ls /sub/*

  3. extended glob can be enable to get more powerful but also easy confusing feature of bash

    bash: shopt -s extglob zsh. : setopt extendedglob explanation

    !(my)* get expand because of * is outside !(), which makes it expland another whole pathname

Command Substitution

we can expansion commands within commands, but must use double-quote "" instead of ''

# this will output contents in hello.h to screen
cat hello.h

# this will expand `cat hello.h` to real contents in 
# file hello.h and concatenate to previous sentence
echo "file hello.h contains contents of $(cat hello.h)"

# this will output 'file hello.h contains contents of $(cat hello.h)'
# without expand command in $()
echo 'file hello.h contains contents of $(cat hello.h)'

In command, $() is called value expansion, it consists of value-expansion prefix $ and subshell (...). A subshell is essentially a small new bash process that is used to run a command while the main bash shell waits for the result.

Parameters

There are three kind of parameters:

  1. Environment Parameter
  2. Positional Parameter
  3. Variables

Environment Parameter

environment variables exist at the process level. That means they are not a feature of the bash shell, but rather a feature of any program process on your system. They can inherit by children, but children’s EV can’t be given to parent.

Positional Parameter

Just as name indicates, these kind of parameters indicate arguments’ position, and always starting from 0.

for example, imaging we have a script rename, arguments could be passed to it to extend its usage:

rename dir name there, we passed dir and name as argument, so that positional parameters in script would be $1 and $2, representing arguments respectively. after $2, such as $3 is unset since there has no more argument.

Positional Parameter is read-only

a new usage: bash -c 'ls "$1"' -- '/home'. This will working like ls /home, dash is necessary since it is first variable in shell we ran commands and it makes positional value of arguments populated after it stand as we expect in shell single-quoted command gonna run in.

Special Parameter

Special parameters are parameters whose name is a single symbolic character, they are used to request certain state information from the bash shell. Like positional parameter, they are read-only.

different kinds of special parameters and the information they hold

Variables

definition: name=value //no space around = like other programming language support

call: like command expansion, using variable is to expand it with prefix $, e.g. $name

  • Keep in mind, Expansion should always be double-quoted *

Parameter expansion

GNU material

we expand parameters by prefixing their name with a $ symbol

e.g. name=me; echo hello "$name". hello me.

In addition, we can put braces ({ and }) around our parameter, which indicates where variable is about to begin and end.

e.g. name=orange; echo there are 4 "${name}s" there are 4 oranges.

name=orange
echo "there are 4 ${name}s."	# there are 4 oranges.
echo "there are 4 $names."	# there are 4 .

here, we put {}aroundnameso that bash can be told that suffix s is not a part of variable. otherwise, it will treat names as parameter and looking for its value, which is none in our example.

parameter expansion brings up a powerful feature: parameter expansion oerators

While expanding a parameter, it is possible to apply an operator to the expanding value without alternate original value.

I use these mostly

# remove string before pattern   	${name#pattern} shortest	${name##pattern} longest
# remove string after pattern    	${name%pattern} longest 	${name%%pattern} shortest
# delete first matching pattern		${name/pattern}
# delete all matching pattern		${name//pattern}
# substitute pattern with string 	${name//pattern/string}

foo="foo-bar-foobar"
echo ${foo#*-}		# echoes 'bar-foobar' (Removes 'foo-' because that matches '*-')
echo ${foo##*-}		# echoes 'foobar' (Removes 'foo-bar-')
echo ${foo%-*}		# echoes 'foo-bar'
echo ${foo%%-*}		# echoes 'foo'
echo ${foo/-}		# echoes 'foobar-foobar'
echo ${foo//-}		# echoes 'foobarfoobar'
echo ${foo//-/_}	# echoes 'foo_bar_foobar'

part of operators as shown picture