Scripting

In this section, we will begin writing and executing short scripts in BASH.

The command line is a text interface for your computer. You can communicate with your computer in a language that the IT understands, using this program. You can go beyond the capability of tasks that are possible through the graphical interface. Some examples of things you can do with the command-line include creating directories and folders, navigating the file system, and writing and executing code.

This module explores the basics of the command line and BASH. It functions as a starter kit that will kickstart your journey into scripting!

Learning Objectives

By the end of this module you will:

Be familiar with a core set of BASH commands
Understand the concepts of piping and redirection
Be able to enter a code editor, write and run scripts
Understand the concept of file permissions
Understand paths and the hierarchy of the filesystem
Understand the meaning of some special characters and their usage patterns
Define and use variables
Write scripts using conditional statements
Define functions and use them in scripts
Understand the concept of operators
Gain a high-level understanding of servers
Use BASH commands to retrieve data from a public database and perform some manipulation on the data to extract the required information

Scripting Introduction

Before we do, it’s worth reviewing and expanding on characters with special meaning in BASH.

More On Special Characters

We described some special characters in a section above. There are a few more that will come in handy when writing scripts.

To remove any special qualities a character may have, you can include a back-slash before it. For example \" does not open an interpreted quote pair, it is just a quote character.

echo "This is not special \""

# Indicates what follows is to be ignored or considered as a comment by the interpreter, except when the first line is #! as in the shebang line:

echo "hello" #Don't echo this

Remember, | pipes the output into another program

history | tail

> redirects standard out to a file

 history > myhistory.txt

* Matches all or wildcard

ls -d *.py

" Preserves (from interpretation) most of the special characters from “String” as STRING.

echo "STRING $HOME"

' Preserves all of the special characters as ‘”STRING”‘ as “STRING”

echo 'STRING $HOME'
STRING $HOME

; Ends the line without a ‘return’

echo hello; echo there

$ Used to refer to a variable

var1=5
var2=test

echo $var1 
echo $var2

${variable} Same as calling the variable as $variable, i.e., value of the variable will be printed. In certain contexts, only the less ambiguous ${variable} form works.

echo ${var1}

/ Separator for folder in the path

echo $PATH

Output:

Cursor Control

Here are some ways to take control of the cursor in the command line

Ctrl+a cursor to beginning of command line
Ctrl+e cursor to end of command line
Ctrl-w Cut last word
Ctrl+k cut to the end of the line
Ctrl+y paste content that was cut earlier (by Ctrl-w or Ctrl-k)

Variables

A variable is an object that stores a piece of information. There are two actions we may perform for variables: Setting a value for a variable; Reading the value for a variable. Variables may have their value set in a few different ways. The most common is to set the value directly or for its value to be set as the result of processing by a command or program.

When setting a variable we leave out the $ sign.

I=5

When referring to or reading a variable we place a $ sign before the variable name.

echo $I

Special Variables

There are a few other variables that the system sets for you to use as well.

$0 – The name of the Bash script.
$1 - $9 – The first 9 arguments to the Bash script. (As mentioned above.)
$# – How many arguments were passed to the Bash script.
$@ – All the arguments supplied to the Bash script.
$? – The exit status of the most recently run process.
$$ – The process ID of the current script.
$USER – The username of the user running the script.
$HOSTNAME – The hostname of the machine the script is running on.
$SECONDS – The number of seconds since the script was started.
$RANDOM – Returns a different random number each time is it referred to.
$LINENO – Returns the current line number in the Bash script.
${#var}Return the length of the variable var.

Example Script 1: SpecialVariableTest.sh

Enter the code editor for the script `SpecialVariableTest.sh` and paste the code below. After which, you may run the script.

Description: The script echos some of the common variables that are available in BASH

#!/bin/bash
echo "The name of my script is $0"
echo "My hostname is $HOSTNAME"  # prints name of server you are on.
echo "The script is now on line $LINENO "
echo "The script has run for a total of $SECONDS seconds"

Output:

Command Substitution in BASH

Command substitution allows us to take the output of a command or program (what would normally be printed to the screen) and save it as the value of a variable. To do this we place it within parentheses, preceded by a $ sign – $(ls) would list directories

Example Script 2: processes.sh

Description: The output of the command ps -ef | grep $USER is stored in the variable $myvar. The command returns the processes that are currently running. Our outputs may look a little different from each other.

#!/bin/bash 
myvar=$( ps -ef | grep $USER) 
echo $myvar

Output:

Reading Standard Input

When a program is executed, the result is either sent to the standard output or to the standard error. We know that we can redirect these using the > sign. Also, a program takes in input in the form of standard input, and we know we can pipe results of one program as standard input into another program. The ability to do this is one of the real strengths of Linux. It turns out that we can easily accommodate this mechanism with our scripts also. By doing so we can create scripts that act as filters to modify data in specific ways for us.

Bash accommodates piping and redirection by way of special files. Each process gets its own set of files (one for STDIN, STDOUT, and STDERR respectively) and they are linked when piping or redirection is invoked. Each process gets the following files:

STDIN – /dev/stdin
STDOUT – /dev/stdout
STDERR – /dev/stderr

Example script 3: makeupper.sh

Description: This standard input is piped in to the script and its letters are transformed to all uppercase letters

#!/bin/bash
cat /dev/stdin | tr a-z A-Z

Output:

Mathematical Operations in BASH

let is a builtin function of Bash that allows us to do simple arithmetic on integers (no decimals). Some operations that can be done using let are:

+, -, /*, / addition, subtraction, multiply, divide
var++ Increase the variable var by 1
var-- Decrease the variable var by 1
% Modulus (Return the remainder after division)
expr is similar to let except instead of saving the result to a variable it instead prints the answer. Unlike let you don’t need to enclose the expression in quotes. You also must have spaces between the items of the expression. It is also common to use expr command substitution to save the output to a variable.

Example script 4: math.sh

Description: some common mathematical operations are performed in the script using the let and expr commands

#!/bin/bash
let a=5+4
echo $a # 9
a=$( expr 10 - 3 )
echo $a # 7

Output:

Conditionals

If statements (and, closely related, case statements) are conditional code that operate only when an argument is true. The test statement is found within a pair of square brackets [ ] and they can be TRUE or FALSE. We list common ones below. When we want to perform a certain set of actions if a statement is true, and another set of actions if it is false, we use the else. If we may have a series of conditions that may lead to different paths, we use if the else if. To complete the conditional segment, the closing fi statement is used.

The format would look something like this:

if[ ]
then
    commands
fi

if [  ]
then
    
else
    
fi

Anything between then and fi (if backward) will be executed only if the test (between the square brackets) is true.

Indentation is important in conditional statements. The commands must be indented by 4 spaces.

Example script 5: bigornot.sh

Description: The first argument from the command line is compared to the number 100. If it is larger that 100, an echo statement is printed.

#!/bin/bash
# Basic if statement
if [ $1 -gt 100 ]
then
    echo Hey that\'s a large number.
    pwd
fi
date

Output:

Operators

Operators are important very important. There are many classes of operators such as assignment, boolean, arithmetic etc. In this module, we’ll focus on the string and file test operators, as they are specific to BASH and take a quick peak into the other classes. In later modules (Python and R), we will be using the others in greater detail.

Some core operators are found below. Based on the arguments they operate on or the function they carry out, they have been split into a few categories.

String Operators:

-n STRING: Checks if a string length is greater than 0, if true, returns the size of the string.
-z STRING: Checks if a string length is equal to 0
STRING1 = STRING2: Checks if the two operands are equal
STRING1 != STRING2: Checks if the two operands are not equal

Comparison Operators:

INTEGER1 -eq INTEGER2: Checks if the two integers are equal
INTEGER1 -gt INTEGER2: Checks if integer 1 is greater than integer 2
INTEGER1 -lt INTEGER2: Checks if integer 1 is less than integer 2

File Check Operators:

-d FILE: Checks if a file is a directory
-e FILE: Checks if a file exists
-r FILE: Checks if a file exists and if read permission is granted
-s FILE: Checks if a file exists and its size is greater than 0
-w FILE: Checks if a file exists and if write permission is granted
-x FILE: Checks if a file exists and if execution permission is granted

Conditional Operators:

! EXPRESSION: The EXPRESSION is false.
&&: Used to connect two conditions, checks if both are true
||:Used to connect two conditions, checks if either is true

Example script 6: useful.sh

Description: Takes in a filename as its second argument. It checks the file to see if it exists and if read permissions are granted. If both the conditions are true, it echos a statement that states that the file is useful.

#!/bin/bash
# and example
if [ -r $1 ] && [ -s $1 ]
then
    echo "This file, $2 is useful."
fi

When you run the script below, use the following syntax:

./useful.sh filename

The script useful.sh will check the permissions of your query (the second file).

Output:

Loops

There are two loops, we will work on: for and while. Essentially, you can specify a set of conditions that the script will check for. If the condition(s) are true, the script will execute the do statements.

While loops

While the expression is true, keep executing these lines of code. The syntax looks like this:

while [condition]
do
    
done

Example script 7: count.sh

Description: While the counter variable is less than or equal to 5, its value will be echoed. Once the condition is false, the loop will terminate and echo “All done”

#!/bin/bash
# Basic while loop
counter=1
while [ $counter -le 5 ]
do
    echo $counter
    ((counter++))
done
echo All done

Output:

For Loop

For each of the items in a given list, perform the given set of commands. It has the following syntax.

for var in list
 do

 done

Example script 8: forcount.sh

Description: For a value in the range of 1 to 5, the value will be printed. For values outside of this range, the script will be terminated.

#!/bin/bash
# Basic range in for loop
for value in {1..5}
do
    echo $value
done

Output:

There are some lesser-used loops, such as select, do, and unless. Also, one can break and continue loops.

Functions

Functions in Bash mirror scripts – they provide mini-programs in someways. You define a function and use its functionality throughout your code. They are a core concept in most languages. It gets confusing, so it’s wise to indent the body of the function.

You can define a function by using the following syntax:

Example script 9 : functions

Description: A function is defined to take 2 arguments as input and echo their values. To use the function, later on in the script, its name is called function_name and two arguments are provided.

#!/bin/bash
function_name () { 
    VAR1=$1
    VAR2=$2
    echo $VAR1
    echo $VAR2
}

function_name "hello","goodbye" #prints hello goodbye

Output:

Passing Variables. In most programming languages, we frequently pass arguments to functions and they are assigned to variables listed in brackets. This is not the case in BASH and we have to manually assign them. They are provided to the functional internal scope as $1, $2, $3… . We may send data to the function in a similar way to passing command-line arguments to a script. We supply the arguments directly after the function name.

Returning. Most languages allow us to return a result. BASH doesn’t do this directly but does allow us to provide a status of whether there was a success or not using return.

Example script 10: WhatPassed.sh

Description: A function is defined to take the first argument from the command line as its own argument and echo it to the screen.

#!/bin/bash
# Setting a return status for a function
what_passed () {
    echo "This Passed |$1|"
    return 1
}

what_passed "MY_TEXT_TO_PASS"

This Passed |MY_TEXT_TO_PASS|

Output:

Variable scope and reach

Scope refers to where a variable can be interpreted. BASH is again a little unique, and by default, variables are global within the script. They can be made local to a function. To do that we use the keyword local in front of the variable the first time we set its value.

local var_name=<var_value>

Setting things local is good practice. This way variables are safer from being inadvertently modified by another part of the script which happens to have a variable with the same name (or vice versa).

Example script 11: local.sh

Description: Var1 is defined as a local variable while var2 is not. When they are echoed, the difference in a local and global variable is seen.

#!/bin/bash
# Showing scope
var_change () {
   local var1='Initialized'  #var1 is local to var_change function
   echo Inside the function: var1 is $var1 : var2 is $var2
   var1='LOCAL:KRAS'
   var2='LOCAL:PTEN'
}
var1='GLOBAL:KRAS'
var2='GLOBAL:PTEN'
echo Before function call: var1 is $var1 : var2 is $var2
var_change
echo After function call: var1 is $var1 : var2 is $var2

Output:

Department of Translational Genomics

Scripting In BASH

Scripting

Learning Objectives

Scripting Introduction

More On Special Characters

Output:

Cursor Control

Variables

Special Variables

Example Script 1: SpecialVariableTest.sh

Command Substitution in BASH

Example Script 2: processes.sh

Reading Standard Input

Example script 3: makeupper.sh

Mathematical Operations in BASH

Example script 4: math.sh

Conditionals

Example script 5: bigornot.sh

Operators

Example script 6: useful.sh

Loops

While loops

Example script 7: count.sh

For Loop

Example script 8: forcount.sh

Functions

Example script 9 : functions

Example script 10: WhatPassed.sh

Variable scope and reach

Example script 11: local.sh