UNIX Shell-Scripting Basics. Agenda What is a shell? A shell script? Introduction to bash Running Commands Applied Shell Programming.

UNIX Shell-Scripting Basics

Agenda

What is a shell? A shell script?Introduction to bashRunning CommandsApplied Shell Programming

What is a shell?

%▌

What is a shell?

/bin/bash

What is a shell?

#!/bin/bash

What is a shell?

INPUT

shell

OUTPUT ERROR

What is a shell?

Any ProgramBut there are a few popular shells…

Bourne Shells

/bin/sh/bin/bash

“Bourne-Again Shell”

Steve Bourne

Other Common Shells

C Shell (/bin/csh)Turbo C Shell (/bin/tcsh)Korn Shell (/bin/ksh)

An aside: What do I mean by /bin ?

C Shell (/bin/csh)Turbo C Shell (/bin/tcsh)Korn Shell (/bin/ksh)

An aside: What do I mean by /bin ?

/bin, /usr/bin, /usr/local/bin/sbin, /usr/sbin, /usr/local/sbin/tmp/dev/home/borwicjh

What is a Shell Script?

A Text FileWith InstructionsExecutable

What is a Shell Script?

% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world

What is a Shell Script? A Text File

% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world

An aside: Redirection

cat > /tmp/myfilecat >> /tmp/myfilecat 2> /tmp/myerrcat < /tmp/myinputcat <<INPUTSome inputINPUT

cat > /tmp/x 2>&1

INPUT

env

OUTPUT ERROR

0

1 2

What is a Shell Script? How To Run

% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world

What is a Shell Script? What To Do

% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world

What is a Shell Script? Executable

% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world

What is a Shell Script? Running it

% cat > hello.sh <<MY_PROGRAM#!/bin/shecho ‘Hello, world’MY_PROGRAM% chmod +x hello.sh% ./hello.shHello, world

Finding the program: PATH

% ./hello.shecho vs. /usr/bin/echo% echo $PATH/bin:/usr/bin:/usr/local/bin:/home/borwicjh/bin

% which echo/usr/bin/echo

Variables and the Environment

% hello.shbash: hello.sh: Command not found

% PATH=“$PATH:.”% hello.shHello, world

An aside: Quoting

% echo ‘$USER’$USER% echo “$USER”borwicjh% echo “\”””% echo “deacnet\\sct”deacnet\sct% echo ‘\”’\”

Variables and the Environment

% env[…variables passed to sub-programs…]% NEW_VAR=“Yes”% echo $NEW_VARYes% env[…PATH but not NEW_VAR…]% export NEW_VAR% env[…PATH and NEW_VAR…]

Welcome to Shell Scripting!

How to Learn

manman bashman catman man

man –kman –k manual

Learning the Bash Shell, 2nd Ed.“Bash Reference” Cardshttp://www.tldp.org/LDP/abs/html/

Introduction to bash

Continuing Lines: \

% echo This \Is \ A \Very \Long \ Command LineThis Is A Very Long Command Line%

Exit Status

$?0 is True

% ls /does/not/exist% echo $?1% echo $?0

Exit Status: exit

% cat > test.sh <<_TEST_exit 3_TEST_% chmod +x test.sh% ./test.sh% echo $?3

Logic: test

% test 1 -lt 10% echo $?0% test 1 == 10% echo $?1

Logic: test

test[ ]

[ 1 –lt 10 ] [[ ]]

[[ “this string” =~ “this” ]](( ))

(( 1 < 10 ))

Logic: test

[ -f /etc/passwd ][ ! –f /etc/passwd ][ -f /etc/passwd –a –f /etc/shadow ][ -f /etc/passwd –o –f /etc/shadow ]

An aside: $(( )) for Math

% echo $(( 1 + 2 ))3% echo $(( 2 * 3 ))6% echo $(( 1 / 3 ))0

Logic: if

if somethingthen :# “elif” a contraction of “else if”:elif something-elsethen :elsethen :fi

Logic: if

if [ $USER –eq “borwicjh” ]then :# “elif” a contraction of “else if”:elif ls /etc/oratabthen :elsethen :fi

Logic: if

# see if a file existsif [ -e /etc/passwd ]then echo “/etc/passwd exists”else echo “/etc/passwd not found!”fi

Logic: for

for i in 1 2 3do echo $idone

Logic: for

for i in /*do echo “Listing $i:” ls -l $i readdone

Logic: for

for i in /*do echo “Listing $i:” ls -l $i readdone

Logic: for

for i in /*do echo “Listing $i:” ls -l $i readdone

Logic: C-style for

for (( expr1 ; expr2 ; expr3 ))do listdone

Logic: C-style for

LIMIT=10for (( a=1 ; a<=LIMIT ; a++ ))do echo –n “$a ”done

Logic: while

while somethingdo :

done

Logic: while

a=0; LIMIT=10while [ "$a" -lt "$LIMIT" ]do echo -n "$a ” a=$(( a + 1 ))done

Counters

COUNTER=0while [ -e “$FILE.COUNTER” ]do COUNTER=$(( COUNTER + 1))done

Note: race condition

Reusing Code: “Sourcing”

% cat > /path/to/my/passwords <<_PW_FTP_USER=“sct”_PW_% echo $FTP_USER

% . /path/to/my/passwords% echo $FTP_USERsct%

Variable Manipulation

% FILEPATH=/path/to/my/output.lis% echo $FILEPATH/path/to/my/output.lis% echo ${FILEPATH%.lis}/path/to/my/output% echo ${FILEPATH#*/}path/to/my/output.lis% echo ${FILEPATH##*/}output.lis

It takes a long time to become a bash guru…

Running Programs

Reasons for Running Programs

Check Return Code$?

Get Job OutputOUTPUT=`echo “Hello”`OUTPUT=$(echo “Hello”)

Send Output SomewhereRedirection: <, >Pipes

Pipes

Lots of Little Tools

echo “Hello” | \ wc -c

INPUT

echo

OUTPUT ERROR

0

1 2

INPUT

wc

OUTPUT ERROR

0

1 2

A Pipe!

Email Notification

% echo “Message” | \mail –s “Here’s your message” \ [email protected]

Dates

% DATESTRING=`date +%Y%m%d`% echo $DATESTRING20060125% man date

FTP the Hard Way

ftp –n –u server.wfu.edu <<_FTP_user username passwordput FILE_FTP_

FTP with wget

wget \ftp://user:[email protected]/file

wget –r \ftp://user:[email protected]/dir/

FTP with curl

curl –T upload-file \ -u username:password \ ftp://server.wfu.edu/dir/file

Searching: grep

% grep rayra /etc/passwd% grep –r rayra /etc% grep –r RAYRA /etc% grep –ri RAYRA /etc% grep –rli rayra /etc

Searching: find

% find /home/borwicjh \ -name ‘*.lis’[all files matching *.lis]% find /home/borwicjh \ -mtime -1 –name ‘*.lis’[*.lis, if modified within 24h]% man find

Searching: locate

% locate .lis[files with .lis in path]% locate log[also finds “/var/log/messages”]

Applied Shell Programming

Make Your Life Easier

TAB completionControl+Rhistorycd -Study a UNIX Editor

pushd/popd

% cd /tmp% pushd /var/log/var/log /tmp% cd ..% pwd/var% popd/tmp

Monitoring processes

psps –efps –u oracleps –C sshdman ps

“DOS” Mode Files

#!/usr/bin/bash^MFTP transfer in ASCII, ordos2unix infile > outfile

sqlplus

JOB=“ZZZTEST”PARAMS=“ZZZTEST_PARAMS”PARAMS_USER=“BORWICJH”sqlplus $BANNER_USER/$BANNER_PW << _EOF_set serveroutput onset sqlprompt ""

EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER');

_EOF_

sqlplus

sqlplus $USER/$PASS @$FILE_SQL \ $ARG1 $ARG2 $ARG3if [ $? –ne 0 ]then exit 1fiif [ -e /file/sql/should/create ]then […use SQL-created file…]fi

Ask Amy Lamy!

Passing Arguments

% cat > test.sh <<_TEST_echo “Your name is \$1 \$2”_TEST_% chmod +x test.sh% ./test.sh John Borwick ignore-this

Your name is John Borwick

INB Job Submission Template

$1: user ID$2: password$3: one-up number$4: process name$5: printer name

% /path/to/your/script $UI $PW \ $ONE_UP $JOB $PRNT

Scheduling Jobs

% crontab -l0 0 * * * daily-midnight-job.sh0 * * * * hourly-job.sh* * * * * every-minute.sh0 1 * * 0 1AM-on-sunday.sh% EDITOR=vi crontab –e% man 5 crontab

Other Questions?

Shells and Shell ScriptsbashRunning Commandsbash and Banner in Practice