Unix Linux Administration II Class 8: Scripting loops. Introduction to sendmail. Reading and printing data.
Unix Linux Administration II
Class 8: Scripting loops. Introduction to sendmail. Reading and printing data.
Agenda discuss Homework. Unit 1: Scripting loops. Unit 2: Introduction to sendmail. Unit 3: Reading and printing data.
Homework review
DNS configs slave and master updates
Configuring views.
Scripting – file management script.
Intermediate certificate, new chained www certificate.
Review: conditionalsExit status, 0 = success, !0 = fail.if test "$user" == “<value>”
you can also just use [][ "$user" == “<value>” ]File tests, such as does the file exist.[ -e /etc/nsswitch.conf ]logical operators-a -o || &&You can use parentheses to alter the order of evaluations.if cmd; then do; else do; fiif [ "$HOME" ]; then echo "Found home!"; else echo "shucks we are homeless!"; fi
Review: PKI
Private keys, Public certificates and CSRpublic CAChain of Trust
Chain certificatesPKI setup
private key, csrsigned cert.
sign other requests (CSR).
Class 8, Unit 1
What we are going to cover: Scripting and loops
What you should leave this session with: Basics to creating loops within your scripts. How to enable debug in your scripts.
Loops.Loops are blocks of code that run until
complete (they can be infinite loops)
The first example is the for loop.
for f in value1 value2 value3
do
cmd
done
For loops - body.for letter in a b c
do
echo “found: $letter”
done.
The “Body” is the content between “do” and “done”.
When the script is executed the value for “letter” is assigned to the first value provided after “in” and then the body of the loop is executed. When complete the second value is assigned to the variable $letter and the process is repeated.
? What happens if you enclose a b c in quotes?
for loops cont.
You can leverage the shells ability for filename substitution in loops. The shell provides for filename substitution in the list provided to the body of the loop.for f in [1-3].txt
do
echo $f
done.
Just as in the other examples, echo is executed 3 times in this example
for loops cont.you can also read in file values and feed those to
the for loop.cat filelist.txt
1.txt
2.txt
3.Txt
for files in $(cat filelist.txt) ; do echo $files; done
or
for files in $(cat filelist.txt) ; do cat $files; done
*example of command substitutions.
Using $* in loops
$* = all arguments
echo “Number of arguments passed in $#“
for variables in $*
do
echo "$variables"
done
Replacing $* with $@You know that $* returns all the values provided at
the command line. However if you use $@ this is actually a comma
separated list of values
for f in “$@”do
echo $fdone
*Best practice to place double quotes around $@
while loopsAnother looping function is "while".
while cmd
do
cmd
done
“cmd” is executed and its exit status is tested. if the exit status is zero the commands between do and done are competed otherwise the script exits with a non zero status code
while script
Similar to saying “while true do”sample “while” script counting to 10
num=1
while [ "$num" -le 10 ]
do
echo $num
num=$(( num+1 ))
done
until
until - the inverse of while, meaning it will run so long as the return code is not 0, or not successful.
Similar to the while blocks, commands between the do and done functions may never be executed if the initial command returns a successful response (zero).
Useful when checking for a status change
until cont.# if NOT successful enter the body
until ps -ef | grep -i "named“ | grep –v grep > /dev/null
do
echo "bind is not running"
sleep 5
done
echo "bind is running“
Break out!Sometimes in a logic loop you want to break
out based on user input such as the user asking to quit. Enter “break”
while truedoread cmdif [ "$cmd" = "quit" ] then break else echo "$cmd"fidone
Continue on…The opposite of break is to continue. Sometimes you
want the loop to simply leave the current loop and continue working through the script. This is where you might use continuefor filedo
if [ ! –e “$file” ]then echo “file not found”continuefi
process rest of file/datadone
Sending the process to background
You can background a process using the & after the done statement. Just as we have done at the command line.
for file in data[1-4]
do
run $file
done &
redirection
I/O redirection on a loop can be obtained using the < or > based on your need.
Write to file:
for i in 1 2 3 4
do
echo $i
done > data.out
Sleep and background
sleep n - where n is a numeric value. Sleep will pause the system for the time specified on the command line.
You can run programs in the background using ampersand "&"
script &
output from this command will tell you the process associated with your process.
Use fg to foreground a background process.
optionsYou can define options in your scripts using syntax
similar to this:if [ "$1" = "-a" ]then option=TRUE
shiftelse option=FALSEfiecho "value for option is: $option"
getoptsThe previous example is fine for simple
options but if you want more flexibility it can become tedious to script. However getopts is available for this purpose.
getopts works within a loop and examines each argument to determine if it is an option based on the existence or absence – before the value.
getopts
The syntax of the getopts command is: getopts optstring option
opstring – is the list of options expected from the command line. option - value used to iterate over the command line options provided.
getopts cont.You can stack your options or pass them
individually. Meaning –abc or –a –b -c
If your option needs an argument add “:”
getopts a:bc name
Now a valid command line looks like:
script.sh –a braeburn –b –c
script.sh –a braeburn
script.sh –b –c
getopts cont.OPTARG used when an option requires an
argument, e.g. –a braeburn
OPTIND is a special variable used by getops which is set to 1 by default and is updated each time getopts complete a loop.
If you reset $OPTIND to 1 at the end of the loop it is possible to use getops again in the same script.
Impact of “:”When an option character not contained in optstring is found, or an option found does not have the required option-argument:
If optstring does NOT begin with a : (colon)
1.Option will be set to a ?
2.OPTARG. will be unset
3.A diagnostic message WILL be written to standard error.
Impact of “:”Alternatively if optstring DOES begin with a : (colon)
1.option will be set to a ? character for an unknown option or to a : (colon) character for a missing required option.
2.OPTARG. will be set to the option character found.
3.no output will be written to standard error.
getopts samplewhile getopts ":ab:c" option; do case $option in a) echo received -a ;; b) echo received -b with $OPTARG ;; c) echo received -c ;; :) echo "option -$OPTARG needs and an ARG" ;; *) echo "invalid option -$OPTARG" ;; esac done
Review: loops and breaksFor loops:for f in a b c; do echo "found: $f"; donefor f in $(cat filelist.txt); do echo $f; donefor f in $(cat filelist.txt); do cat $f; done
$* vs $@, $@ provides a comma separated listUntil and While:while loops, if the exit status is zero the loop is entered.until, if the exit status is NOT zero the loop is entered.Break and continue are used to manipulate the loop behavior.
Review: Options and GETOPTS
Passing options to your script manually.if [ "$1" = "-a" ]then option=TRUE
shift
GETOPTS is a built-in shell function. GETOPTS loops through arguments looking for a “-” before any arguments and determines if it is a valid option.
If arguments are required with the options then you simple add a “:” after the option in your script the GETOPTS will require one.
In class lab 8a
Lab notes for this session can be found here: http://www.ulcert.uw.edu -> Class Content -> InClass labs ->
Class 8, Unit 2
What we are going to cover: Sendmail
What you should leave this session with: DNS mail configuration Basic Sendmail message flow and
configuration.
DNS and mailIn order for mail to routed to your server there
must be a valid MX or mail server record in the DNS domain.
MX records are another type of Resource Record (RR) just as Name Servers are of type NS.
Once we add MX records we should have at least four RR types defined in our domain zone files.
Just as CNAMES and NS RR always need to eventually point to A records, so do MX records.
DNS and mail cont.Mail servers have priority ratings which are
different from other DNS records. The values are somewhat arbitrary but tend to run from 10 to 90
The lower value the higher the priority.
If you have two mail servers one set to 10 and the other to 20 mail will be routed to the lower value unless it is unavailable.
If both had the same value it would be a round robin configuration.
Sample DNS MX configuration
books.ulcert.uw.edu MX 10 mail.books.ulcert.uw.edu
mail.books.ulcert.uw.edu CNAME ns1.books.ulcert.uw.edu
-----------------------------------------------------------------------
Or
----------------------------------------------------------------------- MX 10 mail
mail CNAME ns1
How mail servers work.A client generates a message using one of
many mail clients. This client will either include a built-in SMTP client or it will hand it off to /usr/sbin/sendmail interface. This client then opens a session on port 25 with the SMTP server and begins to send SMTP commands:
HELO, MAIL FROM, RCPT TO, DATA
The message is completed with dot . on a single line. And the message is delivered.
Sendmail historyWritten by Eric Allman who was working and
studying at UC Berkley. The first version was called delivermail and shipped with BSD 4.0 and 4.1.
Sendmail came about as a result of move from NCP (Network Control Protocol) to TCP.
Also namespaces changed from a flat design to a hierarchical namespace (think DNS).
Sendmail first shipped with BSD 4.1c which happened to be the first tcp based version of BSD.
Sendmail successAs Allman has been quoted saying “sendmail
is complex because the world is complex. It is dynamic because the world is dynamic”.
Sendmail strives to accommodate all types of messages. This inclusive goal means rather than denying or rejecting messages that lack the correct header or syntax sendmail tries to compensate for them.
The low cost entry along with a high delivery percentage many consider the primary reason sendmail is so popular today.
Sendmail version info Postfix is the default MTA but sendmail is simple
to install sudo yum install sendmail sendmail-cf
YUM will install sendmail 8.14.x
We can switch between sendmail and postfix using /usr/sbin/alternatives and or enabling services using /sbin/chkconfig
Current stable sendmail version available from sendmail.org is *8.14.8
* now purchased by Proofpoint
Email and Sendmail
There are three primary roles to consider when reviewing mail:MUA – message user agent, examples?MTA – message transfer agent, delivers mail
and transports mail between machines, examples?
MSA – Mail submission agent, capable of altering mail messages such as confirming hostnames are fully qualified, examples?
What are Sendmail, postfix and Exchange?
Basic parts of SendmailThe basic parts to Sendmail The configuration file
/etc/mail/sendmail.cf A queue directory
/var/spool/mqueue Aliases
Sendmail can and will redirect mail destined for one account to another based on defined aliases.
Addresses and Rules Sendmail is based primarily on rules. rules are used to rewrite (modify) mail
addresses, to detect errors in addressing and to select mail delivery agents.
rules are used to detect and reject errors, such as mail with no username
rules examine the address of each envelope recipient and select the appropriate delivery agent.
Rule Sets a sequence of rules are grouped together into rule
sets, each set is similar to a subroutine a rule set is declared with the S command rule sets are numbered or named rule sets such as 0, 3, 4 and 5 are internally
defined by Sendmail 0 resolve mail delivery agent 3 preprocess all addresses 4 post process all address 5 rewrite un-aliased local user
The three parts to a message
All messages have three primary components Header Body Envelope
The Header
Most header lines start with a word followed by a colon.
Received: Date: From: To:Each word indicates the expected value.Not all headers are required.
The BodyThe body of a message consists of
everything following the first blank line
To: user@domain
Subject: Test message, blank line next!
The body start here. Message content here.
Is the subject line required?
The Envelope Because of the diverse recipients, the sendmail
program uses the concept of an envelope. Content that describes the sender or recipient but
is not part of the header is considered envelope information.
Envelope data is used to tell remote machines that mail is ready from a given user. Before sendmail sends the data to a remote MTA it will
send just the envelope-sender address and recipient list to the remote MTA. If ANY of the recipients are accepted the message is sent over otherwise it is not.
Aliases file functions Aliasing is the process of converting one address
into another address. Convert root to mailer-daemon Convert name to list as in mailing list
Sample conversions Bob Barker bbarker geeks allman, schmidt, wall, joy Nobody /dev/null app |/usr/local/bin/myapp
When mail is bounced (returned because it could not be delivered), it is always sent from MAILER-DAEMON. That alias is needed because users might reply to bounced mail without it, replies to bounced mail would themselves bounce.
Queue Management Messages can be temporarily
undeliverable for a variety of reasons. As a result sendmail will queue up messages that are delayed.
These messages are stored in the QueueDirectory which is defined in the sendmail.cf file
Local delivery Sendmail will deliver messages to local
user, meaning a user with a mailbox on the host where sendmail is running.
Local mail is appended to a users mailbox file.The local file is often ~/mbox
Remote delivery Of course sendmail will also deliver mail to
other machines. This happens when sendmail determines the user is not local.
By default Sendmail only supports TCP/IP enabled networks though other options are available (uucp, mfax)
Sendmail modesUsually sendmail runs in Daemon mode –bd, listening for mail but it can be run in:
Test mode –btJust resolve addresses
Verify mode –bvDon’t collect or deliver mail
Mail sender –bmJust send mail
Many others possible, verbose –v…
Sendmail MacrosSendmail macros allow you to reference text
symbolically within the config file. This means you can centrally define values.
Some macros are defined by Sendmail for you such as $u, $h
enter the following to see some of the macros used by sendmail
/usr/lib/sendmail –C/etc/mail/sendmail.cf -bt -d0
*ctrl-+d to exit and no space between –C and /etc…
Sendmail options cont.Sendmail options are defined in sendmail.cf.
Options are declared with an O O QueueDirectory=/var/spool/mqueueOther sample variables are: Timeout
Timeout.queuewarn=4h Timeout.queuereturn=5d
DeliveryMode Background most common
TempFileMode DefaultUser LogLevel
Review:Default MTA in CentOS 6.x is postfix.Installing sendmail provides two MTA options.Mail delivery requires DNS support. MX records are defined
in DNS similar to how we setup NS records.
Three primary roles for mail include: MUA MTA MSA
primary sendmail configuration file /etc/mail/sendmail.cf. This file is not typically edited directly.
Review: Mail is store in the Queue directory before/until
delivered Aliases allow mail to be redirected between
accounts or services as required. sendmail is based on rules and rulesets.
messages are processed by these rulesets before being accepted or denied.
The three primary parts of a message are: header: received, date
body: everything after the first blank line.
envelope: meta data about the message
In class lab 8b
Lab notes for this session can be found here: http://www.ulcert.uw.edu -> Class Content -> InClass labs ->
Class 8, Unit 3
What we are going to cover: Reading and printing data
What you should leave this session with: How to read data in at the cmd line How to format data for printing
Reading in dataTo read in data use
read variable
eg.
read userinput
echo $userinput
Or for multiple variables
read value1 value2 value3
Read cont.If more arguments are entered than variables
the last variable will store the overflow.echo -n "enter names: "
read name names
echo "you entered \$name $name"
echo "then you entered \"$names\" to be stored in \$names “
enter names: TOM SAM JOHN BILL
you entered $name TOM
then you entered "SAM JOHN BILL" to be stored in $names
Read exit code. Read always returns an exit status of zero
unless the end of file condition is detected from input. This usually means Ctrl+d
Knowing this we can use a while loop to read in data at the command line.
while read num1 num2
do
echo $(( $num1 + $num2 ))
done
User input, yes/noUsing the read function and if/then statements we can check for user acceptance.
echo -n "enter yes/no"
read answer
if [ "$answer" = yes ]; then
echo "you agree!"
elif [ "$answer" = no ] ; then
echo "you disagree"
else
echo "I did not understand your answer"
fi
Using $$ for uniqueness
The value for $$ is set to the process id for a given process.
Each process ID on Unix or Linux system is unique for that host. So using this value you can create objects that are very unlikely to conflict with other files on the same system.
grep $USER /etc/passwd >> /tmp/userinfo.$$.tmp
printf: print formatted outputSyntax is
printf “format” arg1 arg2
e.g.
printf “this is a number: %d\n” 10
printf scans the input, sees %d substitutes the first variable with an argument 10
printf conversions.
printf “octal for %d is %o\n” 20 20
Format characters that are NOT preceded by a percent sign are written to stdout.
octal for 20 is 24
Characters that ARE preceded by a percent sign are called “conversion specifications” and will be converted based on the display command.
printf cont.Common printf conversion characters
d integer
c single character
s literal characters
b literal strings with backslash escape char
% percent sign
printf output samples.printf "string contains backslash: %s\n" "test\string"
string contains backslash: test\string
printf "string %s and character %c\n" hello A
string hello and character A
printf “print just the first character: %c\n” QAZW
print just the first character: Q
printf general format
%[flags][width][.precision]typeOnly the % and type are required the others
are modifiers.Flags include- Left justify+ precedes integers with -/+# printf precedes hex integers with 0x or 0X
printf formattingPrintf is typically used to format output. Printf
can align output, set columns and justify content as required.
printf "%+d\n%+d\n%+d\n" 10 -10 20
+10
-10
+20
printf "%-20s%-20s\n" Firstname Lastname
Firstname Lastname
Reviewread variable
echo $variable
read var0 var1 var2
read exit code zero or true unless end of file detected
while true
or
while read input; do ...
read answer
if [ "$answer" = X ] ; then
using process id for file names: file.txt.$$
printf used for formatting output.
printf "%-20s%-20s\n" ColumnA ColumnB
Homework
homework for this week posted later tonight.