unix vr1 02

8/3/2019 unix vr1 02

1/35

The UNIX Operating SystemPipes and Filters

Copyright Zensar Technologies, 2007. All rights reserved.

Objectives

At theend of thissession, you will beable to:

Identify the files: standard input,standard outputandstandard

error

Defineredirection and writecommands which make use of:

Input redirection

Output redirection

Error redirection

Define the term filter

8/3/2019 unix vr1 02

2/35



Objectives (contd.)

Use thegrep filter with the following options:

-n

-c -v

Use thepgandwcfilters

Use thecutandtrfilters

Define the termpipe

8/3/2019 unix vr1 02

3/35



Standard Input, Output and Error Files

Standard input Standard output

Standarderror

Terminal

Utility

8/3/2019 unix vr1 02

4/35



Standard Input, Output and Error Files (contd.)

$cat

Standard input This isa test to depict

Standard output This isa test to depict

$cat temp

Standarderror cat: cannot open temp$

8/3/2019 unix vr1 02

5/35



File Descriptors

Standard input file: 0

Standard output file: 1

Standarderror file: 2

8/3/2019 unix vr1 02

6/35



Input Redirection

Input can be taken from a file other than thestandard input file

(keyboard)

$cat < test1

:

:

< implies input redirection

8/3/2019 unix vr1 02

7/35



Output Redirection

Output can be redirected to a file other than thestandard output file

(VDU)

$cat test1 > test2

:

:

> redirects output to thenamed file

>> appends the output to theexisting file

8/3/2019 unix vr1 02

8/35



Error Redirection

Errorscan be redirected to a file other than thestandard output file

(VDU)

Done using the filedescriptor for thestandarderror file

$cat temp 2> error-mesg

$

8/3/2019 unix vr1 02

9/35



Filters

A filter isa program that:

Takes its input from thestandard input

Processes the input Sends its output to thestandard output

8/3/2019 unix vr1 02

10/35



grep Filter

Stands forglobally search for regular expression and print out

Is used to search a file fora particular pattern ofcharacters

Usage:

grep regular_expression filename

8/3/2019 unix vr1 02

11/35



grep Filter (contd.)

Offers the following options:

-c

displaysacount of lines that match the regularexpression -n

displays linesand their linenumbers that match the regular

expression

-v

displays lines that do not match the regularexpression

8/3/2019 unix vr1 02

12/35




Special symbols used in regularexpressionsare:

[ ] character

to specify a pattern which consists ofany oneset ofcharacters

Example:

grep New[abc]

8/3/2019 unix vr1 02

13/35

8/3/2019 unix vr1 02

14/35




^character

to specify that the pattern following it must occurat the

beginning ofeach line

Example:

grep ^New[abc]

8/3/2019 unix vr1 02

15/35




^ within [ ]

to specify that the pattern must not containany character in

theset/rangespecified within [ ]

Example:

grep New[^a-c]

8/3/2019 unix vr1 02

16/35




$character

to specify that the pattern preceding it must occurat theend

ofeach line

Example:

grep New[abc]$

8/3/2019 unix vr1 02

17/35




. (dot)

to specify any onecharacter

Example:

grep New.[abc]

8/3/2019 unix vr1 02

18/35




\ (backslash)

to specify that grep should ignore thespecial meaning of the

character following it in the regularexpression

Example:

grep New\.\[abc\]

8/3/2019 unix vr1 02

19/35



pgFilter

Is used to display a largedisk file, onescreenful at a time

Waits for the user to press beforedisplaying thenext

screen

Example:

pg data1

8/3/2019 unix vr1 02

20/35


CopyrightZensar Technologies, 2007. All rights reserved.

wcFilter

Used to count thenumber of lines, wordsandcharacters inadisk

file or in thestandard input

Example:wc test

8/3/2019 unix vr1 02

21/35



wcFilter


-l

displaysnumber of lines

-w

displaysnumber of words

-c

displaysnumber ofcharacters

8/3/2019 unix vr1 02

22/35



trFilter

Used to squeeze repeatedcharacters into one

$ who

tony ttyi2 Dec 6 09:06lily ttyj3 Dec 6 10:40

$ who > temp1

$ tr-s < temp1

tony ttyi2 Dec 6 09:06lily ttyj3 Dec 6 10:40

8/3/2019 unix vr1 02

23/35



trFilter (contd.)

Also used incaseconversion

$ tr [a-z] [A-Z]

Thequick brown fox jumps over the lazy dog

THE QUICK BROWN FOX JUMPSOVER THE LAZYDOG

8/3/2019 unix vr1 02

24/35



cutFilter

Used to display selectivecolumns


-f

used to specify columns to be listed

-c

used to display specifiedcharacters

-d

used to specify columnseparator

8/3/2019 unix vr1 02

25/35



cutFilter (contd.)

Example:

$ who > temp1

$ tr-s < temp1 > temp2

$cut -d -f1 < temp2

8/3/2019 unix vr1 02

26/35



Pipes

Used to combine filtersand othercommands/user programs

INPUT cat pg OUTPUT

pipe

Standard output of onecommandcan besent asstandard input toanothercommand

8/3/2019 unix vr1 02

27/35



Pipes (contd.)

Example:

$cat file1 | pg

Instead of:

$cat file1 > tempfile

$ pg tempfile

The vertical bar (|) is the pipecharacter

8/3/2019 unix vr1 02

28/35



Pipes (contd.)

Standard output ofcommand to left of | issent asstandard input to

command to the right of |

command1 | command2 | command3 | command4

Programsdo not have to be rewritten to perform complex tasks

There isno limit to thenumber ofcommands/programs ina

pipeline providedcommands takestandard input and givestandardoutput

Intermediate output ofall commands before the last command is

lost after theentire pipecommand isexecuted

8/3/2019 unix vr1 02

29/35


Copyright

Zensar Technologies, 2007. All rights reserved.

Mindroom Exercise

1. What would happen whena userexecutes the following command

andenters thedetailsasshown below:

$cat >> newfile

This isanew file

d

2. What would happen if in thecommandshown in Q.1, the userdid

not enterany lines but simply pressed ?

3. Rewrite the following commands using filedescriptors:

a. cat > tempfile

b. cat < test1 > temporary

8/3/2019 unix vr1 02

30/35


Copyright


Mindroom Exercise (contd.)

4. Following are thecontents of 2 files, test1 and test2:

This issample test file one (contents of test1)

This issample test file two (contents of test2)

What will thecontents of both files be when the following

commandsareexecuted insuccession?

a. cat test1 > test2

b. cat test1 >> test2

5. Give thecommand to display thecontents of thecurrent directory,

ascreenful at a time.

8/3/2019 unix vr1 02

31/35


Copyright


Solution to Mindroom Exercise

1. A filecallednew_file will becreated with This isanew file as its

contents.

2. A filecallednew_file will becreated which will beempty.

3. Thecommandsare:

cat 1> tempfile 2> errors_file

cat 0< test1 1> tempfile 2> errors_file

4. a. Contents of test 1 and test 2 will be thesame:

This issample test file one

8/3/2019 unix vr1 02

32/35


Copyright


Solution to Mindroom Exercise (contd.)

b. Contents of test1 will be:


Contents of test2 will now be:



(Contents of the file test1 areappended to thecurrent contents of

the file test2)

5. ls | pg

8/3/2019 unix vr1 02

33/35



Summary

Theterminalis thestandardsource forstandard input,standard

output andstandarderrordestination for most UNIXcommands.

The input, output anderrorscan be redirected to a file other than

thestandard file using filedescriptorsand the > or < symbol. Theoutput anderrorcan be redirected in theappend mode to add the

redirected output orerror to anexisting file using the >> symbol.

Output orerror redirectioncausesa file to becreated first and then

the output orerror is written onto orappended to the file.

Filterisacommand or user program that takes input from the

standard input file, processes thedataand gives output on the

standard output file. Examples of filtersare: grep, pg, wc, trand

cut.

8/3/2019 unix vr1 02

34/35



Summary (contd.)

Pipe isa feature by which thestandard output ofacommand or

user program can besent as thestandard input to another

command or user program.

8/3/2019 unix vr1 02

35/35



unix vr1 02

Documents