Lecture 9 sed. sed sed is a stream-oriented editor the input (file/std input) flows through the program sed and is directed the standard output the.

Lecture 9Lecture 9

sedsed

sedsed sed is a stream-oriented editorsed is a stream-oriented editor

the input (file/std input) flows through the the input (file/std input) flows through the program sed and is directed the standard program sed and is directed the standard outputoutput

Used primarily for non interactive operationsUsed primarily for non interactive operations sed [-n] –f script_file file orsed [-n] –f script_file file or

sed [-n] `command` filesed [-n] `command` file sed executes the given command or script file sed executes the given command or script file

that contain commands on each line of the that contain commands on each line of the input (file)input (file)

-n: turn off default printing-n: turn off default printing

sed Commandssed Commands

p: print linep: print line -n prevents lines from being printed twice-n prevents lines from being printed twice

d: delete lined: delete line s/old/new/: substitute old with news/old/new/: substitute old with new s/old/new/g: substitute all occurrences of old s/old/new/g: substitute all occurrences of old

with newwith new !: negates a command!: negates a command Full list of commands can be found on page 129Full list of commands can be found on page 129

sed Examplessed Examples

sed p file.txtsed p file.txt sed –n p file.txtsed –n p file.txt sed d file.txtsed d file.txt sed \!d file.txtsed \!d file.txt p and d seem a bit worthless, don’t they? p and d seem a bit worthless, don’t they?

They purpose will become more clear They purpose will become more clear when we discuss addresses.when we discuss addresses.

sed: Substitutionsed: Substitution

The strongest feature of sedThe strongest feature of sed Syntax is Syntax is s/expression/string/flags/expression/string/flag

expression is a regular expressionexpression is a regular expression string is a stringstring is a string

sed ‘s/|/:/’ data.txtsed ‘s/|/:/’ data.txt substitute the character substitute the character ‘|’‘|’ with the character with the character ‘:’‘:’

sed ‘s/|/:/g’ data.txtsed ‘s/|/:/g’ data.txt

Some Useful Substitution FlagsSome Useful Substitution Flags

g:g: global (replace all matches on the global (replace all matches on the line).line).

p:p: print the line if a successful match print the line if a successful match sed ‘s/old/new/g’ file.txtsed ‘s/old/new/g’ file.txt sed ‘s/old/new/gp’ file.txtsed ‘s/old/new/gp’ file.txt sed –n ‘s/old/new/gp’ file.txtsed –n ‘s/old/new/gp’ file.txt

Regular Expressions for sedRegular Expressions for sed

The usual suspectsThe usual suspects ^, $, ., *, [ ], [^ ], \( \), \<, \>^, $, ., *, [ ], [^ ], \( \), \<, \>

A new operatorA new operator &: the string which matches the expression&: the string which matches the expression

• can be used in the substitution stringcan be used in the substitution string• s/hello/**&**/g replaces all occurrences of hello s/hello/**&**/g replaces all occurrences of hello

with **hello**with **hello**

sed Addressingsed Addressing So far, we have been applying sed commands to So far, we have been applying sed commands to

every lineevery line makes p and d not very usefulmakes p and d not very useful

With addressing, we can apply commands to With addressing, we can apply commands to some, but not all linessome, but not all lines

sed can use sed can use 0 addresses (all lines)0 addresses (all lines) 1 address (a single line)1 address (a single line) 2 addresses (a range of lines)2 addresses (a range of lines)

Address can be line numbers of context (defined Address can be line numbers of context (defined by regular expressions)by regular expressions)

Line Number Addressing ExamplesLine Number Addressing Examples%sed –n ‘3,4p’ foo.txt%sed –n ‘3,4p’ foo.txt

Since sed prints each line anyway, if we only Since sed prints each line anyway, if we only want lines 3 & 4 (instead of all lines with lines want lines 3 & 4 (instead of all lines with lines 3 & 4 duplicated) we use the –n3 & 4 duplicated) we use the –n

%sed –n ‘$p’ foo.txt%sed –n ‘$p’ foo.txt

For each line, if that line is the last line, printFor each line, if that line is the last line, print

%sed –n ‘3,$p’ foo.txt%sed –n ‘3,$p’ foo.txt

For each line, if that line is the third through For each line, if that line is the third through last line, printlast line, print

Context Addressing ExamplesContext Addressing Examples Use patterns/regular expressions rather Use patterns/regular expressions rather

than explicitly specifying line numbersthan explicitly specifying line numbers%sed –n ‘/^From: /p’ $HOME/mbox%sed –n ‘/^From: /p’ $HOME/mbox retrieve all the sender lines from the mailbox retrieve all the sender lines from the mailbox

file, i.e., for each line, if that line starts with file, i.e., for each line, if that line starts with ‘From’, print it. Note that the / / mark the ‘From’, print it. Note that the / / mark the beginning and end of the pattern to matchbeginning and end of the pattern to match

%ls –l | sed –n ‘/^.....w/p’%ls –l | sed –n ‘/^.....w/p’ For each line, if the sixth character is a W, For each line, if the sixth character is a W,

printprint

Context RangesContext Ranges

sed ‘/hello/,/there/d’ file.txtsed ‘/hello/,/there/d’ file.txt delete all lines that occur between a line that delete all lines that occur between a line that

matches hello and a line that matches there. matches hello and a line that matches there. The hello and there lines are also removed.The hello and there lines are also removed.

Multiple contexts are possibleMultiple contexts are possible two contexts specified by a single rangetwo contexts specified by a single range

sed Addressingsed Addressing

Using a ! after the address means all lines Using a ! after the address means all lines which do not match the addresswhich do not match the address sed ‘1\!d’ test.txtsed ‘1\!d’ test.txt

Example fileExample file

northwestnorthwest NWNW Charles Main Charles Main 3.03.0 .98.98 33 3434westernwestern WEWE Sharon GraySharon Gray 5.35.3 .97.97 55 2323southwestsouthwest SWSW Lewis DalsassLewis Dalsass 2.72.7 .8.8 22 1818southernsouthern SOSO Suan ChinSuan Chin 5.15.1 .95.95 44 1515southeastsoutheast SESE Patricia HemePatricia Heme 4.04.0 .7.7 44 1717easterneastern EAEA TB SavageTB Savage 4.44.4 .84.84 55 2020northeastnortheast NENE AM Main Jr.AM Main Jr. 5.15.1 .94.94 33 1313northnorth NONO Margot WebberMargot Webber 4.54.5 .89.89 55 99centralcentral CTCT Ann StephensAnn Stephens 5.75.7 .94.94 55 1313

sed ‘/north/p’ filesed ‘/north/p’ file sed –n ‘s/west/north/g’ filesed –n ‘s/west/north/g’ filesed ‘3,$d’ filesed ‘3,$d’ file sed ‘s/\(Mar\)got/\1ianne/p’ filesed ‘s/\(Mar\)got/\1ianne/p’ filesed ‘s/west/north/g’ filesed ‘s/west/north/g’ file sed ‘/west/,/east/s/$/**VACA**/’ filesed ‘/west/,/east/s/$/**VACA**/’ filesed '/north/a\ sed '/north/a\ hello,word' datafilehello,word' datafile

sed: Using filessed: Using files

Tedious to type in commands at the prompt, Tedious to type in commands at the prompt, especially if commands are repetitiveespecially if commands are repetitive

Can put commands in a file and sed can Can put commands in a file and sed can use themuse them

sed –f cmds.sed data.txtsed –f cmds.sed data.txt

file with commands

sedsed scripts scripts

Series of commands can be put in a file Series of commands can be put in a file and use the ‘-f’ option.and use the ‘-f’ option.

Can also create an sed script:Can also create an sed script:

s/vi/emacs/g

/[Ww]indows/d

p

Another ExampleAnother Example

sed script to remove all HTML tags from a sed script to remove all HTML tags from a file:file:

s/<[^>]*>//g

p

Reading AssignmentReading Assignment

Chapter 5Chapter 5