Part 1 UNIX Stream Editor: sed

Stream Editor is a utility to parse text line-by-line and edit. Here is how a stream editor works:

1. The first line from the input is read into a buffer (known as pattern space). By default this buffer can hold one line at a time.
2. sed applies the instructions in the pattern space.
3. The changed pattern space is output and the buffer is cleared.
4. The process repeats for all the lines provided as input

Let's understand sed through examples

Substitution 
The following sed instruction reads input and replaces the first occurrence of root on each line with ROOT

[user9@localhost ~]$ sed 's/root/ROOT/'
this is root
this is ROOT
ROOT is root
ROOT is ROOT
beetroot
beetroot
root root root
ROOT root root

If you want to replace all occurences of root with ROOT use /g

[user9@localhost ~]$ sed 's/root/ROOT/g'
this is root
this is ROOT
root is root
ROOT is ROOT
every occurrence of root will be replaced by root
every occurrence of ROOT will be replaced by ROOT

Passing input through a file
You can specify a file as input. Each line in the file is read and the instruction is applied on the line. Remember that the input file is not changed, and the changes are made in the pattern space and sent as output.

[user9@localhost ~]$ sed 's/:/***/4' /etc/passwd
root:x:0:0***root:/root:/bin/bash
bin:x:1:1***bin:/bin:/sbin/nologin

Multiple sed instructions
You can use multiple instructions. In the following example hello is replaced with bye, old with new and mc with metachar

[user9@localhost ~]$ sed 's/hello/bye/
> s/old/new/
> s/mc/metachar/'
how may old will be replaced by mc and how many mc times
how may new will be replaced by metachar and how many mc times

Passing instructions through a file
You can write down all the instructions in a file and pass the instruction file with -f option

[user9@localhost ~]$ cat instructionfile
s/mc/metachar/g
s/re/regular expression/g

Don’t use quotes when the commands are placed in file

[user9@localhost ~]$ sed -f instructionfile
hi ms
hi ms
hi mc re
hi metachar regular expression
hi mc mc re re re mc
hi metachar metachar regular expression regular expression regular expression metachar

Substitution on a line address
You can specify which line the instruction should be applied to. In the following example, mc is replaced with metachar only on the 1st line. Similarly, re is replaced with regular expression only on 4th line.

[user9@localhost ~]$ cat instfile2
1 s/mc/metachar/g
4 s/re/regular expression/g


[user9@localhost ~]$ sed -f instfile2
mc re
metachar re
me
me
mc mc
mc mc
mc re mc
mc regular expression mc
mc re
mc re

You can specify a range of lines to which instruction is applied as follows:

[user9@localhost ~]$ cat instfile3
1,4 s/mc/metachar/g
1,6 s/re/regular expression/g


[user9@localhost ~]$ sed -f instfile3
re mc
regular expression metachar
re mc
regular expression metachar
re mc
regular expression metachar
re mc
regular expression metachar
re mc
regular expression mc
re mc
regular expression mc
re mc
re mc

To apply instruction only on the last line, use $ as the line address.

[user9@localhost ~]$ sed '$s/:/<-->/g' /etc/passwd
appli:x:518:518::/home/appli:/bin/bash
user19:x:519:519::/home/user19:/bin/bash
user20:x:520:520::/home/user20:/bin/bash
user23<-->x<-->521<-->521<--><-->/home/user23<-->/bin/bash

Here is an example illustrating how all the instructions are applied to the pattern space and the output is shown only after all the instructions are applied. In this example if you pass hello, the first instruction changes it to bye and the next instruction changes bye to ok. Finally ok is printed for the input hello.

[user9@localhost ~]$ sed 's/hello/bye/g
> s/bye/ok/g
> '
hello
ok
bye
ok
hello bye
ok ok

Substitution on a context address 
Here is an example of context addressing. If you want to replace hello with bye, only on the line containing root, follow this example:

[user9@localhost ~]$ sed '/root/s/hello/bye/'
hello
hello
hello Mr toot
hello Mr toot
hello Mr root
bye Mr root

You can use line address and context addressing too. In the following example, starting from the first line until (including) the line matching root, the instruction to substitute hello with bye is applied.

[user9@localhost ~]$ sed '1,/root/s/hello/bye/'
hello moot
bye moot
hello noot
bye noot
hello oot
bye oot
hello poot
bye poot
hello root
bye root
hello soot
hello soot

The beginning of a line can be addressed by ^. In the following example, each the beginning of each line is replaced with a space.

[user9@localhost ~]$ sed 's/^/ /' /etc/passwd
 root:x:0:0:root:/root:/bin/bash
 bin:x:1:1:bin:/bin:/sbin/nologin
 daemon:x:2:2:daemon:/sbin:/sbin/nologin
 adm:x:3:4:adm:/var/adm:/sbin/nologin
 lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin