Part 1 UNIX Stream Editor: sed


Stream Editor is a utility to parse text line-by-line and edit. Here is how a stream editor works:


  1. The first line from the input is read into a buffer (known as pattern space). By default this buffer can hold one line at a time.
  2. sed applies the instructions in the pattern space.
  3. The changed pattern space is output and the buffer is cleared.
  4. The process repeats for all the lines provided as input

Let's understand sed through examples

Substitution 
The following sed instruction reads input and replaces the first occurrence of root on each line with ROOT

[user9@localhost ~]$ sed 's/root/ROOT/'
this is root
this is ROOT
ROOT is root
ROOT is ROOT
beetroot
beetroot
root root root
ROOT root root

If you want to replace all occurences of root with ROOT use /g

[user9@localhost ~]$ sed 's/root/ROOT/g'
this is root
this is ROOT
root is root
ROOT is ROOT
every occurrence of root will be replaced by root
every occurrence of ROOT will be replaced by ROOT

Passing input through a file
You can specify a file as input. Each line in the file is read and the instruction is applied on the line. Remember that the input file is not changed, and the changes are made in the pattern space and sent as output.

[user9@localhost ~]$ sed 's/:/***/4' /etc/passwd
root:x:0:0***root:/root:/bin/bash
bin:x:1:1***bin:/bin:/sbin/nologin

Multiple sed instructions
You can use multiple instructions. In the following example hello is replaced with bye, old with new and mc with metachar

[user9@localhost ~]$ sed 's/hello/bye/
> s/old/new/
> s/mc/metachar/'
how may old will be replaced by mc and how many mc times
how may new will be replaced by metachar and how many mc times

Passing instructions through a file
You can write down all the instructions in a file and pass the instruction file with -f option

[user9@localhost ~]$ cat instructionfile
s/mc/metachar/g
s/re/regular expression/g

Don’t use quotes when the commands are placed in file

[user9@localhost ~]$ sed -f instructionfile
hi ms
hi ms
hi mc re
hi metachar regular expression
hi mc mc re re re mc
hi metachar metachar regular expression regular expression regular expression metachar

Substitution on a line address
You can specify which line the instruction should be applied to. In the following example, mc is replaced with metachar only on the 1st line. Similarly, re is replaced with regular expression only on 4th line.

[user9@localhost ~]$ cat instfile2
1 s/mc/metachar/g
4 s/re/regular expression/g


[user9@localhost ~]$ sed -f instfile2
mc re
metachar re
me
me
mc mc
mc mc
mc re mc
mc regular expression mc
mc re
mc re

You can specify a range of lines to which instruction is applied as follows:

[user9@localhost ~]$ cat instfile3
1,4 s/mc/metachar/g
1,6 s/re/regular expression/g


[user9@localhost ~]$ sed -f instfile3
re mc
regular expression metachar
re mc
regular expression metachar
re mc
regular expression metachar
re mc
regular expression metachar
re mc
regular expression mc
re mc
regular expression mc
re mc
re mc

To apply instruction only on the last line, use $ as the line address.

[user9@localhost ~]$ sed '$s/:/<-->/g' /etc/passwd
appli:x:518:518::/home/appli:/bin/bash
user19:x:519:519::/home/user19:/bin/bash
user20:x:520:520::/home/user20:/bin/bash
user23<-->x<-->521<-->521<--><-->/home/user23<-->/bin/bash

Here is an example illustrating how all the instructions are applied to the pattern space and the output is shown only after all the instructions are applied. In this example if you pass hello, the first instruction changes it to bye and the next instruction changes bye to ok. Finally ok is printed for the input hello.

[user9@localhost ~]$ sed 's/hello/bye/g
> s/bye/ok/g
> '
hello
ok
bye
ok
hello bye
ok ok

Substitution on a context address 
Here is an example of context addressing. If you want to replace hello with bye, only on the line containing root, follow this example:

[user9@localhost ~]$ sed '/root/s/hello/bye/'
hello
hello
hello Mr toot
hello Mr toot
hello Mr root
bye Mr root

You can use line address and context addressing too. In the following example, starting from the first line until (including) the line matching root, the instruction to substitute hello with bye is applied.

[user9@localhost ~]$ sed '1,/root/s/hello/bye/'
hello moot
bye moot
hello noot
bye noot
hello oot
bye oot
hello poot
bye poot
hello root
bye root
hello soot
hello soot

The beginning of a line can be addressed by ^. In the following example, each the beginning of each line is replaced with a space.

[user9@localhost ~]$ sed 's/^/ /' /etc/passwd
 root:x:0:0:root:/root:/bin/bash
 bin:x:1:1:bin:/bin:/sbin/nologin
 daemon:x:2:2:daemon:/sbin:/sbin/nologin
 adm:x:3:4:adm:/var/adm:/sbin/nologin
 lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

Comments

Post a Comment

Popular posts from this blog

OS/DB Migration - CMD. STR, TOC, EXT, R3load, DDLDBS.TPL and more

Image
With increasing need to improve performance or reduce costs or vanishing support for operating systems/databases it becomes inevitable to migrate to a better OS or DB. We feel take a quick look at the primary ingredients of the tools for OS/DB migration handled by R3SETUP. Tools - R3load, R3ldctl and R3szchk R3ldctl creates STRucture files, TemPLate files by reading the technical settings for table/indexes in SAP data dictionary. R3szchk calculates the database size object sizes (EXTent files) for the target system based on the STR files created by R3ldctl. The EXT and STR files can be later split into smaller groups. The most crucial tool R3load is responsible for unloading and loading data from the database. R3load uses the files created by R3ldctl and R3szchk and dumps the data into compressed binary format as .001, .002 .. files. It uses cmd files to achieve the task. If the cmd files are not present, it creates on its own. Files Command files*.cmd  are creat...
Read more

Fixing Inconsistent Table - Table activation fails due to inconsistency between DD and DB

If you find a table to be available on the database, but not active in ABAP dictionary, try calling SE14 as DDIC user, enter the table name and choose Table > Reconstruct, Execute. If this method fails follow these steps: Call transaction code SE37 and execute the function module  DD_TABL_ACTM with single test On the next screen, do not change any other field except the following and set them as shown below: MODE: 12 NTAB_PUTSTATE: N TABNAME: <enter the correct table name> Click on Execute The export parameter ACT_RESULT should show 0. To verify if the inconsistency is fixed, call transaction SE14 > Extras > Database Object > Check and  SE14 > Extras > Runtime object > Check.
Read more

301 Redirect Using SAP Web Dispatcher

301 redirects are typically used to redirect accesses that include www to non-www or vice-versa. Example, when you call http://blogger.com, the call is redirected to www.blogger.com. An end-user may prefer to call a URL in any manner. You must ensure that all these calls are redirected to relevant URLs with content. SAP web dispatcher can be configured to send a redirect (301) to the client. To enable redirection add icm/HTTP/redirect_<xx> to the web dispatcher's profile file. The parameter uses the following syntax: icm/HTTP/redirect_<xx> = PREFIX=<URL prefix>[, FROM=<pattern for URL>, FROMPROT=<incoming protocol>, FOR=<pattern for host name:port>,TO=<new URL prefix>, PROT=<protocol>, HOST=<host>, PORT=<port number/name>] Let's say you want to redirect all the HTTP requests on mysource.com on 80 port to mytarget.com on HTTPS 443 port icm/HTTP/redirect_0 = PREFIX=/, FROM=*, FROMPROT=http, FOR=mysource.com:80,...
Read more