Wednesday, 28 June 2017

Unix Sed Tutorial: Append, Insert, Replace, and Count File Lines

Sed provides lot of commands to perform number of operations with the lines in a file.
In this article let us review how to append, insert, replace a line in a file and how to get line numbers of a file.
Let us first create ABC.txt file that will be used in all the examples mentioned below.
$cat ABC.txt
Linux Sysadmin
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Windows- Sysadmin, reboot etc.

Append Lines Using Sed Command

Sed provides the command “a” which appends a line after every line with the address or pattern.
Syntax:

#sed 'ADDRESS a\
 Line which you want to append' filename

#sed '/PATTERN/ a\
 Line which you want to append' filename

Sed Append Example 1. Add a line after the 3rd line of the file.

Add the line “Cool gadgets and websites” after the 3rd line. sed “a” command inserts the line after match.
$ sed '3 a\
> Cool gadgets and websites' ABC.txt

Linux Sysadmin
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Cool gadgets and websites
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Windows- Sysadmin, reboot etc.

Sed Append Example 2. Append a line after every line matching the pattern

The below sed command will add the line “Linux Scripting” after every line that matches the pattern “Sysadmin”.
$ sed '/Sysadmin/a \
> Linux Scripting' ABC.txt

Linux Sysadmin
Linux Scripting
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Windows- Sysadmin, reboot etc.
Linux Scripting

Sed Append Example 3. Append a line at the end of the file

The following example, appends the line “Website Design” at the end of the file.
$ sed '$ a\
> Website Design' ABC.txt

Linux Sysadmin
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Windows- Sysadmin, reboot etc.
Website Design

Insert Lines Using Sed Command

Sed command “i” is used to insert a line before every line with the range or pattern.
Syntax:

#sed 'ADDRESS i\
 Line which you want to insert' filename

#sed '/PATTERN/ i\
 Line which you want to insert' filename

Sed Insert Example 1. Add a line before the 4th line of the line.

Add a line “Cool gadgets and websites” before 4th line. “a” command inserts the line after match whereas “i” inserts before match.
$ sed '4 i\
> Cool gadgets and websites' ABC.txt

Linux Sysadmin
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Cool gadgets and websites
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Windows- Sysadmin, reboot etc.

Sed Insert Example 2. Insert a line before every line with the pattern

The below sed command will add a line “Linux Scripting” before every line that matches with the pattern called ‘Sysadmin”.
$ sed '/Sysadmin/i \
> Linux Scripting' ABC.txt

Linux Scripting
Linux Sysadmin
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Linux Scripting
Windows- Sysadmin, reboot etc.

Sed Insert Example 3. Insert a line before the last line of the file.

Append a line “Website Design” before the last line of the file.
$ sed '$ i\
> Website Design' ABC.txt
Linux Sysadmin
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Website Design
Windows- Sysadmin, reboot etc.

Replace Lines Using Sed Command

“c” command in sed is used to replace every line matches with the pattern or ranges with the new given line.
Syntax:

#sed 'ADDRESS c\
 new line' filename

#sed '/PATTERN/ c\
 new line' filename

Sed Replace Example 1. Replace a first line of the file

The below command replaces the first line of the file with the “The Geek Stuff”.
$ sed '1 c\
> The Geek Stuff' ABC.txt

The Geek Stuff
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Windows- Sysadmin, reboot etc.

Sed Replace Example 2. Replace a line which matches the pattern

Replace everyline which has a pattern “Linux Sysadmin” to “Linux Sysadmin – Scripting”.
$ sed '/Linux Sysadmin/c \
> Linux Sysadmin - Scripting' ABC.txt

Linux Sysadmin - Scripting
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Windows- Sysadmin, reboot etc.

Sed Replace Example 3. Replace the last line of the file

Sed command given below replaces the last line of the file with “Last Line of the file”.
$ sed '$ c\
> Last line of the file' ABC.txt

Linux Sysadmin
Databases - Oracle, mySQL etc.
Security (Firewall, Network, Online Security etc)
Storage in Linux
Productivity (Too many technologies to explore, not much time available)
Last line of the file

Print Line Numbers Using Sed Command

“=” is a command in sed to print the current line number to the standard output.
Syntax:

#sed '=' filename
The above send command syntax prints line number in the first line and the original line from the file in the next line .
sed ‘=’ command accepts only one address, so if you want to print line number for a range of lines, you must use the curly braces.
Syntax:

# sed -n '/PATTERN/,/PATTERN/ {
=
p
}' filename

Sed Line Number Example 1. Find the line number which contains the pattern

The below sed command prints the line number for which matches with the pattern “Databases”
$ sed -n '/Databases/=' ABC.txt

2

Sed Line Number Example 2. Printing Range of line numbers

Print the line numbers for the lines matches from the pattern “Oracle” to “Productivity”.
$ sed -n '/Oracle/,/Productivity/{
> =
> p
> }' ABC.txt

2
Databases - Oracle, mySQL etc.
3
Security (Firewall, Network, Online Security etc)
4
Storage in Linux
5
Productivity (Too many technologies to explore, not much time available)

Sed Line Number Example 3. Print the total number of lines in a file

Line number of the last line of the file will be the total lines in a file. Pattern $ specifies the last line of the file.
$ sed -n '$=' ABC.txt

6

Tuesday, 20 June 2017

SORT UNIX COMMAND

Sort Command Examples in Unix / Linux Tutorials

Sort command in unix or linux system is used to order the elements or text. Sort command has the capability of sorting numerical values and strings. The sort command can order the lines in a text file.

The syntax of sort command is:

sort [options] filename

The options are:

-b : Ignores leading spaces in each line
-d : Uses dictionary sort order. Conisders only spaces and alphanumeric characters in sorting
-f : Uses case insensitive sorting.
-M : Sorts based on months. Considers only first 3 letters as month. Eg: JAN, FEB
-n : Uses numeric sorting
-R : Sorts the input file randomly.
-r : Reverse order sorting
-k : Sorts file based on the data in the specified field positions.
-u : Suppresses duplicate lines
-t : input field separator

Sort Command Examples:

Before practicing the examples create the below two files in your unix system:

> cat order.txt
Unix distributed 05 server
Linux virtual 3 server
Unix distributed 05 server
Distributed processing 6 system

> cat delim_sort.txt
Mayday|4
Janmon|1
Declast|12

1. Sorting lines of text

The default sort command uses alphabetical order (ASCII order) to sort the file. It treats each line as a string and then sorts the lines.

> sort order.txt
Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server
Unix distributed 05 server

2. Sorting based on the field positions.

You can specify the field postions using the -k option of sort command. The sort command uses the space or tab as the default delimiter. To sort based on the data in the second field, run the below command:

> sort -k2 order.txt
Unix distributed 05 server
Unix distributed 05 server
Distributed processing 6 system
Linux virtual 3 server

You can also pecify more than field with k option as a comma separated list. The below command uses the second and fourth fields to sort the data.

> sort -k2,4 order.txt

3. Numeric sorting

Instead of the default alphabetical sorting order, you can make the sort command to sort in numeric order using the -n option. This is shown below:

> sort -nk3 order.txt
Linux virtual 3 server
Unix distributed 05 server
Unix distributed 05 server
Distributed processing 6 system

4. Sort in reverse order

By default, the sort command sorts the data in ascending order. You can change this to descending order using the -r option.

> sort -nrk3 order.txt
Distributed processing 6 system
Unix distributed 05 server
Unix distributed 05 server
Linux virtual 3 server

5. Suppressing duplicates or Print only unique values

You can produce only unique values in the output using the - u option of the sort command.

> sort -u order.txt
Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server

Another way is piping the output of sort command to uniq command.

> sort order.txt | uniq

6. Delimited file input

In the second, third and fourth examples we have sorted the data based on the field positions. Here the fields are separted by space or tab character. What if the fields are specifed by any other character? In such cases, we have to specify the input delimiter with the -t option. An example is shown below:

> sort -t'|' -nrk2 delim_sort.txt
Declast|12
Mayday|4
Janmon|1

7. Sorting on months.

We can sort the data in the monthwise using the -M option of the sort command. This is shown below:

> sort -M delim_sort.txt
Janmon|1
Mayday|4
Declast|12

Treats the first 3 characters in the string as month and then sorts in months order.