SED
This is an article about SED. You can read about AWK and GREP in the article
Text preprocessors
By default, I assume that you are working in
WSL2 or similar shell
. How to install bash in Windows 10 is described in detail
here
Basic Sed Commands
In order to apply SED, just enter the command line
echo ice | sed s/ice/fire/
Result:
fire
Replacing a word in a file
Typically, SED is applied to files, such as logs or configs.
Suppose we have a input.txt file with the following contents
Here is a String
Here is an Integer
Here is a Float
We want to replace the word Here to There
sed 's/Here/There/' input.txt
The result will be displayed in the console:
There is a String
There is an Integer
There is a Float
If you need not to output to the console but change the contents of the file, use the -i option
sed -i 's/Here/There/' input.txt
In this case, the original input.txt file will be rewritten
Let's consider an example more difficult. The input.txt file now looks like this:
Here is an Apple. Here is a Pen. Here is an ApplePen
Integer is Here
Here is a Float
Here is a Pen. Here is a Pineapple. Here is a PineapplePen
sed 's/Here/There/' input.txt
As you will now see, a replacement will occur only once per line
There is an Apple. Here is a Pen. Here is an ApplePen
Integer is There
There is a Float
There is a Pen. Here is a Pineapple. Here is a PineapplePen
To replace all words you need option g
sed 's/Here/There/g' input.txt
There is an Apple. There is a Pen. There is an ApplePen
Integer is There
There is a Float
There is a Pen. There is a Pineapple. There is a PineapplePen
Replacing a word in a file and outputting the result to another file
The same replacement, but with output to a new text file, which we will call output:
sed 's/Here/There/' input.txt > output.txt
Replacing a word in multiple files at once
If you need to process several files at once: for example, 1.txt file with the contents
First File: Here
And the 2.txt file with the content
Second File: Here
This can be done using * .txt
sed 's/Here/There/' *.txt > output.txt
The output file output.txt will look like this
First File: There
Second File: There
Drop everything to the left of a certain word
Suppose we have a input.txt file with the following contents
Here is a String it has a Name
Here is an Integer it has a Name
Here is a Float it has a Name
We want to drop everything to the left of the word it, including the word it, and write it to a file.
sed 's/^.*it//' input.txt > output.txt
^ means we start from the beginning of the line Result:
has a Name
has a Name
has a Name
For accessibility, I’ll explain the syntax by comparing two commands. Look carefully when we replace
the word Here on There.
red.
sed 's/Here/There/'
And when we want to delete something, we first describe what we want to delete. For example, everything from
beginning of line before word it.
Now, on the right-hand side of the condition, where there was previously a replacement value,
do not write anything, i.e. replace it with an empty space. I hope the logic is clear.
sed 's/^.*it//' > output.txt
Discard everything to the right of a particular word
Suppose we have a input.txt file with the following contents
Here is a String / it has a Name
Here is an Integer / it has a Name
Here is a Float / it has a Name
We want to drop everything to the right of the word is, including the word is, and write it to a file.
sed 's/is.*//' > output.txt
Result:
Here
Here
Here
Escaping characters in sed
Special characters are escaped with \
Suppose we have a input.txt file with the following contents
Here is a String / it has a Name
Here is an Integer / it has a Name
Here is a Float it / has a Name
We want to drop everything that is to the left of /a, including /a, and write to the file.
sed 's/^.*/a//' > output.txt
As a result, we get an error
-e expression #1, char 15: unknown option to `s'
For the team to work, you need to add \ before /
sed 's/^.*\/a//' > output.txt
Result:
Here is a String
Here is an Integer
Here is a Float
Two conditions at once in sed
Suppose we have a input.txt file with the following contents
Here is a String /b it has a Name
Here is an Integer /b it has a Name
Here is a Float /b it has a Name
We want to drop everything to the left of /b, including /b, and everything to the right
has .
Thus, only the word it should remain in each line.
We need to take into account the need for escaping a special character / as well as we want
send the output to a file.
sed 's/^.*\/b// ; s/has.*//' input.txt > output.txt
Result:
it
it
it
Delete newline breaks
sed ':a;N;$!ba;s/\n//g' file ;
Delete everything after a certain line
Let's say you want to delete all lines after the third
sed 3q input.txt > output.txt
Delete row
Suppose you want to delete all lines where the word Apple appears in input.txt
Here is an Apple Here Pen Here ApplePen
Integer is Here
Here is a Float
Here Pen Here Pineapple Here PineapplePen
Umm Apple Apple Apple Pen
This can be done using the option d
sed '/Apple/d' input.txt > output.txt
Result:
Integer is Here
Here is a Float
Here Pen Here Pineapple Here PineapplePen
Now let's make a more difficult condition - delete all the lines where there is the word Pineapple and the word Integer
sed '/Pineapple\|Integer/d' input.txt > output.txt
| acts as a logical OR
\ need to screen |
Result:
Here is an Apple. Here is a Pen. Here is an ApplePen
Here is a Float
Umm Apple Apple Apple Pen
Get row range
In the case when you work with large files, for example with logs, it is often necessary
get only certain lines, for example, at the time a bug appears.
Copying from the command line UI is not always convenient, but if you have some idea
range of desired lines - you can copy only them and write to a separate file.
For example, you need lines 9570 to 9721
sed -n '9570,9721p;9722q' project-2019-10-03.log > bugFound.txt
Delete blank lines
If the line is really empty, then the command will do.
sed '/^$/d'
Usually life is more cruel, and the lines contain spaces.
You can also delete such lines.
$ sed '/^[[:space:]]*$/d' input.txt > output.txt
Replace empty lines
Replace all blank lines with double line break tags
$ sed 's/^[[:space:]]*$/\<br\>\<br\>/' lin.txt > output.txt
Replace everything between specific characters
Delete everything between square brackets including brackets
sed 's/\[.*\]//' input.txt > output.txt
- Search on this website
- aofeed - Telegram channel for updates
- aofeedchat - Telegram group for questions
|
Share in social media:
|