One-liners with -n and -p

-e, -n and -p

A one-liner is a Perl program written on the command line. The program goes into -e, and two flags wrap it in the loop you would otherwise write by hand.

perl -e 'print "hello\n"'

perl -e 'print "hello\n"'

hello

Name a file after the program and -n supplies the line loop:

printf 'INFO start\nERROR boom\n' > app.log

perl -ne 'print if /ERROR/' app.log

perl -ne 'print if /ERROR/' app.log

ERROR boom

-p prints every line, so the program only has to change it:

printf 'foo foo\n' > notes.txt

perl -pe 's/foo/bar/g' notes.txt

perl -pe 's/foo/bar/g' notes.txt

bar bar

-n wraps the code in while (<>) { ... }, reading every line of every file named on the command line into $_. -p does the same and prints $_ after each iteration, which makes a filter.

Add -l to chomp the input and append a newline on output. That removes most of the chomp calls from your one-liners:

printf 'ada\ngrace\nlinus\n' > names.txt

perl -lne 'print uc' names.txt

perl -lne 'print uc' names.txt

ADA
GRACE
LINUS

$. holds the current line number across all input, and $ARGV names the file being read. With no file arguments the loop reads standard input, so the same command works at the end of a pipe.

Fields with -a and -F

-a splits each line into the array @F, and -F chooses the separator. Together with -n or -p this replaces most of what people use awk for:

printf '10.0.0.1 - - [x] "GET /a HTTP/1.1" 200 5\n10.0.0.2 - - [x] "GET /b HTTP/1.1" 404 0\n' > access.log

perl -lane 'print $F[0]' access.log

perl -lane 'print $F[0]' access.log

10.0.0.1
10.0.0.2

-F takes the separator, so a colon-separated file such as a passwd file splits on colons:

printf 'ada:1000:/bin/sh\ngrace:1001:/bin/bash\n' > users.txt

perl -F: -lane 'print $F[2]' users.txt

perl -F: -lane 'print $F[2]' users.txt

/bin/sh
/bin/bash

To add up a column, clear the accumulator with BEGIN and print it in END. Both blocks run once, outside the loop, which is exactly the awk model:

printf 'north\twidgets\t10420\nsouth\tgadgets\t8000\n' > sales.tsv

perl -lane 'BEGIN { $sum = 0 } $sum += $F[2]; END { print $sum }' sales.tsv

perl -lane 'BEGIN { $sum = 0 } $sum += $F[2]; END { print $sum }' sales.tsv

18420

A nonexistent column is undef and adds as zero, so check the field count when the total looks wrong. -w turns that silence into a warning.

Editing files in place

-i makes -p write the output back to the input file instead of the terminal. Always give an extension, which becomes a backup of the original. Test the same expression without -i first, read the output, and only then edit:

printf '[db]\nhost = localhost\nport = 5432\n' > config.ini

perl -pe 's/localhost/127.0.0.1/g' config.ini

[db]
host = 127.0.0.1
port = 5432

Once the output looks right, add -i and the extension. Without the extension the original is gone, and a broken pattern leaves you with an empty or half-edited file. The command itself prints nothing, so read the file back to see the result:

perl -pi.bak -e 's/localhost/127.0.0.1/g' config.ini

perl -pi.bak -e 's/localhost/127.0.0.1/g' config.ini

cat config.ini

[db]
host = 127.0.0.1
port = 5432

In-place mode rewrites every named file. A wildcard that matches more than you expected rewrites more than you expected, so list the files with ls before you paste the wildcard into the command.

Slurp mode and habits

Setting the input record separator to undef with -0777 reads each file as one string. Multiline substitutions then work without a state machine:

printf 'a\n\n\n\nb\n' > draft.md

perl -0777 -pe 's/\n{3,}/\n\n/g' draft.md

perl -0777 -pe 's/\n{3,}/\n\n/g' draft.md

a

b

Slurping is fine for a few megabytes and dangerous for a log that keeps growing: the whole file lives in memory. For large input keep the line loop and track state in a variable.

Use -M to load a module without writing use. -MData::Dumper plus -e is a fast way to inspect a structure, and -Mstrict -Mwarnings keeps a longer one-liner honest.

perl -MData::Dumper -e 'print Dumper [1, [2, 3]]'

perl -MData::Dumper -e 'print Dumper [1, [2, 3]]'

$VAR1 = [
1,
[
2,
3
]
];

Two rules keep one-liners safe. Quote the whole program with single quotes so the shell leaves $ and @ alone, and read the command once more before pressing Enter, especially when -i or a wildcard is in it.

When a one-liner grows past two lines, save it as a script. A file gives you syntax highlighting, a test, and a place for the comment that explains why the pattern is what it is.

Next: CSV and TSV processing

Article author: Arthur Isaev