In-Line Search And Replace With Perl Regular Expressions.

By | November 20, 2011

Go ahead and skip right to the examples if you’re in a hurry 😉

perl -p -i -e 's/change this/to that/g' file1 file2 file3...

Got Regexes? Regular Expressions (also known as regexes, regexen, and regexps) open up a world of new power tools to you when you need to automate text analysis or manipulations such as massive search and replace operations in one or multiple directories. Such a task oft befalls the sysadmin/developer/commandline ninja.

So case in point: If you find yourself needing to do some file correction in place without the hassle of sed’s inferior pattern matching and all the limitations that go with it, whilst neither feeling the inclination to concoct a convoluted awk block, nor even more a desire to brazenly undertake the time-consuming production of a full blown python script–just to wrap up in a python application the kind of functionality that is just as well performed with a Perl one-liner—— well my friends you’ve come to the right place.

Indeed Perl provides you with in-line search and replace using real regular expressions in all their glory, and in the world of text processing, Perl is king. (Sorry haters/ruby-fanboys/shell-purists, it’s just true!)

So come now, let’s put down the tar, feathers, torches and pitchforks for a moment and consider the following super-cool example (which even creates backup copies of the files you changed, in the event that you flubbed up your regex):

perl -p -i'.backup' -e 's/(?<=replace )this(?= lolc[aA@]t)(?# with )/that/; s/(?# and as for all of the )([[:digit:]]+) (?i:fat|skinny|thirsty) camels(?# you really need those)/$1 white llamas/g;' file1 file2 /path/to/file3

The above powerful search and replace shows you a hefty example of how Perl regexes can be used in your search and replace operation, and the example uses some pretty useful and advanced matching tools you just can’t readily get elsewhere; among other features, the example uses positive look-behind zero-width assertions, positive look-ahead zero-width assertions, atomic sub-match capturing, POSIX character classes, and in-line zero-width comments that make the regexes more readable. In the example, Perl runs two pattern substitutions on every line in each of the three files, and what’s more, it makes backup copies for you!

A more simple version without backups would be:

perl -p -i -e 's/replace this/using that/g' /all/text/files/in/*.txt

The example above replaces any occurrence of the string “replace this” with the string “using that” on all text files inside the directory name given.

So in summary, if you want to use the most powerful search and replace tools on the command line, and do it in the easiest form, use perl -p -i -e 'pattern' file and use it wisely.

…And by “wisely”, I mean it would be wise to brush up on your regular expression fu for free by reading the Perl documentation about its incredible regular expressions.