
Regular Expressions (GREP)
TextSoap now features regular expression support.
This section is not meant as a comprehensive discussion on using regular expressions. Regular Expressions are designed for very advanced users.
Not to worry though. TextSoap cleaners are now shareable, allowing you to use regular expression- based cleaners created by others.
Recommended Reading
For an in depth discussion on Regular Expressions, we recommend:
Mastering Regular Expressions, 2nd Edition
By Jeffery E.F. Friedl.
Publisher: O'Reily Media, Inc.
ISBN: 0596002890
TextSoap's Regular Expression support is based on OgreKit, using the Perl Compatible Regular Expression (PCRE) Syntax.
Note: There can be subtle differences between programs and the syntax that is supported. TextSoap now features regular expression support.
What is a Regular Expression?
A regular expression is another way to describe a find or replace string.
When you choose to find something like "unmarked" you are asking to find a string that begins with a "u" followed by an "n" followed by an "m" followed by an "a" followed by an "r" followed by a "k" followed by an "e" followed by a "d". Wow! Luckily, the way we describe this is to just put the characters next to each other. Then, when all these conditions are met, the search string is considered found.
Regular expressions allow you to use special characters to further describe the search string. For example, we can use a regular expression to say string that begins with a "u" followed by a number of characters, but ends in a "d". The regular expression for this might be "u.*d". The two middle characters act as special characters used to represent "any number of characters".
Understanding Regular Expressions.
Most characters match themselves. The letter "u" will match "u" in text.
Escape Sequences can be used to match special characters that don't naturally match themselves. For example, dot (.) has special meaning to represent any character, but if you want to match an actual dot, you would specify "\." (backslash followed by a dot).
Reference
1. Syntax elements
\ | escape (enable or disable meta character meaning) |
| | alternation |
(...) | group |
[...] | character class |
2. Characters
\t | horizontal tab (0x09) |
\v | vertical tab (0x0B) |
\n | newline (0x0A) |
\r | return (0x0D) |
\b | back space (0x08) |
\f | form feed (0x0C) |
\a | bell (0x07) |
\e | escape (0x1B) |
\nnn | octal char (encoded byte value) |
\xHH | hexadecimal char (encoded byte value) |
\x{7HHHHHHH} | wide hexadecimal char (character code point value) |
\cx | control char (character code point value) |
\C-x | control char (character code point value) |
\M-x | meta (x|0x80) (character code point value) |
\M-\C-x | meta control char (character code point value) |
(* \b is effective in character class [...] only) |
3. Character types
. | any character (except newline) |
\w | word character (alphanumeric, "_" and multibyte char. See also "A-6. Problems") |
\W | non-word char |
\s | whitespace char (\t, \n, \v, \f, \r, \x20) |
\S | non-whitespace char |
\d | digit char |
\D | non-digit char |
4. Quantifier
? | 1 or 0 times |
* | 0 or more times |
+ | 1 or more times |
{n,m} | at least n but not more than m times |
{n,} | at least n times |
{,n} | at least 0 but not more than n times ({0,n}) |
{n} | n times |
5. Anchors
^ | beginning of the line |
$ | end of the line |
\b | word boundary |
\B | not word boundary |
\A | beginning of string |
\Z | end of string, or before newline at the end |
\z | end of string |
\G | matching start position |
6. Character class
^... | negative class (lowest precedence operator) |
x-y | range from x to y |
[...] | set (character class in character class) |
..&&.. | intersection (low precedence at the next of ^) |
ex. [a-w&&[^c-g]z] ==> ([a-w] AND ([^c-g] OR z)) ==> [abh-w]
* If you want to use '[', '-', ']' as a normal character in a character class, you should escape these characters by '\'.
POSIX bracket ([:xxxxx:], negate [:^xxxxx:])
alnum | alphabet or digit char |
alpha | alphabet |
ascii | code value: [0 - 127] |
blank | \t, \x20 |
cntrl | |
digit | 0-9 |
graph | |
lower | |
punct | |
space | \t, \n, \v, \f, \r, \x20 |
upper | |
xdigit | 0-9, a-f, A-F |
7. Extended groups
(?#...) | comment |
(?isx-isx) | option on/off
i: ignore case s: single-line (turn off to have dot(.) match newline) x: extended form |
(?isx-isx:subexp) | option on/off for subexp |
(?:subexp) | not captured group |
(subexp) | captured group |
(?=subexp) | look-ahead |
(?!subexp) | negative look-ahead |
(?<=subexp) | look-behind |
(?<!subexp) | negative look-behind |
Subexp of look-behind must be fixed character length. But different character length is allowed in top level alternatives only.
ex. (?<=a|bc) is OK. (?<=aaa(?:b|cd)) is not allowed.
In negative-look-behind, captured group isn't allowed, but shy group(?:) is allowed.
(?>subexp) atomic group don't backtrack in subexp.
(?<name>subexp) define named group (All characters of the name must be a word character.
And first character must not be a digit or upper case)
Not only a name but a number is assigned like a captured group.
Assigning the same name as two or more subexps is allowed.
In this case, a subexp call can not be performed although the back reference is possible.
8. Back reference
\n | back reference by group number (n >= 1) |
\k | back reference by group name |
In the back reference by the multiplex definition name, a subexp with a large number is referred to preferentially. (When not matched, a group of the small number is referred to.)
* Back reference by group number is forbidden if named group is defined in the pattern.