正则表达式学习指南(七)----Start of String and End of String Anchors

Start of String and End of String Anchors

Thus far, I have explained literal characters and character classes. In both cases, putting one in a regex will cause the regex engine to try to match a single character.

Anchors are a different breed. They do not match any character at all. Instead, they match a position before, after or between characters. They can be used to "anchor" the regex match at a certain position. The caret^ matches the position before the first character in the string. Applying^a to abc matches a. ^b will not match abc at all, because theb cannot be matched right after the start of the string, matched by^. See below for the inside view of the regex engine.

Similarly, $ matches right after the last character in the string.c$ matches c in abc, while a$ does not match at all.

Useful Applications

When using regular expressions in a programming language to validate user input, using anchors is very important. If you use the codeif ($input =~ m/\d+/) in a Perl script to see if the user entered an integer number, it will accept the input even if the user enteredqsdf4ghjk, because \d+ matches the 4. The correct regex to use is^\d+$. Because "start of string" must be matched before the match of\d+, and "end of string" must be matched right after it, the entire string must consist ofdigits for ^\d+$ to be able to match.

It is easy for the user to accidentally type in a space. When Perl reads from a line from a text file, the line break will also be stored in the variable. So before validating input, it is good practice to trim leading and trailingwhitespace. ^\s+ matches leading whitespace and\s+$ matches trailing whitespace. In Perl, you could use $input =~ s/^\s+|\s+$//g. Handy use of alternation and /g allows us to do this in a single line of code.

Using ^ and $ as Start of Line and End of Line Anchors

If you have a string consisting of multiple lines, like first line\nsecond line (where \n indicates a line break), it is often desirable to work with lines, rather than the entire string. Therefore, all the regex engines discussed in this tutorial have the option to expand the meaning of both anchors. ^ can then match at the start of the string (before the f in the above string), as well as after each line break (between\n and s). Likewise, $ will still match at the end of the string (after the last e), and also before every line break (between e and \n).

In text editors like EditPad Pro or GNU Emacs, and regex tools likePowerGREP, the caret and dollar always match at the start and end of each line. This makes sense because those applications are designed to work with entire files, rather than short strings.

In all programming languages and libraries discussed in this help file, exceptRuby, you have to explicitly activate this extended functionality. It is traditionally called "multi-line mode". In Perl, you do this by adding an m after the regex code, like this:m/^regex$/m;. In .NET, the anchors match before and after newlines when you specifyRegexOptions.Multiline, such as in Regex.Match("string", "regex", RegexOptions.Multiline).

Permanent Start of String and End of String Anchors

\A only ever matches at the start of the string. Likewise,\Z only ever matches at the end of the string. These two tokens never match at line breaks. This is true in all regex flavors discussed in this tutorial, even when you turn on "multiline mode". In EditPad Pro and PowerGREP, where the caret and dollar always match at the start and end of lines, \A and\Z only match at the start and the end of the entire file.

JavaScript, POSIX and XML do not support\A and \Z. You're stuck with using the caret and dollar for this purpose.

The GNU extensions to POSIX regular expressions use \` (backtick) to match the start of the string, and \' (single quote) to match the end of the string.

Zero-Length Matches

We saw that the anchors match at a position, rather than matching a character. This means that when a regex only consists of one or more anchors, it can result in a zero-length match. Depending on the situation, this can be very useful or undesirable. Using^\d*$ to test if the user entered a number (notice the use of thestar instead of the plus), would cause the script to accept an empty string as a valid input.See below.

However, matching only a position can be very useful. In email, for example, it is common to prepend a "greater than" symbol and a space to each line of the quoted message. InVB.NET, we can easily do this with Dim Quoted as String = Regex.Replace(Original, "^", "> ", RegexOptions.Multiline). We are using multi-line mode, so the regex^ matches at the start of the quoted message, and after each newline. The Regex.Replace method will remove the regex match from the string, and insert the replacement string (greater than symbol and a space). Since the match does not include any characters, nothing is deleted. However, the match does include a starting position, and the replacement string is inserted there, just like we want it.

Strings Ending with a Line Break

Even though \Z and $ only match at the end of the string (when the option for the caret and dollar to match at embedded line breaks is off), there is one exception. If the string ends with a line break, then\Z and $ will match at the position before that line break, rather than at the very end of the string. This "enhancement" was introduced by Perl, and is copied by many regex flavors, includingJava, .NET and PCRE. In Perl, when reading a line from a file, the resulting string will end with a line break. Reading a line from a file with the text "joe" results in the stringjoe\n. When applied to this string, both ^[a-z]+$ and \A[a-z]+\Z will match joe.

If you only want a match at the absolute very end of the string, use \z (lower case z instead of upper case Z). \A[a-z]+\z does not matchjoe\n. \z matches after the line break, which is not matched by thecharacter class.

Looking Inside the Regex Engine

Let's see what happens when we try to match ^4$ to 749\n486\n4 (where \n represents a newline character) in multi-line mode. As usual, the regex engine starts at the first character:7. The first token in the regular expression is ^. Since this token is a zero-width token, the engine does not try to match it with the character, but rather with the position before the character that the regex engine has reached so far.^ indeed matches the position before 7. The engine then advances to the next regex token:4. Since the previous token was zero-width, the regex engine doesnot advance to the next character in the string. It remains at 7. 4 is a literal character, which does not match 7. There are no other permutations of the regex, so the engine starts again with the first regex token, at the next character:4. This time, ^ cannot match at the position before the 4. This position is preceded by a character, and that character is not a newline. The engine continues at9, and fails again. The next attempt, at \n, also fails. Again, the position before \n is preceded by a character,9, and that character is not a newline.

Then, the regex engine arrives at the second 4 in the string. The^ can match at the position before the 4, because it is preceded by a newline character. Again, the regex engine advances to the next regex token,4, but does not advance the character position in the string.4 matches 4, and the engine advances both the regex token and the string character. Now the engine attempts to match$ at the position before (indeed: before) the 8. The dollar cannot match here, because this position is followed by a character, and that character is not a newline.

Yet again, the engine must try to match the first token again. Previously, it was successfully matched at the second4, so the engine continues at the next character, 8, where the caret does not match. Same at the six and the newline.

Finally, the regex engine tries to match the first token at the third 4 in the string. With success. After that, the engine successfully matches 4 with 4. The current regex token is advanced to $, and the current character is advanced to the very last position in the string: the void after the string. No regex token that needs a character to match can match here. Not even anegated character class. However, we are trying to match a dollar sign, and the mighty dollar is a strange beast. It is zero-width, so it will try to match the position before the current character. It does not matter that this "character" is the void after the string. In fact, the dollar will check the current character. It must be either a newline, or the void after the string, for$ to match the position before the current character. Since that is the case after the example, the dollar matches successfully.

Since $ was the last token in the regex, the engine has found a successful match: the last4 in the string.

Another Inside Look

Earlier I mentioned that ^\d*$ would successfully match an empty string. Let's see why.

There is only one "character" position in an empty string: the void after the string. The first token in the regex is^. It matches the position before the void after the string, because it is preceded by the void before the string. The next token is\d*. As we will see later, one of the star's effects is that it makes the\d, in this case, optional. The engine will try to match \d with the void after the string. That fails, but the star turns the failure of the\d into a zero-width success. The engine will proceed with the next regex token, without advancing the position in the string. So the engine arrives at$, and the void after the string. We already saw that those match. At this point, the entire regex has matched the empty string, and the engine reports success.

Caution for Programmers

A regular expression such as $ all by itself can indeed match after the string. If you would query the engine for the character position, it would return the length of the string if string indices are zero-based, or the length+1 if string indices are one-based in your programming language. If you would query the engine for the length of the match, it would return zero.

What you have to watch out for is that String[Regex.MatchPosition] may cause an access violation or segmentation fault, because MatchPosition can point to the void after the string. This can also happen with^ and ^$ if the last character in the string is a newline.