Prompted by blog reader Mark McDonnell, I've extracted the regex syntax highlighting engine built into RegexPal and made it into its own library, unimaginatively named JavaScript Regex Syntax Highlighter. When combined with the provided CSS, this 1.6 KB self-contained JavaScript file can be used, for instance, to automatically apply regex syntax highlighting to any HTML element with the "regex" class. You can see an example of doing just that on my quick and dirty test page.

Although the library is simple (there's just one function to call), the syntax highlighting is pretty advanced and handles all valid JavaScript regex syntax and errors (with errors highlighted in red). An example of its advanced highlighting support is that it knows, based on the context, whether \10 is backreference 10, backreference 1 followed by a literal zero, octal character index 10, or something else altogether due to its position in the surrounding pattern. Speaking of octal escapes (which are de facto browser extensions; not part of the spec.), they are correctly highlighted according to their subtle differences inside and outside character classes (outside of character classes only, octals can include a fourth digit if the leading digit is a zero).

As far as I'm aware, this is the first JavaScript library for highlighting regex syntax, with or without the level of completeness included here. For people who might feel inclined to use or improve upon my work, I've made the licensing as permissive as possible to avoid getting in your way. RegexPal is already open source under the GNU LGPL 3.0 License, but this new library is released under the MIT License. If you plan to customize or help upgrade this code, note that it could probably use a bit of an overhaul (it's ripped from RegexPal with minimal modification), and might require an overhaul if you want to cleanly add support for additional regex flavors. Another nifty feature I plan to eventually add is explanatory title attributes for each element in the returned HTML, which might be particularly helpful for deciphering any highlighted errors or warnings.

Let me know if this library is useful for you, or if there are any other features you'd like to see added or changed. Thanks!

Link: JavaScript Regex Syntax Highlighter.

For a language as widely used as JavaScript, any realistic change proposal must strongly consider backward compatibility. For this reason, some of the following proposals might not be particularly realistic, but nevertheless I think that a) it's worthwhile to consider what might change if backward compatibility wasn't a concern, and b) in the long run, all of these changes would improve the ease of use and predictability of how regular expressions work in JavaScript.

Remove RegExp.prototype.lastIndex and replace it with an argument for start position

Actual proposal: Deprecate RegExp.prototype.lastIndex and add a "pos" argument to the RegExp.prototype.exec/test methods

JavaScript's lastIndex property serves too many purposes at once:

It lets users manually specify where to start a regex search

You could claim this is not lastIndex's intended purpose, but it's nevertheless an important use since there's no alternative feature that allows this. lastIndex is not very good at this task, though. You need to compile your regex with the /g flag to let lastIndex be used this way; and even then, it only specifies the starting position for the regexp.exec/test methods. It cannot be used to set the start position for the string.match/replace/search/split methods.

It indicates the position where the last match ended

Even though you could derive the match end position by adding the match index and length, this use of lastIndex serves as a convenient and commonly used compliment to the index property on match arrays returned by exec. Like always, using lastIndex like this works only for regexes compiled with /g.

It's used to track the position where the next search should start

This comes into play, e.g., when using a regex to iterate over all matches in a string. However, the fact that lastIndex is actually set to the end position of the last match rather than the position where the next search should start (unlike equivalents in other programming languages) causes a problem after zero-length matches, which are easily possible with regexes like /\w*/g or /^/mg. Hence, you're forced to manually increment lastIndex in such cases. I've posted about this issue in more detail before (see: An IE lastIndex Bug with Zero-Length Regex Matches), as has Jan Goyvaerts (Watch Out for Zero-Length Matches).

Unfortunately, lastIndex's versatility results in it not working ideally for any specific use. I think lastIndex is misplaced anyway; if you need to store a search's ending (or next-start) position, it should be a property of the target string and not the regular expression. Here are three reasons this would work better:

• It would let you use the same regex with multiple strings, without losing track of the next search position within each one.
• It would allow using multiple regexes with the same string and having each one pick up from where the last one left off.
• If you search two strings with the same regex, you're probably not expecting the search within the second string to start from an arbitrary position just because a match was found in the first string.

In fact, Perl uses this approach of storing next-search positions with strings to great effect, and adds various features around it.

So that's my case for lastIndex being misplaced, but I go one further in that I don't think lastIndex should be included in JavaScript at all. Perl's tactic works well for Perl (especially when considered as a complete package), but some other languages (including Python) let you provide a search-start position as an argument when calling regex methods, which I think is an approach that is more natural and easier for developers to understand and use. I'd therefore fix lastIndex by getting rid of it completely. Regex methods and regex-using string methods would use internal search position trackers that are not observable by the user, and the exec and test methods would get a second argument (called pos, for position) that specifies where to start their search. It might be convenient to also give the String methods search, match, replace, and split their own pos arguments, but that is not as important and the functionality it would provide is not currently possible via lastIndex anyway.

Following are examples of how some common uses of lastIndex could be rewritten if these changes were made:

Start search from position 5, using lastIndex (the staus quo):

var regexGlobal = /\w+/g,
    result;

regexGlobal.lastIndex = 5;
result = regexGlobal.test(str);
// must reset lastIndex or future tests will continue from the
// match-end position (defensive coding)
regexGlobal.lastIndex = 0;

var regexNonglobal = /\w+/;

regexNonglobal.lastIndex = 5;
// no go - lastIndex will be ignored. instead, you have to do this
result = regexNonglobal.test(str.slice(5));

Start search from position 5, using pos:

var regex = /\w+/, // flag /g doesn't matter
    result = regex.test(str, 5);

Match iteration, using lastIndex:

var regex = /\w*/g,
    matches = [],
    match;

// the /g flag is required for this regex. if your code was provided a non-
// global regex, you'd need to recompile it with /g, and if it already had /g,
// you'd need to reset its lastIndex to 0 before entering the loop

while (match = regex.exec(str)) {
    matches.push(match);
    // avoid an infinite loop on zero-length matches
    if (regex.lastIndex == match.index) {
        regex.lastIndex++;
    }
}

Match iteration, using pos:

var regex = /\w*/, // flag /g doesn't matter
    pos = 0,
    matches = [],
    match;

while (match = regex.exec(str, pos)) {
    matches.push(match);
    pos = match.index + (match[0].length || 1);
}

Of course, you could easily add your own sugar to further simplify match iteration, or JavaScript could add a method dedicated to this purpose similar to Ruby's scan (although JavaScript already sort of has this via the use of replacement functions with string.replace).

To reiterate, I'm describing what I would do if backward compatibility was irrelevant. I don't think it would be a good idea to add a pos argument to the exec and test methods unless the lastIndex property was deprecated or removed, due to the functionality overlap. If a pos argument existed, people would expect pos to be 0 when it's not specified. Having lastIndex around to sometimes screw up this expectation would be confusing and probably lead to latent bugs. Hence, if lastIndex was deprecated in favor of pos, it should be a means toward the end of removing lastIndex altogether.

Remove String.prototype.match's nonglobal operating mode

Actual proposal: Deprecate String.prototype.match and add a new matchAll method

String.prototype.match currently works very differently depending on whether the /g (global) flag has been set on the provided regex:

• For regexes with /g: If no matches are found, null is returned; otherwise an array of simple matches is returned.
• For regexes without /g: The match method operates as an alias of regexp.exec. If a match is not found, null is returned; otherwise you get an array containing the (single) match in key zero, with any backreferences stored in the array's subsequent keys. The array is also assigned special index and input properties.

The match method's nonglobal mode is confusing and unnecessary. The reason it's unnecessary is obvious: If you want the functionality of exec, just use it (no need for an alias). It's confusing because, as described above, the match method's two modes return very different results. The difference is not merely whether you get one match or all matches—you get a completely different kind of result. And since the result is an array in either case, you have to know the status of the regex's global property to know which type of array you're dealing with.

I'd change string.match by making it always return an array containing all matches in the target string. I'd also make it return an empty array, rather than null, when no matches are found (an idea that comes from Dean Edwards's base2 library). If you want the first match only or you need backreferences and extra match details, that's what regexp.exec is for.

Unfortunately, if you want to consider this change as a realistic proposal, it would require some kind of language version- or mode-based switching of the match method's behavior (unlikely to happen, I would think). So, instead of that, I'd recommend deprecating the match method altogether in favor of a new method (perhaps RegExp.prototype.matchAll) with the changes prescribed above.

Get rid of /g and RegExp.prototype.global

Actual proposal: Deprecate /g and RegExp.prototype.global, and add a boolean replaceAll argument to String.prototype.replace

If the last two proposals were implemented and therefore regexp.lastIndex and string.match were things of the past (or string.match no longer sometimes served as an alias of regexp.exec), the only method where /g would still have any impact is string.replace. Additionally, although /g follows prior art from Perl, etc., it doesn't really make sense to have something that is not an attribute of a regex stored as a regex flag. Really, /g is more of a statement about how you want methods to apply their own functionality, and it's not uncommon to want to use the same pattern with and without /g (currently you'd have to construct two different regexes to do so). If it was up to me, I'd get rid of the /g flag and its corresponding global property, and instead simply give the string.replace method an additional argument that indicates whether you want to replace the first match only (the default handling) or all matches. This would have the additional benefit of allowing replace-all functionality with nonregex searches.

Note that SpiderMonkey already has a proprietary third string.replace argument ("flags") that this proposal would conflict with. I doubt this conflict would cause much heartburn, but in any case, a new replaceAll argument would provide the same functionality that SpiderMonkey's flags argument is most useful for (that is, allowing global replacements with nonregex searches).

Change the behavior of backreferences to nonparticipating groups

Actual proposal: Make backreferences to nonparticipating groups fail to match

I'll keep this brief since David "liorean" Andersson and I have previously argued for this on ES-Discuss and elsewhere. David posted about this in detail on his blog (see: ECMAScript 3 Regular Expressions: A specification that doesn't make sense), and I've previously touched on it here (ECMAScript 3 Regular Expressions are Defective by Design). On several occasions, Brendan Eich has also stated that he'd like to see this changed. The short explanation of this behavior is that, in JavaScript, backreferences to capturing groups that have not (yet) participated in a match always succeed (i.e., they match the empty string), whereas the opposite is true in all other regex flavors: they fail to match and therefore cause the regex engine to backtrack or fail. JavaScript's behavior means that /(a|(b))\2c/.test("ac") returns true. The (negative) implications of this reach quite far when pushing the boundaries of regular expressions.

I think everyone agrees that changing to the traditional backreferencing behavior would be an improvement—it provides far more intuitive handling, compatibility with other regex flavors, and great potential for creative use (e.g., see my post on Mimicking Conditionals). The bigger question is whether it would be safe, in light of backward compatibility. I think it would be, since I imagine that more or less no one uses the unintuitive JavaScript behavior intentionally. The JavaScript behavior amounts to automatically adding a ? quantifier after backreferences to nonparticipating groups, which is what people already do explicitly if they actually want backreferences to nonzero-length subpatterns to be optional. Also note that Safari 3.0 and earlier did not follow the spec on this point and used the more intuitive behavior, although that has changed in more recent versions (notably, this change was due to a write up on my blog rather than reports of real-world errors).

Finally, it's probably worth noting that .NET's ECMAScript regex mode (enabled via the RegexOptions.ECMAScript flag) indeed switches .NET to ECMAScript's unconventional backreferencing behavior.

Make \d \D \w \W \b \B support Unicode (like \s \S . ^ $, which already do)

Actual proposal: Add a /u flag (and corresponding RegExp.prototype.unicode property) that changes the meaning of \d, \w, \b, and related tokens

Unicode-aware digit and word character matching is not an existing JavaScript capability (short of constructing character class monstrosities that are hundreds or thousands of characters long), and since JavaScript lacks lookbehind you can't reproduce a Unicode-aware word boundary. You could therefore say this proposal is outside the stated scope of this post, but I'm including it here because I consider this more of a fix than a new feature.

According to current JavaScript standards, \s, \S, ., ^, and $ use Unicode-based interpretations of whitespace and newline, whereas \d, \D, \w, \W, \b, and \B use ASCII-only interpretations of digit, word character, and word boundary (e.g., /na\b/.test("naïve") unfortunately returns true). See my post on JavaScript, Regex, and Unicode for further details. Adding Unicode support to these tokens would cause unexpected behavior for thousands of websites, but it could be implemented safely via a new /u flag (inspired by Python's re.U or re.UNICODE flag) and a corresponding RegExp.prototype.unicode property. Since it's actually fairly common to not want these tokens to be Unicode enabled in particular regex patterns, a new flag that activates Unicode support would offer the best of both worlds.

Change the behavior of backreference resetting during subpattern repetition

Actual proposal: Never reset backreference values during a match

Like the last backreferencing issue, this too was covered by David Andersson in his post ECMAScript 3 Regular Expressions: A specification that doesn't make sense. The issue here involves the value remembered by capturing groups nested within a quantified, outer group (e.g., /((a)|(b))*/). According to traditional behavior, the value remembered by a capturing group within a quantified grouping is whatever the group matched the last time it participated in the match. So, the value of $1 after /(?:(a)|(b))*/ is used to match "ab" would be "a". However, according to ES3/ES5, the value of backreferences to nested groupings is reset/erased after the outer grouping is repeated. Hence, /(?:(a)|(b))*/ would still match "ab", but after the match is complete $1 would reference a nonparticipating capturing group, which in JavaScript would match an empty string within the regex itself, and be returned as undefined in, e.g., the array returned by the regexp.exec.

My case for change is that current JavaScript behavior breaks from the norm in other regex flavors, does not lend itself to various types of creative patterns (see one example in my post on Capturing Multiple, Optional HTML Attribute Values), and in my opinion is far less intuitive than the more common, alternative regex behavior.

I believe this behavior is safe to change for two reasons. First, this is generally an edge case issue for all but hardcore regex wizards, and I'd be surprised to find regexes that rely on JavaScript's version of this behavior. Second, and more importantly, Internet Explorer does not implement this rule and follows the more traditional behavior.

Add an /s flag, already

Actual proposal: Add an /s flag (and corresponding RegExp.prototype.dotall property) that changes dot to match all characters including newlines

I'll sneak this one in as a change/fix rather than a new feature since it's not exactly difficult to use [\s\S] in place of a dot when you want the behavior of /s. I presume the /s flag has been excluded thus far to save novices from themselves and limit the damage of runaway backtracking, but what ends up happening is that people write horrifically inefficient patterns like (.|\r|\n)* instead.

Regex searches in JavaScript are seldom line-based, and it's therefore more common to want dot to include newlines than to match anything-but-newlines (although both modes are useful). It makes good sense to keep the default meaning of dot (no newlines) since it is shared by other regex flavors and required for backward compatibility, but adding support for the /s flag is overdue. A boolean indicating whether this flag was set should show up on regexes as a property named either singleline (the unfortunate name from Perl, .NET, etc.) or the more descriptive dotall (used in Java, Python, PCRE, etc.).

Personal preferences

Following are a few changes that would suit my preferences, although I don't think most people would consider them significant issues:

• Allow regex literals to use unescaped forward slashes within character clases (e.g., /[/]/). This was already included in the abandoned ES4 change proposals.
• Allow an unescaped ] as the first character in character classes (e.g., []] or [^]]). This is allowed in probably every other regex flavor, but creates an empty class followed by a literal ] in JavaScript. I'd like to imagine that no one uses empty classes intentionally, since they don't work consistently cross-browser and there are widely-used/common-sense alternatives ((?!) instead of [], and [\s\S] instead of [^]). Unfortunately, adherence to this JavaScript quirk is tested in Acid3 (test 89), which is likely enough to kill requests for this backward-incompatible but reasonable change.
• Change the $& token used in replacement strings to $0. It just makes sense. (Equivalents in other replacement text flavors for comparison: Perl: $&; Java: $0; .NET: $0, $&; PHP: $0, \0; Ruby: \0, \&; Python: \g<0>.)
• Get rid of the special meaning of [\b]. Within character classes, the metasequence \b matches a backspace character (equivalent to \x08). This is a worthless convenience since no one cares about matching backspace characters, and it's confusing given that \b matches a word boundary when used outside of character classes. Even though this would break from regex tradition (which I'd usually advocate following), I think that \b should have no special meaning inside character classes and simply match a literal b.

Fixed in ES3: Remove octal character references

ECMAScript 3 removed octal character references from regular expression syntax, although \0 was kept as a convenient exception that allows easily matching a NUL character. However, browsers have generally kept full octal support around for backward compatibility. Octals are very confusing in regular expressions since their syntax overlaps with backreferences and an extra leading zero is allowed outside of character classes. Consider the following regexes:

• /a\1/: \1 is an octal.
• /(a)\1/: \1 is a backreference.
• /(a)[\1]/: \1 is an octal.
• /(a)\1\2/: \1 is a backreference; \2 is an octal.
• /(a)\01\001[\01\001]/: All occurences of \01 and \001 are octals. However, according to the ES3+ specs, the numbers after each \0 should be treated (barring nonstandard extensions) as literal characters, completely changing what this regex matches.
• /(a)\0001[\0001]/: The \0001 outside the character class is an octal; but inside, the octal ends at the third zero (i.e., the character class matches character index zero or "1"). This regex is therefore equivalent to /(a)\x01[\x00\x31]/; although, as mentioned just above, adherence to ES3 would change the meaning.
• /(a)\00001[\00001]/: Outside the character class, the octal ends at the fourth zero and is followed by a literal "1". Inside, the octal ends at the third zero and is followed by a literal "01". And once again, ES3's exclusion of octals and inclusion of \0 could change the meaning.
• /\1(a)/: Given that, in JavaScript, backreferences to capturing groups that have not (yet) participated match the empty string, does this regex match "a" (i.e., \1 is treated as a backreference since a corresponding capturing group appears in the regex) or does it match "\x01a" (i.e., the \1 is treated as an octal since it appears before its corresponding group)? Unsurprisingly, browsers disagree.
• /(\2(a)){2}/: Now things get really hairy. Does this regex match "aa", "aaa", "\x02aaa", "2aaa", "\x02a\x02a", or "2a2a"? All of these options seem plausible, and browsers disagree on the correct choice.

There are other issues to worry about, too, like whether octal escapes go up to \377 (\xFF, 8-bit) or \777 (\u01FF, 9-bit); but in any case, octals in regular expressions are a confusing cluster-cuss. Even though ECMAScript has already cleaned up this mess by removing support for octals, browsers have not followed suit. I wish they would, because unlike browser makers, I don't have to worry about this bit of legacy (I never use octals in regular expressions, and neither should you).

Fixed in ES5: Don't cache regex literals

According to ES3 rules, regex literals did not create a new regex object if a literal with the same pattern/flag combination was already used in the same script or function (this did not apply to regexes created by the RegExp constructor). A common side effect of this was that regex literals using the /g flag did not have their lastIndex property reset in some cases where most developers would expect it. Several browsers didn't follow the spec on this unintuitive behavior, but Firefox did, and as a result it became the second most duplicated JavaScript bug report for Mozilla. Fortunately, ES5 got rid of this rule, and now regex literals must be recompiled every time they're encountered (this change is coming in Firefox 3.7).

———
So there you have it. I've outlined what I think the JavaScript RegExp API got wrong. Do you agree with all of these proposals, or would you if you didn't have to worry about backward compatibility? Are there better ways than what I've proposed to fix the issues discussed here? Got any other gripes with existing JavaScript regex features? I'm eager to hear feedback about this.

Since I've been focusing on the negative in this post, I'll note that I find working with regular expressions in JavaScript to be a generally pleasant experience. There's a hell of a lot that JavaScript got right.

Note that Jan's contest is separate from my ongoing giveaway to promote the release of High Performance JavaScript (ends Feb. 24^th). You can be entered in both contests at the same time.

Following are a couple recipes I wrote for Regular Expressions Cookbook, composing a fairly comprehensive guide to validating and formatting North American and international phone numbers using regular expressions. The regexes in these recipes are all pretty straightforward, but hopefully this gives an example of the depth you can expect from the book.

For more than 100 detailed regular expression recipes that include equal coverage for eight programming languages (C#, Java, JavaScript, Perl, PHP, Python, Ruby, and VB.NET), get your very own copy of Regular Expressions Cookbook. Also available in Russian, German, Japanese, and Czech.

• Validate and Format North American Phone Numbers
  • Variations:
    • Eliminate invalid phone numbers
    • Find phone numbers in documents
    • Allow a leading “1”
    • Allow seven-digit phone numbers
• Validate International Phone Numbers
  • Variations:
    • Validate international phone numbers in EPP format

Validate and Format North American Phone Numbers

Problem

You want to determine whether a user entered a North American phone number in a common format, including the local area code. These formats include 1234567890, 123-456-7890, 123.456.7890, 123 456 7890, (123) 456 7890, and all related combinations. If the phone number is valid, you want to convert it to your standard format, (123) 456-7890, so that your phone number records are consistent.

Solution

A regular expression can easily check whether a user entered something that looks like a valid phone number. By using capturing groups to remember each set of digits, the same regular expression can be used to replace the subject text with precisely the format you want.

Regular expression

Replacement

C#

Regex regexObj =
    new Regex(@"^\(?([0-9]{3})\)?[-. ]?([0-9]{3})[-. ]?([0-9]{4})$");

if (regexObj.IsMatch(subjectString)) {
    string formattedPhoneNumber =
        regexObj.Replace(subjectString, "($1) $2-$3");
} else {
    // Invalid phone number
}

JavaScript

var regexObj = /^\(?([0-9]{3})\)?[-. ]?([0-9]{3})[-. ]?([0-9]{4})$/;

if (regexObj.test(subjectString)) {
    var formattedPhoneNumber =
        subjectString.replace(regexObj, "($1) $2-$3");
} else {
    // Invalid phone number
}

Other programming languages

See Recipes 3.5 and 3.15 for help implementing this regular expression with other programming languages.

Discussion

This regular expression matches three groups of digits. The first group can optionally be enclosed with parentheses, and the first two groups can optionally be followed with a choice of three separators (a hyphen, dot, or space). The following layout breaks the regular expression into its individual parts, omitting the redundant groups of digits:

^        # Assert position at the beginning of the string.
\(       # Match a literal "("...
  ?      #   between zero and one time.
(        # Capture the enclosed match to backreference 1...
  [0-9]  #   Match a digit...
    {3}  #     exactly three times.
)        # End capturing group 1.
\)       # Match a literal ")"...
  ?      #   between zero and one time.
[-. ]    # Match one character from the set "-. "...
  ?      #   between zero and one time.
⋯        # [Match the remaining digits and separator.]
$        # Assert position at the end of the string.

Let’s look at each of these parts more closely. The ^ and $ at the beginning and end of the regular expression are a special kind of metacharacter called an anchor or assertion. Instead of matching text, assertions match a position within the text. Specifically, ^ matches at the beginning of the text, and $ at the end. This ensures that the phone number regex does not match within longer text, such as 123-456-78901.

As we’ve repeatedly seen, parentheses are special characters in regular expressions, but in this case we want to allow a user to enter parentheses and have our regex recognize them. This is a textbook example of where we need a backslash to escape a special character so the regular expression treats it as literal input. Thus, the \( and \) sequences that enclose the first group of digits match literal parenthesis characters. Both are followed by a question mark, which makes them optional. We’ll explain more about the question mark after discussing the other types of tokens in this regular expression.

The parentheses that appear without backslashes are capturing groups and are used to remember the values matched within them so that the matched text can be recalled later. In this case, backreferences to the captured values are used in the replacement text so we can easily reformat the phone number as needed.

Two other types of tokens used in this regular expression are character classes and quantifiers. Character classes allow you to match any one out of a set of characters. [0-9] is a character class that matches any digit. The regular expression flavors covered by this book all include the shorthand character class \d that also matches a digit, but in some flavors \d matches a digit from any language’s character set or script, which is not what we want here. See Recipe 2.3 for more information about \d.

[-. ] is another character class, one that allows any one of three separators. It’s important that the hyphen appears first in this character class, because if it appeared between other characters, it would create a range, as with [0-9]. Another way to ensure that a hyphen inside a character class matches a literal version of itself is to escape it with a backslash. [.\- ] is therefore equivalent.

Finally, quantifiers allow you to repeat a token or group. {3} is a quantifier that causes its preceding element to be repeated exactly three times. The regular expression [0-9]{3} is therefore equivalent to [0-9][0-9][0-9], but is shorter and hopefully easier to read. A question mark (mentioned earlier) is a special quantifier that causes its preceding element to repeat zero or one time. It could also be written as {0,1}. Any quantifier that allows something to be repeated zero times effectively makes that element optional. Since a question mark is used after each separator, the phone number digits are allowed to run together.

Note that although this recipe claims to handle North American phone numbers, it’s actually designed to work with North American Numbering Plan (NANP) numbers. The NANP is the telephone numbering plan for the countries that share the country code “1”. This includes the United States and its territories, Canada, Bermuda, and 16 Caribbean nations. It excludes Mexico and the Central American nations.

Variations

Eliminate invalid phone numbers

So far, the regular expression matches any 10-digit number. If you want to limit matches to valid phone numbers according to the North American Numbering Plan, here are the basic rules:

• Area codes start with a number from 2–9, followed by 0–8, and then any third digit.
• The second group of three digits, known as the central office or exchange code, starts with a number from 2–9, followed by any two digits.
• The final four digits, known as the station code, have no restrictions.

These rules can easily be implemented with a few character classes:

Beyond the basic rules just listed, there are a variety of reserved, unassigned, and restricted phone numbers. Unless you have very specific needs that require you to filter out as many phone numbers as possible, don’t go overboard trying to eliminate unused numbers. New area codes that fit the rules listed earlier are made available regularly, and even if a phone number is valid, that doesn’t necessarily mean it was issued or is in active use.

Find phone numbers in documents

Two simple changes allow the previous regular expression to match phone numbers within longer text:

Here, the ^ and $ assertions that bound the regular expression to the beginning and end of the text have been removed. In their place, word boundary tokens (\b) have been added to ensure that the matched text stands on its own and is not part of a longer number or word.

Similar to ^ and $, \b is an assertion that matches a position rather than any actual text. Specifically, \b matches the position between a word character and either a nonword character or the beginning or end of the text. Letters, numbers, and underscore are all considered word characters (see Recipe 2.6).

Note that the first word boundary token appears after the optional, opening parenthesis. This is important because there is no word boundary to be matched between two nonword characters, such as the opening parenthesis and a preceding space character. The first word boundary is relevant only when matching a number without parentheses, since the word boundary always matches between the opening parenthesis and the first digit of a phone number.

Allow a leading “1”

You can allow an optional, leading “1” for the country code (which covers the North American Numbering Plan region) via the addition shown in the following regex:

In addition to the phone number formats shown previously, this regular expression will also match strings such as +1 (123) 456-7890 and 1-123-456-7890. It uses a noncapturing group, written as (?:⋯). When a question mark follows an unescaped left parenthesis like this, it’s not a quantifier, but instead helps to identify the type of grouping. Standard capturing groups require the regular expression engine to keep track of backreferences, so it’s more efficient to use noncapturing groups whenever the text matched by a group does not need to be referenced later. Another reason to use a noncapturing group here is to allow you to keep using the same replacement string as in the previous examples. If we added a capturing group, we’d have to change $1 to $2 (and so on) in the replacement text shown earlier in this recipe.

The full addition to this version of the regex is (?:\+?1[-. ]?)?. The “1” in this pattern is preceded by an optional plus sign, and optionally followed by one of three separators (hyphen, dot, or space). The entire, added noncapturing group is also optional, but since the “1” is required within the group, the preceding plus sign and separator are not allowed if there is no leading “1”.

Allow seven-digit phone numbers

To allow matching phone numbers that omit the local area code, enclose the first group of digits together with its surrounding parentheses and following separator in an optional, noncapturing group:

Since the area code is no longer required as part of the match, simply replacing any match with ($1) $2-$3 might now result in something like () 123-4567, with an empty set of parentheses. To work around this, add code outside the regex that checks whether group 1 matched any text, and adjust the replacement text accordingly.

See Also

Recipe 4.3 shows how to validate international phone numbers.

The North American Numbering Plan (NANP) is the telephone numbering plan for the United States and its territories, Canada, Bermuda, and 16 Caribbean nations. More information is available at http://www.nanpa.com.

Validate International Phone Numbers

Problem

You want to validate international phone numbers. The numbers should start with a plus sign, followed by the country code and national number.

Solution

Regular expression

JavaScript

function validate (phone) {
    var regex = /^\+(?:[0-9] ?){6,14}[0-9]$/;

    if (regex.test(phone)) {
        // Valid international phone number
    } else {
        // Invalid international phone number
    }
}

Other programming languages

See Recipe 3.5 for help implementing this regular expression with other programming languages.

Discussion

The rules and conventions used to print international phone numbers vary significantly around the world, so it’s hard to provide meaningful validation for an international phone number unless you adopt a strict format. Fortunately, there is a simple, industry-standard notation specified by ITU-T E.123. This notation requires that international phone numbers include a leading plus sign (known as the international prefix symbol), and allows only spaces to separate groups of digits. Although the tilde character (~) can appear within a phone number to indicate the existence of an additional dial tone, it has been excluded from this regular expression since it is merely a procedural element (in other words, it is not actually dialed) and is infrequently used. Thanks to the international phone numbering plan (ITU-T E.164), phone numbers cannot contain more than 15 digits. The shortest international phone numbers in use contain seven digits.

With all of this in mind, let’s look at the regular expression again after breaking it into its pieces. Because this version is written using free-spacing style, the literal space character has been replaced with \x20:

^         # Assert position at the beginning of the string.
\+        # Match a literal "+" character.
(?:       # Group but don't capture...
  [0-9]   #   Match a digit.
  \x20    #   Match a space character...
    ?     #     Between zero and one time.
)         # End the noncapturing group.
  {6,14}  #   Repeat the preceding group between 6 and 14 times.
[0-9]     # Match a digit.
$         # Assert position at the end of the string.

The ^ and $ anchors at the edges of the regular expression ensure that it matches the whole subject text. The noncapturing group—enclosed with (?:⋯)—matches a single digit, followed by an optional space character. Repeating this grouping with the interval quantifier {6,14} enforces the rules for the minimum and maximum number of digits, while allowing space separators to appear anywhere within the number. The second instance of the character class [0-9] completes the rule for the number of digits (bumping it up from between 6 and 14 digits to between 7 and 15), and ensures that the phone number does not end with a space.

Variations

Validate international phone numbers in EPP format

This regular expression follows the international phone number notation specified by the Extensible Provisioning Protocol (EPP). EPP is a relatively recent protocol (finalized in 2004), designed for communication between domain name registries and registrars. It is used by a growing number of domain name registries, including .com, .info, .net, .org, and .us. The significance of this is that EPP-style international phone numbers are increasingly used and recognized, and therefore provide a good alternative format for storing (and validating) international phone numbers.

EPP-style phone numbers use the format +CCC.NNNNNNNNNNxEEEE, where C is the 1–3 digit country code, N is up to 14 digits, and E is the (optional) extension. The leading plus sign and the dot following the country code are required. The literal “x” character is required only if an extension is provided.

See Also

Recipe 4.2 provides more options for validating North American phone numbers.

ITU-T Recommendation E.123 (“Notation for national and international telephone numbers, e-mail addresses and Web addresses”) can be downloaded here: http://www.itu.int/rec/T-REC-E.123.

ITU-T Recommendation E.164 (“The international public telecommunication numbering plan”) can be downloaded at http://www.itu.int/rec/T-REC-E.164.

National numbering plans can be downloaded at http://www.itu.int/ITU-T/inr/nnp.

RFC 4933 defines the syntax and semantics of EPP contact identifiers, including international phone numbers. You can download RFC 4933 at http://tools.ietf.org/html/rfc4933.

Update (2010-02-25): This contest is now closed.

Last year, Yahoo! engineer and all-around JavaScript badass Nicholas Zakas asked if I was interested in writing a chapter for a new book on JavaScript performance that he was working on. I agreed, and that book, High Performance JavaScript, is now available for preorder at Amazon and other fine book retailers.

In addition to the wide-ranging content by Nicholas and a chapter on string and regular expression performance by yours truly, chapters were also contributed by an awesome lineup of JavaScript performance gurus: Ross Harmes, Julien Lecomte, Stoyan Stefanov, and Matt Sweeney. This book is unique in its laser-focus on optimizing the performance of your JavaScript applications, and covers many advanced topics in the process. The chapter on strings and regular expressions provides what I think is easily the most in-depth coverage of cross-browser JavaScript regex performance currently available.

Here's the list of chapters:

1. Loading and Execution
2. Data Access
3. DOM Scripting (Stoyan Stefanov)
4. Algorithms and Flow Control
5. Strings and Regular Expressions (Steven Levithan)
6. Responsive Interfaces
7. Ajax (Ross Harmes)
8. Programming Practices
9. Build and Deployment (Julien Lecomte)
10. Tools (Matt Sweeney)

To celebrate the completion of this book, I'm giving away three copies. O'Reilly Media increased the offer to five books! All you need to do is comment on this post by February 24^th, and I'll pick five people to send a copy to as soon as it's released (Amazon says March 15^th). If you prefer, I'd be happy to send you a copy of Regular Expressions Cookbook instead (please note which book you want in your comment). Four winners will be chosen at random from the pool of unique commenters (I'll be tracking IPs), and the fifth based on the reason given for why you want a copy.

Make sure to include your email address in the comment form, since I'll need it to contact you if you're selected (your email address won't be used for any other purpose). Good luck, and congratulations to Nicholas Zakas and all the other authors on completing a fantastic new book!

Edit (2010-02-05): My blog has been offline more often than not for the first two days after posting this, and many people have reported that they were unable to post a comment. I apologize for the screw-up—my blog is now on a different server, and the problems should be resolved. Please try again!

Edit (2010-02-08): O'Reilly Media kindly offered to pick up the tab for this giveaway, and increased the winnings to five books!

Edit (2010-02-09): Nicholas Zakas posted more information about High Performance JavaScript on his blog: Announcing High Performance JavaScript.

Edit (2010-02-25): This contest is now closed. Winners will be announced here shortly.

Edit (2010-03-03): Following are the winners of this giveaway (the first four were chosen randomly):

1. David Henderson
2. Daniel Trebbien
3. Lea Verou
4. Stefan "schnalle" Schallerl
5. Adam Crabtree

No. 5 Adam Crabtree, who wants to review the book and share it with members of the DallasJS Meetup Group, wins the nonrandom drawing for the best reason to win a copy. Runners up for this selection were Yoav, who promised to donate the book to a high school library after he's done with it; Nick Carter, who threatened me with his wrath if he doesn't win (I'll have to endure); Paul Irish, who kindly offered to have my last name corrected (to that of a sea monster) in exchange for winning; Alexei, a technical editor of a couple of Nicholas Zakas's previous books who'd like to know how many errors this one contains; and Marcel Korpel, who wants to improve his users' health by reducing the "headaches, general stress and insomnia" they suffer while waiting on his websites.

The winners have been informed by email about how to collect their prize. Thanks to everyone for playing!