Regular Expressions Cookbook is Out

As of today, Regular Expressions Cookbook (written by Jan Goyvaerts and me, and published by O'Reilly Media) is listed as In Stock on Amazon.com and other fine bookstores. The book covers seven regular expression flavors (.NET, Java, JavaScript, Perl, PCRE, Python, and Ruby) and eight programming languages (C#, Java, JavaScript, Perl, PHP, Python, Ruby, and VB.NET). It's targeted at people with regex skills from beginner to upper intermediate, and there's a fair amount of information in there even for people who already consider themselves regex experts. For those who'd like to know more, Jan has a good summary on his blog, and here is O'Reilly's press release for the book.

Don't forget to pick up a copy of your very own.

XRegExp 0.5 Released!

Update: This version of XRegExp is outdated. See XRegExp.com for the latest, greatest version.

If you haven't seen the prior versions, XRegExp is an MIT-licensed JavaScript library that provides an augmented, cross-browser implementation of regular expressions, including support for additional modifiers and syntax. Several convenience methods and a new, powerful recursive-construct parser that uses regex delimiters are also included.

Here's what you get beyond the standard JavaScript regex features:

  • Added regex syntax:
    • Comprehensive named capture support. (Improved)
    • Comment patterns: (?#…). (New)
  • Added regex modifiers (flags):
    • s (singleline), to make dot match all characters including newlines.
    • x (extended), for free-spacing and comments.
  • Added awesome:
    • Reduced cross-browser inconsistencies. (More)
    • Recursive-construct parser with regex delimiters. (New)
    • An easy way to cache and reuse regex objects. (New)
    • The ability to safely embed literal text in your regex patterns. (New)
    • A method to add modifiers to existing regex objects.
    • Regex call and apply methods, which make generically working with functions and regexes easier. (New)

All of this can be yours for the low, low price of 2.4 KB. smile Version 0.5 also introduces extensive documentation and code examples.

If you're using a previous version, note that there are a few non-backward compatible changes for the sake of strict ECMA-262 Edition 3 compliance and compatibility with upcoming ECMAScript 4 changes.

  • The XRegExp.overrideNative function has been removed, since it is no longer possible to override native constructors in Firefox 3 or ECMAScript 4 (as proposed).
  • Named capture syntax has been changed from (<name>…) to (?<name>…), which is the standard in most regex libraries and under consideration for ES4. Named capture is now always available, and does not require the k modifier.
  • Due to cross-browser compatibility issues, previous versions enforced that a leading, unescaped ] within a character class was treated as a literal character, which is how things work in most regex flavors. XRegExp now follows ECMA-262 Edition 3 on this point. [] is an empty set and never matches (this is enforced in all browsers).

Get it while it's hot! Check out the new XRegExp documentation and source code.

Date Format 1.1

I've just updated my ColdFusion-inspired JavaScript Date Format script to version 1.0 1.1, and updated the documentation in the old post along with it. The new release includes "Z" (US timezone abbreviation) and "o" (UTC offset) flags as well as brevity enhancements from Scott Trenda, along with several other new features including a standalone dateFormat function, named and default masks (plus you can easily add your own), easier internationalization, etc.

This update includes one change which is not backwards compatible: mask characters and sequences no longer have to comprise entire words for them to be treated specially. The former handling was intended to make it dead-easy to mix literal characters into date masks, but ended up mostly just being a slight nuisance since most people didn't use it to embed dates in larger strings.

Check out the new Date Format!


Edit: Date Format is now integrated into two JavaScript frameworks:

  • CFJS is a library of almost 70 ColdFusion functions written in JavaScript by Chris Jordan. CFJS has used Date Format, which was a natural fit since it's largely based on ColdFusion's dateFormat and timeFormat functions, since version 0.1.
  • Chiron is an innovative, emerging JavaScript library by Kris Kowal. It's based on Python idioms, and at its heart is an advanced module loader and isolation system the likes of which hasn't been seen yet in the JavaScript world. In addition to integrating Date Format as a module called date.js, Chiron has also integrated my XRegExp library, and uses regular expressions from parseUri in its core. Expect to hear more about Chiron as it gets closer to 0.1 release.

RegexPal Now Open Source

RegexPal (easily the most del.icio.used regex tester wink) is now released under the Creative Commons Attribution-Share Alike 3.0 License GNU LGPL.

There are certainly many more features that can be added to the app and things that can be improved, so if you are interested in helping out or creating your own version, you are welcome to do so. If there is interest I'll create a Google Code project, but for now there is a package you can download which includes all files for the regexpal.com website. Two of the files in the package (xregexp.js and helpers.js) are dual-licensed under the MIT License.

If you're only interested in the JavaScript, you can see the three source files here:

For regex aficionados particularly, there is some stuff here you might find interesting, including the latest, as-yet-unreleased version of my XRegExp library, and the regex syntax parser used for RegexPal's syntax highlighting (which includes lots of details on the minutiae of regex syntax and cross-browser regex handling).

RegexPal: Web-Based Regex Testing Reinvented

Yes I know, there are many other JavaScript regex testers available. Why did I create yet another? RegexPal brings several new things to the table for such web-based apps, and in my (biased) opinion it's easier to use and more helpful towards learning regular expressions than the others currently available. Additionally, most other such tools are very slow for the kind of data I often work with. They might appear fast when displaying 10 matches, but what about 100, 1000, or 5000? Try generating 5,000 matches (which is easy to do with an any-character pattern such as a dot) in your favorite existing web-based tool and see if your browser ever recovers (doubtful). The same task takes RegexPal less than half a second, and what's more, results overlay the text while you're typing it.

At the moment, RegexPal is short on features, but here are the highlights:

  • Real-time regex syntax highlighting with backwards and forwards context awareness.
  • Lightning-fast match highlighting with alternating styles.
  • Inverted matches (match any text not matched by the regex).
regexpal.com screenshot

I'm not sure when I'll add additional features, but there are lots of things I'm considering. If there is something you'd like to see, let me know.

A few things to be aware of:

  • The approach I've used for scrollable rich-text editing (which I haven't seen elsewhere) is fast but a bit buggy. Firefox 2 and IE7 have the least issues, but it more or less works in other browsers as well.
  • The syntax highlighting generally marks corner-case issues that create cross-browser inconsistencies as errors even if they are the result of browser bugs or missing behavior documentation in ECMA-262 v3.
  • There are different forms of line breaks cross-platform/browser. E.g., Firefox uses \n even on Windows where nearly all programs use \r\n. This can affect the results of certain regexes.

At least for me, RegexPal is lots of fun to play with and helps to make learning regular expressions easy through its instant feedback. I encourage you to just go play with it and discover its results on your own, but for the curious, I'll keep rambling…

Regex syntax parsing (needed for the syntax highlighting) is somewhat complex, due to the numerous backwards and forwards context awareness issues involved. Take, for example, the pattern \10. What does it mean?

  • Backreference 10, if not inside a character class and at least 10 capturing groups are opened before that point.
  • Backreference 1, followed by a literal "0", if not inside a character class and between 1 and 9 capturing groups are opened before that point.
  • Octal character index 10 (decimal 8), if inside a character class, or if no capturing groups are opened before that point.
  • The three literal characters "\", "1", and "0", if preceded by an unescaped "\" character.
  • An incomplete token in a couple other situations.

Another example is the "-" character. Outside a character class it's always a literal hyphen, but inside a character class…

  • It creates a range between tokens if:
    • There is a preceding and following token in the class, or it's preceded by a token and is the last character in an unclosed character class (caveats follow).
  • It's a literal character if:
    • It's the first or last character in the class.
    • It's preceded by an unescaped "\".
    • It follows a token which is the end index for a range.
    • It follows a hyphen which creates a range.
  • It's an error if:
    • It's creating a range between tokens in reverse character index order (e.g., z-a, @-!, \uFFFF-\b, or \127-\cB).
    • It would otherwise create a range, but it's followed or preceded by a token which represents more than one character index (e.g., \d). In fact, in some cases browsers take this to mean that the hyphen should be treated as a literal, but browser bugs cause it to be handled inconsistently so RegexPal flags it as a range error.

Here are a few more things which aren't errors but are flagged as such:

  • Empty, top-level alternation, except at the end of the pattern, where such an alternation is ignored when highlighting matches in order to create a less surprising experience while the user is in the middle of constructing the regex. Empty, top-level alternation is flagged as an error because it effectively truncates the regex at that point (since it will always match). If a zero-length, top-level alteration is really needed, there are other easy ways to do that more explicitly.
  • Lookaround quantifiers (e.g., the plus sign in (?!x)+). This would be an actual error with some regex libraries (e.g., PCRE), and although that's not the case in most web browsers, such constructs add no value. As a result, RegexPal flags such quantifiers as an error, since they are almost certainly a user mistake.
  • \c when not followed by A–Z, \x when not followed by two hex characters, and \u when not followed by four hex characters. Although these do not cause most browsers to throw errors, they are handled inconsistently cross-browser and are hence flagged as errors. They would almost certainly be a user mistake even if the cross-browser issues didn't exist.

Credit to osteele.com from where the text of the short-and-sweet Quick Reference is based, and to RegexBuddy from JGsoft for inspiring many of RegexPal's features. The name RegexPal is, in part, a nod to RegexBuddy, but also selected because it contains both "regex" and "regexp." wink