Note
----

For testing and comparison with the current 're' module the new implementation is in the form of a module called 'regex'.


Flags
-----

There are 2 kinds of flag: scoped and global. Scoped flags can apply to only part of a pattern and can be turned on or off; global flags apply to the entire pattern and can only be turned on.

The scoped flags are: ``IGNORECASE``, ``MULTILINE``, ``DOTALL``, ``VERBOSE``.

The global flags are: ``ASCII``, ``LOCALE``, ``REVERSE``, ``UNICODE``, ``ZEROWIDTH``.


Notes on named capture groups
-----------------------------

All capture groups have a group number, starting from 1.

Groups with the same group name will have the same group number, and groups with a different group name will have a different group number.

The same group name can be used on different branches of an alternation because they are mutually exclusive, eg. ``(?<foo>first)|(?<foo>second)``. They will, of course, have the same group number.

Group numbers will be reused, where possible, across different branches of a branch reset, eg. ``(?|(first)|(second))`` has only group 1. If capture groups have different group names then they will, of course, have different group numbers, eg. ``(?|(?<foo>first)|(?<bar>second))`` has group 1 ("foo") and group 2 ("bar").


Additional features
-------------------

* Atomic grouping (issue #433030)

    ``(?>...)``

    If the following pattern subsequently fails, then the subpattern as a whole will fail.

* Possessive quantifiers.
    ``(?:...)?+`` ; ``(?:...)*+`` ; ``(?:...)++`` ; ``(?:...){min,max}+``

    The subpattern is matched up to 'max' times. If the following pattern subsequently fails, then all of the repeated subpatterns will fail as a whole. For example, ``(?:...)++`` is equivalent to ``(?>(?:...)+)``.

* Scoped flags (issue #433028)

    ``(?flags-flags:...)``

    The flags will apply only to the subpattern. Flags can be turned on or off.

* Inline flags (#433024, #433027)

    ``(?flags-flags)``

    The flags will apply to the end of the group or pattern. Flags can be turned on or off.

* Repeated repeats (#2537)

    A regex like ``((x|y+)*)*`` will be accepted and will work correctly, but should complete more quickly.

* Definition of 'word' character (#1693050)

    The definition of a 'word' character has been expanded for Unicode. This applies to ``\w``, ``\W``, ``\b`` and ``\B``.

* Groups in lookahead and lookbehind (#814253)

    Groups and group references are permitted in both lookahead and lookbehind.

* Variable-length lookbehind

    A lookbehind can match a variable-length string.

* Correct handling of charset with ignore case flag (#3511)

    Ranges within charsets are handled correctly when the ignore-case flag is turned on.

* Unmatched group in replacement (#1519638)

    An unmatched group is treated as an empty string in a replacement template.

* 'Pathological' patterns (#1566086, #1662581, #1448325, #1721518, #1297193)

    'Pathological' patterns should complete more quickly.

* Flags argument for regex.split, regex.sub and regex.subn (#3482)

    ``regex.split``, ``regex.sub`` and ``regex.subn`` support a 'flags' argument.

* 'Overlapped' argument for regex.findall and regex.finditer

    ``regex.findall`` and ``regex.finditer`` support an 'overlapped' flag which permits overlapped matches.

* Unicode escapes (#3665)

    The Unicode escapes ``\uxxxx`` and ``\Uxxxxxxxx`` are supported.

* Large patterns (#1160)

    Patterns can be much larger.

* Zero-width match with regex.finditer (#1647489)

    ``regex.finditer`` behaves correctly when it splits at a zero-width match.

* Zero-width split with regex.split (#3262)

    ``regex.split`` can split at a zero-width match if the zero-width flag is turned on. When the flag is turned off the current behaviour is unchanged because the BDFL thinks that some existing software might depend on it.

* Splititer

    ``regex.splititer`` has been added. It's a generator equivalent of ``regex.split``.

* Subscripting for groups

    A match object accepts access to the captured groups via subscripting and slicing:

    >>> m = regex.search(r"(?<before>.*?)(?<num>\\d+)(?<after>.*)", "pqr123stu")
    >>> print m["before"]
    pqr
    >>> print m["num"]
    123
    >>> print m["after"]
    stu
    >>> print len(m)
    4
    >>> print m[:]
    ('pqr123stu', 'pqr', '123', 'stu')

* Named groups

    Named groups can be named with ``(?<name>...)`` as well as the current ``(?P<name>...)``.

* Group references

    Groups can be referenced within a pattern with ``\g<name>``. This also allows there to be more than 99 groups.

* Named characters

    ``\N{name}``

    Named characters are supported.

* Unicode codepoint properties, blocks and scripts

    ``\p{name}`` ; ``\P{name}``

    Unicode properties, blocks and scripts are supported. ``\p{name}`` matches a character which has property 'name' and ``\P{name}`` matches a character which doesn't have property 'name'.

    In order to avoid ambiguity, block names should start with ``In`` and script names should start with ``Is``. If a name lacks such a prefix and it could be a block or a script, script will take priority, for example:

    1. ``InBasicLatin`` or ``BasicLatin``, the 'BasicLatin' **block**.

    2. ``IsLatin`` or ``Latin``, the 'Latin' **script**.

    3. ``InCyrillic``, the 'Cyrillic' **block**.

    4. ``IsCyrillic`` or ``Cyrillic``, the 'Cyrillic' **script**.

* Posix character classes

    ``[[:alpha:]]``

    Posix character classes are supported.

* Search anchor

    ``\G``

    A search anchor has been added. It matches at the position where each search started/continued and can be used for contiguous matches or in negative variable-length lookbehinds to limit how far back the lookbehind goes:

    >>> regex.findall(r"\w{2}", "abcd ef")
    ['ab', 'cd', 'ef']
    >>> regex.findall(r"\G\w{2}", "abcd ef")
    ['ab', 'cd']

    1. The search starts at position 0 and matches 2 letters 'ab'.

    2. The search continues at position 2 and matches 2 letters 'cd'.

    3. The search continues at position 4 and fails to match any letters.

    4. The anchor stops the search start position from being advanced, so there are no more results.

* Reverse searching

    Searches can now work backwards:

    >>> regex.findall(r".", "abc")
    ['a', 'b', 'c']
    >>> regex.findall(r"(?r).", "abc")
    ['c', 'b', 'a']

    Note: the result of a reverse search is not necessarily the reverse of a forward search:

    >>> regex.findall(r"..", "abcde")
    ['ab', 'cd']
    >>> regex.findall(r"(?r)..", "abcde")
    ['de', 'bc']

* Multithreading

    The regex module now releases the GIL when matching, enabling other Python threads to run concurrently.

* Matching a single grapheme

    ``\X``

    The grapheme matcher is supported. It's equivalent to ``\P{M}\p{M}*``.

* Branch reset

	(?|...|...)

    Capture group numbers will be reused across the alternatives.
