Contents
Initial StepsI/O, Database
Stream I/O
BOM Values
Constants & Globals
Data Types
Platform String API
TCHAR String API
References
Here is a cheat sheet for converting Microsoft C or C++ source code to support Unicode.It does not attempt to explain much and presumes you are generally familiar with Microsoft'sapproach to Unicode. The goal is just to have a single place to look for names, correct spellings, etc.of relevant data types, functions, etc.
Also see: Cheat Sheet: Unicode-enabling Microsoft C/C++ Source Code in Chinese
Unicode glyph cheat sheet Common Unicode characters with digraph or code point, layed out for quick location. Includes general symbols, arrows, drawing characters, and IPA letters. Unicode is a computing standard for the consistent encoding symbols. It was created in 1991. It’s just a table, which shows glyphs position to encoding system. Encoding takes symbol from table, and tells font what should be painted. But computer can understand binary code only. So, encoding is used number 1 or 0 to represent characters.
Initial Steps for Unicode-enabling Microsoft C/C++ Source
- Define _UNICODE, undefine _MBCS if defined.
- Convert literal strings to use L or _T
- Convert string functions to use Wide or TCHAR versions.
- Clarify string lengths in API as byte or character counts.For character-based display or printing (as opposed to GUI which is pixel-based)use column counts, not byte or character.
- Replace character pointer arithmetic with GetNext style, ascharacters may consist of more than one Unicode code unit.
- Watch buffer size and buffer overflows- changing encodingsmay require either larger buffers or limiting string lengths. Ifcharacter size changes from 1 byte to as many as 4 bytes, andstring length was formerly 20 characters and 20 bytes, eitherexpand the string buffer(s) from 20 to 80 bytes or limit thestring to 5 characters (and therefore 20 bytes).Note maximum buffer expansion may be constrained (for example to 65 KB).Reducing string length to a fixed number of characters may break existing applications.Limiting strings to a fixed byte length is dangerous. For example, allowing anystring that fits into 20 bytes. Simple operations such as uppercasinga string may cause it to grow and exceed the byte length.
- Replace functions that accept or return arguments of a single character, with functions that use strings instead.(International) Operations on a single character may result in more than one code point being returned. For example,upper('ß')returns 'SS'.
- Use wmain instead of main.The environment variable is then _wenviron instead of _environ.
wmain( int argc, wchar_t *argv[ ], wchar_t *envp[ ] ). - MFC Unicode applications use wWinMain as the entry point.
In the Output page of the Linker folder in the project's Property Pages dialog box,set the Entry Point symbol to wWinMainCRTStartup. - Consider fonts. Identify the fonts that will render each language or script used.
File I/O, Database, Transfer Protocol Considerations
- Consider whether to read/write UTF-8 or UTF-16 in files, databases, and for data exchange.
- Consider Endian-ness in UTF-16 files.
Read/Write Big-Endian on networks. Use Big-Endian if you don't produce a BOM.
Endian-ness of files will depend on the file format and/or the architectureof the source or target machine.
When reading files encoded in UTF-16 or UTF-32,be prepared to swap-bytes to convert endian-ness.
Also consider streams and transfer protocols and the encoding used in each. - Label files or protocols for data exchange with the correct character encoding.E.g. set HTTP, HTML, XML to UTF-8 or UTF-16.
- Consider Unicode BOM (Byte Order Marker) and whetherit should be written with data. Remove it when reading data.
- Consider encoding conversion of legacy data and files, import and export, transfer protocols.(MultiByteToWideChar,WideCharToMultiByte,mbtowc,wctomb,wctombs,mbstowcs )
- Consider writing to the Clipboard-
use CF_TEXT format and write native character encoding (ANSI) text, and
use CF_UNICODETEXT format and write Unicode text. - Database applications should consider Data Type(NCHAR, NVARCHAR)and Schema Changes, Triggers, Stored Procedures, and Queries. Data Storagegrowth, Indexes and Performance.
Note that the Unicode schema changes will have different impacts and concerns on different vendors' databases.If database portability is a requirement, the features and behaviors of each database need to be taken into account.
(I know this item is seriously understated. To be expanded sometime in the future.)
Stream I/O
Streams are difficult in Microsoft C++. You may run into 3 types of problems:
- Unicode filenames are not supported.The workaround is to useFILE * _wfopen and if needed, use the FILE handle in subsequent stream I/O.
- Stream I/O will convert Unicode data from/to native (ANSI) code page on read/write,not UTF-8 or UTF-16.However the stream class can be modified to read/write UTF-8.You can implement a facet to convert between Unicode and UTF-8.codecvt <wchar_t, char_traits <wchar_t> >
- To read/write UTF-16 with stream I/O, use binary opens and binary I/O. To set binary I/O:
Also see the Microsoft run-time library reference: 'Unicode Stream I/O in Text and Binary Modes'.
Note: There aren't TCHAR equivalents forcout/wcout, cin/wcin, etc.You may want to make your own preprocessor definition for 'tout',if you are compiling code both ways.
Internationalization, Advanced Unicode, Platform and Other Considerations
- Consider using locale-based routines and further internationalization.
- For Windows 95, 98 and ME, consider using theMicrosoft MSLU (Microsoft Layer for Unicode)
- Consider string compares and sorting, Unicode Collation Algorithm
- Consider Unicode Normalization
- Consider Character Folding
- Reconsider doing this on your own. Bring in an experiencedUnicode consultant,and deploy your existing resources on the tasks they do best.(Hey, an I18nGuy's gotta earn a living...)
Unicode BOM Encoding Values
| The Byte Order Marker (BOM) is Unicode character U+FEFF.(It can also represent a Zero Width No-break Space.)The code point U+FFFE is illegal in Unicode, and shouldnever appear in a Unicode character stream. Therefore the BOM can be used in the first characterof a file (or more generally a string), as an indicator of endian-ness. With UTF-16,if the first character is read as bytes FE FF then the texthas the same endian-ness as the machine reading it. If the character is read as bytesFF FE, thenthe endian-ness is reversed and all 16-bit words should be byte-swapped as they are read-in.In the same way, the BOM indicates the endian-ness of text encoded with UTF-32. Note that not all files start with a BOM however. In fact, the Unicode Standard says that text thatdoes not begin with a BOM MUST be interpreted in big-endian form. The character U+FEFF also serves as an encoding signaturefor the Unicode Encoding Forms.The tableshows the encoding of U+FEFF in each of the Unicode encoding forms.Note that by definition, text labeled as UTF-16BE, UTF-32BE, UTF-32LE or UTF-16LE shouldnot have a BOM. The endian-ness isindicated in the label. For text that is compressed with the SCSU (Standard Compression Scheme for Unicode) algorithm,there is also a recommended signature. |
Constant and Global Variables
ANSI | Wide | TCHAR |
---|---|---|
EOF | WEOF | _TEOF |
_environ | _wenviron | _tenviron |
_pgmptr | _wpgmptr | _tpgmptr |
Data Types
ANSI | Wide | TCHAR |
---|---|---|
char | wchar_t | _TCHAR |
_finddata_t | _wfinddata_t | _tfinddata_t |
__finddata64_t | __wfinddata64_t | _tfinddata64_t |
_finddatai64_t | _wfinddatai64_t | _tfinddatai64_t |
int | wint_t | _TINT |
signed char | wchar_t | _TSCHAR |
unsigned char | wchar_t | _TUCHAR |
char | wchar_t | _TXCHAR |
L | _T or _TEXT | |
LPSTR (char *) | LPWSTR (wchar_t *) | LPTSTR (_TCHAR *) |
LPCSTR (const char *) | LPCWSTR (const wchar_t *) | LPCTSTR (const _TCHAR *) |
LPOLESTR (For OLE) | LPWSTR | LPTSTR |
Platform SDK String Functions
There are many Windows API that compile into ANSI or Wide forms,depending on whether the symbol UNICODE isdefined. Modules that operate on both ANSI and Wide characters, need tobe aware of this. Otherwise, using the Character Data Type-independent namerequires no changes, just compile with the symbol UNICODEdefined.
The following list is by no means all of the Character Data Type-dependent API, justsome character and string related ones. Look in WinNLS.h for some code pageand locale related API.
ANSI | Wide | Character Data Type- Independent Name |
---|---|---|
CharLowerA | CharLowerW | CharLower |
CharLowerBuffA | CharLowerBuffW | CharLowerBuff |
CharNextA | CharNextW | CharNext |
CharNextExA | CharNextExW | CharNextEx |
CharPrevA | CharPrevW | CharPrev |
CharPrevExA | CharPrevExW | CharPrevEx |
CharToOemA | CharToOemW | CharToOem |
CharToOemBuffA | CharToOemBuffW | CharToOemBuff |
CharUpperA | CharUpperW | CharUpper |
CharUpperBuffA | CharUpperBuffW | CharUpperBuff |
CompareStringA | CompareStringW | CompareString |
FoldStringA | FoldStringW | FoldString |
GetStringTypeA | GetStringTypeW | GetStringType |
GetStringTypeExA | GetStringTypeExW | GetStringTypeEx |
IsCharAlphaA | IsCharAlphaW | IsCharAlpha |
IsCharAlphaNumericA | IsCharAlphaNumericW | IsCharAlphaNumeric |
IsCharLowerA | IsCharLowerW | IsCharLower |
IsCharUpperA | IsCharUpperW | IsCharUpper |
LoadStringA | LoadStringW | LoadString |
lstrcatA | lstrcatW | lstrcat |
lstrcmpA | lstrcmpW | lstrcmp |
lstrcmpiA | lstrcmpiW | lstrcmpi |
lstrcpyA | lstrcpyW | lstrcpy |
lstrcpynA | lstrcpynW | lstrcpyn |
lstrlenA | lstrlenW | lstrlen |
OemToCharA | OemToCharW | OemToChar |
OemToCharBuffA | OemToCharBuffW | OemToCharBuff |
wsprintfA | wsprintfW | wsprintf |
wvsprintfA | wvsprintfW | wvsprintf |
TCHAR String Functions
Functions sorted by ANSI name, for ease of converting to Unicode.
ANSI | Wide | TCHAR |
---|---|---|
_access | _waccess | _taccess |
_atoi64 | _wtoi64 | _tstoi64 |
_atoi64 | _wtoi64 | _ttoi64 |
_cgets | _cgetws | cgetts |
_chdir | _wchdir | _tchdir |
_chmod | _wchmod | _tchmod |
_cprintf | _cwprintf | _tcprintf |
_cputs | _cputws | _cputts |
_creat | _wcreat | _tcreat |
_cscanf | _cwscanf | _tcscanf |
_ctime64 | _wctime64 | _tctime64 |
_execl | _wexecl | _texecl |
_execle | _wexecle | _texecle |
_execlp | _wexeclp | _texeclp |
_execlpe | _wexeclpe | _texeclpe |
_execv | _wexecv | _texecv |
_execve | _wexecve | _texecve |
_execvp | _wexecvp | _texecvp |
_execvpe | _wexecvpe | _texecvpe |
_fdopen | _wfdopen | _tfdopen |
_fgetchar | _fgetwchar | _fgettchar |
_findfirst | _wfindfirst | _tfindfirst |
_findnext64 | _wfindnext64 | _tfindnext64 |
_findnext | _wfindnext | _tfindnext |
_findnexti64 | _wfindnexti64 | _tfindnexti64 |
_fputchar | _fputwchar | _fputtchar |
_fsopen | _wfsopen | _tfsopen |
_fullpath | _wfullpath | _tfullpath |
_getch | _getwch | _gettch |
_getche | _getwche | _gettche |
_getcwd | _wgetcwd | _tgetcwd |
_getdcwd | _wgetdcwd | _tgetdcwd |
_ltoa | _ltow | _ltot |
_makepath | _wmakepath | _tmakepath |
_mkdir | _wmkdir | _tmkdir |
_mktemp | _wmktemp | _tmktemp |
_open | _wopen | _topen |
_popen | _wpopen | _tpopen |
_putch | _putwch | _puttch |
_putenv | _wputenv | _tputenv |
_rmdir | _wrmdir | _trmdir |
_scprintf | _scwprintf | _sctprintf |
_searchenv | _wsearchenv | _tsearchenv |
_snprintf | _snwprintf | _sntprintf |
_snscanf | _snwscanf | _sntscanf |
_sopen | _wsopen | _tsopen |
_spawnl | _wspawnl | _tspawnl |
_spawnle | _wspawnle | _tspawnle |
_spawnlp | _wspawnlp | _tspawnlp |
_spawnlpe | _wspawnlpe | _tspawnlpe |
_spawnv | _wspawnv | _tspawnv |
_spawnve | _wspawnve | _tspawnve |
_spawnvp | _wspawnvp | _tspawnvp |
_spawnvpe | _wspawnvpe | _tspawnvpe |
_splitpath | _wsplitpath | _tsplitpath |
_stat64 | _wstat64 | _tstat64 |
_stat | _wstat | _tstat |
_stati64 | _wstati64 | _tstati64 |
_strdate | _wstrdate | _tstrdate |
_strdec | _wcsdec | _tcsdec |
_strdup | _wcsdup | _tcsdup |
_stricmp | _wcsicmp | _tcsicmp |
_stricoll | _wcsicoll | _tcsicoll |
_strinc | _wcsinc | _tcsinc |
_strlwr | _wcslwr | _tcslwr |
_strncnt | _wcsncnt | _tcsnbcnt |
_strncnt | _wcsncnt | _tcsnccnt |
_strncnt | _wcsncnt | _tcsnccnt |
_strncoll | _wcsncoll | _tcsnccoll |
_strnextc | _wcsnextc | _tcsnextc |
_strnicmp | _wcsnicmp | _tcsncicmp |
_strnicmp | _wcsnicmp | _tcsnicmp |
_strnicoll | _wcsnicoll | _tcsncicoll |
_strnicoll | _wcsnicoll | _tcsnicoll |
_strninc | _wcsninc | _tcsninc |
_strnset | _wcsnset | _tcsncset |
_strnset | _wcsnset | _tcsnset |
_strrev | _wcsrev | _tcsrev |
_strset | _wcsset | _tcsset |
_strspnp | _wcsspnp | _tcsspnp |
_strtime | _wstrtime | _tstrtime |
_strtoi64 | _wcstoi64 | _tcstoi64 |
_strtoui64 | _wcstoui64 | _tcstoui64 |
_strupr | _wcsupr | _tcsupr |
_tempnam | _wtempnam | _ttempnam |
_ui64toa | _ui64tow | _ui64tot |
_ultoa | _ultow | _ultot |
_ungetch | _ungetwch | _ungettch |
_unlink | _wunlink | _tunlink |
_utime64 | _wutime64 | _tutime64 |
_utime | _wutime | _tutime |
_vscprintf | _vscwprintf | _vsctprintf |
_vsnprintf | _vsnwprintf | _vsntprintf |
asctime | _wasctime | _tasctime |
atof | _wtof | _tstof |
atoi | _wtoi | _tstoi |
atoi | _wtoi | _ttoi |
atol | _wtol | _tstol |
atol | _wtol | _ttol |
character compare | Maps to macro or inline function | _tccmp |
character copy | Maps to macro or inline function | _tccpy |
character length | Maps to macro or inline function | _tclen |
ctime | _wctime | _tctime |
fgetc | fgetwc | _fgettc |
fgets | fgetws | _fgetts |
fopen | _wfopen | _tfopen |
fprintf | fwprintf | _ftprintf |
fputc | fputwc | _fputtc |
fputs | fputws | _fputts |
freopen | _wfreopen | _tfreopen |
fscanf | fwscanf | _ftscanf |
getc | getwc | _gettc |
getchar | getwchar | _gettchar |
getenv | _wgetenv | _tgetenv |
gets | getws | _getts |
isalnum | iswalnum | _istalnum |
isalpha | iswalpha | _istalpha |
isascii | iswascii | _istascii |
iscntrl | iswcntrl | _istcntrl |
isdigit | iswdigit | _istdigit |
isgraph | iswgraph | _istgraph |
islead (Always FALSE) | (Always FALSE) | _istlead |
isleadbyte (Always FALSE) | isleadbyte (Always FALSE) | _istleadbyte |
islegal (Always TRUE) | (Always TRUE) | _istlegal |
islower | iswlower | _istlower |
isprint | iswprint | _istprint |
ispunct | iswpunct | _istpunct |
isspace | iswspace | _istspace |
isupper | iswupper | _istupper |
isxdigit | iswxdigit | _istxdigit |
main | wmain | _tmain |
perror | _wperror | _tperror |
printf | wprintf | _tprintf |
putc | putwc | _puttc |
putchar | putwchar | _puttchar |
puts | _putws | _putts |
remove | _wremove | _tremove |
rename | _wrename | _trename |
scanf | wscanf | _tscanf |
setlocale | _wsetlocale | _tsetlocale |
sprintf | swprintf | _stprintf |
sscanf | swscanf | _stscanf |
strcat | wcscat | _tcscat |
strchr | wcschr | _tcschr |
strcmp | wcscmp | _tcscmp |
strcoll | wcscoll | _tcscoll |
strcpy | wcscpy | _tcscpy |
strcspn | wcscspn | _tcscspn |
strerror | _wcserror | _tcserror |
strftime | wcsftime | _tcsftime |
strlen | wcslen | _tcsclen |
strlen | wcslen | _tcslen |
strncat | wcsncat | _tcsncat |
strncat | wcsncat | _tcsnccat |
strncmp | wcsncmp | _tcsnccmp |
strncmp | wcsncmp | _tcsncmp |
strncpy | wcsncpy | _tcsnccpy |
strncpy | wcsncpy | _tcsncpy |
strpbrk | wcspbrk | _tcspbrk |
strrchr | wcsrchr | _tcsrchr |
strspn | wcsspn | _tcsspn |
strstr | wcsstr | _tcsstr |
strtod | wcstod | _tcstod |
strtok | wcstok | _tcstok |
strtol | wcstol | _tcstol |
strtoul | wcstoul | _tcstoul |
strxfrm | wcsxfrm | _tcsxfrm |
system | _wsystem | _tsystem |
tmpnam | _wtmpnam | _ttmpnam |
tolower | towlower | _totlower |
toupper | towupper | _totupper |
ungetc | ungetwc | _ungettc |
vfprintf | vfwprintf | _vftprintf |
vprintf | vwprintf | _vtprintf |
vsprintf | vswprintf | _vstprintf |
WinMain | wWinMain | _tWinMain |
References
- 'MSLU: Develop Unicode Applications for Windows 9x Platforms with the Microsoft Layer for Unicode'Michael Kaplan, Cathy Wissink
- 'Design a Single Unicode App that Runs on Both Windows 98 and Windows 2000'F. Avery Bishop, April 1999.
- 'Supporting Multilanguage Text Layout and Complex Scripts with Windows NT 5.0'F. Avery Bishop, David C Brown and Davis M Meltzer, November 1998.
- Newsgroup: MSDN MSLU
- Newsgroup: MSDN Internationalization
Copyright © 2003-2010 Tex Texin. All rights reserved.
Top of page
The tables are not exhaustive, for two reasons. First, every regex flavor is different, and I didn't want to crowd the page with overly exotic syntax. For a full reference to the particular regex flavors you'll be using, it's always best to go straight to the source. In fact, for some regex engines (such as Perl, PCRE, Java and .NET) you may want to check once a year, as their creators often introduce new features.
The other reason the tables are not exhaustive is that I wanted them to serve as a quick introduction to regex. If you are a complete beginner, you should get a firm grasp of basic regex syntax just by reading the examples in the tables. I tried to introduce features in a logical order and to keep out oddities that I've never seen in actual use, such as the 'bell character'. With these tables as a jumping board, you will be able to advance to mastery by exploring the other pages on the site.
How to use the tables
The tables are meant to serve as an accelerated regex course, and they are meant to be read slowly, one line at a time. On each line, in the leftmost column, you will find a new element of regex syntax. The next column, 'Legend', explains what the element means (or encodes) in the regex syntax. The next two columns work hand in hand: the 'Example' column gives a valid regular expression that uses the element, and the 'Sample Match' column presents a text string that could be matched by the regular expression.You can read the tables online, of course, but if you suffer from even the mildest case of online-ADD (attention deficit disorder), like most of us… Well then, I highly recommend you print them out. You'll be able to study them slowly, and to use them as a cheat sheet later, when you are reading the rest of the site or experimenting with your own regular expressions.
Enjoy!
If you overdose, make sure not to miss the next page, which comes back down to Earth and talks about some really cool stuff: The 1001 ways to use Regex.
Regex Accelerated Course and Cheat Sheet
For easy navigation, here are some jumping points to various sections of the page:✽ Characters
✽ Quantifiers
✽ More Characters
✽ Logic
Unicode Cheat Sheet Free
✽ More White-Space
✽ More Quantifiers
✽ Character Classes
✽ Anchors and Boundaries
✽ POSIX Classes
✽ Inline Modifiers
✽ Lookarounds
✽ Character Class Operations
✽ Other Syntax
(direct link)
Characters
Character | Legend | Example | Sample Match |
---|---|---|---|
d | Most engines: one digit from 0 to 9 | file_dd | file_25 |
d | .NET, Python 3: one Unicode digit in any script | file_dd | file_9੩ |
w | Most engines: 'word character': ASCII letter, digit or underscore | w-www | A-b_1 |
w | .Python 3: 'word character': Unicode letter, ideogram, digit, or underscore | w-www | 字-ま_۳ |
w | .NET: 'word character': Unicode letter, ideogram, digit, or connector | w-www | 字-ま‿۳ |
s | Most engines: 'whitespace character': space, tab, newline, carriage return, vertical tab | asbsc | a b c |
s | .NET, Python 3, JavaScript: 'whitespace character': any Unicode separator | asbsc | a b c |
D | One character that is not a digit as defined by your engine's d | DDD | ABC |
W | One character that is not a word character as defined by your engine's w | WWWWW | *-+=) |
S | One character that is not a whitespace character as defined by your engine's s | SSSS | Yoyo |
(direct link)
Quantifiers
Quantifier | Legend | Example | Sample Match |
---|---|---|---|
+ | One or more | Version w-w+ | Version A-b1_1 |
{3} | Exactly three times | D{3} | ABC |
{2,4} | Two to four times | d{2,4} | 156 |
{3,} | Three or more times | w{3,} | regex_tutorial |
* | Zero or more times | A*B*C* | AAACC |
? | Once or none | plurals? | plural |
(direct link)
More Characters
Character | Legend | Example | Sample Match |
---|---|---|---|
. | Any character except line break | a.c | abc |
. | Any character except line break | .* | whatever, man. |
. | A period (special character: needs to be escaped by a ) | a.c | a.c |
Escapes a special character | .*+? $^/ | .*+? $^/ | |
Escapes a special character | [{()}] | [{()}] |
(direct link)
Logic
Logic | Legend | Example | Sample Match |
---|---|---|---|
| | Alternation / OR operand | 22|33 | 33 |
( … ) | Capturing group | A(nt|pple) | Apple (captures 'pple') |
1 | Contents of Group 1 | r(w)g1x | regex |
2 | Contents of Group 2 | (dd)+(dd)=2+1 | 12+65=65+12 |
(?: … ) | Non-capturing group | A(?:nt|pple) | Apple |
(direct link)
More White-Space
Character | Legend | Example | Sample Match |
---|---|---|---|
t | Tab | Ttw{2} | T ab |
r | Carriage return character | see below | |
n | Line feed character | see below | |
rn | Line separator on Windows | ABrnCD | AB CD |
N | Perl, PCRE (C, PHP, R…): one character that is not a line break | N+ | ABC |
h | Perl, PCRE (C, PHP, R…), Java: one horizontal whitespace character: tab or Unicode space separator | ||
H | One character that is not a horizontal whitespace | ||
v | .NET, JavaScript, Python, Ruby: vertical tab | ||
v | Perl, PCRE (C, PHP, R…), Java: one vertical whitespace character: line feed, carriage return, vertical tab, form feed, paragraph or line separator | ||
V | Perl, PCRE (C, PHP, R…), Java: any character that is not a vertical whitespace | ||
R | Perl, PCRE (C, PHP, R…), Java: one line break (carriage return + line feed pair, and all the characters matched by v) |
(direct link)
More Quantifiers
Quantifier | Legend | Example | Sample Match |
---|---|---|---|
+ | The + (one or more) is 'greedy' | d+ | 12345 |
? | Makes quantifiers 'lazy' | d+? | 1 in 12345 |
* | The * (zero or more) is 'greedy' | A* | AAA |
? | Makes quantifiers 'lazy' | A*? | empty in AAA |
{2,4} | Two to four times, 'greedy' | w{2,4} | abcd |
? | Makes quantifiers 'lazy' | w{2,4}? | ab in abcd |
(direct link)
Character Classes
Character | Legend | Example | Sample Match |
---|---|---|---|
[ … ] | One of the characters in the brackets | [AEIOU] | One uppercase vowel |
[ … ] | One of the characters in the brackets | T[ao]p | Tap or Top |
- | Range indicator | [a-z] | One lowercase letter |
[x-y] | One of the characters in the range from x to y | [A-Z]+ | GREAT |
[ … ] | One of the characters in the brackets | [AB1-5w-z] | One of either: A,B,1,2,3,4,5,w,x,y,z |
[x-y] | One of the characters in the range from x to y | [ -~]+ | Characters in the printable section of the ASCII table. |
[^x] | One character that is not x | [^a-z]{3} | A1! |
[^x-y] | One of the characters not in the range from x to y | [^ -~]+ | Characters that are not in the printable section of the ASCII table. |
[dD] | One character that is a digit or a non-digit | [dD]+ | Any characters, inc- luding new lines, which the regular dot doesn't match |
[x41] | Matches the character at hexadecimal position 41 in the ASCII table, i.e. A | [x41-x45]{3} | ABE |
(direct link)

Anchors and Boundaries
Anchor | Legend | Example | Sample Match |
---|---|---|---|
^ | Start of string or start of line depending on multiline mode. (But when [^inside brackets], it means 'not') | ^abc .* | abc (line start) |
$ | End of string or end of line depending on multiline mode. Many engine-dependent subtleties. | .*? the end$ | this is the end |
A | Beginning of string (all major engines except JS) | Aabc[dD]* | abc (string... ...start) |
z | Very end of the string Not available in Python and JS | the endz | this is...n...the end |
Z | End of string or (except Python) before final line break Not available in JS | the endZ | this is...n...the endn |
G | Beginning of String or End of Previous Match .NET, Java, PCRE (C, PHP, R…), Perl, Ruby | ||
b | Word boundary Most engines: position where one side only is an ASCII letter, digit or underscore | Bob.*bcatb | Bob ate the cat |
b | Word boundary .NET, Java, Python 3, Ruby: position where one side only is a Unicode letter, digit or underscore | Bob.*bкошкаb | Bob ate the кошка |
B | Not a word boundary | c.*BcatB.* | copycats |
(direct link)
POSIX Classes
Character | Legend | Example | Sample Match |
---|---|---|---|
[:alpha:] | PCRE (C, PHP, R…): ASCII letters A-Z and a-z | [8[:alpha:]]+ | WellDone88 |
[:alpha:] | Ruby 2: Unicode letter or ideogram | [[:alpha:]d]+ | кошка99 |
[:alnum:] | PCRE (C, PHP, R…): ASCII digits and letters A-Z and a-z | [[:alnum:]]{10} | ABCDE12345 |
[:alnum:] | Ruby 2: Unicode digit, letter or ideogram | [[:alnum:]]{10} | кошка90210 |
[:punct:] | PCRE (C, PHP, R…): ASCII punctuation mark | [[:punct:]]+ | ?!.,:; |
[:punct:] | Ruby: Unicode punctuation mark | [[:punct:]]+ | ‽,:〽⁆ |
(direct link)
Inline Modifiers
None of these are supported in JavaScript. In Ruby, beware of (?s) and (?m).Modifier | Legend | Example | Sample Match |
---|---|---|---|
(?i) | Case-insensitive mode (except JavaScript) | (?i)Monday | monDAY |
(?s) | DOTALL mode (except JS and Ruby). The dot (.) matches new line characters (rn). Also known as 'single-line mode' because the dot treats the entire input as a single line | (?s)From A.*to Z | From A to Z |
(?m) | Multiline mode (except Ruby and JS) ^ and $ match at the beginning and end of every line | (?m)1rn^2$rn^3$ | 1 2 3 |
(?m) | In Ruby: the same as (?s) in other engines, i.e. DOTALL mode, i.e. dot matches line breaks | (?m)From A.*to Z | From A to Z |
(?x) | Free-Spacing Mode mode (except JavaScript). Also known as comment mode or whitespace mode | (?x) # this is a # comment abc # write on multiple # lines [ ]d # spaces must be # in brackets | abc d |
(?n) | .NET, PCRE 10.30+: named capture only | Turns all (parentheses) into non-capture groups. To capture, use named groups. | |
(?d) | Java: Unix linebreaks only | The dot and the ^ and $ anchors are only affected by n | |
(?^) | PCRE 10.32+: unset modifiers | Unsets ismnx modifiers |
(direct link)
Lookarounds
Lookaround | Legend | Example | Sample Match |
---|---|---|---|
(?=…) | Positive lookahead | (?=d{10})d{5} | 01234 in 0123456789 |
(?<=…) | Positive lookbehind | (?<=d)cat | cat in 1cat |
(?!…) | Negative lookahead | (?!theatre)thew+ | theme |
(?<!…) | Negative lookbehind | w{3}(?<!mon)ster | Munster |
(direct link)
Character Class Operations
Class Operation | Legend | Example | Sample Match |
---|---|---|---|
[…-[…]] | .NET: character class subtraction. One character that is in those on the left, but not in the subtracted class. | [a-z-[aeiou]] | Any lowercase consonant |
[…-[…]] | .NET: character class subtraction. | [p{IsArabic}-[D]] | An Arabic character that is not a non-digit, i.e., an Arabic digit |
[…&&[…]] | Java, Ruby 2+: character class intersection. One character that is both in those on the left and in the && class. | [S&&[D]] | An non-whitespace character that is a non-digit. |
[…&&[…]] | Java, Ruby 2+: character class intersection. | [S&&[D]&&[^a-zA-Z]] | An non-whitespace character that a non-digit and not a letter. |
[…&&[^…]] | Java, Ruby 2+: character class subtraction is obtained by intersecting a class with a negated class | [a-z&&[^aeiou]] | An English lowercase letter that is not a vowel. |
[…&&[^…]] | Java, Ruby 2+: character class subtraction | [p{InArabic}&&[^p{L}p{N}]] | An Arabic character that is not a letter or a number |
(direct link)
Other Syntax
Syntax | Legend | Example | Sample Match |
---|---|---|---|
Keep Out Perl, PCRE (C, PHP, R…), Python's alternate regex engine, Ruby 2+: drop everything that was matched so far from the overall match to be returned | prefixKd+ | 12 | |
Perl, PCRE (C, PHP, R…), Java: treat anything between the delimiters as a literal string. Useful to escape metacharacters. | Q(C++ ?)E | (C++ ?) |

and The Best Regex Trick Ever!!!
The 1001 ways to use Regex
Thankyou very much for compiling these. I am new to text analytics and is struggling a lot with regex. This is helping me a lot pick up. Great work
Yahoo Fantasy Football Cheat Sheets
