Unicode Cheat Sheet

Posted on  by 



Contents

Initial Steps
I/O, Database
Stream I/O
BOM Values
Constants & Globals
Data Types
Platform String API
TCHAR String API
References

Here is a cheat sheet for converting Microsoft C or C++ source code to support Unicode.It does not attempt to explain much and presumes you are generally familiar with Microsoft'sapproach to Unicode. The goal is just to have a single place to look for names, correct spellings, etc.of relevant data types, functions, etc.
Also see: Cheat Sheet: Unicode-enabling Microsoft C/C++ Source Code in Chinese

Unicode glyph cheat sheet Common Unicode characters with digraph or code point, layed out for quick location. Includes general symbols, arrows, drawing characters, and IPA letters. Unicode is a computing standard for the consistent encoding symbols. It was created in 1991. It’s just a table, which shows glyphs position to encoding system. Encoding takes symbol from table, and tells font what should be painted. But computer can understand binary code only. So, encoding is used number 1 or 0 to represent characters.

Initial Steps for Unicode-enabling Microsoft C/C++ Source

  • Define _UNICODE, undefine _MBCS if defined.
  • Convert literal strings to use L or _T
  • Convert string functions to use Wide or TCHAR versions.
  • Clarify string lengths in API as byte or character counts.For character-based display or printing (as opposed to GUI which is pixel-based)use column counts, not byte or character.
  • Replace character pointer arithmetic with GetNext style, ascharacters may consist of more than one Unicode code unit.
  • Watch buffer size and buffer overflows- changing encodingsmay require either larger buffers or limiting string lengths. Ifcharacter size changes from 1 byte to as many as 4 bytes, andstring length was formerly 20 characters and 20 bytes, eitherexpand the string buffer(s) from 20 to 80 bytes or limit thestring to 5 characters (and therefore 20 bytes).Note maximum buffer expansion may be constrained (for example to 65 KB).Reducing string length to a fixed number of characters may break existing applications.Limiting strings to a fixed byte length is dangerous. For example, allowing anystring that fits into 20 bytes. Simple operations such as uppercasinga string may cause it to grow and exceed the byte length.
  • Replace functions that accept or return arguments of a single character, with functions that use strings instead.(International) Operations on a single character may result in more than one code point being returned. For example,upper('ß')returns 'SS'.
  • Use wmain instead of main.The environment variable is then _wenviron instead of _environ.
    wmain( int argc, wchar_t *argv[ ], wchar_t *envp[ ] ).
  • MFC Unicode applications use wWinMain as the entry point.
    In the Output page of the Linker folder in the project's Property Pages dialog box,set the Entry Point symbol to wWinMainCRTStartup.
  • Consider fonts. Identify the fonts that will render each language or script used.

File I/O, Database, Transfer Protocol Considerations

  • Consider whether to read/write UTF-8 or UTF-16 in files, databases, and for data exchange.
  • Consider Endian-ness in UTF-16 files.
    Read/Write Big-Endian on networks. Use Big-Endian if you don't produce a BOM.
    Endian-ness of files will depend on the file format and/or the architectureof the source or target machine.
    When reading files encoded in UTF-16 or UTF-32,be prepared to swap-bytes to convert endian-ness.
    Also consider streams and transfer protocols and the encoding used in each.
  • Label files or protocols for data exchange with the correct character encoding.E.g. set HTTP, HTML, XML to UTF-8 or UTF-16.
  • Consider Unicode BOM (Byte Order Marker) and whetherit should be written with data. Remove it when reading data.
  • Consider encoding conversion of legacy data and files, import and export, transfer protocols.(MultiByteToWideChar,WideCharToMultiByte,mbtowc,wctomb,wctombs,mbstowcs )
  • Consider writing to the Clipboard-
    use CF_TEXT format and write native character encoding (ANSI) text, and
    use CF_UNICODETEXT format and write Unicode text.
  • Database applications should consider Data Type(NCHAR, NVARCHAR)and Schema Changes, Triggers, Stored Procedures, and Queries. Data Storagegrowth, Indexes and Performance.
    Note that the Unicode schema changes will have different impacts and concerns on different vendors' databases.If database portability is a requirement, the features and behaviors of each database need to be taken into account.
    (I know this item is seriously understated. To be expanded sometime in the future.)

Stream I/O

Streams are difficult in Microsoft C++. You may run into 3 types of problems:

  1. Unicode filenames are not supported.The workaround is to useFILE * _wfopen and if needed, use the FILE handle in subsequent stream I/O.
  2. Stream I/O will convert Unicode data from/to native (ANSI) code page on read/write,not UTF-8 or UTF-16.However the stream class can be modified to read/write UTF-8.You can implement a facet to convert between Unicode and UTF-8.
    codecvt <wchar_t, char_traits <wchar_t> >
  3. To read/write UTF-16 with stream I/O, use binary opens and binary I/O. To set binary I/O:
    Also see the Microsoft run-time library reference: 'Unicode Stream I/O in Text and Binary Modes'.

Note: There aren't TCHAR equivalents forcout/wcout, cin/wcin, etc.You may want to make your own preprocessor definition for 'tout',if you are compiling code both ways.

Internationalization, Advanced Unicode, Platform and Other Considerations

  • Consider using locale-based routines and further internationalization.
  • For Windows 95, 98 and ME, consider using theMicrosoft MSLU (Microsoft Layer for Unicode)
  • Consider string compares and sorting, Unicode Collation Algorithm
  • Consider Unicode Normalization
  • Consider Character Folding
  • Reconsider doing this on your own. Bring in an experiencedUnicode consultant,and deploy your existing resources on the tasks they do best.(Hey, an I18nGuy's gotta earn a living...)

Unicode BOM Encoding Values

Encoding FormBOM Encoding
UTF-8EF BB BF
UTF-16
(big-endian)
FE FF
UTF-16
(little-endian)
FF FE
UTF-16BE, UTF-32BE
(big-endian)
No BOM!
UTF-16LE, UTF-32LE
(little-endian)
No BOM!
UTF-32
(big-endian)
00 00 FE FF
UTF-32
(little-endian)
FF FE 00 00
SCSU
(compression)
0E FE FF

The Byte Order Marker (BOM) is Unicode character U+FEFF.(It can also represent a Zero Width No-break Space.)The code point U+FFFE is illegal in Unicode, and shouldnever appear in a Unicode character stream. Therefore the BOM can be used in the first characterof a file (or more generally a string), as an indicator of endian-ness. With UTF-16,if the first character is read as bytes FE FF then the texthas the same endian-ness as the machine reading it. If the character is read as bytesFF FE, thenthe endian-ness is reversed and all 16-bit words should be byte-swapped as they are read-in.In the same way, the BOM indicates the endian-ness of text encoded with UTF-32.

Note that not all files start with a BOM however. In fact, the Unicode Standard says that text thatdoes not begin with a BOM MUST be interpreted in big-endian form.

The character U+FEFF also serves as an encoding signaturefor the Unicode Encoding Forms.The tableshows the encoding of U+FEFF in each of the Unicode encoding forms.Note that by definition, text labeled as UTF-16BE, UTF-32BE, UTF-32LE or UTF-16LE shouldnot have a BOM. The endian-ness isindicated in the label.

For text that is compressed with the SCSU (Standard Compression Scheme for Unicode) algorithm,there is also a recommended signature.

Constant and Global Variables

ANSIWideTCHAR
EOFWEOF_TEOF
_environ_wenviron_tenviron
_pgmptr_wpgmptr_tpgmptr

Data Types

ANSIWideTCHAR
charwchar_t_TCHAR
_finddata_t_wfinddata_t_tfinddata_t
__finddata64_t__wfinddata64_t_tfinddata64_t
_finddatai64_t_wfinddatai64_t_tfinddatai64_t
intwint_t_TINT
signed charwchar_t_TSCHAR
unsigned charwchar_t_TUCHAR
charwchar_t_TXCHAR
L_T or _TEXT
LPSTR
(char *)
LPWSTR
(wchar_t *)
LPTSTR
(_TCHAR *)
LPCSTR
(const char *)
LPCWSTR
(const wchar_t *)
LPCTSTR
(const _TCHAR *)
LPOLESTR
(For OLE)
LPWSTRLPTSTR

Platform SDK String Functions

There are many Windows API that compile into ANSI or Wide forms,depending on whether the symbol UNICODE isdefined. Modules that operate on both ANSI and Wide characters, need tobe aware of this. Otherwise, using the Character Data Type-independent namerequires no changes, just compile with the symbol UNICODEdefined.

The following list is by no means all of the Character Data Type-dependent API, justsome character and string related ones. Look in WinNLS.h for some code pageand locale related API.

ANSIWideCharacter Data Type-
Independent Name
CharLowerACharLowerWCharLower
CharLowerBuffACharLowerBuffWCharLowerBuff
CharNextACharNextWCharNext
CharNextExACharNextExWCharNextEx
CharPrevACharPrevWCharPrev
CharPrevExACharPrevExWCharPrevEx
CharToOemACharToOemWCharToOem
CharToOemBuffACharToOemBuffWCharToOemBuff
CharUpperACharUpperWCharUpper
CharUpperBuffACharUpperBuffWCharUpperBuff
CompareStringACompareStringWCompareString
FoldStringAFoldStringWFoldString
GetStringTypeAGetStringTypeWGetStringType
GetStringTypeExAGetStringTypeExWGetStringTypeEx
IsCharAlphaAIsCharAlphaWIsCharAlpha
IsCharAlphaNumericAIsCharAlphaNumericWIsCharAlphaNumeric
IsCharLowerAIsCharLowerWIsCharLower
IsCharUpperAIsCharUpperWIsCharUpper
LoadStringALoadStringWLoadString
lstrcatAlstrcatWlstrcat
lstrcmpAlstrcmpWlstrcmp
lstrcmpiAlstrcmpiWlstrcmpi
lstrcpyAlstrcpyWlstrcpy
lstrcpynAlstrcpynWlstrcpyn
lstrlenAlstrlenWlstrlen
OemToCharAOemToCharWOemToChar
OemToCharBuffAOemToCharBuffWOemToCharBuff
wsprintfAwsprintfWwsprintf
wvsprintfAwvsprintfWwvsprintf

TCHAR String Functions

Functions sorted by ANSI name, for ease of converting to Unicode.

ANSIWideTCHAR
_access_waccess_taccess
_atoi64_wtoi64_tstoi64
_atoi64_wtoi64_ttoi64
_cgets_cgetwscgetts
_chdir_wchdir_tchdir
_chmod_wchmod_tchmod
_cprintf_cwprintf_tcprintf
_cputs_cputws_cputts
_creat_wcreat_tcreat
_cscanf_cwscanf_tcscanf
_ctime64_wctime64_tctime64
_execl_wexecl_texecl
_execle_wexecle_texecle
_execlp_wexeclp_texeclp
_execlpe_wexeclpe_texeclpe
_execv_wexecv_texecv
_execve_wexecve_texecve
_execvp_wexecvp_texecvp
_execvpe_wexecvpe_texecvpe
_fdopen_wfdopen_tfdopen
_fgetchar_fgetwchar_fgettchar
_findfirst_wfindfirst_tfindfirst
_findnext64_wfindnext64_tfindnext64
_findnext_wfindnext_tfindnext
_findnexti64_wfindnexti64_tfindnexti64
_fputchar_fputwchar_fputtchar
_fsopen_wfsopen_tfsopen
_fullpath_wfullpath_tfullpath
_getch_getwch_gettch
_getche_getwche_gettche
_getcwd_wgetcwd_tgetcwd
_getdcwd_wgetdcwd_tgetdcwd
_ltoa_ltow_ltot
_makepath_wmakepath_tmakepath
_mkdir_wmkdir_tmkdir
_mktemp_wmktemp_tmktemp
_open_wopen_topen
_popen_wpopen_tpopen
_putch_putwch_puttch
_putenv_wputenv_tputenv
_rmdir_wrmdir_trmdir
_scprintf_scwprintf_sctprintf
_searchenv_wsearchenv_tsearchenv
_snprintf_snwprintf_sntprintf
_snscanf_snwscanf_sntscanf
_sopen_wsopen_tsopen
_spawnl_wspawnl_tspawnl
_spawnle_wspawnle_tspawnle
_spawnlp_wspawnlp_tspawnlp
_spawnlpe_wspawnlpe_tspawnlpe
_spawnv_wspawnv_tspawnv
_spawnve_wspawnve_tspawnve
_spawnvp_wspawnvp_tspawnvp
_spawnvpe_wspawnvpe_tspawnvpe
_splitpath_wsplitpath_tsplitpath
_stat64_wstat64_tstat64
_stat_wstat_tstat
_stati64_wstati64_tstati64
_strdate_wstrdate_tstrdate
_strdec_wcsdec_tcsdec
_strdup_wcsdup_tcsdup
_stricmp_wcsicmp_tcsicmp
_stricoll_wcsicoll_tcsicoll
_strinc_wcsinc_tcsinc
_strlwr_wcslwr_tcslwr
_strncnt_wcsncnt_tcsnbcnt
_strncnt_wcsncnt_tcsnccnt
_strncnt_wcsncnt_tcsnccnt
_strncoll_wcsncoll_tcsnccoll
_strnextc_wcsnextc_tcsnextc
_strnicmp_wcsnicmp_tcsncicmp
_strnicmp_wcsnicmp_tcsnicmp
_strnicoll_wcsnicoll_tcsncicoll
_strnicoll_wcsnicoll_tcsnicoll
_strninc_wcsninc_tcsninc
_strnset_wcsnset_tcsncset
_strnset_wcsnset_tcsnset
_strrev_wcsrev_tcsrev
_strset_wcsset_tcsset
_strspnp_wcsspnp_tcsspnp
_strtime_wstrtime_tstrtime
_strtoi64_wcstoi64_tcstoi64
_strtoui64_wcstoui64_tcstoui64
_strupr_wcsupr_tcsupr
_tempnam_wtempnam_ttempnam
_ui64toa_ui64tow_ui64tot
_ultoa_ultow_ultot
_ungetch_ungetwch_ungettch
_unlink_wunlink_tunlink
_utime64_wutime64_tutime64
_utime_wutime_tutime
_vscprintf_vscwprintf_vsctprintf
_vsnprintf_vsnwprintf_vsntprintf
asctime_wasctime_tasctime
atof_wtof_tstof
atoi_wtoi_tstoi
atoi_wtoi_ttoi
atol_wtol_tstol
atol_wtol_ttol
character compareMaps to macro or inline function_tccmp
character copyMaps to macro or inline function_tccpy
character lengthMaps to macro or inline function_tclen
ctime_wctime_tctime
fgetcfgetwc_fgettc
fgetsfgetws_fgetts
fopen_wfopen_tfopen
fprintffwprintf_ftprintf
fputcfputwc_fputtc
fputsfputws_fputts
freopen_wfreopen_tfreopen
fscanffwscanf_ftscanf
getcgetwc_gettc
getchargetwchar_gettchar
getenv_wgetenv_tgetenv
getsgetws_getts
isalnumiswalnum_istalnum
isalphaiswalpha_istalpha
isasciiiswascii_istascii
iscntrliswcntrl_istcntrl
isdigitiswdigit_istdigit
isgraphiswgraph_istgraph
islead (Always FALSE)(Always FALSE)_istlead
isleadbyte (Always FALSE)isleadbyte (Always FALSE)_istleadbyte
islegal (Always TRUE)(Always TRUE)_istlegal
isloweriswlower_istlower
isprintiswprint_istprint
ispunctiswpunct_istpunct
isspaceiswspace_istspace
isupperiswupper_istupper
isxdigitiswxdigit_istxdigit
mainwmain_tmain
perror_wperror_tperror
printfwprintf_tprintf
putcputwc_puttc
putcharputwchar_puttchar
puts_putws_putts
remove_wremove_tremove
rename_wrename_trename
scanfwscanf_tscanf
setlocale_wsetlocale_tsetlocale
sprintfswprintf_stprintf
sscanfswscanf_stscanf
strcatwcscat_tcscat
strchrwcschr_tcschr
strcmpwcscmp_tcscmp
strcollwcscoll_tcscoll
strcpywcscpy_tcscpy
strcspnwcscspn_tcscspn
strerror_wcserror_tcserror
strftimewcsftime_tcsftime
strlenwcslen_tcsclen
strlenwcslen_tcslen
strncatwcsncat_tcsncat
strncatwcsncat_tcsnccat
strncmpwcsncmp_tcsnccmp
strncmpwcsncmp_tcsncmp
strncpywcsncpy_tcsnccpy
strncpywcsncpy_tcsncpy
strpbrkwcspbrk_tcspbrk
strrchrwcsrchr_tcsrchr
strspnwcsspn_tcsspn
strstrwcsstr_tcsstr
strtodwcstod_tcstod
strtokwcstok_tcstok
strtolwcstol_tcstol
strtoulwcstoul_tcstoul
strxfrmwcsxfrm_tcsxfrm
system_wsystem_tsystem
tmpnam_wtmpnam_ttmpnam
tolowertowlower_totlower
touppertowupper_totupper
ungetcungetwc_ungettc
vfprintfvfwprintf_vftprintf
vprintfvwprintf_vtprintf
vsprintfvswprintf_vstprintf
WinMainwWinMain_tWinMain

References

  • 'MSLU: Develop Unicode Applications for Windows 9x Platforms with the Microsoft Layer for Unicode'Michael Kaplan, Cathy Wissink
  • 'Design a Single Unicode App that Runs on Both Windows 98 and Windows 2000'F. Avery Bishop, April 1999.
  • 'Supporting Multilanguage Text Layout and Complex Scripts with Windows NT 5.0'F. Avery Bishop, David C Brown and Davis M Meltzer, November 1998.
  • Newsgroup: MSDN MSLU
  • Newsgroup: MSDN Internationalization

Copyright © 2003-2010 Tex Texin. All rights reserved.
Top of page

The tables below are a reference to basic regex. While reading the rest of the site, when in doubt, you can always come back and look here. (It you want a bookmark, here's a direct link to the regex reference tables). I encourage you to print the tables so you have a cheat sheet on your desk for quick reference.
The tables are not exhaustive, for two reasons. First, every regex flavor is different, and I didn't want to crowd the page with overly exotic syntax. For a full reference to the particular regex flavors you'll be using, it's always best to go straight to the source. In fact, for some regex engines (such as Perl, PCRE, Java and .NET) you may want to check once a year, as their creators often introduce new features.
The other reason the tables are not exhaustive is that I wanted them to serve as a quick introduction to regex. If you are a complete beginner, you should get a firm grasp of basic regex syntax just by reading the examples in the tables. I tried to introduce features in a logical order and to keep out oddities that I've never seen in actual use, such as the 'bell character'. With these tables as a jumping board, you will be able to advance to mastery by exploring the other pages on the site.

How to use the tables

The tables are meant to serve as an accelerated regex course, and they are meant to be read slowly, one line at a time. On each line, in the leftmost column, you will find a new element of regex syntax. The next column, 'Legend', explains what the element means (or encodes) in the regex syntax. The next two columns work hand in hand: the 'Example' column gives a valid regular expression that uses the element, and the 'Sample Match' column presents a text string that could be matched by the regular expression.
You can read the tables online, of course, but if you suffer from even the mildest case of online-ADD (attention deficit disorder), like most of us… Well then, I highly recommend you print them out. You'll be able to study them slowly, and to use them as a cheat sheet later, when you are reading the rest of the site or experimenting with your own regular expressions.
Enjoy!
If you overdose, make sure not to miss the next page, which comes back down to Earth and talks about some really cool stuff: The 1001 ways to use Regex.

Regex Accelerated Course and Cheat Sheet

For easy navigation, here are some jumping points to various sections of the page:
✽ Characters
✽ Quantifiers
✽ More Characters
✽ Logic

Unicode Cheat Sheet Free


✽ More White-Space
✽ More Quantifiers
✽ Character Classes
✽ Anchors and Boundaries
✽ POSIX Classes
✽ Inline Modifiers
✽ Lookarounds
✽ Character Class Operations
✽ Other Syntax
(direct link)

Characters

CharacterLegendExampleSample Match
dMost engines: one digit
from 0 to 9
file_ddfile_25
d.NET, Python 3: one Unicode digit in any scriptfile_ddfile_9੩
wMost engines: 'word character': ASCII letter, digit or underscorew-wwwA-b_1
w.Python 3: 'word character': Unicode letter, ideogram, digit, or underscorew-www字-ま_۳
w.NET: 'word character': Unicode letter, ideogram, digit, or connectorw-www字-ま‿۳
sMost engines: 'whitespace character': space, tab, newline, carriage return, vertical tabasbsca b
c
s.NET, Python 3, JavaScript: 'whitespace character': any Unicode separatorasbsca b
c
DOne character that is not a digit as defined by your engine's dDDDABC
WOne character that is not a word character as defined by your engine's wWWWWW*-+=)
SOne character that is not a whitespace character as defined by your engine's sSSSSYoyo

(direct link)

Quantifiers

QuantifierLegendExampleSample Match
+One or moreVersion w-w+Version A-b1_1
{3}Exactly three timesD{3}ABC
{2,4}Two to four timesd{2,4}156
{3,}Three or more timesw{3,}regex_tutorial
*Zero or more timesA*B*C*AAACC
?Once or noneplurals?plural

(direct link)

More Characters

CharacterLegendExampleSample Match
.Any character except line breaka.cabc
.Any character except line break.*whatever, man.
.A period (special character: needs to be escaped by a )a.ca.c
Escapes a special character.*+? $^/.*+? $^/
Escapes a special character[{()}][{()}]

(direct link)

Logic

LogicLegendExampleSample Match
| Alternation / OR operand22|3333
( … )Capturing groupA(nt|pple)Apple (captures 'pple')
1Contents of Group 1r(w)g1xregex
2Contents of Group 2(dd)+(dd)=2+112+65=65+12
(?: … )Non-capturing groupA(?:nt|pple)Apple

(direct link)

More White-Space

CharacterLegendExampleSample Match
tTabTtw{2}T ab
rCarriage return charactersee below
nLine feed charactersee below
rnLine separator on WindowsABrnCDAB
CD
NPerl, PCRE (C, PHP, R…): one character that is not a line breakN+ABC
hPerl, PCRE (C, PHP, R…), Java: one horizontal whitespace character: tab or Unicode space separator
HOne character that is not a horizontal whitespace
v.NET, JavaScript, Python, Ruby: vertical tab
vPerl, PCRE (C, PHP, R…), Java: one vertical whitespace character: line feed, carriage return, vertical tab, form feed, paragraph or line separator
VPerl, PCRE (C, PHP, R…), Java: any character that is not a vertical whitespace
RPerl, PCRE (C, PHP, R…), Java: one line break (carriage return + line feed pair, and all the characters matched by v)

(direct link)

More Quantifiers

QuantifierLegendExampleSample Match
+The + (one or more) is 'greedy'd+12345
?Makes quantifiers 'lazy'd+?1 in 12345
*The * (zero or more) is 'greedy'A*AAA
?Makes quantifiers 'lazy'A*?empty in AAA
{2,4}Two to four times, 'greedy'w{2,4}abcd
?Makes quantifiers 'lazy'w{2,4}?ab in abcd

(direct link)

Character Classes

CharacterLegendExampleSample Match
[ … ]One of the characters in the brackets[AEIOU]One uppercase vowel
[ … ]One of the characters in the bracketsT[ao]pTap or Top
-Range indicator[a-z]One lowercase letter
[x-y]One of the characters in the range from x to y[A-Z]+GREAT
[ … ]One of the characters in the brackets[AB1-5w-z]One of either: A,B,1,2,3,4,5,w,x,y,z
[x-y]One of the characters in the range from x to y[ -~]+Characters in the printable section of the ASCII table.
[^x]One character that is not x[^a-z]{3}A1!
[^x-y]One of the characters not in the range from x to y[^ -~]+Characters that are not in the printable section of the ASCII table.
[dD]One character that is a digit or a non-digit[dD]+Any characters, inc-
luding new lines, which the regular dot doesn't match
[x41]Matches the character at hexadecimal position 41 in the ASCII table, i.e. A[x41-x45]{3}ABE

(direct link)
Sheet

Anchors and Boundaries

AnchorLegendExampleSample Match
^Start of string or start of line depending on multiline mode. (But when [^inside brackets], it means 'not')^abc .*abc (line start)
$End of string or end of line depending on multiline mode. Many engine-dependent subtleties..*? the end$this is the end
ABeginning of string
(all major engines except JS)
Aabc[dD]*abc (string...
...start)
zVery end of the string
Not available in Python and JS
the endzthis is...n...the end
ZEnd of string or (except Python) before final line break
Not available in JS
the endZthis is...n...the endn
GBeginning of String or End of Previous Match
.NET, Java, PCRE (C, PHP, R…), Perl, Ruby
bWord boundary
Most engines: position where one side only is an ASCII letter, digit or underscore
Bob.*bcatbBob ate the cat
bWord boundary
.NET, Java, Python 3, Ruby: position where one side only is a Unicode letter, digit or underscore
Bob.*bкошкаbBob ate the кошка
BNot a word boundaryc.*BcatB.*copycats

(direct link)

POSIX Classes

CharacterLegendExampleSample Match
[:alpha:]PCRE (C, PHP, R…): ASCII letters A-Z and a-z[8[:alpha:]]+WellDone88
[:alpha:]Ruby 2: Unicode letter or ideogram[[:alpha:]d]+кошка99
[:alnum:]PCRE (C, PHP, R…): ASCII digits and letters A-Z and a-z[[:alnum:]]{10}ABCDE12345
[:alnum:]Ruby 2: Unicode digit, letter or ideogram[[:alnum:]]{10}кошка90210
[:punct:]PCRE (C, PHP, R…): ASCII punctuation mark[[:punct:]]+?!.,:;
[:punct:]Ruby: Unicode punctuation mark[[:punct:]]+‽,:〽⁆

(direct link)

Inline Modifiers

None of these are supported in JavaScript. In Ruby, beware of (?s) and (?m).
ModifierLegendExampleSample Match
(?i)Case-insensitive mode
(except JavaScript)
(?i)MondaymonDAY
(?s)DOTALL mode (except JS and Ruby). The dot (.) matches new line characters (rn). Also known as 'single-line mode' because the dot treats the entire input as a single line(?s)From A.*to ZFrom A
to Z
(?m)Multiline mode
(except Ruby and JS) ^ and $ match at the beginning and end of every line
(?m)1rn^2$rn^3$1
2
3
(?m)In Ruby: the same as (?s) in other engines, i.e. DOTALL mode, i.e. dot matches line breaks(?m)From A.*to ZFrom A
to Z
(?x)Free-Spacing Mode mode
(except JavaScript). Also known as comment mode or whitespace mode
(?x) # this is a
# comment
abc # write on multiple
# lines
[ ]d # spaces must be
# in brackets
abc d
(?n).NET, PCRE 10.30+: named capture onlyTurns all (parentheses) into non-capture groups. To capture, use named groups.
(?d)Java: Unix linebreaks onlyThe dot and the ^ and $ anchors are only affected by n
(?^)PCRE 10.32+: unset modifiersUnsets ismnx modifiers

(direct link)

Lookarounds

LookaroundLegendExampleSample Match
(?=…)Positive lookahead(?=d{10})d{5}01234 in 0123456789
(?<=…)Positive lookbehind(?<=d)catcat in 1cat
(?!…)Negative lookahead(?!theatre)thew+theme
(?<!…)Negative lookbehindw{3}(?<!mon)sterMunster

(direct link)

Character Class Operations

Class OperationLegendExampleSample Match
[…-[…]].NET: character class subtraction. One character that is in those on the left, but not in the subtracted class.[a-z-[aeiou]]Any lowercase consonant
[…-[…]].NET: character class subtraction.[p{IsArabic}-[D]]An Arabic character that is not a non-digit, i.e., an Arabic digit
[…&&[…]]Java, Ruby 2+: character class intersection. One character that is both in those on the left and in the && class.[S&&[D]]An non-whitespace character that is a non-digit.
[…&&[…]]Java, Ruby 2+: character class intersection.[S&&[D]&&[^a-zA-Z]]An non-whitespace character that a non-digit and not a letter.
[…&&[^…]]Java, Ruby 2+: character class subtraction is obtained by intersecting a class with a negated class[a-z&&[^aeiou]]An English lowercase letter that is not a vowel.
[…&&[^…]]Java, Ruby 2+: character class subtraction[p{InArabic}&&[^p{L}p{N}]]An Arabic character that is not a letter or a number

(direct link)

Other Syntax

SyntaxLegendExampleSample Match
KKeep Out
Perl, PCRE (C, PHP, R…), Python's alternate regex engine, Ruby 2+: drop everything that was matched so far from the overall match to be returned
prefixKd+12
Q…EPerl, PCRE (C, PHP, R…), Java: treat anything between the delimiters as a literal string. Useful to escape metacharacters.Q(C++ ?)E(C++ ?)
Sheet
Don't Miss The Regex Style Guide
and The Best Regex Trick Ever!!!

The 1001 ways to use Regex

1-10 of 17 Threads
Subject: Very thoughtful and useful cheat sheet

Unlike lots of other cheat sheets or regex web sites, I was able (without much persistent regex knowledge) to apply the rules and to solve my problem. THANK YOU :)
Subject: Thanks a lot

Thanks a lot for the quick guide. It's really helpful.
Subject: Very useful site

Thank you soooooo much for this site. I'm using python regex for natural language processing in sentiment analysis and this helped me a lot.
Subject: Thank you! Excellent resource for any student

Thank you so much for this incredible cheatsheet! It is facilitating a lot my regex learning! God bless you and your passion!
Subject: Thank you for doing such a geat work.

I am now learning regex and for finding such a well organized site is a blessing! You are a good soul! Thank you for everything and stay inspired!
Subject: Simple = perfect

Subject: Congratulations

Well done, very useful page. Thank you for your effort. T
Subject: Thank you very much

Hi Rex,
Thankyou very much for compiling these. I am new to text analytics and is struggling a lot with regex. This is helping me a lot pick up. Great work
Subject: Nice summary

Nice summary of regex. I was trying to remember how to group and I found the example above. Thanks.
Subject: Best Regex site ever

Yahoo Fantasy Football Cheat Sheets


This is the best regex site ever on the internet. Regular Expressions are like any other language, they require time and effort to learn. RexEgg makes it an easy journey. Great work Author. Kudos to you.







Coments are closed