How to get the number of characters in a std::string?

Question

How should I get the number of characters in a string in C++?

Answer 1

For Unicode

Several answers here have addressed that .length() gives the wrong results with multibyte characters, but there are 11 answers and none of them have provided a solution.

The case of Z͉̳̺ͥͬ̾a̴͕̲̒̒͌̋ͪl̨͎̰̘͉̟ͤ̀̈̚͜g͕͔̤͖̟̒͝ͅo̵̡̡̼͚̐ͯ̅ͪ̆ͣ̚

First of all, it's important to know what you mean by "length". For a motivating example, consider the string "Z͉̳̺ͥͬ̾a̴͕̲̒̒͌̋ͪl̨͎̰̘͉̟ͤ̀̈̚͜g͕͔̤͖̟̒͝ͅo̵̡̡̼͚̐ͯ̅ͪ̆ͣ̚" (note that some languages, notably Thai, actually use combining diacritical marks, so this isn't just useful for 15-year-old memes, but obviously that's the most important use case). Assume it is encoded in UTF-8. There are 3 ways we can talk about the length of this string:

95 bytes

00000000: 5acd a5cd accc becd 89cc b3cc ba61 cc92  Z............a..
00000010: cc92 cd8c cc8b cdaa ccb4 cd95 ccb2 6ccd  ..............l.
00000020: a4cc 80cc 9acc 88cd 9ccc a8cd 8ecc b0cc  ................
00000030: 98cd 89cc 9f67 cc92 cd9d cd85 cd95 cd94  .....g..........
00000040: cca4 cd96 cc9f 6fcc 90cd afcc 9acc 85cd  ......o.........
00000050: aacc 86cd a3cc a1cc b5cc a1cc bccd 9a    ...............

50 codepoints

LATIN CAPITAL LETTER Z
COMBINING LEFT ANGLE BELOW
COMBINING DOUBLE LOW LINE
COMBINING INVERTED BRIDGE BELOW
COMBINING LATIN SMALL LETTER I
COMBINING LATIN SMALL LETTER R
COMBINING VERTICAL TILDE
LATIN SMALL LETTER A
COMBINING TILDE OVERLAY
COMBINING RIGHT ARROWHEAD BELOW
COMBINING LOW LINE
COMBINING TURNED COMMA ABOVE
COMBINING TURNED COMMA ABOVE
COMBINING ALMOST EQUAL TO ABOVE
COMBINING DOUBLE ACUTE ACCENT
COMBINING LATIN SMALL LETTER H
LATIN SMALL LETTER L
COMBINING OGONEK
COMBINING UPWARDS ARROW BELOW
COMBINING TILDE BELOW
COMBINING LEFT TACK BELOW
COMBINING LEFT ANGLE BELOW
COMBINING PLUS SIGN BELOW
COMBINING LATIN SMALL LETTER E
COMBINING GRAVE ACCENT
COMBINING DIAERESIS
COMBINING LEFT ANGLE ABOVE
COMBINING DOUBLE BREVE BELOW
LATIN SMALL LETTER G
COMBINING RIGHT ARROWHEAD BELOW
COMBINING LEFT ARROWHEAD BELOW
COMBINING DIAERESIS BELOW
COMBINING RIGHT ARROWHEAD AND UP ARROWHEAD BELOW
COMBINING PLUS SIGN BELOW
COMBINING TURNED COMMA ABOVE
COMBINING DOUBLE BREVE
COMBINING GREEK YPOGEGRAMMENI
LATIN SMALL LETTER O
COMBINING SHORT STROKE OVERLAY
COMBINING PALATALIZED HOOK BELOW
COMBINING PALATALIZED HOOK BELOW
COMBINING SEAGULL BELOW
COMBINING DOUBLE RING BELOW
COMBINING CANDRABINDU
COMBINING LATIN SMALL LETTER X
COMBINING OVERLINE
COMBINING LATIN SMALL LETTER H
COMBINING BREVE
COMBINING LATIN SMALL LETTER A
COMBINING LEFT ANGLE ABOVE

5 graphemes

Z with some s**t
a with some s**t
l with some s**t
g with some s**t
o with some s**t

Finding the lengths using ICU

There are C++ classes for ICU, but they require converting to UTF-16. You can use the C types and macros directly to get some UTF-8 support:

#include 
#include 
#include 
#include 
#include 

//
// C++ helpers so we can use RAII
//
// Note that ICU internally provides some C++ wrappers (such as BreakIterator), however these only seem to work
// for UTF-16 strings, and require transforming UTF-8 to UTF-16 before use.
// If you already have UTF-16 strings or can take the performance hit, you should probably use those instead of
// the C functions. See: http://icu-project.org/apiref/icu4c/
//
struct UTextDeleter { void operator()(UText* ptr) { utext_close(ptr); } };
struct UBreakIteratorDeleter { void operator()(UBreakIterator* ptr) { ubrk_close(ptr); } };
using PUText = std::unique_ptr;
using PUBreakIterator = std::unique_ptr;

void checkStatus(const UErrorCode status)
{
    if(U_FAILURE(status))
    {
        throw std::runtime_error(u_errorName(status));
    }
}

size_t countGraphemes(UText* text)
{
    // source for most of this: http://userguide.icu-project.org/strings/utext
    UErrorCode status = U_ZERO_ERROR;
    PUBreakIterator it(ubrk_open(UBRK_CHARACTER, "en_us", nullptr, 0, &status));
    checkStatus(status);
    ubrk_setUText(it.get(), text, &status);
    checkStatus(status);
    size_t charCount = 0;
    while(ubrk_next(it.get()) != UBRK_DONE)
    {
        ++charCount;
    }
    return charCount;
}

size_t countCodepoints(UText* text)
{
    size_t codepointCount = 0;
    while(UTEXT_NEXT32(text) != U_SENTINEL)
    {
        ++codepointCount;
    }
    // reset the index so we can use the structure again
    UTEXT_SETNATIVEINDEX(text, 0);
    return codepointCount;
}

void printStringInfo(const std::string& utf8)
{
    UErrorCode status = U_ZERO_ERROR;
    PUText text(utext_openUTF8(nullptr, utf8.data(), utf8.length(), &status));
    checkStatus(status);

    std::cout << "UTF-8 string (might look wrong if your console locale is different): " << utf8 << std::endl;
    std::cout << "Length (UTF-8 bytes): " << utf8.length() << std::endl;
    std::cout << "Length (UTF-8 codepoints): " << countCodepoints(text.get()) << std::endl;
    std::cout << "Length (graphemes): " << countGraphemes(text.get()) << std::endl;
    std::cout << std::endl;
}

void main(int argc, char** argv)
{
    printStringInfo(u8"Hello, world!");
    printStringInfo(u8"หวัดดีชาวโลก");
    printStringInfo(u8"\xF0\x9F\x90\xBF");
    printStringInfo(u8"Z͉̳̺ͥͬ̾a̴͕̲̒̒͌̋ͪl̨͎̰̘͉̟ͤ̀̈̚͜g͕͔̤͖̟̒͝ͅo̵̡̡̼͚̐ͯ̅ͪ̆ͣ̚");
}

This prints:

UTF-8 string (might look wrong if your console locale is different): Hello, world!
Length (UTF-8 bytes): 13
Length (UTF-8 codepoints): 13
Length (graphemes): 13

UTF-8 string (might look wrong if your console locale is different): หวัดดีชาวโลก
Length (UTF-8 bytes): 36
Length (UTF-8 codepoints): 12
Length (graphemes): 10

UTF-8 string (might look wrong if your console locale is different):