unicode-string

In standard C++ we have char and wchar_t for storing characters. char can store values between 0x00 and 0xFF . And wchar_t can store values between 0x0000 and 0xFFFF . std::string uses char , so it can store 1-byte characters only. std::wstring uses wchar_t , so it can store characters up to 2-byte width. This is what I know about strings in C++. Please correct me if I said anything wrong up to this point. I read the article for UTF-8 in Wikipedia, and I learned that some Unicode characters consume up to 4-byte space. For example, the Chinese character 𤭢 has a Unicode code point 0x24B62 ,

This code: for root, dirs, files in os.walk('.'): print(root) Gives me this error: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc3' in position 27: surrogates not allowed How do I walk through a file tree without getting toxic strings like this? On Linux, filenames are 'just a bunch of bytes', and are not necessarily encoded in a particular encoding. Python 3 tries to turn everything into Unicode strings. In doing so the developers came up with a scheme to translate byte strings to Unicode strings and back without loss, and without knowing the original encoding. They used

I am trying hard to get the count of unicode string and tried various options. Looks like a small problem but struck in a big way. Here I am trying to get the length of the string str1. I am getting it as 6. But actually it is 3. moving the cursor over the string "குமார்" also shows it as 3 chars. Basically I want to measure the length and print each character. like "கு", "மா", "ர்" . public class one { public static void main(String[] args) { String str1 = new String("குமார்"); System.out.print(str1.length()); } } PS : It is tamil language. halex Found a solution to your problem. Based on

I am using MySQL and PHP 5.3 and tried this code. $dbhost = 'localhost'; $dbuser = 'root'; $dbpass = ''; $con = mysql_connect("localhost", "root", ""); mysql_set_charset('utf8'); if (!$con) { die('Could not connect: ' . mysql_error()); } mysql_select_db("kdict", $con); $sql = "SELECT * FROM `en-kh` where english='a'"; echo $sql; $result = mysql_query($sql); while($row = mysql_fetch_array($result)) { echo $row['english'] . " </br> " . $row['khmer']; echo "<br />"; } ?> => I got good UTF-8 render display, well done. But for now I create a class PDO to keep easy to extend and more easy class crud

I have a file that has strings hand typed as \u00C3. I want to create a unicode character that is being represented by that unicode in java. I tried but could not find how. Help. Edit: When I read the text file String will contain "\u00C3" not as unicode but as ASCII chars '\' 'u' '0' '0' '3'. I would like to form unicode character from that ASCII string. I picked this up somewhere on the web: String unescape(String s) { int i=0, len=s.length(); char c; StringBuffer sb = new StringBuffer(len); while (i < len) { c = s.charAt(i++); if (c == '\\') { if (i < len) { c = s.charAt(i++); if (c == 'u')