utf-16

I'm working on a twitter app and just stumbled into the world of utf-8(16). It seems the majority of javascript string functions are as blind to surrogate pairs as I was. I've got to recode some stuff to make it wide character aware. I've got this function to parse strings into arrays while preserving the surrogate pairs. Then I'll recode several functions to deal with the arrays rather than strings. function sortSurrogates(str){ var cp = []; // array to hold code points while(str.length){ // loop till we've done the whole string if(/[\uD800-\uDFFF]/.test(str.substr(0,1))){ // test the first

From here Essentially, string uses the UTF-16 character encoding form But when saving vs StreamWriter : This constructor creates a StreamWriter with UTF-8 encoding without a Byte-Order Mark (BOM), I've seen this sample (broken link removed): And it looks like utf8 is smaller for some strings while utf-16 is smaller in some other strings. So Why .net uses utf16 as default encoding for string while utf8 for saving file ? Thank you. p.s. Ive already read the famous article If you're happy ignoring surrogate pairs (or equivalently, the possibility of your app needing characters outside the Basic

Recently I read lots of thing about unicode code points and how they evolved over time and sure I read http://www.joelonsoftware.com/articles/Unicode.html this also. But something I couldn't find the real reason why Java uses UTF-16 for a char. For example If I had the string which contains 1024 letter of ASCII scoped charachter string. It means 1024 * 2 bytes which equals to 2KB string memory it will consume in anyway. So if Java base char would be UTF-8 it would be just 1KB of data. Even if the string has any charachter which needs to 2bytes for example 10 charachter of "字" naturally it will

We have a bunch of surrogate pair (or 2-byte utf8?) characters such as �� which is the prayer hands emojis stored as UTF8 as 2 characters. When rendered in a browser this string renders as two ?? example: I need to convert those to the hands emjoi using php but I simply cannot find a combination of iconv, utf8_decode, html_entity_decode etc to pull it off. This site converts the �� properly: http://www.convertstring.com/EncodeDecode/HtmlDecode Paste in there the following string Please join me in this prayer. ��❤️ You will notice the surragate pair ( &

I have an .inf file exported from Resource Hacker. The file is in UTF-16 LE encoding. EXTRALARGELEGENDSII_INI TEXTFILE "Data.bin" LARGEFONTSLEGENDSII_INI TEXTFILE "Data_2.bin" NORMALLEGENDSII_INI TEXTFILE "Data_3.bin" THEMES_INI TEXTFILE "Data_4.bin" When I load it using the LoadStringFromFile function : procedure LoadResources; var RESOURCE_INFO: AnsiString; begin LoadStringFromFile(ExpandConstant('{tmp}\SKINRESOURCE - INFO.inf'), RESOURCE_INFO); Log(String(RESOURCE_INFO)); end; I am getting this in the Debug Output : E Please tell me how to fix this issue. Thanks in advance. It seems the

From Core Java , vol. 1, 9th ed., p. 69: The character ℤ requires two code units in the UTF-16 encoding. Calling String sentence = "ℤ is the set of integers"; // for clarity; not in book char ch = sentence.charAt(1) doesn't return a space but the second code unit of ℤ. But it seems that sentence.charAt(1) does return a space. For example, the if statement in the following code evaluates to true . String sentence = "ℤ is the set of integers"; if (sentence.charAt(1) == ' ') System.out.println("sentence.charAt(1) returns a space"); Why? I'm using JDK SE 1.7.0_09 on Ubuntu 12.10, if it's relevant.